TaskPool

All Implemented Interfaces:

ai.platon.pulsar.common.config.Parameterized , kotlin.Comparable
```
public final class TaskPool
 implements Comparable<TaskPool>, Parameterized
                    
```
This class handles FetchTasks which come from the same host ID (be it a proto/hostname or proto/IP pair).
It also keeps track of requests in progress and elapsed time between requests.

Nested Class Summary

Nested Classes
Modifier and Type	Class	Description
`public enum`	`TaskPool.Status`
`public class`	`TaskPool.Companion`

Field Summary

Fields
Modifier and Type	Field	Description
`private TaskPool.Status`	`status`
`private final Integer`	`priority`
`private final String`	`protocol`
`private final String`	`host`
`private final Boolean`	`isSlow`
`private final Boolean`	`isActive`
`private final Boolean`	`isInactive`
`private final Boolean`	`isRetired`
`private final Integer`	`numReadyTasks`
`private final Integer`	`numPendingTasks`
`private final Integer`	`numTotalFinishedTasks`
`private final Integer`	`numSlowTasks`
`private final Double`	`averageTime`
`private final Double`	`averageRecentTimeCost`
`private final Double`	`averageTps`
`private final Double`	`averageRecentTps`
`private final String`	`timeReport`
`private final PoolId`	`id`
`private final URLUtil.GroupMode`	`groupMode`

Constructor Summary

Constructors
Constructor	Description
`TaskPool(PoolId id, URLUtil.GroupMode groupMode, Integer allowedThreads, Duration crawlDelay, Duration minCrawlDelay, Duration pendingTimeout)`

Method Summary

Modifier and Type	Method	Description
`final TaskPool.Status`	`getStatus()`	If a fetch queue is inactive, the queue does not accept any tasks, nor serve any requests, but still hold pending tasks, waiting to finish
`final Unit`	`setStatus(TaskPool.Status status)`
`final Integer`	`getPriority()`
`final String`	`getProtocol()`
`final String`	`getHost()`
`final Boolean`	`getIsSlow()`
`final Boolean`	`getIsActive()`
`final Boolean`	`getIsInactive()`
`final Boolean`	`getIsRetired()`
`final Integer`	`getNumReadyTasks()`
`final Integer`	`getNumPendingTasks()`
`final Integer`	`getNumTotalFinishedTasks()`
`final Integer`	`getNumSlowTasks()`
`final Double`	`getAverageTime()`
`final Double`	`getAverageRecentTimeCost()`
`final Double`	`getAverageTps()`
`final Double`	`getAverageRecentTps()`
`final String`	`getTimeReport()`
`final PoolId`	`getId()`
`final URLUtil.GroupMode`	`getGroupMode()`	Host group mode : can be by ip, by host or by domain
`Params`	`getParams()`
`final Unit`	`produce(JobFetchTask task)`	Produce a task to this queue.
`final JobFetchTask`	`consume()`	Ask a task from this queue.
`final Boolean`	`finish(Integer itemId, Boolean asap)`
`final Boolean`	`finish(JobFetchTask fetchTask, Boolean asap)`	Note : We have set response time for each page, @see {HttpBase#getProtocolOutput}
`final JobFetchTask`	`getPendingTask(Integer itemID)`
`final Boolean`	`hasTasks()`
`final Boolean`	`hasReadyTasks()`
`final Boolean`	`hasPendingTasks()`
`final Boolean`	`pendingTaskExists(Integer itemId)`
`final Boolean`	`isSlow(Duration threshold)`
`final Unit`	`enable()`
`final Unit`	`disable()`
`final Unit`	`retire()`
`final Integer`	`clearReadyQueue()`
`final Integer`	`clearPendingQueue()`
`final Integer`	`clearPendingTasksIfFew(Integer threshold)`
`final Unit`	`dump(Boolean drop)`
`Boolean`	`equals(Object other)`
`Integer`	`hashCode()`
`Integer`	`compareTo(TaskPool other)`
`String`	`toString()`

Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

TaskPool

TaskPool(PoolId id, URLUtil.GroupMode groupMode, Integer allowedThreads, Duration crawlDelay, Duration minCrawlDelay, Duration pendingTimeout)

Method Detail

getStatus
```
 final TaskPool.Status getStatus()
```
If a fetch queue is inactive, the queue does not accept any tasks, nor serve any requests, but still hold pending tasks, waiting to finish

setStatus

 final Unit setStatus(TaskPool.Status status)

getPriority
```
 final Integer getPriority()
```

getProtocol
```
 final String getProtocol()
```

getHost
```
 final String getHost()
```

getIsSlow
```
 final Boolean getIsSlow()
```

getIsActive
```
 final Boolean getIsActive()
```

getIsInactive
```
 final Boolean getIsInactive()
```

getIsRetired
```
 final Boolean getIsRetired()
```

getNumReadyTasks
```
 final Integer getNumReadyTasks()
```

getNumPendingTasks
```
 final Integer getNumPendingTasks()
```

getNumTotalFinishedTasks

 final Integer getNumTotalFinishedTasks()

getNumSlowTasks
```
 final Integer getNumSlowTasks()
```

getAverageTime
```
 final Double getAverageTime()
```

getAverageRecentTimeCost

 final Double getAverageRecentTimeCost()

getAverageTps
```
 final Double getAverageTps()
```

getAverageRecentTps
```
 final Double getAverageRecentTps()
```

getTimeReport
```
 final String getTimeReport()
```

getId
```
 final PoolId getId()
```

getGroupMode
```
 final URLUtil.GroupMode getGroupMode()
```
Host group mode : can be by ip, by host or by domain

getParams
```
 Params getParams()
```

produce
```
 final Unit produce(JobFetchTask task)
```
Produce a task to this queue. Retired queues do not accept any tasks

consume
```
 final JobFetchTask consume()
```
Ask a task from this queue. Retired queues do not assign any tasks

finish

 final Boolean finish(Integer itemId, Boolean asap)

finish
```
 final Boolean finish(JobFetchTask fetchTask, Boolean asap)
```
Note : We have set response time for each page, @see {HttpBase#getProtocolOutput}

getPendingTask

 final JobFetchTask getPendingTask(Integer itemID)

hasTasks
```
 final Boolean hasTasks()
```

hasReadyTasks
```
 final Boolean hasReadyTasks()
```

hasPendingTasks
```
 final Boolean hasPendingTasks()
```

pendingTaskExists

 final Boolean pendingTaskExists(Integer itemId)

isSlow

 final Boolean isSlow(Duration threshold)

enable
```
 final Unit enable()
```

disable
```
 final Unit disable()
```

retire
```
 final Unit retire()
```

clearReadyQueue
```
 final Integer clearReadyQueue()
```

clearPendingQueue
```
 final Integer clearPendingQueue()
```

clearPendingTasksIfFew

 final Integer clearPendingTasksIfFew(Integer threshold)

dump
```
 final Unit dump(Boolean drop)
```

equals
```
 Boolean equals(Object other)
```

hashCode
```
 Integer hashCode()
```

compareTo
```
 Integer compareTo(TaskPool other)
```

toString
```
 String toString()
```

Class TaskPool

Nested Class Summary

Field Summary

Constructor Summary

Method Summary

Methods inherited from class java.lang.Object

Constructor Detail

TaskPool

Method Detail

getStatus

setStatus

getPriority

getProtocol

getHost

getIsSlow

getIsActive

getIsInactive

getIsRetired

getNumReadyTasks

getNumPendingTasks

getNumTotalFinishedTasks

getNumSlowTasks

getAverageTime

getAverageRecentTimeCost

getAverageTps

getAverageRecentTps

getTimeReport

getId

getGroupMode

getParams

produce

consume

finish

finish

getPendingTask

hasTasks

hasReadyTasks

hasPendingTasks

pendingTaskExists

isSlow

enable

disable

retire

clearReadyQueue

clearPendingQueue

clearPendingTasksIfFew

dump

equals

hashCode

compareTo

toString