-
- All Implemented Interfaces:
-
ai.platon.pulsar.common.config.Parameterized,ai.platon.pulsar.crawl.common.JobInitialized,java.lang.AutoCloseable
public final class TaskMonitor implements Parameterized, JobInitialized, AutoCloseable
Tasks Monitor
-
-
Nested Class Summary
Nested Classes Modifier and Type Class Description public classTaskMonitor.Companion
-
Field Summary
Fields Modifier and Type Field Description private final DurationpoolPendingTimeoutprivate final AtomicIntegernumReadyTasksprivate final AtomicIntegernumPendingTasksprivate final AtomicIntegernumFinishedTasksprivate final IntegernumTasks
-
Constructor Summary
Constructors Constructor Description TaskMonitor(CoreMetrics coreMetrics, MiscMessageWriter metrics, ImmutableConfig conf)
-
Method Summary
Modifier and Type Method Description final DurationgetPoolPendingTimeout()Once timeout, the pending items should be put to the ready pool again. final AtomicIntegergetNumReadyTasks()Task counters final AtomicIntegergetNumPendingTasks()final AtomicIntegergetNumFinishedTasks()final IntegergetNumTasks()Unitsetup(ImmutableConfig jobConf)ParamsgetParams()final UnitsetFeederCompleted()final Unitproduce(Integer jobID, WebPage page)final Unitproduce(JobFetchTask task)final JobFetchTaskconsume(PoolId poolId)final Unitfinish(JobFetchTask item)final UnitfinishAsap(JobFetchTask item)final Unitmaintain()final Unitreport()final JobFetchTaskfindPendingTask(Integer priority, URL url, Integer itemID)final JobFetchTaskfindPendingTask(PoolId poolId, Integer itemID)Get a pending task, the task can be in working pools or in detached pools Unitclose()final IntegerclearReadyTasks()-
-
Constructor Detail
-
TaskMonitor
TaskMonitor(CoreMetrics coreMetrics, MiscMessageWriter metrics, ImmutableConfig conf)
-
-
Method Detail
-
getPoolPendingTimeout
final Duration getPoolPendingTimeout()
Once timeout, the pending items should be put to the ready pool again.
-
getNumReadyTasks
final AtomicInteger getNumReadyTasks()
Task counters
-
getNumPendingTasks
final AtomicInteger getNumPendingTasks()
-
getNumFinishedTasks
final AtomicInteger getNumFinishedTasks()
-
getNumTasks
final Integer getNumTasks()
-
getParams
Params getParams()
-
setFeederCompleted
final Unit setFeederCompleted()
-
produce
@Synchronized() final Unit produce(Integer jobID, WebPage page)
-
produce
@Synchronized() final Unit produce(JobFetchTask task)
-
consume
@Synchronized() final JobFetchTask consume(PoolId poolId)
-
finish
@Synchronized() final Unit finish(JobFetchTask item)
-
finishAsap
@Synchronized() final Unit finishAsap(JobFetchTask item)
-
report
@Synchronized() final Unit report()
-
findPendingTask
@Synchronized() final JobFetchTask findPendingTask(Integer priority, URL url, Integer itemID)
-
findPendingTask
@Synchronized() final JobFetchTask findPendingTask(PoolId poolId, Integer itemID)
Get a pending task, the task can be in working pools or in detached pools
-
close
@Synchronized() Unit close()
-
clearReadyTasks
@Synchronized() final Integer clearReadyTasks()
-
-
-
-