TaskScheduler

All Implemented Interfaces:

ai.platon.pulsar.common.config.Parameterized , ai.platon.pulsar.crawl.common.JobInitialized , java.lang.AutoCloseable
```
public final class TaskScheduler
 implements Parameterized, JobInitialized, AutoCloseable
                    
```

Nested Class Summary

Nested Classes
Modifier and Type	Class	Description
`public final class`	`TaskScheduler.Status`
`public class`	`TaskScheduler.Companion`

Field Summary

Fields
Modifier and Type	Field	Description
`private final Integer`	`id`
`private Boolean`	`skipTruncated`
`private Boolean`	`storeContent`
`private Boolean`	`indexJIT`
`private Boolean`	`parse`
`private Instant`	`lastTaskStartTime`
`private Instant`	`lastTaskFinishTime`
`private final String`	`name`
`private final TaskMonitor`	`tasksMonitor`
`private final PageParser`	`pageParser`
`private final JITIndexer`	`jitIndexer`
`private final CoreMetrics`	`coreMetrics`
`private final MiscMessageWriter`	`messageWriter`
`private final ImmutableConfig`	`immutableConfig`

Constructor Summary

Constructors
Constructor	Description
`TaskScheduler(TaskMonitor tasksMonitor, PageParser pageParser, JITIndexer jitIndexer, CoreMetrics coreMetrics, MiscMessageWriter messageWriter, ImmutableConfig immutableConfig)`

Method Summary

Modifier and Type	Method	Description
`final Integer`	`getId()`
`final Boolean`	`getSkipTruncated()`
`final Unit`	`setSkipTruncated(Boolean skipTruncated)`
`final Boolean`	`getStoreContent()`
`final Unit`	`setStoreContent(Boolean storeContent)`
`final Boolean`	`getIndexJIT()`
`final Unit`	`setIndexJIT(Boolean indexJIT)`
`final Boolean`	`getParse()`
`final Unit`	`setParse(Boolean parse)`
`final Instant`	`getLastTaskStartTime()`
`final Unit`	`setLastTaskStartTime(Instant lastTaskStartTime)`
`final Instant`	`getLastTaskFinishTime()`
`final Unit`	`setLastTaskFinishTime(Instant lastTaskFinishTime)`
`final String`	`getName()`
`final TaskMonitor`	`getTasksMonitor()`
`final PageParser`	`getPageParser()`
`final JITIndexer`	`getJitIndexer()`
`final CoreMetrics`	`getCoreMetrics()`
`final MiscMessageWriter`	`getMessageWriter()`
`final ImmutableConfig`	`getImmutableConfig()`
`Unit`	`setup(ImmutableConfig jobConf)`
`Params`	`getParams()`
`final JobFetchTask`	`schedule()`	Schedule a queue with the given priority and given poolId
`final List<JobFetchTask>`	`schedule(Integer number)`	Schedule the queues with top priority
`final List<JobFetchTask>`	`schedule(PoolId poolId, Integer number)`	Null queue id means the queue with top priority Consume a fetch item and try to download the target web page
`final Unit`	`finishUnchecked(JobFetchTask fetchTask)`	Finish the fetch item anyway, even if it's failed to download the target page
`final Unit`	`finish(PoolId poolId, Integer itemId)`	Finished downloading the web pageMultiple threaded, non-synchronized class member variables are not allowed inside this method.
`final Unit`	`updateCounters()`	Wait for a while and report task statusTODO: use metrics system instead
`Unit`	`close()`

Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

TaskScheduler

TaskScheduler(TaskMonitor tasksMonitor, PageParser pageParser, JITIndexer jitIndexer, CoreMetrics coreMetrics, MiscMessageWriter messageWriter, ImmutableConfig immutableConfig)

Method Detail

getId
```
 final Integer getId()
```

getSkipTruncated
```
 final Boolean getSkipTruncated()
```

setSkipTruncated

 final Unit setSkipTruncated(Boolean skipTruncated)

getStoreContent
```
 final Boolean getStoreContent()
```

setStoreContent

 final Unit setStoreContent(Boolean storeContent)

getIndexJIT
```
 final Boolean getIndexJIT()
```

setIndexJIT

 final Unit setIndexJIT(Boolean indexJIT)

getParse
```
 final Boolean getParse()
```

setParse
```
 final Unit setParse(Boolean parse)
```

getLastTaskStartTime
```
 final Instant getLastTaskStartTime()
```

setLastTaskStartTime

 final Unit setLastTaskStartTime(Instant lastTaskStartTime)

getLastTaskFinishTime
```
 final Instant getLastTaskFinishTime()
```

setLastTaskFinishTime

 final Unit setLastTaskFinishTime(Instant lastTaskFinishTime)

getName
```
 final String getName()
```

getTasksMonitor
```
 final TaskMonitor getTasksMonitor()
```

getPageParser
```
 final PageParser getPageParser()
```

getJitIndexer
```
 final JITIndexer getJitIndexer()
```

getCoreMetrics
```
 final CoreMetrics getCoreMetrics()
```

getMessageWriter

 final MiscMessageWriter getMessageWriter()

getImmutableConfig

 final ImmutableConfig getImmutableConfig()

setup
```
 Unit setup(ImmutableConfig jobConf)
```

getParams
```
 Params getParams()
```

schedule
```
 final JobFetchTask schedule()
```
Schedule a queue with the given priority and given poolId

schedule

 final List<JobFetchTask> schedule(Integer number)

Schedule the queues with top priority

schedule
```
 final List<JobFetchTask> schedule(PoolId poolId, Integer number)
```
Null queue id means the queue with top priority Consume a fetch item and try to download the target web page

finishUnchecked
```
 final Unit finishUnchecked(JobFetchTask fetchTask)
```
Finish the fetch item anyway, even if it's failed to download the target page

finish
```
 final Unit finish(PoolId poolId, Integer itemId)
```
Finished downloading the web page
Multiple threaded, non-synchronized class member variables are not allowed inside this method.

updateCounters
```
 final Unit updateCounters()
```
Wait for a while and report task status
TODO: use metrics system instead

close
```
 Unit close()
```

Class TaskScheduler

Nested Class Summary

Field Summary

Constructor Summary

Method Summary

Methods inherited from class java.lang.Object

Constructor Detail

TaskScheduler

Method Detail

getId

getSkipTruncated

setSkipTruncated

getStoreContent

setStoreContent

getIndexJIT

setIndexJIT

getParse

setParse

getLastTaskStartTime

setLastTaskStartTime

getLastTaskFinishTime

setLastTaskFinishTime

getName

getTasksMonitor

getPageParser

getJitIndexer

getCoreMetrics

getMessageWriter

getImmutableConfig

setup

getParams

schedule

schedule

schedule

finishUnchecked

finish

updateCounters

close