-
- All Implemented Interfaces:
-
ai.platon.pulsar.common.config.Parameterized,ai.platon.pulsar.crawl.common.JobInitialized,java.lang.AutoCloseable
public final class FetchMonitor implements Parameterized, JobInitialized, AutoCloseable
Created by vincent on 16-9-24. Copyright @ 2013-2016 Platon AI. All rights reserved
The fetch monitor
-
-
Nested Class Summary
Nested Classes Modifier and Type Class Description public classFetchMonitor.Companion
-
Field Summary
Fields Modifier and Type Field Description private final BooleanisMissionCompleteprivate FetchOptionsoptionsprivate StringjobNameprivate final BooleanisActive
-
Constructor Summary
Constructors Constructor Description FetchMonitor(FetchComponent fetchComponent, ParseComponent parseComponent, TaskMonitor taskMonitor, TaskSchedulers taskSchedulers, JITIndexer jitIndexer, ImmutableConfig conf)
-
Method Summary
Modifier and Type Method Description final BooleangetIsMissionComplete()final FetchOptionsgetOptions()Initialize in setup using job conf final UnitsetOptions(FetchOptions options)Initialize in setup using job conf final StringgetJobName()Initialize in setup using job conf final UnitsetJobName(String jobName)Initialize in setup using job conf final BooleangetIsActive()Unitsetup(ImmutableConfig jobConf)ParamsgetParams()final Unitstart(ReducerContext<IntWritable, out IFetchEntry, String, GWebPage> context)final UnitregisterFeedThread(FeedLoop feedLoop)final UnitunregisterFeedThread(FeedLoop feedLoop)final UnitregisterFetchLoop(FetchLoop fetchLoop)final UnitunregisterFetchLoop(FetchLoop fetchLoop)Unitclose()-
-
Constructor Detail
-
FetchMonitor
FetchMonitor(FetchComponent fetchComponent, ParseComponent parseComponent, TaskMonitor taskMonitor, TaskSchedulers taskSchedulers, JITIndexer jitIndexer, ImmutableConfig conf)
-
-
Method Detail
-
getIsMissionComplete
final Boolean getIsMissionComplete()
-
getOptions
final FetchOptions getOptions()
Initialize in setup using job conf
-
setOptions
final Unit setOptions(FetchOptions options)
Initialize in setup using job conf
-
getJobName
final String getJobName()
Initialize in setup using job conf
-
setJobName
final Unit setJobName(String jobName)
Initialize in setup using job conf
-
getIsActive
final Boolean getIsActive()
-
getParams
Params getParams()
-
start
final Unit start(ReducerContext<IntWritable, out IFetchEntry, String, GWebPage> context)
-
registerFeedThread
final Unit registerFeedThread(FeedLoop feedLoop)
-
unregisterFeedThread
final Unit unregisterFeedThread(FeedLoop feedLoop)
-
registerFetchLoop
final Unit registerFetchLoop(FetchLoop fetchLoop)
-
unregisterFetchLoop
final Unit unregisterFetchLoop(FetchLoop fetchLoop)
-
-
-
-