Package 

Class NewsFetchSchedule

  • All Implemented Interfaces:
    ai.platon.pulsar.common.config.Parameterized , ai.platon.pulsar.crawl.schedule.FetchSchedule

    
    public final class NewsFetchSchedule
    extends AdaptiveFetchSchedule
                        

    This class implements an adaptive re-fetch algorithm.

    NOTE: values of DEC_FACTOR and INC_FACTOR higher than 0.4f may destabilize the algorithm, so that the fetch interval either increases or decreases infinitely, with little relevance to the page changes

    • Method Summary

      Modifier and Type Method Description
      Duration getMaxFetchInterval()
      final ImmutableConfig getConf()
      final MiscMessageWriter getMessageWriter()
      Unit setFetchSchedule(WebPage page, ModifyInfo m) Sets the fetchInterval and fetchTime on a successfully fetched page.
      • Methods inherited from class ai.platon.pulsar.crawl.schedule.NewsFetchSchedule

        getParams, setPageGoneSchedule
      • Methods inherited from class ai.platon.pulsar.crawl.schedule.AdaptiveFetchSchedule

        estimatePrevFetchTime, forceRefetch, initializeSchedule, setPageRetrySchedule, shouldFetch
      • Methods inherited from class java.lang.Object

        clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
    • Constructor Detail

      • NewsFetchSchedule

        NewsFetchSchedule(ImmutableConfig conf, MiscMessageWriter messageWriter)