-
public final class FatLinkExtractor
-
-
Nested Class Summary
Nested Classes Modifier and Type Class Description public classFatLinkExtractor.Companion
-
Field Summary
Fields Modifier and Type Field Description private final FatLinkExtractor.Companion.Counterscountersprivate final PulsarSessionsessionprivate final UrlNormalizerPipelinenormalizer
-
Constructor Summary
Constructors Constructor Description FatLinkExtractor(PulsarSession session, UrlNormalizerPipeline normalizer)
-
Method Summary
Modifier and Type Method Description final FatLinkExtractor.Companion.CountersgetCounters()final PulsarSessiongetSession()final UrlNormalizerPipelinegetNormalizer()final Unitparse(WebPage page, FeaturedDocument document, LoadOptions options)final PageFatLinkcreateFatLink(NormUrl seed)final PageFatLinkcreateFatLink(NormUrl seed, WebPage page, Collection<UrlAware> denyList)final PageFatLinkcreateFatLink(NormUrl seed, Collection<UrlAware> denyList)final PageFatLinkcreateFatLink(WebPage page, FeaturedDocument document, LoadOptions options)final PageFatLinkcreateFatLink(NormUrl seed, WebPage page, FeaturedDocument document)final PageFatLinkcreateFatLink(NormUrl seed, WebPage page, FeaturedDocument document, Collection<UrlAware> denyList)Create a fat link. final BooleanshouldFetchVividPage(String url, Duration expires, Instant now)TODO: the logic is different from the one in LoadComponent -
-
Constructor Detail
-
FatLinkExtractor
FatLinkExtractor(PulsarSession session, UrlNormalizerPipeline normalizer)
-
-
Method Detail
-
getCounters
final FatLinkExtractor.Companion.Counters getCounters()
-
getSession
final PulsarSession getSession()
-
getNormalizer
final UrlNormalizerPipeline getNormalizer()
-
parse
final Unit parse(WebPage page, FeaturedDocument document, LoadOptions options)
-
createFatLink
final PageFatLink createFatLink(NormUrl seed)
-
createFatLink
final PageFatLink createFatLink(NormUrl seed, WebPage page, Collection<UrlAware> denyList)
-
createFatLink
final PageFatLink createFatLink(NormUrl seed, Collection<UrlAware> denyList)
-
createFatLink
final PageFatLink createFatLink(WebPage page, FeaturedDocument document, LoadOptions options)
-
createFatLink
final PageFatLink createFatLink(NormUrl seed, WebPage page, FeaturedDocument document)
-
createFatLink
final PageFatLink createFatLink(NormUrl seed, WebPage page, FeaturedDocument document, Collection<UrlAware> denyList)
Create a fat link. If the document is not null, parse links from the document, or if the document is null, try to load the page's vivid link, the vivid link can be parsed and saved recently
-
shouldFetchVividPage
final Boolean shouldFetchVividPage(String url, Duration expires, Instant now)
TODO: the logic is different from the one in LoadComponent
-
-
-
-