CoreMetrics

All Implemented Interfaces:: ai.platon.pulsar.common.config.Parameterized , java.lang.AutoCloseable


public final class CoreMetrics
 implements Parameterized, AutoCloseable

Nested Class Summary

Nested Classes
Modifier and Type	Class	Description
`public class`	`CoreMetrics.Companion`

Field Summary

Fields
Modifier and Type	Field	Description
`private final URLUtil.GroupMode`	`groupMode`
`private final Integer`	`maxHostFailureEvents`
`private final Instant`	`startTime`
`private final Duration`	`elapsedTime`
`private final ConcurrentHashMap<String, UrlStat>`	`urlStatistics`
`private final ConcurrentSkipListSet<String>`	`unreachableHosts`
`private final ConcurrentSkipListSet<String>`	`timeoutUrls`
`private final ConcurrentSkipListSet<String>`	`movedUrls`
`private final ConcurrentSkipListSet<String>`	`failedUrls`
`private final ConcurrentSkipListSet<String>`	`deadUrls`
`private final ConcurrentHashMultiset<String>`	`failedHosts`
`private final Meter`	`meterTotalNetworkIFsRecvMBytes`
`private final Meter`	`tasks`
`private final Meter`	`successTasks`
`private final Meter`	`finishedTasks`
`private final MultiMetric`	`persists`
`private final MultiMetric`	`contentPersists`
`private final MultiMetric`	`meterContentMBytes`
`private final MultiMetric`	`persistContentMBytes`
`private final Meter`	`meterContentBytes`
`private final Histogram`	`histogramContentBytes`
`private final Histogram`	`pageImages`
`private final Histogram`	`pageAnchors`
`private final Histogram`	`pageNumbers`
`private final Histogram`	`pageSmallTexts`
`private final Histogram`	`pageHeights`
`private final Long`	`realTimeNetworkIFsRecvBytes`
`private final Long`	`initNetworkIFsRecvBytes`
`private volatile Long`	`totalNetworkIFsRecvBytes`
`private final Long`	`networkIFsRecvBytes`
`private final Long`	`networkIFsRecvBytesPerSecond`
`private final Long`	`networkIFsRecvBytesPerPage`
`private final Float`	`finishedTasksPerSecond`
`private final Float`	`successTasksPerSecond`

Constructor Summary

Constructors
Constructor	Description
`CoreMetrics(MiscMessageWriter messageWriter, ImmutableConfig conf)`

Method Summary

Modifier and Type	Method	Description
`final URLUtil.GroupMode`	`getGroupMode()`
`final Integer`	`getMaxHostFailureEvents()`
`final Instant`	`getStartTime()`
`final Duration`	`getElapsedTime()`
`final ConcurrentHashMap<String, UrlStat>`	`getUrlStatistics()`	Tracking statistics for each host
`final ConcurrentSkipListSet<String>`	`getUnreachableHosts()`	Tracking unreachable hosts
`final ConcurrentSkipListSet<String>`	`getTimeoutUrls()`	Tracking hosts who is failed to fetch tasks.
`final ConcurrentSkipListSet<String>`	`getMovedUrls()`
`final ConcurrentSkipListSet<String>`	`getFailedUrls()`
`final ConcurrentSkipListSet<String>`	`getDeadUrls()`
`final ConcurrentHashMultiset<String>`	`getFailedHosts()`
`final Meter`	`getMeterTotalNetworkIFsRecvMBytes()`
`final Meter`	`getTasks()`	The total bytes of page content of all success web pages
`final Meter`	`getSuccessTasks()`
`final Meter`	`getFinishedTasks()`
`final MultiMetric`	`getPersists()`
`final MultiMetric`	`getContentPersists()`
`final MultiMetric`	`getMeterContentMBytes()`
`final MultiMetric`	`getPersistContentMBytes()`
`final Meter`	`getMeterContentBytes()`
`final Histogram`	`getHistogramContentBytes()`
`final Histogram`	`getPageImages()`
`final Histogram`	`getPageAnchors()`
`final Histogram`	`getPageNumbers()`
`final Histogram`	`getPageSmallTexts()`
`final Histogram`	`getPageHeights()`
`final Long`	`getRealTimeNetworkIFsRecvBytes()`
`final Long`	`getInitNetworkIFsRecvBytes()`	The total all bytes received by the hardware at the application startup
`final Long`	`getTotalNetworkIFsRecvBytes()`	The total all bytes received by the hardware last read from system
`final Unit`	`setTotalNetworkIFsRecvBytes(Long totalNetworkIFsRecvBytes)`	The total all bytes received by the hardware last read from system
`final Long`	`getNetworkIFsRecvBytes()`
`final Long`	`getNetworkIFsRecvBytesPerSecond()`
`final Long`	`getNetworkIFsRecvBytesPerPage()`
`final Float`	`getFinishedTasksPerSecond()`
`final Float`	`getSuccessTasksPerSecond()`
`Params`	`getParams()`
`final Boolean`	`isReachable(String url)`
`final Boolean`	`isUnreachable(String url)`
`final Boolean`	`isFailed(String url)`
`final Unit`	`trackFailedUrl(String url)`
`final Unit`	`trackFailedUrls(Collection<String> urls)`
`final Boolean`	`isTimeout(String url)`
`final Unit`	`markTaskStart()`
`final Unit`	`markTaskStart(Integer size)`
`final Unit`	`trackSuccess(WebPage page)`	Available hosts statistics
`final Unit`	`trackTimeout(String url)`
`final Unit`	`trackMoved(String url)`
`final Boolean`	`trackHostUnreachable(String url, Integer occurrences)`
`final Integer`	`countHostTasks(String host)`
`final String`	`formatStatus()`
`Unit`	`close()`
`final Unit`	`updateSystemInfo()`

Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

CoreMetrics

CoreMetrics(MiscMessageWriter messageWriter, ImmutableConfig conf)

Method Detail

getGroupMode
```
 final URLUtil.GroupMode getGroupMode()
```

getMaxHostFailureEvents

 final Integer getMaxHostFailureEvents()

getStartTime
```
 final Instant getStartTime()
```

getElapsedTime
```
 final Duration getElapsedTime()
```

getUrlStatistics

 final ConcurrentHashMap<String, UrlStat> getUrlStatistics()

Tracking statistics for each host

getUnreachableHosts

 final ConcurrentSkipListSet<String> getUnreachableHosts()

Tracking unreachable hosts

getTimeoutUrls
```
 final ConcurrentSkipListSet<String> getTimeoutUrls()
```
Tracking hosts who is failed to fetch tasks. A host is considered to be a unreachable host if there are too many failure

getMovedUrls

 final ConcurrentSkipListSet<String> getMovedUrls()

getFailedUrls

 final ConcurrentSkipListSet<String> getFailedUrls()

getDeadUrls

 final ConcurrentSkipListSet<String> getDeadUrls()

getFailedHosts

 final ConcurrentHashMultiset<String> getFailedHosts()

getMeterTotalNetworkIFsRecvMBytes

 final Meter getMeterTotalNetworkIFsRecvMBytes()

getTasks
```
 final Meter getTasks()
```
The total bytes of page content of all success web pages

getSuccessTasks
```
 final Meter getSuccessTasks()
```

getFinishedTasks
```
 final Meter getFinishedTasks()
```

getPersists
```
 final MultiMetric getPersists()
```

getContentPersists
```
 final MultiMetric getContentPersists()
```

getMeterContentMBytes

 final MultiMetric getMeterContentMBytes()

getPersistContentMBytes

 final MultiMetric getPersistContentMBytes()

getMeterContentBytes
```
 final Meter getMeterContentBytes()
```

getHistogramContentBytes

 final Histogram getHistogramContentBytes()

getPageImages
```
 final Histogram getPageImages()
```

getPageAnchors
```
 final Histogram getPageAnchors()
```

getPageNumbers
```
 final Histogram getPageNumbers()
```

getPageSmallTexts
```
 final Histogram getPageSmallTexts()
```

getPageHeights
```
 final Histogram getPageHeights()
```

getRealTimeNetworkIFsRecvBytes

 final Long getRealTimeNetworkIFsRecvBytes()

getInitNetworkIFsRecvBytes
```
 final Long getInitNetworkIFsRecvBytes()
```
The total all bytes received by the hardware at the application startup

getTotalNetworkIFsRecvBytes
```
 final Long getTotalNetworkIFsRecvBytes()
```
The total all bytes received by the hardware last read from system

setTotalNetworkIFsRecvBytes
```
 final Unit setTotalNetworkIFsRecvBytes(Long totalNetworkIFsRecvBytes)
```
The total all bytes received by the hardware last read from system

getNetworkIFsRecvBytes
```
 final Long getNetworkIFsRecvBytes()
```

getNetworkIFsRecvBytesPerSecond

 final Long getNetworkIFsRecvBytesPerSecond()

getNetworkIFsRecvBytesPerPage

 final Long getNetworkIFsRecvBytesPerPage()

getFinishedTasksPerSecond

 final Float getFinishedTasksPerSecond()

getSuccessTasksPerSecond
```
 final Float getSuccessTasksPerSecond()
```

getParams
```
 Params getParams()
```

isReachable
```
 final Boolean isReachable(String url)
```

isUnreachable

 final Boolean isUnreachable(String url)

isFailed
```
 final Boolean isFailed(String url)
```

trackFailedUrl
```
 final Unit trackFailedUrl(String url)
```

trackFailedUrls

 final Unit trackFailedUrls(Collection<String> urls)

isTimeout
```
 final Boolean isTimeout(String url)
```

markTaskStart
```
 final Unit markTaskStart()
```

markTaskStart

 final Unit markTaskStart(Integer size)

trackSuccess
```
 final Unit trackSuccess(WebPage page)
```
Available hosts statistics

trackTimeout
```
 final Unit trackTimeout(String url)
```

trackMoved
```
 final Unit trackMoved(String url)
```

trackHostUnreachable

 final Boolean trackHostUnreachable(String url, Integer occurrences)

Parameters:: url - The url

countHostTasks

 final Integer countHostTasks(String host)

formatStatus
```
 final String formatStatus()
```

close
```
 Unit close()
```

updateSystemInfo
```
 final Unit updateSystemInfo()
```

Class CoreMetrics

Nested Class Summary

Field Summary

Constructor Summary

Method Summary

Methods inherited from class java.lang.Object

Constructor Detail

CoreMetrics

Method Detail

getGroupMode

getMaxHostFailureEvents

getStartTime

getElapsedTime

getUrlStatistics

getUnreachableHosts

getTimeoutUrls

getMovedUrls

getFailedUrls

getDeadUrls

getFailedHosts

getMeterTotalNetworkIFsRecvMBytes

getTasks

getSuccessTasks

getFinishedTasks

getPersists

getContentPersists

getMeterContentMBytes

getPersistContentMBytes

getMeterContentBytes

getHistogramContentBytes

getPageImages

getPageAnchors

getPageNumbers

getPageSmallTexts

getPageHeights

getRealTimeNetworkIFsRecvBytes

getInitNetworkIFsRecvBytes

getTotalNetworkIFsRecvBytes

setTotalNetworkIFsRecvBytes

getNetworkIFsRecvBytes

getNetworkIFsRecvBytesPerSecond

getNetworkIFsRecvBytesPerPage

getFinishedTasksPerSecond

getSuccessTasksPerSecond

getParams

isReachable

isUnreachable

isFailed

trackFailedUrl

trackFailedUrls

isTimeout

markTaskStart

markTaskStart

trackSuccess

trackTimeout

trackMoved

trackHostUnreachable

countHostTasks

formatStatus

close

updateSystemInfo