FetchOptions

All Implemented Interfaces:: ai.platon.pulsar.common.config.Parameterized


public final class FetchOptions
extends CommonOptions

Nested Class Summary

Nested Classes
Modifier and Type	Class	Description
`public class`	`FetchOptions.Companion`

Field Summary

Fields
Modifier and Type	Field	Description
`private String`	`crawlId`
`private String`	`batchId`
`private Integer`	`round`
`private FetchMode`	`fetchMode`
`private Boolean`	`strictDf`
`private Integer`	`numReduceTasks`
`private Integer`	`numPoolThreads`
`private Boolean`	`resume`
`private Boolean`	`parse`
`private Integer`	`limit`
`private Boolean`	`index`
`private String`	`indexerUrl`
`private String`	`indexerCollection`
`private String`	`zkHostString`
`private Integer`	`verbose`
`private Boolean`	`isHelp`
`private Boolean`	`expandAtSign`
`private Boolean`	`acceptUnknownOptions`
`private Boolean`	`allowParameterOverwriting`
`private final String`	`args`
`private final Array<String>`	`argv`

Constructor Summary

Constructors
Constructor	Description
`FetchOptions(ImmutableConfig conf)`
`FetchOptions(Array<String> argv, ImmutableConfig conf)`

Method Summary

Modifier and Type	Method	Description
`final String`	`getCrawlId()`
`final Unit`	`setCrawlId(String crawlId)`
`final String`	`getBatchId()`
`final Unit`	`setBatchId(String batchId)`
`final Integer`	`getRound()`
`final Unit`	`setRound(Integer round)`
`final FetchMode`	`getFetchMode()`
`final Unit`	`setFetchMode(FetchMode fetchMode)`
`final Boolean`	`getStrictDf()`
`final Unit`	`setStrictDf(Boolean strictDf)`
`final Integer`	`getNumReduceTasks()`	Note: in browser mode, reducer tasks/fetch threads/pool threads are not used use browser instances/active browser tabs instead
`final Unit`	`setNumReduceTasks(Integer numReduceTasks)`	Note: in browser mode, reducer tasks/fetch threads/pool threads are not used use browser instances/active browser tabs instead
`final Integer`	`getNumPoolThreads()`
`final Unit`	`setNumPoolThreads(Integer numPoolThreads)`
`final Boolean`	`getResume()`
`final Unit`	`setResume(Boolean resume)`
`final Boolean`	`getParse()`
`final Unit`	`setParse(Boolean parse)`
`final Integer`	`getLimit()`
`final Unit`	`setLimit(Integer limit)`
`final Boolean`	`getIndex()`
`final Unit`	`setIndex(Boolean index)`
`final String`	`getIndexerUrl()`
`final Unit`	`setIndexerUrl(String indexerUrl)`
`final String`	`getIndexerCollection()`
`final Unit`	`setIndexerCollection(String indexerCollection)`
`final String`	`getZkHostString()`
`final Unit`	`setZkHostString(String zkHostString)`
`final Integer`	`getVerbose()`
`final Unit`	`setVerbose(Integer verbose)`
`Boolean`	`getIsHelp()`
`Unit`	`setIsHelp(Boolean isHelp)`
`final Boolean`	`getExpandAtSign()`
`final Unit`	`setExpandAtSign(Boolean expandAtSign)`
`final Boolean`	`getAcceptUnknownOptions()`
`final Unit`	`setAcceptUnknownOptions(Boolean acceptUnknownOptions)`
`final Boolean`	`getAllowParameterOverwriting()`
`final Unit`	`setAllowParameterOverwriting(Boolean allowParameterOverwriting)`
`final String`	`getArgs()`
`final Array<String>`	`getArgv()`	The argument vector
`final Params`	`toParams()`

Methods inherited from class ai.platon.pulsar.common.options.CommonOptions
addObjects, equals, hashCode, parse, parseOrExit, setObjects, toArgsMap, toArgv, toCmdLine, toMutableArgsMap, toString, usage
Methods inherited from class ai.platon.pulsar.common.options.PulsarOptions
getParams
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

FetchOptions
```
FetchOptions(ImmutableConfig conf)
```

FetchOptions

FetchOptions(Array<String> argv, ImmutableConfig conf)

Method Detail

getCrawlId
```
 final String getCrawlId()
```

setCrawlId
```
 final Unit setCrawlId(String crawlId)
```

getBatchId
```
 final String getBatchId()
```

setBatchId
```
 final Unit setBatchId(String batchId)
```

getRound
```
 final Integer getRound()
```

setRound
```
 final Unit setRound(Integer round)
```

getFetchMode
```
 final FetchMode getFetchMode()
```

setFetchMode

 final Unit setFetchMode(FetchMode fetchMode)

getStrictDf
```
 final Boolean getStrictDf()
```

setStrictDf

 final Unit setStrictDf(Boolean strictDf)

getNumReduceTasks
```
 final Integer getNumReduceTasks()
```
Note: in browser mode, reducer tasks/fetch threads/pool threads are not used use browser instances/active browser tabs instead

setNumReduceTasks
```
 final Unit setNumReduceTasks(Integer numReduceTasks)
```
Note: in browser mode, reducer tasks/fetch threads/pool threads are not used use browser instances/active browser tabs instead

getNumPoolThreads
```
 final Integer getNumPoolThreads()
```

setNumPoolThreads

 final Unit setNumPoolThreads(Integer numPoolThreads)

getResume
```
 final Boolean getResume()
```

setResume
```
 final Unit setResume(Boolean resume)
```

getParse
```
 final Boolean getParse()
```

setParse
```
 final Unit setParse(Boolean parse)
```

getLimit
```
 final Integer getLimit()
```

setLimit
```
 final Unit setLimit(Integer limit)
```

getIndex
```
 final Boolean getIndex()
```

setIndex
```
 final Unit setIndex(Boolean index)
```

getIndexerUrl
```
 final String getIndexerUrl()
```

setIndexerUrl

 final Unit setIndexerUrl(String indexerUrl)

getIndexerCollection
```
 final String getIndexerCollection()
```

setIndexerCollection

 final Unit setIndexerCollection(String indexerCollection)

getZkHostString
```
 final String getZkHostString()
```

setZkHostString

 final Unit setZkHostString(String zkHostString)

getVerbose
```
 final Integer getVerbose()
```

setVerbose

 final Unit setVerbose(Integer verbose)

getIsHelp
```
 Boolean getIsHelp()
```

setIsHelp
```
 Unit setIsHelp(Boolean isHelp)
```

getExpandAtSign
```
 final Boolean getExpandAtSign()
```

setExpandAtSign

 final Unit setExpandAtSign(Boolean expandAtSign)

getAcceptUnknownOptions

 final Boolean getAcceptUnknownOptions()

setAcceptUnknownOptions

 final Unit setAcceptUnknownOptions(Boolean acceptUnknownOptions)

getAllowParameterOverwriting

 final Boolean getAllowParameterOverwriting()

setAllowParameterOverwriting

 final Unit setAllowParameterOverwriting(Boolean allowParameterOverwriting)

getArgs
```
 final String getArgs()
```

getArgv
```
 final Array<String> getArgv()
```
The argument vector

toParams
```
 final Params toParams()
```

Class FetchOptions

Nested Class Summary

Field Summary

Constructor Summary

Method Summary

Methods inherited from class ai.platon.pulsar.common.options.CommonOptions

Methods inherited from class ai.platon.pulsar.common.options.PulsarOptions

Methods inherited from class java.lang.Object

Constructor Detail

FetchOptions

FetchOptions

Method Detail

getCrawlId

setCrawlId

getBatchId

setBatchId

getRound

setRound

getFetchMode

setFetchMode

getStrictDf

setStrictDf

getNumReduceTasks

setNumReduceTasks

getNumPoolThreads

setNumPoolThreads

getResume

setResume

getParse

setParse

getLimit

setLimit

getIndex

setIndex

getIndexerUrl

setIndexerUrl

getIndexerCollection

setIndexerCollection

getZkHostString

setZkHostString

getVerbose

setVerbose

getIsHelp

setIsHelp

getExpandAtSign

setExpandAtSign

getAcceptUnknownOptions

setAcceptUnknownOptions

getAllowParameterOverwriting

setAllowParameterOverwriting

getArgs

getArgv

toParams