--- apiVersion: v1 kind: List items: - apiVersion: v1 kind: ConfigMap metadata: labels: funktion.fabric8.io/kind: Connector provider: fabric8 project: connector-file version: 1.1.9 group: io.fabric8.funktion.connector name: file data: deployment.yml: | --- apiVersion: extensions/v1beta1 kind: Deployment metadata: labels: funktion.fabric8.io/kind: Subscription connector: file spec: replicas: 1 template: metadata: labels: funktion.fabric8.io/kind: Subscription connector: file spec: containers: - image: fabric8/connector-file:1.1.9 name: connector schema.yml: | --- component: kind: component scheme: file syntax: file:directoryName title: File description: The file component is used for reading or writing files. label: core,file deprecated: false async: false javaType: org.apache.camel.component.file.FileComponent groupId: org.apache.camel artifactId: camel-core version: 2.18.1 componentProperties: {} properties: directoryName: kind: path group: common required: true type: string javaType: java.io.File deprecated: false secret: false description: The starting directory charset: kind: parameter group: common type: string javaType: java.lang.String deprecated: false secret: false description: This option is used to specify the encoding of the file. You can use this on the consumer to specify the encodings of the files which allow Camel to know the charset it should load the file content in case the file content is being accessed. Likewise when writing a file you can use this option to specify which charset to write the file as well. doneFileName: kind: parameter group: common type: string javaType: java.lang.String deprecated: false secret: false description: 'Producer: If provided then Camel will write a 2nd done file when the original file has been written. The done file will be empty. This option configures what file name to use. Either you can specify a fixed name. Or you can use dynamic placeholders. The done file will always be written in the same folder as the original file. Consumer: If provided Camel will only consume files if a done file exists. This option configures what file name to use. Either you can specify a fixed name. Or you can use dynamic placeholders.The done file is always expected in the same folder as the original file. Only $file.name and $file.name.noext is supported as dynamic placeholders.' fileName: kind: parameter group: common type: string javaType: java.lang.String deprecated: false secret: false description: 'Use Expression such as File Language to dynamically set the filename. For consumers it''s used as a filename filter. For producers it''s used to evaluate the filename to write. If an expression is set it take precedence over the CamelFileName header. (Note: The header itself can also be an Expression). The expression options support both String and Expression types. If the expression is a String type it is always evaluated using the File Language. If the expression is an Expression type the specified Expression type is used - this allows you for instance to use OGNL expressions. For the consumer you can use it to filter filenames so you can for instance consume today''s file using the File Language syntax: mydata-$date:now:yyyyMMdd.txt. The producers support the CamelOverruleFileName header which takes precedence over any existing CamelFileName header; the CamelOverruleFileName is a header that is used only once and makes it easier as this avoids to temporary store CamelFileName and have to restore it afterwards.' bridgeErrorHandler: kind: parameter group: consumer label: consumer type: boolean javaType: boolean optionalPrefix: consumer. deprecated: false secret: false defaultValue: false description: Allows for bridging the consumer to the Camel routing Error Handler which mean any exceptions occurred while the consumer is trying to pickup incoming messages or the likes will now be processed as a message and handled by the routing Error Handler. By default the consumer will use the org.apache.camel.spi.ExceptionHandler to deal with exceptions that will be logged at WARN/ERROR level and ignored. delete: kind: parameter group: consumer label: consumer type: boolean javaType: boolean deprecated: false secret: false defaultValue: false description: If true the file will be deleted after it is processed successfully. moveFailed: kind: parameter group: consumer label: consumer type: string javaType: java.lang.String deprecated: false secret: false description: 'Sets the move failure expression based on Simple language. For example to move files into a .error subdirectory use: .error. Note: When moving the files to the fail location Camel will handle the error and will not pick up the file again.' noop: kind: parameter group: consumer label: consumer type: boolean javaType: boolean deprecated: false secret: false defaultValue: false description: If true the file is not moved or deleted in any way. This option is good for readonly data or for ETL type requirements. If noop=true Camel will set idempotent=true as well to avoid consuming the same files over and over again. preMove: kind: parameter group: consumer label: consumer type: string javaType: java.lang.String deprecated: false secret: false description: Expression (such as File Language) used to dynamically set the filename when moving it before processing. For example to move in-progress files into the order directory set this value to order. recursive: kind: parameter group: consumer label: consumer type: boolean javaType: boolean deprecated: false secret: false defaultValue: false description: If a directory will look for files in all the sub-directories as well. sendEmptyMessageWhenIdle: kind: parameter group: consumer label: consumer type: boolean javaType: boolean optionalPrefix: consumer. deprecated: false secret: false defaultValue: false description: If the polling consumer did not poll any files you can enable this option to send an empty message (no body) instead. directoryMustExist: kind: parameter group: consumer (advanced) label: consumer,advanced type: boolean javaType: boolean deprecated: false secret: false defaultValue: false description: Similar to startingDirectoryMustExist but this applies during polling recursive sub directories. exceptionHandler: kind: parameter group: consumer (advanced) label: consumer,advanced type: object javaType: org.apache.camel.spi.ExceptionHandler optionalPrefix: consumer. deprecated: false secret: false description: To let the consumer use a custom ExceptionHandler. Notice if the option bridgeErrorHandler is enabled then this options is not in use. By default the consumer will deal with exceptions that will be logged at WARN/ERROR level and ignored. exchangePattern: kind: parameter group: consumer (advanced) label: consumer,advanced type: string javaType: org.apache.camel.ExchangePattern enum: - InOnly - RobustInOnly - InOut - InOptionalOut - OutOnly - RobustOutOnly - OutIn - OutOptionalIn deprecated: false secret: false description: Sets the default exchange pattern when creating an exchange. extendedAttributes: kind: parameter group: consumer (advanced) label: consumer,advanced type: string javaType: java.lang.String deprecated: false secret: false description: 'To define which file attributes of interest. Like posix:permissionsposix:ownerbasic:lastAccessTime it supports basic wildcard like posix: basic:lastAccessTime' inProgressRepository: kind: parameter group: consumer (advanced) label: consumer,advanced type: object javaType: org.apache.camel.spi.IdempotentRepository deprecated: false secret: false description: A pluggable in-progress repository org.apache.camel.spi.IdempotentRepository. The in-progress repository is used to account the current in progress files being consumed. By default a memory based repository is used. localWorkDirectory: kind: parameter group: consumer (advanced) label: consumer,advanced type: string javaType: java.lang.String deprecated: false secret: false description: When consuming a local work directory can be used to store the remote file content directly in local files to avoid loading the content into memory. This is beneficial if you consume a very big remote file and thus can conserve memory. onCompletionExceptionHandler: kind: parameter group: consumer (advanced) label: consumer,advanced type: object javaType: org.apache.camel.spi.ExceptionHandler deprecated: false secret: false description: To use a custom org.apache.camel.spi.ExceptionHandler to handle any thrown exceptions that happens during the file on completion process where the consumer does either a commit or rollback. The default implementation will log any exception at WARN level and ignore. pollStrategy: kind: parameter group: consumer (advanced) label: consumer,advanced type: object javaType: org.apache.camel.spi.PollingConsumerPollStrategy optionalPrefix: consumer. deprecated: false secret: false description: A pluggable org.apache.camel.PollingConsumerPollingStrategy allowing you to provide your custom implementation to control error handling usually occurred during the poll operation before an Exchange have been created and being routed in Camel. In other words the error occurred while the polling was gathering information for instance access to a file network failed so Camel cannot access it to scan for files. The default implementation will log the caused exception at WARN level and ignore it. probeContentType: kind: parameter group: consumer (advanced) label: consumer,advanced type: boolean javaType: boolean deprecated: false secret: false defaultValue: false description: Whether to enable probing of the content type. If enable then the consumer uses link FilesprobeContentType(java.nio.file.Path) to determine the content-type of the file and store that as a header with key link ExchangeFILE_CONTENT_TYPE on the Message. processStrategy: kind: parameter group: consumer (advanced) label: consumer,advanced type: object javaType: org.apache.camel.component.file.GenericFileProcessStrategy deprecated: false secret: false description: A pluggable org.apache.camel.component.file.GenericFileProcessStrategy allowing you to implement your own readLock option or similar. Can also be used when special conditions must be met before a file can be consumed such as a special ready file exists. If this option is set then the readLock option does not apply. startingDirectoryMustExist: kind: parameter group: consumer (advanced) label: consumer,advanced type: boolean javaType: boolean deprecated: false secret: false defaultValue: false description: Whether the starting directory must exist. Mind that the autoCreate option is default enabled which means the starting directory is normally auto created if it doesn't exist. You can disable autoCreate and enable this to ensure the starting directory must exist. Will thrown an exception if the directory doesn't exist. fileExist: kind: parameter group: producer label: producer type: string javaType: org.apache.camel.component.file.GenericFileExist enum: - Override - Append - Fail - Ignore - Move - TryRename deprecated: false secret: false defaultValue: Override description: What to do if a file already exists with the same name. Override which is the default replaces the existing file. Append - adds content to the existing file. Fail - throws a GenericFileOperationException indicating that there is already an existing file. Ignore - silently ignores the problem and does not override the existing file but assumes everything is okay. Move - option requires to use the moveExisting option to be configured as well. The option eagerDeleteTargetFile can be used to control what to do if an moving the file and there exists already an existing file otherwise causing the move operation to fail. The Move option will move any existing files before writing the target file. TryRename Camel is only applicable if tempFileName option is in use. This allows to try renaming the file from the temporary name to the actual name without doing any exists check.This check may be faster on some file systems and especially FTP servers. flatten: kind: parameter group: producer label: producer type: boolean javaType: boolean deprecated: false secret: false defaultValue: false description: Flatten is used to flatten the file name path to strip any leading paths so it's just the file name. This allows you to consume recursively into sub-directories but when you eg write the files to another directory they will be written in a single directory. Setting this to true on the producer enforces that any file name in CamelFileName header will be stripped for any leading paths. moveExisting: kind: parameter group: producer label: producer type: string javaType: java.lang.String deprecated: false secret: false description: 'Expression (such as File Language) used to compute file name to use when fileExist=Move is configured. To move files into a backup subdirectory just enter backup. This option only supports the following File Language tokens: file:name file:name.ext file:name.noext file:onlyname file:onlyname.noext file:ext and file:parent. Notice the file:parent is not supported by the FTP component as the FTP component can only move any existing files to a relative directory based on current dir as base.' tempFileName: kind: parameter group: producer label: producer type: string javaType: java.lang.String deprecated: false secret: false description: The same as tempPrefix option but offering a more fine grained control on the naming of the temporary filename as it uses the File Language. tempPrefix: kind: parameter group: producer label: producer type: string javaType: java.lang.String deprecated: false secret: false description: This option is used to write the file using a temporary name and then after the write is complete rename it to the real name. Can be used to identify files being written and also avoid consumers (not using exclusive read locks) reading in progress files. Is often used by FTP when uploading big files. allowNullBody: kind: parameter group: producer (advanced) label: producer,advanced type: boolean javaType: boolean deprecated: false secret: false defaultValue: false description: Used to specify if a null body is allowed during file writing. If set to true then an empty file will be created when set to false and attempting to send a null body to the file component a GenericFileWriteException of 'Cannot write null body to file.' will be thrown. If the fileExist option is set to 'Override' then the file will be truncated and if set to append the file will remain unchanged. chmod: kind: parameter group: producer (advanced) label: producer,advanced type: string javaType: java.lang.String deprecated: false secret: false description: Specify the file permissions which is sent by the producer the chmod value must be between 000 and 777; If there is a leading digit like in 0755 we will ignore it. chmodDirectory: kind: parameter group: producer (advanced) label: producer,advanced type: string javaType: java.lang.String deprecated: false secret: false description: Specify the directory permissions used when the producer creates missing directories the chmod value must be between 000 and 777; If there is a leading digit like in 0755 we will ignore it. eagerDeleteTargetFile: kind: parameter group: producer (advanced) label: producer,advanced type: boolean javaType: boolean deprecated: false secret: false defaultValue: true description: Whether or not to eagerly delete any existing target file. This option only applies when you use fileExists=Override and the tempFileName option as well. You can use this to disable (set it to false) deleting the target file before the temp file is written. For example you may write big files and want the target file to exists during the temp file is being written. This ensure the target file is only deleted until the very last moment just before the temp file is being renamed to the target filename. This option is also used to control whether to delete any existing files when fileExist=Move is enabled and an existing file exists. If this option copyAndDeleteOnRenameFails false then an exception will be thrown if an existing file existed if its true then the existing file is deleted before the move operation. forceWrites: kind: parameter group: producer (advanced) label: producer,advanced type: boolean javaType: boolean deprecated: false secret: false defaultValue: true description: Whether to force syncing writes to the file system. You can turn this off if you do not want this level of guarantee for example if writing to logs / audit logs etc; this would yield better performance. keepLastModified: kind: parameter group: producer (advanced) label: producer,advanced type: boolean javaType: boolean deprecated: false secret: false defaultValue: false description: 'Will keep the last modified timestamp from the source file (if any). Will use the Exchange.FILE_LAST_MODIFIED header to located the timestamp. This header can contain either a java.util.Date or long with the timestamp. If the timestamp exists and the option is enabled it will set this timestamp on the written file. Note: This option only applies to the file producer. You cannot use this option with any of the ftp producers.' autoCreate: kind: parameter group: advanced label: advanced type: boolean javaType: boolean deprecated: false secret: false defaultValue: true description: Automatically create missing directories in the file's pathname. For the file consumer that means creating the starting directory. For the file producer it means the directory the files should be written to. bufferSize: kind: parameter group: advanced label: advanced type: integer javaType: int deprecated: false secret: false defaultValue: "131072" description: Write buffer sized in bytes. copyAndDeleteOnRenameFail: kind: parameter group: advanced label: advanced type: boolean javaType: boolean deprecated: false secret: false defaultValue: true description: Whether to fallback and do a copy and delete file in case the file could not be renamed directly. This option is not available for the FTP component. renameUsingCopy: kind: parameter group: advanced label: advanced type: boolean javaType: boolean deprecated: false secret: false defaultValue: false description: Perform rename operations using a copy and delete strategy. This is primarily used in environments where the regular rename operation is unreliable (e.g. across different file systems or networks). This option takes precedence over the copyAndDeleteOnRenameFail parameter that will automatically fall back to the copy and delete strategy but only after additional delays. synchronous: kind: parameter group: advanced label: advanced type: boolean javaType: boolean deprecated: false secret: false defaultValue: false description: Sets whether synchronous processing should be strictly used or Camel is allowed to use asynchronous processing (if supported). antExclude: kind: parameter group: filter label: consumer,filter type: string javaType: java.lang.String deprecated: false secret: false description: Ant style filter exclusion. If both antInclude and antExclude are used antExclude takes precedence over antInclude. Multiple exclusions may be specified in comma-delimited format. antFilterCaseSensitive: kind: parameter group: filter label: consumer,filter type: boolean javaType: boolean deprecated: false secret: false defaultValue: true description: Sets case sensitive flag on ant fiter antInclude: kind: parameter group: filter label: consumer,filter type: string javaType: java.lang.String deprecated: false secret: false description: Ant style filter inclusion. Multiple inclusions may be specified in comma-delimited format. eagerMaxMessagesPerPoll: kind: parameter group: filter label: consumer,filter type: boolean javaType: boolean deprecated: false secret: false defaultValue: true description: Allows for controlling whether the limit from maxMessagesPerPoll is eager or not. If eager then the limit is during the scanning of files. Where as false would scan all files and then perform sorting. Setting this option to false allows for sorting all files first and then limit the poll. Mind that this requires a higher memory usage as all file details are in memory to perform the sorting. exclude: kind: parameter group: filter label: consumer,filter type: string javaType: java.lang.String deprecated: false secret: false description: Is used to exclude files if filename matches the regex pattern (matching is case in-senstive). Notice if you use symbols such as plus sign and others you would need to configure this using the RAW() syntax if configuring this as an endpoint uri. See more details at configuring endpoint uris filter: kind: parameter group: filter label: consumer,filter type: object javaType: org.apache.camel.component.file.GenericFileFilter deprecated: false secret: false description: Pluggable filter as a org.apache.camel.component.file.GenericFileFilter class. Will skip files if filter returns false in its accept() method. filterDirectory: kind: parameter group: filter label: consumer,filter type: string javaType: java.lang.String deprecated: false secret: false description: Filters the directory based on Simple language. For example to filter on current date you can use a simple date pattern such as $date:now:yyyMMdd filterFile: kind: parameter group: filter label: consumer,filter type: string javaType: java.lang.String deprecated: false secret: false description: Filters the file based on Simple language. For example to filter on file size you can use $file:size 5000 idempotent: kind: parameter group: filter label: consumer,filter type: boolean javaType: java.lang.Boolean deprecated: false secret: false defaultValue: false description: Option to use the Idempotent Consumer EIP pattern to let Camel skip already processed files. Will by default use a memory based LRUCache that holds 1000 entries. If noop=true then idempotent will be enabled as well to avoid consuming the same files over and over again. idempotentKey: kind: parameter group: filter label: consumer,filter type: string javaType: java.lang.String deprecated: false secret: false description: 'To use a custom idempotent key. By default the absolute path of the file is used. You can use the File Language for example to use the file name and file size you can do: idempotentKey=$file:name-$file:size' idempotentRepository: kind: parameter group: filter label: consumer,filter type: object javaType: org.apache.camel.spi.IdempotentRepository deprecated: false secret: false description: A pluggable repository org.apache.camel.spi.IdempotentRepository which by default use MemoryMessageIdRepository if none is specified and idempotent is true. include: kind: parameter group: filter label: consumer,filter type: string javaType: java.lang.String deprecated: false secret: false description: Is used to include files if filename matches the regex pattern (matching is case in-senstive). Notice if you use symbols such as plus sign and others you would need to configure this using the RAW() syntax if configuring this as an endpoint uri. See more details at configuring endpoint uris maxDepth: kind: parameter group: filter label: consumer,filter type: integer javaType: int deprecated: false secret: false defaultValue: "2147483647" description: The maximum depth to traverse when recursively processing a directory. maxMessagesPerPoll: kind: parameter group: filter label: consumer,filter type: integer javaType: int deprecated: false secret: false description: 'To define a maximum messages to gather per poll. By default no maximum is set. Can be used to set a limit of e.g. 1000 to avoid when starting up the server that there are thousands of files. Set a value of 0 or negative to disabled it. Notice: If this option is in use then the File and FTP components will limit before any sorting. For example if you have 100000 files and use maxMessagesPerPoll=500 then only the first 500 files will be picked up and then sorted. You can use the eagerMaxMessagesPerPoll option and set this to false to allow to scan all files first and then sort afterwards.' minDepth: kind: parameter group: filter label: consumer,filter type: integer javaType: int deprecated: false secret: false description: The minimum depth to start processing when recursively processing a directory. Using minDepth=1 means the base directory. Using minDepth=2 means the first sub directory. move: kind: parameter group: filter label: consumer,filter type: string javaType: java.lang.String deprecated: false secret: false description: Expression (such as Simple Language) used to dynamically set the filename when moving it after processing. To move files into a .done subdirectory just enter .done. exclusiveReadLockStrategy: kind: parameter group: lock label: consumer,lock type: object javaType: org.apache.camel.component.file.GenericFileExclusiveReadLockStrategy deprecated: false secret: false description: Pluggable read-lock as a org.apache.camel.component.file.GenericFileExclusiveReadLockStrategy implementation. readLock: kind: parameter group: lock label: consumer,lock type: string javaType: java.lang.String enum: - none - markerFile - fileLock - rename - changed - idempotent deprecated: false secret: false description: 'Used by consumer to only poll the files if it has exclusive read-lock on the file (i.e. the file is not in-progress or being written). Camel will wait until the file lock is granted. This option provides the build in strategies: none - No read lock is in use markerFile - Camel creates a marker file (fileName.camelLock) and then holds a lock on it. This option is not available for the FTP component changed - Changed is using file length/modification timestamp to detect whether the file is currently being copied or not. Will at least use 1 sec to determine this so this option cannot consume files as fast as the others but can be more reliable as the JDK IO API cannot always determine whether a file is currently being used by another process. The option readLockCheckInterval can be used to set the check frequency. fileLock - is for using java.nio.channels.FileLock. This option is not avail for the FTP component. This approach should be avoided when accessing a remote file system via a mount/share unless that file system supports distributed file locks. rename - rename is for using a try to rename the file as a test if we can get exclusive read-lock. idempotent - (only for file component) idempotent is for using a idempotentRepository as the read-lock. This allows to use read locks that supports clustering if the idempotent repository implementation supports that. Notice: The various read locks is not all suited to work in clustered mode where concurrent consumers on different nodes is competing for the same files on a shared file system. The markerFile using a close to atomic operation to create the empty marker file but its not guaranteed to work in a cluster. The fileLock may work better but then the file system need to support distributed file locks and so on. Using the idempotent read lock can support clustering if the idempotent repository supports clustering such as Hazelcast Component or Infinispan.' readLockCheckInterval: kind: parameter group: lock label: consumer,lock type: integer javaType: long deprecated: false secret: false defaultValue: "1000" description: 'Interval in millis for the read-lock if supported by the read lock. This interval is used for sleeping between attempts to acquire the read lock. For example when using the changed read lock you can set a higher interval period to cater for slow writes. The default of 1 sec. may be too fast if the producer is very slow writing the file. Notice: For FTP the default readLockCheckInterval is 5000. The readLockTimeout value must be higher than readLockCheckInterval but a rule of thumb is to have a timeout that is at least 2 or more times higher than the readLockCheckInterval. This is needed to ensure that amble time is allowed for the read lock process to try to grab the lock before the timeout was hit.' readLockDeleteOrphanLockFiles: kind: parameter group: lock label: consumer,lock type: boolean javaType: boolean deprecated: false secret: false defaultValue: true description: Whether or not read lock with marker files should upon startup delete any orphan read lock files which may have been left on the file system if Camel was not properly shutdown (such as a JVM crash). If turning this option to false then any orphaned lock file will cause Camel to not attempt to pickup that file this could also be due another node is concurrently reading files from the same shared directory. readLockLoggingLevel: kind: parameter group: lock label: consumer,lock type: string javaType: org.apache.camel.LoggingLevel enum: - TRACE - DEBUG - INFO - WARN - ERROR - OFF deprecated: false secret: false defaultValue: WARN description: 'Logging level used when a read lock could not be acquired. By default a WARN is logged. You can change this level for example to OFF to not have any logging. This option is only applicable for readLock of types: changed fileLock rename.' readLockMarkerFile: kind: parameter group: lock label: consumer,lock type: boolean javaType: boolean deprecated: false secret: false defaultValue: true description: Whether to use marker file with the changed rename or exclusive read lock types. By default a marker file is used as well to guard against other processes picking up the same files. This behavior can be turned off by setting this option to false. For example if you do not want to write marker files to the file systems by the Camel application. readLockMinAge: kind: parameter group: lock label: consumer,lock type: integer javaType: long deprecated: false secret: false defaultValue: "0" description: This option applied only for readLock=change. This option allows to specify a minimum age the file must be before attempting to acquire the read lock. For example use readLockMinAge=300s to require the file is at last 5 minutes old. This can speedup the changed read lock as it will only attempt to acquire files which are at least that given age. readLockMinLength: kind: parameter group: lock label: consumer,lock type: integer javaType: long deprecated: false secret: false defaultValue: "1" description: This option applied only for readLock=changed. This option allows you to configure a minimum file length. By default Camel expects the file to contain data and thus the default value is 1. You can set this option to zero to allow consuming zero-length files. readLockRemoveOnCommit: kind: parameter group: lock label: consumer,lock type: boolean javaType: boolean deprecated: false secret: false defaultValue: false description: This option applied only for readLock=idempotent. This option allows to specify whether to remove the file name entry from the idempotent repository when processing the file is succeeded and a commit happens. By default the file is not removed which ensures that any race-condition do not occur so another active node may attempt to grab the file. Instead the idempotent repository may support eviction strategies that you can configure to evict the file name entry after X minutes - this ensures no problems with race conditions. readLockRemoveOnRollback: kind: parameter group: lock label: consumer,lock type: boolean javaType: boolean deprecated: false secret: false defaultValue: true description: This option applied only for readLock=idempotent. This option allows to specify whether to remove the file name entry from the idempotent repository when processing the file failed and a rollback happens. If this option is false then the file name entry is confirmed (as if the file did a commit). readLockTimeout: kind: parameter group: lock label: consumer,lock type: integer javaType: long deprecated: false secret: false defaultValue: "10000" description: 'Optional timeout in millis for the read-lock if supported by the read-lock. If the read-lock could not be granted and the timeout triggered then Camel will skip the file. At next poll Camel will try the file again and this time maybe the read-lock could be granted. Use a value of 0 or lower to indicate forever. Currently fileLock changed and rename support the timeout. Notice: For FTP the default readLockTimeout value is 20000 instead of 10000. The readLockTimeout value must be higher than readLockCheckInterval but a rule of thumb is to have a timeout that is at least 2 or more times higher than the readLockCheckInterval. This is needed to ensure that amble time is allowed for the read lock process to try to grab the lock before the timeout was hit.' backoffErrorThreshold: kind: parameter group: scheduler label: consumer,scheduler type: integer javaType: int optionalPrefix: consumer. deprecated: false secret: false description: The number of subsequent error polls (failed due some error) that should happen before the backoffMultipler should kick-in. backoffIdleThreshold: kind: parameter group: scheduler label: consumer,scheduler type: integer javaType: int optionalPrefix: consumer. deprecated: false secret: false description: The number of subsequent idle polls that should happen before the backoffMultipler should kick-in. backoffMultiplier: kind: parameter group: scheduler label: consumer,scheduler type: integer javaType: int optionalPrefix: consumer. deprecated: false secret: false description: To let the scheduled polling consumer backoff if there has been a number of subsequent idles/errors in a row. The multiplier is then the number of polls that will be skipped before the next actual attempt is happening again. When this option is in use then backoffIdleThreshold and/or backoffErrorThreshold must also be configured. delay: kind: parameter group: scheduler label: consumer,scheduler type: integer javaType: long optionalPrefix: consumer. deprecated: false secret: false defaultValue: "500" description: Milliseconds before the next poll. The default value is 500. You can also specify time values using units such as 60s (60 seconds) 5m30s (5 minutes and 30 seconds) and 1h (1 hour). greedy: kind: parameter group: scheduler label: consumer,scheduler type: boolean javaType: boolean optionalPrefix: consumer. deprecated: false secret: false defaultValue: false description: If greedy is enabled then the ScheduledPollConsumer will run immediately again if the previous run polled 1 or more messages. initialDelay: kind: parameter group: scheduler label: consumer,scheduler type: integer javaType: long optionalPrefix: consumer. deprecated: false secret: false defaultValue: "1000" description: Milliseconds before the first poll starts. The default value is 1000. You can also specify time values using units such as 60s (60 seconds) 5m30s (5 minutes and 30 seconds) and 1h (1 hour). runLoggingLevel: kind: parameter group: scheduler label: consumer,scheduler type: string javaType: org.apache.camel.LoggingLevel enum: - TRACE - DEBUG - INFO - WARN - ERROR - OFF optionalPrefix: consumer. deprecated: false secret: false defaultValue: TRACE description: The consumer logs a start/complete log line when it polls. This option allows you to configure the logging level for that. scheduledExecutorService: kind: parameter group: scheduler label: consumer,scheduler type: object javaType: java.util.concurrent.ScheduledExecutorService optionalPrefix: consumer. deprecated: false secret: false description: Allows for configuring a custom/shared thread pool to use for the consumer. By default each consumer has its own single threaded thread pool. This option allows you to share a thread pool among multiple consumers. scheduler: kind: parameter group: scheduler label: consumer,scheduler type: string javaType: org.apache.camel.spi.ScheduledPollConsumerScheduler enum: - none - spring - quartz2 optionalPrefix: consumer. deprecated: false secret: false defaultValue: none description: 'Allow to plugin a custom org.apache.camel.spi.ScheduledPollConsumerScheduler to use as the scheduler for firing when the polling consumer runs. The default implementation uses the ScheduledExecutorService and there is a Quartz2 and Spring based which supports CRON expressions. Notice: If using a custom scheduler then the options for initialDelay useFixedDelay timeUnit and scheduledExecutorService may not be in use. Use the text quartz2 to refer to use the Quartz2 scheduler; and use the text spring to use the Spring based; and use the text myScheduler to refer to a custom scheduler by its id in the Registry. See Quartz2 page for an example.' schedulerProperties: kind: parameter group: scheduler label: consumer,scheduler type: object javaType: java.util.Map prefix: scheduler. multiValue: true deprecated: false secret: false description: To configure additional properties when using a custom scheduler or any of the Quartz2 Spring based scheduler. startScheduler: kind: parameter group: scheduler label: consumer,scheduler type: boolean javaType: boolean optionalPrefix: consumer. deprecated: false secret: false defaultValue: true description: Whether the scheduler should be auto started. timeUnit: kind: parameter group: scheduler label: consumer,scheduler type: string javaType: java.util.concurrent.TimeUnit enum: - NANOSECONDS - MICROSECONDS - MILLISECONDS - SECONDS - MINUTES - HOURS - DAYS optionalPrefix: consumer. deprecated: false secret: false defaultValue: MILLISECONDS description: Time unit for initialDelay and delay options. useFixedDelay: kind: parameter group: scheduler label: consumer,scheduler type: boolean javaType: boolean optionalPrefix: consumer. deprecated: false secret: false defaultValue: true description: Controls if fixed delay or fixed rate is used. See ScheduledExecutorService in JDK for details. shuffle: kind: parameter group: sort label: consumer,sort type: boolean javaType: boolean deprecated: false secret: false defaultValue: false description: To shuffle the list of files (sort in random order) sortBy: kind: parameter group: sort label: consumer,sort type: string javaType: java.lang.String deprecated: false secret: false description: Built-in sort by using the File Language. Supports nested sorts so you can have a sort by file name and as a 2nd group sort by modified date. sorter: kind: parameter group: sort label: consumer,sort type: object javaType: java.util.Comparator> deprecated: false secret: false description: Pluggable sorter as a java.util.Comparator class. documentation.adoc: "[[File2-FileComponent]]\nFile Component\n~~~~~~~~~~~~~~\n\ \nThe File component provides access to file systems, allowing files to be\n\ processed by any other Camel link:components.html[Components] or\nmessages from\ \ other components to be saved to disk.\n\n[[File2-URIformat]]\nURI format\n\ ^^^^^^^^^^\n\n[source,java]\n----------------------------\nfile:directoryName[?options]\n\ ----------------------------\n\nor\n\n[source,java]\n------------------------------\n\ file://directoryName[?options]\n------------------------------\n\nWhere *directoryName*\ \ represents the underlying file directory.\n\nYou can append query options\ \ to the URI in the following format,\n`?option=value&option=value&...`\n\n\ *Only directories*\n\nCamel supports only endpoints configured with a starting\ \ directory. So\nthe *directoryName* must be a directory. +\n If you want to\ \ consume a single file only, you can use the *fileName*\noption, e.g. by setting\ \ `fileName=thefilename`. +\n Also, the starting directory must not contain\ \ dynamic expressions with\n$\\{ } placeholders. Again use the `fileName` option\ \ to specify the\ndynamic part of the filename.\n\nWARNING:*Avoid reading files\ \ currently being written by another\napplication*\n\nBeware the JDK File IO\ \ API is a bit limited in detecting whether another\napplication is currently\ \ writing/copying a file. And the implementation\ncan be different depending\ \ on OS platform as well. This could lead to\nthat Camel thinks the file is\ \ not locked by another process and start\nconsuming it. Therefore you have\ \ to do you own investigation what suites\nyour environment. To help with this\ \ Camel provides different `readLock`\noptions and `doneFileName` option that\ \ you can use. See also the section\n_Consuming files from folders where others\ \ drop files directly_.\n\n[[File2-URIOptions]]\nURI Options\n^^^^^^^^^^^\n\n\ [[File2-Options]]\nFile2 Options\n+++++++++++++\n\n\n// component options: START\n\ The File component has no options.\n// component options: END\n\n\n\n\n// endpoint\ \ options: START\nThe File component supports 81 endpoint options which are\ \ listed below:\n\n{% raw %}\n[width=\"100%\",cols=\"2,1,1m,1m,5\",options=\"\ header\"]\n|=======================================================================\n\ | Name | Group | Default | Java Type | Description\n| directoryName | common\ \ | | File | *Required* The starting directory\n| charset | common | | String\ \ | This option is used to specify the encoding of the file. You can use this\ \ on the consumer to specify the encodings of the files which allow Camel to\ \ know the charset it should load the file content in case the file content\ \ is being accessed. Likewise when writing a file you can use this option to\ \ specify which charset to write the file as well.\n| doneFileName | common\ \ | | String | Producer: If provided then Camel will write a 2nd done file\ \ when the original file has been written. The done file will be empty. This\ \ option configures what file name to use. Either you can specify a fixed name.\ \ Or you can use dynamic placeholders. The done file will always be written\ \ in the same folder as the original file. Consumer: If provided Camel will\ \ only consume files if a done file exists. This option configures what file\ \ name to use. Either you can specify a fixed name. Or you can use dynamic placeholders.The\ \ done file is always expected in the same folder as the original file. Only\ \ $file.name and $file.name.noext is supported as dynamic placeholders.\n| fileName\ \ | common | | String | Use Expression such as File Language to dynamically\ \ set the filename. For consumers it's used as a filename filter. For producers\ \ it's used to evaluate the filename to write. If an expression is set it take\ \ precedence over the CamelFileName header. (Note: The header itself can also\ \ be an Expression). The expression options support both String and Expression\ \ types. If the expression is a String type it is always evaluated using the\ \ File Language. If the expression is an Expression type the specified Expression\ \ type is used - this allows you for instance to use OGNL expressions. For the\ \ consumer you can use it to filter filenames so you can for instance consume\ \ today's file using the File Language syntax: mydata-$date:now:yyyyMMdd.txt.\ \ The producers support the CamelOverruleFileName header which takes precedence\ \ over any existing CamelFileName header; the CamelOverruleFileName is a header\ \ that is used only once and makes it easier as this avoids to temporary store\ \ CamelFileName and have to restore it afterwards.\n| bridgeErrorHandler | consumer\ \ | false | boolean | Allows for bridging the consumer to the Camel routing\ \ Error Handler which mean any exceptions occurred while the consumer is trying\ \ to pickup incoming messages or the likes will now be processed as a message\ \ and handled by the routing Error Handler. By default the consumer will use\ \ the org.apache.camel.spi.ExceptionHandler to deal with exceptions that will\ \ be logged at WARN/ERROR level and ignored.\n| delete | consumer | false |\ \ boolean | If true the file will be deleted after it is processed successfully.\n\ | moveFailed | consumer | | String | Sets the move failure expression based\ \ on Simple language. For example to move files into a .error subdirectory use:\ \ .error. Note: When moving the files to the fail location Camel will handle\ \ the error and will not pick up the file again.\n| noop | consumer | false\ \ | boolean | If true the file is not moved or deleted in any way. This option\ \ is good for readonly data or for ETL type requirements. If noop=true Camel\ \ will set idempotent=true as well to avoid consuming the same files over and\ \ over again.\n| preMove | consumer | | String | Expression (such as File Language)\ \ used to dynamically set the filename when moving it before processing. For\ \ example to move in-progress files into the order directory set this value\ \ to order.\n| recursive | consumer | false | boolean | If a directory will\ \ look for files in all the sub-directories as well.\n| sendEmptyMessageWhenIdle\ \ | consumer | false | boolean | If the polling consumer did not poll any files\ \ you can enable this option to send an empty message (no body) instead.\n|\ \ directoryMustExist | consumer (advanced) | false | boolean | Similar to startingDirectoryMustExist\ \ but this applies during polling recursive sub directories.\n| exceptionHandler\ \ | consumer (advanced) | | ExceptionHandler | To let the consumer use a custom\ \ ExceptionHandler. Notice if the option bridgeErrorHandler is enabled then\ \ this options is not in use. By default the consumer will deal with exceptions\ \ that will be logged at WARN/ERROR level and ignored.\n| exchangePattern |\ \ consumer (advanced) | | ExchangePattern | Sets the default exchange pattern\ \ when creating an exchange.\n| extendedAttributes | consumer (advanced) | \ \ | String | To define which file attributes of interest. Like posix:permissionsposix:ownerbasic:lastAccessTime\ \ it supports basic wildcard like posix: basic:lastAccessTime\n| inProgressRepository\ \ | consumer (advanced) | | String> | A pluggable in-progress repository org.apache.camel.spi.IdempotentRepository.\ \ The in-progress repository is used to account the current in progress files\ \ being consumed. By default a memory based repository is used.\n| localWorkDirectory\ \ | consumer (advanced) | | String | When consuming a local work directory\ \ can be used to store the remote file content directly in local files to avoid\ \ loading the content into memory. This is beneficial if you consume a very\ \ big remote file and thus can conserve memory.\n| onCompletionExceptionHandler\ \ | consumer (advanced) | | ExceptionHandler | To use a custom org.apache.camel.spi.ExceptionHandler\ \ to handle any thrown exceptions that happens during the file on completion\ \ process where the consumer does either a commit or rollback. The default implementation\ \ will log any exception at WARN level and ignore.\n| pollStrategy | consumer\ \ (advanced) | | PollingConsumerPollStrategy | A pluggable org.apache.camel.PollingConsumerPollingStrategy\ \ allowing you to provide your custom implementation to control error handling\ \ usually occurred during the poll operation before an Exchange have been created\ \ and being routed in Camel. In other words the error occurred while the polling\ \ was gathering information for instance access to a file network failed so\ \ Camel cannot access it to scan for files. The default implementation will\ \ log the caused exception at WARN level and ignore it.\n| probeContentType\ \ | consumer (advanced) | false | boolean | Whether to enable probing of the\ \ content type. If enable then the consumer uses link FilesprobeContentType(java.nio.file.Path)\ \ to determine the content-type of the file and store that as a header with\ \ key link ExchangeFILE_CONTENT_TYPE on the Message.\n| processStrategy | consumer\ \ (advanced) | | GenericFileProcessStrategy | A pluggable org.apache.camel.component.file.GenericFileProcessStrategy\ \ allowing you to implement your own readLock option or similar. Can also be\ \ used when special conditions must be met before a file can be consumed such\ \ as a special ready file exists. If this option is set then the readLock option\ \ does not apply.\n| startingDirectoryMustExist | consumer (advanced) | false\ \ | boolean | Whether the starting directory must exist. Mind that the autoCreate\ \ option is default enabled which means the starting directory is normally auto\ \ created if it doesn't exist. You can disable autoCreate and enable this to\ \ ensure the starting directory must exist. Will thrown an exception if the\ \ directory doesn't exist.\n| fileExist | producer | Override | GenericFileExist\ \ | What to do if a file already exists with the same name. Override which is\ \ the default replaces the existing file. Append - adds content to the existing\ \ file. Fail - throws a GenericFileOperationException indicating that there\ \ is already an existing file. Ignore - silently ignores the problem and does\ \ not override the existing file but assumes everything is okay. Move - option\ \ requires to use the moveExisting option to be configured as well. The option\ \ eagerDeleteTargetFile can be used to control what to do if an moving the file\ \ and there exists already an existing file otherwise causing the move operation\ \ to fail. The Move option will move any existing files before writing the target\ \ file. TryRename Camel is only applicable if tempFileName option is in use.\ \ This allows to try renaming the file from the temporary name to the actual\ \ name without doing any exists check.This check may be faster on some file\ \ systems and especially FTP servers.\n| flatten | producer | false | boolean\ \ | Flatten is used to flatten the file name path to strip any leading paths\ \ so it's just the file name. This allows you to consume recursively into sub-directories\ \ but when you eg write the files to another directory they will be written\ \ in a single directory. Setting this to true on the producer enforces that\ \ any file name in CamelFileName header will be stripped for any leading paths.\n\ | moveExisting | producer | | String | Expression (such as File Language) used\ \ to compute file name to use when fileExist=Move is configured. To move files\ \ into a backup subdirectory just enter backup. This option only supports the\ \ following File Language tokens: file:name file:name.ext file:name.noext file:onlyname\ \ file:onlyname.noext file:ext and file:parent. Notice the file:parent is not\ \ supported by the FTP component as the FTP component can only move any existing\ \ files to a relative directory based on current dir as base.\n| tempFileName\ \ | producer | | String | The same as tempPrefix option but offering a more\ \ fine grained control on the naming of the temporary filename as it uses the\ \ File Language.\n| tempPrefix | producer | | String | This option is used\ \ to write the file using a temporary name and then after the write is complete\ \ rename it to the real name. Can be used to identify files being written and\ \ also avoid consumers (not using exclusive read locks) reading in progress\ \ files. Is often used by FTP when uploading big files.\n| allowNullBody | producer\ \ (advanced) | false | boolean | Used to specify if a null body is allowed during\ \ file writing. If set to true then an empty file will be created when set to\ \ false and attempting to send a null body to the file component a GenericFileWriteException\ \ of 'Cannot write null body to file.' will be thrown. If the fileExist option\ \ is set to 'Override' then the file will be truncated and if set to append\ \ the file will remain unchanged.\n| chmod | producer (advanced) | | String\ \ | Specify the file permissions which is sent by the producer the chmod value\ \ must be between 000 and 777; If there is a leading digit like in 0755 we will\ \ ignore it.\n| chmodDirectory | producer (advanced) | | String | Specify the\ \ directory permissions used when the producer creates missing directories the\ \ chmod value must be between 000 and 777; If there is a leading digit like\ \ in 0755 we will ignore it.\n| eagerDeleteTargetFile | producer (advanced)\ \ | true | boolean | Whether or not to eagerly delete any existing target file.\ \ This option only applies when you use fileExists=Override and the tempFileName\ \ option as well. You can use this to disable (set it to false) deleting the\ \ target file before the temp file is written. For example you may write big\ \ files and want the target file to exists during the temp file is being written.\ \ This ensure the target file is only deleted until the very last moment just\ \ before the temp file is being renamed to the target filename. This option\ \ is also used to control whether to delete any existing files when fileExist=Move\ \ is enabled and an existing file exists. If this option copyAndDeleteOnRenameFails\ \ false then an exception will be thrown if an existing file existed if its\ \ true then the existing file is deleted before the move operation.\n| forceWrites\ \ | producer (advanced) | true | boolean | Whether to force syncing writes to\ \ the file system. You can turn this off if you do not want this level of guarantee\ \ for example if writing to logs / audit logs etc; this would yield better performance.\n\ | keepLastModified | producer (advanced) | false | boolean | Will keep the last\ \ modified timestamp from the source file (if any). Will use the Exchange.FILE_LAST_MODIFIED\ \ header to located the timestamp. This header can contain either a java.util.Date\ \ or long with the timestamp. If the timestamp exists and the option is enabled\ \ it will set this timestamp on the written file. Note: This option only applies\ \ to the file producer. You cannot use this option with any of the ftp producers.\n\ | autoCreate | advanced | true | boolean | Automatically create missing directories\ \ in the file's pathname. For the file consumer that means creating the starting\ \ directory. For the file producer it means the directory the files should be\ \ written to.\n| bufferSize | advanced | 131072 | int | Write buffer sized in\ \ bytes.\n| copyAndDeleteOnRenameFail | advanced | true | boolean | Whether\ \ to fallback and do a copy and delete file in case the file could not be renamed\ \ directly. This option is not available for the FTP component.\n| renameUsingCopy\ \ | advanced | false | boolean | Perform rename operations using a copy and\ \ delete strategy. This is primarily used in environments where the regular\ \ rename operation is unreliable (e.g. across different file systems or networks).\ \ This option takes precedence over the copyAndDeleteOnRenameFail parameter\ \ that will automatically fall back to the copy and delete strategy but only\ \ after additional delays.\n| synchronous | advanced | false | boolean | Sets\ \ whether synchronous processing should be strictly used or Camel is allowed\ \ to use asynchronous processing (if supported).\n| antExclude | filter | |\ \ String | Ant style filter exclusion. If both antInclude and antExclude are\ \ used antExclude takes precedence over antInclude. Multiple exclusions may\ \ be specified in comma-delimited format.\n| antFilterCaseSensitive | filter\ \ | true | boolean | Sets case sensitive flag on ant fiter\n| antInclude | filter\ \ | | String | Ant style filter inclusion. Multiple inclusions may be specified\ \ in comma-delimited format.\n| eagerMaxMessagesPerPoll | filter | true | boolean\ \ | Allows for controlling whether the limit from maxMessagesPerPoll is eager\ \ or not. If eager then the limit is during the scanning of files. Where as\ \ false would scan all files and then perform sorting. Setting this option to\ \ false allows for sorting all files first and then limit the poll. Mind that\ \ this requires a higher memory usage as all file details are in memory to perform\ \ the sorting.\n| exclude | filter | | String | Is used to exclude files if\ \ filename matches the regex pattern (matching is case in-senstive). Notice\ \ if you use symbols such as plus sign and others you would need to configure\ \ this using the RAW() syntax if configuring this as an endpoint uri. See more\ \ details at configuring endpoint uris\n| filter | filter | | GenericFileFilter\ \ | Pluggable filter as a org.apache.camel.component.file.GenericFileFilter\ \ class. Will skip files if filter returns false in its accept() method.\n|\ \ filterDirectory | filter | | String | Filters the directory based on Simple\ \ language. For example to filter on current date you can use a simple date\ \ pattern such as $date:now:yyyMMdd\n| filterFile | filter | | String | Filters\ \ the file based on Simple language. For example to filter on file size you\ \ can use $file:size 5000\n| idempotent | filter | false | Boolean | Option\ \ to use the Idempotent Consumer EIP pattern to let Camel skip already processed\ \ files. Will by default use a memory based LRUCache that holds 1000 entries.\ \ If noop=true then idempotent will be enabled as well to avoid consuming the\ \ same files over and over again.\n| idempotentKey | filter | | String | To\ \ use a custom idempotent key. By default the absolute path of the file is used.\ \ You can use the File Language for example to use the file name and file size\ \ you can do: idempotentKey=$file:name-$file:size\n| idempotentRepository |\ \ filter | | String> | A pluggable repository org.apache.camel.spi.IdempotentRepository\ \ which by default use MemoryMessageIdRepository if none is specified and idempotent\ \ is true.\n| include | filter | | String | Is used to include files if filename\ \ matches the regex pattern (matching is case in-senstive). Notice if you use\ \ symbols such as plus sign and others you would need to configure this using\ \ the RAW() syntax if configuring this as an endpoint uri. See more details\ \ at configuring endpoint uris\n| maxDepth | filter | 2147483647 | int | The\ \ maximum depth to traverse when recursively processing a directory.\n| maxMessagesPerPoll\ \ | filter | | int | To define a maximum messages to gather per poll. By default\ \ no maximum is set. Can be used to set a limit of e.g. 1000 to avoid when starting\ \ up the server that there are thousands of files. Set a value of 0 or negative\ \ to disabled it. Notice: If this option is in use then the File and FTP components\ \ will limit before any sorting. For example if you have 100000 files and use\ \ maxMessagesPerPoll=500 then only the first 500 files will be picked up and\ \ then sorted. You can use the eagerMaxMessagesPerPoll option and set this to\ \ false to allow to scan all files first and then sort afterwards.\n| minDepth\ \ | filter | | int | The minimum depth to start processing when recursively\ \ processing a directory. Using minDepth=1 means the base directory. Using minDepth=2\ \ means the first sub directory.\n| move | filter | | String | Expression (such\ \ as Simple Language) used to dynamically set the filename when moving it after\ \ processing. To move files into a .done subdirectory just enter .done.\n| exclusiveReadLockStrategy\ \ | lock | | GenericFileExclusiveReadLockStrategy | Pluggable read-lock\ \ as a org.apache.camel.component.file.GenericFileExclusiveReadLockStrategy\ \ implementation.\n| readLock | lock | | String | Used by consumer to only\ \ poll the files if it has exclusive read-lock on the file (i.e. the file is\ \ not in-progress or being written). Camel will wait until the file lock is\ \ granted. This option provides the build in strategies: none - No read lock\ \ is in use markerFile - Camel creates a marker file (fileName.camelLock) and\ \ then holds a lock on it. This option is not available for the FTP component\ \ changed - Changed is using file length/modification timestamp to detect whether\ \ the file is currently being copied or not. Will at least use 1 sec to determine\ \ this so this option cannot consume files as fast as the others but can be\ \ more reliable as the JDK IO API cannot always determine whether a file is\ \ currently being used by another process. The option readLockCheckInterval\ \ can be used to set the check frequency. fileLock - is for using java.nio.channels.FileLock.\ \ This option is not avail for the FTP component. This approach should be avoided\ \ when accessing a remote file system via a mount/share unless that file system\ \ supports distributed file locks. rename - rename is for using a try to rename\ \ the file as a test if we can get exclusive read-lock. idempotent - (only for\ \ file component) idempotent is for using a idempotentRepository as the read-lock.\ \ This allows to use read locks that supports clustering if the idempotent repository\ \ implementation supports that. Notice: The various read locks is not all suited\ \ to work in clustered mode where concurrent consumers on different nodes is\ \ competing for the same files on a shared file system. The markerFile using\ \ a close to atomic operation to create the empty marker file but its not guaranteed\ \ to work in a cluster. The fileLock may work better but then the file system\ \ need to support distributed file locks and so on. Using the idempotent read\ \ lock can support clustering if the idempotent repository supports clustering\ \ such as Hazelcast Component or Infinispan.\n| readLockCheckInterval | lock\ \ | 1000 | long | Interval in millis for the read-lock if supported by the read\ \ lock. This interval is used for sleeping between attempts to acquire the read\ \ lock. For example when using the changed read lock you can set a higher interval\ \ period to cater for slow writes. The default of 1 sec. may be too fast if\ \ the producer is very slow writing the file. Notice: For FTP the default readLockCheckInterval\ \ is 5000. The readLockTimeout value must be higher than readLockCheckInterval\ \ but a rule of thumb is to have a timeout that is at least 2 or more times\ \ higher than the readLockCheckInterval. This is needed to ensure that amble\ \ time is allowed for the read lock process to try to grab the lock before the\ \ timeout was hit.\n| readLockDeleteOrphanLockFiles | lock | true | boolean\ \ | Whether or not read lock with marker files should upon startup delete any\ \ orphan read lock files which may have been left on the file system if Camel\ \ was not properly shutdown (such as a JVM crash). If turning this option to\ \ false then any orphaned lock file will cause Camel to not attempt to pickup\ \ that file this could also be due another node is concurrently reading files\ \ from the same shared directory.\n| readLockLoggingLevel | lock | WARN | LoggingLevel\ \ | Logging level used when a read lock could not be acquired. By default a\ \ WARN is logged. You can change this level for example to OFF to not have any\ \ logging. This option is only applicable for readLock of types: changed fileLock\ \ rename.\n| readLockMarkerFile | lock | true | boolean | Whether to use marker\ \ file with the changed rename or exclusive read lock types. By default a marker\ \ file is used as well to guard against other processes picking up the same\ \ files. This behavior can be turned off by setting this option to false. For\ \ example if you do not want to write marker files to the file systems by the\ \ Camel application.\n| readLockMinAge | lock | 0 | long | This option applied\ \ only for readLock=change. This option allows to specify a minimum age the\ \ file must be before attempting to acquire the read lock. For example use readLockMinAge=300s\ \ to require the file is at last 5 minutes old. This can speedup the changed\ \ read lock as it will only attempt to acquire files which are at least that\ \ given age.\n| readLockMinLength | lock | 1 | long | This option applied only\ \ for readLock=changed. This option allows you to configure a minimum file length.\ \ By default Camel expects the file to contain data and thus the default value\ \ is 1. You can set this option to zero to allow consuming zero-length files.\n\ | readLockRemoveOnCommit | lock | false | boolean | This option applied only\ \ for readLock=idempotent. This option allows to specify whether to remove the\ \ file name entry from the idempotent repository when processing the file is\ \ succeeded and a commit happens. By default the file is not removed which ensures\ \ that any race-condition do not occur so another active node may attempt to\ \ grab the file. Instead the idempotent repository may support eviction strategies\ \ that you can configure to evict the file name entry after X minutes - this\ \ ensures no problems with race conditions.\n| readLockRemoveOnRollback | lock\ \ | true | boolean | This option applied only for readLock=idempotent. This\ \ option allows to specify whether to remove the file name entry from the idempotent\ \ repository when processing the file failed and a rollback happens. If this\ \ option is false then the file name entry is confirmed (as if the file did\ \ a commit).\n| readLockTimeout | lock | 10000 | long | Optional timeout in\ \ millis for the read-lock if supported by the read-lock. If the read-lock could\ \ not be granted and the timeout triggered then Camel will skip the file. At\ \ next poll Camel will try the file again and this time maybe the read-lock\ \ could be granted. Use a value of 0 or lower to indicate forever. Currently\ \ fileLock changed and rename support the timeout. Notice: For FTP the default\ \ readLockTimeout value is 20000 instead of 10000. The readLockTimeout value\ \ must be higher than readLockCheckInterval but a rule of thumb is to have a\ \ timeout that is at least 2 or more times higher than the readLockCheckInterval.\ \ This is needed to ensure that amble time is allowed for the read lock process\ \ to try to grab the lock before the timeout was hit.\n| backoffErrorThreshold\ \ | scheduler | | int | The number of subsequent error polls (failed due some\ \ error) that should happen before the backoffMultipler should kick-in.\n| backoffIdleThreshold\ \ | scheduler | | int | The number of subsequent idle polls that should happen\ \ before the backoffMultipler should kick-in.\n| backoffMultiplier | scheduler\ \ | | int | To let the scheduled polling consumer backoff if there has been\ \ a number of subsequent idles/errors in a row. The multiplier is then the number\ \ of polls that will be skipped before the next actual attempt is happening\ \ again. When this option is in use then backoffIdleThreshold and/or backoffErrorThreshold\ \ must also be configured.\n| delay | scheduler | 500 | long | Milliseconds\ \ before the next poll. The default value is 500. You can also specify time\ \ values using units such as 60s (60 seconds) 5m30s (5 minutes and 30 seconds)\ \ and 1h (1 hour).\n| greedy | scheduler | false | boolean | If greedy is enabled\ \ then the ScheduledPollConsumer will run immediately again if the previous\ \ run polled 1 or more messages.\n| initialDelay | scheduler | 1000 | long |\ \ Milliseconds before the first poll starts. The default value is 1000. You\ \ can also specify time values using units such as 60s (60 seconds) 5m30s (5\ \ minutes and 30 seconds) and 1h (1 hour).\n| runLoggingLevel | scheduler |\ \ TRACE | LoggingLevel | The consumer logs a start/complete log line when it\ \ polls. This option allows you to configure the logging level for that.\n|\ \ scheduledExecutorService | scheduler | | ScheduledExecutorService | Allows\ \ for configuring a custom/shared thread pool to use for the consumer. By default\ \ each consumer has its own single threaded thread pool. This option allows\ \ you to share a thread pool among multiple consumers.\n| scheduler | scheduler\ \ | none | ScheduledPollConsumerScheduler | Allow to plugin a custom org.apache.camel.spi.ScheduledPollConsumerScheduler\ \ to use as the scheduler for firing when the polling consumer runs. The default\ \ implementation uses the ScheduledExecutorService and there is a Quartz2 and\ \ Spring based which supports CRON expressions. Notice: If using a custom scheduler\ \ then the options for initialDelay useFixedDelay timeUnit and scheduledExecutorService\ \ may not be in use. Use the text quartz2 to refer to use the Quartz2 scheduler;\ \ and use the text spring to use the Spring based; and use the text myScheduler\ \ to refer to a custom scheduler by its id in the Registry. See Quartz2 page\ \ for an example.\n| schedulerProperties | scheduler | | Map | To configure\ \ additional properties when using a custom scheduler or any of the Quartz2\ \ Spring based scheduler.\n| startScheduler | scheduler | true | boolean | Whether\ \ the scheduler should be auto started.\n| timeUnit | scheduler | MILLISECONDS\ \ | TimeUnit | Time unit for initialDelay and delay options.\n| useFixedDelay\ \ | scheduler | true | boolean | Controls if fixed delay or fixed rate is used.\ \ See ScheduledExecutorService in JDK for details.\n| shuffle | sort | false\ \ | boolean | To shuffle the list of files (sort in random order)\n| sortBy\ \ | sort | | String | Built-in sort by using the File Language. Supports nested\ \ sorts so you can have a sort by file name and as a 2nd group sort by modified\ \ date.\n| sorter | sort | | GenericFile> | Pluggable sorter as a java.util.Comparator\ \ class.\n|=======================================================================\n\ {% endraw %}\n// endpoint options: END\n\n\n\n[[File2-Defaultbehaviorforfileproducer]]\n\ Default behavior for file producer\n++++++++++++++++++++++++++++++++++\n\n*\ \ By default it will override any existing file, if one exist with the\nsame\ \ name.\n\n[[File2-MoveandDeleteoperations]]\nMove and Delete operations\n^^^^^^^^^^^^^^^^^^^^^^^^^^\n\ \nAny move or delete operations is executed after (post command) the\nrouting\ \ has completed; so during processing of the `Exchange` the file\nis still located\ \ in the inbox folder.\n\nLets illustrate this with an example:\n\n[source,java]\n\ -----------------------------------------------------------\n from(\"file://inbox?move=.done\"\ ).to(\"bean:handleOrder\");\n-----------------------------------------------------------\n\ \nWhen a file is dropped in the `inbox` folder, the file consumer notices\n\ this and creates a new `FileExchange` that is routed to the\n`handleOrder` bean.\ \ The bean then processes the `File` object. At this\npoint in time the file\ \ is still located in the `inbox` folder. After the\nbean completes, and thus\ \ the route is completed, the file consumer will\nperform the move operation\ \ and move the file to the `.done` sub-folder.\n\nThe *move* and the??*preMove*\ \ options are considered as a directory name\n(though if you use an expression\ \ such as link:file-language.html[File\nLanguage], or link:simple.html[Simple]\ \ then the result of the expression\nevaluation is the file name to be used\ \ - eg if you set\n\n[source,java]\n-----------------------------------\nmove=../backup/copy-of-${file:name}\n\ -----------------------------------\n\nthen that's using the link:file-language.html[File\ \ Language] which we\nuse return the file name to be used), which can be either\ \ relative or\nabsolute. If relative, the directory is created as a sub-folder\ \ from\nwithin the folder where the file was consumed.\n\nBy default, Camel\ \ will move consumed files to the `.camel` sub-folder\nrelative to the directory\ \ where the file was consumed.\n\nIf you want to delete the file after processing,\ \ the route should be:\n\n[source,java]\n-------------------------------------------------------------\n\ \ from(\"file://inobox?delete=true\").to(\"bean:handleOrder\");\n-------------------------------------------------------------\n\ \nWe have introduced a *pre* move operation to move files *before* they\nare\ \ processed. This allows you to mark which files have been scanned as\nthey\ \ are moved to this sub folder before being processed.\n\n[source,java]\n-------------------------------------------------------------------\n\ \ from(\"file://inbox?preMove=inprogress\").to(\"bean:handleOrder\");\n-------------------------------------------------------------------\n\ \nYou can combine the *pre* move and the regular move:\n\n[source,java]\n------------------------------------------------------------------------------\n\ \ from(\"file://inbox?preMove=inprogress&move=.done\").to(\"bean:handleOrder\"\ );\n------------------------------------------------------------------------------\n\ \nSo in this situation, the file is in the `inprogress` folder when being\n\ processed and after it's processed, it's moved to the `.done` folder.\n\n[[File2-FinegrainedcontroloverMoveandPreMoveoption]]\n\ Fine grained control over Move and PreMove option\n+++++++++++++++++++++++++++++++++++++++++++++++++\n\ \nThe *move* and *preMove* options\nare??link:expression.html[Expression]-based,\ \ so we have the full power of\nthe link:file-language.html[File Language] to\ \ do advanced configuration\nof the directory and name pattern. +\n Camel will,\ \ in fact, internally convert the directory name you enter\ninto a link:file-language.html[File\ \ Language] expression. So when we\nenter `move=.done` Camel will convert this\ \ into:\n`${``file:parent``}/.done/${``file:onlyname`}. This is only done if\n\ Camel detects that you have not provided a $\\{ } in the option value\nyourself.\ \ So when you enter a $\\{ } Camel will *not* convert it and thus\nyou have\ \ the full power.\n\nSo if we want to move the file into a backup folder with\ \ today's date as\nthe pattern, we can do:\n\n[source,java]\n---------------------------------------------\n\ move=backup/${date:now:yyyyMMdd}/${file:name}\n---------------------------------------------\n\ \n[[File2-AboutmoveFailed]]\nAbout moveFailed\n++++++++++++++++\n\nThe `moveFailed`\ \ option allows you to move files that *could not* be\nprocessed succesfully\ \ to another location such as a error folder of your\nchoice. For example to\ \ move the files in an error folder with a\ntimestamp you can use\n`moveFailed=/error/${``file:name.noext``}-${date:now:yyyyMMddHHmmssSSS}.${``file:ext`}.\n\ \nSee more examples at link:file-language.html[File Language]\n\n[[File2-MessageHeaders]]\n\ Message Headers\n^^^^^^^^^^^^^^^\n\nThe following headers are supported by this\ \ component:\n\n[[File2-Fileproduceronly]]\nFile producer only\n++++++++++++++++++\n\ \n[width=\"100%\",cols=\"10%,90%\",options=\"header\",]\n|=======================================================================\n\ |Header |Description\n\n|`CamelFileName` |Specifies the name of the file to\ \ write (relative to the endpoint\ndirectory). This name can be a `String`;\ \ a `String` with a\nlink:file-language.html[File Language] or link:simple.html[Simple]\n\ expression; or an link:expression.html[Expression] object. If it's\n`null` then\ \ Camel will auto-generate a filename based on the message\nunique ID.\n\n|`CamelFileNameProduced`\ \ |The actual absolute filepath (path + name) for the output file that was\n\ written. This header is set by Camel and its purpose is providing\nend-users\ \ with the name of the file that was written.\n\n|`CamelOverruleFileName` |*Camel\ \ 2.11:* Is used for overruling `CamelFileName` header and use the\nvalue instead\ \ (but only once, as the producer will remove this header\nafter writing the\ \ file). The value can be only be a String. Notice that\nif the option `fileName`\ \ has been configured, then this is still being\nevaluated.\n|=======================================================================\n\ \n[[File2-Fileconsumeronly]]\nFile consumer only\n++++++++++++++++++\n\n[width=\"\ 100%\",cols=\"10%,90%\",options=\"header\",]\n|=======================================================================\n\ |Header |Description\n\n|`CamelFileName` |Name of the consumed file as a relative\ \ file path with offset from the\nstarting directory configured on the endpoint.\n\ \n|`CamelFileNameOnly` |Only the file name (the name with no leading paths).\n\ \n|`CamelFileAbsolute` |A `boolean` option specifying whether the consumed file\ \ denotes an\nabsolute path or not. Should normally be `false` for relative\ \ paths.\nAbsolute paths should normally not be used but we added to the move\n\ option to allow moving files to absolute paths. But can be used\nelsewhere as\ \ well.\n\n|`CamelFileAbsolutePath` |The absolute path to the file. For relative\ \ files this path holds the\nrelative path instead.\n\n|`CamelFilePath` |The\ \ file path. For relative files this is the starting directory + the\nrelative\ \ filename. For absolute files this is the absolute path.\n\n|`CamelFileRelativePath`\ \ |The relative path.\n\n|`CamelFileParent` |The parent path.\n\n|`CamelFileLength`\ \ |A `long` value containing the file size.\n\n|`CamelFileLastModified` |A `Long`\ \ value containing the last modified timestamp of the file. In\n*Camel 2.10.3\ \ and older* the type is `Date`.\n|=======================================================================\n\ \n[[File2-BatchConsumer]]\nBatch Consumer\n^^^^^^^^^^^^^^\n\nThis component\ \ implements the link:batch-consumer.html[Batch Consumer].\n\n[[File2-ExchangeProperties,fileconsumeronly]]\n\ Exchange Properties, file consumer only\n+++++++++++++++++++++++++++++++++++++++\n\ \nAs the file consumer implements the??`BatchConsumer` it supports batching\n\ the files it polls. By batching we mean that Camel will add the\nfollowing additional\ \ properties to the link:exchange.html[Exchange], so\nyou know the number of\ \ files polled, the current index, and whether the\nbatch is already completed.\n\ \n[width=\"100%\",cols=\"10%,90%\",options=\"header\",]\n|=======================================================================\n\ |Property |Description\n\n|`CamelBatchSize` |The total number of files that\ \ was polled in this batch.\n\n|`CamelBatchIndex` |The current index of the\ \ batch. Starts from 0.\n\n|`CamelBatchComplete` |A `boolean` value indicating\ \ the last link:exchange.html[Exchange] in\nthe batch. Is only `true` for the\ \ last entry.\n|=======================================================================\n\ \nThis allows you for instance to know how many files exist in this batch\n\ and for instance let the link:aggregator2.html[Aggregator2] aggregate\nthis\ \ number of files.\n\n[[File2-Usingcharset]]\nUsing charset\n^^^^^^^^^^^^^\n\ \n*Available as of Camel 2.9.3* +\n The charset option allows for configuring\ \ an encoding of the files on\nboth the consumer and producer endpoints. For\ \ example if you read utf-8\nfiles, and want to convert the files to iso-8859-1,\ \ you can do:\n\n[source,java]\n---------------------------------------\nfrom(\"\ file:inbox?charset=utf-8\")\n .to(\"file:outbox?charset=iso-8859-1\")\n---------------------------------------\n\ \nYou can also use the `convertBodyTo` in the route. In the example below\n\ we have still input files in utf-8 format, but we want to convert the\nfile\ \ content to a byte array in iso-8859-1 format. And then let a bean\nprocess\ \ the data. Before writing the content to the outbox folder using\nthe current\ \ charset.\n\n[source,java]\n--------------------------------------------\n\ from(\"file:inbox?charset=utf-8\")\n .convertBodyTo(byte[].class, \"iso-8859-1\"\ )\n .to(\"bean:myBean\")\n .to(\"file:outbox\");\n--------------------------------------------\n\ \nIf you omit the charset on the consumer endpoint, then Camel does not\nknow\ \ the charset of the file, and would by default use \"UTF-8\". However\nyou\ \ can configure a JVM system property to override and use a different\ndefault\ \ encoding with the key `org.apache.camel.default.charset`.\n\nIn the example\ \ below this could be a problem if the files is not in\nUTF-8 encoding, which\ \ would be the default encoding for read the\nfiles. +\n In this example when\ \ writing the files, the content has already been\nconverted to a byte array,\ \ and thus would write the content directly as\nis (without any further encodings).\n\ \n[source,java]\n--------------------------------------------\nfrom(\"file:inbox\"\ )\n .convertBodyTo(byte[].class, \"iso-8859-1\")\n .to(\"bean:myBean\")\n\ \ .to(\"file:outbox\");\n--------------------------------------------\n\nYou\ \ can also override and control the encoding dynamic when writing\nfiles, by\ \ setting a property on the exchange with the key\n`Exchange.CHARSET_NAME`.\ \ For example in the route below we set the\nproperty with a value from a message\ \ header.\n\n[source,java]\n------------------------------------------------------------------\n\ from(\"file:inbox\")\n .convertBodyTo(byte[].class, \"iso-8859-1\")\n .to(\"\ bean:myBean\")\n .setProperty(Exchange.CHARSET_NAME, header(\"someCharsetHeader\"\ ))\n .to(\"file:outbox\");\n------------------------------------------------------------------\n\ \nWe suggest to keep things simpler, so if you pickup files with the same\n\ encoding, and want to write the files in a specific encoding, then favor\nto\ \ use the `charset` option on the endpoints.\n\nNotice that if you have explicit\ \ configured a `charset` option on the\nendpoint, then that configuration is\ \ used, regardless of the\n`Exchange.CHARSET_NAME` property.\n\nIf you have\ \ some issues then you can enable DEBUG logging on\n`org.apache.camel.component.file`,\ \ and Camel logs when it reads/write a\nfile using a specific charset. +\n For\ \ example the route below will log the following:\n\n[source,java]\n---------------------------------------\n\ from(\"file:inbox?charset=utf-8\")\n .to(\"file:outbox?charset=iso-8859-1\"\ )\n---------------------------------------\n\nAnd the logs:\n\n[source,java]\n\ ----------------------------------------------------------------------------------------------------------------------------------------------\n\ DEBUG GenericFileConverter - Read file /Users/davsclaus/workspace/camel/camel-core/target/charset/input/input.txt\ \ with charset utf-8\nDEBUG FileOperations - Using Reader to\ \ write file: target/charset/output.txt with charset: iso-8859-1\n----------------------------------------------------------------------------------------------------------------------------------------------\n\ \n[[File2-Commongotchaswithfolderandfilenames]]\nCommon gotchas with folder\ \ and filenames\n^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n\nWhen Camel is producing\ \ files (writing files) there are a few gotchas\naffecting how to set a filename\ \ of your choice. By default, Camel will\nuse the message ID as the filename,\ \ and since the message ID is normally\na unique generated ID, you will end\ \ up with filenames such as:\n`ID-MACHINENAME-2443-1211718892437-1-0`. If such\ \ a filename is not\ndesired, then you must provide a filename in the `CamelFileName`\ \ message\nheader. The constant, `Exchange.FILE_NAME`, can also be used.\n\n\ The sample code below produces files using the message ID as the\nfilename:\n\ \n[source,java]\n------------------------------------------------\nfrom(\"direct:report\"\ ).to(\"file:target/reports\");\n------------------------------------------------\n\ \nTo use `report.txt` as the filename you have to do:\n\n[source,java]\n-------------------------------------------------------------------------------------------------------\n\ from(\"direct:report\").setHeader(Exchange.FILE_NAME, constant(\"report.txt\"\ )).to( \"file:target/reports\");\n-------------------------------------------------------------------------------------------------------\n\ \n... the same as above, but with `CamelFileName`:\n\n[source,java]\n----------------------------------------------------------------------------------------------------\n\ from(\"direct:report\").setHeader(\"CamelFileName\", constant(\"report.txt\"\ )).to( \"file:target/reports\");\n----------------------------------------------------------------------------------------------------\n\ \nAnd a syntax where we set the filename on the endpoint with the\n*fileName*\ \ URI option.\n\n[source,java]\n---------------------------------------------------------------------\n\ from(\"direct:report\").to(\"file:target/reports/?fileName=report.txt\");\n\ ---------------------------------------------------------------------\n\n[[File2-FilenameExpression]]\n\ Filename Expression\n^^^^^^^^^^^^^^^^^^^\n\nFilename can be set either using\ \ the *expression* option or as a\nstring-based link:file-language.html[File\ \ Language] expression in the\n`CamelFileName` header. See the link:file-language.html[File\ \ Language]\nfor syntax and samples.\n\n[[File2-Consumingfilesfromfolderswhereothersdropfilesdirectly]]\n\ Consuming files from folders where others drop files directly\n^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n\ \nBeware if you consume files from a folder where other applications write\n\ files to directly. Take a look at the different readLock options to see\nwhat\ \ suits your use cases. The best approach is however to write to\nanother folder\ \ and after the write move the file in the drop folder.\nHowever if you write\ \ files directly to the drop folder then the option\nchanged could better detect\ \ whether a file is currently being\nwritten/copied as it uses a file changed\ \ algorithm to see whether the\nfile size / modification changes over a period\ \ of time. The other\nreadLock options rely on Java File API that sadly is not\ \ always very\ngood at detecting this. You may also want to look at the doneFileName\n\ option, which uses a marker file (done file) to signal when a file is\ndone\ \ and ready to be consumed.\n\n[[File2-Usingdonefiles]]\nUsing done files\n\ ^^^^^^^^^^^^^^^^\n\n*Available as of Camel 2.6*\n\nSee also section _writing\ \ done files_ below.\n\nIf you want only to consume files when a done file exists,\ \ then you can\nuse the `doneFileName` option on the endpoint.\n\n[source,java]\n\ -----------------------------------\nfrom(\"file:bar?doneFileName=done\");\n\ -----------------------------------\n\nWill only consume files from the bar\ \ folder, if a done _file_ exists in\nthe same directory as the target files.\ \ Camel will automatically delete\nthe _done file_ when it's done consuming\ \ the files. From Camel *2.9.3*\nonwards Camel will not automatically delete\ \ the _done file_ if\n`noop=true` is configured.\n\nHowever it is more common\ \ to have one _done file_ per target file. This\nmeans there is a 1:1 correlation.\ \ To do this you must use dynamic\nplaceholders in the `doneFileName` option.\ \ Currently Camel supports the\nfollowing two dynamic tokens: `file:name` and\ \ `file:name.noext` which\nmust be enclosed in $\\{ }. The consumer only supports\ \ the static part of\nthe _done file_ name as either prefix or suffix (not both).\n\ \n[source,java]\n------------------------------------------------\nfrom(\"file:bar?doneFileName=${file:name}.done\"\ );\n------------------------------------------------\n\nIn this example only\ \ files will be polled if there exists a done file\nwith the name _file name_.done.\ \ For example\n\n* `hello.txt` - is the file to be consumed\n* `hello.txt.done`\ \ - is the associated done file\n\nYou can also use a prefix for the done file,\ \ such as:\n\n[source,java]\n-------------------------------------------------\n\ from(\"file:bar?doneFileName=ready-${file:name}\");\n-------------------------------------------------\n\ \n* `hello.txt` - is the file to be consumed\n* `ready-hello.txt` - is the associated\ \ done file\n\n[[File2-Writingdonefiles]]\nWriting done files\n^^^^^^^^^^^^^^^^^^\n\ \n*Available as of Camel 2.6*\n\nAfter you have written a file you may want\ \ to write an additional _done_\n_file_ as a kind of marker, to indicate to\ \ others that the file is\nfinished and has been written. To do that you can\ \ use the `doneFileName`\noption on the file producer endpoint.\n\n[source,java]\n\ ----------------------------------\n.to(\"file:bar?doneFileName=done\");\n----------------------------------\n\ \nWill simply create a file named `done` in the same directory as the\ntarget\ \ file.\n\nHowever it is more common to have one done file per target file.\ \ This\nmeans there is a 1:1 correlation. To do this you must use dynamic\n\ placeholders in the `doneFileName` option. Currently Camel supports the\nfollowing\ \ two dynamic tokens: `file:name` and `file:name.noext` which\nmust be enclosed\ \ in $\\{ }.\n\n[source,java]\n-----------------------------------------------\n\ .to(\"file:bar?doneFileName=done-${file:name}\");\n-----------------------------------------------\n\ \nWill for example create a file named `done-foo.txt` if the target file\nwas\ \ `foo.txt` in the same directory as the target file.\n\n[source,java]\n-----------------------------------------------\n\ .to(\"file:bar?doneFileName=${file:name}.done\");\n-----------------------------------------------\n\ \nWill for example create a file named `foo.txt.done` if the target file\nwas\ \ `foo.txt` in the same directory as the target file.\n\n[source,java]\n-----------------------------------------------------\n\ .to(\"file:bar?doneFileName=${file:name.noext}.done\");\n-----------------------------------------------------\n\ \nWill for example create a file named `foo.done` if the target file was\n`foo.txt`\ \ in the same directory as the target file.\n\n[[File2-Samples]]\nSamples\n\ ^^^^^^^\n\n[[File2-Readfromadirectoryandwritetoanotherdirectory]]\nRead from\ \ a directory and write to another directory\n++++++++++++++++++++++++++++++++++++++++++++++++++++\n\ \n[source,java]\n-----------------------------------------------------------\n\ from(\"file://inputdir/?delete=true\").to(\"file://outputdir\")\n-----------------------------------------------------------\n\ \n[[File2-Readfromadirectoryandwritetoanotherdirectoryusingaoverruledynamicname]]\n\ Read from a directory and write to another directory using a overrule dynamic\ \ name\n++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++\n\ \n[source,java]\n---------------------------------------------------------------------------------------------\n\ from(\"file://inputdir/?delete=true\").to(\"file://outputdir?overruleFile=copy-of-${file:name}\"\ )\n---------------------------------------------------------------------------------------------\n\ \nListen on a directory and create a message for each file dropped there.\n\ Copy the contents to the `outputdir` and delete the file in the\n`inputdir`.\n\ \n[[File2-Readingrecursivelyfromadirectoryandwritingtoanother]]\nReading recursively\ \ from a directory and writing to another\n+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++\n\ \n[source,java]\n--------------------------------------------------------------------------\n\ from(\"file://inputdir/?recursive=true&delete=true\").to(\"file://outputdir\"\ )\n--------------------------------------------------------------------------\n\ \nListen on a directory and create a message for each file dropped there.\n\ Copy the contents to the `outputdir` and delete the file in the\n`inputdir`.\ \ Will scan recursively into sub-directories. Will lay out the\nfiles in the\ \ same directory structure in the `outputdir` as the\n`inputdir`, including\ \ any sub-directories.\n\n[source,java]\n--------------------\ninputdir/foo.txt\n\ inputdir/sub/bar.txt\n--------------------\n\nWill result in the following output\ \ layout:\n\n[source,java]\n---------------------\noutputdir/foo.txt\noutputdir/sub/bar.txt\n\ ---------------------\n\n[[File2-Usingflatten]]\nUsing flatten\n\nIf you want\ \ to store the files in the outputdir directory in the same\ndirectory, disregarding\ \ the source directory layout (e.g. to flatten out\nthe path), you just add\ \ the `flatten=true` option on the file producer\nside:\n\n[source,java]\n---------------------------------------------------------------------------------------\n\ from(\"file://inputdir/?recursive=true&delete=true\").to(\"file://outputdir?flatten=true\"\ )\n---------------------------------------------------------------------------------------\n\ \nWill result in the following output layout:\n\n[source,java]\n-----------------\n\ outputdir/foo.txt\noutputdir/bar.txt\n-----------------\n\n[[File2-Readingfromadirectoryandthedefaultmoveoperation]]\n\ Reading from a directory and the default move operation\n+++++++++++++++++++++++++++++++++++++++++++++++++++++++\n\ \nCamel will by default move any processed file into a `.camel`\nsubdirectory\ \ in the directory the file was consumed from.\n\n[source,java]\n--------------------------------------------------------------------------\n\ from(\"file://inputdir/?recursive=true&delete=true\").to(\"file://outputdir\"\ )\n--------------------------------------------------------------------------\n\ \nAffects the layout as follows: +\n *before*\n\n[source,java]\n--------------------\n\ inputdir/foo.txt\ninputdir/sub/bar.txt\n--------------------\n\n*after*\n\n\ [source,java]\n---------------------------\ninputdir/.camel/foo.txt\ninputdir/sub/.camel/bar.txt\n\ outputdir/foo.txt\noutputdir/sub/bar.txt\n---------------------------\n\n[[File2-Readfromadirectoryandprocessthemessageinjava]]\n\ Read from a directory and process the message in java\n+++++++++++++++++++++++++++++++++++++++++++++++++++++\n\ \n[source,java]\n-----------------------------------------------------------\n\ from(\"file://inputdir/\").process(new Processor() {\n public void process(Exchange\ \ exchange) throws Exception {\n Object body = exchange.getIn().getBody();\n\ \ // do some business logic with the input body\n }\n});\n-----------------------------------------------------------\n\ \nThe body will be a `File` object that points to the file that was just\ndropped\ \ into the `inputdir` directory.\n\n[[File2-Writingtofiles]]\nWriting to files\n\ ++++++++++++++++\n\nCamel is of course also able to write files, i.e. produce\ \ files. In the\nsample below we receive some reports on the SEDA queue that\ \ we process\nbefore they are being written to a directory.\n\n[[File2-WritetosubdirectoryusingExchange.FILE_NAME]]\n\ Write to subdirectory using `Exchange.FILE_NAME`\n++++++++++++++++++++++++++++++++++++++++++++++++\n\ \nUsing a single route, it is possible to write a file to any number of\nsubdirectories.\ \ If you have a route setup as such:\n\n[source,xml]\n-----------------------------------\n\ \ \n \n \n \n-----------------------------------\n\nYou can have `myBean`\ \ set the header `Exchange.FILE_NAME` to values such\nas:\n\n[source,java]\n\ --------------------------------------------------------------\nExchange.FILE_NAME\ \ = hello.txt => /rootDirectory/hello.txt\nExchange.FILE_NAME = foo/bye.txt\ \ => /rootDirectory/foo/bye.txt\n--------------------------------------------------------------\n\ \nThis allows you to have a single route to write files to multiple\ndestinations.\n\ \n[[File2-Writingfilethroughthetemporarydirectoryrelativetothefinaldestination]]\n\ Writing file through the temporary directory relative to the final destination\n\ ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++\n\ \nSometime you need to temporarily write the files to some directory\nrelative\ \ to the destination directory. Such situation usually happens\nwhen some external\ \ process with limited filtering capabilities is\nreading from the directory\ \ you are writing to. In the example below\nfiles will be written to the?? `/var/myapp/filesInProgress`\ \ directory and\nafter data transfer is done, they will be atomically moved\ \ to\nthe`??/var/myapp/finalDirectory `directory.\n\n[source,java]\n-------------------------------------------------------------------------\n\ from(\"direct:start\").\n to(\"file:///var/myapp/finalDirectory?tempPrefix=/../filesInProgress/\"\ );\n-------------------------------------------------------------------------\n\ \n[[File2-Usingexpressionforfilenames]]\nUsing expression for filenames\n++++++++++++++++++++++++++++++\n\ \nIn this sample we want to move consumed files to a backup folder using\ntoday's\ \ date as a sub-folder name:\n\n[source,java]\n-----------------------------------------------------------------------------\n\ from(\"file://inbox?move=backup/${date:now:yyyyMMdd}/${file:name}\").to(\"...\"\ );\n-----------------------------------------------------------------------------\n\ \nSee link:file-language.html[File Language] for more samples.\n\n[[File2-Avoidingreadingthesamefilemorethanonceidempotentconsumer]]\n\ Avoiding reading the same file more than once (idempotent consumer)\n^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n\ \nCamel supports link:idempotent-consumer.html[Idempotent Consumer]\ndirectly\ \ within the component so it will skip already processed files.\nThis feature\ \ can be enabled by setting the `idempotent=true` option.\n\n[source,java]\n\ -----------------------------------------------\nfrom(\"file://inbox?idempotent=true\"\ ).to(\"...\");\n-----------------------------------------------\n\nCamel uses\ \ the absolute file name as the idempotent key, to detect\nduplicate files.\ \ From *Camel 2.11* onwards you can customize this key by\nusing an expression\ \ in the idempotentKey option. For example to use both\nthe name and the file\ \ size as the key\n\n[source,xml]\n------------------------------------------------------------------------------------------\n\ \ \n \n \n \n------------------------------------------------------------------------------------------\n\ \nBy default Camel uses a in memory based store for keeping track of\nconsumed\ \ files, it uses a least recently used cache holding up to 1000\nentries. You\ \ can plugin your own implementation of this store by using\nthe `idempotentRepository`\ \ option using the `#` sign in the value to\nindicate it's a referring to a\ \ bean in the link:registry.html[Registry]\nwith the specified `id`.\n\n[source,xml]\n\ --------------------------------------------------------------------------------\n\ \ \n \n\n \n \n \n \n--------------------------------------------------------------------------------\n\ \nCamel will log at `DEBUG` level if it skips a file because it has been\nconsumed\ \ before:\n\n[source,java]\n-------------------------------------------------------------------------------------------------------------------------\n\ DEBUG FileConsumer is idempotent and the file has been consumed before. Will\ \ skip this file: target\\idempotent\\report.txt\n-------------------------------------------------------------------------------------------------------------------------\n\ \n[[File2-Usingafilebasedidempotentrepository]]\nUsing a file based idempotent\ \ repository\n++++++++++++++++++++++++++++++++++++++++\n\nIn this section we\ \ will use the file based idempotent repository\n`org.apache.camel.processor.idempotent.FileIdempotentRepository`\ \ instead\nof the in-memory based that is used as default. +\n This repository\ \ uses a 1st level cache to avoid reading the file\nrepository. It will only\ \ use the file repository to store the content of\nthe 1st level cache. Thereby\ \ the repository can survive server restarts.\nIt will load the content of the\ \ file into the 1st level cache upon\nstartup. The file structure is very simple\ \ as it stores the key in\nseparate lines in the file. By default, the file\ \ store has a size limit\nof 1mb. When the file grows larger Camel will truncate\ \ the file store,\nrebuilding the content by flushing the 1st level cache into\ \ a fresh\nempty file.\n\nWe configure our repository using Spring XML creating\ \ our file\nidempotent repository and define our file consumer to use our repository\n\ with the `idempotentRepository` using `#` sign to indicate\nlink:registry.html[Registry]\ \ lookup:\n\n[[File2-UsingaJPAbasedidempotentrepository]]\nUsing a JPA based\ \ idempotent repository\n+++++++++++++++++++++++++++++++++++++++\n\nIn this\ \ section we will use the JPA based idempotent repository instead\nof the in-memory\ \ based that is used as default.\n\nFirst we need a persistence-unit in `META-INF/persistence.xml`\ \ where we\nneed to use the class\n`org.apache.camel.processor.idempotent.jpa.MessageProcessed`\ \ as model.\n\n[source,xml]\n---------------------------------------------------------------------------------\n\ \n org.apache.camel.processor.idempotent.jpa.MessageProcessed\n\ \ \n \n \n \n \n \n \n \n\ \n---------------------------------------------------------------------------------\n\ \nNext, we can create our JPA idempotent repository in the spring\nXML file\ \ as well:\n\n[source,xml]\n---------------------------------------------------------------------------------\n\ \n\n \n \n \n \n\n---------------------------------------------------------------------------------\n\ \nAnd yes then we just need to refer to the *jpaStore* bean in the file\nconsumer\ \ endpoint using the `idempotentRepository` using the `#` syntax\noption:\n\n\ [source,xml]\n---------------------------------------------------------------------------------\n\ \ \n \n \n \n---------------------------------------------------------------------------------\n\ \n[[File2-Filterusingorg.apache.camel.component.file.GenericFileFilter]]\nFilter\ \ using org.apache.camel.component.file.GenericFileFilter\n^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n\ \nCamel supports pluggable filtering strategies. You can then configure\nthe\ \ endpoint with such a filter to skip certain files being processed.\n\nIn the\ \ sample we have built our own filter that skips files starting\nwith `skip`\ \ in the filename:\n\nAnd then we can configure our route using the *filter*\ \ attribute to\nreference our filter (using `#` notation) that we have defined\ \ in the\nspring XML file:\n\n[source,xml]\n----------------------------------------------------------\n\ \ \n \n \n \n \n \n \n----------------------------------------------------------\n\ \n[[File2-FilteringusingANTpathmatcher]]\nFiltering using ANT path matcher\n\ ++++++++++++++++++++++++++++++++\n\nTIP:*New options from Camel 2.10 onwards*\n\ There are now `antInclude` and `antExclude` options to make it easy to\nspecify\ \ ANT style include/exclude without having to define the filter.\nSee the URI\ \ options above for more information.\n\nThe ANT path matcher is shipped out-of-the-box\ \ in the *camel-spring*\njar. So you need to depend on *camel-spring* if you\ \ are using Maven. +\n The reasons is that we leverage Spring's\nhttp://static.springframework.org/spring/docs/2.5.x/api/org/springframework/util/AntPathMatcher.html[AntPathMatcher]\n\ to do the actual matching.\n\nThe file paths is matched with the following rules:\n\ \n* `?` matches one character\n* `*` matches zero or more characters\n* `**`\ \ matches zero or more directories in a path\n\nThe sample below demonstrates\ \ how to use it:\n\n[[File2-SortingusingComparator]]\nSorting using Comparator\n\ ^^^^^^^^^^^^^^^^^^^^^^^^\n\nCamel supports pluggable sorting strategies. This\ \ strategy it to use the\nbuild in `java.util.Comparator` in Java. You can then\ \ configure the\nendpoint with such a comparator and have Camel sort the files\ \ before\nbeing processed.\n\nIn the sample we have built our own comparator\ \ that just sorts by file\nname:\n\nAnd then we can configure our route using\ \ the *sorter* option to\nreference to our sorter (`mySorter`) we have defined\ \ in the spring XML\nfile:\n\n[source,xml]\n-----------------------------------------------------------\n\ \ \n \n\n \n \n \n \n-----------------------------------------------------------\n\ \nTIP:*URI options can reference beans using the # syntax*\nIn the Spring DSL\ \ route above notice that we can refer to beans in the\nlink:registry.html[Registry]\ \ by prefixing the id with `#`. So writing\n`sorter=#mySorter`, will instruct\ \ Camel to go look in the\nlink:registry.html[Registry] for a bean with the\ \ ID, `mySorter`.\n\n[[File2-SortingusingsortBy]]\nSorting using sortBy\n^^^^^^^^^^^^^^^^^^^^\n\ \nCamel supports pluggable sorting strategies. This strategy it to use the\n\ link:file-language.html[File Language] to configure the sorting. The\n`sortBy`\ \ option is configured as follows:\n\n[source,java]\n----------------------------------\n\ sortBy=group 1;group 2;group 3;...\n----------------------------------\n\nWhere\ \ each group is separated with semi colon. In the simple situations\nyou just\ \ use one group, so a simple example could be:\n\n[source,java]\n----------------\n\ sortBy=file:name\n----------------\n\nThis will sort by file name, you can reverse\ \ the order by prefixing\n`reverse:` to the group, so the sorting is now Z..A:\n\ \n[source,java]\n------------------------\nsortBy=reverse:file:name\n------------------------\n\ \nAs we have the full power of link:file-language.html[File Language] we\ncan\ \ use some of the other parameters, so if we want to sort by file size\nwe do:\n\ \n[source,java]\n------------------\nsortBy=file:length\n------------------\n\ \nYou can configure to ignore the case, using `ignoreCase:` for string\ncomparison,\ \ so if you want to use file name sorting but to ignore the\ncase then we do:\n\ \n[source,java]\n---------------------------\nsortBy=ignoreCase:file:name\n\ ---------------------------\n\nYou can combine ignore case and reverse, however\ \ reverse must be\nspecified first:\n\n[source,java]\n-----------------------------------\n\ sortBy=reverse:ignoreCase:file:name\n-----------------------------------\n\n\ In the sample below we want to sort by last modified file, so we do:\n\n[source,java]\n\ --------------------\nsortBy=file:modified\n--------------------\n\nAnd then\ \ we want to group by name as a 2nd option so files with same\nmodifcation is\ \ sorted by name:\n\n[source,java]\n------------------------------\nsortBy=file:modified;file:name\n\ ------------------------------\n\nNow there is an issue here, can you spot it?\ \ Well the modified timestamp\nof the file is too fine as it will be in milliseconds,\ \ but what if we\nwant to sort by date only and then subgroup by name? +\n Well\ \ as we have the true power of link:file-language.html[File\nLanguage] we can\ \ use??its date command that supports patterns. So this\ncan be solved as:\n\ \n[source,java]\n-----------------------------------\nsortBy=date:file:yyyyMMdd;file:name\n\ -----------------------------------\n\nYeah, that is pretty powerful, oh by\ \ the way you can also use reverse\nper group, so we could reverse the file\ \ names:\n\n[source,java]\n-------------------------------------------\nsortBy=date:file:yyyyMMdd;reverse:file:name\n\ -------------------------------------------\n\n[[File2-UsingGenericFileProcessStrategy]]\n\ Using GenericFileProcessStrategy\n^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n\nThe option\ \ `processStrategy` can be used to use a custom\n`GenericFileProcessStrategy`\ \ that allows you to implement your own\n_begin_, _commit_ and _rollback_ logic.\ \ +\n For instance lets assume a system writes a file in a folder you should\n\ consume. But you should not start consuming the file before another\n_ready_\ \ file has been written as well.\n\nSo by implementing our own `GenericFileProcessStrategy`\ \ we can implement\nthis as:\n\n* In the `begin()` method we can test whether\ \ the special _ready_ file\nexists. The begin method returns a `boolean` to\ \ indicate if we can\nconsume the file or not.\n* In the `abort()` method (Camel\ \ 2.10) special logic can be executed in\ncase the `begin` operation returned\ \ `false`, for example to cleanup\nresources etc.\n* in the `commit()` method\ \ we can move the actual file and also delete\nthe _ready_ file.\n\n[[File2-Usingfilter]]\n\ Using filter\n^^^^^^^^^^^^\n\nThe `filter` option allows you to implement a\ \ custom filter in Java code\nby implementing the `org.apache.camel.component.file.GenericFileFilter`\n\ interface. This interface has an `accept` method that returns a boolean.\nReturn\ \ `true` to include the file, and `false` to skip the file. From\nCamel 2.10\ \ onwards, there is a `isDirectory` method on `GenericFile`\nwhether the file\ \ is a directory. This allows you to filter unwanted\ndirectories, to avoid\ \ traversing down unwanted directories.\n\nFor example to skip any directories\ \ which starts with `\"skip\"` in the\nname, can be implemented as follows:\n\ \n[[File2-HowtousetheCamelerrorhandlertodealwithexceptionstriggeredoutsidetheroutingengine]]\n\ How to use the Camel error handler to deal with exceptions triggered outside\ \ the routing engine\n^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n\ \nThe file and ftp consumers, will by default try to pickup files. Only if\n\ that is successful then a Camel link:exchange.html[Exchange] can be\ncreated\ \ and passed in the Camel routing engine. +\n When the link:exchange.html[Exchange]\ \ is processed by the routing\nengine, then the Camel link:error-handling-in-camel.html[Error\ \ Handling]\ntakes over (eg the onException / errorHandler in the routes). +\n\ \ However outside the scope of the routing engine, any exceptions\nhandling\ \ is component specific. Camel offers a\n`org.apache.camel.spi.ExceptionHandler`\ \ that allows components +\n to use that as a pluggable hook for end users to\ \ use their own\nimplementation. Camel offers a default `LoggingExceptionHandler`\ \ that\nwill log the exception at ERROR/WARN level. +\n For the file and ftp\ \ components this would be the case. However if you\nwant to bridge the `ExceptionHandler`\ \ so it uses the Camel\nlink:error-handling-in-camel.html[Error Handling], then\ \ +\n you need to implement a custom `ExceptionHandler` that will handle the\n\ exception by creating a Camel link:exchange.html[Exchange] and send it\nto the\ \ routing engine; then the error handling of the routing engine can\nget triggered.\n\ \nHere is such an example based upon an unit test.\n\nFirst we have a custom\ \ `ExceptionHandler` where you can see we deal with\nthe exception by sending\ \ it to a Camel link:endpoint.html[Endpoint]\nnamed \"direct:file-error\":\n\ \n*MyExceptionHandler*\n\nThen we have a Camel route that uses the Camel routing\ \ error handler,\nwhich is the `onException` where we handle any IOException\ \ being\nthrown. +\n We then send the message to the same \"direct:file-error\"\ \ endpoint,\nwhere we handle it by transforming it to a message, and then being\ \ sent\nto a link:mock.html[Mock] endpoint. +\n This is just for testing purpose.\ \ You can handle the exception in any\ncustom way you want, such as using a\ \ link:bean.html[Bean] or sending an\nemail etc.\n\nNotice how we configure\ \ our custom `MyExceptionHandler` by using the\n`consumer.exceptionHandler`\ \ option to refer to `#myExceptionHandler`\nwhich is a id of the bean registered\ \ in the\nlink:registry.html[Registry]. If using Spring XML or OSGi Blueprint,\n\ then that would be a :\n\n*Camel route with routing engine error handling*\n\nThe source code for\ \ this example can be seen\nhttps://svn.apache.org/repos/asf/camel/trunk/camel-core/src/test/java/org/apache/camel/component/file/FileConsumerCustomExceptionHandlerTest.java[here]\n\ \n[[File2-Usingconsumer.bridgeErrorHandler]]\nUsing consumer.bridgeErrorHandler\n\ +++++++++++++++++++++++++++++++++\n\n*Available as of Camel 2.10*\n\nIf you\ \ want to use the Camel link:error-handler.html[Error Handler] to\ndeal with\ \ any exception occurring in the file consumer, then you can\nenable the `consumer.bridgeErrorHandler`\ \ option as shown below:\n\n*Using consumer.bridgeErrorHandler*\n\nSo all you\ \ have to do is to enable this option, and the error handler in\nthe route will\ \ take it from there. [Info]\n\n*Important when using\nconsumer.bridgeErrorHandler*\n\ \nWhen using consumer.bridgeErrorHandler, then\nlink:intercept.html[interceptors],\ \ link:oncompletion.html[OnCompletion]s\ndoes *not* apply. The link:exchange.html[Exchange]\ \ is processed directly\nby the Camel link:error-handler.html[Error Handler],\ \ and does not allow\nprior actions such as interceptors, onCompletion to take\ \ action.\n\n[[File2-Debuglogging]]\nDebug logging\n^^^^^^^^^^^^^\n\nThis component\ \ has log level *TRACE* that can be helpful if you have\nproblems.\n\n[[File2-SeeAlso]]\n\ See Also\n^^^^^^^^\n\n* link:configuring-camel.html[Configuring Camel]\n* link:component.html[Component]\n\ * link:endpoint.html[Endpoint]\n* link:getting-started.html[Getting Started]\n\ \n* link:file-language.html[File Language]\n* link:ftp2.html[FTP]\n* link:polling-consumer.html[Polling\ \ Consumer]\n\n"