--- apiVersion: v1 kind: List items: - apiVersion: v1 kind: ConfigMap metadata: labels: funktion.fabric8.io/kind: Connector provider: fabric8 project: connector-sjms-batch version: 1.1.37 group: io.fabric8.funktion.connector name: sjms-batch data: deployment.yml: | --- apiVersion: extensions/v1beta1 kind: Deployment metadata: labels: funktion.fabric8.io/kind: Subscription connector: sjms-batch spec: replicas: 1 template: metadata: labels: funktion.fabric8.io/kind: Subscription connector: sjms-batch spec: containers: - image: funktion/connector-sjms-batch:1.1.37 name: connector schema.yml: | --- component: kind: component scheme: sjms-batch syntax: sjms-batch:destinationName title: Simple JMS Batch description: The sjms-batch component is a specialized for highly performant transactional batch consumption from a JMS queue. label: messaging deprecated: false async: false consumerOnly: true javaType: org.apache.camel.component.sjms.batch.SjmsBatchComponent groupId: org.apache.camel artifactId: camel-sjms version: 2.18.1 componentProperties: connectionFactory: kind: property type: object javaType: javax.jms.ConnectionFactory deprecated: false secret: false description: A ConnectionFactory is required to enable the SjmsBatchComponent. order: 0 properties: destinationName: kind: path group: consumer required: true type: string javaType: java.lang.String deprecated: false secret: false description: The destination name. Only queues are supported names may be prefixed by 'queue:'. order: 0 aggregationStrategy: kind: parameter group: consumer required: true type: object javaType: org.apache.camel.processor.aggregate.AggregationStrategy deprecated: false secret: false description: The aggregation strategy to use which merges all the batched messages into a single message order: 1 allowNullBody: kind: parameter group: consumer type: boolean javaType: boolean deprecated: false secret: false defaultValue: true description: Whether to allow sending messages with no body. If this option is false and the message body is null then an JMSException is thrown. order: 2 bridgeErrorHandler: kind: parameter group: consumer label: consumer type: boolean javaType: boolean optionalPrefix: consumer. deprecated: false secret: false defaultValue: false description: Allows for bridging the consumer to the Camel routing Error Handler which mean any exceptions occurred while the consumer is trying to pickup incoming messages or the likes will now be processed as a message and handled by the routing Error Handler. By default the consumer will use the org.apache.camel.spi.ExceptionHandler to deal with exceptions that will be logged at WARN/ERROR level and ignored. order: 3 completionInterval: kind: parameter group: consumer type: integer javaType: int deprecated: false secret: false defaultValue: "1000" description: The completion interval in millis which causes batches to be completed in a scheduled fixed rate every interval. The batch may be empty if the timeout triggered and there was no messages in the batch. Notice you cannot use both completion timeout and completion interval at the same time only one can be configured. order: 4 completionPredicate: kind: parameter group: consumer type: string javaType: java.lang.String deprecated: false secret: false description: The completion predicate which causes batches to be completed when the predicate evaluates as true. The predicate can also be configured using the simple language using the string syntax. You may want to set the option eagerCheckCompletion to true to let the predicate match the incoming message as otherwise it matches the aggregated message. order: 5 completionSize: kind: parameter group: consumer type: integer javaType: int deprecated: false secret: false defaultValue: "200" description: The number of messages consumed at which the batch will be completed order: 6 completionTimeout: kind: parameter group: consumer type: integer javaType: int deprecated: false secret: false defaultValue: "500" description: The timeout in millis from receipt of the first first message when the batch will be completed. The batch may be empty if the timeout triggered and there was no messages in the batch. Notice you cannot use both completion timeout and completion interval at the same time only one can be configured. order: 7 consumerCount: kind: parameter group: consumer type: integer javaType: int deprecated: false secret: false defaultValue: "1" description: The number of JMS sessions to consume from order: 8 eagerCheckCompletion: kind: parameter group: consumer type: boolean javaType: boolean deprecated: false secret: false defaultValue: false description: Use eager completion checking which means that the completionPredicate will use the incoming Exchange. As opposed to without eager completion checking the completionPredicate will use the aggregated Exchange. order: 9 includeAllJMSXProperties: kind: parameter group: consumer type: boolean javaType: boolean deprecated: false secret: false defaultValue: false description: 'Whether to include all JMSXxxx properties when mapping from JMS to Camel Message. Setting this to true will include properties such as JMSXAppID and JMSXUserID etc. Note: If you are using a custom headerFilterStrategy then this option does not apply.' order: 10 mapJmsMessage: kind: parameter group: consumer type: boolean javaType: boolean deprecated: false secret: false defaultValue: true description: Specifies whether Camel should auto map the received JMS message to a suited payload type such as javax.jms.TextMessage to a String etc. See section about how mapping works below for more details. order: 11 pollDuration: kind: parameter group: consumer type: integer javaType: int deprecated: false secret: false defaultValue: "1000" description: The duration in milliseconds of each poll for messages. completionTimeOut will be used if it is shorter and a batch has started. order: 12 sendEmptyMessageWhenIdle: kind: parameter group: consumer type: boolean javaType: boolean deprecated: false secret: false defaultValue: false description: If using completion timeout or interval then the batch may be empty if the timeout triggered and there was no messages in the batch. If this option is true and the batch is empty then an empty message is added to the batch so an empty message is routed. order: 13 exceptionHandler: kind: parameter group: consumer (advanced) label: consumer,advanced type: object javaType: org.apache.camel.spi.ExceptionHandler optionalPrefix: consumer. deprecated: false secret: false description: To let the consumer use a custom ExceptionHandler. Notice if the option bridgeErrorHandler is enabled then this options is not in use. By default the consumer will deal with exceptions that will be logged at WARN/ERROR level and ignored. order: 14 exchangePattern: kind: parameter group: consumer (advanced) label: consumer,advanced type: string javaType: org.apache.camel.ExchangePattern enum: - InOnly - RobustInOnly - InOut - InOptionalOut - OutOnly - RobustOutOnly - OutIn - OutOptionalIn deprecated: false secret: false description: Sets the exchange pattern when the consumer creates an exchange. order: 15 headerFilterStrategy: kind: parameter group: advanced label: advanced type: object javaType: org.apache.camel.spi.HeaderFilterStrategy deprecated: false secret: false description: To use a custom HeaderFilterStrategy to filter header to and from Camel message. order: 16 jmsKeyFormatStrategy: kind: parameter group: advanced label: advanced type: object javaType: org.apache.camel.component.sjms.jms.JmsKeyFormatStrategy deprecated: false secret: false description: 'Pluggable strategy for encoding and decoding JMS keys so they can be compliant with the JMS specification. Camel provides two implementations out of the box: default and passthrough. The default strategy will safely marshal dots and hyphens (. and -). The passthrough strategy leaves the key as is. Can be used for JMS brokers which do not care whether JMS header keys contain illegal characters. You can provide your own implementation of the org.apache.camel.component.jms.JmsKeyFormatStrategy and refer to it using the notation.' order: 17 messageCreatedStrategy: kind: parameter group: advanced label: advanced type: object javaType: org.apache.camel.component.sjms.jms.MessageCreatedStrategy deprecated: false secret: false description: To use the given MessageCreatedStrategy which are invoked when Camel creates new instances of javax.jms.Message objects when Camel is sending a JMS message. order: 18 synchronous: kind: parameter group: advanced label: advanced type: boolean javaType: boolean deprecated: false secret: false defaultValue: false description: Sets whether synchronous processing should be strictly used or Camel is allowed to use asynchronous processing (if supported). order: 19 timeoutCheckerExecutorService: kind: parameter group: advanced label: advanced type: object javaType: java.util.concurrent.ScheduledExecutorService deprecated: false secret: false description: If using the completionInterval option a background thread is created to trigger the completion interval. Set this option to provide a custom thread pool to be used rather than creating a new thread for every consumer. order: 20 documentation.adoc: | [[ConfluenceContent]] [[SJMSBatch-SJMSBatchComponent]] SJMS Batch Component ~~~~~~~~~~~~~~~~~~~~ *Available as of Camel 2.16* SJMS Batch is a specialized component for highly performant, transactional batch consumption from a JMS queue. It can be thought of as a hybrid of a consumer-only component and an link:aggregator2.html[aggregator]. A common use case in Camel is to consume messages from a queue and aggregate them before sending the aggregated state to another endpoint. In order to ensure that data is not lost if the system performing the processing fails, it is typically consumed within a transaction from the queue, and once aggregated stored in a persistent `AggregationRepository`, such as the one found in the link:jdbc.html[JDBC Component]. The behavior of the aggregator pattern involves fetching data from the `AggregationRepository` before an incoming message is aggregated, and writing back the result afterwards. By nature, the reads and writes take progressively longer as the number of aggregated artifacts increases. A rough example of this using arbitrary time units that demonstrates the impact of this is as follows: [cols=",,,",options="header",] |======================================= |Item |Read Time |Write Time |Total Time |0 |0 |1 |1 |1 |1 |2 |4 |2 |2 |3 |9 |3 |3 |4 |16 |4 |4 |5 |25 |5 |5 |6 |36 |6 |6 |7 |49 |7 |7 |8 |64 |8 |8 |9 |81 |9 |9 |10 |100 |======================================= In contrast, consumption performance using the SJMS Batch component is linear. Each message is consumed and aggregated using an `AggregationStrategy` before the next one is fetched. As all of the consumption and aggregation is performed in a single JMS transaction no external storage is required to persist the intermediate state - this avoids the read and write costs described above. In practice, this yields multiple orders of magnitude higher throughput. Once a completion condition is met, either by size or period since first message, the aggregated `Exchange` is passed into the route. During the processing of this `Exchange`, if an exception is thrown or the system shuts down, all of the original consumed messages end up back on the queue (or are placed on the dead-letter queue depending on the broker configuration). Unlike using a regular aggregator, there is no facility for an aggregation condition, meaning that it is not possible to batch consume into multiple groups of messages. All consumed messages are aggregated together into a single batch. Multiple JMS consumer support is available which allows you to consume in parallel using the one route, and at the same time use facilities like JMS message groups to group related messages. Maven users will need to add the following dependency to their `pom.xml` for this component: [source,xml] ---- org.apache.camel camel-sjms x.x.x ---- [[SJMSBatch-URIformat]] URI format ++++++++++ [source] ---- sjms:[queue:]destinationName[?options] ---- Where `destinationName` is a JMS queue. By default, the `destinationName` is interpreted as a queue name. [source] ---- sjms:FOO.BAR ---- You can include the optional `queue:` prefix, if you prefer: [source] ---- sjms:queue:FOO.BAR ---- Topic consumption is not supported, as there is no advantage to using batch consumption within that context. Topic messages are usually non-persistent, and loss is acceptable. If consumed within a transaction that fails, a topic message will likely not be redelivered by the broker. A plain link:sjms.html[SJMS] consumer endpoint can be used in conjunction with a regular non-persistence backed link:aggregator2.html[aggregator] in this scenario. [[SJMSBatch-ComponentOptionsandConfigurations]] Component Options and Configurations ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ // component options: START The Simple JMS Batch component supports 1 options which are listed below. {% raw %} [width="100%",cols="2,1m,7",options="header"] |======================================================================= | Name | Java Type | Description | connectionFactory | ConnectionFactory | A ConnectionFactory is required to enable the SjmsBatchComponent. |======================================================================= {% endraw %} // component options: END // endpoint options: START The Simple JMS Batch component supports 21 endpoint options which are listed below: {% raw %} [width="100%",cols="2,1,1m,1m,5",options="header"] |======================================================================= | Name | Group | Default | Java Type | Description | destinationName | consumer | | String | *Required* The destination name. Only queues are supported names may be prefixed by 'queue:'. | aggregationStrategy | consumer | | AggregationStrategy | *Required* The aggregation strategy to use which merges all the batched messages into a single message | allowNullBody | consumer | true | boolean | Whether to allow sending messages with no body. If this option is false and the message body is null then an JMSException is thrown. | bridgeErrorHandler | consumer | false | boolean | Allows for bridging the consumer to the Camel routing Error Handler which mean any exceptions occurred while the consumer is trying to pickup incoming messages or the likes will now be processed as a message and handled by the routing Error Handler. By default the consumer will use the org.apache.camel.spi.ExceptionHandler to deal with exceptions that will be logged at WARN/ERROR level and ignored. | completionInterval | consumer | 1000 | int | The completion interval in millis which causes batches to be completed in a scheduled fixed rate every interval. The batch may be empty if the timeout triggered and there was no messages in the batch. Notice you cannot use both completion timeout and completion interval at the same time only one can be configured. | completionPredicate | consumer | | String | The completion predicate which causes batches to be completed when the predicate evaluates as true. The predicate can also be configured using the simple language using the string syntax. You may want to set the option eagerCheckCompletion to true to let the predicate match the incoming message as otherwise it matches the aggregated message. | completionSize | consumer | 200 | int | The number of messages consumed at which the batch will be completed | completionTimeout | consumer | 500 | int | The timeout in millis from receipt of the first first message when the batch will be completed. The batch may be empty if the timeout triggered and there was no messages in the batch. Notice you cannot use both completion timeout and completion interval at the same time only one can be configured. | consumerCount | consumer | 1 | int | The number of JMS sessions to consume from | eagerCheckCompletion | consumer | false | boolean | Use eager completion checking which means that the completionPredicate will use the incoming Exchange. As opposed to without eager completion checking the completionPredicate will use the aggregated Exchange. | includeAllJMSXProperties | consumer | false | boolean | Whether to include all JMSXxxx properties when mapping from JMS to Camel Message. Setting this to true will include properties such as JMSXAppID and JMSXUserID etc. Note: If you are using a custom headerFilterStrategy then this option does not apply. | mapJmsMessage | consumer | true | boolean | Specifies whether Camel should auto map the received JMS message to a suited payload type such as javax.jms.TextMessage to a String etc. See section about how mapping works below for more details. | pollDuration | consumer | 1000 | int | The duration in milliseconds of each poll for messages. completionTimeOut will be used if it is shorter and a batch has started. | sendEmptyMessageWhenIdle | consumer | false | boolean | If using completion timeout or interval then the batch may be empty if the timeout triggered and there was no messages in the batch. If this option is true and the batch is empty then an empty message is added to the batch so an empty message is routed. | exceptionHandler | consumer (advanced) | | ExceptionHandler | To let the consumer use a custom ExceptionHandler. Notice if the option bridgeErrorHandler is enabled then this options is not in use. By default the consumer will deal with exceptions that will be logged at WARN/ERROR level and ignored. | exchangePattern | consumer (advanced) | | ExchangePattern | Sets the exchange pattern when the consumer creates an exchange. | headerFilterStrategy | advanced | | HeaderFilterStrategy | To use a custom HeaderFilterStrategy to filter header to and from Camel message. | jmsKeyFormatStrategy | advanced | | JmsKeyFormatStrategy | Pluggable strategy for encoding and decoding JMS keys so they can be compliant with the JMS specification. Camel provides two implementations out of the box: default and passthrough. The default strategy will safely marshal dots and hyphens (. and -). The passthrough strategy leaves the key as is. Can be used for JMS brokers which do not care whether JMS header keys contain illegal characters. You can provide your own implementation of the org.apache.camel.component.jms.JmsKeyFormatStrategy and refer to it using the notation. | messageCreatedStrategy | advanced | | MessageCreatedStrategy | To use the given MessageCreatedStrategy which are invoked when Camel creates new instances of javax.jms.Message objects when Camel is sending a JMS message. | synchronous | advanced | false | boolean | Sets whether synchronous processing should be strictly used or Camel is allowed to use asynchronous processing (if supported). | timeoutCheckerExecutorService | advanced | | ScheduledExecutorService | If using the completionInterval option a background thread is created to trigger the completion interval. Set this option to provide a custom thread pool to be used rather than creating a new thread for every consumer. |======================================================================= {% endraw %} // endpoint options: END The `completionSize` endpoint attribute is used in conjunction with `completionTimeout`, where the first condition to be met will cause the aggregated `Exchange` to be emitted down the route.