Package org.apache.pulsar.functions.source.batch

Class BatchSourceExecutor<T>

  • All Implemented Interfaces:
    java.lang.AutoCloseable, org.apache.pulsar.io.core.Source<T>
    public class BatchSourceExecutor<T>
extends java.lang.Object
implements org.apache.pulsar.io.core.Source<T>
    BatchSourceExecutor wraps BatchSource as Source. Thus from Pulsar IO perspective, it is running a regular streaming source. The BatchSourceExecutor orchestrates the lifecycle of BatchSource. The current implementation uses an intermediate topic between the discovery process and the actual batchsource instances. The Discovery is run on 0th instance. Any tasks discovered during the discover are written to the intermediate topic. All the instances consume tasks from this intermediate topic using a shared subscription.

    • Method Summary

      All Methods Instance Methods Concrete Methods 
      Modifier and Type Method Description
      void close()  
      void open​(java.util.Map<java.lang.String,​java.lang.Object> config, org.apache.pulsar.io.core.SourceContext sourceContext)  
      org.apache.pulsar.functions.api.Record<T> read()  

      • Methods inherited from class java.lang.Object

        clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

    • Constructor Detail

      • BatchSourceExecutor

        public BatchSourceExecutor()

    • Method Detail

      • open

        public void open​(java.util.Map<java.lang.String,​java.lang.Object> config,
                 org.apache.pulsar.io.core.SourceContext sourceContext)
          throws java.lang.Exception
        Specified by:
        open in interface org.apache.pulsar.io.core.Source<T>
        Throws:
        java.lang.Exception

      • read

        public org.apache.pulsar.functions.api.Record<T> read()
                                               throws java.lang.Exception
        Specified by:
        read in interface org.apache.pulsar.io.core.Source<T>
        Throws:
        java.lang.Exception

      • close

        public void close()
           throws java.lang.Exception
        Specified by:
        close in interface java.lang.AutoCloseable
        Throws:
        java.lang.Exception