Let’s try something a bit more complicated and swap out the time source for something else. Another supported source type is http, which accepts data for ingestion over HTTP POSTs. Note that the http source accepts data on a different port from the Data Flow Server (default 8080). By default the port is randomly assigned.

To create a stream using an http source, but still using the same log sink, we would change the original command above to

dataflow:> stream create --definition "http | log" --name myhttpstream --deploy

which will produce the following output from the server

2016-01-13 18:42:19.162  INFO 19463 --- [nio-9393-exec-7] o.s.c.d.a.s.l.OutOfProcessModuleDeployer : deploying module org.springframework.cloud.stream.module:log-sink:jar:exec:1.0.0.BUILD-SNAPSHOT instance 0
   Logs will be in /var/folders/hs/h87zy7z17qs6mcnl4hj8_dp00000gp/T/spring-cloud-data-flow-2185888994718649403/myhttpstream.log
2016-01-13 18:42:19.180  INFO 19463 --- [nio-9393-exec-7] o.s.c.d.a.s.l.OutOfProcessModuleDeployer : deploying module org.springframework.cloud.stream.module:http-source:jar:exec:1.0.0.BUILD-SNAPSHOT instance 0
   Logs will be in /var/folders/hs/h87zy7z17qs6mcnl4hj8_dp00000gp/T/spring-cloud-data-flow-2185888994718649403/myhttpstream.http

Note that we don’t see anydefin other output this time until we actually post some data (using shell command). In order to see the randomly assigned port on which the http source is listening, execute:

dataflow:> runtime modules

You should see that the corresponding http source has a url property containing the host and port information on which it is listening. You are now ready to post to that url, e.g.:

dataflow:> http post --target http://localhost:1234 --data "hello"
dataflow:> http post --target http://localhost:1234 --data "goodbye"

and the stream will then funnel the data from the http source to the output log implemented by the log sink

2016-01-13 21:15:34.825  INFO 54348 --- [hannel-adapter1] log.sink                                 : hello
2016-01-13 21:17:36.544  INFO 54348 --- [hannel-adapter1] log.sink                                 : goodbye

Of course, we could also change the sink implementation. You could pipe the output to a file (file), to hadoop (hdfs) or to any of the other sink modules which are provided. You can also define your own modules.