How to write spark structured streaming data to REST API?

Question

I would like to push my spark structured streaming processed data to the REST API. can someone share the examples of same. i have found few but all are related to spark streaming, not the structured streaming.

Jacek Laskowski · Accepted Answer · 2019-01-28 18:41:09Z

1

I have not heard about a REST API sink for Spark Structured Streaming, but you could write one yourself. Start from org.apache.spark.sql.execution.streaming.Source.

The easiest would however be to use DataStreamWriter.foreach or foreachBatch (since 2.4).

answered Jan 28, 2019 at 18:41

Jacek Laskowski

75k28 gold badges253 silver badges440 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Mehdi LAMRANI Over a year ago

This is a big hole in the API. Rest should be made available as Source and Sink, as it is as Ubiquitous as Json. I was a bit shocked there was none embedded already. Planning to write one for the community if I manage to find some time to do it.

Collectives™ on Stack Overflow

How to write spark structured streaming data to REST API?

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related