Azure Event Hubs integration with Apache Spark in Python

Question

I would like to know if there are something like this in python?

If so, could you provided any web with examples?

EXTRA:

If you are trying to run it like a normal python program it will give you an error:

Run it using:

spark-submit --packages com.microsoft.azure:azure-eventhubs-spark_2.11:2.3.9 nameoffile.py

simon_dmorias · Accepted Answer · 2019-01-25 08:54:58Z

5

https://docs.databricks.com/_static/notebooks/structured-streaming-python.html

https://github.com/Azure/azure-event-hubs-spark/blob/master/docs/PySpark/structured-streaming-pyspark.md

# Source with default settings
connectionString = "YOUR.CONNECTION.STRING"
ehConf = {
  'eventhubs.connectionString' : connectionString
}

df = spark \
  .readStream \
  .format("eventhubs") \
  .options(**ehConf) \
  .load()

edited Jan 25, 2019 at 8:54

answered Jan 24, 2019 at 19:51

simon_dmorias

2,4933 gold badges23 silver badges38 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

IoT user Over a year ago

Believe me when I ask something I try to look for it before.The web you posted is the same I saw before and that is not what I asking/looking for. In the title of the post I talk about Integrating Apache Spark with Azure Event Hubs. I saw that with scala but not with python.So I would like to know about any python connector to work like this org.apache.spark.eventhubs

simon_dmorias Over a year ago

I've updated accordingly. Apologies I thought the original mentioned event hub as well.

IoT user Over a year ago

Thanks @HaukeMallow

Collectives™ on Stack Overflow

Azure Event Hubs integration with Apache Spark in Python

1 Answer 1

3 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

3 Comments

Your Answer

Sign up or log in

Post as a guest

Related