How do you integrate this new Spark libary for python?

Question

I am using spark with python using the pyspark command.

I came across this new library

which I want to integrate into my spark program written in python.

I have downloaded the complete codebase shown in the above link but not sure how to proceed from there. Should I just copy its contents somewhere?

Any guidance would help as I am still learning scala and spark and want to use the above library code.

zero323 · Accepted Answer · 2015-12-03 14:47:30Z

2

Just to repeat information from the README file. To use spark-csv in your script you should use --packages option for each execution of spark-submit or pyspark shell.

answered Dec 3, 2015 at 14:47

community wiki

zero323

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

How do you integrate this new Spark libary for python?

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related