How to store JavaRDD as a sequence file using spark java API?

Question

I want to store JavaRDD as a sequence file instead of textfile. But i don't see any Java API for that. Is there a way for this? Please let me know. Thanks!

Just use Scala :) You will make your life so much easier.

samthebest
– samthebest

2014-06-20 13:08:47 +00:00
Commented Jun 20, 2014 at 13:08 — samthebest
– samthebest, Commented Jun 20, 2014 at 13:08

samthebest · Accepted Answer · 2014-06-20 13:11:12Z

4

Maybe JavaPairRDD has a method .saveAsSequenceFile if not I think you can use .saveAsHadoopFile and have SequenceFileOutputFormat.class as the format param. It seems to be explained a bit here: http://apache-spark-user-list.1001560.n3.nabble.com/Spark-SequenceFile-Java-API-Repeat-Key-Values-td353.html

answered Jun 20, 2014 at 13:11

samthebest

31.7k25 gold badges106 silver badges153 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Fisher Coder Over a year ago

Thanks @samthebest for pointing this approach, but I tried and faced with an issue, here's my question: stackoverflow.com/questions/44187041/…, do you have any ideas please?

Collectives™ on Stack Overflow

How to store JavaRDD as a sequence file using spark java API?

1 Answer 1

1 Comment

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Linked

Related