how to insert data to existing collection in mongodb with mongodb-spark connector

Question

I'm using the mongo-spark connector to connect between spark and MongoDB. and I could not insert data to MongoDB because of spark default save mode is to error if the "table" (collection) exists and the I try this.

MongoSpark.write(centenarians).option("collection", "hundredClub")*.option("mode","append")*.save();

but this is not an effect. how can I solve this problem please help me thanks!

This is error message:

Exception in thread "main" java.lang.UnsupportedOperationException: MongoCollection already exists

Krzysztof Atłasik · Accepted Answer · 2019-04-16 10:47:33Z

4

I solved this problem by myself... I share this solution (I'm using spark language by Scala):

centenarians
  .write
  .format("com.mongodb.spark.sql.DefaultSource")
  .option("collection", "hundredClub")
  .mode("append")
  .save()

edited Apr 16, 2019 at 10:47

Krzysztof Atłasik

22.8k6 gold badges57 silver badges79 bronze badges

answered Aug 26, 2017 at 2:42

min heo

15110 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

j ' · Accepted Answer · 2018-07-11 11:16:48Z

1

I'm usign mongo-spark-connector_2.11. If you have your spark data in form of Dataset or dataframe , you can save it using different modes of save method :

For example , (Java Code)

MongoSpark.write(yourDataSet).mode("append").option("replaceDocument","false").save();

It will append new data in your existing document matching it with _id on mongo collection.

answered Jul 11, 2018 at 11:16

j '

2111 gold badge3 silver badges15 bronze badges

Collectives™ on Stack Overflow

how to insert data to existing collection in mongodb with mongodb-spark connector

2 Answers 2

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related