How to write data into CouchBase using Spark & Scala?

Question

I am newbie in CouchBase.I am trying to write data into CouchBase in local mode.My sample code is following,

 val cfg = new SparkConf()
.setAppName("couchbaseQuickstart")
.setMaster("local[*]")
.set("com.couchbase.bucket.MyBucket","pwd") 

    val sc = new SparkContext(cfg)
    val doc1 = JsonDocument.create("doc1", JsonObject.create().put("some","content"))
    val doc2 = JsonArrayDocument.create("doc2", JsonArray.from("more", "content", "in", "here"))
    val data = sc.parallelize(Seq(doc1, doc2))

But I can't access data.saveToCouchbase().

I am using Spark 1.6.1 & Scala 2.11.8

I gave following dependencies in built.sbt

libraryDependencies += "org.apache.spark" % "spark-core_2.11" % "1.6.1"
libraryDependencies += "com.couchbase.client" % "spark-connector_2.11" % "1.2.1"

How can I write data into CouchBase using Spark & Scala?

jstuartmill · Accepted Answer · 2016-10-05 04:22:04Z

1

Looks like you are just missing an import statement that'll enable you to use Couchbase functions on RDDs and dataframes:

import com.couchbase.spark._

val cfg = new SparkConf()
.setAppName("couchbaseQuickstart")
.setMaster("local[*]")
.set("com.couchbase.bucket.MyBucket","pwd") 

val sc = new SparkContext(cfg)
val doc1 = JsonDocument.create("doc1", 

JsonObject.create().put("some","content"))
val doc2 = JsonArrayDocument.create("doc2", JsonArray.from("more", "content", "in", "here"))

val data = sc.parallelize(Seq(doc1, doc2))

data.saveToCouchbase()

answered Oct 5, 2016 at 4:22

jstuartmill

2885 silver badges11 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

How to write data into CouchBase using Spark & Scala?

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related