Spark - SQL : value implicits is not a member of org.apache.spark.sql.SQLContext

Question

Hi please find below code and corresponding errors :Even though i have used import statements but still giving errors

import org.apache.spark.sql._

val sparkConf = new SparkConf().setAppName("new_proj")
implicit val sc = new SparkContext(sparkConf)

val sqlContext = new org.apache.spark.sql.SQLContext(sc)
import sqlContext._
import sqlContext.implicits._

val projects = sqlContext.read.json("/part-m-00000.json")

[error] /mapr/trans.scala:25: value implicits is not a member of org.apache.spark.sql.SQLContext [error] import sqlContext.implicits._ [error] ^ [error] /mapr/ppm_trans.scala:28: value read is not a member of org.apache.spark.sql.SQLContext [error] val projects = sqlContext.read.json("/mapr//part-m-00000.json")

MapReddy Usthili · Accepted Answer · 2015-12-02 09:58:45Z

2

I'm able to compile code by changing the following lines in build.sbt :

libraryDependencies ++= Seq(
  "org.apache.spark"  % "spark-core_2.10"              % "1.4.0" % "provided",
  "org.apache.spark"  % "spark-sql_2.10"               % "1.4.0",
  "org.apache.spark"  % "spark-sql_2.10"               % "1.4.0",
  "org.apache.spark"  % "spark-mllib_2.10"             % "1.4.0"
  )

answered Dec 2, 2015 at 9:58

MapReddy Usthili

2881 gold badge7 silver badges23 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Spark - SQL : value implicits is not a member of org.apache.spark.sql.SQLContext

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related