SBT dependency error for Scala Spark on Intellij

Question

I am a noob to Spark and Intellij.I want to run Spark using Scala I initially installed Scala 2.12 and created the SBT accordingly.Then I got a NoSuchMethod runtime error on

       sc =  new SparkContext(conf)

From the solution posted NoSuchMethodError when using Sparka and IntelliJ I used Scala version 2.11.3 while creating the project and used the SBT

version := "0.1"

scalaVersion := "2.11.3"

libraryDependencies += "org.apache.spark" % "spark-core_2.11" % "2.0.2"

I am now getting the error

   Error:scalac: No 'scala-library*.jar' in Scala compiler classpath in Scala SDK SBT: org.scala-lang:scala-library:2.11.3:jar

This is the library on the External libraries section

I tried creating the project from scratch and Cache Invalidate/Restart option.Same result

I also tried downloading via Maven through File -> Project Structure.Only found spark-core 2.10.Showed the same NoSuchMethod Error

Can anyone identify the problem?

can you see scala-compiler.jar, scala-library.jar, scala-reflect.jar libraries in Project Structure -> modules ? — koiralo
– koiralo, Commented Oct 11, 2017 at 12:16
Thanks @Shankar Koirala.I could not see them before but I just uninstalled scala plugin and restarted and reinstalled and it worked.It was just magic..:) — Solo
– Solo, Commented Oct 11, 2017 at 12:36

koiralo · Accepted Answer · 2017-10-11 12:44:17Z

1

It clearly says that, It could not find scala-library*.jar file

So go to

"Project Structure -> Modules"

And see these jar files

scala-compiler.jar, scala-library.jar, scala-reflect.jar

If they are absent from the Modules add them manually or Reinstall the scala and scala plugin.

Hope this should work!

answered Oct 11, 2017 at 12:44

koiralo

23.2k6 gold badges57 silver badges77 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

SBT dependency error for Scala Spark on Intellij

1 Answer 1

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related