java.lang.AbstractMethodError in Java Spark and cassandra connection

Question

working with spark1.6.0 and cassandra-3.1.1 and I tried to connect to cassandra database using Java spark. there is no error while building but getting the following error while i run the application

vException in thread "main" java.lang.AbstractMethodError
at org.apache.spark.Logging$class.log(Logging.scala:51)
at com.datastax.spark.connector.cql.CassandraConnector$.log(CassandraConnector.scala:144)
at org.apache.spark.Logging$class.logDebug(Logging.scala:62)
at com.datastax.spark.connector.cql.CassandraConnector$.logDebug(CassandraConnector.scala:144)
at com.datastax.spark.connector.cql.CassandraConnector$.com$datastax$spark$connector$cql$CassandraConnector$$createSession(CassandraConnector.scala:154)
at com.datastax.spark.connector.cql.CassandraConnector$$anonfun$4.apply(CassandraConnector.scala:151)
at com.datastax.spark.connector.cql.CassandraConnector$$anonfun$4.apply(CassandraConnector.scala:151)
at com.datastax.spark.connector.cql.RefCountedCache.createNewValueAndKeys(RefCountedCache.scala:36)
at com.datastax.spark.connector.cql.RefCountedCache.acquire(RefCountedCache.scala:61)
at com.datastax.spark.connector.cql.CassandraConnector.openSession(CassandraConnector.scala:72)
at com.test.cassandra.spark.Main.generateData(Main.java:30)
at com.test.cassandra.spark.Main.run(Main.java:21)
at com.test.cassandra.spark.Main.main(Main.java:163)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:731)
at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:181)
at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:206)
at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:121)
at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)

my code

import com.datastax.driver.core.Session;
import com.datastax.spark.connector.cql.CassandraConnector;
import org.apache.spark.SparkConf;
import org.apache.spark.api.java.JavaSparkContext;
import java.io.Serializable;

public class Main implements Serializable {
    private transient SparkConf sconf;
    private static final String keySpaceName = "java_api";
    private static final String primaryTableName = "test_cassandra";

    private Main(SparkConf conf) {
        this.sconf = conf;
    }
    private void run() {
      JavaSparkContext sc = new JavaSparkContext(sconf);
    generateData(sc); 
    sc.stop();
    }
    private void generateData(JavaSparkContext sc) {

        CassandraConnector connector = CassandraConnector.apply(sc.getConf());

        try (Session session = connector.openSession()) {
            System.out.println("connected to cassandra");
        session.execute("DROP KEYSPACE IF EXISTS java_api");
        session.execute("CREATE KEYSPACE java_api WITH replication = {'class': 'SimpleStrategy', 'replication_factor': 1}");            
        session.execute("CREATE TABLE java_api.sales (id UUID PRIMARY KEY, product INT, price DECIMAL)");
        session.execute("CREATE TABLE java_api.summaries (product INT PRIMARY KEY, summary DECIMAL)");
        System.out.println("connected");
        }
    }
public static void main(String[] args) {
        if (args.length != 2) {
            System.err
                    .println("Syntax: com.datastax.spark.demo.Main <Spark Master URL> <Cassandra contact point>");
            System.exit(1);
        }
        SparkConf conf = new SparkConf()
                .set("spark.cassandra.connection.host", "localhost")
                .set("spark.cassandra.connection.native.port", "9042");
        conf.setAppName("Java API demo");
        conf.setMaster(args[0]);
        //conf.set("spark.cassandra.connection.host", "127.0.0.1");
        Main app = new Main(conf);
        app.run();
}
}

my pom.xml

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <groupId>com.test</groupId>
    <artifactId>cassandra-spark</artifactId>
    <version>1.0</version>
    <packaging>jar</packaging>
    <properties>
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
        <maven.compiler.source>1.7</maven.compiler.source>
        <maven.compiler.target>1.7</maven.compiler.target>
    </properties>
    <dependencies>
        <dependency>
            <groupId>junit</groupId>
            <artifactId>junit</artifactId>
            <version>3.8.1</version>
            <scope>test</scope>
        </dependency>
        <!--Spark Cassandra Connector -->
        <dependency>
            <groupId>com.datastax.spark</groupId>
            <artifactId>spark-cassandra-connector_2.10</artifactId>
            <version>1.5.0-M3</version>
        </dependency>
        <dependency>
            <groupId>com.datastax.spark</groupId>
            <artifactId>spark-cassandra-connector-java_2.10</artifactId>
            <version>1.5.0-M3</version>
        </dependency>
        <dependency>
            <groupId>com.datastax.cassandra</groupId>
            <artifactId>cassandra-driver-core</artifactId>
            <version>3.0.0-rc1</version>
        </dependency>

        <dependency>
            <groupId>org.apache.spark</groupId>
            <artifactId>spark-core_2.10</artifactId>
            <version>1.6.0</version>
        </dependency>
        <dependency>
            <groupId>org.apache.spark</groupId>
            <artifactId>spark-streaming_2.10</artifactId>
            <version>1.6.0</version>
        </dependency>
    </dependencies>
</project>

Casandra Connector for Spark 1.6 is still not released. Would suggest to try with Spark 1.5 or below. see here for Compatibility Matrix. — Sumit
– Sumit, Commented Feb 1, 2016 at 11:19

Community · Accepted Answer · 2017-05-23 12:01:46Z

1

This may come from the fact that

some class has incompatibly changed since the currently executing method was last compiled.

This may come from the java version for example

See the response to this question: Spark streaming StreamingContext.start() - Error starting receiver 0

edited May 23, 2017 at 12:01

CommunityBot

11 silver badge

answered Nov 29, 2016 at 12:38

Pop

12.5k5 gold badges57 silver badges71 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Priyanshu Singh · Accepted Answer · 2018-12-18 05:14:08Z

1

Seems this issue is because of conflict in logging of spark and Cassandra.I was getting this error while using below dependency.

libraryDependencies += "com.datastax.spark" %% "spark-cassandra-connector" % "1.6.2"

I used below Cassandra connector to resolve this issue.. libraryDependencies += "com.datastax.spark" %% "spark-cassandra-connector" % "1.6.5"

answered Dec 18, 2018 at 5:14

Priyanshu Singh

7248 silver badges12 bronze badges

Collectives™ on Stack Overflow

java.lang.AbstractMethodError in Java Spark and cassandra connection

2 Answers 2

Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related