Unable to deserialize Kafka stream to pojo. Could not find class specified in writer's schema

Question

Getting exception while reading from kafka topic: Caused by: org.apache.kafka.common.errors.SerializationException: Error deserializing Avro message for id 1 Caused by: org.apache.kafka.common.errors.SerializationException: Could not find class USERS specified in writer's schema whilst finding reader's schema for a SpecificRecord.

I think deserialising is not correct and i am not able to find any proper example also.

    Properties props = new Properties();

    props.put(StreamsConfig.APPLICATION_ID_CONFIG , "TEST");
    props.put(StreamsConfig.BOOTSTRAP_SERVERS_CONFIG,"localhost:9092");

    props.put(AbstractKafkaAvroSerDeConfig.SCHEMA_REGISTRY_URL_CONFIG, "http://localhost:8081");
            props.put(StreamsConfig.DEFAULT_KEY_SERDE_CLASS_CONFIG,Serdes.String().getClass().getName());
                    props.put(StreamsConfig.DEFAULT_VALUE_SERDE_CLASS_CONFIG,SpecificAvroSerde.class);

                    props.put(ConsumerConfig.AUTO_OFFSET_RESET_CONFIG, "earliest");
                    props.put(KafkaAvroDeserializerConfig.SPECIFIC_AVRO_READER_CONFIG, true);
    StreamsBuilder builder = new StreamsBuilder();




    KStream<String,USERS> valid = builder.stream("testUSERS");

valid.foreach((k,v)-> System.out.println("v ="+v.getUSERNAME()));
valid.foreach((k,v)-> System.out.println("k ="+k));

KafkaStreams streams = new KafkaStreams(builder.build(),props);

streams.cleanUp();

streams.start();

Pedro Bacchini · Accepted Answer · 2021-08-19 12:38:59Z

6

You are using kafka's configuration to require a specific schema in this case the consumed schema must exist in your class path so that it is possible to parse it, if not, configure to use GenericRecord

props.put(KafkaAvroDeserializerConfig.SPECIFIC_AVRO_READER_CONFIG, false);

answered Aug 19, 2021 at 12:38

Pedro Bacchini

93510 silver badges13 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

LunaticJape Over a year ago

I face the same issue, what's the solution if we use a remote schema registry?

Pedro Bacchini Over a year ago

same solution being remote does not change the way lib will handle the serialization of avros.

Collectives™ on Stack Overflow

Unable to deserialize Kafka stream to pojo. Could not find class specified in writer's schema

1 Answer 1

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related