Failed to create topics","exception":"\norg.apache.kafka.common.errors.UnsupportedVersionException

Question

{
   "mdc":{
      
   },
   "timestamp":"2021-05-11 11:48:04.055",
   "level":"ERROR",
   "logger":"org.springframework.cloud.stream.binder.kafka.provisioning.KafkaTopicProvisioner",
   "message":"Failed to create topics",
   "exception":"\"\norg.apache.kafka.common.errors.UnsupportedVersionException: Creating topics with default partitions/replication factor are only supported in CreateTopicRequest version 4+. The following topics need values for partitions and replicas:"

Please suggest what changes are required as i am getting this error.

Gary Russell · Accepted Answer · 2021-05-13 13:12:38Z

5

I see you are new here. You should always include version information and full stack trace for questions like this.

Upgrade your broker to >= 2.4 or set the binder replication factor property.

See https://github.com/spring-cloud/spring-cloud-stream-binder-kafka/commit/4161f875ede0446ab1d485730c51e6a2c5baa37a

Change default replication factor to -1

Binder now uses a default value of -1 for replication factor signaling the broker to use defaults. Users who are on Kafka brokers older than 2.4, need to set this to the previous default value of 1 used in the binder.

In either case, if there is an admin policy that requires replication factor > 1, then that value must be used instead.

answered May 13, 2021 at 13:12

Gary Russell

176k14 gold badges168 silver badges194 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

shagun Over a year ago

Using Kafka Version <kafka.version>2.6.0</kafka.version> but still getting this issue

Gary Russell Over a year ago

That's the client version; the broker needs to be 2.4 or later; set the binder property to the replication factor you want.

shagun Over a year ago

Thanks Gary, added binder property to the replication factor "1" it works... But previously we are using spring-cloud-stream-binder-kafka.version "3.0.7" at that time issue didn't occurs , currently we are using "3.1.2" and issue occurs. Is it because of replication factor changes from "1" to "-1” ? So, we need to explicitly add the replication factor property.

Gary Russell Over a year ago

Yes, it was a new feature added to Kafka Brokers at 2.4; users complained that SCSt overrode that behavior and forced them to configure it in both places. In order to satisfy those users, SCSt 3.1 changed its default to -1 so the broker configuration would win. As the documentation stated, this meant that users with old brokers would have to change it to the old default. Price of progress.

Roee Gavirel · Accepted Answer · 2021-09-19 11:23:33Z

4

Overriding the default replication factor(-1) with a non negative value fixed the issue for me.

spring.cloud.stream.kafka.binder.replication-factor=1

For application.yaml file use:

spring.cloud.stream.kafka.binder.replicationFactor: 1

edited Sep 19, 2021 at 11:23

Roee Gavirel

19.6k13 gold badges69 silver badges98 bronze badges

answered Jun 24, 2021 at 20:15

javachipsy

5165 silver badges6 bronze badges

Comments

Tanmay Patil · Accepted Answer · 2023-02-11 15:08:13Z

0

Change the code to following . The fluent api allows to give partitions and replicas.

@Bean
    NewTopic topicBytes() {
        return TopicBuilder.name("reflectoring-bytes").partitions(1).replicas(1).build();
    }

answered Feb 11, 2023 at 15:08

Tanmay Patil

6991 gold badge5 silver badges21 bronze badges

Comments

rajkumar21 · Accepted Answer · 2023-04-20 16:20:17Z

0

I had the same issue. And the following link helped me to solve it.

https://github.com/spring-cloud/spring-cloud-stream-binder-kafka/issues/1134

The property path for cloud kafka stream is different. Instead of spring.cloud.stream.kafka.binder we have to use spring.cloud.stream.kafka.streams.binder

answered Apr 20, 2023 at 16:20

rajkumar21

2534 silver badges9 bronze badges

Collectives™ on Stack Overflow

Failed to create topics","exception":"\norg.apache.kafka.common.errors.UnsupportedVersionException

4 Answers 4

4 Comments

Comments

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

4 Answers 4

4 Comments

Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related