Kafka Producer Not Able to connect via Code

Question

I'm unable to connect to my locally running Kafka instance via code - I can connect successfully using Kafka-Console-Producer and Kafka-Console-Consumer but when I use Kafka Java SDK and simply use the Java Producer to connect to & produce any message, it fails with the following error:

[Producer clientId=producer-1] Connection to node -1 (localhost/127.0.0.1:9092) could not be established. Broker may not be available.
[kafka-producer-network-thread | producer-1] WARN org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Bootstrap broker localhost:9092 (id: -1 rack: null) disconnected
[kafka-producer-network-thread | producer-1] WARN org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Connection to node -1 (localhost/127.0.0.1:9092) could not be established. Broker may not be available.
[kafka-producer-network-thread | producer-1] WARN org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Bootstrap broker localhost:9092 (id: -1 rack: null) disconnected
[kafka-producer-network-thread | producer-1] WARN org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Connection to node -1 (localhost/127.0.0.1:9092) could not be established. Broker may not be available.
[kafka-producer-network-thread | producer-1] WARN org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Bootstrap broker localhost:9092 (id: -1 rack: null) disconnected

Not sure if that matters but I'm using WSL2 on Windows and Kafka 2.12-2.5.0 to test this. How can I enable more verbose logging and debug this ?

Snippet of my Producer code

Properties properties = new Properties();
properties.setProperty(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG, "localhost:9092");
properties.setProperty(ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG, StringSerializer.class.getName());
properties.setProperty(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG, StringSerializer.class.getName());

ProducerRecord<String, String> record = new ProducerRecord<String, String>("first_topic","first_value");

KafkaProducer<String, String> kafkaProducer = new KafkaProducer<String, String>(properties);
kafkaProducer.send(record);

i am trying to do the same thing, connect from a kafka client in windows to a kafka instance running in WSL2. I get a similar error. But I do not have the option of running my client in wsl2 since it is a windows app. Does anyone know how to fix / configure this scenario so a windows kafka client can connect to a wsl2 kafka server? — Ed Ost
– Ed Ost, Commented Oct 31, 2020 at 19:41
I had the same problem. I could only fix it by going back to wsl1, instead of wsl2. Seems like wsl2 is not able to connect to windows localhost. — Augusto Acioli Pinho Vanderley
– Augusto Acioli Pinho Vanderley, Commented Nov 13, 2020 at 6:20

AndreyCh · Accepted Answer · 2023-10-05 13:01:14Z

3

I had the same issue when I tried to connect to the Kafka Broker running under WSL2, from the IntelliJ Java application running on Windows.

In such a case, the solution is to disable IPv6 in WSL2 for this to work properly. Per official Kafka's installation tutorial for WSL2 on Windows guidance:

WSL2 currently has a networking issue that prevents outside programs to connect to Kafka running on WSL2 (for example your Java programs, Conduktor, etc...); To fix this, we recommend disabling IPv6 on WSL2. Your Windows password will be prompted on the first command:

sudo sysctl -w net.ipv6.conf.all.disable_ipv6=1
sudo sysctl -w net.ipv6.conf.default.disable_ipv6=1

https://www.conduktor.io/kafka/how-to-install-apache-kafka-on-windows/

answered Oct 5, 2023 at 13:01

AndreyCh

1591 silver badge5 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

MSingh Over a year ago

Disabling IPV6 Worked!!! Had exactly the same issue. installed Confluent Kafka in Windows using WSL2 and i was unable to Connect with the Spring boot consumer.

Agraj · Accepted Answer · 2020-05-25 08:27:09Z

2

Sorry for the false alarm. My issue was somewhat linked to WSL2 - I was starting my ZK and Kafka Broker in WSL2 in the linux distro but was mistakenly trying to access the broker from a Java App running in Windows !

When I try to run that Java app from Linux itself. It works !

answered May 25, 2020 at 8:27

Agraj

4745 silver badges20 bronze badges

Collectives™ on Stack Overflow

Kafka Producer Not Able to connect via Code

2 Answers 2

1 Comment

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

1 Comment

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related