Spring boot app cannot connect with Kafka within the container

In short To repeat the issue, you may clone the project from: https://github.com/ethanalef/event-driven-model Then build both listener and publisher projects with maven, build the image with Dockerfile insider each project then run docker-compose.yml. You may find the failed logs in listener project.

Background I have a simple spring boot app running with rabbitMQ successfully within a container. Then I add Kafka setting in the docker-compose.yml; and application.properties for the app such that the app will connect to both rabbitMQ and Kafka within the same container.

However, the app cannot connected to Kafka:

org.apache.kafka.clients.NetworkClient   : [Consumer clientId=consumer-myId-1, groupId=myId] Connection to node -1 (localhost/127.0.0.1:9092) could not be established. Node may not be available.

While the Kafka log shows:

2024-12-16 17:38:08 [2024-12-16 09:38:08,360] INFO [Controller id=1, targetBrokerId=1] Node 1 disconnected. (org.apache.kafka.clients.NetworkClient)
2024-12-16 17:38:08 [2024-12-16 09:38:08,361] WARN [Controller id=1, targetBrokerId=1] Connection to node 1 (kafka/172.18.0.4:9092) could not be established. Node may not be available. (org.apache.kafka.clients.NetworkClient)
2024-12-16 17:38:08 [2024-12-16 09:38:08,362] WARN [RequestSendThread controllerId=1] Controller 1's connection to broker kafka:9092 (id: 1 rack: null) was unsuccessful (kafka.controller.RequestSendThread)
2024-12-16 17:38:08 java.io.IOException: Connection to kafka:9092 (id: 1 rack: null) failed.
2024-12-16 17:38:08     at org.apache.kafka.clients.NetworkClientUtils.awaitReady(NetworkClientUtils.java:71)
2024-12-16 17:38:08     at kafka.controller.RequestSendThread.brokerReady(ControllerChannelManager.scala:299)
2024-12-16 17:38:08     at kafka.controller.RequestSendThread.doWork(ControllerChannelManager.scala:252)
2024-12-16 17:38:08     at org.apache.kafka.server.util.ShutdownableThread.run(ShutdownableThread.java:135)

application.properties:

spring.kafka.bootstrap-servers=localhost:9092
spring.kafka.consumer.group-id=orderGroup
spring.kafka.consumer.auto-offset-reset=earliest

Dockerfile:

FROM openjdk:21-jdk-slim
VOLUME /tmp
COPY target/listener-0.0.1-SNAPSHOT.jar app.jar
ENTRYPOINT ["java","-jar","/app.jar"]

docker-compose.yml:

version: '3'
services:
  rabbitmq:
    image: rabbitmq:3-management
    ports:
      - "5672:5672"
      - "15672:15672"
    healthcheck:
      test: ["CMD", "rabbitmq-diagnostics", "check_port_connectivity"]
      interval: 5s
      timeout: 5s
      retries: 5

  zookeeper:
    image: confluentinc/cp-zookeeper:latest
    environment:
      ZOOKEEPER_CLIENT_PORT: 2181
      ZOOKEEPER_TICK_TIME: 2000
    healthcheck:
      test: [ "CMD", "echo", "ruok", "|", "nc", "localhost", "2181", "|", "grep", "imok" ]
      interval: 10s
      timeout: 5s
      retries: 5

  kafka:
    image: confluentinc/cp-kafka:latest
    depends_on:
      - zookeeper
    ports:
      - "9092:9092"
    environment:
      KAFKA_BROKER_ID: 1
      KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
      KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka:9092
      KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT
      KAFKA_INTER_BROKER_LISTENER_NAME: PLAINTEXT
      KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
    healthcheck:
      test: [ "CMD", "kafka-broker-api-versions", "--bootstrap-server", "localhost:9092" ]
      interval: 10s
      timeout: 5s
      retries: 5

  listener:
    build: ./listener
    ports:
      - "8081:8081"
    depends_on:
      rabbitmq:
        condition: service_healthy
      kafka:
        condition: service_healthy
    restart: unless-stopped

I cannot identify what is wrong so please help.

I try to revise application.properties and docker-compose.yml with AI like copilot, cursor etc and I expect it should work like rabbitMQ. But no hints at the moment.