Not able to create kafka topic using docker-compose

Question

I am trying to set up Kafka rest proxy using a docker. But the topic I am providing in the configuration isn't creating.

I am checking Kafka Topic with API: curl "http://metrics-kafka-rest:38082/topics" and I am getting this response: ["__confluent.support.metrics","_schemas"]

Below in the config that I have used in docker-compose:

    image: confluentinc/cp-zookeeper:5.3.0
    container_name: 'metrics-zookeeper'
    restart: always
    ports:
      - "32181:32181"
    environment:
      ZOOKEEPER_CLIENT_PORT: 32181
      ZOOKEEPER_TICK_TIME: 2000
      ZOOKEEPER_SYNC_LIMIT: 2
      ZOOKEEPER_SASL_ENABLED: "FALSE"

  metrics-kafka:
    image: confluentinc/cp-kafka:5.3.0
    container_name: 'metrics-kafka'
    restart: always
    ports:
      - "29092:29092"
    depends_on:
      - metrics-zookeeper
    environment:
      KAFKA_BROKER_ID: 1
      KAFKA_AUTO_CREATE_TOPICS_ENABLE: "true"
      KAFKA_ZOOKEEPER_CONNECT: metrics-zookeeper:32181
      KAFKA_INTER_BROKER_LISTENER_NAME: PLAINTEXT
      KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT
      KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://metrics-kafka:29092
      KAFKA_CREATE_TOPICS: "Notification:1:1"
      KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
      CONFLUENT_METRICS_ENABLE: 'false'

  metrics-schema-registry:
    image: confluentinc/cp-schema-registry:5.3.0
    container_name: 'metrics-schema-registry'
    restart: always
    ports:
      - "38081:38081"
    depends_on:
      - metrics-kafka
    environment:
      SCHEMA_REGISTRY_KAFKASTORE_CONNECTION_URL: metrics-zookeeper:32181
      SCHEMA_REGISTRY_HOST_NAME: metrics-schema-registry
      SCHEMA_REGISTRY_LISTENERS: "http://metrics-schema-registry:38081"
      SCHEMA_REGISTRY_DEBUG: "true"

  metrics-kafka-rest:
    image: confluentinc/cp-kafka-rest:5.3.0
    container_name: 'metrics-kafka-rest'
    restart: always
    ports:
      - "38082:38082"
    depends_on:
      - metrics-schema-registry
    environment:
      KAFKA_REST_ZOOKEEPER_CONNECT: metrics-zookeeper:32181
      KAFKA_REST_SCHEMA_REGISTRY_URL: "http://metrics-schema-registry:38081"
      KAFKA_REST_HOST_NAME: metrics-kafka-rest
      KAFKA_REST_LISTENERS: "http://metrics-kafka-rest:38082"
      KAFKA_REST_DEBUG: "true"

I expect that when I hit API of getting the list of topics then it should contain topic Notification.

Robin Moffatt · Accepted Answer · 2019-08-13 12:38:46Z

17

KAFKA_CREATE_TOPICS is not a supported Environment variable for the cp-kafka image that you're using.

Since you already have KAFKA_AUTO_CREATE_TOPICS_ENABLE: "true" then you can just start using the broker and topics will be created as and when they're first referenced by the producer or consumer.

answered Aug 13, 2019 at 12:38

Robin Moffatt

32.4k5 gold badges74 silver badges103 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

beatrice Over a year ago

So how should we create default topics with cp-kafka at container startup time?

João Zarate Over a year ago

But is it possible to define a name previously? [Edited] I think that it will answer my question environment: KAFKA_CREATE_TOPICS: "Topic1:1:3,Topic2:1:1:compact" See: registry.hub.docker.com/r/wurstmeister/kafka

琴罢倚松玩鹤 · Accepted Answer · 2022-08-11 01:43:59Z

5

Agree with @Robin, and give some config references because some guys mix the env variables KAFKA_CREATE_TOPICS for images wurstmeister/kafka: https://registry.hub.docker.com/r/wurstmeister/kafka and confluentinc/cp-kafka:https://docs.confluent.io/platform/current/installation/docker/config-reference.html#optional-confluent-enterprise-ak-settings This is to say KAFKA_CREATE_TOPICS is not a supported Environment variable for the cp-kafka, it's for wurstmeister/kafka

answered Aug 11, 2022 at 1:43

琴罢倚松玩鹤

892 silver badges6 bronze badges

Collectives™ on Stack Overflow

Not able to create kafka topic using docker-compose

2 Answers 2

2 Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

2 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related