Starting a multi-node cluster with Docker Compose - where do you configure host actual names?

Question

I'm new to Docker and I'm trying to run an Elasticsearch cluster with docker. I have 3 machines with IP addresses ip1,ip2,ip3 and I want to run the cluster using them.

Inside Elasticsearch site, they provide the following docker compose file:

version: '2.2'
services:
  es01:
    image: docker.elastic.co/elasticsearch/elasticsearch:7.12.1
    container_name: es01
    environment:
      - node.name=es01
      - cluster.name=es-docker-cluster
      - discovery.seed_hosts=es02,es03
      - cluster.initial_master_nodes=es01,es02,es03
      - bootstrap.memory_lock=true
      - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
    ulimits:
      memlock:
        soft: -1
        hard: -1
    volumes:
      - data01:/usr/share/elasticsearch/data
    ports:
      - 9200:9200
    networks:
      - elastic
  es02:
    image: docker.elastic.co/elasticsearch/elasticsearch:7.12.1
    container_name: es02
    environment:
      - node.name=es02
      - cluster.name=es-docker-cluster
      - discovery.seed_hosts=es01,es03
      - cluster.initial_master_nodes=es01,es02,es03
      - bootstrap.memory_lock=true
      - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
    ulimits:
      memlock:
        soft: -1
        hard: -1
    volumes:
      - data02:/usr/share/elasticsearch/data
    networks:
      - elastic
  es03:
    image: docker.elastic.co/elasticsearch/elasticsearch:7.12.1
    container_name: es03
    environment:
      - node.name=es03
      - cluster.name=es-docker-cluster
      - discovery.seed_hosts=es01,es02
      - cluster.initial_master_nodes=es01,es02,es03
      - bootstrap.memory_lock=true
      - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
    ulimits:
      memlock:
        soft: -1
        hard: -1
    volumes:
      - data03:/usr/share/elasticsearch/data
    networks:
      - elastic

volumes:
  data01:
    driver: local
  data02:
    driver: local
  data03:
    driver: local

networks:
  elastic:
    driver: bridge

so how do I configure the 3 machines I have in this docker-compose file? under node.name? like a full URL of the machine? the IP address? something else?

Do I need to run 'docker-compose up' in all of the machines? The machines reside in the same local network, and all have docker daemon running.

You could run docker-compose up with what you already have and check which are the container names. Since they are in the same network, they will be able to talk to each other. — Marko E
– Marko E, Commented Apr 29, 2021 at 16:08

Michal Rosenbaum · Accepted Answer · 2021-04-29 16:22:00Z

2

Do I need to run 'docker-compose up' in all of the machines? The machines reside in the same local network, and all have docker daemon running.

All these elasticsearch nodes will be just containers on the same Docker host, so if you do that you'll have a total of 9 containers on these 3 machines.

To spread these three containers across multiple machines you need to set up your machines into Docker Swarm and prepare docker-compose file, which is eligible for docker stack command. Then you deploy the docker-compose file from the manager.

https://docs.docker.com/engine/swarm/swarm-tutorial/

edited Apr 29, 2021 at 16:22

answered Apr 29, 2021 at 16:13

Michal Rosenbaum

2,1011 gold badge15 silver badges22 bronze badges

Sign up to request clarification or add additional context in comments.

7 Comments

nadavgam Over a year ago

Micahl - i don't have 'docker-machine' installed on my machines, it it mandatory?

Michal Rosenbaum Over a year ago

@nadavgam it's not mandatory. Follow these instructions to initialize the docker swarm on the first machine: docs.docker.com/engine/swarm/swarm-tutorial/create-swarm and then join others to the swarm by using the command which will be returned from docker swarm init this step is described here docs.docker.com/engine/swarm/swarm-tutorial/add-nodes

nadavgam Over a year ago

thats what i did, and 'docker node ls' shows the two nodes, but i dont think anything is running. 'docker container ps' shows nothing...

nadavgam Over a year ago

docker stack services vossibility ID NAME MODE REPLICAS IMAGE PORTS rfcppcod0xmv vossibility_es02 replicated 0/1 ger-registry.caas.mycom.com/hpc-middleware/docker.elastic.co/elasticsearch/elasticsearch:latest stzsspyyore1 vossibility_es01 replicated 0/1 ger-registry.caas.mycom.com/hpc/docker.elastic.co/elasticsearch/elasticsearch:latest *:9200->9200/tcp

Michal Rosenbaum Over a year ago

@nadavgam replicated 0/1 suggests that the docker stack command worked, but containers struggle to start, maybe they are exiting early due to some error. Could you try docker ps -a?

|

Collectives™ on Stack Overflow

Starting a multi-node cluster with Docker Compose - where do you configure host actual names?

1 Answer 1

7 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

7 Comments

Your Answer

Sign up or log in

Post as a guest

Related