Elasticsearch does not persist data, docker-compose

Question

I have this configuration, I'd like the data to be persistent among reboots but right no it is not happening, everything works fine but when I bring the ELK down and up it starts empty and all the dashboards are also lost.

What's wrong?

    version: '3.3'
services:
  logstash:
    container_name: logstash
    image: docker.elastic.co/logstash/logstash:6.7.0
    command: bash -c 'bin/logstash -e "input { http { port => 5044 } } filter { split {} csv { separator => \",\" columns => [\"Job\", \"BuildId\", \"Start\", \"Start_date\", \"Start_time\", \"Stop_time\", \"Stop_date\", \"Stop_time\", \"Execution_time\", \"Queue_time\", \"Executor\", \"Result\", \"Parent\", \"ChangeId\", \"Repo\", \"User\"] convert => { \"Start_time\" => \"date\" \"Stop_time\" => \"date\" } } mutate { convert => { \"Execution_time\" => \"float\" \"Queue_time\" => \"float\"}} date { match => [ \"Start time\", \"yyyy-MM-dd HH:mm:ss\" ] } } output { elasticsearch { hosts => [\"elasticsearch:9200\"] index => \"job-executions\" } }"'
    networks:
      - elastic-net
    ports:
      - 5044:5044

  elasticsearch:
    container_name: elasticsearch
    image: docker.elastic.co/elasticsearch/elasticsearch:6.7.0
    environment:
      - cluster.name=docker-cluster
    ulimits:
      memlock:
        soft: -1
        hard: -1
    volumes:
      - esdata01:/root/elasticKibana/data
    networks:
      - elastic-net
    ports:
      - "9200:9200"
  kibana:
    container_name: kibana
    image: docker.elastic.co/kibana/kibana:6.7.0
    networks:
      - elastic-net
        driver: bridge
    volumes:
      esdata01:
        driver: local
        driver_opts:
          type: 'none'
          o: 'bind'
          device: '/root/elasticKibana/data'

Thanks in advance

Briefly reading the fine manual, /root/elasticKibana/data does not look like the correct path to mount a volume in the container to persist data. Shouldn't it be /usr/share/elasticsearch/data ? Moreover, it seems your docker-compose file has indentation issues. — Zeitounator
– Zeitounator, Commented Aug 19, 2020 at 15:20
In my machine that path did not exist, I have created it and modified the configuration. So... this means that the data is only persisted to /usr/share/elasticsearch/data? I mean, you cannot decide where to store it? — RaistlinMolina
– RaistlinMolina, Commented Aug 20, 2020 at 11:05
You store the data wherever you want on your machine. But it usually has to be mounted to a specific path in the container unless you customize and change this, which is not the case as far as I can tell from the example you show. — Zeitounator
– Zeitounator, Commented Aug 20, 2020 at 11:14
Aha, and what do my configuration lacks? because right now no data is persisted between container reboots. — RaistlinMolina
– RaistlinMolina, Commented Aug 20, 2020 at 12:19
As is, your configuration is just mounting the data volume on the wrong path inside the container, as reported in my very first comment, containing a link to the documentation that you should probably read in depth. — Zeitounator
– Zeitounator, Commented Aug 20, 2020 at 13:23

Pankaj Saini · Accepted Answer · 2020-08-19 17:27:50Z

2

For Elastic search i think the container PATH you need to mount is "/usr/share/elasticsearch/data" rather "/root/elasticKibana/data". At least that's the case in my helm chart and docker-compose should not be different as I am using same Image.

answered Aug 19, 2020 at 17:27

Pankaj Saini

1,65810 silver badges13 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

RaistlinMolina Over a year ago

In my machine that path did not exist, I have created it and modified the configuration. So... this means that the data is only persisted to /usr/share/elasticsearch/data? I mean, you cannot decide where to store it?

Pankaj Saini Over a year ago

Hello, you can decide on the target to anything on your host/machine. What I am referring to is the source that is defined in the image/container. You can't change that.

RaistlinMolina Over a year ago

Aha, and what do my configuration lacks? because right now no data is persisted between container reboots.

Pankaj Saini Over a year ago

Change line 22 i.e. "- esdata01:/root/elasticKibana/data" to "- esdata01:/usr/share/elasticsearch/data". Also bear in mind at the moment you are persisting data only for Elasticsearch container and not for Kibana as per your code.

Collectives™ on Stack Overflow

Elasticsearch does not persist data, docker-compose

1 Answer 1

4 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

4 Comments

Your Answer

Sign up or log in

Post as a guest

Related