Syslog driver not working with docker compose and elk stack

Question

I want to send logs from one container running my_service to another running the ELK stack with the syslog driver (so I will need the logstash-input-syslog plugin installed).

I am tweaking this elk image (and tagging it as elk-custom) via the following Dockerfile-elk

(using port 514 because this seems to be the default port)

FROM sebp/elk

WORKDIR /opt/logstash/bin

RUN ./logstash-plugin install logstash-input-syslog

EXPOSE 514

Running my services via a docker-compose as follows more or less:

 elk-custom:
    # image: elk-custom
    build:
      context: .
      dockerfile: Dockerfile-elk
    ports:
      - 5601:5601
      - 9200:9200
      - 5044:5044
      - 514:514

  my_service:
    image: some_image_from_my_local_registry
    depends_on:
      - elk-custom
    logging:
     driver: syslog
     options:
       syslog-address: "tcp://elk-custom:514"

However:

ERROR: for b4cd17dc1142_namespace_my_service_1 Cannot start service my_service: failed to initialize logging driver: dial tcp: lookup elk-custom on 10.14.1.31:53: server misbehaving

ERROR: for api Cannot start service my_service: failed to initialize logging driver: dial tcp: lookup elk-custom on 10.14.1.31:53: server misbehaving ERROR: Encountered errors while bringing up the project.

Any suggestions?

UPDATE: Apparently nothing seems to be listening on port 514, cause from within the container, the command netstat -a shows nothing on this port....no idea why...

Try changing tcp://elk-custom:514 to tcp://127.0.0.1:514 and see if that helps — Tarun Lalwani
– Tarun Lalwani, Commented Sep 25, 2017 at 14:04
yep. that seems to do the job ... my service stack now seems to start... proceeding with further logstash configuration — pkaramol
– pkaramol, Commented Sep 25, 2017 at 14:15

Tarun Lalwani · Accepted Answer · 2017-09-25 14:21:22Z

5

You need to use tcp://127.0.0.1:514 instead of tcp://elk-custom:514. Reason being this address is being used by docker and not by the container. That is why elk-custom is not reachable.

So this will only work when you map the port (which you have done) and the elk-service is started first (which you have done) and the IP is reachable from the docker host, for which you would use tcp://127.0.0.1:514

answered Sep 25, 2017 at 14:21

Tarun Lalwani

147k11 gold badges217 silver badges278 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

pkaramol Over a year ago

So the actual log forwarding will be done by the docker daemon, and he is not aware of the hostnames, whilst each container / service defined within docker-compose is aware of each other's host name? Is this the case?

pkaramol Over a year ago

Any suggestion how this works on rancher? Neither of 127.0.0.1 or elk-custom (i.e. the service name) work, as they cause the same issue: dial tcp: lookup elk-custom on 10.14.1.31:53: server misbehaving

Tarun Lalwani Over a year ago

I think you might have to use rancher.com/docs/rancher/latest/en/cattle/internal-dns-service. <Service>.<stackname> or something like that. Check the article

Collectives™ on Stack Overflow

Syslog driver not working with docker compose and elk stack

1 Answer 1

3 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

3 Comments

Your Answer

Sign up or log in

Post as a guest

Related