1

I have a docker swarm cluster and am able to get all docker "container" logs to ELK stack.

But am unable to get docker daemon logs. Can someone please guide me to achieve this.

FYI : My stack is in Linux.

Improve this question
1
  • Did you find a solution? Even if I posted an answer, I'm also interested for a better solution. Commented Aug 2, 2018 at 8:06

2 Answers 2

Reset to default
1

You can use Filebeat plugin to send the logs from the daemon logs file to your ELK (plugin presentation page.

There is an article on this point on the elasic.co blog. Your configuration will be different since you don't want containers logs but Docker daemon logs found at the path /var/log/docker.log or /var/log/daemon.log.

EDIT 1:

Since in your environment, the logs are readable with journalctl, I digged up the internet and I have found an ELK plugin that allows you to send the logs from the journald: https://github.com/logstash-plugins/logstash-input-journald

I Hope it'll help.

Improve this answer
Sign up to request clarification or add additional context in comments.

3 Comments

You can refer to Sabin's post about Docker daemon logs file maybe
Am getting docker logs under journalctl but am unable to redirect them to ELK.
I updated my answer because I found something maybe interesting in your case.
1

1st: you'd need to find out where your docker daemon is saving the logs, which depends on linux distribution. See this response with a list of possible places:

https://stackoverflow.com/a/30970134/3165889

2nd: you can use the suggestion of Paul Rey and use Filebeat. As an alternative, I also suggest the use of Fluentd, which usually you can use in place of Logstash, then having EFK instead of ELK, or simply as an extra tool to your ELK environment.

It can also read from a file using the tail input plugin

It can also insert data to Elasticsearch using the elasticsearch out plugin

This tutorial teaches how to log containers, but then you'd need to change your input plugin to tail from that file: Docker logging via EFK

I'd also like to add that, if you're interested in logging the daemon, you probably want to log even if docker is failing to start. So I'd install Fluentd directly on the host. NOT in a container.

Improve this answer

3 Comments

i can get logs from journalctl but from there how can i continuously push logs to ELK?
This means that you probably have systemd logging your process. journalctl is used by systemd but saves logs to binary files. That's why you have to use it to read the logs. Although, If you're lucky and you have a newer version of systemd, you can setup systemd to save logs to a specific file, see: stackoverflow.com/a/48052152/3165889. After having it on a file, you can use simply the tail input plugin from fluentd
Another option, which I haven't tested, but looks like someone already wrote a plugin to integrate systemd journal and fluend: github.com/reevoo/fluent-plugin-systemd

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.