Specify logstash configuration on command line with docker

Question

Logstash 6.0 documentation for Stashing your first event shows the following command to run logstash with a simple pipeline which simply pipes stdin to stdout:

$ bin/logstash -e 'input { stdin { } } output { stdout {} }'

This does however not work when running logstash from the official docker container and fails with an error message about incompatible command line options:

$ docker run --rm -it docker.elastic.co/logstash/logstash-oss:6.0.0 \
  -e 'input { stdin { } } output { stdout {} }'
ERROR: Settings 'path.config' (-f) and 'config.string' (-e) can't be used simultaneously.

I'm not specifying -f or path.config. I simply want to test if logstash can be started and is working for this simple case. How to get around this error without mounting a config file?

knittl · Accepted Answer · 2017-12-02 10:49:19Z

Looking into the startup scripts inside the logstash docker image, one can find the following function in the logstash-core/lib/logstash/runner.rb file:

# where can I find the logstash.yml file?
# 1. look for a "--path.settings path"
# 2. look for a "--path.settings=path"
# 3. check if the LS_SETTINGS_DIR environment variable is set
# 4. return nil if not found
def fetch_settings_path(cli_args)
  if i=cli_args.find_index("--path.settings")
    cli_args[i+1]
  elsif settings_arg = cli_args.find {|v| v.match(/--path.settings=/) }
    match = settings_arg.match(/--path.settings=(.*)/)
    match[1]
  elsif ENV['LS_SETTINGS_DIR']
    ENV['LS_SETTINGS_DIR']
  else
    nil
  end
end

So the solution seems to either pass --path.settings= (=-syntax with empty value) or set the LS_SETTINGs_DIR variable to a falsy value. And indeed:

$ docker run --rm -it docker.elastic.co/logstash/logstash-oss:6.0.0 \
  --path.settings= -e 'input { stdin { } } output { stdout { } }'
The stdin plugin is now waiting for input:

herm · Accepted Answer · 2018-01-18 08:33:25Z

0

As Knittl stated --path.settings is consulted for logstash settings which are in a logstash.yml file that looks something like this (taken from the docker.elastic.co/logstash/logstash:6.1.2 image:

http.host: "0.0.0.0" path.config: /usr/share/logstash/pipeline xpack.monitoring.elasticsearch.url: http://elasticsearch:9200 xpack.monitoring.elasticsearch.username: logstash_system xpack.monitoring.elasticsearch.password: changeme

The problem is the path.config points to a directory containing only the default logstash.conf.

The solution with no side effects is:

$ docker run --rm -it docker.elastic.co/logstash/logstash-oss:6.0.0 \ --path.config="" -e 'input { stdin { } } output { stdout { } }'

This way the other settings in logstash.yml will persist.

edited Jan 18, 2018 at 8:33

answered Jan 18, 2018 at 8:27

herm

16.8k7 gold badges45 silver badges65 bronze badges

Collectives™ on Stack Overflow

Specify logstash configuration on command line with docker

2 Answers 2

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related