AND query in elasticsearch with curl

Question

Let's inject some data in Elasticsearch

curl -XPUT 'localhost:9200/customer/external/1' -d '{ "author": "John", "published_from":"2016-08-03" }'
curl -XPUT 'localhost:9200/customer/external/2' -d '{ "author": "Jeanne", "published_from":"2016-08-03" }'
curl -XPUT 'localhost:9200/customer/external/3' -d '{ "author": "Jean", "published_from":"2016-08-05" }'

I am trying to query document with published_from=2016-08-03 and author=John. I try to do it with this curl command:

curl -g "localhost:9200/customer/external/_search?pretty&filter_path=hits.hits._source.author&q=+author:John+published_from:2016-08-03"

Yet, the output displays Jeanne

{
  "hits" : {
    "hits" : [
      {
        "_source" : {
          "author" : "John"
        }
      },
      {
        "_source" : {
          "author" : "Jeanne"
        }
      }
    ]
  }
}

When I try this curl command :

curl "localhost:9200/customer/external/_search?pretty&filter_path=hits.hits._source.author&q=%2Bauthor%3AJohn+%2Bpublished_from%3A2016-08-03"

The output is exactly what I want.

{
  "hits" : {
    "hits" : [
      {
        "_source" : {
          "author" : "John"
        }
      }
    ]
  }
}

Why is the first command not working as expected ?

randomir · Accepted Answer · 2017-10-30 22:48:41Z

3

The + signs in the first URL:

...&q=+author:John+published_from:2016-08-03

are interpreted (on server-side), in accordance to the percent-encoding rules as spaces. The space is usually encoded as %20, but for historical reasons, + is also a valid encoding of the space character.

That means the query string that ElasticSearch gets looks like:

author:John published_from:2016-08-03

According to query string syntax, it will find any document that contains one or more of author:John or published_from:2016-08-03.

When you properly encode the Elastic's + operator (in the second URL), the query received is:

+author:John +published_from:2016-08-03

Note the + was decoded as space , and %2B as +.

Simple query parameter encoding in `curl`

To avoid manual (percent-)encoding of queries, you can use the --data-urlencode option, and provide raw key=value pairs.

For example:

curl -G 'localhost:9200/customer/external/_search?pretty&filter_path=hits.hits._source.author' --data-urlencode 'q=+author:John +published_from:2016-08-03'

Here curl will combine the query parameters from the URL with those provided with the -d/--data-urlencode options. We need -G to force a GET request, since -d/--data-urlencode defaults to a POST request.

edited Oct 30, 2017 at 22:48

answered Oct 20, 2017 at 13:28

randomir

18.8k1 gold badge46 silver badges60 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

g.lahlou Over a year ago

I thought the option -g in curl was preventing the encoding. --data-urlencode was exactly what I was looking for ! Thanks !

randomir Over a year ago

The -g option prevents {}[] expansion (you don't really need it in your example), but it doesn't encode query params. You're welcome!

Collectives™ on Stack Overflow

AND query in elasticsearch with curl

1 Answer 1

Simple query parameter encoding in `curl`

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Simple query parameter encoding in curl

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related

Simple query parameter encoding in `curl`