get unique values from elasticsearch

Question

I am trying to get unique values from elasticsearch using this dsl query

{
  "aggs": {
    "distinct": {
      "terms": {
        "field": "model",
        "size": 0
      }
    }
  }
}

The problem with this is that if I have two models i.e. moto g and htc one, I get four results, moto, g, htc, one. How can I configure it to return two results?

Val · Accepted Answer · 2016-02-23 14:36:27Z

2

You need to modify the mapping of your model field to add a not_analyzed sub-field instead and run the aggregation on that sub-field instead, like this:

curl -XPUT localhost:9200/your_index/_mapping/your_type -d '{
  "properties": {
    "model": {
      "type": "string",
      "fields": {                      <-- add this section
        "raw": {
          "type": "string",
          "index": "not_analyzed"
        }
      }
    }
  }
}'

Once your mapping is updated, you need to reindex your data and then you'll be able to run your aggregation like this:

{
  "aggs": {
    "distinct": {
      "terms": {
        "field": "model.raw",          <--- use the new sub-field
        "size": 0
      }
    }
  }
}

answered Feb 23, 2016 at 14:36

Val

218k14 gold badges377 silver badges384 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

Keshav Agarwal Over a year ago

I am using logstash to send my logs to elasticsearch. When I used model.raw directly, it threw errors ---> doc_count_error_upper_bound:0 and sum_other_doc_count:0. Does logstash not send raw fields? Or do I have to modify the query in some other format?

Val Over a year ago

You can either modify the elasticsearch output in the Logstash config in order to apply your own template with manage_template => true or use an index template in ES. Whichever way suits you best.

Keshav Agarwal Over a year ago

The change in logstash filter didn't work for some unknown reason as no errors were thrown and I didn't want to create a new template in my elasticsearch. I ended up updating the mapping and re-indexing the data. But it worked! Thanks a ton!

Collectives™ on Stack Overflow

get unique values from elasticsearch

1 Answer 1

3 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

3 Comments

Your Answer

Sign up or log in

Post as a guest

Related