finding duplicate field values in elasticsearch

Question

Using elasticsearch 0.19.4 (I know this is old, but its what is required by a dependency)

I have a field "digest" in an elasticsearch index - and I would like to execute a query that will return me all the cases where there are duplicate values of digest. Can this be done?

For the records that have duplicate values, I would like to return other values - such as "url" which may not be duplicated.

I am also looking for the same but till now did not find anything. If anyone knows the answer of this question then please reply. Thanks in Advance!! — Mukesh
– Mukesh, Commented Jun 16, 2015 at 5:54

Giriraj Sharma · Accepted Answer · 2015-10-01 06:13:55Z

3

You can use Terms Aggregation for this.

POST <index>/<type>/_search?search_type=count
{
    "aggs": {
       "duplicateNames": {
           "terms": {
               "field": "digest",
               "size": 0,
               "min_doc_count": 2
            }
        }
    }
}

This will return all values of the field digest which occur in at least 2 documents. I agree this does not exactly match to your use case but it might help.

answered Oct 1, 2015 at 6:13

Giriraj Sharma

3053 silver badges13 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

finding duplicate field values in elasticsearch

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related