Find docs where field value does not exists in other docs using ElasticSearch query

Question

I have a problem retrieving all the docs, that do not have corresponding other logs by field value. I would appreciate any help given. The easiest to explain is with an example:

I have the following 5 docs:

DOC1:

{
     message: **Step1**,
     id: 100
}

DOC2:

{
     message: **Step2**,
     id: 100
}

DOC3:

{
     message: **Step1**,
     id: 200
}

DOC4:

{
     message: **Step2**,
     id: 300
}

DOC5:

{
     message: **StepX**,
     id: 400
}

So what I would like as a response from a query, would be all the docs, where Step1 does not have the corresponding Step2, where id value is the same. In the example below, this is (RESPONSE I WANT): DOC3:

{
     message: Step1,
     id: 200
}

What I tried so far is not the result I would like. This returns count of docs by id and ascending order, but since we have many other Steps, this is not ok, and results with count 1 will not be always correct.

GET index/_search
{
      "size": 0, 
      "aggs" : {
          "langs" : {
              "terms" : { 
                    "field" : "id",  
                    "size" : 10, 
                    "order": {
                      "_count": "asc"
                     }
               }
          }
     } 
}

MORE DETAILS In response, I need docs, that does not have 2 docs related with one id field (each result should have only 1 doc with related id and message=Step1).

@Suresh For the result, I would need all the docs, where "Step2" value is missing on "message" field with "id" field of value "x" -> in above example this is 200. — tadej
– tadej, Commented Jun 19, 2019 at 11:56

Suresh · Accepted Answer · 2019-06-19 12:09:35Z

0

Try this...

{
  "query": {
    "bool": {
      "must": [
        {
          "term": {
            "id": "x"
          }
        }
      ],
      "must_not": [
        {
          "exists": {
            "field": "message"
          }
        }
      ]
    }
  }
}

answered Jun 19, 2019 at 12:09

Suresh

5,99912 gold badges56 silver badges84 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

tadej Over a year ago

This would be ok if I would have those ids (x in this example). I want this dynamic. It means that I will get docs for each id, that does not have the corresponding message. And btw, field "message" must always exist. I need docs, that does not have 2 docs related with one id field (one with message=Step1, another with message=Step2)

Collectives™ on Stack Overflow

Find docs where field value does not exists in other docs using ElasticSearch query

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related