ElasticSearch: search multiple elements in array of object

Question

I'm on Elastic Search 6.8.22

I have multiple users and each one has multiple papers ("valid" or not):

{"name":"Amy",
    "papers":[
        {"type":"idcard", "country":"fr", "valid":"no"},
        {"type":"idcard", "country":"us", "valid":"yes"}
    ]}

{"name":"Brittany",
    "papers":[
        {"type":"idcard", "country":"fr", "valid":"no"},
        {"type":"idcard", "country":"us", "valid":"no"}
    ]}

{"name":"Chloe",
    "papers":[
        {"type":"idcard", "country":"fr", "valid":"yes"},
        {"type":"idcard", "country":"us", "valid":"no"}
    ]}

I'm trying to find only user with a paper: "valid" for "fr":

{"query": {
    "bool": {
      "filter": [
              {"match":{"papers.valid": "yes"}},
              {"match":{"papers.country": "fr"}}
      ]}}}

It returns Chloe, which is fine (she has a paper which is both "valid" and "fr"). But it also returns Amy; because she has one "valid" paper and another one which is "fr". This is due to the fact that ES doesn't understand array of objects and flattens everything into values with arrays (as far as I understand).

I've tried using "combined term queries" from this link, but I guess it only works for arrays of "primitive" (not complex objects).

I've seen that I can transform arrays into nested objects to do what I need, but it seems to be overcomplicated and would slow down the queries (because of hidden joins).

My question is: Is there any way I can search if a document has in its array of objects, one that match multiple criteria at the same time ?

(Originally, I wanted a query that checks if every "papers" in the array matched criteria, but that seems impossible, ex. all papers of type "idcard" must be "valid")

As far as i know only nested type can solve your problem. Another option you can use it painless script and remove element at response time but again it will add overhead on response time. — Sagar Patel
– Sagar Patel, Commented Aug 17, 2022 at 9:34

Ron Serruya · Accepted Answer · 2022-08-17 09:41:47Z

You need to define papers as a nested field in the mapping, then you can run a nested search on it

https://www.elastic.co/guide/en/elasticsearch/reference/current/nested.html

So if for example, your mapping will be this:

{
  "mappings": {
    "properties": {
      "name": {
        "type": "keyword"
      },
      "papers": {
        "type": "nested",
        "properties": {
          "type": {
            "type": "keyword"
          },
          "country": {
            "type": "keyword"
          },
          "valid": {
            "type": "keyword"
          }
        }
      }
    }
  }
}

this query will work

{
  "query": {
    "nested": {
      "path": "papers",
      "query": {
        "bool": {
          "filter": [
            {
              "term": {
                "papers.valid": "yes"
              }
            },
            {
              "term": {
                "papers.country": "fr"
              }
            }
          ]
        }
      }
    }
  }
}

Collectives™ on Stack Overflow

ElasticSearch: search multiple elements in array of object

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related