ElasticSearch - Get only matching nested objects with All Top level fields in search response

Question

let say I have following Document:

{
    id: 1,
    name: "xyz",
    users: [
        {
            name: 'abc',
            surname: 'def'
        },
        {
            name: 'xyz',
            surname: 'wef'
        },
        {
            name: 'defg',
            surname: 'pqr'
        }
    ]
}

I want to Get only matching nested objects with All Top level fields in search response. I mean If I search/filter for users with name 'abc', I want below response

{
    id: 1,
    name: "xyz",
    users: [
        {
            name: 'abc',
            surname: 'def'
        }
    ]
}

How can I do that?

Reference : select matching objects from array in elasticsearch

So you basically need the GET query in order to retrieve the above like result ? — Kulasangar
– Kulasangar, Commented Nov 24, 2016 at 7:19
How about the previous answer in the referenced question but excluding the nested users field from the root source document? You'd get all root fields (except the nested one) and then only the matching inner hit from the nested field — Val
– Val, Commented Nov 24, 2016 at 7:21

Val · Accepted Answer · 2016-11-24 08:06:08Z

50

If you're ok with having all root fields except the nested one and then only the matching inner hits in the nested field, then we can re-use the previous answer like this by specifying a slightly more involved source filtering parameter:

{
  "_source": {
    "includes": [ "*" ],
    "excludes": [ "users" ]
  },
  "query": {
    "nested": {
      "path": "users",
      "inner_hits": {        <---- this is where the magic happens
        "_source": [
          "name", "surname"
        ]
      },
      "query": {
        "bool": {
          "must": [
            {
              "term": {
                "users.name": "abc"
              }
            }
          ]
        }
      }
    }
  }
}

answered Nov 24, 2016 at 8:06

Val

218k14 gold badges377 silver badges384 bronze badges

Sign up to request clarification or add additional context in comments.

6 Comments

Jay Shah Over a year ago

Btw, we can get all fields of nested objects by keeping inner_hits empty i.e. "inner_hits": { }

Val Over a year ago

Yes, you can, I added the _source in the inner_hits section since your expected answer only mentioned name and surname

Val Over a year ago

Anything more needed?

Jay Shah Over a year ago

No, as I think getting only matching nested objects with all top level fields in source is not possible and using inner_hits is the only way. Thanks

Anirudh Over a year ago

@JayShah Hi, any change in ES 5 to support this (getting only matching nested objects with all top level fields in source ) now?

|

Mathieu Giboulet · Accepted Answer · 2017-08-29 12:54:59Z

Maybe late, I use nested sorting to limit element on my nested relation, here a example :

"sort": {
    "ouverture.periodesOuvertures.dateDebut": {
      "order": "asc",
      "mode": "min",
      "nested_filter": {
        "range": {
          "ouverture.periodesOuvertures.dateFin": {
            "gte": "2017-08-29",
            "format": "yyyy-MM-dd"
          }
        }
      },
      "nested_path": "ouverture.periodesOuvertures"
    }
  },

Since 5.5 ES (I think) you can use filter on nested query. Here a example of nested query filter I use:

{
            "nested": {
              "path": "ouverture.periodesOuvertures",
              "query": {
                "bool": {
                  "must": [
                    {
                      "range": {
                        "ouverture.periodesOuvertures.dateFin": {
                          "gte": "2017-08-29",
                          "format": "yyyy-MM-dd"
                        }
                      }
                    },
                    {
                      "range": {
                        "ouverture.periodesOuvertures.dateFin": {
                          "lte": "2017-09-30",
                          "format": "yyyy-MM-dd"
                        }
                      }
                    }
                  ],
                  "filter": [
                    {
                      "range": {
                        "ouverture.periodesOuvertures.dateFin": {
                          "gte": "2017-08-29",
                          "format": "yyyy-MM-dd"
                        }
                      }
                    },
                    {
                      "range": {
                        "ouverture.periodesOuvertures.dateFin": {
                          "lte": "2017-09-30",
                          "format": "yyyy-MM-dd"
                        }
                      }
                    }
                  ]
                }
              }
            }
          }

Hope this can help ;)

Plus if you ES is not in the last version (5.5) inner_hits could slow your query Including inner hits drastically slows down query results

user8393839 · Accepted Answer · 2020-06-08 10:06:38Z

0

https://www.elastic.co/guide/en/elasticsearch/reference/5.6/search-request-inner-hits.html#nested-inner-hits-source

  "inner_hits": {
    "_source" : false,
    "stored_fields" : ["name", "surname"]
  }

but you may need to change mapping to set those fields as "stored_fields" , otherwise you can use "inner_hits": {} to get a result that not that perfect.

answered Jun 8, 2020 at 10:06

user8393839

1

Comments

vl4deee11 · Accepted Answer · 2021-07-08 11:14:49Z

0

You can make such a request, but the response will have internal fields starting with _

{
  "_source": {
    "includes": [ "*" ],
    "excludes": [ "users" ]
  },
  "query": {
    "nested": {
      "path": "users",
      "inner_hits": {},
      "query": {
        "bool": {
          "must": [
            { "match": { "users.name":  "abc" }}
          ]
        }
      }
    }
  }
}

answered Jul 8, 2021 at 11:14

vl4deee11

819 bronze badges

Comments

Brijesh Wani · Accepted Answer · 2021-08-05 11:19:34Z

In one of my projects, My expectation was to retrieve unique conversation messages text(inner fields like messages.text) having specific tags. So instead of using inner_hits, I used aggregation like below,

final NestedAggregationBuilder aggregation = AggregationBuilders.nested("parentPath", "messages").subAggregation(AggregationBuilders.terms("innerPath").field("messages.tag"));
        final NativeSearchQuery searchQuery = new NativeSearchQueryBuilder()
                .addAggregation(aggregation).build();
        final Aggregations aggregations = elasticsearchOperations.search(searchQuery, Conversation.class).getAggregations();
        final ParsedNested parentAgg = (ParsedNested) aggregations.asMap().get("parentPath");
        final Aggregations childAgg = parentAgg.getAggregations();
        final ParsedStringTerms childParsedNested = (ParsedStringTerms) childAgg.asMap().get("innerPath");
        // Here you will get unique expected inner fields in key part.
        Map<String, Long> agg = childParsedNested.getBuckets().stream().collect(Collectors.toMap(Bucket::getKeyAsString, Bucket::getDocCount));

Oleg Dmitrochenko · Accepted Answer · 2021-11-12 09:00:22Z

I use the following body to get that result (I have set the full path to the values):

   {
  "_source": {
    "includes": [ "*" ],
    "excludes": [ "users" ]
  },
  "query": {
    "nested": {
      "path": "users",
      "inner_hits": {        
        "_source": [
          "users.name", "users.surname"
        ]
      },
      "query": {
        "bool": {
          "must": [
            {
              "term": {
                "users.name": "abc"
              }
            }
          ]
        }
      }
    }
  }
}

Also another way exists:


   {
  "_source": {
    "includes": [ "*" ],
    "excludes": [ "users" ]
  },
  "query": {
    "nested": {
      "path": "users",
      "inner_hits": {      
        "_source": false,
        "docvalue_fields": [
          "users.name", "users.surname"
        ]
      },
      "query": {
        "bool": {
          "must": [
            {
              "term": {
                "users.name": "abc"
              }
            }
          ]
        }
      }
    }
  }
}

See results in inner_hits of the result hits.

https://www.elastic.co/guide/en/elasticsearch/reference/7.15/inner-hits.html#nested-inner-hits-source

Collectives™ on Stack Overflow

ElasticSearch - Get only matching nested objects with All Top level fields in search response

6 Answers 6

6 Comments

Comments

Comments

Comments

Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

6 Answers 6

6 Comments

Comments

Comments

Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related