ElasticSearch: Multi Field query with query_string over property and nested object

Question

I'm trying to solve the following high level requirement:

save arbitrary number of key value pairs on a document which already has properties: name and description
values can be numbers which should be 'range searchable'
values can include geo_points which need to be 'geo searchable'

I've created the following index.

{
    "settings": {
        "index" : {
            "number_of_shards" : 3, 
            "number_of_replicas" : 1
        }
    },
    "mappings": {
        "_doc": {
            "dynamic": "strict",
            "properties": {
                "name": {
                    "type": "text",
                    "analyzer": "german",
                },
                "description": {
                    "type": "text",
                    "analyzer": "german"
                },
                "attributes": {
                    "type": "nested",
                    "properties": {
                        "key": { "type": "text" },
                        "val_bool":   { "type": "boolean" },
                        "val_int":    { "type": "integer" },
                        "val_float":  { "type": "float" },
                        "val_string": { "type": "text" },
                        "val_geo":    { "type": "geo_point" },
                        "val_date":   { "type": "date" }
                    }
                }
            }
        }
    }
}

We use nested objects to be able to save a list of key-value-pairs for each document. Each key-value-pair uses one typed val_* property to persist the typed value. Thus enabling special searches for special types, Range-Query for example.

To search in documents we use query_string query to allow users to be very specific in searches. For example. Search documents where name:foo AND description:bar. (that works as expected)

The same scenario should be possible with key-value-pairs.So for example: attributes.key:someKey AND attributes.val_string:someStringValue. This scenario requires a nested query which we use and works as expected.

What isnt't working: If we search for name:foo AND attributes.key:someKey we get no results.

It's seems that 'nested query_string queries' AND 'just query_string queries' arent't supported in combination. Is that true? What's a feasible workaround to implement the described requirement?

The query looks like this:

{
    "query": {
        "bool": {
            "should": [
                {
                    "query_string": {
                        "query": "attributes.key:someKey AND name:foo",
                        "default_operator": "and",
                        "fields": [
                            "name",
                            "description"
                        ]
                    }
                },
                {
                    "nested": {
                        "query": {
                            "query_string": {
                                "query": "attributes.key:someKey AND name:foo",
                                "default_operator": "and",
                                "fields": [
                                    "attributes.key",
                                    "attributes.val_string"
                                ]
                            }
                        },
                        "path": "attributes"
                    }
                }
            ]
        }
    }
}

Any help is greatly appreciated. Thank you in advance.

Flogex · Accepted Answer · 2018-11-06 06:04:37Z

Nested objects are indexed as separate documents. That is, you are effectively querying two documents, but neither of them matches the given query:

_doc does not have a (not nested) field attributes, and attributes does not contain a field name. Since both terms in your query are linked logically with AND, the search results in zero hits.

A similar problem appears for query strings like attributes.key:someKey AND attributes.key:otherKey when trying to retrieve all root parent documents that contain two nested objects with one of the specified keys. Because both nested objects are independent from one each other, this query is looking for a nested object with both keys - but each object can only have one.

To work around this, you must somehow split your query into the parts concerning the root parent document and the ones for nested objects. After that, you have to create a nested query for each term referring to a nested object. In other words, name:foo AND attributes.key:someKey must eventually look like this:

{
    "query": {
        "bool": {
            "must": [
                {
                    "query_string": {
                        "query": "name:foo"       
                    }
                },
                {
                    "nested": {
                        "query": {
                            "query_string": {
                                "query": "attributes.key:someKey"
                            }
                        },
                        "path": "attributes"
                    }
                }
            ]
        }
    }
}

One possible solution is to provide several inputs, one for querying the root parent document itself and another for querying the nested objects. You can then combine both query strings by manually creating a query consisting of a nested part and one that is not nested.

Alternatively, you may think about parsing the query by yourself, generating a nested query every time come across attributes.[field].

Collectives™ on Stack Overflow

ElasticSearch: Multi Field query with query_string over property and nested object

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related