Elasticsearch: Issue with multiple prefix and multiple fields search

Question

hope someone can enlighten me on this one. Suppose I have the following data:

{ "index": { "_index": "courses_test", "_id": 1 } }
{ "Course Name": "Bachelor of Arts in Music", "Job Role": "Theatre & Media Director, Video Engineer" }
{ "index": { "_index": "courses_test", "_id": 2 } }
{ "Course Name": "Bachelor of Arts in Engineering", "Job Role": "Graduate policy officer, editorial assistant, communications and campaigns assistant, assistant advocacy officer, employment consultant." }

My objective is to match "Bachelor" AND "Engineering" in their Course Name AND Job Role fields. With the query below, not quite sure why 2 courses are being returned but document ID 2 does not satisfy the condition.

It works as expected if I search in "Course Name" only. Searching in "Job Role" returns 0, which is correct too.

I am using query string and using * so that even if the user just typed in prefixes e.g. 'bach eng', it should still match.

Full query:

{
    "query": {
        "bool": {
            "must": [
                {
                    "query_string": {
                        "query": "Bachelor* AND Engineer*",
                        "fields": [
                            "Course Name",
                            "Job Role"
                        ]
                    }
                }
            ]
        }
    }
}

Response:

{
    "took": 1,
    "timed_out": false,
    "_shards": {
        "total": 1,
        "successful": 1,
        "skipped": 0,
        "failed": 0
    },
    "hits": {
        "total": {
            "value": 2,
            "relation": "eq"
        },
        "max_score": 2.0,
        "hits": [
            {
                "_index": "courses_test",
                "_type": "_doc",
                "_id": "1",
                "_score": 2.0,
                "_source": {
                    "Course Name": "Bachelor of Arts in Music",
                    "Job Role": "Theatre & Media Director, Video Engineer"
                }
            },
            {
                "_index": "courses_test",
                "_type": "_doc",
                "_id": "2",
                "_score": 2.0,
                "_source": {
                    "Course Name": "Bachelor of Arts in Engineering",
                    "Job Role": "Graduate policy officer, editorial assistant, communications and campaigns assistant, assistant advocacy officer, employment consultant"
                }
            }
        ]
    }
}

Thank you for your help!

So I asked this question here: discuss.elastic.co/t/multi-fields-multi-prefixes-search/276504 and just to update this that I found the solution to meet my objective. After the explanation below from @ibexit re AND to OR. I thought of separating queries per field and used 'should' instead of 'must'. — Jan
– Jan, Commented Jun 23, 2021 at 3:09

ibexit · Accepted Answer · 2021-06-21 10:33:59Z

1

The Query String Query will expand your query to a OR query for each field you provide. Please have a look here. At the end, all documents will match having at least one match in any field.

Probably you´ll need to rewrite the query using a https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-multi-match-query.html AND/OR https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-bool-query.html

For future debugging: There is a API endpoint capable of explaining why a document matches:

https://www.elastic.co/guide/en/elasticsearch/reference/current/search-explain.html

In your case this should give you the related insights (please note the index name and document id in the url):

GET /courses_test/_explain/1  
{
  "query": {
    "bool": {
      "must": [
        {
          "query_string": {
            "query": "Bachelor* AND Engineer*",
            "fields": [
              "Course Name",
              "Job Role"
            ]
          }
        }
      ]
    }
  }
}

answered Jun 21, 2021 at 10:33

ibexit

3,7021 gold badge14 silver badges27 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Jan Over a year ago

I forgot to check that documentation and just focused on opendistro documentation, but that reference to query_string made more sense there! Thank you! And I've come across the explain but always returns 'no matching clause'. But thanks again for your input!

Collectives™ on Stack Overflow

Elasticsearch: Issue with multiple prefix and multiple fields search

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related