Elasticsearch Query NOT searching in the specified fields

Question

I am struggling with an elasticsearch query. In the fields option, we have specified '*' which means it should look in all fields as well as given the higher weights to a few fields. But it isn't working as it should. This query was written by my colleague, it'd be great if you could explain it as well as point out the solution. Here's my query:

{
  "query": {
    "bool": {
      "must": [
        {
          "simple_query_string": {
            "query": "Atoms for Peace",
            "default_operator": "AND",
            "flags": "PREFIX|PHRASE|NOT|AND|OR|FUZZY|WHITESPACE",
            "fields": [
              "*",
              "systemNumber^5",
              "global_search",
              "objectType^2",
              "partTypes.text",
              "partTypes.id",
              "gs_am_people^2",
              "gs_am_person^2",
              "gs_am_org^2",
              "gs_title^2",
              "_currentLocation.displayName",
              "briefDescription",
              "physicalDescription",
              "summaryDescription",
              "_flatPersonsNameId",
              "_flatPeoplesNameId",
              "_flatOrganisationsNameId",
              "_primaryDate",
              "_primaryDateEarliest",
              "_primaryDateLatest"
            ]
          }
        }
      ]
    }
  }

The query 'Atoms for Peace' exist in the title of a record but this query shows zero result although we have specified 'gs_title' with a weight of 2. And it also happens for other searched queries where they are present in the data but this query returns zero result or inaccurate result. — sheharbano
– sheharbano, Commented Oct 10, 2022 at 11:28
Can you add your mapping and sample document which contains "Atoms for Peace" to recreate it. In my view your query should work fine — jaspreet chahal
– jaspreet chahal, Commented Oct 10, 2022 at 11:35
Mapping of the index is quite long, should I just add it in the question? — sheharbano
– sheharbano, Commented Oct 10, 2022 at 11:42

jaspreet chahal · Accepted Answer · 2022-10-10 12:26:05Z

2

Your query is fine but it will not work on field with "nested" data type.

From doc

Searching across all eligible fields does not include nested documents. Use a nested query to search those documents.

You need to use nested query

{
  "query": {
    "bool": {
      "minimum_should_match": 1, 
      "should": [
        {
          "simple_query_string": {
            "query": "Atoms for Peace",
            "default_operator": "AND",
            "flags": "PREFIX|PHRASE|NOT|AND|OR|FUZZY|WHITESPACE",
            "fields": [
              "*",
              "systemNumber^5",
              "global_search",
              "objectType^2",
              "partTypes.text",
              "partTypes.id",
              "gs_am_people^2",
              "gs_am_person^2",
              "gs_am_org^2",
              "gs_title^2",
              "_currentLocation.displayName",
              "briefDescription",
              "physicalDescription",
              "summaryDescription",
              "_flatPersonsNameId",
              "_flatPeoplesNameId",
              "_flatOrganisationsNameId",
              "_primaryDate",
              "_primaryDateEarliest",
              "_primaryDateLatest"
            ]
          }
        },
        {
          "nested": {
            "path": "record",
            "query": {
              "simple_query_string": {
                "query": "Atoms for Peace",
                "default_operator": "AND",
                "flags": "PREFIX|PHRASE|NOT|AND|OR|FUZZY|WHITESPACE",
                "fields": [
                  "*"
                ]
              }
            }
          }
        }
      ]
    }
  }
}

edited Oct 10, 2022 at 12:26

answered Oct 10, 2022 at 11:54

jaspreet chahal

9,1392 gold badges14 silver badges30 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

sheharbano Over a year ago

The query failed with this exception "reason" : "failed to create query: [nested] failed to find nested object under path [records]" 'Record' is just added in the API but not in the mapping itself. We do have a mix of fields with few nested fields. So is there any way to give a path to an index like you used 'records'?

jaspreet chahal Over a year ago

@sheharbano i have modified query . Can you try it. Your field name is record, i was using records

jaspreet chahal Over a year ago

For fields which are not of type nested, your query is fine and should give result. Nested data type is a special case in elasticsearch which needs use of nested queries. Separate nested queries are needed for separate nested fields . Another option is to use copy_to which will copy all data in a single sentence in a single field. To use it changes will be needed in mapping and reindexing of data and keywords will be searched across of all fields

Collectives™ on Stack Overflow

Elasticsearch Query NOT searching in the specified fields

1 Answer 1

3 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

3 Comments

Your Answer

Sign up or log in

Post as a guest

Related