Elasticsearch match all query using Java API

Question

We are using Elasticsearch version 7.3.1 and Java API.

We are implementing a "free search", which means a search for a value in ALL the fields in each segment in Elasticsearch. If that value appears in at least one of the segment's fields, it should be returned.

Until now, we have used the following: QueryBuilders.multiMatchQuery(value) and it worked well.

As for today, we have added to the mapping file some nested (Elasticsearch data type) type fields.

After that change, the code above does not return the expected results anymore.

How can I implement a search in ALL the fields in a segment without specifying each field to search?

Alkis Kalogeris · Accepted Answer · 2020-04-27 19:40:06Z

5

You could implement the logic of _all of previous elasticsearch versions (this was removed after version 6 I believe).

PUT stackoverflow
{
  "mappings": {
    "properties": {
      "all": {
        "type": "text"
      },
      "group": {
        "type": "text",
        "copy_to": "all"
      },
      "user": {
        "type": "nested",
        "properties": {
          "email": {
            "type": "keyword",
            "copy_to": "all"
          },
          "info": {
            "type": "text",
            "copy_to": "all"
          }
        }
      }
    }
  }
}

Basically you add the copy_to parameter.

PUT stackoverflow/_doc/1
{
  "group" : "programmers",
  "user" : [
    {
      "email" : "[email protected]",
      "info" :  "java developer"
    },
    {
      "email" : "[email protected]",
      "info" :  "css guru"
    }
  ]
}

Then you can search against the all field

GET stackoverflow/_search
{
  "query": {
    "match": {
      "all": "guru"
    }
  }
}

Update

Here is an example of how to modify your query in order for it to work without copy_to

GET stackoverflow/_search
{
  "query": {
    "bool": {
      "minimum_should_match": 1,
      "should": [
        {
          "multi_match": {
            "query": "SEARCH_INPUT_HERE",
            "fields": [
              "group"
            ]
          }
        },
        {
          "nested": {
            "path": "user",
            "query": {
              "multi_match": {
                "query": "SEARCH_INPUT_HERE",
                "fields": [
                  "user.email", "user.info"
                ]
              }
            }
          }
        }
      ]
    }
  }
}

Update 2

public static void main(String[] args) {
    String queryInput = "QUERY_INPUT_HERE";
    String[] nested = {"user", "product"};

    MultiMatchQueryBuilder multiMatchQueryBuilder = QueryBuilders.multiMatchQuery(queryInput, "*");

    BoolQueryBuilder boolQueryBuilder = QueryBuilders.boolQuery();
    List<QueryBuilder> shouldQueryBuilders = boolQueryBuilder.should();

    shouldQueryBuilders.add(multiMatchQueryBuilder);
    for(String path : nested) {
        NestedQueryBuilder nestedQueryBuilder = QueryBuilders.nestedQuery(path, multiMatchQueryBuilder, ScoreMode.Avg);
        shouldQueryBuilders.add(nestedQueryBuilder);
    }

    SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
    searchSourceBuilder.query(boolQueryBuilder);

    SearchRequest searchRequest = new SearchRequest();
    searchRequest.source(searchSourceBuilder);
    searchRequest.indices("MY_INDEX");

    System.out.println(searchRequest.toString());
}

Output

{
  "query": {
    "bool": {
      "should": [
        {
          "multi_match": {
            "query": "QUERY_INPUT_HERE",
            "fields": [
              "*^1.0"
            ],
            "type": "best_fields",
            "operator": "OR",
            "slop": 0,
            "prefix_length": 0,
            "max_expansions": 50,
            "zero_terms_query": "NONE",
            "auto_generate_synonyms_phrase_query": true,
            "fuzzy_transpositions": true,
            "boost": 1.0
          }
        },
        {
          "nested": {
            "query": {
              "multi_match": {
                "query": "QUERY_INPUT_HERE",
                "fields": [
                  "*^1.0"
                ],
                "type": "best_fields",
                "operator": "OR",
                "slop": 0,
                "prefix_length": 0,
                "max_expansions": 50,
                "zero_terms_query": "NONE",
                "auto_generate_synonyms_phrase_query": true,
                "fuzzy_transpositions": true,
                "boost": 1.0
              }
            },
            "path": "user",
            "ignore_unmapped": false,
            "score_mode": "avg",
            "boost": 1.0
          }
        },
        {
          "nested": {
            "query": {
              "multi_match": {
                "query": "QUERY_INPUT_HERE",
                "fields": [
                  "*^1.0"
                ],
                "type": "best_fields",
                "operator": "OR",
                "slop": 0,
                "prefix_length": 0,
                "max_expansions": 50,
                "zero_terms_query": "NONE",
                "auto_generate_synonyms_phrase_query": true,
                "fuzzy_transpositions": true,
                "boost": 1.0
              }
            },
            "path": "product",
            "ignore_unmapped": false,
            "score_mode": "avg",
            "boost": 1.0
          }
        }
      ],
      "adjust_pure_negative": true,
      "boost": 1.0
    }
  }
}

edited Apr 27, 2020 at 19:40

answered Apr 27, 2020 at 17:51

Alkis Kalogeris

17.9k15 gold badges62 silver badges118 bronze badges

Sign up to request clarification or add additional context in comments.

7 Comments

Roy Leibovitz Over a year ago

This "copy_to" is copying the field to the all also? It won't use 2x of space? There is no other way to do that?

Alkis Kalogeris Over a year ago

This is the easiest way without changing your search every time you add a field. It won't double the size, but it will increase it. I'll add an example in my answer for without copy_to.

Alkis Kalogeris Over a year ago

I've added an example of your query should look like in order for it to work with nested and simple fields.

Alkis Kalogeris Over a year ago

The query for the nested fields as well as the other one, are multi_match queries. The fields section can be the wildcard * which means all fields. You can do this "fields": ["*"] as well as this "fields": ["user.*"] if you want more control

Alkis Kalogeris Over a year ago

Let us continue this discussion in chat.

|

Collectives™ on Stack Overflow

Elasticsearch match all query using Java API

1 Answer 1

7 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

7 Comments

Your Answer

Sign up or log in

Post as a guest

Related