Elasticsearch "_index" field optimizations

Question

I tried to do some queries to alias with multiple indices in Kibana with profiler and it seems that when you make high-level filter on _index field - it runs some really fast MatchNoDocQuery queries on all indices except the needed one.

E.g.: let's say we have two indices: test.book and test.film. We have an alias test with pattern test.*. Also each index has product_type field which may be "book" or "film".

It seems that this query:

GET test/_search
{
  "query": {
    "bool": {
      "filter": [
        {
          "terms": {
            "_index": ["test.film"]
          }
        }
      ]
    }
}

Is much faster than this query:

GET test/_search
{
  "query": {
    "bool": {
      "filter": [
        {
          "terms": {
            "product_type": ["film"]
          }
        }
      ]
    }
  }
}

Are there any optimizations when running filter on "_index" field?

Val · Accepted Answer · 2021-12-01 10:54:38Z

1

The first query is faster because the query knows how to go through all the segments of the test.film index only and not check the segments of the test.book index. Whereas in the second, the query has no idea that there are multiple indexes and it will have to go through all the shards.

Say both indexes have 10 segments, which means 20 segments in total when running the query of the alias. The first query will only go through 10 segments, whereas the second will have to go through the 20 segments in order to figure out the documents that satisfy the condition.

edited Dec 1, 2021 at 10:54

answered Dec 1, 2021 at 10:18

Val

218k14 gold badges377 silver badges384 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

DenisNovac Over a year ago

I understand it but i tried to do some queries in Kibana with profiler to alias with multiple indices and it seems that when you make filter on _index - it runs some really fast MatchNoDocQuery queries on all indices except the needed one. And i can't google anything about it. So i wonder if there is some not obvious logic.

Val Over a year ago

Of course filter will reduce the document set on which the query has to run, so it will be faster for sure, post_filter is for a totally different use case

DenisNovac Over a year ago

I tried to make a question more clear. Sorry for inconvenient reference.

Val Over a year ago

Thanks, makes much more sense now, I've updated my answer

Collectives™ on Stack Overflow

Elasticsearch "_index" field optimizations

1 Answer 1

4 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

4 Comments

Your Answer

Sign up or log in

Post as a guest

Related