Elasticsearch: Multiple small queries vs one bigger query - efficiency?

Question

Is it more efficient to do multiple single queries in a php loop or to keep everything in an array and build one bigger query for elastic?

Option 1:

foreach ($rowset as $row) { 
//execute query:
"query": {
                     "bool" : {
                          "must" : [
                            {"term" : { "a" : "'.$row['a'].'" }},
                            {"term" : { "b" : "'.$row['b'].'" }}
                          ]
                    }
                }

}

Option 2:

$search = "";

        foreach ($rowset as $key => $row) {

            if($key > 0) {

                $search .= ',';
            }

            $search .= '"must" : [
                            {"term" : { "a" : "'.$row['a'].'" }},
                            {"term" : { "b" : "'.$row['b'].'" }}
                          ]'
        }


"query": {
                     "bool" : {
                         "should" : [
                              "must" : [
                                '.$search.'
                              ]
                         ]
                    }
                }

The syntax may be incorrect in this minimal example but I hope the idea becomes clear. I would expect Option 2 to be faster. I didn't test it yet since I have multiple nested loops right now and wanted an optinion before I rewrite my code.

Justinas · Accepted Answer · 2018-07-12 13:28:48Z

3

External resources requests are always slow. So it's better to create single request. External DB will process it with same speed, but you will save requests time.

In the other hand you can get such big query, that your client (e.g. browser) will timeout or you can't put everything to memory for processing.

Use single query with reasonable amount of data.

answered Jul 12, 2018 at 13:28

Justinas

43.9k5 gold badges72 silver badges108 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Sergej Schelle · Accepted Answer · 2018-07-12 13:34:50Z

1

Single request using filter is the best option: https://www.elastic.co/guide/en/elasticsearch/reference/current/query-filter-context.html

answered Jul 12, 2018 at 13:34

Sergej Schelle

4203 silver badges9 bronze badges

Collectives™ on Stack Overflow

Elasticsearch: Multiple small queries vs one bigger query - efficiency?

2 Answers 2

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related