Multiple Mongo Aggregation Query Performance Improvement

Question

I have 5 mongo db aggregator queries. Each query has same group part but different match criteria. forexample :

1- count records by email
2- count records by phone
3- count records by xxx...

db.getCollection("myCol").aggregate([
    {
        "$match": {
                   
               { "contact.email": "[email protected]" }
                 //.. some more conditions       
                
        }
    }
    {
        "$group": {
            "_id": "$myId",
            "count": { "$sum": 1 }
        }
    }
    
])

each query is very fast (as index is being scanned) but since i need to call these group of 5 queries multiple times (say 5* 37) so its making the end-to-end api very slow .

what is the best appriach for this in mongo db ? as i am not very expect in mongoDB.

Any suggestions please?

if there is common match part for some queries and you always need to execute the 5x queries together perhaps try to combine them in a single $facet aggregation: mongodb.com/docs/manual/reference/operator/aggregation/facet — R2D2
– R2D2, Commented Jan 8, 2024 at 15:30

Tom Slabbaert · Accepted Answer · 2024-01-08 17:04:17Z

Let's break down the performance area's your describing:

Application network (Client -> App calls)
DB network (App -> db calls)
DB processing.

You want to be able to reduce each of them separately.

Let's start with the point you're asking about point #3.

Here it Mongo has to scan each index tree separately to count the documents, therefor it doesn't matter if it's 37 different calls, or 1 call using an aggregation.

For point #2

Here the amount of calls does matter, as each time to send a request to the DB and wait for it to return there is overhead, it's the reason operations like bulkWrite exists for example - so you could execute multiple updates while saving the network overhead.

In your case you just want to make sure you're doing a single db call using a single aggregation pipeline.

#1 Goes without saying, I'm assuming the client already sends a single call but in the case he does not change it.

Good luck.

Collectives™ on Stack Overflow

Multiple Mongo Aggregation Query Performance Improvement

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related