Asynchronous Mongoose processing

Question

I have a collection "users" in MongoDB that have some documents such as:

{
    "email": "[email protected]",
    "privileges" : [
        {
            "domain" : "test.com:7777", 
            "_id" : ObjectId("59636fbf0b61c61f659d0365"), 
            "role" : "admin"
        }, 
        {
            "domain" : "test2.com:7777", 
            "_id" : ObjectId("59636fbf0b61c61f659d0365"), 
            "role" : "admin"
        }
    ], 
},
{
    "email": "[email protected]",
    "privileges" : [
        {
            "domain" : "testxyz.com:7777", 
            "_id" : ObjectId("59636fbf0b61c61f659d0365"), 
            "role" : "admin"
        }
    ]
},    
{
    "email": "[email protected]",
    "privileges" : [
        {
            "domain" : "testabc.com", 
            "_id" : ObjectId("59636fbf0b61c61f659d0365"), 
            "role" : "admin"
        }
    ]
}

Please notice the "case sensitive" in each email.

Now what I need to do is that I need to combine these emails into only one with lowercase and merge all privileges.

So the result should be:

 {
    "email": "[email protected]",
    "privileges" : [
        {
            "domain" : "test.com:7777", 
            "_id" : ObjectId("59636fbf0b61c61f659d0365"), 
            "role" : "admin"
        }, 
        {
            "domain" : "test2.com:7777", 
            "_id" : ObjectId("59636fbf0b61c61f659d0365"), 
            "role" : "admin"
        },
        {
            "domain" : "testxyz.com:7777", 
            "_id" : ObjectId("59636fbf0b61c61f659d0365"), 
            "role" : "admin"
        },
        {
            "domain" : "testabc.com", 
            "_id" : ObjectId("59636fbf0b61c61f659d0365"), 
            "role" : "admin"
        }
    ], 
}

Here is what I did:

const _ = require('lodash');
const async = require('async');

module.exports = (req, res, next) => {
  Users.find({}).then((users) => {
    async.mapLimit(users, 1, async (user) => {
      let userEmail = user.email;
      console.log("Loading user " + userEmail);
      // Find all users that have the similar emails (case insensitive) but not the current email
      let regexPattern = new RegExp(`^${userEmail}$`, "i");

      let duplicateUsers = await Users.find({
        $and: [
          {
            email: { $ne: userEmail }
          },
          { email: regexPattern }
        ]
      });

      _.each(duplicateUsers, async (duplicateUser) => {
        // Merge privileges
        user.privileges = _.union(user.privileges, duplicateUser.privileges);
        // Remove duplicate user
        try {
          await duplicateUser.remove();
          console.log(duplicateUser.email + " is removed");
        } catch (err) {
          return err;
        }
      });

      // Convert email to lowercase
      user.email = user.email.toLowerCase();
      user.save().then(() => {
        return user;
      }, (err) => {
        return err;
      });
    }, (err) => {
      if (err) throw err;
      // results is now an array of the users
      res.success('Done');
      res.end();
    });
  }, next);
};

The problem is, in the end, all 3 emails are removed.

I believe that this issue happened because in the loop mapLimit, all users are processed at the same time (javascript asynchronous), because as I logged this

console.log("Loading user " + userEmail);

How should I fix this problem? Thank you.

Rahul Sharma · Accepted Answer · 2018-03-14 05:46:33Z

2

Try this, No need to perform the manual check operation. Save result of aggregate result.

db.getCollection('TEST').aggregate([{
    $match: {
        "email": /[email protected]/i
    }
}, {
    "$unwind": "$privileges"
}, {
    $group: {
        "_id": {
            $toLower: "$email"
        },
        "email": {
            $first: {
                $toLower: "$email"
            }
        },
        "privileges": {
            $addToSet: "$privileges"
        }
    }
}, {
    $project: {
        _id: 0
    }
}])

Response

{
    "email" : "[email protected]",
    "privileges" : [ 
        {
            "domain" : "test.com:7777",
            "_id" : ObjectId("59636fbf0b61c61f659d0365"),
            "role" : "admin"
        }, 
        {
            "domain" : "test2.com:7777",
            "_id" : ObjectId("59636fbf0b61c61f659d0365"),
            "role" : "admin"
        }, 
        {
            "domain" : "testxyz.com:7777",
            "_id" : ObjectId("59636fbf0b61c61f659d0365"),
            "role" : "admin"
        }, 
        {
            "domain" : "testabc.com",
            "_id" : ObjectId("59636fbf0b61c61f659d0365"),
            "role" : "admin"
        }
    ]
}

edited Mar 14, 2018 at 5:46

answered Mar 13, 2018 at 7:37

Rahul Sharma

10.1k1 gold badge20 silver badges41 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

0xh8h Over a year ago

do you mean that I only need to use the mongodb console, right?

Rahul Sharma Over a year ago

@HoangTrinh No no, You are getting all result and then checking for unique privileges manually in nodejs. You can do that in mongo itself.

0xh8h Over a year ago

But what if "[email protected]" is only one of those duplicate emails in database? Do I still need the Users.find({}).then((users) => { async.mapLimit(users, 1, async (user) => { ... at the beginning? Sorry but I still didn't understand this. Thank you

0xh8h Over a year ago

I tried this in an aggregation editor and it worked as you said. But the result now contains "_id" : "[email protected]", which is not what I want. I want the email to be lowercase version, not the _id. What should I do?

Collectives™ on Stack Overflow

Asynchronous Mongoose processing

1 Answer 1

4 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

4 Comments

Your Answer

Sign up or log in

Post as a guest

Related