nodejs mongodb $in [array] not working if array is a variable

Question

This is very strange to me. If put the array onlyIds in the aggregation query of my db I get no results. If I however put the content of onlyIds that I get printed from line 5 which looks like:

["52e953942a13df5be22cf792","52e953942a13df5be22cf793","52e953942a13df5be22cf797"...]

Then it works. But not if I use the variable.

This function:

var onlyIds = [];
for (var i = 0; i < users.length; i++) {
    onlyIds.push(users[i]._id);
}
console.log("ids: " + JSON.stringify(onlyIds));      <---------- not empty

db.collection('posts', function(err, collection) {
    collection.aggregate([
        {$match: {user_id: {$in: onlyIds}}},   <------- not working
        {$match: {created:{$gte: 0}}},
        {$sort:{"created": -1}},
        {$skip: req.body.skip},
        {$limit: req.body.limit}
    ], 

    function(err, posts) {
        var errorNo, content, message;
        if (err) {
            errorNo = resSend.errorDB;
            message = JSON.stringify(err);
        } else {
            errorNo = resSend.errorNo;
            content = posts;
            message = "";
-->         console.log(JSON.stringify(posts));
        }
        resSend.sendResponse(res,  resSend.errorNo, content, message);
    });
});

So in short, why does this work:

{$match: {user_id: {$in: ["52e953942a13df5be22cf792","52e953942a13df5be22cf793","52e953942a13df5be22cf797"...]}}}

and this doesn't:

{$match: {user_id: {$in: onlyIds}}}

And the line that does not work, works perfectly in another function. Any ideas or enlightenments?

EDIT: Switching to find and using the below answer like this:

collection.find({'user_id': {$in: onlyIdsX}}).toArray(function(err, posts)

does not work either.

ANSWER:

As the selected answer indicates below is when the variable you search for is an ObjectId or a string. For anyone else, make sure that the variable in the db is the same type as the one you try to match it with. In my case both were supposed to be strings, but one's in "onlyIds" were ObjectIds.

Are you sure it passes this condition? if (req.body.greater !== undefined) and this date_search = {$match: {created:{$gte: req.body.greater}}}; is getting executed? You are not using if(cond){..} you skipped {} so its difficult to see what is happening.. — Amol M Kulkarni
– Amol M Kulkarni, Commented Apr 7, 2014 at 8:42
@AmolMKulkarni yes, I'm sure, I updated the code so its a little bit easier to see. The date hasn't done anything to the query's result. — just_user
– just_user, Commented Apr 7, 2014 at 8:50
Kind of agreeing with the first comment. Instead construct the pipeline outside of the aggregation function, then log the pipeline to console before you call aggregate. I think something else is up with your inputs. — Neil Lunn
– Neil Lunn, Commented Apr 7, 2014 at 9:03
@NeilLunn if I log "onlyIds" just before I do the aggregate function I still get all the content I expect. Could something happen to after that in the above code? — just_user
– just_user, Commented Apr 7, 2014 at 9:08
As I said, you are assuming that the array is the problem, it possibly is something else. Log the whole pipeline. When using other logging packages with other language I trace level log things like this all the time. — Neil Lunn
– Neil Lunn, Commented Apr 7, 2014 at 9:20

Amol M Kulkarni · Accepted Answer · 2014-04-07 09:45:58Z

3

Try following code to modify your loop:

var ids = ["52e953942a13df5be22cf792","52e953942a13df5be22cf793","52e953942a13df5be22cf797"];
var obj_ids = [];
for (var i = 0; i < users.length; i++) {
    obj_ids.push(new ObjectID(users[i]._id.toString()));    
    var obj_ids.push(users[i]._id);    // <== This will not work if your DB has _id : ObjectID("xyz") [i.e. you are not overiding defaults]
}

And you should include var ObjectID = require('mongodb').ObjectID; into your code.
You should use .toArray(function(err,.. (Not in your case since you used aggregation framework).This will also cause issue if you are not using findOne() (For more info on this here is the link)

Following is the example which spots the issue (in comments) & working Code:

var mongo = require('mongodb'),
    Server = mongo.Server,
    Db = mongo.Db,
    ObjectID = require('mongodb').ObjectID;
var BSON = require('mongodb').BSONPure;
var server = new Server('localhost', 27017, {
    auto_reconnect: true
}); 
var MongoClient = require('mongodb').MongoClient

//let id = your _id, smth like '6dg27sh2sdhsdhs72hsdfs2sfs'...
var users = ["52e953942a13df5be22cf792","52cbd028e9f43a090ca0c1af","52e953942a13df5be22cf797"];
var obj_ids = [];
for (var i = 0; i < users.length; i++) {
    obj_ids.push(new ObjectID(users[i].toString()));    
    //obj_ids.push(users[i]._id);    // <== This will not work if your DB has _id : ObjectID("xyz") [i.e. you are not overiding defaults]
}

MongoClient.connect('mongodb://127.0.0.1:27017/YourDBName', function(err, db) {
    console.log('err'  +  err);
    db.collection('posts', function(error, collection) {
            //collection.find({_id:{$in: users}}),function(err, docs) { //This will not work
            collection.find({_id:{$in: obj_ids}}).toArray(function(err, docs) {
              console.log("Printing docs from Array. count " + docs.length);
              docs.forEach(function(doc) {
                console.log("Doc from Array ");
                console.dir(doc);
              });
            });
    });
});

edited Apr 7, 2014 at 9:45

answered Apr 7, 2014 at 9:23

Amol M Kulkarni

21.7k34 gold badges125 silver badges165 bronze badges

Sign up to request clarification or add additional context in comments.

5 Comments

Neil Lunn Over a year ago

The driver "should" stringify and ObjectId passed in as an argument on the conversion to BSON for over the while. Are you sure that sending in just a plain ObjectId is the problem?

Amol M Kulkarni Over a year ago

Yes because I faced this issue very long back. And driver is not doing only because you have control to override it to custom String, Number then you can match straight away, By Default it is BSON's ObjectID type. Go through the above code and let me know if you cant find the solution. For more info. You can refer this link for types

just_user Over a year ago

@AmolMKulkarni thanks for your response. It doesn't work for me though. With or without ObjectId and switch to find instead of aggregate only looking for the array doesn't work. See my edited entry.

Amol M Kulkarni Over a year ago

@just_user: Can you please update your question with a actual document stored in MongoDB?

just_user Over a year ago

@AmolMKulkarni thanks to your answer I figured it out. It was the opposite way. I modified my question with the answer that worked for me. Cheers!

Collectives™ on Stack Overflow

nodejs mongodb $in [array] not working if array is a variable

1 Answer 1

5 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

5 Comments

Your Answer

Sign up or log in

Post as a guest

Related