Aggregate and flatten an array field in MongoDB

Question

I have a Schema:

var ProjectSchema = new Schema({
    name: {
        type: String,
        default: ''
    },
    topics: [{
        type: Schema.ObjectId,
        ref: 'Topic'
    }],
    user: {
        type: Schema.ObjectId,
        ref: 'User'
    }
});

What I want to do is get an array with all topics from all projects. I cannot query Topic directly and get a full list because some topics are unassigned and they do not hold a reference back to a Project (for reasons of avoiding two way references). So I need to query Project and aggregate some how. I am doing something like:

Project.aggregate([{$project:{topics:1}}]);

But this is giving me an array of Project objects with the topics field. What I want is an array with topic objects.

How can I do this?

Please post the sample output

karthick
– karthick

2015-06-26 08:28:54 +00:00
Commented Jun 26, 2015 at 8:28 — karthick
– karthick, Commented Jun 26, 2015 at 8:28

Blakes Seven · Accepted Answer · 2015-06-26 08:45:05Z

8

When dealing with arrays you typically want to use $unwind on the array members first and then $group to find the distinct entries:

Project.aggregate(
    [
        { "$unwind": "$topics" },
        { "$group": { "_id": "$topics._id" } }
    ],
    function(err,docs) {

    }
)

But for this case, it is probably simplier to just use .distinct() which will do the same as above, but with just an array of results rather than documents:

Project.distinct("topics._id",function(err,topics) {

});

But wait there a minute because I know what you are really asking here. It's not the _id values you want but your Topic data has a property on it like "name".

Since your items are "referenced" and in another collection, you cannot do an aggregation pipeline or .distinct() operation on the property of a document in another collection. Put basically "MongoDB does not perform Joins" and mongoose .populate() is not a join, just something that "emulates" that with additional query(ies).

But you can of course just find the "distinct" values from "Project" and then fetch the information from "Topic". As in:

Project.distinct("topics._id",function(err,topics) {
    Topic.find({ "_id": { "$in": topics } },function(err,topics) {

    });
});

Which is handy because the .distinct() function already returned an array suitable for use with $in.

answered Jun 26, 2015 at 8:45

Blakes Seven

50.5k14 gold badges132 silver badges136 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Mike M Over a year ago

Thank you for the detailed answer. It almost worked as long as I wrote Project.distinct("topics", function(err, topicIDs) {});

Blakes Seven Over a year ago

@MikeM My bad, as I generally have "embedding" on the brain or did when writing. With a referenced array of course the value is just the ObjectId value in the array with no sub-properties as you have noted.

Collectives™ on Stack Overflow

Aggregate and flatten an array field in MongoDB

1 Answer 1

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related