MongoDB: Concatenate Multiple Arrays

Question

I have 3 arrays of ObjectIds I want to concatenate into a single array, and then sort by creation date. $setUnion does precisely what I want, but I'd like to try without using it.

Schema of object I want to sort:

var chirpSchema = new mongoose.Schema({
    interactions: { 
      _liked      : ["55035390d3e910505be02ce2"] // [{ type: $oid, ref: "interaction" }]
    , _shared     : ["507f191e810c19729de860ea", "507f191e810c19729de860ea"] //  [{ type: $oid, ref: "interaction" }]
    , _viewed     : ["507f1f77bcf86cd799439011"] //  [{ type: $oid, ref: "interaction" }]
  }
});

Desired result: Concatenate _liked, _shared, and _viewed into a single array, and then sort them by creation date using aggregate pipeline. See below

["507f1f77bcf86cd799439011", "507f191e810c19729de860ea", "507f191e810c19729de860ea", "55035390d3e910505be02ce2"]

I know I'm suppose to use $push, $each, $group, and $unwind in some combination or other, but I'm having trouble piecing together the documenation to make this happen.

Update: Query

model_user.aggregate([
      { $match    : { '_id' : { $in : following } } }
    , { $project  : { 'interactions' : 1 } }
    , { $project  : {
          "combined": { $setUnion : [ 
            "$interactions._liked"
          , "$interactions._shared"
          , "$interactions._viewed"
        ]}
      }}
])
.exec(function (err, data) {
  if (err) return next(err);
  next(data); // Combined is returning null
})

Whist "concatenating" the array is possible, "sorting them by date" is not. You seem to be refering to information such as "date" in the referenced data in the other collection. MongoDB does not perform joins in any way, and mongoose populate is also not a join nor can it be performed in the middle of an aggregation pipeline. That said, ObjectId values contain a date component so they will at least sort in order of their creation if that is enough. — Blakes Seven
– Blakes Seven, Commented Sep 30, 2015 at 3:14
@BlakesSeven, sorry I wasn't more specific. Sorting by order of creation is what I need. Can you please show how the concatenating is done, or point me to a resource? — dmr07
– dmr07, Commented Sep 30, 2015 at 3:19

Blakes Seven · Accepted Answer · 2015-09-30 05:54:15Z

2

If all the Object _id values are "unique" then $setUnion is your best option. It is of course not "ordered" in any way as it works with a "set", and that does not guarantee order. But you can always unwind and $sort.

[
    { "$project": {
        "combined": { "$setUnion": [ 
            { "$ifNull": [ "$interactions._liked", [] ] },
            { "$ifNull": [ "$interactions._shared", [] ] },
            { "$ifNull", [ "$interactions._viewed", [] ] }
        ]}
    }},
    { "$unwind": "$combined" },
    { "$sort": { "combined": 1 } },
    { "$group": {
        "_id": "$_id",
        "combined": { "$push": "$combined" }
    }}
]

Of course again since this is a "set" of distinct values you can do the old way instead with $addToSet, after processing $unwind on each array:

[
    { "$unwind": "$interactions._liked" },
    { "$unwind": "$interactions._shared" },
    { "$unwind": "$interactions._viewed" },
    { "$project": {
        "interactions": 1,
        "type": { "$const": [ "liked", "shared", "viewed" ] }
    }}
    { "$unwind": "$type" },
    { "$group": {
        "_id": "$_id",
        "combined": {
            "$addToSet": {
                "$cond": [
                   { "$eq": [ "$type", "liked" ] },
                   "$interactions._liked",
                   { "$cond": [
                       { "$eq": [ "$type", "shared" ] },
                       "$interactions._shared",
                       "$interactions._viewed"
                   ]}
                ]
            }
        }
    }},
    { "$unwind": "$combined" },
    { "$sort": { "combined": 1 } },
    { "$group": {
        "_id": "$_id",
        "combined": { "$push": "$combined" }
    }}
]

But still the same thing applies to ordering.

Future releases even have the ability to concatenate arrays without reducing to a "set":

[
    { "$project": {
        "combined": { "$concatArrays": [ 
            "$interactions._liked",
            "$interactions._shared",
            "$interactions._viewed"
        ]}
    }},
    { "$unwind": "$combined" },
    { "$sort": { "combined": 1 } },
    { "$group": {
        "_id": "$_id",
        "combined": { "$push": "$combined" }
    }}
]

But still there is no way to re-order the results without procesing $unwind and $sort.

You might therefore consider that unless you need this grouped across multiple documents, that the basic "contenate and sort" operation is best handled in client code. MongoDB has no way to do this "in place" on the array at present, so per document in client code is your best bet.

But if you do need to do this grouping over multiple documents, then the sort of approaches as shown here are for you.

Also note that "creation" here means creation of the ObjectId value itself and not other properties from your referenced objects. If you need those, then you perform a populate on the id values after the aggregation or query instead, and of course sort in client code.

edited Sep 30, 2015 at 5:54

answered Sep 30, 2015 at 3:33

Blakes Seven

50.5k14 gold badges132 silver badges136 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

dmr07 Over a year ago

Thanks for the answer! Having some trouble, I might be doing something wrong - combined is returning null for me. Please see question update.

Blakes Seven Over a year ago

@danm07 Please make sure you are matching something that actually has all arrays present. You can wrap each element with $ifNull to return an empty array where it does not exist. Also loose the prior $project. It does nothing useful anyway and is probably your SQL thinking that makes you think you need it. Added the $ifNull wrapping as an example.

dmr07 Over a year ago

I figured as much. I was trying to update the schema -.-' Out of curiosity, do you know if Mongoose automatically updates its schema when I make a change to it, or do I need to do a manual schema update?

Blakes Seven Over a year ago

@danm07 Application restarts are always required for code changes. Nothing to to with mongoose, but compilers in general.

Collectives™ on Stack Overflow

MongoDB: Concatenate Multiple Arrays

1 Answer 1

4 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

4 Comments

Your Answer

Sign up or log in

Post as a guest

Related