MongoDB Aggregation Pipeline Multiple Groups Complicating Pipeline

Question

Here is my Document structure:

{
 "_id" : ObjectId("50dcd7ff4de274a2c4a31df0"),
 "seq_name" : "169:D18M6ACXX:1:1111:17898:82486:GTGACA_10",
 "raw_seq" : "TTGACCTGAGGAGACGGTGACCAGGGTTCCCTGGCCCCAGTAGTCAACGGGAGTTAGACTTCTCGCACAGTAATAAACAGCCGTGTCCTCGGCTCTCAGGCTGTTCATTTGCAGA",
 "seq_aa" : "LQMNSLRAEDTAVYYCARSLTPVDYWGQGTLVTVSSGQ",
 "cdr3_seq" : "GCGAGAAGTCTAACTCCCGTTGACTAC",
 "cdr3_seq_aa" : "ARSLTPVDY",
 "cdr3_seq_len" : 27,
 "cdr3_seq_aa_len" : 9,
 "vg" : "IGHV3-48*03",
 "dg" : "IGHD3-10*02R",
 "jg" : "IGHJ4*02",
 "donor" : 10
}

I really enjoy MongoDB framework but I'm having trouble with this grouping pipeline and since I can't $out to another collection yet. I can do this multi-grouping pipeline.

db.collection.aggregate({$match:{cdr3_seq_aa_len:{$gt:3}},
   {$group:{_id:$cdr3_seq_aa,other_set:{$addToSet:$cdr3_seq_aa_len}}},
   {$group:{_id:$other_set,sum:{$sum:1}}})

Which gives me how many unique$cdr3_seq_aa's there are grouped by length.

{ id:40, sum:1002031,
  id:41, sum:1949402,....

However The first operation I would like to do is group by donor. So I can first know how many unique cdr3_seq_aa strings there are among each donor. Then I would like to group it by length and count how many strings group with the length.

Sambhav Sharma · Accepted Answer · 2015-03-22 21:28:29Z

5

If I understand the question correctly, this is what you're looking for. The key concept is that you can construct compound _id's from multiple fields.

db.collection.aggregate(
[
    {$match: {cdr3_seq_aa_len: {$gt: 3}}},
    {$group: 
         {
              _id: {donor: "$donor", cdr3_seq_aa: "$cdr3_seq_aa"},
              donor_cdr3_seq_aa_count: {$sum: 1},
              cdr3_seq_aa_len: {$first: "$cdr3_seq_aa_len"}
         }
    },
    {$group:
         {
             _id: {donor: "$_id.donor", len: "$cdr3_seq_aa_len"},
             num_strings_with_this_length: {$sum: 1},
             total_doc_count_by_length:
                  {$sum: "$donor_cdr3_seq_aa_count"}
         }
    }
])

edited Mar 22, 2015 at 21:28

Sambhav Sharma

5,86910 gold badges59 silver badges101 bronze badges

answered Jan 18, 2013 at 4:29

mjhm

16.8k10 gold badges47 silver badges56 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

Eve Freeman Over a year ago

Some issues: 1. Need another } at end of {$match 2. Need group operation for cdr3_seq_aa_len in first $group ($first works) 3. Need $cdr3_seq_aa_len in last $group _id, instead of $cdr_seq_aa_len

Eve Freeman Over a year ago

No problem. You beat me to the answer so I just ran your query through my test data, which made it easy to debug. :)

jwillis0720 Over a year ago

Thank you. The only thing I needed was to actually not group by donors anymore as I wanted them to be grouped by their length, and then ask for the unique strings. So the workflow would be. First grab all unique strings among donors. Then combine the donors and ask for all total unique strings grouped by length. I just simply had to take out the donor _id grouping and it worked perfectly!

Collectives™ on Stack Overflow

MongoDB Aggregation Pipeline Multiple Groups Complicating Pipeline

1 Answer 1

3 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

3 Comments

Your Answer

Sign up or log in

Post as a guest

Related