Running MongoDB Queries in Map/Reduce

Question

Is it possible to run MongoDB commands like a query to grab additional data or to do an update from with in MongoDB's MapReduce command. Either in the Map or the Reduce function?

Is this completely ludicrous to do anyways? Currently I have some documents that refer to separate collections using the MongoDB DBReference command.

Thanks for the help!

Gates VP · Accepted Answer · 2011-10-04 18:35:24Z

4

Is it possible to run MongoDB commands... from within MongoDB's MapReduce command.

In theory, this is possible. In practice there are lots of problems with this.

Problem #1: exponential work. M/R is already pretty intense and poorly logged. Adding queries can easily make M/R run out of control.
Problem #2: context. Imagine that you're running a sharded M/R and you are querying into an unsharded collection. Does the current context even have that connection?

You're basically trying to implement JOIN logic and MongoDB has no joins. Instead, you may need to build the final data in a couple of phases by running a few loops on a few sets of data.

answered Oct 4, 2011 at 18:35

Gates VP

45.3k11 gold badges108 silver badges108 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

whobutsb Over a year ago

Thank you for the reply. Basically I have a situation where I have a group of products for merchants and I wanted to do some aggregation of the product prices. Attached to the Products I have the MerchantID from the DB Ref. My Key in my emit would be the MerchantID and then emit the product price / quantity. Just to tidy things up for the final result collection I wanted to add in the Merchant Name rather than the MerchantID. The Merchant Names are stored in a separate collection.

Gates VP Over a year ago

In your case, you have two options. (1) -> denormalize and store the merchant name with the product data. (2) -> after the M/R, update the output collection with the merchant name. Takes an extra step, but you'll get the right result and it's easier to shard. Option (3) -> query the required names when running queries against the data. It's a manual join, but if you want to display more than just the name on that screen, you'll end up doing this anyways.

Collectives™ on Stack Overflow

Running MongoDB Queries in Map/Reduce

1 Answer 1

2 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related