node.js deferred misunderstanding

Question

I`m learning q.js and trying to query 3 collections simultaneously with its help (avoiding callback hell):

var Q = require('q')
var deferred = Q.defer();

users() is a wrapper of db.collection.find()

var users = function (){
    Auth.listUsers({role:'user'}, call)
    return deferred.promise
}

call() is a shorthand for exporting promises

var call = function (err,data){
    if (err) {
        deferred.reject(err);
    } else {
        deferred.resolve(data);
    }
}

loop() is main loop, which gets cursor and loops through entries

var loop = function (result) {
    var list = []
    var promises = [];
    result.each(function (err,data){
        if (err) {
            deferred.reject(err);
        } else {
            deferred.resolve(data);
            promises.push(deferred.promise)
            console.log('promises_:', promises) // <- internal
        }
    })
    console.log('promises:', promises) // <- external
    return Q.all(promises)
}

code:

users()
  .then(loop)
  .then(function(ful){
            console.log('ful:', ful);  // <- here should come the queries to other collections
        })

result of console logging at the end:

promises: []    //external is empty
ful: []
promises_: [ [object Object] ]  // internal is being filled
promises_: [ [object Object], [object Object] ]

As you can see, the callback of .each is executed later than pushing promises to array. I believe it can be made by using result.toArray() instead of .each, but how can it be done with the help of .each loop?

result is a cursor, returned by mongodb driver after db.collection.find() call (http://mongodb.github.io/node-mongodb-native/api-generated/collection.html#find).

Where are you getting each() from? Have you tried it with the Javascript function forEach()? It might work better for you. Also, I'm not sure that I follow the result.each() call having the function (err,data) does your each call with an error? — barry-johnson
– barry-johnson, Commented Mar 3, 2014 at 15:33
Where are you getting each of those deferred variables from? — Bergi
– Bergi, Commented Mar 3, 2014 at 15:42
Got it, thanks - I am not a mongo person, but if I am following this correctly, I think your general problem is that the .each() method is getting executed asynchronously, but isn't part of your promise scheme. You might want to use Q.ninvoke, Q.denodeify, or some similar strategy so that the each() is returning a promise. Also, and I know you are clearly stripping out some code, but as written, you just have one deferred you are operating on. Inside your loop, you are just populating the array with multiple copies of the same promise, which in the line prior line you are resolving. — barry-johnson
– barry-johnson, Commented Mar 3, 2014 at 16:29
You are right, .each() is async. Couldn't understand the Q functions (.fcall, .ncall, etc.). Can you recommend a good tutorial or a detailed manual on this library? I was not satisfied with its homepage. — Schtz
– Schtz, Commented Mar 3, 2014 at 19:42

Community · Accepted Answer · 2020-06-20 09:12:55Z

1

var deferred = Q.defer();
…
deferred.resolve(…);
…
deferred.resolve(…);
call() is a shorthand for exporting promises

That will not work! You need to create a new deferred for each promise that you want. However, you shouldn't use Deferreds anyway! Instead, use the many Node callback helper functions.

As you can see, the callback of .each is executed later than pushing promises to array. I believe it can be made by using result.toArray() instead of .each, but how can it be done with the help of .each loop?

It cannot, unless you know beforehand how often each will be called and how many promises will need to be created. Q.all is a bit useless in here, since the promises are not created at once and execute their tasks in parallel - instead, they are a stream.

You really should use toArray here, to get a single callback with which you resolve the promise.

Well, there is a way, but it's ugly and less efficient than toArray. You can have one deferred that will always wait, and is only resolved with a promise for the rest of the stream.

function loop(result) {
    var deferred = Q.defer();
    result.each(function (err,data){
        if (err) {
            deferred.reject(err);
        } else if (data == null) {
            deferred.resolve([]); // end of the stream
        } else {
            var nextDeferred = Q.defer();
            deferred.resolve(nextDeferred.promise.then(function(rest) {
                return [data].concat(rest);
            }));
            deferred = nextDeferred;
        }
    })
    return deferred.promise;
}

edited Jun 20, 2020 at 9:12

CommunityBot

11 silver badge

answered Mar 3, 2014 at 16:33

Bergi

671k162 gold badges1k silver badges1.5k bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Schtz Over a year ago

Thanks for your attention. I will try .toArray(). Can it be, that someArray.forEach(callbackFunction()) is executed syncronously? In mongodb driver .each() is documented to be async.

Bergi Over a year ago

Yes, forEach != each.

Collectives™ on Stack Overflow

node.js deferred misunderstanding

1 Answer 1

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related