2

I have some code that I cant get my head around, I am trying to return an array of object using a callback, I have a function that is returning the values and then pushing them into an array but I cant access this outside of the function, I am doing something stupid here but can't tell what ( I am very new to Node.JS )

                    for (var index in res.response.result) {
                    var marketArray = [];
                    (function () {
                        var market = res.response.result[index];
                        createOrUpdateMarket(market, eventObj , function (err, marketObj) {
                            marketArray.push(marketObj)
                            console.log('The Array is %s',marketArray.length) //Returns The Array is 1.2.3..etc
                        });
                        console.log('The Array is %s',marketArray.length) // Returns The Array is 0

                    })();

                }
3
  • FYI: At first this may not appear as a duplicate to hat question, but it actually is one of the many variations of the same problem: The failure to understand how asynchronous code works. The linked question provides a canonical explanation and multiple solutions. Commented Sep 26, 2015 at 18:43
  • 1
    @Tomalak - there are several aspects of this question that are not covered in the dup you originally marked it of. Yes, the core async problem is covered in other answers, but the various solutions available in node.js and the issues with where marketArray is initialized are not covered there. Commented Sep 27, 2015 at 14:20
  • @jfriend00 Fair enough. I would say that the general warning jshint emits ("don't make functions within a loop") provides a good hint about about what's wrong here, but I admit that getting to the point of recognition for that particular problem isn't trivial. (Upvoted your answer to balance the unexplained downvotes.) Commented Sep 27, 2015 at 17:49

1 Answer 1

5

You have multiple issues going on here. A core issue is to gain an understanding of how asynchronous responses work and which code executes when. But, in addition to that you also have to learn how to manage multiple async responses in a loop and how to know when all the responses are done and how to get the results in order and what tools can best be used in node.js to do that.

Your core issue is a matter of timing. The createOrUpdateMarket() function is probably asynchronous. That means that it starts its operation when the function is called, then calls its callback sometime in the future. Meanwhile the rest of your code continues to run. Thus, you are trying to access the array BEFORE the callback has been called.

Because you cannot know exactly when that callback will be called, the only place you can reliably use the callback data is inside the callback or in something that is called from within the callback.

You can read more about the details of the async/callback issue here: Why is my variable unaltered after I modify it inside of a function? - Asynchronous code reference

To know when a whole series of these createOrUpdateMarket() operations are all done, you will have to code especially to know when all of them are done and you cannot rely on a simple for loop. The modern way to do that is to use promises which offer tools for helping you manage the timing of one or more asynchronous operations.

In addition, if you want to accumulate results from your for loop in marketArray, you have to declare and initialize that before your for loop, not inside your for loop. Here are several solutions:

Manually Coded Solution

var len = res.response.result.length;
var marketArray = new Array(len), cntr = 0;
for (var index = 0, index < len; index++) {
    (function(i) {
        createOrUpdateMarket(res.response.result[i], eventObj , function (err, marketObj) {
            ++cntr;
            if (err) {
                // need error handling here
            }
            marketArray[i] = marketObj;
            // if last response has just finished
            if (cntr === len) {
                // here the marketArray is fully populated and all responses are done
                // put your code to process the marketArray here
            }
        });
    })(index);
}

Standard Promises Built Into Node.js

// make a version of createOrUpdateMarket that returns a promise
function createOrUpdateMarketAsync(a, b) {
    return new Promise(function(resolve, reject) {
        createOrUpdateMarket(a, b, function(err, marketObj) {
            if (err) {
                reject(err);
                return;
            }
            resolve(marketObj);
        });
    });
}

var promises = [];
for (var i = 0; i < res.response.result.length; i++) {
    promises.push(createorUpdateMarketAsync(res.response.result[i], eventObj));
}
Promise.all(promises).then(function(marketArray) {
    // all results done here, results in marketArray
}, function(err) {
    // an error occurred
});

Enhanced Promises with the Bluebird Promise library

The bluebird promise library offers Promise.map() which will iterate over your array of data and produce an array of asynchronously obtained results.

// make a version of createOrUpdateMarket that returns a promise
var Promise = require('bluebird');
var createOrUpdateMarketAsync = Promise.promisify(createOrUpdateMarket);

// iterate the res.response.result array and run an operation on each item
Promise.map(res.response.result, function(item) {
    return createOrUpdateMarketAsync(item, eventObj);
}).then(function(marketArray) {
    // all results done here, results in marketArray
}, function(err) {
    // an error occurred
});

Async Library

You can also use the async library to help manage multiple async operations. In this case, you can use async.map() which will create an array of results.

var async = require('async');
async.map(res.response.result, function(item, done) {
    createOrUpdateMarker(item, eventObj, function(err, marketObj) {
        if (err) {
            done(err);
        } else {
            done(marketObj);
        }
    });
}, function(err, results) {
    if (err) {
        // an error occurred
    } else {
        // results array contains all the async results
    }
});
Sign up to request clarification or add additional context in comments.

4 Comments

Added multiple different code options for solving this problem.
I'm completely baffled as to why the downvotes? Please explain - can't improve my answer without knowing why you don't like it.
Thanks for this detailed answer, Its helped me a lot in understanding what went wrong and what to do about it
This answer is very thorough, I don't understand why you would get downvotes, +1

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.