Multiple async http get requests

Question

I'm struggling with asynchronous events in Node.js.

My app is doing a first HTTP GET request and then based on some information found in the body of the response it is looping a certain amount of times.

At each iteration it calls a function which sends other HTTP GET requests and returns a result pushed in the elements array.

My problem is that I want to console.log() the elements array when all the http requests are done. Here's my code :

http.get('mysite.com', function(res) {
    var body = '';
    res.on('data', function(chunk) {
        body += chunk;
    });
    res.on('end', function() {
        var last = 5,
            elements = [];
        for(var i=0; i < last; i++) {
            elements.push(myRequest(i));
        }
        console.log(elements);
        // Do some stuff with the elements array after it has been populated
    });
}).on('error', function(error) {
    console.log(error);
});

var myRequest = function(i) {
    http.get('mysite.com/page' + i, function(res) {
        var body = '';
        res.on('data', function(chunk) {
            body += chunk;
        });
        res.on('end', function() {
            // Do parsing here
            return parsed;
        });
    }).on('error', function(error) {
        console.log(error);
    });
};

I thought about using the async module but I'm not sure how to use it in this use case.

Thanks in advance !

A http get is by definition asynchronuous. Dont really see what you want to do. — asgoth
– asgoth, Commented Dec 23, 2012 at 11:43
I just want to do something with the results of all the http requests when they are finished. — m_vdbeek
– m_vdbeek, Commented Dec 23, 2012 at 12:18

mindandmedia · Accepted Answer · 2012-12-23 11:47:23Z

2

I would use the async library and maybe use a queue. but if you want to do it by hand, without having tested anything below and not really sure it's the cleanest way, but it might give you ideas.

pass a callback to your request-function:

var myRequest = function(i, cb){
   ...
   res.on('end', function(){
      cb(parsed);
   });
}

and above onEnd-call would look more like this:

var last = 5, elements = [];

var rep = function(i){
  if( i < last ){
     myRequest(i, function( result ){
        elements.push(result);              
        rep(i+1);
     });
  } else {
    console.log(elements);
    // Do some stuff with the elements array after it has been populated
  }
}

res.on('end', function() {
   rep(0);
});

using async would come out a lot nicer looking. gotta run now though...

answered Dec 23, 2012 at 11:47

mindandmedia

6,8551 gold badge26 silver badges33 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

m_vdbeek Over a year ago

Hum it doesn't seem to work. The app just hangs and does nothing / doesn't print anything.

m_vdbeek Over a year ago

From what I can see, the "end" event of the first http.get request is never fired.

m_vdbeek Over a year ago

I fixed some scope issued and it works now ! Thank you very much.

Collectives™ on Stack Overflow

Multiple async http get requests

1 Answer 1

3 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

3 Comments

Your Answer

Sign up or log in

Post as a guest

Related