NodeJS eventloop execution order

Question

When sending several requests serially, it seems like

callbacks are executed when all requests are sent.
requests seems to get added into a queue but not actually getting executed before the loop is done.

Example

var http = require('http');

var i=0;
var postData = [];

while (i++ < 12) {
    var largeObject = [
        'this' + i,
        'that' + i,
        'then' + i
    ];

    postData.push(largeObject);

    if (postData.length >= 2) {
        console.log('request');
        var options = {
            hostname: 'httpbin.org',
            port: 80,
            path: '/post',
            method: 'POST',
            headers: {
                'Content-Type': 'application/json',
            }
        };
        var req = http.request(options, function(res) {
            console.log(res.statusCode);
            res.on('data', function(chunk) {
                console.log(chunk.toString());
            });

            res.on('end', function() {
                console.log('finished reading response');
            });
        });
        req.end(JSON.stringify(postData), function() {
            console.log('request stream finished');
        });

        postData = [];
    }
}

The output here would look like

request
request
request
request
request
request
request stream finished
request stream finished
request stream finished
request stream finished
request stream finished
request stream finished
200
{
  "args": {}, 
  "data": "[[\"this7\",\"that7\",\"then7\"],[\"this8\",\"that8\",\"then8\"]]", 
  "files": {}, 
  "form": {}, 
  "headers": {
    "Content-Length": "53", 
    "Content-Type": "application/json", 
    "Host": "httpbin.org"
  }, 
  "json": [
    [
      "this7", 
      "that7", 
      "then7"
    ], 
    [
      "this8", 
      "that8", 
      "then8"
    ]
  ], 
  "origin": "xxxxx", 
  "url": "http://httpbin.org/post"
}

finished reading response
200
{
  "args": {}, 
  "data": "[[\"this1\",\"that1\",\"then1\"],[\"this2\",\"that2\",\"then2\"]]", 
  "files": {}, 
  "form": {}, 
  "headers": {
    "Content-Length": "53", 
    "Content-Type": "application/json", 
    "Host": "httpbin.org"
  }, 
  "json": [
    [
      "this1", 
      "that1", 
      "then1"
    ], 
    [
      "this2", 
      "that2", 
      "then2"
    ]
  ], 
  "origin": "xxxx", 
  "url": "http://httpbin.org/post"
}
...

Is there any way to finish one request before the next one is getting executed?

I don't really care about the order, it's more about memory - I'd like to get rid of large objects I am posting

e.g.

request1
response1
request2
response2
request3
response3
request4
request5
response5
response4

would be absolutely fine. Any suggestion?

soyuka · Accepted Answer · 2016-03-08 16:08:40Z

1

Of course, just use some control-flow modules like async or bluebird.

As you don't care about order, I'd advise you to use either async#parallel or bluebird#map. The bluebird#map has a concurrency argument, which can be nice when you need some more control over loops and their amount (so does async#parallelLimit).

If this doesn't seem straightforward, please comment and I'll add an example.

answered Mar 8, 2016 at 16:08

soyuka

9,1554 gold badges42 silver badges55 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Michel Feldheim Over a year ago

Thank you for the answer. Maybe the question wasn't clear enough. In my real code, I am preparing a large object in each loop cycle and I'd like to submit and clear it from memory as soon as possible. I have updated above example to better reflect what I am doing. I don't see how async.parallel for example would help me with that. An example would be great.

soyuka Over a year ago

What do you mean by "submit" the object? Async.parallel for example would allow you to call a function when each request is done, and then you'll have the last callback when everything is done. IMO you can implement what you need with a simple EventEmitter.

Collectives™ on Stack Overflow

NodeJS eventloop execution order

1 Answer 1

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related