Handling multiple parallel HTTP requests in Node.js

Question

I know that Node is non-blocking, but I just realized that the default behaviour of http.listen(8000) means that all HTTP requests are handled one-at-a-time. I know I shouldn't have been surprised at this (it's how ports work), but it does make me seriously wonder how to write my code so that I can handle multiple, parallel HTTP requests.

So what's the best way to write a server so that it doesn't hog port 80 and long-running responses don't result in long request queues?

To illustrate the problem, try running the code below and loading it up in two browser tabs at the same time.

var http = require('http');
http.createServer(function (req, res) {
    res.setHeader('Content-Type', 'text/html; charset=utf-8');
    res.write("<p>" + new Date().toString() + ": starting response");
    setTimeout(function () {
        res.write("<p>" + new Date().toString() + ": completing response and closing connection</p>");
        res.end();
    }, 4000);
}).listen(8080);

blog.mixu.net/2011/02/01/understanding-the-node-js-event-loop — Paul
– Paul, Commented Jun 16, 2013 at 2:49

Peter Lyons · Accepted Answer · 2013-06-16 02:51:11Z

17

You are misunderstanding how node works. The above code can accept TCP connections from hundreds or thousands of clients, read the HTTP requests, and then wait the 4000 ms timeout you have baked in there, and then send the responses. Each client will get a response in about 4000 + a small number of milliseconds. During that setTimeout (and during any I/O operation) node can continue processing. This includes accepting additional TCP connections. I tested your code and the browsers each get a response in 4s. The second one does NOT take 8s, if that is how you think it works.

I ran curl -s localhost:8080 in 4 tabs as quickly as I can via the keyboard and the seconds in the timestamps are:

54 to 58
54 to 58
55 to 59
56 to 00

There's no issue here, although I can understand how you might think there is one. Node would be totally broken if it worked as your post suggested.

Here's another way to verify:

for i in 1 2 3 4 5 6 7 8 9 10; do curl -s localhost:8080 &;done

edited Jun 16, 2013 at 2:51

answered Jun 16, 2013 at 2:44

Peter Lyons

147k32 gold badges285 silver badges281 bronze badges

Sign up to request clarification or add additional context in comments.

6 Comments

Andrew Over a year ago

You are correct. It's Chrome that's doing it. For me at least, the second request won't be issued until the first one is returned.

Peter Lyons Over a year ago

Even chrome should normally allow up to 6 concurrent connections per origin.

Andrew Over a year ago

I know. That's why I was surprised and assumed it was Node.

edi9999 Over a year ago

Do you know why Chrome couldn't send the 2 requests at the same time ?

Peter Lyons Over a year ago

Frankly I don't believe the OP's claim that chrome is serializing the request as per my above comment. The full interactions between a browser and a server are complex. But without code to repetitive the issue i cant comment further.

|

user3286817 · Accepted Answer · 2014-03-08 16:47:40Z

4

Your code can accept multiple connections because the job is done in callback function of the setTimeout call.

But if you instead of setTimeout do a heavy job... then it is true that node.js will not accept other multiple connections! SetTimeout accidentally frees the process so the node.js can accept other jobs and you code is executed in other "thread".

I don't know which is the correct way to implement this. But this is how it seems to work.

answered Mar 8, 2014 at 16:47

user3286817

1892 silver badges2 bronze badges

Comments

Amar T · Accepted Answer · 2017-07-01 19:30:35Z

2

Browser blocks the other same requests. If you call it from different browsers then this will work parallelly.

answered Jul 1, 2017 at 19:30

Amar T

3651 silver badge9 bronze badges

Comments

NAVIN · Accepted Answer · 2017-05-22 12:24:42Z

I used following code to test request handling

app.get('/', function(req, res) {
  console.log('time', MOMENT());  
  setTimeout( function() {
    console.log(data, '          ', MOMENT());
    res.send(data);
    data = 'changing';
  }, 50000);
  var data = 'change first';
  console.log(data);
});

Since this request doesn't take that much processing time, except for 50 sec of setTimeout and all the time-out were processed together like usually do.

Response 3 request together-

time moment("2017-05-22T16:47:28.893")
change first
time moment("2017-05-22T16:47:30.981")
change first
time moment("2017-05-22T16:47:33.463")
change first
change first            moment("2017-05-22T16:48:18.923")
change first            moment("2017-05-22T16:48:20.988")
change first            moment("2017-05-22T16:48:23.466")

After this i moved to second phase... i.e., what if my request takes so much time to process a sync file or some thing else that take time.

app.get('/second', function(req, res) {
    console.log(data);
    if(req.headers.data === '9') {
        res.status(200);
        res.send('response from api');
    } else {
        console.log(MOMENT());
        for(i = 0; i<9999999999; i++){}
        console.log('Second MOMENT', MOMENT());
        res.status(400);
        res.send('wrong data');
    }

    var data = 'second test';
});

As my first request was still in process so my second didn't get accepted by Node. Thus i got following response of 2 request-

undefined
moment("2017-05-22T17:43:59.159")
Second MOMENT moment("2017-05-22T17:44:40.609")
undefined
moment("2017-05-22T17:44:40.614")
Second MOMENT moment("2017-05-22T17:45:24.643")

Thus For all Async functions theres a virtual thread in Node and Node does accept other request before completing previous requests async work like(fs, mysql,or calling API), however it keeps it self as single thread and does not process other request until all previous ones are completed.

Collectives™ on Stack Overflow

Handling multiple parallel HTTP requests in Node.js

4 Answers 4

6 Comments

Comments

Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

4 Answers 4

6 Comments

Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related