How to design parallel multi-request in nodejs using Axios?

Question

I want to design a method for 100 users to get self data,

API 1

(xxx.xxx.xxx/api/login)
POST body {"id":"xxx","password":"xxxx"},
response: {"token":"xxx"}

API 2

(xxx.xxx.xxx/api/getData)
POST body {"token":"xxx"}

How to design this case in nodejs using axios?

Idea 1:

Using a "for loop"

for(int i=0;i<100;i++){
    axios.post(api1,postdata1).then{
      //if api1 excuted successful
      axios.post(api2,postdata2)
    }
}

Idea 2:

axios.all

Idea 3:

await Promise.all + await axios.post

Thanks for reading.

Christian · Accepted Answer · 2020-09-30 08:03:52Z

1

To make the requests in parallel use Promise.all

const posts = [{api: api1, postdata: postdata1}, ... , { api: apin, postdata: postdatan}].map({a, p} => { return axios.post(a, p); });
await Promise.all(posts);

answered Sep 30, 2020 at 8:03

Christian

7,9025 gold badges42 silver badges66 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Anuj Pancholi · Accepted Answer · 2020-09-30 08:44:30Z

Idea 1 just seems too inefficient and cumbersome.

Idea 2 is deprecated, as per the axios github page

Idea 3 could be considered somewhat viable, however, given that you have to do this for 100 users concurrently (100 login calls + 100 getData calls = at least 200 API calls), I would not recommend this. It may put too much pressure on the server hosting the API and will allocate too much memory for concurrent requests, might cause issues and be slow.

What I would do is use a library called Bluebird.js, which has a .map() function which takes an array, a callback function and an object of options as params (one of which is concurrency, the number of promises to be executed at once) and executes the promises (requests, in this case) in batches.

Your code could look something like:

const BluebirdPromise = require("bluebird");

BluebirdPromise.map(usersForLogin, function(userData) {
    // Promise.map awaits for returned promises as well.
    return axios.post(`http://api.url.here/api/login`,{
        id: userData.id,
        password: userData.password
    });
},{
    concurrency: 10
}).then(function(loginResponseData) {
    console.log("Logins done");
    
    return BluebirdPromise.map(usersForLogin,function(userData,index){
        return axios.post(`http://api.url.here/api/login`,{
            id: userData.id,
            token: loginResponseData[index].token
        })
    },{
        concurrency: 10
    })

}).then(function(resultsArray){
    console.log(`All results fetched: resultsArray`);
});

This will hit the login API for 100 users in the usersForLogin array, 10 at a time, so it won't cause one big spike in the server, or the client.

Collectives™ on Stack Overflow

How to design parallel multi-request in nodejs using Axios?

2 Answers 2

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related