Waiting for async function to complete inside map

Question

consider the following simple code code:

await Promise.all(arrayOfObjects.map(async (obj) => {
    return await someAsyncFunctionOnObj(obj);
}));

My problem is, arrayOfObjects, and someAsyncFunctionOnObj, take too much memory while executing, because the loop doesn't wait for the execution to finish, instead it calls someAsyncFunctionOnObj(obj), on each one, and waits till all are resolved, not necessary in order, this causes OOM crash. Iv'e tried using a recursive async function, which does solve the order problem, but still causes OOM crash.

The flow I want to achieve is a synchronous loop, meaning

await someAsyncFunctionOnObj(obj1); // wait for it to finish
await someAsyncFunctionOnObj(obj2); // wait for it to finish
...

Any suggestions on how to correctly implement it?

First of all, it should be return await someAsyncFunctionOnObj(obj); But you really don't need async/await if you use Promise.all. Or just await Promise.all(arrayOfObjects.map(someAsyncFunctionOnObj)); — dfsq
– dfsq, Commented Oct 18, 2017 at 11:12
Why do you use Promise.all at all when you rather want to chain them? — Rob
– Rob, Commented Oct 18, 2017 at 11:16
@Robert that was exactly my question, what is the best way to chain them. — vrachlin
– vrachlin, Commented Oct 18, 2017 at 11:23
You probably want the example from here, just replace RSVP with Promise in the first code example. It explains it pretty well too — tocallaghan
– tocallaghan, Commented Oct 18, 2017 at 11:32

Wing · Accepted Answer · 2018-03-09 17:23:40Z

3

Solution

async function queueAsyncFns(fns) {
  const values = [];

  await fns.reduce((previous, current, index, array) => {
    const thenable = index === 1 ? previous() : previous;
    return thenable.then(value => {
      values.push(value);
      return index === array.length - 1 ? current().then(value => values.push(value)) : current();
    });
  });

  return values;
}

Example

const anArray = [1, 2, 3];
const doSomething = async (id) => await fetch(`https://jsonplaceholder.typicode.com/users/${id}`).then(res => res.json());

queueAsyncFns(anArray.map((val) => () => doSomething(val))).then((val) => console.log(val));

The above function should solve your issue. Here's a brief overview of what it does:

queueAsyncFns accepts an array of functions that returns the result of calling an async function. This array is reduced by calling each function and returning the Promise to the next call of the reducer. With each iteration the value of the async call is accumulated into an array called values which is returned after all items have been iterated through.

The accurate behaviour of the function can be determined visually by looking at the waterfall graph when running the example. You can see each network call is only made after the previous one has been completed.

edited Mar 9, 2018 at 17:23

answered Oct 18, 2017 at 13:05

Wing

9,8104 gold badges44 silver badges54 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

vrachlin Over a year ago

Thank you for that explanation, this better answers my question!

Faly · Accepted Answer · 2017-10-18 11:30:58Z

1

If you want to wait for someAsyncFunctionOnObj(obj1) to finish before doing the same but with the next object (obj2, obj3, ...), I think you have to chain your promises:

var promises = arrayOfObjects.map(obj => someAsyncFunctionOnObj(obj));
await promises.reduce((m, o) => m.then(() => o), Promise.resolve());

edited Oct 18, 2017 at 11:30

answered Oct 18, 2017 at 11:20

Faly

13.4k2 gold badges21 silver badges39 bronze badges

2 Comments

vrachlin Over a year ago

Could you explain the second line?

Faly Over a year ago

It takes all the promises in var promises and waits for them, one after the other, by the use of array reduce (array reduce use m as memory of the previous promise so we can do .then on m and return the next promise o, o becoming the new value of m and so on)

tocallaghan · Accepted Answer · 2017-10-18 11:49:33Z

0

(async function() {
  async function executeSequentially() {
    const tasks = [1,2]
    const total = []
    for (const fn of tasks) {
      const res = await fetch(endpoint);
      const res2 = await res.json();
      console.log(res2[0]);
      total.push(...res2);
    }

    return total;
  }

  const res = await executeSequentially();
  console.log(res);
})();

answered Oct 18, 2017 at 11:49

tocallaghan

9,9221 gold badge30 silver badges23 bronze badges

Collectives™ on Stack Overflow

Waiting for async function to complete inside map

3 Answers 3

Solution

1 Comment

2 Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

Solution

1 Comment

2 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related