0

I'm new to working with async programming, so there may be something simple I'm missing out on here.

I have an express project, I'm passing an array in the body of my request.

Inside my function, I validate the body then parse the array and use a promise as I map over the array.

const games = JSON.parse(JSON.stringify(req.body.games));

const gamesMap = games.map((game) => gameSearch(game));

return Promise.all(gamesMap)
    .then(function(g) {
        // async is still running here, I want to wait until it returns 
        console.log(g); // returns [ undefined, undefined, ... ]
    });

The game search function uses puppeteer to use a headless browser to return prices of the game passed in array. However, it doesn't wait until the array is returned before Promise.all is called, so the console.log(g); above returns an undefined array. I assume it is something to do with using async await inside the gameSearch function, although I'm not sure what I am supposed to do here? Any help would be greatly appreciated.

function gameSearch(game) {
  (async () => {
    const url = '.....' + game;
    try {
      const browser = await puppeteer.launch();
      const page = await browser.newPage();
      await page.setUserAgent('Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/69.0.3497.100 Safari/537.36');
      await page.goto(url);
      const selector = '.searchRcrd';
      await page.waitForSelector(selector);

      const searchRcrds = await page.$$(selector);

      const records = [];

      for (let i = 0; i < searchRcrds.length; i++) {
        const searchRcrd = searchRcrds[i];
        const title = await searchRcrd.$eval('h1', (h1) => h1.innerText.trim());
        const buyFor = await searchRcrd.$eval('.desc .prodPrice div:nth-child(2) .priceTxt:nth-child(1)', (buy) => buy.innerText.trim());
        const inStoreFor = await searchRcrd.$eval('.desc .priceTxt:nth-child(2)', (inStore) => inStore.innerText.trim());
        const imgSrc = await searchRcrd.$eval('div.thumb > a > img', (img) => img.src.trim());

        records.push({
          'title': title,
          'buyFor': buyFor,
          'inStoreFor': inStoreFor,
          'imgSrc': imgSrc
        });
      }

      await browser.close();

      return records;
    } catch (err) {
      next(err);
    }
  })();
}
2
  • Why does your gameSearch function not return anything? What good is the IIFE inside it? And why does it still use the callback approach for errors (next(err))? Commented Oct 16, 2018 at 21:06
  • @Bergi It should return records. I was just following a tutorial online, this was how they approached this. Commented Oct 16, 2018 at 21:11

1 Answer 1

2

The return records returns from the (async () => {…})(); IIFE. Drop that and make gameSearch itself an async function that returns (a promise for) the array.

async function gameSearch(game) {
  const url = '.....' + game;
  const browser = await puppeteer.launch();
  const page = await browser.newPage();
  await page.setUserAgent('Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/69.0.3497.100 Safari/537.36');
  await page.goto(url);
  const selector = '.searchRcrd';
  await page.waitForSelector(selector);

  const searchRcrds = await page.$$(selector);

  const records = [];

  for (let i = 0; i < searchRcrds.length; i++) {
    const searchRcrd = searchRcrds[i];
    const title = await searchRcrd.$eval('h1', (h1) => h1.innerText.trim());
    const buyFor = await searchRcrd.$eval('.desc .prodPrice div:nth-child(2) .priceTxt:nth-child(1)', (buy) => buy.innerText.trim());
    const inStoreFor = await searchRcrd.$eval('.desc .priceTxt:nth-child(2)', (inStore) => inStore.innerText.trim());
    const imgSrc = await searchRcrd.$eval('div.thumb > a > img', (img) => img.src.trim());

    records.push({
      'title': title,
      'buyFor': buyFor,
      'inStoreFor': inStoreFor,
      'imgSrc': imgSrc
    });
  }

  await browser.close();

  return records;
}
Sign up to request clarification or add additional context in comments.

1 Comment

Perfect, thank you, I knew it would be something simple.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.