Running a loop within an instance of Puppeteer

Question

Just getting started with Puppeteer and i can launch the browser, go to a url, run a bunch of actions and then close the browser. What i am looking to see if i can do though is open the browser and loop over a set of actions in the same session.

I have a JSON object with urls i want to visit, so want to loop over that

// teams.js
module.exports = {
  premier_league: [
    { team_name: "Team 1", url: "https://url-of-site/team_1"},
    { team_name: "Team 2", url: "https://url-of-site/team_2"}
  ]
}

My script to launch puppeteer is as follows

// index.js
const TEAM = require('./teams');
const puppeteer = require('puppeteer');

(async () => {
  // Initialise Browser
  const browser = await puppeteer.launch({headless: false});
  const page = await browser.newPage();
  await page.setViewport({
    width: 1280,
    height: 800
  });
  await page.goto('login page');

  await page.click('login_box');
  await page.keyboard.type('username');

  await page.click('login_password');
  await page.keyboard.type('password');

  await page.click('login_button');
  await page.waitForNavigation();

  // Go To Team URL
  await page.goto('Team URL')

  await browser.close();
})();

So to loop over my JSON object I can use

Object.keys(TEAM['premier_league']).forEach(function(key) {

  // Output url of each team
  console.log(TEAM['premier_league'][key]['url'])

});

If i wrap my go to url with my loop, then page is no longer accessible

// index.js
const TEAM = require('./teams');
const puppeteer = require('puppeteer');

(async () => {
  // Initialise Browser
  const browser = await puppeteer.launch({headless: false});
  const page = await browser.newPage();
  await page.setViewport({
    width: 1280,
    height: 800
  });
  await page.goto('login page');

  await page.click('login_box');
  await page.keyboard.type('username');

  await page.click('login_password');
  await page.keyboard.type('password');

  await page.click('login_button');
  await page.waitForNavigation();

  Object.keys(TEAM['premier_league']).forEach(function(key) {

    // Go To Team URL
    await page.goto(TEAM['premier_league'][key]['url'])

  });

  await browser.close();
})();

The actual error is

await page.goto(TEAM[args][key]['url'])
    ^^^^

SyntaxError: Unexpected identifier

yue you · Accepted Answer · 2018-02-06 23:18:16Z

1

Your Object.keys callback function need to use async as well in order to use await inside. Try to change as below

Object.keys(TEAM['premier_league']).forEach( async function(key) {

    // Go To Team URL
    await page.goto(TEAM['premier_league'][key]['url'])

});

Hope it helps

answered Feb 6, 2018 at 23:18

yue you

2,2641 gold badge14 silver badges31 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

Richlewis Over a year ago

ah ok, thanks very much, good news is that i can log out the urls but as far as visiting the actual url i get Unhandled promise rejection (rejection id: 1): Error: Protocol error (Page.navigate): Target closed.

yue you Over a year ago

Put your async function into a try catch closure and check the error. Seems your page.goto() is failed for some reason

Lukáš Křivka Over a year ago

I would not do .forEach with async, it will not await the overall Object.keys loop so the code execution will continue. For sequential async/await, it is easier to use for (const ... of ...) { // await stuff }

Collectives™ on Stack Overflow

Running a loop within an instance of Puppeteer

1 Answer 1

3 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

3 Comments

Your Answer

Sign up or log in

Post as a guest

Related