Loop to Open A WebPage with PhantomJS (With Different Parameters Per Loop Iteration) Not Working

Question

When I try to use the following loop to open a page:

      console.log("Just before time, endTime and loopnumber gets declared.");
      var time, endTime;
      var loopnumber = 6;
      console.log("time declared as '"+time+"', loopnumber as '"+ loopnumber+"'");
      for (time = 1490543999999999, endTime = time - 518400000000 * 10; time > endTime; time -= 518400000000) {
        console.log("inside loop. time declared as '"+time+"', loopnumber as '"+ loopnumber+"'");
        var mintime = time - 518399999999;

        page.open('https://myactivity.google.com/item?min='+mintime+'&max='+time+'&product=5,6,7,10,11,12,13,15,19,20,25,27,28', function () {
        console.log("inside callback");
        page.render('/Users/jMac-NEW/Documents/FILEMAKER OCLC/login_test33.1_brkpoint'+loopnumber+'.png');
        fs.write('/Users/jMac-NEW/Documents/FILEMAKER OCLC/login_test33.1_brkpoint'+loopnumber+'html.html',page.content,'w');

      });
      loopnumber++;
      }

console.log("inside loop. time declared as '"+time+"', loopnumber as '"+ loopnumber+"'"); gets called, but not the console.log inside the callback function of page.open. page.render and fs.write also does not work (I do not see the expected result file in my directory.)

How come?

EDIT: If it's just because page.open doesn't have enough time, how can I give it enough time? with setInterval()?

EDIT2: What I am trying to get the code to do is to call myactivity.google.com/item with a different set of arguments for each iteration of the loop, save the webpage as a .png file and an .html file.

Yes, I have imported fs before this code segment, just that it isn't shown here.

How fast does your loop iterating? (sorry, won't calculate that myself :) It's possible that page.open just doesn't have enought time to open a page and issue a callback. — Vaviloff
– Vaviloff, Commented Mar 27, 2017 at 8:33
That largely depends on what you're trying to do. Maybe edit the question with description of your task? (but keep the code to show what you've tried) — Vaviloff
– Vaviloff, Commented Mar 27, 2017 at 8:40
you simply load the next page before the first one is finished, that's why you will have only have 1 callback at the end. also, did you import fs var fs = require('fs')? — Alex K
– Alex K, Commented Mar 27, 2017 at 8:49
ok I have included a description @Vaviloff, and yes Alex I have imported it. — forgodsakehold
– forgodsakehold, Commented Mar 27, 2017 at 12:58

Artjom B. · Accepted Answer · 2017-03-27 17:58:16Z

Perhaps the easiest way is to have the next page load be trigger by the callback from the previous page load. That way you know that the previous page has finished loading (note however, that loaded doesn't necessarily mean rendered and if there's a bunch of Javascript running on your page to render something - like a chart for example, you still my be rendering your screen before it's finished)

Something like this:

var time = 1490543999999999;
var endTime = time - 518400000000 * 10;
var mintime = time - 518399999999;

page.open('https://myactivity.google.com/item?min=' + mintime + '&max=' + time + '&product=5,6,7,10,11,12,13,15,19,20,25,27,28', callback);

function callback() {
  console.log("inside callback");
  page.render('/Users/jMac-NEW/Documents/FILEMAKER OCLC/login_test33.1_brkpoint' + loopnumber + '.png');
  fs.write('/Users/jMac-NEW/Documents/FILEMAKER OCLC/login_test33.1_brkpoint' + loopnumber + 'html.html', page.content, 'w');
  time -= 518400000000;
  if (time > endTime) {
    var mintime = time - 518399999999;
    page.open('https://myactivity.google.com/item?min=' + mintime + '&max=' + time + '&product=5,6,7,10,11,12,13,15,19,20,25,27,28', callback);
  }
}

Collectives™ on Stack Overflow

Loop to Open A WebPage with PhantomJS (With Different Parameters Per Loop Iteration) Not Working

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related