Load a web page, execute its JavaScript and dump resulting HTML to a file

Question

I need to load a web page, execute its JavaScript (and all js files included with the tags) and dump resulting HTLM to a file. This needs to be done on the server. I have tried node.js with zombie.js but it seems it is too immature to work in the real world. More often than not it just throws a bogus exception while a real browser (FireFox) has no issues with the page.

My node.js code is:

var zombie = require("zombie"),
    sys = require('sys');

// Load the page
var browser = new zombie.Browser({ debug: false });
browser.visit('http://www.dba.dk', function (error, browser, status) {
    if (error) { console.log('Error:' + error.message); }
    if (!error && browser.statusCode == 200) {
        sys.puts(browser.html);
    }
});

and it exits with an exception "TypeError: Cannot call method 'toString' of null"

Jaxer is not really an option.. I need to download a 3rd party page and execute it on my server. How would I do that with Jaxer

Martijn · Accepted Answer · 2011-03-23 09:36:38Z

5

Perhaps that’s because you are using err.message whereas err is not defined? error, on the other hand, is defined.

Update

Did you check out PhantomJS?

Also, it looks like Aptana Jaxer could do what you want. To quote John Resig:

Imagine ripping off the visual rendering part of Firefox and replacing it with a hook to Apache instead - roughly speaking that's what Jaxer is.

edited Mar 23, 2011 at 9:36

answered Mar 22, 2011 at 14:24

Martijn

13.7k4 gold badges50 silver badges60 bronze badges

Sign up to request clarification or add additional context in comments.

8 Comments

Oscar Kilhed Over a year ago

If that line of code was causing the error err would have to be null which it can't be, because null evaluates to false. However, it would mean that the real error message is not logged to the console.

Martijn Over a year ago

@OscarKilhed: good point! So that can’t be the actual cause of the error. Still, the code as posted is not correct.

intellion Over a year ago

@Martijn: My fault, I have corrected the code. However this mistake has no effect on the actual error. Are there any other tools serving a similar purpose? I tried to use jsdom but couldn't find a way to execute page's javascript.

Martijn Over a year ago

@andrew: I’ve added a few suggestions to my answer

intellion Over a year ago

@Martijn: Checking out PhantomJS. Looks promising. I will let you know about results later today

|

Collectives™ on Stack Overflow

Load a web page, execute its JavaScript and dump resulting HTML to a file

1 Answer 1

8 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

8 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related