2

I am trying to use a headless browser in .Net and I need it to execute JavaScript because the information I need to read with it is loaded via AJAX. After some research I believe my best option is HtmlUnit. I have followed the instructions in Steven Sanderson's blog http://blog.stevensanderson.com/2010/03/30/using-htmlunit-on-net-for-headless-browser-automation/ in the main this works great. However, as hard as I try I cannot get it to wait for the JavaScript to load. Essentially after selecting something from a selectbox an AJAX request fires and populated another selectbox. HtmlUnit never sees the new options.

I have tried combinations of all of the examples in the FAQ http://htmlunit.sourceforge.net/faq.html#AJAXDoesNotWork so I can only assume that it is a problem with the fact it has been converted to run in .Net. I am hoping someone has some experience in this who can shed some light on what is wrong. I feel like I am going the right way with HtmlUnit and it has been recommended on here before Is there a headless browser for .NET (for automated Web-UI-Tests)? and headless internet browser? but I have reached a dead end.

So if anyone has either a better choice for a headles browser that supports JavaScript, and ideas how to get HtmlUnit to wait for the JavaScript or even just some advice on how to debug this further.

Thanks.

Simon

Improve this question
1
  • I don't suppose you could integrate jQuery and its document.ready() function? Might take some tweaking but this way the browser would wait for the document to load completely before HtmlUnit tries to do its thing. Commented Mar 27, 2012 at 14:14

1 Answer 1

Reset to default
3
+100

I think you need to use webClient.waitForBackgroundJavaScript() or webClient.waitForBackgroundJavaScriptStartingBefore().

You can also see these links that might help you little.

  1. HTMLUnit doesn't wait for Javascript
  2. HtmlUnit doesn't load externally referenced javascript
  3. htmlunit javascript question
Improve this answer
Sign up to request clarification or add additional context in comments.

1 Comment

Sadly I have tried both of those and neither work. Because of the evidence pointing to the fact that they should work I can only come the the conclusion that it is something to do with how the website is built and not something I can resolve within HTMLUnit.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.