Is there a way in VBA to get elements produced by javascript?

Question

I've been doing Google detective work for a few hours (including searching StackOverflow) for a technique to allow targeting of HTML elements produced by JavaScript in VBA.

As an example, I cannot use ie.Document.getElementById(id) on this: http://www.kisfutures.com/electronic.html?page=quote&sym=NGU12&mode=i

However, that method is quite capable of finding elements on static pages, as tested on google.com.

I've played with simply getting all the InnerText from Document.body and then parsing the file for my desired TD values, but I ran into a roadblock when trying to split two values that are on different lines. As an illustration, the following was understood by Split(..) as "2040":

20
40

From what I understand, the problem with getElementById(id) is that the website's table is generated by JavaScript after the page has loaded, and thus any elements created by that JavaScript cannot be targeted by my VBA code. Is there anyway to have my VBA code see this JavaScript-generated content?

Thank you for your help!

Edit

The VBA code being used:

Function Quote(Market, Parameter)
    Set ie = CreateObject("InternetExplorer.Application")
    ie.Navigate "http://www.kisfutures.com/electronic.html?page=quote&sym=NGU12&mode=i"
    While ie.Busy
        DoEvents
    Wend
    Dim id As String
    id = "dt1_" & Market & "_" & StrConv(Parameter, vbLowerCase)
    Quote = ie.Document.getElementByID(id).InnerText
    ie.Quit
End Function

Basically I try to build an HTML element ID using the column header for the Market. So if, for example the market is "NGU12" and the column header is "Open", the ID built is: "dt1_NGU12_open", which is the ID for the TD element containing the "Open" value for market NGU12.

Rather than (or as well as) linking to a full page, can you please show a cut-down version of the VBA code in question? Wouldn't hurt to show a few lines of the JS too... Are you sure you're running your VBA after the elements have been created by JS? — nnnnnn
– nnnnnn, Commented Aug 5, 2012 at 22:33
Sorry, I'm having trouble understanding what you're doing and what you're trying to do. It sounds like you're using JS to grab the elements on the page, but you're trying to do this with VBA? Is there a way to have JavaScript write the necessary elements for Document.getElementById(..) before it is called? Write where? Before what is called? — Zhihao
– Zhihao, Commented Aug 5, 2012 at 22:34
As a side note, I'd like to mention that using innerText will be tricky. For example, Firefox does not support innerText, and uses the W3C standard textContent instead. Whitespace will also be treated differently (e.g. You won't get line breaks with FF textContent), and you will get the script tag contents as well. Apparently Opera (I haven't tried it myself) has innerText, but it's treated as textContent. I feel the only reliable way of getting consistent content is with innerHTML, but then you will be parsing markup, which is a whole other problem. — Zhihao
– Zhihao, Commented Aug 5, 2012 at 22:38
I've edited my original post with the VBA code in question. I am not running any JavaScript code myself. The website (linked in OP) generates large tables of data using JS. I am having trouble using getElementByID(id) because it does not retrieve anything on these types of dynamic pages. — adairdavid
– adairdavid, Commented Aug 5, 2012 at 22:44
Once the page has completed loading (and any js-generated elements have been added to the document) then there's no difference between "static" and generated elements as far as the DOM goes: they should be equally accessible to scripting. If you add a slight delay between the page loading and when you try accessing its elements that should be sufficient to make any dynamic content available to you. — Tim Williams
– Tim Williams, Commented Aug 5, 2012 at 23:22

mkingston · Accepted Answer · 2012-08-06 02:13:51Z

1

As Tim has mentioned there should be no difference between statically and dynamically generated content when attempting access. Try this:

Function Quote(Market, Parameter)
    Set ie = CreateObject("InternetExplorer.Application")
    ie.Navigate "http://www.kisfutures.com/electronic.html?page=quote&sym=NGU12&mode=i"
    While ie.Busy
        DoEvents
    Wend
    Application.Wait Now + TimeSerial(0,0,1) 'Pause 1 second, hopefully
                                             'the dynamic content is 
                                             'generated within this 
                                             'time-frame
    Dim id As String
    id = "dt1_" & Market & "_" & StrConv(Parameter, vbLowerCase)
    Quote = ie.Document.getElementByID(id).InnerText
    ie.Quit
End Function

There are better ways of halting execution (see Windows Sleep API) but I'll leave that to you if you need more fine-grained control.

Also, if you'd like to test whether a specific id exists after the page has been generated you could load the page manually in IE and add a bookmarklet containing the following as the URL:

javascript:void((function() {var a = document.getElementById(prompt('Please enter an id to test','dt1_NGU12_open')); if (a) {alert('Element found');} else {alert('Element not found')};})())

answered Aug 6, 2012 at 2:13

mkingston

2,74820 silver badges27 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

mkingston Over a year ago

Note that using the bookmarklet I've written here I couldn't find an element with the example id you've provided on the page you've provided. You could try using FF or Chrome to load the page and use their developer tools to inspect. I'm not sure what IE9 dev tools are like but IE8 (which I'm sadly forced to use at work) is rubbish, and it'd be a bit of work to find the id. Give it a go, let me know if you don't have any luck- could look again when I get home.

Collectives™ on Stack Overflow

Is there a way in VBA to get elements produced by javascript?

1 Answer 1

1 Comment

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Linked

Related