Python Selenium StaleElementReferenceException while iterating through Select options

Question

I have been tasked with providing a selenium test (using Python) on a webpage for which I did not write the source code (I don't know much about web development, but from what I was told, it was written using D3 and JQuery). The website generates graphs, and there are a few dropdown menus with options. I'm trying to create every possible graph with these options. To do so, I create a Select object from the dropdown menu WebElement and extract the text of each option in the dropdown. Later on, I use this text to recursively go through each dropdown and enter each valid option to generate the graphs. However, I sometimes (not always, and it seems to be relatively random) encounter a StaleElementReferenceException while trying to extract this list of options, and it is because of the below code (it happens on the options.append(op.get_attribute("text")) line). I have been searching previous questions and documentation on this exception, and I have yet to find an answer than helps me.

#find all of the options 
# create list of string options before, so as to avoid StaleElementReferenceException's
# as much as possible
el = driver.find_element(By.ID,node.element)
d = Select(el)
options = [] 
for op in d.options:
    try:
        options.append(op.get_attribute("text"))
    except StaleElementReferenceException:
        print >>sys.stderr, 'StaleElementReferenceException getting text from \'' + node.element + '\' element'

Any input is very helpful! Thank you.

EDIT: Here is an HTML snippet example of one of the dropdowns:

<select class="dropdown" id="datatype" name="datatype" style="display: none;">
    <option value="solar_radiation" selected="selected">solar radiation</option>
    <option value="windspeed">windspeed</option>
    <option value="airtemperature">airtemperature</option>
    ...
</select>

Thanks for the advice! I have actually looked through that documentation already, and I understand pretty well what the exception means. I'm just not sure why I'm getting it in this instance, or how to avoid it. — awerchniak
– awerchniak, Commented Dec 12, 2016 at 19:14
does selecting an option refresh the page or change the DOM? If so, that is causing your problem. you are using a reference to an element that has changed — Corey Goldberg
– Corey Goldberg, Commented Dec 12, 2016 at 19:35
I believe it changes the DOM, as elements in other lists depend on the option you choose. In the above code, though, I'm not even selecting any options. All I'm doing is iterating through the available options and saving their values in a list — awerchniak
– awerchniak, Commented Dec 12, 2016 at 20:35

Linh Nguyen · Accepted Answer · 2016-12-14 14:53:10Z

1

+50

I am suspicious d = Select(el)

If you just want to get all the texts, maybe you can do like:

optionNodes = driver.find_element_by_id("datatype").find_elements_by_tag_name("option")
options = []
for op in optionNodes:
    options.append(op.get_attribute("text"))

If the code above doesn't work (strange, though), maybe you can try to execute script, like:

driver.execute_script("return Array.apply(null, document.getElementById('datatype').options).map(function (option) {return option.text})");

edited Dec 14, 2016 at 14:53

answered Dec 12, 2016 at 22:55

Linh Nguyen

1,1388 silver badges14 bronze badges

Sign up to request clarification or add additional context in comments.

14 Comments

awerchniak Over a year ago

Unfortunately this didn't solve the problem. Still same StaleElementReferenceException on that get_attribute line

Linh Nguyen Over a year ago

Could you provide the html for the Select?

awerchniak Over a year ago

selenium.common.exceptions.WebDriverException: Message: unknown error: Cannot read property 'options' of null Perhaps I am executing this wrong, I've never used Javascript before.

Linh Nguyen Over a year ago

I used the id = 'datatype' in the code sample, but maybe you have a different id for your select?

awerchniak Over a year ago

It would appear that the execute script is working now... I must have typed something in wrong somewhere. Thanks for all of your help.

|

Guy · Accepted Answer · 2016-12-15 05:36:42Z

1

A dirty workaround will be to iterate over the options list with indexes and repopulate the list when you encounter the exception

el = driver.find_element(By.ID,node.element)
select = Select(el)
options = select.options
options_texts = []
for i in range len(options)):
try:
    options_texts.append(options[i].get_attribute("text"))
except StaleElementReferenceException:
    el = driver.find_element(By.ID,node.element)
    select = Select(el)
    options = select.options

answered Dec 15, 2016 at 5:36

Guy

51.2k10 gold badges49 silver badges96 bronze badges

Collectives™ on Stack Overflow

Python Selenium StaleElementReferenceException while iterating through Select options

2 Answers 2

14 Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

14 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related