Python Selenium - StaleElementReferenceException: stale element not found

Question

I am trying to click on the next button at the bottom of the page to access the following page as in the image below:

However, when I reach page 3 I receive the following error:

selenium. common. exceptions. StaleElementReferenceException: Message: stale element reference: stale element not found

from the code line

next_button = wait.until(EC.element_to_be_clickable(span_element[1])).click()

Overall, the code I am using for this looks like

region_content = driver.find_element(By.CLASS_NAME, "pager__item--next")
span_element = region_content.find_elements(By.CSS_SELECTOR, "a > span")  
next_button = wait.until(EC.element_to_be_clickable(span_element[1])).click()

and the HTML code where the button can be found is the following:

<li class="pager__item pager__item--next">
          <a href="?field_headquarters_of_company_target_id=All&amp;ajax_page_state%5Btheme%5D=bootstrap&amp;ajax_page_state%5Blibraries%5D=addtoany/addtoany%2Casset_injector/css/css_injector%2Cbootstrap/popover%2Cbootstrap/tooltip%2Ccore/drupal.autocomplete%2Ccore/drupal.autocomplete%2Ccore/drupal.autocomplete%2Ccore/drupal.autocomplete%2Ccore/drupal.autocomplete%2Ccore/drupal.autocomplete%2Ccore/drupal.autocomplete%2Ccore/drupal.autocomplete%2Ccore/drupal.autocomplete%2Ccore/html5shiv%2Cextlink/drupal.extlink%2Cgoogle_analytics/google_analytics%2Csuperfish/superfish%2Csuperfish/superfish_hoverintent%2Csuperfish/superfish_smallscreen%2Csuperfish/superfish_style_white%2Csuperfish/superfish_supersubs%2Csuperfish/superfish_supposition%2Csuperfish/superfish_touchscreen%2Csystem/base%2Cviews/views.ajax%2Cviews/views.ajax%2Cviews/views.ajax%2Cviews/views.ajax%2Cviews/views.ajax%2Cviews/views.ajax%2Cviews/views.ajax%2Cviews/views.ajax%2Cviews/views.ajax%2Cviews/views.module%2Cviews/views.module%2Cviews/views.module%2Cviews/views.module%2Cviews/views.module%2Cviews/views.module%2Cviews/views.module%2Cviews/views.module%2Cviews/views.module%2Cwebform_bootstrap/webform_bootstrap&amp;ajax_page_state%5Btheme_token%5D=&amp;_wrapper_format=drupal_ajax&amp;title=&amp;field_company_category_primary_target_id&amp;field_market_cap_jul302023_value=&amp;field_company_website_uri=&amp;page=3" title="Go to next page" rel="next">
            <span class="visually-hidden">Next page</span>
            <span aria-hidden="true">Next ›</span>
          </a>
        </li>

I have tried with a try/except statement to refresh the page in case of stale element but still does not work. What is a bit puzzling to me is that to go from page 1 to 2 and from 2 to 3 there is no error.

Would anyone be willing to help out?
Thank you

Is the page publicly accessible? If yes, what's the url?

Barry the Platipus
– Barry the Platipus

2023-08-28 11:01:33 +00:00
Commented Aug 28, 2023 at 11:01 — Barry the Platipus
– Barry the Platipus, Commented Aug 28, 2023 at 11:01
yes it is! You can access it here. Thanks :)

econnoob5
– econnoob5

2023-08-28 11:06:48 +00:00
Commented Aug 28, 2023 at 11:06 — econnoob5
– econnoob5, Commented Aug 28, 2023 at 11:06

Patryk Smoliński · Accepted Answer · 2023-08-28 11:06:35Z

-1

What browser you use? From my experience, it's best to click elements by xpath.

Try something like:

while True:
    try:
        driver.find_element('xpath', 'some_xpath').click()
        break
    except:
        pass

edited Aug 28, 2023 at 11:06

answered Aug 28, 2023 at 11:06

Patryk Smoliński

12 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Python Selenium - StaleElementReferenceException: stale element not found

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related