Python selenium refresh if wait more than 10s

Question

from selenium import webdriver    
driver=webdriver.Firefox() 
driver.get(url)

Sometimes the webdriver is stuck on a file or response and the page is never full-loaded so the line

driver.get(url)

is never finished. But I already got enough source code to run the rest of my code. I am wondering how can I bypass or refresh the page if the page is not full-loaded in 10 seconds.

I have tried

from selenium import webdriver
from selenium.common.exceptions import TimeoutException    
driver=webdriver.Firefox() 
driver.set_page_load_timeout(10)
while True:
    try:
        driver.get(url)
    except TimeoutException:
        print("Timeout, retrying...")
        continue
    else:
        break

but the line

driver.set_page_load_timeout(10)

always gives me following error

  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/selenium/webdriver/remote/webdriver.py", line 727, in set_page_load_timeout
'pageLoad': int(float(time_to_wait) * 1000)})
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/selenium/webdriver/remote/webdriver.py", line 238, in execute
self.error_handler.check_response(response)
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/selenium/webdriver/remote/errorhandler.py", line 193, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message:

This is nothing after Message:. I can't identify the type of error. It's weird that my laptop can't run

driver.set_page_load_timeout(10)

My next step is to click a button on the page, but that button doesn't always exist even after full-loaded. Thus I can't use explicit wait.

Thanks

Can you update the stack trace to include whatever is after "Message:"? WebDriverException is the generic base exception for selenium and can be raised for many different reasons. — Levi Noecker
– Levi Noecker, Commented Mar 24, 2017 at 16:00
There is nothing after Message: I don't know what type of error it is. — Jimmy Lee
– Jimmy Lee, Commented Mar 24, 2017 at 16:02
@JimmyLee it's not an good idea, however why not you catch this generic webdriver exception as well ? — Gaurang Shah
– Gaurang Shah, Commented Mar 24, 2017 at 16:12

Levi Noecker · Accepted Answer · 2017-03-24 17:37:32Z

5

(In your code snippet you don't define URL, but I'll assume URL is defined somewhere in your actual code.)

You could combine the retry and timeout-decorator packages for this:

from retry import retry
from timeout_decorator import timeout, TimeoutError
from selenium import webdriver
from selenium.common.exceptions import TimeoutException    

@retry(TimeoutError, tries=3)
@timeout(10)
def get_with_retry(driver, url):
    driver.get(url)


def main():
    url = "http://something.foo"

    driver=webdriver.Firefox() 
    try:
        get_with_retry(driver, url)
        foo(driver) # do whatever it is you need to do
    finally:
        driver.quit()


if __name__ == "__main__":
    main()

Note that you would need to either not set driver.set_page_load_timeout to anything, or set it to something higher than 10 seconds.

answered Mar 24, 2017 at 17:37

Levi Noecker

3,3191 gold badge19 silver badges31 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Python selenium refresh if wait more than 10s

1 Answer 1

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related