How can I use WebDriverWait from Selenium properly through Python?

Question

I just thought I'd add an edit now that this has been resolved. Replacing those two time.sleep() took my program from 180 seconds down to 30. WebDriverWait creates a substantial improvement in runtime.

I am just trying to determine if I'm setting up WebDriverWait correctly. This is my working script, I use time.sleep()

for x,sequence in enumerate(table.find_elements_by_xpath('//*[@id="gwzSngrOrderResultPanelRoot"]/table/tbody/tr/td[9]'),1):
        driver.find_element_by_xpath(f'//*[@id="gwzSngrOrderResultPanelRoot"]/table/tbody/tr[{x}]/td[9]/span[2]').click()
        time.sleep(5)
        element = WebDriverWait(driver, 5).until(EC.presence_of_element_located((By.XPATH, '//*[@id="gwzViewResultsModalDialog"]/div/div/div[3]/button')))
        seq_info=driver.find_element_by_xpath('//*[@id="gwzViewResultsModalDialog"]/div/div/div[2]/div')
        seq_list.append([seq_info.text])
        driver.find_element_by_xpath('//*[@id="gwzViewResultsModalDialog"]/div/div/div[3]/button').click()
        time.sleep(5)

In short, it goes through a table, clicks a button which opens a pop up, extracts text from pop up, and closes said pop up. I have to wait for the pop up to open, and to fully close. I'm currently using time.sleep(), but I'm trying to switch to WebDriverWait. This is how I've implemented it.

for x,sequence in enumerate(table.find_elements_by_xpath('//*[@id="gwzSngrOrderResultPanelRoot"]/table/tbody/tr/td[9]'),1):
        driver.find_element_by_xpath(f'//*[@id="gwzSngrOrderResultPanelRoot"]/table/tbody/tr[{x}]/td[9]/span[2]').click()
        element = WebDriverWait(driver, 5).until(EC.presence_of_element_located((By.XPATH, '//*[@id="gwzViewResultsModalDialog"]/div/div/div[3]/button')))
        seq_info=driver.find_element_by_xpath('//*[@id="gwzViewResultsModalDialog"]/div/div/div[2]/div')
        seq_list.append([seq_info.text])
        driver.find_element_by_xpath('//*[@id="gwzViewResultsModalDialog"]/div/div/div[3]/button').click()
        time.sleep(5)

However, the above does not work. And I receive this error:

driver.find_element_by_xpath('//*[@id="gwzViewResultsModalDialog"]/div/div/div[3]/button').click()
selenium.common.exceptions.ElementNotInteractableException: Message: element not interactable

This is gone if I return to time.sleep(), thus making me think I must have setup my WebDriverWait incorrectly. The waiting is for the opening and closing of the browser, so we have to wait till the button appears, thus I put the xpath of the button itself in the WebDriverWait. Is this the proper setup?

Edit: Thank you @DebanjanB for the answer. However, I have come across another issue in trying to remove my time.sleep(), this is what I currently have.

for x,sequence in enumerate(table.find_elements_by_xpath('//*[@id="gwzSngrOrderResultPanelRoot"]/table/tbody/tr/td[9]'),1):
        driver.find_element_by_xpath(f'//*[@id="gwzSngrOrderResultPanelRoot"]/table/tbody/tr[{x}]/td[9]/span[2]').click()
        time.sleep(5)
        #WebDriverWait(driver, 10).until(EC.presence_of_element_located((By.XPATH, '//*[@id="gwzViewResultsModalDialog"]/div/div/div[2]/div')))
        seq_info=driver.find_element_by_xpath('//*[@id="gwzViewResultsModalDialog"]/div/div/div[2]/div')
        seq_list.append([seq_info.text])
        WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.XPATH, '//*[@id="gwzViewResultsModalDialog"]/div/div/div[3]/button'))).click()

The time.sleep() above is waiting for the pop up to open (from the previous click), before it extracts the data. If I remove the time.sleep with the below #WebDriver, it still moves forward, but for some reason seq_info.text is now blank (it no longer finds the text). I don't quite understand why this is happening. This is not a click or button, I just wanted to check that the pop up is open before I extract the info from it.

imgur.com/a/fKsiw6a I'm clicking on the clipboard in seq_file, that opens up a pop up on the same page with the info to extract — samman
– samman, Commented Jul 6, 2020 at 14:44

Peter Mortensen · Accepted Answer · 2022-10-31 23:06:38Z

0

Short answer, no, though syntactically correct, but you aren't using WebDriverWait optimally.

Along with WebDriverWait you are also using time.sleep().

time.sleep(secs)

time.sleep(secs) suspends the execution of the current thread for the given number of seconds. The argument may be a floating point number to indicate a more precise sleep time. The actual suspension time may be less than that requested because any caught signal will terminate the sleep() following execution of that signal’s catching routine. Also, the suspension time may be longer than requested by an arbitrary amount because of the scheduling of other activity in the system.

You can find a detailed discussion in How to sleep webdriver in python for milliseconds

Moreover,

In the for loop as you intend to iterate instead of /tr[{x}] you need //tr[{x}]
To collect the desired text you you need to use visibility_of_element_located().
<button> are interactive in nature, so instead of presence_of_element_located() you need to use element_to_be_clickable() just when you need to interact with them.

A probable solution:

for x,sequence in enumerate(table.find_elements_by_xpath('//*[@id="gwzSngrOrderResultPanelRoot"]/table/tbody/tr/td[9]'),1):
      WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.XPATH, '//*[@id="gwzSngrOrderResultPanelRoot"]/table/tbody//tr[{x}]/td[9]/span[2]'))).click()
      seq_list.append(WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.XPATH, "//*[@id='gwzViewResultsModalDialog']/div/div/div[2]/div"))).text)
      WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.XPATH, '//*[@id="gwzViewResultsModalDialog"]/div/div/div[3]/button'))).click()

edited Oct 31, 2022 at 23:06

Peter Mortensen

31.4k22 gold badges110 silver badges134 bronze badges

answered Jul 6, 2020 at 14:34

undetected Selenium

194k44 gold badges304 silver badges387 bronze badges

Sign up to request clarification or add additional context in comments.

9 Comments

samman Over a year ago

I was only using time.sleep() because I wanted to change things 1 step at a time (I know time.sleep() works, but I don't know how to use WebDriverWait, thus if the program failed, I knew exactly where/why it failed). I agree WebDriverWait is far better than time.sleep(), however it is not working in my above situation. Is it because I am also using time.sleep() as well? And if I am not using it optimally, how may I optimize its usage?

undetected Selenium Over a year ago

@samman Checkout the updated answer and let me know the status.

samman Over a year ago

Oh, so can WebDriverWait also click the object? I.E. Replace the .click() option? Also, why //tr instead of /tr? /tr was working for me?

undetected Selenium Over a year ago

@samman 1) element_to_be_clickable() is effective, efficient and proven, there shouldn't be any doubts. 2) click() should be always invoked through WebDriverWait, no matter it's a <span> or a <button>.

undetected Selenium Over a year ago

@samman Checkout the updated answer and let me know the status.

|

Collectives™ on Stack Overflow

How can I use WebDriverWait from Selenium properly through Python?

1 Answer 1

time.sleep(secs)

9 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

time.sleep(secs)

9 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related