Selenium: struggle clicking a second button on a page

Question

I am on a page in which I need to click on a variable's logo/name/arrow (3 button options). The variable is called HISCO occupation. HTML from URL: https://icem.data-archive.ac.uk/#step2:

<div class="clearfix ice-tab-wrapper" id="workhistoryunit_category" ng-class="{active:tabActive[child.name]}">
:: before
<button class="fa fa-certificate ice-icon" ng-click="toggleTab(child.name, $event)">
:: before
<span class= "sr-only ng-binding"> HISCO OCCUPATION </span>
</button>
< button .....> </button>

My entire code so far:

ETUPfrom selenium.webdriver.support.ui import WebDriverWait
import selenium
from selenium import webdriver as wd
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC

chrome_path = r'C:\webdrivers\chromedriver.exe'

webD=wd.Chrome(executable_path=chrome_path)
webD.get('https://icem.data-archive.ac.uk/#step1')


## STEP 1: SELECTING A YEAR, HERE 1851

webD.find_element_by_xpath('/html/body/div/section/section[1]/article[1]/div[1]/div/div[1]/label/input').click()


## STEP 2: SELECTING ENGLAND

WebDriverWait(webD, 20).until(EC.element_to_be_clickable(
    (By.XPATH, '//b[@id = "country_england"]/preceding-sibling::input'))).click()


## STEP 3: MOVE ON TO VARIABLES

webD.find_element_by_xpath('//html/body/div/section/section[1]/article[2]/div/div/button').click()


## STEP 4.1: COUNTIES, OPEN MENU

WebDriverWait(webD, 20).until(EC.element_to_be_clickable(
    (By.XPATH, '//*[@id="county_category"]/button[1]'))).click()


## STEP 4.2 COUNTIES, MORE VARIABLES
WebDriverWait(webD, 20).until(EC.element_to_be_clickable(
    (By.XPATH, '//*[@id="county_category"]/div[2]/div/div[2]/button'))).click()

## VERSION 1, SELECTION 1 COUNTY/ 1 HISCO_OCC
## STEP 4.3 COUNTIES, SELECT VARIABLES

WebDriverWait(webD, 20).until(EC.element_to_be_clickable(
    (By.XPATH, '/html/body/div[3]/div/div/div[3]/div[1]/label/input'))).click()

## STEP 4.3.2 CLICK APPLY

WebDriverWait(webD, 20).until(EC.element_to_be_clickable((By.XPATH, "//button[contains(.,'Apply')]"))).click()


# STEP 5, HISCO OCCUPATION, OPEN MENU

WebDriverWait(webD, 20).until(EC.element_to_be_clickable((By.ID, "workhistoryunit"))).click()

As of now, I have the following (unsuccessful) code trying to click on either of these buttons:

WebDriverWait(webD, 20).until(EC.element_to_be_clickable((By.XPATH, '//*[@id="workhistoryunit_category"]/button[1]'))).click()

or

webD.find_element_by_xpath('//*[@id="workhistoryunit_category"]/button[1]').click()

The action prior to clicking this button is closing a window (applying a selection).

This code is based on the code from a previous analogous click on the same page, just another variable (hence different ID) that works:

WebDriverWait(webD, 20).until(EC.element_to_be_clickable((By.XPATH, '//*[@id="county_category"]/button[1]'))).click()

Any ideas what the problem maybe?

You've been using only xpath as the locator in both the above examples, why not try class or other locators to find the element? — Ananth
– Ananth, Commented Feb 1, 2021 at 17:06
Welcome to Stack Overflow! Please read why a screenshot of code is a bad idea. Paste the code and properly format it instead. This includes HTML. — JeffC
– JeffC, Commented Feb 1, 2021 at 17:08
Use following xpath to click WebDriverWait(webD, 20).until(EC.element_to_be_clickable((By.XPATH, "//button[./b[text()='HISCO classified occupation']]"))).click() — KunduK
– KunduK, Commented Feb 1, 2021 at 17:20
@JeffC, thank you for the information. I've edited accordingly. — Rye Rye
– Rye Rye, Commented Feb 1, 2021 at 17:24

KunduK · Accepted Answer · 2021-02-01 17:39:01Z

2

Use following XPath to click on the button.

WebDriverWait(driver,20).until(EC.element_to_be_clickable((By.XPATH,"//button[./span[text()='HISCO classified occupation']]"))).click()

answered Feb 1, 2021 at 17:39

KunduK

33.4k5 gold badges19 silver badges42 bronze badges

Sign up to request clarification or add additional context in comments.

8 Comments

Rye Rye Over a year ago

Thank you for your help! It seems like it shound work... But still no success... Any additional information I could give to be more precise?

KunduK Over a year ago

@RyeRye : what error are you getting while clicking?

Rye Rye Over a year ago

No error! The button is simply not clicking, so I cannot go forward in my automation.

KunduK Over a year ago

@RyeRye : I got the catch. Application is too fast and selenium unable to sync properly. use time.sleep(3) before clicking It should work. I have just tested with code.

Rye Rye Over a year ago

So embed time.sleep(3).click()? I'll try! Thank you very much for all the effort!

|

JeffC · Accepted Answer · 2021-02-01 19:13:20Z

1

# STEP 5, HISCO OCCUPATION, OPEN MENU


WebDriverWait(webD, 20).until(
    EC.presence_of_element_located((By.XPATH, '//span[@class = "ice-allow-download-alert ng-animate ng-hide-remove ng-hide-remove-active"]')))

WebDriverWait(webD, 20).until(
    EC.invisibility_of_element_located((By.XPATH, '//span[@class = "ice-allow-download-alert ng-animate ng-hide-remove ng-hide-remove-active"]')))

WebDriverWait(webD, 20).until(
    EC.presence_of_element_located((By.XPATH, '//b[text()[contains(.,"HISCO classified occupation")]]/..'))).click()

Try this there is small tool tip being displayed for few seconds that is removing the reference , so use this code first to check its presence then its invisibility and then click that button

There is a green tool tip that comes up same time the tab you want to click is loaded , so it resets the element reference and causes stale element exception. You need to wait for that elemnt to disappear first. But as the element is displayed only for few seconds we cannot find its locator . To find its locator open inspect and right click the html element and add break on > subtree modification and keep resuming execution still you get that tool tip

edited Feb 1, 2021 at 19:13

JeffC

26.4k5 gold badges35 silver badges57 bronze badges

answered Feb 1, 2021 at 17:46

PDHide

20.1k2 gold badges43 silver badges57 bronze badges

12 Comments

Rye Rye Over a year ago

Smart approach but still doesn't click... It's a bit frustrating, it seems many of these propositions should work...

PDHide Over a year ago

@RyeRye it work just add a sleep after that line you shuld add that time.sleep(5) ,

PDHide Over a year ago

@DebanjanB Updated the answer , there is a tool tip that shows for few seconds that causes the stale element for the target button . It was hectic to use dom break point and find its locators thats why used time.sleep() instead . But anyways added the wait answer

PDHide Over a year ago

@RyeRye there is a green tool tip that comes up same time the tab you want to click is loaded , so it resets the element reference and causes stale element exception. You need to wait for that elemnt to disappear first. But as the element is displayed only for few seconds we cannot find its locator . To find its locator open inspect and right click the html element and add break on > subtree modification and keep resuming execution still you get that tool tip :D uff long story and effort i know

Rye Rye Over a year ago

@PDHide I'm very impressed!! I'm really just starting out so this was way out of reach for me. Thank you very very much for helping out!!

|

undetected Selenium · Accepted Answer · 2021-02-01 17:54:32Z

0

To click on the element with text as HISCO OCCUPATION you need to induce WebDriverWait for the element_to_be_clickable() and you can use either of the following Locator Strategies:

Using XPATH:

WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//button[@class='ice-tab-button ng-binding']/b[@class='ng-binding' and contains(., 'HISCO classified occupation')]"))).click()

Note: You have to add the following imports :

from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC

edited Feb 1, 2021 at 17:54

answered Feb 1, 2021 at 17:29

undetected Selenium

194k44 gold badges304 silver badges387 bronze badges

5 Comments

Rye Rye Over a year ago

Thank you for your help, sadly neither solution is successful in clicking the button...

undetected Selenium Over a year ago

@RyeRye Checkout the updated answer and let me know the results.

Rye Rye Over a year ago

Other users have suggester it isn't actually the locator strategy being an issue but the speed and suggested to include a time.sleep(5) right before, I'm experimenting with that. I'll try your suggestion next and will let you know. Thank you!

undetected Selenium Over a year ago

@RyeRye I understand you wanted to avoid WebDriverWait as I suggested in your initial question which you have avoided. time.sleep() isn't a good practice. Honestly sooner or later you have to implement WebDriverWait

Rye Rye Over a year ago

Hmmm I think I'm a bit confused, I was using both sleep and wait... I think you have a point there. In any case, nothing has yet worked, including your suggestion...

Collectives™ on Stack Overflow

Selenium: struggle clicking a second button on a page

3 Answers 3

8 Comments

12 Comments

5 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

8 Comments

12 Comments

5 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related