How do I click on clickable element with Selenium in shadow-root (closed)?

Question

An "Agree with the terms" button appears on https://www.sreality.cz/hledani/prodej/domy I am trying to go through that with a .click() using Selenium and Python. The button element is:

<button data-testid="button-agree" type="button" class="scmp-btn scmp-btn--default w-button--footer sm:scmp-ml-sm md:scmp-ml-md lg:scmp-ml-dialog">Souhlasím</button>

My approach is:

driver = webdriver.Chrome()
driver.implicitly_wait(20)

driver.get("https://www.sreality.cz/hledani/prodej/domy")

button = driver.find_element_by_css_selector("button[data-testid='button-agree']")
button.click()

Any idea what to change to make it work? Thanks! :)

you can not access it as it is not on the page source, but inside the #shadow-root (closed). stackoverflow.com/questions/71282237/… — Ajeet Verma
– Ajeet Verma, Commented Apr 12, 2023 at 7:58

Shawn · Accepted Answer · 2023-04-19 10:27:02Z

3

+50

Check the below working workaround solution:

driver = webdriver.Chrome()
driver.implicitly_wait(10)
driver.get("https://www.sreality.cz/hledani/prodej/domy")
driver.maximize_window()

# Below line creates instance of ActionChains class 
action = ActionChains(driver)
# Below line locates and stores an element which is outside the shadow-root
element_outside_shadow = driver.find_element(By.XPATH, "//div[@class='szn-cmp-dialog-container']")
# Below 2 lines clicks on the browser at an offset of co-ordinates x=5 and y=5
action.move_to_element_with_offset(element_outside_shadow, 5, 5)
action.click()
# Below 2 lines presses TAB key 9 times so that pointer moves to "Souhlasím" button and presses ENTER key once
action.send_keys(Keys.TAB * 9).perform()
action.send_keys(Keys.ENTER).perform()

Imports required:

from selenium import webdriver
from selenium.webdriver import ActionChains
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.by import By

NOTE:

This is a workaround solution, as there is no direct selenium solution to handle it.

answered Apr 19, 2023 at 10:27

Shawn

9,1774 gold badges19 silver badges39 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

Mr.Slow Over a year ago

Thanks, I will try. I also found out that workpin solutions could be: to locate and access a closest parent element and the perform an action with Tab button.

Life is complex Over a year ago

This solution worked perfectly. Thanks for posting. Definitely worth an upvote.

Life is complex Over a year ago

@Mr.Slow. There are some videos on SelectorsHub's YouTube channel on dealing with shadow root elements.

Shawn Over a year ago

@Mr.Slow - Yes, locating the element just outside #shadow-root(closed) and clicking it did not work for some reason. Locate the element and click little away from the element by offset, and then press TAB button works. Try it.

Deborisz · Accepted Answer · 2023-04-15 10:11:40Z

Your code looks correct, but the issue might be related to the timing of the click you want to execute. Page may take longer to load and you have to click the button after it is ready to be clicked. Try to use this code instead:

from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium import webdriver

driver = webdriver.Chrome()
driver.implicitly_wait(20)

driver.get("https://www.sreality.cz/hledani/prodej/domy")

wait = WebDriverWait(driver, 10)
button = wait.until(EC.element_to_be_clickable((By.CSS_SELECTOR, "button[data-testid='button-agree']")))
button.click()

We are using the WebDriverWait class to wait for the button to be clickable before clicking on it. The EC.element_to_be_clickable method checks if the element is both visible and enabled, so we are sure we can click on it. We are using a timeout of 10 seconds for the wait, but you can adjust it.

The idea came from this post: Wait for element to be clickable using python and Selenium

Yaroslavm · Accepted Answer · 2023-11-14 20:14:21Z

I know it's a bit too late, but it is possible to access shadow-root (closed) if it is not placed in cross origin iframe (in your case it is not placed).

Since Selenium implemented CDP protocol API, you can override Element class prototype method attachShadow and evaluate script on new document before page load.

So when shadow-root node would be attached in page context, it would be automatically created with status open.

However, if shadow-root (closed) is placed inside iframe from resource you are not controlling, override iframeElement.contentWindow.Element would be impossible because of CORS.

How to override Element properties

How to executed script on load via CDP Protocol

Solution without workaround for your case:

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

driver = webdriver.Chrome()
wait = WebDriverWait(driver, 10)

url = "https://www.sreality.cz/hledani/prodej/domy"

driver.execute_cdp_cmd('Page.addScriptToEvaluateOnNewDocument', {'source': """
Element.prototype._attachShadow = Element.prototype.attachShadow;
Element.prototype.attachShadow = function () {
    return this._attachShadow( { mode: "open" } );
};
"""})
driver.get(url)

closed_shadow_host = wait.until(EC.presence_of_element_located((By.CSS_SELECTOR, '.szn-cmp-dialog-container')))
shadow_root = driver.execute_script('return arguments[0].shadowRoot', closed_shadow_host)

button = shadow_root.find_element(By.CSS_SELECTOR, "button[data-testid='button-agree']")
button.click()

Collectives™ on Stack Overflow

How do I click on clickable element with Selenium in shadow-root (closed)?

3 Answers 3

4 Comments

Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

4 Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related