Python Selenium: How to click on see-more buttons in a loop

Question

I am trying to scrap the website https://www.wlw.de/de/suche?employeeCounts=200%2B_50-199&q=blechbearbeitung&supplierTypes=Dienstleister_Hersteller with multiple repetitive divs elements that each has a see-more button to show the hidden data in the div I tried to find buttons using Selenium in Python with this code

for i in range(len(company_names)):
    seemore_button = driver.find_elements(By.CLASS_NAME, 'toggle-button')[i]
    seemore_button.click()
    time.sleep(2)

Where company_names is the name of each div and I am using its length to loop on all divs but it doesn't work, when I run this code the chrome driver acts as clicking on buttons but when it finishes I find that not all the buttons have been clicked I thought at first that it may be because of the page can't load as fast as the code clicks the buttons so I added a sleep time of 2 seconds and the same result, sometimes the buttons are clicked and sometimes no

Would be great if you could provide some more details (HTML/URL/ Is there always a button to click,...) to clarify. Based on your information it is not possible, to investigate what exactly went wrong. Thanks — HedgeHog
– HedgeHog, Commented Sep 3, 2022 at 10:32

undetected Selenium · Accepted Answer · 2022-09-03 21:22:51Z

-1

To click on all the show more links in a loop you can use either of the following locator strategies:

Using CSS_SELECTOR:

driver.get('https://www.wlw.de/de/suche?employeeCounts=200%2B_50-199&q=blechbearbeitung&supplierTypes=Dienstleister_Hersteller')
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "a#CybotCookiebotDialogFooterButtonAcceptAll>button"))).click()
elements = WebDriverWait(driver, 20).until(EC.visibility_of_all_elements_located((By.CSS_SELECTOR, "a.toggle-button span")))
for elem in elements:
    elem.click()

Using XPATH:

driver.get('https://www.wlw.de/de/suche?employeeCounts=200%2B_50-199&q=blechbearbeitung&supplierTypes=Dienstleister_Hersteller')
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//a[@id='CybotCookiebotDialogFooterButtonAcceptAll']/button"))).click()
elements = WebDriverWait(driver, 20).until(EC.visibility_of_all_elements_located((By.XPATH, "//a[@class='toggle-button']//span")))
for elem in elements:
    elem.click()

Note : You have to add the following imports :

from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC

answered Sep 3, 2022 at 21:22

undetected Selenium

194k44 gold badges304 silver badges387 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

RF1991 Over a year ago

Dear @undetectedSelenium can you look at my question? your vast knowledge can solve my issue better

Collectives™ on Stack Overflow

Python Selenium: How to click on see-more buttons in a loop

1 Answer 1

1 Comment

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Linked

Related