click on element with classname selenium

Question

I am trying to scrape the opening hours of bars from a website. There is a list of bars which then if you navigate to you the opening hours are available. I am having an issue clicking on an element when it has a class name.

I have written the code to get the hours from one venuw, however, I am unable to navigate to each venue from the first link.

This code works when I get the hours for one venue

from selenium import webdriver

driver = webdriver.Firefox()

driver.get('https://www.designmynight.com/london/bars/soho/six-storeys')

hours = driver.find_element_by_xpath('//li[@id="hours"]')
hours.click()

hoursTable = driver.find_elements_by_css_selector("table.opening-times tr")
for row in hoursTable:
    print(row.text)

The issue is when I try to navigate to this page to I am unable to click into each link. Can anyone see what I am doing wrong?

from selenium import webdriver

driver = webdriver.Firefox()

driver.get('https://www.designmynight.com/london/search-results#!?type_of_venue=512b2019d5d190d2978c9ea9&type_of_venue=512b2019d5d190d2978c9ea8&type_of_venue=512b2019d5d190d2978c9ead&type_of_venue=512b2019d5d190d2978c9eaa&type_of_venue=512b2019d5d190d2978c9eab&type=&q=&radius=')

venue = driver.find_element_by_xpath('//a[@class="ng-binding"]')
venue.click()

//this should then lead me to the following link ->
driver.get('https://www.designmynight.com/london/bars/soho/six-storeys')

hours = driver.find_element_by_xpath('//li[@id="hours"]')
hours.click()

hoursTable = driver.find_elements_by_css_selector("table.opening-times tr")
for row in hoursTable:
    print(row.text)

Andersson · Accepted Answer · 2017-03-18 18:00:11Z

All links with ng-binding class names are generated dynamically, so you need to wait untill link appears in DOM and it's also clickable:

from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.support.ui import WebDriverWait as wait
from selenium.webdriver.common.by import By
from selenium import webdriver

driver = webdriver.Firefox()

driver.get('https://www.designmynight.com/london/search-results#!?type_of_venue=512b2019d5d190d2978c9ea9&type_of_venue=512b2019d5d190d2978c9ea8&type_of_venue=512b2019d5d190d2978c9ead&type_of_venue=512b2019d5d190d2978c9eaa&type_of_venue=512b2019d5d190d2978c9eab&type=&q=&radius=')

venue = wait(driver, 10).until(EC.element_to_be_clickable((By.XPATH, '//a[@class="ng-binding"]')))
venue.click()

But if you want to follow each link I'd suggest you not to click those links, but get the list of references and then open each one as below:

xpath = '//a[@class="ng-binding"]'
wait(driver, 10).until(EC.element_to_be_clickable((By.XPATH, xpath)))
links = [venue.get_attribute('href') for venue in driver.find_elements_by_xpath(xpath)]

for link in links:
    driver.get(link)
    hours = driver.find_element_by_xpath('//li[@id="hours"]')
    hours.click()
    hoursTable = driver.find_elements_by_css_selector("table.opening-times tr")
    for row in hoursTable:
        print(row.text)

DrNightmare · Accepted Answer · 2017-03-18 15:44:40Z

0

I feel the issue is that the driver is trying to locate the element before it is available in the DOM. Try waiting until the element is present, instead of directly trying to find it. You could replace this line:

venue = driver.find_element_by_xpath('//a[@class="ng-binding"]')

with:

venue = WebDriverWait(driver, 10).until(EC.presence_of_element_located((By.XPATH, '//a[@class="ng-binding"]')))

Note that you'll need to do a few imports to do this:

from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

Source : http://selenium-python.readthedocs.io/waits.html

answered Mar 18, 2017 at 15:44

DrNightmare

1561 gold badge2 silver badges8 bronze badges

2 Comments

DrNightmare Over a year ago

It does. After replacing that line as I mentioned above, does it redirect to that link you provided (designmynight.com/london/bars/soho/six-storeys)?

peter flanagan Over a year ago

and do you know how i can loop through each of those links?

Collectives™ on Stack Overflow

click on element with classname selenium

2 Answers 2

Comments

2 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

2 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related