Copied XPATH doesn't work in Selenium Python

Question

I have tried to scrape my school's website and it's not working. I tried a lot of stuff but on most of the time the XPATH that I passed was incorrect. (the XPATH was copied from the HTML) There are a lot of elements like this that aren't responding after clicking or they just can't be find. My code:

from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

def element(driver, by_x, html_element):
    try:
        element = WebDriverWait(driver, 5).until(
            EC.presence_of_element_located((by_x, html_element))
        )
        return element
    except:
        print("Element not found")

class EduPage:
    def __init__(self, name, password, edu_link):
        options = webdriver.ChromeOptions()
        options.add_experimental_option("excludeSwitches", ["enable-logging"])
        PATH = "C:\Programming Modules\Drivers\chromedriver.exe"
        self.driver = webdriver.Chrome(executable_path=PATH, options=options)
        self.name = name
        self.password = password
        self.driver.get(edu_link)

    def log_in(self):
        login_name = element(self.driver, By.ID, "login_Login_1e1")
        login_name.send_keys(self.name)
        login_pass = element(self.driver, By.ID, "login_Login_1e2")
        login_pass.send_keys(self.password, Keys.RETURN)

    def go_to_class(self):
        online_class = element(self.driver, By.XPATH, "//*[@id='jwb98b9ff6_md']/div/div[2]")
        online_class.click()


edupage = EduPage("my name", "my password", "https://gymstrop.edupage.org/login/")
edupage.log_in()
edupage.go_to_class

HTML code (I want to click the highlighted element)

KunduK · Accepted Answer · 2021-01-19 14:42:57Z

2

The reason it is failing because parent element id attribute is dynamic. I would suggest use following xpath to identify the element..

//div[@class='userTopOnlineLesson']/a

so your code would be like

online_class = element(self.driver, By.XPATH, "//div[@class='userTopOnlineLesson']/a")

Or following css selector.

div.userTopOnlineLesson>a.userTopBlackboard

answered Jan 19, 2021 at 14:42

KunduK

33.4k5 gold badges19 silver badges42 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

KunduK Over a year ago

@themm1 : what error are you getting? timeout?

Panchdev Singh · Accepted Answer · 2021-01-19 18:23:59Z

0

Solution provided by KunduK should work and if it is not working, I also would like to see the actual selenium error that you are getting.

Meanwhile could you please try to increase the wait time to 60, just to figure out if that has nothing to do long waiting time

try:
        element = WebDriverWait(driver, 60).until(
            EC.presence_of_element_located((by_x, html_element))
        )
        return element
    except:
        print("Element not found")

answered Jan 19, 2021 at 18:23

Panchdev Singh

1511 silver badge6 bronze badges

Collectives™ on Stack Overflow

Copied XPATH doesn't work in Selenium Python

2 Answers 2

1 Comment

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

1 Comment

Comments

Your Answer

Sign up or log in

Post as a guest

Related