I apologize if this is a poor explanation, but I don't know how to word it better.
From the image you can see that there is a text called UPC and directly below it there is a number. I would like to get the number directly below the UPC. I would like to find it by searching the website for the text "UPC" and getting the number directly below it if possible.
If you are owner of this page it would be better to use unique elements id's to locate them. Anyway there is another solution. As it's written in selenium documentation you can use css selector and use just "next element" css-selector "~". It can looks like this:
content = driver.find_element_by_css_selector('div:contains("UPC") ~ div')
You can read more about CSS selectors here
Good luck!
Update: correct answer is:
driver.find_element(By.XPATH, '//div[text()="UPC"]/following-sibling::div')
Note:
find_element_by_*commands are deprecated. Please use find_element() instead
To print the number 840.... you can use either of the following Locator Strategies:
Using selenium3 approach and xpath:
print(driver.find_element_by_xpath("//div[contains(., 'UPC')]//following-sibling::div[1]").text)
Using selenium4 approach and xpath:
print(driver.find_element(By.XPATH, "//div[contains(., 'UPC')]//following-sibling::div[1]").text)
Ideally you need to induce WebDriverWait for the visibility_of_element_located() and you can use either of the following Locator Strategies:
Using selenium3 approach and xpath:
print(WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.XPATH, "//div[contains(., 'UPC')]//following-sibling::div[1]"))).text)
Using selenium4 approach and xpath:
print(WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.XPATH, "//div[contains(., 'UPC')]//following-sibling::div[1]"))).text)
You can find a relevant discussion in How to retrieve the text of a WebElement using Selenium - Python
