How to extract text from multiple div class using Selenium with Python

Question

I'm trying to get text class price-value from Page_inspect

Used driver.find_element_by_xpath and WebDriverWait.

rateText=WebDriverWait(driver, 10).until(EC.presence_of_all_elements_located((By.XPATH, '//div[starts-with(@class,"price")]//div[contains(@class,"price-value")]')))
for ratevalue in rateText:
      print (ratevalue.text)

result not found :

Traceback (most recent call last): File "D:\project\totempop\webscraping\asrPOP.py", line 22, in rateText=WebDriverWait(driver, 10).until(EC.presence_of_all_elements_located((By.XPATH, '//div[starts-with(@class,"price")]//div[contains(@class,"price-value")]'))) File "C:\Python310\lib\site-packages\selenium\webdriver\support\wait.py", line 89, in until raise TimeoutException(message, screen, stacktrace) selenium.common.exceptions.TimeoutException: Message:

Thanks in advance

Makarenko Maxim · Accepted Answer · 2022-04-21 18:27:59Z

0

Probably, you have the problem with XPath. You should try this one: './/div[contains(@class,"price-value")]/text()'

answered Apr 21, 2022 at 18:27

Makarenko Maxim

11 bronze badge

Sign up to request clarification or add additional context in comments.

Comments

Zzoomrus · Accepted Answer · 2022-04-21 18:45:14Z

0

You can just use this:

WebDriverWait(driver, 10).until(EC.presence_of_all_elements_located((By.CLASS_NAME, "price-value")))
elements = driver.find_elements(By.CLASS_NAME, "price-value")
for element in elements:
    print(element.text)

answered Apr 21, 2022 at 18:45

Zzoomrus

293 bronze badges

Comments

undetected Selenium · Accepted Answer · 2022-04-21 20:23:39Z

0

Seems you were close enough but you need some minor adjustments as follows:

The <div class="price"> have a single classname value, so you can drop the contains() part.
The <strong class="price-value"> is within the parent <div>, so you have to adjust the locator strategy.

Ideally to get text you need to induce WebDriverWait for the visibility_of_element_located() and you can use either of the following locator strategies:

Using CSS_SELECTOR and text attribute:

print(WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.CSS_SELECTOR, "div.price div.price-line > strong.price-value"))).text)

Using XPATH and get_attribute("innerHTML"):

print(WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.XPATH, "//div[@class='price']//div[@class='price-line']/strong[@class='price-value']"))).get_attribute("innerHTML"))

Note : You have to add the following imports :

from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC

edited Apr 21, 2022 at 20:23

answered Apr 21, 2022 at 20:09

undetected Selenium

194k44 gold badges304 silver badges387 bronze badges

1 Comment

MasNo Over a year ago

its work, thanks bro

Collectives™ on Stack Overflow

How to extract text from multiple div class using Selenium with Python

3 Answers 3

Comments

Comments

1 Comment

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

Comments

Comments

1 Comment

Your Answer

Sign up or log in

Post as a guest

Linked

Related