How to get a specific class name value using selenium?

Question

I am trying to get the price from this website, however my code does not return the price. I am wondering if I can get a specific network item header values.

For example network with name 759454 and header where it contains fingerprint? With that I can scrape the whole thing. Currently using class name does not return the price

import requests
import pandas as pd
from bs4 import BeautifulSoup
from selenium import webdriver
from webdriver_manager.chrome import ChromeDriverManager
from selenium.webdriver.common.by import By


url='https://alfagift.id/p/elmondo-jas-hujan-ceria-setelan-jaket-celana-759454'

options = webdriver.ChromeOptions() 
options.add_argument("start-maximized")
options.add_experimental_option("excludeSwitches", ["enable-automation"])
options.add_experimental_option('useAutomationExtension', False)
driver = webdriver.Chrome(ChromeDriverManager().install(),options=options)
driver.execute_script("Object.defineProperty(navigator, 'webdriver', {get: () => undefined})")
driver.execute_cdp_cmd('Network.setUserAgentOverride', {"userAgent": 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.53 Safari/537.36'})
print(driver.execute_script("return navigator.userAgent;"))

driver.get(url)

price= driver.find_elements(By.CLASS_NAME,"text-xlg fw7 text-primary mb-1")
print(price)

for x in price:
    print(x.text)

KunduK · Accepted Answer · 2022-12-07 16:06:34Z

0

By.CLASS_NAME() only accepts single class name not multiple class names. Instead use By.CSS_SELECTOR().

Use WebDriverWait() and wait for presence_of_element_located() and use textContent attribute to get the value. Since element is hidden in the DOM tree.

You need to import below libraries as well

from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By

Code

options = webdriver.ChromeOptions() 
options.add_argument("start-maximized")
options.add_experimental_option("excludeSwitches", ["enable-automation"])
options.add_experimental_option('useAutomationExtension', False)
driver = webdriver.Chrome(service=Service(ChromeDriverManager().install()),options=options)
driver.get('https://alfagift.id/p/elmondo-jas-hujan-ceria-setelan-jaket-celana-759454')
price=WebDriverWait(driver,10).until(EC.presence_of_element_located((By.CSS_SELECTOR, "p.text-primary.mb-1"))).get_attribute("textContent")
print(price)

Output:

answered Dec 7, 2022 at 16:06

KunduK

33.4k5 gold badges19 silver badges42 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

Hal Over a year ago

If I am trying to get multiple items do I have to do WebDriverWait too? for example product_name=WebDriverWait(driver,10).until(EC.presence_of_element_located((By.CSS_SELECTOR, "text-xl fw5 d-lg-none"))).get_attribute("textContent") price=WebDriverWait(driver,10).until(EC.presence_of_element_located((By.CSS_SELECTOR, "p.text-primary.mb-1"))).get_attribute("textContent")

KunduK Over a year ago

Best practices said if we use webdriverwait we can avoid page element sync issue.

Hal Over a year ago

do I just do a 1 time webdriverwait or after that I can get multiple items?

Collectives™ on Stack Overflow

How to get a specific class name value using selenium?

1 Answer 1

3 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

3 Comments

Your Answer

Sign up or log in

Post as a guest

Related