Print item from new window- Selenium/Python

Question

My code goes into a webpage, finds the table, clicks on each row,

(which each row when clicked, opens a new window)

and from this point I want to scrape 1 piece of information (faculty) from this new window which I cant seem to figure out.

Here is my code

from selenium import webdriver

from bs4 import BeautifulSoup
import time
import requests
driver = webdriver.Chrome()
driver.get('https://aaaai.planion.com/Web.User/SearchSessions?ACCOUNT=AAAAI&CONF=AM2021&USERPID=PUBLIC&ssoOverride=OFF')
time.sleep(3)
page_source = driver.page_source
soup = BeautifulSoup(page_source,'html.parser')
eachRow=driver.find_elements_by_class_name('clickdiv')

for item in eachRow:
    item.click() #opens the new window per each row
    time.sleep(2)
    faculty=driver.find_elements_by_xpath('//*[@id="W1"]/div/div/div/div[2]/div[2]/table/tbody/tr[7]/td/table/tbody/tr/td[2]/b')
    print(faculty)
    driver.find_element_by_class_name('XX').click()#closes window

what is the error? Is it not finding what you are looking for? — Insula
– Insula, Commented Feb 1, 2021 at 23:36
correct. i tried many approaches, some print an error message some print blanks. — Void S
– Void S, Commented Feb 1, 2021 at 23:40

Arundeep Chohan · Accepted Answer · 2021-02-02 02:37:48Z

1

Use find_element and .text.

driver.get('https://aaaai.planion.com/Web.User/SearchSessions?ACCOUNT=AAAAI&CONF=AM2021&USERPID=PUBLIC&ssoOverride=OFF')
time.sleep(3)
page_source = driver.page_source
soup = BeautifulSoup(page_source,'html.parser')
eachRow=driver.find_elements_by_class_name('clickdiv')

for item in eachRow:
    item.click() #opens the new window per each row
    time.sleep(2)
    faculty=driver.find_element_by_xpath("//td[@valign='MIDDLE']/b")
    print(faculty.text)
    driver.find_element_by_class_name('XX').click()

A better way:

wait = WebDriverWait(driver, 5)
 
for item in eachRow:
    item.click() #opens the new window per each row
    faculty=wait.until(EC.presence_of_element_located((By.XPATH, "//td[@valign='MIDDLE']/b")))
    print(faculty.text)
    driver.find_element_by_class_name('XX').click()

Import

from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait 
from selenium.webdriver.support import expected_conditions as EC

Outputs

Zaimat Beiro
Débora Shibayama Guterres, DÉBORA S GUTERRES
Joong K. Cho
Tao Zhu
Caroline Horner, MD FAAAAI

edited Feb 2, 2021 at 2:37

answered Feb 1, 2021 at 23:56

Arundeep Chohan

9,9995 gold badges17 silver badges36 bronze badges

Sign up to request clarification or add additional context in comments.

6 Comments

Void S Over a year ago

Thanks but this does not work. The output prints an address it seems, not the value of the faculty. I had a similar output when I was doing my code, however this is not the expected output. Expected output for row 1 example - > Zaimat Beiro

Arundeep Chohan Over a year ago

Can you post an example output.

Void S Over a year ago

hm your output seems right, however mine is printing - > <selenium.webdriver.remote.webelement.WebElement (session="f354ba32a8836f4b57b89c1966e4b86e", element="0.43124300990102404-650")> as the output?

Arundeep Chohan Over a year ago

Did you add faculty.text.

Arundeep Chohan Over a year ago

Mine is a more custom one using the the data-attribute of the td it was in.

|

Collectives™ on Stack Overflow

Print item from new window- Selenium/Python

1 Answer 1

6 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

6 Comments

Your Answer

Sign up or log in

Post as a guest

Related