How to select whole page and copy using selenium python?

Question

im using the selemium to extract data .i need to select all the text and copy to the variable .but it select all the text but it not copying.any can help this?

from selenium import webdriver
from selenium.webdriver.common.keys import Keys
driver = webdriver.Firefox()
driver.get('https://en.wikipedia.org/wiki/Apple_Inc.')
element=driver.find_element_by_css_selector('body')
all_text=element.send_keys(Keys.CONTROL+'a')
copy=element.send_keys(Keys.CONTROL+'c')
print copy
driver.close()

I do not know why you are using selenium, but most of the people are using complex solutions (such as selenium) instead of simpler ones (using urllib3 and bs4). Is getting all the elements what you need? I could help you with a simple script without selenium — BlueSheepToken
– BlueSheepToken, Commented Oct 11, 2018 at 10:16
@EDv What exactly do you mean by all the text? Entire html? — undetected Selenium
– undetected Selenium, Commented Oct 11, 2018 at 10:45
Why are you scraping wikipedia? That's going to break their ToS. What is it you are trying to do? — JeffC
– JeffC, Commented Oct 11, 2018 at 18:00
i know about the bs4 and urllib.but i dont want to use that .i need to select all page like cntrl+a ,and copy that text into a variable @DebanjanB — EDv
– EDv, Commented Oct 12, 2018 at 8:48
Still not sure about your usecase for which you want to do cntrl+a but if you want to _...copy the text... @Infern0 Answer looks just perfect to cater to your requirement. Let me know if you are looking for something else. — undetected Selenium
– undetected Selenium, Commented Oct 12, 2018 at 9:04

Infern0 · Accepted Answer · 2018-10-11 10:26:40Z

3

to get the text

element=driver.find_element_by_id("bodyContent") 
print(element.text)

to get the html content

driver.get('https://en.wikipedia.org/wiki/Apple_Inc.') 
content = driver.page_source

soup = BeautifulSoup(content) 
print soup

answered Oct 11, 2018 at 10:26

Infern0

2,8241 gold badge11 silver badges24 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

How to select whole page and copy using selenium python?

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related