Using selenium and python to visit multiple sites

Question

I'm new to programming and I was just messing around with selenium. I started simple by using it to visit a website. It got me thinking, how would one visit multiple sites, specifically one after the other? How would I go about doing that in Python?

I guess what I'm asking is how would I use selenium to visit a list of sites one after another, waiting about 10 seconds in between going to the sites.

Here's what I have so far:

from selenium import webdriver
from selenium.webdriver.common.keys import Keys

driver = webdriver.Firefox(executable_path='PATH_TO_WEBDRIVER')

url = "http://www.google.com"

driver.get(url)
print(driver.title)

HedgeHog · Accepted Answer · 2021-03-15 07:40:10Z

2

Create a list of urls to visit:

urls = ['http://www.google.com','https://stackoverflow.com']

Loop the list of urls:

for url in urls:
    driver.get(url)
    print(driver.title)
    time.sleep(10)

Example

from selenium import webdriver
from selenium.webdriver.common.keys import Keys
import time

driver = webdriver.Firefox(executable_path='PATH_TO_WEBDRIVER')
urls = ['http://www.google.com','https://stackoverflow.com']

for url in urls:
    driver.get(url)
    print(driver.title)

driver.close()

Output

Google
Stack Overflow - Where Developers Learn, Share, & Build Careers

edited Mar 15, 2021 at 7:40

answered Mar 12, 2021 at 20:21

HedgeHog

25.4k5 gold badges18 silver badges43 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

LearningGuy15 Over a year ago

Thank you so much! This looks like it'll work.

Vova Over a year ago

@LearningGuy15 don't forget to close/quite driver

hugoncosta · Accepted Answer · 2021-03-13 09:04:05Z

0

To add to HedgeHog's response, should you want to visit websites in succession and the links are not known in advance (example, going to a newspaper and then the link(s) of the first article/first suggested article), I'd follow something like:

driver.get(url)
driver.get(driver.find_element_by_xpath(new_link))

answered Mar 13, 2021 at 9:04

hugoncosta

715 bronze badges

Collectives™ on Stack Overflow

Using selenium and python to visit multiple sites

2 Answers 2

2 Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

2 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related