How to open content in a new tab using Python + Selenium

Question

I am using Python + Selenium.

Browsing this page

you will see many activities.

Using Python + Selenium, how can I open each activity in a new tab, and switch to the new tab?

I tried the following code, but it doesn't open in a new tab.

link = WebDriverWait(driver, 30).until(EC.element_to_be_clickable(driver.find_elements(By.CLASS_NAME, "css-vurnku")[0]))

actions = ActionChains(driver)
actions.key_down(Keys.CONTROL)
actions.click(on_element=link)
actions.perform()

Any help is highly appreciated.

I see you've added a bounty. Can you explain why Jakobs answer does not suffice and what needs to be different for you to accept the answer? — thetaco
– thetaco, Commented Feb 22, 2024 at 18:59
@thetaco I am still struggling to get Jakobs codes to run on my computer. It throws a bunch of errors on my computer. Actually my setup is very basic, I didn't configure anything out of the ordinary I think, so not too sure why there are errors. — J K
– J K, Commented Feb 23, 2024 at 7:35
@browsermator Can you please provide code snippet how to do it? I am new to Python and Selenium. Thanks. — J K
– J K, Commented Feb 23, 2024 at 7:37
that's it right there... then the driver will be in the new tab and you can navigate to the site and click your link(s) — browsermator
– browsermator, Commented Feb 23, 2024 at 17:09

Michael Mintz · Accepted Answer · 2024-02-23 04:04:55Z

1

+25

Here's a SeleniumBase script that goes to that site and opens up the first 20 activities in new tabs.

pip install seleniumbase, and run with python:

from seleniumbase import SB

with SB() as sb:  # By default, browser="chrome" if not set.
    sb.open("https://rovo.co/explore/activities")
    for i in range(20):
        sb.open_new_tab(switch_to=True)
        sb.open("https://rovo.co/explore/activities")
        sb.wait_for_element("div.css-uq1pv2")
        sb.find_elements("div.css-uq1pv2")[i].click()
        sb.sleep(0.25)
    breakpoint()

answered Feb 23, 2024 at 4:04

Michael Mintz

16.1k9 gold badges51 silver badges109 bronze badges

Sign up to request clarification or add additional context in comments.

6 Comments

J K Over a year ago

Can we open rovo.co/explore/activities in the first tab, then open each activity in a new tab? The reason I wanna do this is because, I will actually need to scroll down the activities page to lazy load more activities, then only open each activity in a new tab, scrap some info, and close the tab.

Michael Mintz Over a year ago

That’s exactly what the above script does. Have you tried it?

J K Over a year ago

I have already tried the above script, it is not doing what I want. The above script opens a new tab, opens the activities page in this new tab, then opens an activity in this new tab, then repeats. This is not what I want, although eventually the end result may seem like what I want. What I really want is this --> Open the activities page in the 1st tab. This only happens ONE time, and in the 1st tab only. Then, opens activity 1 in tab 2. Then, opens activity 2 in tab 3. Then, opens activity 3 in tab 4, ... Notice, the activities page stays open in tab 1 and is never opened in other tabs.

Michael Mintz Over a year ago

On that page, it doesn't give you the URL for an activity until after you've already clicked on it. So you would need to inject JS into the page to do an event.preventDefault() so that you can try to grab the URL without leaving the page, and then open the activity page in a new URL. That sounds like stackoverflow.com/q/7610871/7058266

J K Over a year ago

I have no clue how to inject JS into the page and do all that. Could you please provide code snippet to achieve that?

|

Jakob Bagterp · Accepted Answer · 2024-02-20 17:04:18Z

0

How I would solve this with the Browserist extension of the Selenium web driver. It simply makes browser automation much easier.

Simply install the package with pip install browserist and you're ready to go.

How I would solve your problem:

Normally you would get a list of activity URLs from the <a href="https://example.com">...</a> tags, but since Rovo is a web application, it doesn't expose the links in the HTML. Instead we need to click and open each activity to get its URL. That's the first iteration, and hereafter we open each URL in a new tab.

from browserist import Browser

activities_url = "https://rovo.co/explore/activities"
activities_xpath = "//*[@id='bp3-tab-panel_explore-tabs_activities']/div/div"
activity_clickable_partial_xpath = "/div/div[1]"

with Browser() as browser:
    activity_links: list[str] = []

    browser.open.url(activities_url)
    number_of_activities = browser.tool.count_elements(activities_xpath)

    for i in range(1, number_of_activities + 1):
        browser.click.button(f"{activities_xpath}[{i}]{activity_clickable_partial_xpath}")
        browser.wait.until.url.contains("https://rovo.co/plays/")
        activity_links.append(browser.get.url.current())
        browser.back()

    for i, link in enumerate(activity_links):
        browser.window.open.new_tab(link, f"tab_{i + 1}")

Excerpt of how it works:

edited Feb 20, 2024 at 17:04

answered Feb 20, 2024 at 13:50

Jakob Bagterp

1,49114 silver badges18 bronze badges

5 Comments

J K Over a year ago

I installed the package with pip install browserist, then I have the following browser = Browser() browser.open.url("rovo.co/explore/activities")

J K Over a year ago

I get the following error: return self._repopulate_pool_static(self._ctx, self.Process, return self._repopulate_pool_static(self._ctx, self.Process, File "D:\Program Files\Python310\lib\multiprocessing\pool.py", line 326, in _repopulate_pool_static File "D:\Program Files\Python310\lib\multiprocessing\pool.py", line 326, in _repopulate_pool_static w.start() File "D:\Program Files\Python310\lib\multiprocessing\process.py", line 121, in start w.start()

J K Over a year ago

Here is a screenshot of the error I get from Browserist i.sstatic.net/7mLG0.gif

Jakob Bagterp Over a year ago

I haven’t seen this issue before. Seems like you’re running Python 3.10, right? Maybe a quick fix would be trying with newer versions 3.11 or 3.12?

Jakob Bagterp Over a year ago

Find more mitigation options here: github.com/jakob-bagterp/browserist/issues/517

Jakob Bagterp · Accepted Answer · 2024-04-26 08:56:17Z

That seems like cumbersome, lengthy code. This is one of the reason why I created the Browserist package as an extension to Selenium so you don't have to hassle with ActionChains and other details.

Find more information about methods to open tabs and windows in the documentation.

Maybe something like this would work?

from browserist import Browser

with Browser() as browser:
    browser.window.open.new_tab("https://example.com", "tab_1")
    browser.window.open.new_tab("https://google.com", "tab_2")
    browser.window.switch_to("tab_1")

In full disclosure, I'm the author of the Browserist package. Browserist is lightweight, less verbose extension of the Selenium web driver that makes browser automation even easier.

Simply install the package with pip install browserist and you're ready to go.

Also, note that you often don't need explicit or implicit waits like WebDriverWait when using Browserist. Before interacting with any element, it simply waits for it to be active, which leads to leaner, more efficient code. For instance, you only need this to click a button:

    browser.click.button("//*[@class='css-vurnku'][1]")

Collectives™ on Stack Overflow

How to open content in a new tab using Python + Selenium

3 Answers 3

6 Comments

5 Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

6 Comments

5 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related