560 questions
0
votes
1
answer
2k
views
Playwright: is there a selector for the text *inside* an <input>?
I'm trying to do some automated testing in Dynamics 365, whose tables look like this
.
Each 'cell' here is an <input> element.
Inside that element is a link of some kind that directs you to ...
- 101
1
vote
1
answer
2k
views
Printing results in playwright python eg. print(result)
I just learnt playwright python and wanted to know how one can print the results of a page to the terminal. I have tried printing to the terminal in the playwright function but it does nothing. Also ...
- 13
2
votes
2
answers
1k
views
Python playwright unable to access elements
I want to scrape the words which reside in the <li> elements. The results return an empty list. Are they resided within a frame because as I can see they are not within any <iframe><\...
- 388
0
votes
0
answers
868
views
Why i am getting this Proxy server connection error
i get this error requests.exceptions.SSLError: HTTPSConnectionPool(host='ipinfo.io', port=443): Max retries exceeded with url: /json (Caused by SSLError(SSLZeroReturnError(6, 'TLS/SSL connection has ...
1
vote
1
answer
1k
views
Scraping Websites With Playwright
I am trying to scrape tweets from twitter with Playwright.
The problem I am facing is - after I scroll down with "page.mouse.wheel", I can only get the tweets("page.content"), from ...
- 48
1
vote
0
answers
629
views
Accessing aria-hidden elements with Scrapy-Playwright
I'm using scrapy to access a response element. Since this element is hidden, it returns an empty object.
How can I make this object visible?
This is the div class I like to access or click on it.
<...
- 21
0
votes
1
answer
620
views
How to extract text from an iframe in Playwright Python? [duplicate]
I need to extract the text from the second doctype using playwright (mostly) or another tool. (python only)
Can anyone help?
<!DOCTYPE html>
<html data-lang-tag="" lang="ru&...
- 19
-1
votes
1
answer
1k
views
Playwright Python: How to locate an iframe object that has a dynamic ip and no name/url?
I'm trying to click a button within an iframe that has a dynamically generated id. This is the HTML of it:
<iframe frameborder="0" src="Container.aspx?type=topgen&menu_id=...
- 1
0
votes
1
answer
2k
views
Open multiple tabs in a browser using Playwright
I am trying to open multiple tabs from the same browser using the mentioned playwright code. The browser I'm trying to use is Firefox. Firefox is opening multiple windows instead of tabs. With Chrome ...
- 329
3
votes
1
answer
3k
views
open specific chrome profile using playwright python
I'm trying to open a specific chrome profile using playwright python using the following code
from playwright.sync_api import sync_playwright
with sync_playwright() as p:
# browser = p.chromium....
- 71
1
vote
0
answers
794
views
Why playwright isn't modifying request body?
I need to modify the request body that is sent by the browser, i do it like this:
async def handle_uri_route(route: Route):
response = await route.fetch()
response_json = await response.json()
...
- 986
0
votes
1
answer
2k
views
Why playwright miss pattern url?
I need to handle request with certain url and im trying to do it like this:
await page.route("**/api/common/v1/play?**", handle_data_route)
But it also handles a url like this: api/common/...
- 986
3
votes
1
answer
1k
views
Reuse log in state for all tests with pytest-playwright and Django [closed]
I'm trying to create a pytest fixture that will allow all my tests to reuse a single authenticated context. The goal is for the fixture to run once, yielding a context. Then the other tests can just ...
- 79
1
vote
1
answer
396
views
Why when installing cron along with playwright,errors occur in the absence of the playwright module
The launch takes place on a VPS. So, if you run the code without cron-a, then everything is OK, the sites are parsed. When I add cron, everything flies into a heap with errors. Here is what my log ...
- 19
3
votes
3
answers
11k
views
Error in Python Playwright: playwright._impl._api_types.Error: net::ERR_ABORTED at ... waiting until "load" error Python Playwright
I've struggled a while now trying to download a pdf in Python Playwright which is rendered from a php web page (not included in this example as it has sensitive code - instead I've included a link to ...
- 142
1
vote
1
answer
1k
views
Error generating PDF (blank or format error) - Playwright Python
Context:
Playwright Version: 1.29.1
Operating System: Windows
Python version: 3.8.2
Browser: Chromium
Describe the bug
This error happens in some specific situations, usually when directly or ...
3
votes
1
answer
344
views
Python tqdm download progress with playwright
I have following code to download something (in this example it's a video).
from playwright.sync_api import sync_playwright
import func
import os, time, shutil
import requests
from tqdm.auto import ...
- 31
0
votes
2
answers
6k
views
How to pass all headers from playwright python?
Currently, I can send the user-agent from playwright as well the viewport sizes. But i want to send all the headers informations like accept, accept_encoding, accept_language,referer, cookies,etc. I ...
- 140
0
votes
1
answer
238
views
Scrapy Crawl (referer: None)
I am new to scrapy and python I am scrapping data from Aliexpress.com with playwright method and it returns (referer: None): Here is my code
class AliSpider(scrapy.Spider):
name = "aliex&...
- 99
6
votes
4
answers
14k
views
Python Playwright start maximized window
I have a problem starting Playwright in Python maximized. I found some articles for other languages but doesn't work in Python, also nothing is written about maximizing window in Python in the ...
- 375
1
vote
2
answers
6k
views
Clicking a dynamic button with Playwright: How to identify by class name
How can I use Playwright to click on the button in the image, identified by its class name? It's important to note that the button's name changes daily, starting with 'Collect 10 Candies' and ...
- 85
1
vote
2
answers
3k
views
Scrapy playwright (python) giving 403 in headless = True and 200 in headless = false?
I am Scraping with scrapy-playwright an ecommerce site where when I scrap with headless: True, I am getting 403 error but, with Headless False I am getting 200,I even tried randomizing User agent ...
0
votes
0
answers
330
views
How to make sure text inside a column is fully visible using Playwright?
I have a table in my web page. Let us assume that one of the columns has a very long text, and the column is at its default width.
expect(self.page.locator('text=ABCDEFGHIJKLMNOPQRSTUVWXYZ')).\
...
- 1,846
-1
votes
1
answer
1k
views
How to make Playwright not to raise exceptions when the browser is closed
I have a interface which has a button that closes the playwright context and browser, but when i close it, a lot of exceptions (because the playwright browser and context were closed during some ...
- 39
0
votes
1
answer
1k
views
Playwright - how to find input that will contain a value?
I have an Input element with value like '123 456'.
How can I validate that the Input element contains '123' using an Expect?
input_locator = 'input[id="edition"]'
expect(self.page.locator(...
- 1,846
0
votes
1
answer
426
views
Scrapy crawl return ModuleNotFoundError: No module named '_lzma'
I am currently experiencing a problem when trying to run a scrapy crawl in my dedicated virtual environment. Indeed it returns me the error quoted in the title of this topic.
So I start by checking ...
- 15
0
votes
1
answer
3k
views
How to open multiple persistent chrome profiles using playwright
I have some Chrome profiles in which I have my different accounts logged in. I'm writing a multi-threaded script that will run each profile using launch_persistent_context so that I'm already logged ...
- 712
1
vote
0
answers
258
views
get SSL certificate details from web ui using selenium or playwright [closed]
Is there a way to get SSL certificate details from web UI using selenium or playwright? I know I can get them using openssl commands but I am looking to get from UI using selenium or playwright.
1
vote
1
answer
4k
views
playwright-python fails with "Navigation failed because page was closed!"
I have multiple pages to scrape with Playwright, but my script keeps failing with playwright._impl._api_types.Error: Navigation failed because page was closed!.
Here is my code:
urls = ['http://www....
- 66.1k
0
votes
1
answer
480
views
How to scrape a website with multiple pages with the same url adress using scrapy-playwright
I am trying to scrape a website with multiple pages with the same url using scrapy-playwright.
the following script returned only the data of the second page and did not continue to the rest of the ...
0
votes
4
answers
4k
views
How to asynchronizely save image with Playwright in Python?
I'm implementing a python web scraper using Playwright and I'm interested in saving images given their urls (i.e. each url contains and only contains the corresponding image). However, I have trouble ...
- 1
0
votes
1
answer
5k
views
page.close() not working as expected in Playwright and asyncio
I have written a web scraper which needs to scrape few hundred pages asynchronously in Playwright-Python after login.
I've came across aiometer from @Florimond Manca (https://github.com/florimondmanca/...
- 142
1
vote
1
answer
1k
views
Scrape info from popup window with Playwright in Python and store in pandas df
I'm using Playwright in a Jupyter Notebook to obtain building construction years from a property appraiser's website. Some web pages, like this one, have multiple buildings' data viewable only after ...
- 335
0
votes
1
answer
1k
views
Can't scrape multiple pages using scrapy-playwright api
CONTEXT: I'm just a newbie in web scraping. I was trying to scrape a local e-commerce site. It's a dynamic website so I am using scrapy-playwright(chromium) with proxies.
PROBLEM: It was running ...
- 1
-1
votes
1
answer
406
views
playwright doesn't respond with systemctl service
My server is running on Ubuntu 22.04.1 LTS and I have a python flask app which runs perfectly with an active virtual environment (source bin/activate) using wsgi [python 3.10]. While all my other ...
0
votes
0
answers
140
views
Extract complete URL from a link
I am scrapping amanzon.co.in using scrapy-playwright. I am able to extract description, rating and price of desired item. However for going to next page I want to extract href for Next Page button at ...
- 375
5
votes
3
answers
4k
views
Extract images from playwright page without requesting them again?
Let's say I've requested a page and it's fully loaded. Is it possible to save the images from the rendered/loaded page without sending another request for the image? This would be to avoid just ...
- 2,255
0
votes
1
answer
3k
views
Why can't I interact (fill, click, etc) with this element using Playwright in my Python code?
I'm using Playwright to access and interact with a website and it was going perfect till I found myself in the page where I can't interact any button ou search bar to apply a filter. I can use ....
0
votes
1
answer
737
views
Playwright error: 'Page.inner_html' was never awaited RuntimeWarning: Enable tracemalloc to get the object allocation traceback
I have written a Playwright async function which extracts laboratory results from our lab information system. However, I struggle to get rid of the following error despite "await"-ing every ...
- 142
3
votes
2
answers
7k
views
Get element text behind shadow DOM element using Playwright
I am trying to use Playwright to get contents of the open shadow root element which looks like this.
<some-element>
#shadow-root
ABC
</some-element>
Here #shadow-root contains text ...
- 704
0
votes
1
answer
2k
views
How to conditionally disable proxies for some requests playwright python
try:
with sync_playwright() as p:
driver = p.firefox.launch(hedless=headless, proxy={
"server": 'fa****y.com:10000',
'...
- 1
1
vote
1
answer
2k
views
how to make playwright to type in each character in a variable in to a search bar
there is a website (company website - cant share login) which has a search box containing locations
i was using fill to type in a randomly selected location from a dictionary
playwright fill in the ...
- 33
-1
votes
1
answer
953
views
Playwright Python - find fillable form by partial placeholder
As described in the title, I'm trying to find a fillable field on a webpage whose placeholder contains the word 'name'. Examples of ones I'd like it to select are ones with placeholder text 'First ...
-1
votes
1
answer
2k
views
playwright is timing out before running page.pause()
everything was running fine the code opened a website and carried out some functions
i took a 60 min break and now playwright refuses to do even the most basic function
it loads the website and times ...
- 33
1
vote
2
answers
2k
views
Handling pagination in python playwright when the url doesn't change
I am trying to scrape this site https://franchisedisclosure.gov.au/Register with playwright and the url doesn't change after you click on the next button. How do I solve this pagination problem?
Here'...
0
votes
1
answer
1k
views
Assigning the contents of XPath result from Playwright into a list
Using Playwright, the code below successfully prints the title of stackoverflow.co. This works because the Xpath query (//title) produces only ONE result (there is only one title).
page.goto("...
- 5
2
votes
1
answer
1k
views
Installing Chrome with Playwright on Google Cloud Run for Python
I'm getting a Executable doesn't exist at /root/.cache/ms-playwright/chromium-1019/chromium-1019/chrome-linux/chrome error on Google Cloud Run whenever I try to install Playwright for Python, and I ...
0
votes
1
answer
1k
views
Trouble waiting for changes to complete that are triggered by Python Playwright `select_option`
I'm trying to scrape a site that reports internet service availability by address. Addresses can be selected from a list created for a specific postcode. After an address is selected, a table is ...
- 33
0
votes
1
answer
2k
views
How to get class name in python playwright?
How to get a Class name using playwright and pyton?
I tried that but without success.
It could also be the color that is contained in the css
page.locator('xpath=//*[@id="__next"]/div[1]/div/...
2
votes
1
answer
3k
views
SEC_ERROR_UNKNOWN_ISSUER, playwright python inside docker
My code is quite simple:
from playwright.sync_api import sync_playwright
pw = sync_playwright().start()
firefox = pw.firefox.launch(headless=True)
context=firefox.new_context()
page= context.new_page()...
