1

I have a really strange problem: starting my program with this headless=False option everything works correctly, pages are opened, buttons are clicked, in a nutshell Playwright does its job. As soon as I remove the headless mode or put headless=True, the program starts but closes after 1-2 seconds, doing absolutely nothing. (I notice this because it should return a .txt file to me).

PlaywrightAutomation.py



from playwright.sync_api import Playwright


class PlaywrightAutomation:
    def __init__(self, playwright: Playwright):
        self.playwright = playwright

    def start_driver(self, url):
        browser = self.playwright.chromium.launch(
            headless=False,
            args=[
                "--user-agent=Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 "
                "(KHTML, like Gecko) Chrome/71.0.3578.98 Safari/537.36",
                "--log-level=3",
                '--no-sandbox',
                "--enable-automation=False",
                "--disable-blink-features=AutomationControlled",
                "--window-size='1920,1080'",
                "--start-maximized"
            ]
        )
        context = browser.new_context(no_viewport=True)
        page = context.new_page()
        page.goto(url)
        return browser, context, page

    @staticmethod
    def press_button(page, prompt):
        page.wait_for_selector(prompt, state="visible")
        page.click(prompt)



This file is called from another file.py:

from bs4 import BeautifulSoup
from concurrent.futures import ThreadPoolExecutor
from playwright.sync_api import sync_playwright
from PlaywrightAutomation import PlaywrightAutomation


def playwright_stuffs(url, day):
    with sync_playwright() as p:
        automation = PlaywrightAutomation(p)
        b, c, pg = automation.start_driver(url)

        ...
        ...
        ...

        html_content = pg.content()

        c.close()
        b.close()

        with open(f'pinnacle_{day}.txt', 'w') as file:
            file.write(html_content)

        return html_content


def main():
    def pinnacle_day(day):
        html = playwright_stuffs('...', day)
        soup = BeautifulSoup(html, 'html.parser')

    ...
    ...
    ...

    days = ['Lunedì', 'Martedì', 'Mercoledì', 'Giovedì', 'Venerdì', 'Sabato', 'Domenica']

    with ThreadPoolExecutor(max_workers=len(days)) as executor:
        executor.map(pinnacle_day, days)


main()


I wouldn't really know what to try, because in headless=False everything works perfectly.

2
  • Once I've solved a problem like this by adding a logger and setting the log level to debug. Then I've found the error in the logs. Commented Jan 6 at 14:30
  • The site is probably detecting your script as a bot and blocking you. What's the site? How to bypass the block is totally site-specific, but running headfully is much header to detect than headless. Commented Aug 31 at 4:15

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.