How can I to use Python with Selenium to run automatically a website (chrome) in the background when I want to press a button and download?

Question

I would like to create an automated program in python with the help of selenium that opens a website, navigates to a section where I press a button and downloads the data. If the program is not running in the background, it works fine (then I don't use the chrome_options.add_argument("--headless=old") ). If the website runs in the background, the program runs without error, but the data is not downloaded. I don't think press the button. I made the following chrome settings:

chrome_options = Options()
chrome_options.add_argument("--headless=old") # "--headless" or "--headless=new" can't work
chrome_options.add_argument("--disable-gpu")
chrome_options.add_argument("--no-sandbox")
chrome_options.add_argument("--disable-dev-shm-usage")
chrome_options.add_argument("--window-size=1920,1080") 
chrome_options.add_argument("--log-level=3") 
chrome_options.add_argument("--disable-extensions")

...

driver = webdriver.Chrome(service=Service(ChromeDriverManager().install()), options=chrome_options)
driver.get(""https://www.example.com"")  
time.sleep(1)

button press

...

button = driver.find_element(By.XPATH, '//*[@data-form-button="primary"]')
button.click()

or

button = wait.until(EC.element_to_be_clickable((By.XPATH, '//*[@id="dialog-region"]/form/div/footer/ul/li[2]/button')))
driver.execute_script("arguments[0].click();", button)

or

form = driver.find_element(By.XPATH, '//*[@id="dialog-region"]/form')
form.submit()

It's can't work.

When I use the next chrome setup: chrome_options.add_argument("--headless") or chrome_options.add_argument("--headless=new") then a white panel appears until I close the program. The program only runs with the background open when I use "--headless=old". I use this for the button press:

button = driver.find_element(By.XPATH, '//*[@data-form-button="primary"]')
button.click()

Version of Chrome: 129.0.6668.101

Version of Selenium : 4.25.0

Operation system: Windows 10

Thank you in advance for your help.

ryyyn · Accepted Answer · 2024-10-11 19:28:12Z

1

Looks like downloads don't work in headless mode unless you set a download path via Chrome experimental options before initializing the driver: See implementation here

That being said, I would personally try to do this using http requests. Then you don't have to worry about the headaches of web automation (page flakiness, Selenium instability, the time to download the driver each time, etc)

answered Oct 11, 2024 at 19:28

ryyyn

6293 silver badges12 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Sevi Over a year ago

Thank you for your help. It's work, I need to set the download path to Chorme before request to the website.Unfortunatly I need to use selenium because. I have to enter a website where they ask for a password and username, and I have to download data and press buttons to download the right things, and that's why I use Selenium. Perhaps a good solution is to use selenium and bs4 supplements in parallel.

Collectives™ on Stack Overflow

How can I to use Python with Selenium to run automatically a website (chrome) in the background when I want to press a button and download?

1 Answer 1

1 Comment

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Linked

Related