request_html.render() working on windows, but not on Ubuntu

Question

I have following code:

    from requests_html import HTMLSession()
    session = HTMLSession()

    response = session.get(url)
    response.html.render()  # <- causes error
    page_soup = soup(response.html.html, "html.parser")
    response.close()

which I'm trying to parse website with JS. Independently of which website I tried to render i get following error on Ubuntu:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/project1/project1_venv/lib/python3.8/site-packages/requests_html.py", line 586, in render
    self.browser = self.session.browser  # Automatically create a event loop and browser
  File "/home/project1/project1_venv/lib/python3.8/site-packages/requests_html.py", line 730, in browser
    self._browser = self.loop.run_until_complete(super().browser)
  File "/usr/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/home/project1/project1_venv/lib/python3.8/site-packages/requests_html.py", line 714, in browser
    self._browser = await pyppeteer.launch(ignoreHTTPSErrors=not(self.verify), headless=True, args=self.__browser_args)
  File "/home/project1/project1_venv/lib/python3.8/site-packages/pyppeteer/launcher.py", line 306, in launch
    return await Launcher(options, **kwargs).launch()
  File "/home/project1/project1_venv/lib/python3.8/site-packages/pyppeteer/launcher.py", line 167, in launch
    self.browserWSEndpoint = get_ws_endpoint(self.url)
  File "/home/project1/project1_venv/lib/python3.8/site-packages/pyppeteer/launcher.py", line 226, in get_ws_endpoint
    raise BrowserError('Browser closed unexpectedly:\n')
pyppeteer.errors.BrowserError: Browser closed unexpectedly:

When I run the same code on my local computer with Windows 10 everything works fine.

In both environments the same versions of library:

requests-html==0.10.0

But on windows I have Python 3.7.2 and on Linux server Python 3.8.6. It's problem? In both environments I using venvs.

I have installed chromium on Ubuntu.

It seems that chromium needs some lib to run headlessly, maybe this and this can help. — Dorian Massoulier
– Dorian Massoulier, Commented Jun 30, 2021 at 9:57

a14stoner · Accepted Answer · 2025-01-31 06:49:52Z

4

I faced the same issue. The resolution which helped me I found on: https://techoverflow.net/2020/09/29/how-to-fix-pyppeteer-pyppeteer-errors-browsererror-browser-closed-unexpectedly/ . I executed the following command:

apt install -y gconf-service libasound2 libatk1.0-0 libc6 libcairo2 libcups2 libdbus-1-3 libexpat1 libfontconfig1 libgcc1 libgconf-2-4 libgdk-pixbuf2.0-0 libglib2.0-0 libgtk-3-0 libnspr4 libpango-1.0-0 libpangocairo-1.0-0 libstdc++6 libx11-6 libx11-xcb1 libxcb1 libxcomposite1 libxcursor1 libxdamage1 libxext6 libxfixes3 libxi6 libxrandr2 libxrender1 libxss1 libxtst6 ca-certificates fonts-liberation libappindicator1 libnss3 lsb-release xdg-utils wget

Now it works without any problem.

edited Jan 31 at 6:49

answered Jul 19, 2021 at 5:20

a14stoner

871 silver badge10 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Sikandar Hayat · Accepted Answer · 2021-06-30 04:26:25Z

-1

I face same issue on windows working fine while on Ubuntu show error for render()

when I remove it from code on windows then page not load properly and show error

while on ubuntu works fine. I think on ubuntu no need of including render().

Remove render() from code and try on it .

Hope it will work.

Thanks!

answered Jun 30, 2021 at 4:26

Sikandar Hayat

71 silver badge2 bronze badges

Collectives™ on Stack Overflow

request_html.render() working on windows, but not on Ubuntu

2 Answers 2

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related