Using 'requests_html' with 'FastAPI'

Question

I'm trying to use the requests_html library with js rendering along with FastAPI:

script.py

from fastapi import FastAPI
from requests_html import HTMLSession

app = FastAPI()

@app.get('/')
def func():
    with HTMLSession() as session:
        r = session.get('https://stackoverflow.com')
        r.html.render()
        return r.text

when running using uvicorn script:app --reload and accessing http://127.0.0.1:8000/ ,I get the following error:

...
 r.html.render()
  File "c:\users\a\appdata\local\programs\python\python37\lib\site-packages\requests_html.py", line 586, in render
    self.browser = self.session.browser  # Automatically create a event loop and browser
  File "c:\users\a\appdata\local\programs\python\python37\lib\site-packages\requests_html.py", line 727, in browser
    self.loop = asyncio.get_event_loop()
  File "c:\users\a\appdata\local\programs\python\python37\lib\asyncio\events.py", line 644, in get_event_loop
    % threading.current_thread().name)
RuntimeError: There is no current event loop in thread 'ThreadPoolExecutor-0_0'.

Any idea how can I make them work together?

Mehmet · Accepted Answer · 2020-12-24 09:49:08Z

1

You have to use AsyncHTMLSession in Uvicorn

from fastapi import FastAPI
from requests_html import AsyncHTMLSession

app = FastAPI()

@app.get('/')
async def func():
    asession = AsyncHTMLSession()
    r = await asession.get('https://stackoverflow.org/')

    await r.html.arender()

    return r.text

edited Dec 24, 2020 at 9:49

answered Dec 24, 2020 at 8:49

Mehmet

344 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Mezo Over a year ago

Works great, thank u. I also get this warning on the 1st request ``` .\scratch.py:13: RuntimeWarning: coroutine 'HTML.arender' was never awaited r.html.arender() RuntimeWarning: Enable tracemalloc to get the object allocation traceback ``` is that an issue?

Mehmet Over a year ago

please replace "r.html.arender()" to "await r.html.arender()"

Collectives™ on Stack Overflow

Using 'requests_html' with 'FastAPI'

1 Answer 1

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related