server in a thread (Python3.9.0+aiohttp) : RuntimeError: can't register atexit after shutdown

Question

This snippet of code (a minimal server running in a thread, code taken from here) works fine with Python3.8.3 but raises an error message with Python3.9.0:

import asyncio
import threading
from aiohttp import web


def aiohttp_server():
    def say_hello(request):
        return web.Response(text='Hello, world')

    app = web.Application()
    app.add_routes([web.get('/', say_hello)])
    runner = web.AppRunner(app)
    return runner


def run_server(runner):
    loop = asyncio.new_event_loop()
    asyncio.set_event_loop(loop)
    loop.run_until_complete(runner.setup())
    site = web.TCPSite(runner, 'localhost', 8080)
    loop.run_until_complete(site.start())
    loop.run_forever()


t = threading.Thread(target=run_server, args=(aiohttp_server(),))
t.start()

The error message:

Exception in thread Thread-1:
Traceback (most recent call last):
  File "/usr/lib/python3.9/threading.py", line 954, in _bootstrap_inner
    self.run()
  File "/usr/lib/python3.9/threading.py", line 892, in run
    self._target(*self._args, **self._kwargs)
  File "/home/alkhinoos/nikw/nikw/z2.py", line 21, in run_server
    loop.run_until_complete(site.start())
  File "/usr/lib/python3.9/asyncio/base_events.py", line 642, in run_until_complete
    return future.result()
  File "/usr/lib/python3.9/site-packages/aiohttp/web_runner.py", line 121, in start
    self._server = await loop.create_server(
  File "/usr/lib/python3.9/asyncio/base_events.py", line 1460, in create_server
    infos = await tasks.gather(*fs, loop=self)
  File "/usr/lib/python3.9/asyncio/base_events.py", line 1400, in _create_server_getaddrinfo
    infos = await self._ensure_resolved((host, port), family=family,
  File "/usr/lib/python3.9/asyncio/base_events.py", line 1396, in _ensure_resolved
    return await loop.getaddrinfo(host, port, family=family, type=type,
  File "/usr/lib/python3.9/asyncio/base_events.py", line 856, in getaddrinfo
    return await self.run_in_executor(
  File "/usr/lib/python3.9/asyncio/base_events.py", line 809, in run_in_executor
    executor = concurrent.futures.ThreadPoolExecutor(
  File "/usr/lib/python3.9/concurrent/futures/__init__.py", line 49, in __getattr__
    from .thread import ThreadPoolExecutor as te
  File "/usr/lib/python3.9/concurrent/futures/thread.py", line 37, in <module>
    threading._register_atexit(_python_exit)
  File "/usr/lib/python3.9/threading.py", line 1374, in _register_atexit
    raise RuntimeError("can't register atexit after shutdown")
RuntimeError: can't register atexit after shutdown

What's going on ? The same problem appears with Python 3.9.1. Is this problem solved with Python 3.9.2 ? Maybe a relative issue here.

Jason Pan · Accepted Answer · 2021-06-30 03:04:57Z

14

As mentioned in Python manual - Thread Objects

Other threads can call a thread’s join() method. This blocks the calling thread until the thread whose join() method is called is terminated.

After calling t.start() in main thread, the main thread will exit. Then the process is ended.

If you want to run the child thread forever or until it exits, you should call t.join() in main thread after t.start().

answered Jun 30, 2021 at 3:04

Jason Pan

8129 silver badges25 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Anwar Husain · Accepted Answer · 2021-03-29 16:21:24Z

6

Not sure what you did, but I used 127.0.0.1 instead of localhost and the error is resolved!

answered Mar 29, 2021 at 16:21

Anwar Husain

1,90418 silver badges20 bronze badges

1 Comment

Alireza Haji gholamreza Over a year ago

man you saved me... i working on more +8h with chatgpt and not solving until i saw your answer, but why localhost not working?

Alex · Accepted Answer · 2023-02-07 21:37:18Z

2

I managed to get this sorted by importing the ThreadPoolExecutor module, before calling any of my main application code - so at the top of my main.py start-up script.

thread_pool_ref = concurrent.futures.ThreadPoolExecutor

Just the act of importing the module early (before any threads are initialised) was enough to fix the error. There is an on-going issue around how this module must be present in the main thread, before any child threads import or use any related threading library code.

The inspiration for this fix came from this post on the Python bugs site. My issue was specifically around boto3 library, but the fix is applicable across the board.

answered Feb 7, 2023 at 21:37

Alex

4,9338 gold badges46 silver badges61 bronze badges

Comments

openwld · Accepted Answer · 2025-01-30 08:57:00Z

0

This could also be caused by processes instead of threads. Found some good explanation here: https://github.com/python/cpython/issues/86813#issuecomment-1246097184

"You need to manually import concurrent.futures.thread or import concurrent.futures.process if their respective concurrent.futures Executor classes are to be used from an atexit handler."

Basically, the workaround is to add these 2 lines of code at the start of your main program, before any other lines of code:

import concurrent.futures.thread
import concurrent.futures.process

edited Jan 30 at 8:57

openwld

1,1945 silver badges21 bronze badges

answered Jan 29 at 12:04

Liviu Iacob

216 bronze badges

Collectives™ on Stack Overflow

server in a thread (Python3.9.0+aiohttp) : RuntimeError: can't register atexit after shutdown

4 Answers 4

Comments

1 Comment

Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

4 Answers 4

Comments

1 Comment

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related