datetime.strptime() is blocking asyncio.Queue.get() [duplicate]

Question

In the following code datetime.strptime() is blocking asyncio.Queue.get() operation

import time
import asyncio
from datetime import datetime
from functools import partial

def write(queue):

    data = dict(fname = "Stack", lname = "Overflow")
    
    # If you comment the below line thing are working fine
    dob = datetime.strptime("2025-02-12", "%Y-%m-%d")

    queue.put_nowait(data)
    print("Writing to queue complete")
    time.sleep(3600)  # sleep forever

async def reader(queue):
    loop = asyncio.get_running_loop()
    print("This print means this thread is not blocked")
    while msg := await queue.get():
        print("Got my message", msg)
        loop.call_soon(print, msg)

async def main():
    queue = asyncio.Queue()
    loop = asyncio.get_running_loop()
    loop.run_in_executor(None, partial(write, queue))
    await reader(queue)

asyncio.run(main())

If I comment out datetime.strptime() function call I get the following output

Writing to queue complete
This print means this thread is not blocked
Got my message {'fname': 'Stack', 'lname': 'Overflow'}
{'fname': 'Stack', 'lname': 'Overflow'}

But if datetime.strptime() is not commented I get the following output

This print means this thread is not blocked
Writing to queue complete

Why is datetime.strptime() blocking asyncio.Queue.get() operation?

user2357112 · Accepted Answer · 2025-02-13 03:47:13Z

3

The problem isn't datetime.strptime. The problem is that it is not safe to call queue.put_nowait from outside the event loop. The docs specifically mention that

asyncio queues are not thread-safe

so you cannot safely call queue.put_nowait from an executor thread. The time needed to execute datetime.strptime simply changed the execution order enough for the unsafe call to cause a problem.

edited Feb 13 at 3:47

answered Feb 13 at 3:40

user2357112

286k32 gold badges490 silver badges571 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

datetime.strptime() is blocking asyncio.Queue.get() [duplicate]

1 Answer 1

Comments

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Linked

Related