I have a multiprocess program in which one process puts many items on a queue and then sets an event after it has finished putting items on the queue. Another process waits for that event and then empties the queue in a while loop as in the code below, checking with an empty() call. The problem is that every once in awhile (maybe 1 time out of 10), empty() returns True when there are still items on the queue. During the while loop, no other process or thread is touching the queue. The issue can be reproduced with the below code in a single process. Most of the time, the output is:
The size of my_queue initially is 400
The size of my_queue is 0
But every once in a while, I will get something like:
The size of my_queue initially is 400
The size of my_queue is 377
Here is the code:
from multiprocessing import Queue as MPQueue
my_list = []
my_queue = MPQueue()
for i in range(0, 400):
my_queue.put(f"The value of i is {i}")
print(f"The size of my_queue initially is {my_queue.qsize()}")
while not my_queue.empty():
my_list.append(my_queue.get())
print(f"The size of my_queue is {my_queue.qsize()}")
Is this a bug in the multiprocessing.Queue module or am I missing something? I've also tried using get_nowait() and then catching the Queue.Empty exception, but I get the same result. Any help would be much appreciated. Thanks.
