python multiprocessing and queue access issues

Question

I've got the following code:

import multiprocessing
import queue
import time

to_delete = queue.Queue()

def parallel(args):
    return para_print(*args)

def para_print(sngl, dbl, iter):
    to_delete.put(f"{sngl}, {dbl}, {iter}")

if __name__ == '__main__':
    multiprocessing.freeze_support()

    expression_list = [('a', 'aa', 1), ('b', 'bb', 2), ('c', 'cc', 3), ('d', 'dd', 4)]
    pool = multiprocessing.Pool(multiprocessing.cpu_count() - 1)
    result = pool.map(parallel, expression_list)

    print(to_delete.qsize())
    while not to_delete.empty():
        print(to_delete.get())

The result is '0' printed as the queue size, with nothing getting put correctly into the queue - or pulled from it. What on earth am I doing wrong here?

bivouac0 · Accepted Answer · 2019-11-07 05:40:00Z

1

You should be using the queue from multiprocessing. The standard one doesn't work correctly between processes. Here's the revised code..

import multiprocessing

to_delete = multiprocessing.Queue()

def parallel(args):
    return para_print(*args)

def para_print(sngl, dbl, iter):
    to_delete.put(f"{sngl}, {dbl}, {iter}")


if __name__ == '__main__':
    multiprocessing.freeze_support()

    expression_list = [('a', 'aa', 1), ('b', 'bb', 2), ('c', 'cc', 3), ('d', 'dd', 4)]
    pool = multiprocessing.Pool(multiprocessing.cpu_count() - 1)
    result = pool.map(parallel, expression_list)

    print(to_delete.qsize())
    while not to_delete.empty():
        print(to_delete.get())

edited Nov 7, 2019 at 5:40

answered Nov 7, 2019 at 5:31

bivouac0

2,5701 gold badge15 silver badges31 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

python multiprocessing and queue access issues

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related