passing user input from main to inside a pool python

Question

I'm trying to pass some user input from main or outside main to the function running inside a multiprocessing pool. Code is something like below:

import multiprocessing

global var1
var1 = input("enter input: ")

def function1(arg1_A):
    x = var1 + 'string'
    do stuff

def MCprocess():
    pool = multiprocessing.Pool(4)
    pool.map(function1, listarg1)

if __name__ == '__main__':
    MCprocess()

Where do I put the input() function in order for all instances of function1 to receive it? Is there a better way of doing this?

To clarify. I've tried declaring var1 as global var1 within the function. That hasn't worked.

what does the modules module do and what is arg1? also could you say what you mean by "that hasn't worked". also is multiprocessing can be OS dependant, which OS do you care about. see stackoverflow.com/help/mcve — Sam Mason
– Sam Mason, Commented Nov 9, 2020 at 17:16
@SamMason multiprocessing is a python module. you need an import statement to import it. I've updated the code to show that I'm importing that module. I am importing others. Thanks for pointing out arg1. I've updated that as well. in MCprocess, listarg1 is a list of files to be processed. in function1, arg1_A is an individual (any individual) belonging to that list. — NorthLand
– NorthLand, Commented Nov 9, 2020 at 17:39
I've not tested my answer under windows, but I expect it to work — Sam Mason
– Sam Mason, Commented Nov 9, 2020 at 19:53

Sam Mason · Accepted Answer · 2020-11-09 17:54:06Z

1

making this work portably (i.e. under Windows) is somewhat fiddly, but this should do the right thing:

from multiprocessing import Pool

def init(*args):
   global var1
   [var1] = args

def process(path):
    print(repr(var1), 'processing', path)

def MCprocess():
    var1 = input("enter input: ")
    with Pool(initializer=init, initargs=[var1]) as pool:
        pool.map(process, ['first', 'second'])

if __name__ == '__main__':
    MCprocess()

note that if you just care about Linux or OSX (i.e. where it forks processes) you could get away with:

from multiprocessing import Pool

def process(path):
    print(repr(var1), 'processing', path)

var1 = input("enter input: ")
with Pool() as pool:
    pool.map(process, ['first', 'second'])

answered Nov 9, 2020 at 17:54

Sam Mason

16.5k1 gold badge49 silver badges71 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

passing user input from main to inside a pool python

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related