Python Multiprocessing and global variables

Question

import os
from multiprocessing import Process

b = {
        'x':{
            'd':[]
            },
        'y':{
            'd':[]
            },
}

def fmt():
    global b
    for k in b:
        if not b[k]['d']:
            b[k]['d'].append("fb1")
        print b
        t = Process(target=fb2, args=(k,))
        t.daemon = False
        t.start()

def fb2(k="x"):
    print os.getpid(), k, b[k]

if __name__ == '__main__':
    fmt()

Windows output:

C:\Python27\python.exe C:/Users/qchen/PycharmProjects/syntax_test/syntax_test.py
{'y': {'d': ['fb1']}, 'x': {'d': []}}
{'y': {'d': ['fb1']}, 'x': {'d': ['fb1']}}
4412 y {'d': []}
5972 x {'d': []}

Linux Output:

qchen@ubuntu:~/PycharmProjects/for_test$ python syntax_test.py 
{'y': {'d': ['fb1']}, 'x': {'d': []}}
{'y': {'d': ['fb1']}, 'x': {'d': ['fb1']}}    
23547 y {'d': ['fb1']}
23548 x {'d': ['fb1']}

I don't know why it is different between Windows OS and Linux OS; The difference is because of the difference of Process Fork and management in two OS

Why are you using global variables in the first place? Don't be that guy. — ThiefMaster
– ThiefMaster, Commented Mar 12, 2014 at 7:33
Looks like on Linux you printed b[k] (despite your code) while on Windows you really printed b (as the code states). Since there is a difference in the used code (at least that's the simplest explanation), you probably messed up sth else. I propose to compare both versions you run on the two systems, remove all differences, and retry that experiment. Probably the peculiarities are gone then. — Alfe
– Alfe, Commented Mar 12, 2014 at 8:08
To test on Linux, you could use Python 3.4 and multiprocessing.set_start_method('spawn') to avoid fork-semantics. — jfs
– jfs, Commented Mar 12, 2014 at 15:15
global b is useless unless there is b = .. in the function (or other binding statement) — jfs
– jfs, Commented Mar 12, 2014 at 15:23

jfs · Accepted Answer · 2014-03-12 15:33:24Z

4

To make the code behave similar on both Windows and Linux, pass b explicitly:

Process(target=fb2, args=(k, b))

The difference is that on Linux fork is used by default and it copies whatever state the parent process had to the child process. That is why the changes made inside fmt() are visible in children.

Windows uses spawn start method by default that reproduces the global state only partially e.g., values that are set during the import time are visible but changes made inside fmt() are not.

answered Mar 12, 2014 at 15:33

jfs

417k210 gold badges1k silver badges1.7k bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Python Multiprocessing and global variables

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related