I'm curious in how the Global Interpreter Lock in python actually works. If I have a c++ application launch four separate instances of a python script will they run in parallel on separate cores, or does the GIL go even deeper then just the single process that was launched and control all python process's regardless of the process that spawned it?
Add a comment
|
2 Answers
The GIL only affects threads within a single process. The multiprocessing module is in fact an alternative to threading that lets Python programs use multiple cores &c. Your scenario will easily allow use of multiple cores, too.
2 Comments
S.Lott
A "Plain Old Unix Pipeline" of Python applications is often a better design than multiple threads. A pipeline of processes avoids all GIL issues. It also avoids any potential thread-safety issues in any library you happen to be using. It's easy to create using the shell. The OS handles synchronization for you -- you just read from sys.stdin and write to sys.stdout.
Charlie G
@S.Lott does that involve running scripts from
subprocess if calling from a python app?As Alex Martelli points out you can indeed avoid the GIL by running multiple processes, I just want to add and point out that the GIL is a limitation of the implementation (CPython) and not of Python in general, it's possible to implement Python without this limitation. Stackless Python comes to mind.
1 Comment
Jay Taylor
This is a common misconception about stackless. Stackless does not help take advantage of multiple cores. See stackless.com/pipermail/stackless/2007-August/001963.html