0

I have a Python 3.6 script which is multi-threaded and runs on a windows 10 machine with 12 cores.

I want to run multiple instances of the script but I am worried about Pythons GIL issue in terms of the performance of each instance of the script.

Would doing any of the below work around this as I am under the impression that when I run an instance of the script, the Python process it runs within is running on just one CPU core and each thread called from within the script 'time slices' on that one core...

Therefore would:

A: Starting each instance of the script in its own CMD window allow the OS to automatically deal with starting each scripts parent Python processes on its own core and prevent any locking taking place...

or

B: Starting each CMD session via a shortcut that sets its affinity to a specific core and then run the Python script so that the python process and its threads run on the specific core the CMD process has been set to use...

or

C: My understanding of how threading and the Python GIL works is not correct and I need understand...

Any help will be much appreciated.

Improve this question
1
  • 1
    Python's threading uses real system threads and as such, your threaded code is not necessarily restricted to just one CPU - the underlying OS's thread scheduler can, and sometimes will spread them over different CPUs. However, no two threads may run at the same time due to the dreaded GIL so parallel execution is out of question. If you run multiple instances of your script, however, they will be running in different Python interpreter instances, therefore each will have its own GIL so multiple instances may run in parallel. Commented Jun 10, 2018 at 22:31

1 Answer 1

Reset to default
3

I want to run multiple instances of the script

Then go ahead and do it!

I am worried about Pythons GIL issue in terms of the performance of each instance of the script.

The "GIL issue" is only an issue within any single multi-threaded Python process: No more than one thread belonging to the same Python process will ever execute Python code at the same instant. But, each Python process has its own GIL, and nothing prevents a thread in one Python process from running at the same time as a thread in some other Python process.

would (A) Starting each instance of the script in its own CMD window [...run each] Python processes on its own core.

No, because it is not normal for any process on an SMP platform (i.e., most multi-processor systems) to have "its own core." Cores are a resource that the operating system uses to run threads/processes, and applications normally have no reason to know or care which core is running which thread/process at any given moment in time.

(B) Starting each CMD session via a shortcut that sets its affinity to a specific core.

Some operating systems let you do that, but usually there is no benefit unless your application has very special needs. Normally, you should just trust the operating system scheduler to schedule processes and threads as efficiently as possible.

Improve this answer
Sign up to request clarification or add additional context in comments.

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.