Python multithreading works only a single CPU [duplicate]

Question

I am trying to create different threads to work with 8 cores. However I see the code creates 8 threads but only uses around on 100% on my macos. Why?

def runner(i):
    # do random stuff
    for a in range(0,1000000):
        i+=1
        5000 / 34 * i
        i + 400
        i / 20000
        i * 24440
        i+=1
        5000 / 34 * i
        i + 400
        i / 20000

q = queue.Queue()
threads = list()
for x in range(0,80):
    th = threading.Thread(target=runner,args=(x,))
    threads.append(th)

for th in threads:
    th.start()
for th in threads:
    th.join()

Also see stackoverflow.com/questions/3044580/… and stackoverflow.com/questions/18114285/… — PM 2Ring
– PM 2Ring, Commented Nov 21, 2017 at 9:40

Chen A. · Accepted Answer · 2017-11-21 09:33:26Z

2

This is due to Python GIL (Global Interpreter Lock). It blocks Python threads to work on a single CPU. You can read more about it at https://wiki.python.org/moin/GlobalInterpreterLock

Python GIL is a mutex that protects access to Python objects, preventing multiple threads from executing Python bytecodes at once. This lock is necessary mainly because CPython's memory management is not thread-safe.

There are multiple questions on the subject, check out the list here on SO

If you want your code to work on multiple CPUs, check out the multiprocessing module.

edited Nov 21, 2017 at 9:33

answered Nov 21, 2017 at 9:31

Chen A.

11.5k4 gold badges55 silver badges75 bronze badges

Sign up to request clarification or add additional context in comments.

7 Comments

Artsiom Praneuski Over a year ago

Or you can disable GIL, but it's not so safety :)

Open AI - Opting Out Over a year ago

@ArtsiomPraneuski in what ways is it safe to disable the GIL?

Chen A. Over a year ago

@ArtsiomPraneuski that's really not a good advice

Artsiom Praneuski Over a year ago

where did I write that this is safe :)

Artsiom Praneuski Over a year ago

@Vinny yeah, I know. that's why I left smile there :)

|

Collectives™ on Stack Overflow

Python multithreading works only a single CPU [duplicate]

1 Answer 1

7 Comments

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

7 Comments

Linked

Related