Why does Python switch threads?

Question

In the Python documentation about threads and the GIL, it says:

In order to emulate concurrency of execution, the interpreter regularly tries to switch threads (see sys.setswitchinterval())

Why would it do this? These context switches appear to do nothing other than waste time. Wouldn't it be quicker to run each process until it releases the GIL, and then run the next?

It makes a difference if some of your "threads" (green threads) are stuck doing IO operations. — devoured elysium
– devoured elysium, Commented Jan 18, 2019 at 15:33
You are misunderstanding the docs. sys.setswitchinterval is setting the interval a thread maximally would hold the GIL. @devoured elysium Python-threads are real OS-threads, not green-threads. — Darkonaut
– Darkonaut, Commented Jan 18, 2019 at 15:41
@Darkonaut but why have a maximum time that a thread can hold the GIL if it is automatically released in the event of blocking I/O? Why would you want more context switching? Why bother 'emulating concurrency' if it slower? — BHC
– BHC, Commented Jan 18, 2019 at 15:54
Because then you would have only context switches on I/O. A thread doesn't necessarily have any I/O. If you want to trade simplicity for more efficiency you'll have to look into asyncio for single-threaded concurrency. — Darkonaut
– Darkonaut, Commented Jan 18, 2019 at 16:01
You could have one thread doing number crunching, another handling I/O. The number-chrunching thread with your proposal would never drop the GIL so the other thread could handle the I/O. — Darkonaut
– Darkonaut, Commented Jan 18, 2019 at 16:06

Darkonaut · Accepted Answer · 2019-01-18 16:29:07Z

4

A thread doesn't neccessarly have any I/O. You could have one thread doing number crunching, another handling I/O. The number-crunching thread with your proposal would never drop the GIL so the other thread could handle the I/O.

To ensure every thread gets to run, a thread will by default drop the GIL after 5 ms (Python 3) if it hasn't done so before because of waiting for I/O.

You can change this interval with sys.setswitchinterval().

Threading is a simple concurrency technique. For a more efficient concurrency technique look into asyncio which offers single-threaded concurrency using coroutines.

edited Jan 18, 2019 at 16:29

answered Jan 18, 2019 at 16:21

Darkonaut

22k7 gold badges61 silver badges73 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Why does Python switch threads?

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related