python threading local unexpected value

Question

Why is getattr not working? I am not trying to set a default value to threadLocal, I want to know why getattr does not work the way I want it to? tried to lock, same output

Expected Output

0
1
2
3
4
main thread

Current Output

0
0
0
0
0
main thread

Code

from concurrent.futures import ThreadPoolExecutor
from threading import local

threadLocal = local()
threadLocal.x = 'main thread'

def f(x):
    # threadLocal.x = x # this works
    threadLocal.x = getattr(threadLocal, 'x', x) # this does not work
    return threadLocal.x

pool = ThreadPoolExecutor(5)
result = pool.map(f, range(0, 5))
for i in result:
    print(i)

print(threadLocal.x)

A thread local is attached to the thread. Changes in other threads do not chance the version in the main thread. And the new threads don't even see x from the main thread. — Klaus D.
– Klaus D., Commented Nov 12, 2021 at 5:56
@KlausD maybe I did not explain it clearly, but threadLocal.x = x does what I want, why does not getting the value using getattr work? — Jake
– Jake, Commented Nov 12, 2021 at 5:57
Maybe I was not clear enough as well: because it does not exist in that thread. — Klaus D.
– Klaus D., Commented Nov 12, 2021 at 6:06
the x in getattr(threadLocal, 'x', x) will not be a function local the one in the function argument? @KlausD. — Jake
– Jake, Commented Nov 12, 2021 at 6:08

MisterMiyagi · Accepted Answer · 2021-11-12 08:11:54Z

1

A Thread pool shares its worker threads between work items. That means that if a work item is finished before the next, both can run on the same thread and thus see the same thread locals.

This is exactly what happens here: since f is very quick to execute, most or all work items run in the first worker thread. The first task f(0) sets threadLocal.x = 0 here, and other tasks running in the same thread thus read threadLocal.x = 0.

You can circumvent this by (artificially) increasing the runtime of f.

def f(x):
    time.sleep(0.2)  # delay for other work items to start running
    threadLocal.x = getattr(threadLocal, 'x', x) # this works now
    return threadLocal.x

Note that any additional operation may be sufficient to break the timing issue: that includes printing the argument and thread local, for example.

edited Nov 12, 2021 at 8:11

answered Nov 12, 2021 at 7:58

MisterMiyagi

53.4k14 gold badges131 silver badges138 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

python threading local unexpected value

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related