Python interpreted code optimisation

Question

Consider the following code snippet:

dict [name] = 0
dict [name] += 1
dict [name] += 1

Does the python interpreter automatically recognise the repeated references to the dictionary value and use a cached local reference instead?, somewhat akin to the aliasing optimisations of C/C++, becoming something like so:

value = dict [name]
value = 0
value += 1
value += 1

Obviously, it's not a big deal to do this manually but I'm curious if it's actually necessary. any insight, feedback, etc is appreciated.

That's the interpreter's implementation detail you should not worry about. Besides, this sort of micro-optimization is not all that meaningful on such a layer as Python's, stratosphere-high as it is. Any particular reason you are concerned about it? — Santa
– Santa, Commented Aug 12, 2011 at 18:23
Curiosity and a tendency to fret over every aspect of my code :) — Gearoid Murphy
– Gearoid Murphy, Commented Aug 12, 2011 at 18:26

Brent Newey · Accepted Answer · 2011-08-12 18:23:01Z

You can run it through the disassembler to find out:

import dis

def test():
    name = 'test'
    tdict = {}
    tdict[name] = 0
    tdict[name] += 1
    tdict[name] += 1

dis.dis(test)

Running this we get:

 13           0 LOAD_CONST               1 ('test')
              3 STORE_FAST               0 (name)

 14           6 BUILD_MAP                0
              9 STORE_FAST               1 (tdict)

 15          12 LOAD_CONST               2 (0)
             15 LOAD_FAST                1 (tdict)
             18 LOAD_FAST                0 (name)
             21 STORE_SUBSCR        

 16          22 LOAD_FAST                1 (tdict)
             25 LOAD_FAST                0 (name)
             28 DUP_TOPX                 2
             31 BINARY_SUBSCR       
             32 LOAD_CONST               3 (1)
             35 INPLACE_ADD         
             36 ROT_THREE           
             37 STORE_SUBSCR        

 17          38 LOAD_FAST                1 (tdict)
             41 LOAD_FAST                0 (name)
             44 DUP_TOPX                 2
             47 BINARY_SUBSCR       
             48 LOAD_CONST               3 (1)
             51 INPLACE_ADD         
             52 ROT_THREE           
             53 STORE_SUBSCR        
             54 LOAD_CONST               0 (None)
             57 RETURN_VALUE

It looks like, in this case, that LOAD_FAST is loading up the values of tdict and name each time we try to access it to perform the increment, so the answer would appear to be no.

Ned Batchelder · Accepted Answer · 2011-08-12 18:30:25Z

9

That type of optimization isn't possible simply by inspecting the code. Your name dict could refer not to a native dictionary, but a user-defined object that implements __setitem__, and that method has to be called three times. At runtime, a sophisticated implementation could note the actual value of the name, and make an optimization, but it can't be done before runtime without breaking some Python semantics.

answered Aug 12, 2011 at 18:30

Ned Batchelder

378k77 gold badges583 silver badges675 bronze badges

Comments

Jochen Ritzel · Accepted Answer · 2011-08-12 18:38:32Z

1

No, because that would not work, and you even demonstated that with your own code - it's actually not equivalent:

>>> a = {}
>>> name = 'x'
>>> a[name] = 0
>>> a[name] += 1
>>> a[name] += 1
>>> a[name] # ok no suprises so far
2
>>> a = {}
>>> a[name] = 0
>>> x = a[name] # x is now literally `0`, not some sort of reference to a[name]
>>> x
0
>>> x += 1
>>> x += 1
>>> a[name] # so this never changed
0
>>>

Python does not have C-ish "references". What you had in mind would work only for mutable types such as list. This is a very fundamental property of Python and you should probably forget everything C taught you about variables when programming Python.

edited Aug 12, 2011 at 18:38

answered Aug 12, 2011 at 18:33

Jochen Ritzel

108k33 gold badges205 silver badges196 bronze badges

2 Comments

brandizzi Over a year ago

Do you mean "C++ like references" instead of "C like references"?

Karl Knechtel Over a year ago

I would say that on the contrary, Python does have "C++-like" references; what it doesn't have is variables of value type. Values are values, and variables refer to values. Well - they're like C++ references in that they're always valid and non-null, and "automatically dereferenced"; but like C++ pointers in that they can be re-seated and that assigning to them re-seats them; and unlike C++ entirely in that they're dynamically typed.

xubuntix · Accepted Answer · 2011-08-12 18:55:43Z

1

Changing your two examples into something like this:

#v1.py
di = {}
name = "hallo"
di[name] = 0
for i in range(2000000):
    di[name] += 1

and

#v2.py
di = {}
name = "hallo"
di[name] = 0
value = di[name]
for i in range(2000000):
    value += 1

You can see in the following tests, that v2 is faster, but pypy is much faster :-)

$ time python2.7 v1.py
real    0m0.788s
user    0m0.700s
sys     0m0.080s

$ time python2.7 v2.py
real    0m0.586s
user    0m0.490s
sys     0m0.090s

$ time pypy v1.py
real    0m0.203s
user    0m0.210s
sys     0m0.000s

$ time pypy v2.py
real    0m0.117s
user    0m0.080s
sys     0m0.030s

SO: it's not good to optimize code for a single interpreter (I have not tested Jython for example...), but it's great when someone optimizes the interpreter...

answered Aug 12, 2011 at 18:55

xubuntix

2,33318 silver badges20 bronze badges

2 Comments

xubuntix Over a year ago

I'm aware of that. But it's the same in the question: in v1, dict[name] is 2, in v2 value is 2. The question is whether both ways are treated (roughly) the same by the interpreter. And they are not, otherwise they would be (roughly) equally fast.

agf Over a year ago

I had forgotten that was what the question did between reading the question and reading your answer :)

Collectives™ on Stack Overflow

Python interpreted code optimisation

4 Answers 4

1 Comment

Comments

2 Comments

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

4 Answers 4

1 Comment

Comments

2 Comments

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related