Why does initializing a variable via a python default variable keep state across object instantiation?

Question

I hit an interesting python bug today in which instantiating a class repeatedly appears to be holding state. In later instantiation calls the variable is already defined.

I boiled down the issue into the following class/shell interaction. I realize that this is not the best way to initialize a class variable, but it sure should not be behaving like this. Is this a true bug or is this a "feature"? :D

tester.py:

class Tester():
        def __init__(self):
                self.mydict = self.test()

        def test(self,out={}):
                key = "key"
                for i in ['a','b','c','d']:
                        if key in out:
                                out[key] += ','+i
                        else:   
                                out[key] = i 
                return out

Python prompt:

Python 2.6.6 (r266:84292, Oct  6 2010, 00:44:09) 
[GCC 4.2.1 (Apple Inc. build 5664)] on darwin
>>> import tester
>>> t = tester.Tester()
>>> print t.mydict
{'key': 'a,b,c,d'}
>>> t2 = tester.Tester()
>>> print t2.mydict
{'key': 'a,b,c,d,a,b,c,d'}

Could it be interpreting out as a global variable, thus keeps adding to it whenever you call test? — Assaf Lavie
– Assaf Lavie, Commented Oct 8, 2010 at 1:27
Possible duplicate of "Least Astonishment" and the Mutable Default Argument — David Cain
– David Cain, Commented May 9, 2017 at 14:04

David Cain · Accepted Answer · 2017-05-09 14:15:13Z

15

It is a feature that pretty much all Python users run into once or twice. The main usage is for caches and the like to avoid repetitive lengthy calculations (simple memoizing, really), although I am sure people have found other uses for it.

The reason for this is that the def statement only gets executed once, which is when the function is defined. Thus the initializer value only gets created once. For a reference type (as opposed to an immutable type which cannot change) like a list or a dictionary, this ends up as a visible and surprising pitfall, whereas for value types, it goes unnoticed.

Usually, people work around it like this:

def test(a=None):
    if a is None:
        a = {}
    # ... etc.

edited May 9, 2017 at 14:15

David Cain

17.5k14 gold badges69 silver badges76 bronze badges

answered Oct 8, 2010 at 1:36

Stigma

1,74614 silver badges28 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

James Sapam Over a year ago

Ya good explanation. I want to add another line. function is an object in python, when we call for the first time function object is created, after that we are reusing the same object. you can confirm by printing the id(Test) after every call. you will get the same function object id, which means only one object is sharing.

Community · Accepted Answer · 2017-05-23 12:10:36Z

5

In general, default method arguments shouldn't be mutable. Instead do:

def test(self, out=None):
   out = out or {}
   # other code goes here.

See these links for more details on why this is necessary and why it's a "feature" of the python language rather than a bug.

edited May 23, 2017 at 12:10

CommunityBot

11 silver badge

answered Oct 8, 2010 at 1:33

Sam Dolan

32.6k10 gold badges93 silver badges84 bronze badges

1 Comment

John Machin Over a year ago

out = out or {} not only has a perlish whiff about it but also will fail if the caller passes in their own empty mapping object. if out is None: out = {} is preferable.

detly · Accepted Answer · 2010-10-08 01:34:39Z

3

You are modifying the value of the function keyword parameter out in your method.

This blog post explains it succintly:

expressions in default arguments are calculated when the function is defined, not when it’s called.

The function is defined when the class is created, not for each instance. If you modify it like so, the problem goes away:

def test(self,out=None):
        if out is None:
                out = {}
        key = "key"
        for i in ['a','b','c','d']:
                if key in out:
                        out[key] += ','+i
                else:   
                        out[key] = i 
        return out

answered Oct 8, 2010 at 1:34

detly

30.6k20 gold badges101 silver badges161 bronze badges

Collectives™ on Stack Overflow

Why does initializing a variable via a python default variable keep state across object instantiation?

3 Answers 3

1 Comment

1 Comment

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

1 Comment

1 Comment

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related