Python doctest execution context

Question

I have the following function which I do unit testing with doctest.

from collections import deque

def fill_q(histq=deque([])):
    """
    >>> fill_q()
    deque([1, 2, 3])
    >>> fill_q()
    deque([1, 2, 3])
    """
    if histq:
        assert(len(histq) == 0)
    histq.append(1)
    histq.append(2)
    histq.append(3)
    return histq

if __name__ == "__main__":
    import doctest
    doctest.testmod()

the first case passes, but the second call to fill_q fails, yet it's the same code:

**********************************************************************
File "trial.py", line 7, in __main__.fill_q
Failed example:
    fill_q()
Exception raised:
    Traceback (most recent call last):
      File "/usr/lib/python2.7/doctest.py", line 1289, in __run
        compileflags, 1) in test.globs
      File "<doctest __main__.fill_q[1]>", line 1, in <module>
        fill_q()
      File "trial.py", line 11, in fill_q
        assert(len(histq) == 0)
    AssertionError
**********************************************************************
1 items had failures:
   1 of   2 in __main__.fill_q
***Test Failed*** 1 failures.

It looks like that doctest re-uses the local variable histq from the first test call, why is it doing this? This is very silly behaviour (provided it's not me doing sth crazy here).

Wouldn't your condition be True only if histq was empty? — Blender
– Blender, Commented Oct 3, 2012 at 21:40
@Blender, yes, this example is just to illustrate the problem. — Yanshuai Cao
– Yanshuai Cao, Commented Oct 3, 2012 at 23:05

K Z · Accepted Answer · 2012-10-03 22:38:24Z

The problem is not with doctest, but the default parameter you are using in def fill_q(histq=deque([])). It is similar to this:

>>> from collections import deque
>>> 
>>> def fill_q(data=deque([])):
...     data.append(1)
...     return data
... 
>>> fill_q()
deque([1])
>>> fill_q()
deque([1, 1])
>>> fill_q()
deque([1, 1, 1])

This seemingly odd behaviour happens when you use a mutable object as a default value like a list or a dictionary. It is in fact using the same object:

>>> id(fill_q())
4485636624
>>> id(fill_q())
4485636624
>>> id(fill_q())
4485636624

Why?

Default parameter values are always evaluated when and only when the def statement they belong to is executed [ref].

How to avoid this mistake:

Use None as default parameter instead, or for arbitrary object:

my_obj = object()
def sample_func(value=my_obj):
    if value is my_obj:
        value = expression
    # then modify value

When to use it?:

local rebinding of global names:

import math

def fast_func(sin=math.sin, cos=math.cos):

can be used for memoization (e.g., make certain recursions run faster)

Peter Sobot · Accepted Answer · 2012-10-03 21:45:28Z

1

You're making a very common Python mistake - if you set an object to be a default constructor for a function, it will not be reinitialized on the next invocation of that function - and any changes to that object will persist across function calls.

A better strategy that avoids this problem is to set the default to some known value, and check for it:

def fill_q(histq=None):
    if histq is None:
        histq = deque([])
    ...

answered Oct 3, 2012 at 21:45

Peter Sobot

2,55721 silver badges20 bronze badges

Collectives™ on Stack Overflow

Python doctest execution context

2 Answers 2

Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related