Python unittest, results vary depending on print statement

Question

I'm mildly confused. I'm testing a django application with python's unittest library. All of a sudden, after running my tests with 100 % success for some minutes, suddenly an error appears. Ok I thought, I must have just added some stupid syntax error. I started looking at the test and then my code, I then tried to print out the results which are being compared with assertEqual before they are compared. Suddenly if I do that, the test runs!!! :o

Why is this? has anyone experienced this before. I swear, the only change I made was adding a print statement inside my test function. I'll post this function before and after

Before (Fails)

def test_swap_conditionals(self):
    """
    Test conditional template keys
    """
    testStr = "My email is: {?email}"
    swapStr = self.t.swap(testStr)
    # With email
    self.assertEqual(swapStr, "My email is: [email protected]")

    # Without email
    self.t.template_values = {"phone" : "00458493"}
    swapStr = self.t.swap(testStr)
    self.assertEqual(swapStr, "My email is: ")

After (Success)

def test_swap_conditionals(self):
    """
    Test conditional template keys
    """
    testStr = "My email is: {?email}"
    swapStr = self.t.swap(testStr)
    print(swapStr) #diff here
    # With email
    self.assertEqual(swapStr, "My email is: [email protected]")

    # Without email
    self.t.template_values = {"phone" : "00458493"}
    swapStr = self.t.swap(testStr)
    self.assertEqual(swapStr, "My email is: ")

@RickyA correct, this was my fault when copying this in here. The snippets are exactly the same except in the second one I print out swapStr and for some reason this makes the test run — Donna
– Donna, Commented Dec 15, 2014 at 11:23
what happens when you remove the print again now? and on what assert does it fails? — RickyA
– RickyA, Commented Dec 15, 2014 at 11:25
With the print removed it fails unless I comment out the part #Without email. So it's failing on this assertion: self.assertEqual(swapStr, "My email is: "). self.t.template_values contains a dictionary with values that are suppose to be swapped out if they are present, the {?email} means, check if email is a valid key, if it's not present then return "", that's why I'm expecting to get "My email is: " when I overwrite self.t.template_values to contain only the key phone. — Donna
– Donna, Commented Dec 15, 2014 at 11:31
can it be self.t still contains the email after the second call to self.t.swap? print(self.t) after each swap and print(swapStr) after each assignment. — RickyA
– RickyA, Commented Dec 15, 2014 at 11:35

Michel Keijzers · Accepted Answer · 2014-12-15 11:25:16Z

1

It looks like there is some external reason.

What you can check:

Rerun the test several times under the same conditions. Is it always failing or passing? Or is it a 'flipper' test? This might be caused by timing issues (although unlikely).
Put the test in its own class, so there are no side effects from other unit tests.
If the test in its own test class was passing the reason is a side effect by:
- other unit tests
- startup/teardown functionality

answered Dec 15, 2014 at 11:25

Michel Keijzers

15.4k28 gold badges100 silver badges124 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Donna Over a year ago

Ok so I put both tests in their own isolated functions, they are not depending on anything from setUp, they create their own instance of the Template class, fill in the dictionary themselves and then swap out the values. This works, I've tried running the tests couple of times and they always return success.

Donna · Accepted Answer · 2014-12-15 13:42:05Z

Ok well embarrassing, but this was completely my fault. The swap function was looking up every conditional template variable on the line and then iterating over that list one conditional template variable at a time, so either it missed keys it already had or it got lucky and it happened to hit that key.

Example

line: "This is my {?email}, and this is my {?phone}"

finds:
[{?email}, {?phone}]

iterates over [{?email}, {?phone}]
1. {?email}
key being compared = phone : '00549684'

It has phone as a key but it completely disregards it and does not swap it out because it's just holding {?email} so it simply returns "".

I'm sincerely sorry to waste all your time here. Thanks for good answers. I am refactoring the code now for the better, and definitely taking a coffee break :D

Collectives™ on Stack Overflow

Python unittest, results vary depending on print statement

2 Answers 2

1 Comment

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

1 Comment

Comments

Your Answer

Sign up or log in

Post as a guest

Related