7

Assuming the following test suite:

# test_module.py
import unittest

class Tests(unittest.TestCase):

  @unittest.skip
  def test_1(self):
    print("This should run only if explicitly asked to but not by default")

  # assume many other test cases and methods with or without the skip marker

When invoking the unittest library via python -m unittest are there any arguments I can pass to it actually run and not skip Tests.test_1 without modifying the test code and running any other skipped tests?

python -m unittest test_module.Tests.test_1 correctly selects this as the only test to run, but it still skips it.

If there is no way to do it without modifying the test code, what is the most idiomatic change I can make to conditionally undo the @unittest.skip and run one specific test case test case?

In all cases, I still want python -m unittest discover (or any other invocation that doesn't explicitly turn on the test) to skip the test.

Improve this question
3
  • The question is why are these tests skipped in the first place? Usually skipping a test involves some condition in which case it makes no sense to run the test anyway because the required resources might not be available. Unconditionally skipping a test makes only sense if you remove the skip some time in the future, so that involves source code modification anyway. Otherwise you could just remove the test from the suite completely. Commented Mar 24, 2020 at 18:40
  • 1
    Thank you @a_guest. These are long-running tests that could be run any time but they are off by default because of the time they take. I want to manually execute some of them (but not all at the same time). Commented Mar 24, 2020 at 18:43
  • Another case is a running test I expect to fail because of a bug in the system under test. But I might want to run it anyway to see if a change in the code fixed it to see if the skip decorator can be removed. Commented Mar 24, 2020 at 18:45

1 Answer 1

Reset to default
6

If you want to skip some expensive tests you can use a conditional skip together with a custom environment variable:

@skipIf(int(os.getenv('TEST_LEVEL', 0)) < 1)
def expensive_test(self):
    ...

Then you can include this test by specifying the corresponding environment variable:

TEST_LEVEL=1 python -m unittest discover
TEST_LEVEL=1 python -m unittest test_module.Tests.test_1

If you want to skip a test because you expect it to fail, you can use the dedicated expectedFailure decorator.

By the way, pytest has a dedicated decorator for marking slow tests.

Improve this answer
Sign up to request clarification or add additional context in comments.

4 Comments

Thank you! This almost works for me, but this seems to imply that I either define a separate environment variable for each test method or otherwise TEST_LEVEL=1 python -m unittest discover will run all my tests instead of the one I want.
You can combine this with pattern matching of test methods, so TEST_LEVEL=1 just means activate expensive tests and then what tests you will run depends on the pattern you specify.
I edited the answer to add this. Accepted and thanks!
Sadly, not applicable for tests that I have no control on the source. For example, on exercism.org, a student can download and work on the exercises locally, but any change to test files are ignored upon submission. It'd really be useful to have a command line flag that I could set when running locally, such as --include-skipped.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.