Is it possible to create an abstract TestCase, that will have some test_* methods, but this TestCase won't be called and those methods will only be used in subclasses? I think I am going to have one abstract TestCase in my test suite and it will be subclassed for a few different implementation of a single interface. This is why all test methods are the some, only one, internal method changes. How can I do it in elegant way?
15 Answers
I didn't quite understand what do you plan to do -- the rule of thumb is "not to be smart with tests" - just have them there, plain written.
But to achieve what you want, if you inherit from unittest.TestCase, whenever you call unittest.main() your "abstract" class will be executed - I think this is the situation you want to avoid.
Just do this: Create your "abstract" class inheriting from "object", not from TestCase. And for the actual "concrete" implementations, just use multiple inheritance: inherit from both unittest.TestCase and from your abstract class.
import unittest
class Abstract(object):
def test_a(self):
print "Running for class", self.__class__
class Test(Abstract, unittest.TestCase):
pass
unittest.main()
update: reversed the inheritance order - Abstract first so that its defintions are not overriden by TestCase defaults, as well pointed in the comments bellow.
11 Comments
class Test(Abstract, unittest.TestCase). Otherwise the empty setUp method in unittest.TestCase takes precedence over the implemented one in Abstract.TestCase?SomethingMixin.There's a very simple way that everyone has missed so far. And unlike several of the answers, it works with all test drivers, rather than failing the minute you switch between them.
Simply use inheritence as usual, then add:
del AbstractTestCase
at the end of the module. This will prevent the Abstract class from being executed.
Example:
import unittest
from abc import abstractmethod, ABC
class AbstractSomethingTest(ABC, unittest.TestCase):
@abstractmethod
def implement_me(self):
pass
def test_implemented_method_should_return_inri(self):
self.assertEqual("INRI", self.implement_me())
class ImplementedSomethingTest(AbstractSomethingTest):
def implement_me(self):
return "INRI"
class AnotherClassImplementedSomethingTest(AbstractSomethingTest):
def implement_me(self):
return "INRI"
del AbstractSomethingTest
7 Comments
super still works, and everyone should really be using that.TypeError: Can't instantiate abstract class AbstractTestCase with abstract methods my_help_method,... Explain where to put the del (in the modules that inherit the base) and what to name the module that contains the base so it is ignored.If you really want to use inheritance instead of mixins, a simple solution is to nest the abstract test in another class.
It avoids issues with test runner discovery and you can still import the abstract test from another module.
import unittest
class AbstractTests:
class AbstractTest(unittest.TestCase)
def test_a(self):
print "Running for class", self.__class__
class Test(AbstractTests.AbstractTest):
pass
4 Comments
Multiple inheritance isn't a great option here, chiefly for the two following reasons:
- None of the methods in
TestCaseusesuper()so you'd have to list your class first for methods likesetUp()andtearDown()to work. - pylint will warn that the base class uses
self.assertEquals()etc which aren't defined onselfat that point.
Here's the kludge I came up with: turn run() into a no-op for the base class only.
class TestBase( unittest.TestCase ):
def __init__( self, *args, **kwargs ):
super( TestBase, self ).__init__( *args, **kwargs )
self.helper = None
# Kludge alert: We want this class to carry test cases without being run
# by the unit test framework, so the `run' method is overridden to do
# nothing. But in order for sub-classes to be able to do something when
# run is invoked, the constructor will rebind `run' from TestCase.
if self.__class__ != TestBase:
# Rebind `run' from the parent class.
self.run = unittest.TestCase.run.__get__( self, self.__class__ )
else:
self.run = lambda self, *args, **kwargs: None
def newHelper( self ):
raise NotImplementedError()
def setUp( self ):
print "shared for all subclasses"
self.helper = self.newHelper()
def testFoo( self ):
print "shared for all subclasses"
# test something with self.helper
class Test1( TestBase ):
def newHelper( self ):
return HelperObject1()
class Test2( TestBase ):
def newHelper( self ):
return HelperObject2()
1 Comment
Just to put in my two-cents, although it likely goes against some convention, you could define your abstract test case as a protected member to prevent its execution. I've implemented the following in Django and works as required. See example below.
from django.test import TestCase
class _AbstractTestCase(TestCase):
"""
Abstract test case - should not be instantiated by the test runner.
"""
def test_1(self):
raise NotImplementedError()
def test_2(self):
raise NotImplementedError()
class TestCase1(_AbstractTestCase):
"""
This test case will pass and fail.
"""
def test_1(self):
self.assertEqual(1 + 1, 2)
class TestCase2(_AbstractTestCase):
"""
This test case will pass successfully.
"""
def test_1(self):
self.assertEqual(2 + 2, 4)
def test_2(self):
self.assertEqual(12 * 12, 144)
2 Comments
Raise unittest.SkipTest in setUpClass()
Just another approach is to raise a unittest.SkipTest in setUpClass() of the base class and override setUpClass() in child classes:
class BaseTestCase(TestCase):
@classmethod
def setUpClass(cls):
"Child classes must override this method and define cls.x and cls.y"
raise unittest.SkipTest
def test_x(self):
self.assertEqual(self.x * 3, self.x)
def test_y(self):
self.assertEqual(self.y * 3, self.y + self.y + self.y)
def test_z(self):
self.assertEqual(self.x + self.y, self.y)
class IntegerTestCase(BaseTestCase):
@classmethod
def setUpClass(cls):
cls.x = 0
cls.y = 2
class StringTestCase(BaseTestCase):
@classmethod
def setUpClass(cls):
cls.x = ''
cls.y = 'zuzuka'
If you need to use custom TestCase that defines its own setUpClass() and you need to call super().setUpClass(), you can define you own method to "set up data" and raise SkipTest only inside that method:
class BaseTestCase(ThidPartyTestCase):
@classmethod
def setUpClass(cls):
super().setUpClass() # if ThirdPartyTestCase has own setUpClass()
cls.setUpTestCaseData()
@classmethod
def setUpTestCaseData(cls):
"Override and set up cls.x and cls.y here"
raise unittest.SkipTest
... # tests
class IntegerTestCase(BaseTestCase):
@classmethod
def setUpTestCaseData(cls):
cls.x = 0
cls.y = 2
The unittest module provides several options for skipping tests.
My preferred solution is to override the setUpClass method in the "abstract" base class to raise a unittest.SkipTest exception if needed:
class BaseTestCase(unittest.TestCase):
@classmethod
def setUpClass(cls):
if cls is BaseTestCase:
raise unittest.SkipTest("%s is an abstract base class" % cls.__name__)
else:
super(BaseTestCase, cls).setUpClass()
1 Comment
If you follow the convention of explicitly listing all test classes in run_unittest (see e.g. the Python test suite for many uses of that convention), then it will be straight-forward to not list a specific class.
If you want to continue using unittest.main, and if you can allow using unittest2 (e.g. from Python 2.7), you can use its load_tests protocol to specify which classes contain test cases). In earlier versions, you will have to subclass TestLoader, and override loadTestsFromModule.
1 Comment
Python unittest library has load_tests protocol, which can be used to achieve exactly what do you want:
# Add this function to module with AbstractTestCase class
def load_tests(loader, tests, _):
result = []
for test_case in tests:
if type(test_case._tests[0]) is AbstractTestCase:
continue
result.append(test_case)
return loader.suiteClass(result)
1 Comment
IndexError: list index out of rangeAnother reason for wanting to do what the OP is doing is to create a highly-parameterized base class which implements much of a set of core tests which need to be reproduced in several environments/scenarios. What I'm describing is essentially creating a parameterized fixture, a la pytest, using unittest.
Assuming you (like me) decide to run away as fast as you can from any multiple-inheritance-based solutions, one might have the following problem with using load_tests() to filter out your base class from the loaded suite:
In the standard TestLoader, load_tests is called after the auto-loading-from-class is done. Because: * this auto-loading-from-class will attempt to construct instances from your base class with the standard signature init(self, name), and * you may want this base class to have a very different ctor signature, or * you may want to skip construction-then-removal of your base class instances for some other reason
.. you may want to completely prevent this auto-loading of test instances from base classes.
EDIT: Vadim's solution in this other thread is a more elegant, concise, and independent way to do this. I have implemented the "nested class trick" and confirmed it works beautifully for the purpose of preventing TestLoader from "finding" your TestCase bases.
I originally had done this by modifying TestLoader.loadTestsFromModule to simply skip any TestCase classes which serve as base classes for any other TestCase classes in the module:
for name in dir(module):
obj = getattr(module, name)
# skip TestCase classes:
# 1. without any test methods defined
# 2. that are base classes
# (we don't allow instantiating TestCase base classes, which allows test designers
# to implement actual test methods in highly-parametrized base classes.)
if isinstance(obj, type) and issubclass(obj, unittest.TestCase) and \
self.getTestCaseNames(obj) and not isbase(obj, module):
loaded_suite = self.loadTestsFromTestCase(obj)
# ignore empty suites
if loaded_suite.countTestCases():
tests.append(loaded_suite)
where:
def isbase(cls, module):
'''Returns True if cls is base class to any classes in module, else False.'''
for name in dir(module):
obj = getattr(module, name)
if obj is not cls and isinstance(obj, type) and issubclass(obj, cls):
return True
return False
The parametrization I spoke of above is implemented by having each child class define it's fixture details (the parameters) and pass them to the base class TestCase ctor so that all of its common impl methods (the "fixturey" ones setUp*/tearDown*/cleanup* and the test methods themselves) have all the info that defines the now very specific fixture that that child TestCase class is to operate on.
For me, this was a temporary solution for quickly implementing some parametrized fixtures in unittest, since I plan to move my team's tests to pytest asap.
Comments
If you set __test__ = False in the base test class it should disable its tests. Quoting from this link:
class MessageTestBase(unittest.TestCase): __test__ = False def setUp(self): self.status = 'running' def tearDown(self): self.status = 'not running' def test_common_test(self): self.assertEqual(self.status, 'running') class TestSlack(MessageTestMixin): __test__ = TrueNotice the differences. Our mixin becomes a base class that inherits from TestCase. we include
__test__ = Falsein the base class to prevent the test runner from executing tests in this class. Then the child class only inherits from MessageTestBase and includes__test__ = Trueto instruct the test runner to run our tests.
More details here: How does __test__ = False magic attribute work for test discovery
1 Comment
unittest module.Here's a relatively simple approach that allows your common tests to inherit from TestCase (so type checking and IDE tooling stays happy), that uses only documented unittest features, and that avoids the "skip" test status:
import unittest
class CommonTestCases(unittest.TestCase):
def __init__(self, methodName='runTest'):
if self.__class__ is CommonTestCases:
# don't run these tests on the abstract base implementation
methodName = 'runNoTestsInBaseClass'
super().__init__(methodName)
def runNoTestsInBaseClass(self):
print('not running tests in abstract base class')
pass
def test_common(self):
# This will run *only* in subclasses. Presumably, this would
# be a test you need to repeat in several different contexts.
self.assertEqual(2 + 2, 4)
class SomeTests(CommonTestCases):
# inherited test_common *will* be run here
def test_something(self):
self.assertTrue(True)
# Also plays nicely with MRO, if needed:
class SomeOtherTests(CommonTestCases, django.test.SimpleTestCase):
# inherited test_common *will* be run here
def test_something_else(self):
self.client.get('/') # ...
How it works: per the unittest.TestCase documentation, "Each instance of TestCase will run a single base method: the method named methodName." The default "runTests" runs all the test* methods on the class—that's how TestCase instances normally work. But when running in the abstract base class itself, you can simply override that behavior with a method that does nothing.
A side effect is your test count will increase by one: the runNoTestsInBaseClass "test" gets counted as a successful test when it's run on CommonTestCases.
Comments
I know it's an old thread but it might help someone: I had a similar problem with my tests in a Django app that I had to upgrade from version 2.2 to version 4.2.
Previously, I’ve been using the django-nose library which has a nose based custom unit test runner, built on top of Django's own unit test runner. The feature asked by the OP was covered by the nose test runner. I think django-nose only run the tests of a test class if the name of the class starts as Test - hence I adopted a convention to name my abstract test classes as _Test.
Since django-nose (and nose) is in maintenance mode I had drop it but Django's built-in test runner (unittest2) collects and run all tests whose class is a subclass of unittest.TestCase.
So my solution was to subclass Django's DiscoverRunner the following way:
from unittest import TestSuite
from django.test.runner import DiscoverRunner
class CustomDiscoverRunner(DiscoverRunner):
def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
self.filtered_out = 0
def build_suite(self, test_labels=None, extra_tests=None, **kwargs) -> TestSuite:
print("Custom DiscoverRunner starts up...")
suite = super().build_suite(test_labels=test_labels, extra_tests=extra_tests, **kwargs)
# noinspection PyProtectedMember
# pylint: disable=protected-access
filtered_tests = [t for t in suite._tests if self._needed(t)]
print(f"Running {len(filtered_tests)} tests -- filtered out: {self.filtered_out}")
return TestSuite(tests=filtered_tests)
def _needed(self, test_name) -> bool:
"""
DiscoverRunner will collect all tests from all classes that are subclassing unittest.TestCase
This custom runner should only keep those tests whose class name starts with "Test{ExampleClass}"
This is to make sure that tests in abstract test classes will be filtered out
As a guideline, such classes should follow the "_Test{AbstractTestClass}" naming pattern
Tests with _Failed class name component are kept to make sure errors will be reported correctly:
if there is an error at test load time, test gets the class assigned as "unittest.loader._FailedTest"
This happens when using incorrect test labels: e.g.: "python manage.py test NoModuleLikeThis"
Keeping such failures in the suite will make sure, they will "run" and the error of incorrect label is reported
"""
if (
test_name.__class__.__name__.startswith("Test") or
test_name.__class__.__name__.startswith("_Failed")
):
return True
self.filtered_out += 1
return False
After this the custom runner can be setup to be the default runner by setting the corresponding settings variable:
TEST_RUNNER = 'myproject.unittest_runner.CustomDiscoverRunner'
Comments
I agree with the answer of @tsuna and the comment of @interDist in this similar question.
A further improvement can be using a decorator to explicitly mark test classes which are meant to be base or abstract and their test methods must only be executed for the derived classes:
from unittest import TestCase
def abstract_test(cls):
def run(self, result=None):
return result if self.__class__ is cls else TestCase.run(self, result)
cls.run = run
return cls
@abstract_test
class Base(TestCase):
"""This test shall only be executed in subclasses."""
def test_b(self):
print(f"{Base.__doc__} {self.__class__.__name__}.test_b from {Base.__name__}")
@abstract_test
class Base2(Base):
"""This too."""
def test_b2(self):
print(f"{Base2.__doc__} {self.__class__.__name__}.test_b2 from {Base2.__name__}")
class Derived(Base2):
def test_c(self):
print(f"{self.__class__.__name__}.test_c")
The decorator replaces the run method inherited from TestCase instead of overriding it in a base test class. Using the decorator avoids the code repetition and makes it more obvious which class is a base or abstract test.
However, executing the same inherited test in derived test cases may indicate that the units (under test) are not well designed:
- The code tested by the inherited test should be in a separate unit;
- If this code fails, it breaks several tests instead of a single one;
- It lasts longer or consumes more resources to test the same logic multiple times.
Comments
I have done it following way, maybe it can inspire you:
class AbstractTest(TestCase):
def setUp(self):
pass
def tearDown(self):
pass
def _test_1(self):
# your test case here
class ConcreteTest(AbstractTest)
def test_1(self):
self._test_1()
Although it's not the most convenient solution, it lets you escape from multi inheritance. Also, the solution suggested by Dan Ward didn't work with Django testing in PyCharm.
__test__=Falsein your base class.