Python: Running tests with all combinations of feature flags

We have several modules that require mandatory feature / backout flags. These flags are defined at module level.

module.py:

from enabled import is_enabled

FLAGS = {flag : is_enabled(flag) for flag in ("foo", "bar")}

if FLAGS.get("foo"):
    def baz():
        print("New baz")
else:
    def baz():
        print("Old baz")

print(f"Loaded module with {FLAGS=}")
print("#"*100)

enabled.py:

def is_enabled(_):
    return True

I can use mock.patch to patch is_enabled and by using importlib.reload I can test all combinations of True and False for all flags for a module(max number of flags at the same time is 5 or 6 so combination is not going to explode).

My main problem is now running the test with all combinations; I have tried:

1. Creating a decorator : this fails because setup and teardown methods are not called between repeated runs in the decorator causing assertions to fail. I can either call self.setUp and self.tearDown in each test OR pass those methods to the decorator and call them but it feels like a messy approach which creates dependencies and is fragile. I'm going to avoid this if possible.

2. Creating a test runner (subclass of TextTestRunner) : Unfortunately a test runner already exists and is setup. I'm not sure if it is possible / how to use my own test runner instead of the existing test runner (which can't be easily swapped out / refactored as it is used a lot). Is there a way for me to override / ignore that and use my own? This approach is cleaner and technically more correct as the the test suite is ran multiple times with all the necessary setup /tear-down, captured results, etc.

given the unit test below, can you please demonstrate how to run the test_baz with all combinations of "foo" and "bar" being True and False?

module_test.py

import module
from importlib import reload
from unittest import TestCase
from itertools import product
from functools import wraps, partial
from unittest.mock import patch, DEFAULT


def run_all_combinations_decorator(func):
    flags = sorted(module.FLAGS.keys())
    combinations = list(product((True, False), repeat=len(flags)))

    def is_enabled(new_flags, flag):
        return new_flags.get(flag, DEFAULT)

    @wraps(func)
    def wrapper(*args, **kwargs):
        for combination in combinations:
            new_flags = {flag: enabled for flag, enabled in zip(flags, combination)}
            print(f"Running with {new_flags=}")
            with patch("enabled.is_enabled", side_effect=partial(is_enabled, new_flags)):
                reload(module)
                result = func(*args, **kwargs)  # Not ideal as only the last result is captured
        return result

    return wrapper


class ModuleTests(TestCase):
    def setUp(self):
        self.x = 0

    @run_all_combinations_decorator
    def test_baz(self):
        # self.setUp() # Uncommenting this makes the test pass but is not ideal
        module.baz()
        self.x += 1
        self.assertEqual(self.x, 1)  # Fails because setup is not called between runs

If there is a better approach to this that I have not yet tried, please suggest that as well.