Testing multiple versions of the same programs with pytest

Question

I have several Python programs with multiple variants, e.g.:

sort_bubble_0.py
sort_bubble_1.py
sort_insertion_0.py
sort_insertion_1.py
sort_selection_0.py
...
unique_elements_0.py
unique_elements_1.py
unique_elements_2.py
...

In each group, all variants have the same interface.

In pytest, is it possible to have a unique test suite for each group, which would be launched successively on all the variants?

Ideally, a small snippet of code in the test suite would be able to discover all the appropriate variants based on a regular expression (e.g., r"sort_.*\.py"), import and test them one by one.

For now on, I just duplicate the same test suite as many times as needed, and manually change the imported name in each copy! I guess there should be a better way to do this.

MrBean Bremen · Accepted Answer · 2020-08-25 09:52:20Z

1

A simple solution would be to parametrize the tests with the modules:

import pytest


def get_tested_modules(module_name):
    modules = []
    for i in range(100):
        try:
            modules.append(importlib.import_module(module_name + str(i)))
        except ImportError:
            break
    return modules
    
@pytest.mark.parametrize("module", get_tested_modules("sort_bubble_"))
def test_sort_bubble(module):
    assert module.sort([3, 2, 1]) == [1, 2, 3]

(simplified after comment by OP)

edited Aug 25, 2020 at 9:52

answered Aug 25, 2020 at 9:16

MrBean Bremen

17.1k3 gold badges36 silver badges58 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

Aristide Over a year ago

Thanks a lot! I can certainly try to adapt your first approach by creating a list of programs with Path.glob(pattern), then importing all of them with importlib and feeding the list to @pytest.mark.parametrize(). For now, I am not familiar enough with pytest to appreciate your second solution, but I will certainly study it too.

MrBean Bremen Over a year ago

Actually, you are right: you don't need the second approach at all. I will simplify the answer.

Aristide Over a year ago

Great! I have not thought using a function with side-effect. Besides, in case you don't know it, I would recommend you to give pathlib module a look: its function glob can simplify the enumeration of the relevant files, while avoiding the exception construct.

Collectives™ on Stack Overflow

Testing multiple versions of the same programs with pytest

1 Answer 1

3 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

3 Comments

Your Answer

Sign up or log in

Post as a guest

Related