pytest dynamically parametrized test

Question

I have created a 'dynamic' list of parameters which I pass to parametrize.

OPTIONS = ['a', 'b', 'c']

def get_unique_pairs():
    unique_list = []
    for first in OPTIONS:
        for second OPTIONS:
            if first == second:
                continue
            unique_list.append({'first':first, 'second':second))
    return unique_list

def some_func()
    unique_pairs = get_unique_pairs()
    result = []
    for pair in unique_pair:
        if test(pair):
           continue
        else:
           result.append(pair)
    return pair

@pytest.mark.parametrize('param', some_fnc())
def test_fnc(param):
    first = param['first']
    second = param['second']

The input I wish to pass to test_fnc is [('a','b'),('a','c')...('c','b')] where the first and second elements are never the same. There is some additional logic I am using to further remove specific pairs.

When I run the test I get the output:

::test_fnc[param0] PASSED
::test_fnc[param1] PASSED
::test_fnc[param2] PASSED

I have two issues:

I'm not entirely sure how to describe what I am doing to find further documentation / help.
I would like more descriptive output (i.e. not param0), and I'd like to continue using dictionaries to pass data to the test.

I'm under the impression that your get_unique_pairs can be replaced by list(itertools.combinations(OPTIONS, 2)). Also, take a look at generator functions and generator expressions, they can make your code more concise and idiomatic. — Paulo Scardine
– Paulo Scardine, Commented Jun 12, 2018 at 13:06
Thats pretty cool - thanks. I'll have a play, I'd like it to generate ['c', 'a'] - inverting the combinations. — F. Elliot
– F. Elliot, Commented Jun 12, 2018 at 13:55
Combinatoric iterators are: product, permutations, combinations and combinations_with_replacement. The combinations(p, r) iterator returns r-length tuples in sorted order, no repeated elements. Product is equivalent to your nested for loop: [(a, b) for a, b in product(OPTIONS, OPTIONS) if a != b] — Paulo Scardine
– Paulo Scardine, Commented Jun 12, 2018 at 14:09
itertools.permutations(OPTIONS, 2) is a much cleaner way of writing what I was doing! — F. Elliot
– F. Elliot, Commented Jun 12, 2018 at 14:13
Everyday you learn a new thing. Welcome to StackOverflow! Python community here is awesome. — Paulo Scardine
– Paulo Scardine, Commented Jun 12, 2018 at 14:17

Paulo Scardine · Accepted Answer · 2018-06-12 15:37:57Z

I would write it like this:

import pytest
import itertools


OPTIONS = ['a', 'b', 'c']

@pytest.mark.parametrize(
    'param',
    itertools.permutations(OPTIONS, 2),
    ids=lambda pair: "first={}, second={}".format(*pair)
)
def test_fn(param):
    first, second = param
    assert first != second

Result:

> pytest --verbose 
================================ test session starts =================================
platform linux2 -- Python 2.7.12, pytest-3.6.1, py-1.5.3, pluggy-0.6.0 -- /usr/bin/python
cachedir: .pytest_cache
rootdir: /home/paulos/work/lixo/tests, inifile:
collected 6 items

test_foo.py::test_fn[first=a, second=b] PASSED                                     [ 16%]
test_foo.py::test_fn[first=a, second=c] PASSED                                     [ 33%]
test_foo.py::test_fn[first=b, second=a] PASSED                                     [ 50%]
test_foo.py::test_fn[first=b, second=c] PASSED                                     [ 66%]
test_foo.py::test_fn[first=c, second=a] PASSED                                     [ 83%]
test_foo.py::test_fn[first=c, second=b] PASSED                                     [100%]

================================ 6 passed in 0.03 seconds ================================

[update]

I think this answer is the solution I will use. Every day is a school day! (Out of curiosity is there a way I could be the 'param' into something like 'first, second' so to have test_foo(param) be test_foo(first, second). I'm not sure that actually helps anything... but I am curious – F. Elliot

If you don't mind a test_fn[a-b] instead of test_fn[a, b]:

@pytest.mark.parametrize(
    'first,second',
    itertools.permutations(OPTIONS, 2),
)
def test_fn(first, second):
    assert first != second

In practice we don't really run tests with --verbose anyway, so most of the time the output will be just a dot for each test.

I think this answer is the solution I will use. Every day is a school day! (Out of curiosity is there a way I could be the 'param' into something like 'first, second' so to have test_foo(param) be test_foo(first, second). I'm not sure that actually helps anything... but I am curious

hoefling · Accepted Answer · 2018-06-12 13:17:43Z

5

As Paulo Scardine mentioned in the comment, don't reinvent the wheel, use itertools.combinations.
You are looking for the ids argument in parametrize hook. It's either list of names for each argument, or a function that accepts the argument and returns a string - repr is a simple solution for builtin types.

Combined example:

import itertools
import pytest


opts = ['a', 'b', 'c']


@pytest.mark.parametrize('param', itertools.combinations(opts, 2), ids=repr)
def test_fn(param):
    first = param[0]
    second = param[1]
    assert first != second

answered Jun 12, 2018 at 13:17

hoefling

67.8k15 gold badges182 silver badges228 bronze badges

4 Comments

F. Elliot Over a year ago

I found this and Paulos post useful - thank you for the link also, I didnt realise ids could be a function. (I spent some time trying seeing if I could separate param out by using a dict, but I guess I'll just kick my obsession with dictionaries for now!)

hoefling Over a year ago

You can have dicts if you want - just build them from itertools.combinations(opts, 2), e.g. [{'first': first, 'second': second} for first, second in itertools.combinations(opts, 2)].

F. Elliot Over a year ago

How would I changed 'ids'? I feel like the keys of the dictionary are being passed in

hoefling Over a year ago

You can continue using repr, or write a custom format function, for example def dump(param): return ', '.join('{}={}'.format(k, v) for k, v in param.items()) and use it with ids=dump. This will give you test names like test_fn[first=a, second=b].

Collectives™ on Stack Overflow

pytest dynamically parametrized test

2 Answers 2

1 Comment

4 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

1 Comment

4 Comments

Your Answer

Sign up or log in

Post as a guest

Related