How can I configure pytest to generate useful names when generating tests?

Question

I'm using py.test to execute a suite of selenium tests. I'm essentially running a collector in my conftest.py that generates tests like this (I stole this from the pytest documentation):

def pytest_generate_tests(metafunc):
    funcarglist = metafunc.cls.cases[metafunc.function.__name__]
    argnames = list(funcarglist[0])
    metafunc.parametrize(argnames, [[funcargs[i] for i in argnames]
                                for funcargs in funcarglist])

My test cases are placed in objects that look like this:

class TestObject(object):

def __init__(
        self,
        parameter_1,

):
    self.parameter_1 = parameter_1
    self.parameter_2 = parameter_2

I instantiate them something like this:

test_cases_values = {
    "friendly_case_name_1": TestObject(
        "parameter_1_value",
        "parameter_2_value"
    ),
    "friendly_case_name_2": TestObject(
        "parameter_1_value",
        "parameter_2_value"
    ),
}

My browsers are attached to a grid server, I make a list of them like this:

BROWSERS = [
    "('browser_1', SERVER_URL)",
    "('browser_2', SERVER_URL)"
]

I store the target environment in a config file that is an instantiation of an object like this:

class Environment(object):

    def __init__(self, url=URL, port=PORT):
        self.url = url
        self.port = port

    def __name__(self):
        return self.url + ":" + self.port

ENVIRONMENT = Environment()

Then I have a test class that creates a list of test cases like this - the test object parameters are actually strings that allow for self generating code. I'm oversimplifying as I pass them in as fill ins to broader exec statements:

class TestClass(object):

    cases = {"test_function": []}
    for i in test_cases.values():
        for j in BROWSERS:
            cases["test_function"].append(
                dict(
                    browser=j,
                    environment=ENVIRONMENT
                    test_object=i
                )
            )

    @pytest.mark.run()
    def test_function(
        self,
        browser,
        environment,
        test_object
    ):
        exec(test_object.parameter_1)
        exec(test_object.parameter_2)
        assert my_assertion

When the collector runs, it looks like this:

collected # items
    <Module 'tests.py'>
      <Class 'TestClass'>
        <Instance '()'>
          <Function "test_function[environment0-test_object0-('browser_1', GRID_SERVER)]">
          <Function "test_function[environment1-test_object1-('browser_2', GRID_SERVER)]">
          <Function "test_function[environment2-test_object2-('browser_1', GRID_SERVER)]">
          <Function "test_function[environment3-test_object3-('browser_2', GRID_SERVER)]">

I want to have the collector work in such a way that I get back useful information about each item - I've messed around with setting __str__, __repr__, and __name__ methods in various places but haven't had the results I expected. I'd like to be able to roll this into reporting - there are over 200 tests that this generates in production and I have to trace through stack traces currently to figure out exactly what was being tested for each failure.

I'm not really sure where I'm making my mistakes here, should I modify my implementation of pytest_generate_tests, or the way I'm creating my TestClass, or set up the cases in a different way? Ideally, I want something that can be mapped back via an ORM to include test metadata as well.

Check this out ,could be useful github.com/allure-framework/allure-python/issues/47 — Sanju
– Sanju, Commented Jul 14, 2015 at 16:02

Brovaccie · Accepted Answer · 2015-07-14 21:06:33Z

Well, I figured it out. Turns out that the metafunc.parametrize function accepts "ids" as a parameter. All I had to do was define the __repr__ of the objects I was looking to name, and expanded the list comprehension so I could return two things from the same loop.

def pytest_generate_tests(metafunc):
    funcarglist = metafunc.cls.cases[metafunc.function.__name__]
    argnames = list(funcarglist[0])
    argvalues = []
    ids = []
    for i in funcarglist:
        inner_argvalues_list = []
        inner_ids_list = []
        for j in argnames:
            inner_argvalues_list.append(i[j])
            if type(i[j]) != str:
                inner_ids_list.append(i[j].__repr__())
            else:
                inner_ids_list.append(i[j])
        argvalues.append(inner_argvalues_list)
        ids.append(inner_ids_list)
    metafunc.parametrize(argnames, argvalues, ids=ids)

Collectives™ on Stack Overflow

How can I configure pytest to generate useful names when generating tests?

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related