Django profiling tests

Question

So I found this article from 2016 detailing how to create a custom test runner that captures the time it takes to run tests in Django:

runner.py:

from unittest.runner import TextTestResult
from django.test.runner import DiscoverRunner

import time

class TimeLoggingTestRunner(DiscoverRunner):
    def __init__(self, slow_test_threshold=0.0, *args, **kwargs):
        self.slow_test_threshold = slow_test_threshold
        return super().__init__(
            resultclass=TimeLoggingTestResult,
            *args,
            **kwargs,
        )
    def run(self, test):
        result = super().run(test)
        self.stream.writeln(
            "\nSlow Tests (>{:.03}s):".format(
                self.slow_test_threshold))
        for name, elapsed in result.getTestTimings():
            if elapsed > self.slow_test_threshold:
                self.stream.writeln(
                    "({:.03}s) {}".format(
                        elapsed, name))
        return result

    def get_resultclass(self):
        return TimeLoggingTestResult

class TimeLoggingTestResult(TextTestResult):
    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)
        self.test_timings = []
    def startTest(self, test):
        self._test_started_at = time.time()
        super().startTest(test)
    def addSuccess(self, test):
        elapsed = time.time() - self._test_started_at
        name = self.getDescription(test)
        self.test_timings.append((name, elapsed))
        super().addSuccess(test)
    def getTestTimings(self):
        return self.test_timings

settings.py:

TEST_RUNNER = 'core.utils.test_runner.runner.TimeLoggingTestRunner'

However, nothing prints at the end. I know the code runs, because debug statements fire in both classes - but the actual end result (run) doesn't appear to get called.

What I have managed to do is get the timings to print (by adding print statements inside TimeLoggingTestResult), but I can't seem to get them to all print out at the end.

Does anyone have any experience with doing something like this? The run method doesn't appear to be accessed at all.

Using Django 1.11.

I don't understand all your problem ( i'm not a django master) . But i created my own test_runner too, few days ago, and I am overriding run_tests method and not run. — AntoineLB
– AntoineLB, Commented Jun 18, 2018 at 15:19
Yeah, I saw that in the docs, but run_tests works differently, and I'm not sure how to get it to work with this code. — Darkstarone
– Darkstarone, Commented Jun 18, 2018 at 15:30
You're right, there is a run method in DiscoverRunner. Could you print what happen if you run tests with this runner but removing your own run method ? — AntoineLB
– AntoineLB, Commented Jun 18, 2018 at 15:37
So the tests run (because I get output from TimeLoggingTestResult) but I can't get the final result to print out. — Darkstarone
– Darkstarone, Commented Jun 18, 2018 at 15:41
I mean without overriding method ̀run` ? And also i don't understand what's missing in your output. — AntoineLB
– AntoineLB, Commented Jun 18, 2018 at 15:47

Bamse · Accepted Answer · 2021-04-30 07:12:20Z

I know this is an old question, but I came across it looking for a way to profile my tests and tried to implement it myself. I'm on Django 3.0.8 but had the same issue as the Asker.

After looking into it, I found that DiscoverRunner does not inherit from TextTestRunner, it uses composition. Instead of calling self.run() it calls runner.run(). See code below.

class DiscoverRunner:
    ...
    test_runner = unittest.TextTestRunner
    ...
    def run_suite(self, suite, **kwargs):
        kwargs = self.get_test_runner_kwargs()
        runner = self.test_runner(**kwargs)
        return runner.run(suite)

What I did was to make TimeLoggingTestRunner inherit both DiscoverRunner and TextTestRunner and overriding run_suite().

class TimeLoggingTestRunner(DiscoverRunner, TextTestRunner):
    def __init__(self, *args, slow_test_threshold=0.0, **kwargs):
        self.slow_test_threshold = slow_test_threshold
        DiscoverRunner.__init__(self, *args, **kwargs)
        TextTestRunner.__init__(self, *args, **self.get_test_runner_kwargs())
    [...]
    def run_suite(self, suite, **kwargs):
        return self.run(suite)

There's probably better ways to deal with the inheritance, but it's working.

Another option might be to create two TestRunners, inheriting DiscoverRunner and TextTestRunner separately, and then composing them the same way DiscoverRunner does.

Collectives™ on Stack Overflow

Django profiling tests

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related