0

Basically I want to be able to write a unit test that will tell me if a change I just made has changed performance. I'm aware of a similar question, but the answer to that was kind of "that's not what you want", so I'm rephrasing.

I've noticed when doing performance profiling in python that a key issue is lines of python code, and that more lines of code == worse performance. Hard to disagree with. Frequently the biggest performance gains I've got are by identifying a routine that is called unnecessarily frequently, and reducing it.

Therefore what I want is to be able to write a unit test like this:

def test_performance_of_critical_routine():
    start_counting_statement_executions()
    critical_routine()
    stop_counting_statement_executions()
    assert number_of_executions() == previously_accepted_count

The idea is that this is deterministic and the same on all platforms. If the test fails, then I can decide whether I think the regression / improvement is reasonable, and it's easy to update the test. But if there's a big change and it was unexpected, I've caught an issue at UT stage, which is always good!

I know that there are many things that affect application performance, but CPU is occasionally one of them, and I'd like to be able to catch it earlier.

Any suggestions?

Improve this question
2
  • You should probably time the function, not count the statements. Statement number doesn't matter much. Some operations are O(N), O(1) or O(N·log N) for instance, that matters much more than how many of them you write. Also, that would tempt devs to write complex one-liners to reduce statement number, gaining nothing in actual performance while cutting maintainability a lot. Commented Nov 25, 2016 at 12:10
  • @spectras, yeah, I'm aware it's not perfect, but I still think it can be useful, and I think for a large framework statement number can be very significant. Stopping devs from being idiots is the job of code review anyway :-) Commented Nov 25, 2016 at 12:15
Related questions
56
How to know time spent on each test when using unittest?
1
assertIn with count
13
Display number of assertions in python unit tests
Load 7 more related questions Show fewer related questions

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.