Coverage.py vs. Python vs. TestNG vs. scikit-image Comparison Chart

Coverage.py is a tool for measuring code coverage of Python programs. It monitors your program, noting which parts of the code have been executed, then analyzes the source to identify code that could have been executed but was not. Coverage measurement is typically used to gauge the effectiveness of tests. It can show which parts of your code are being exercised by tests, and which are not. Use coverage run to run your test suite and gather data. However you normally run your test suite, and you can run your test runner under coverage. If your test runner command starts with “python”, just replace the initial “python” with “coverage run”. To limit coverage measurement to code in the current directory, and also find files that weren’t executed at all, add the source argument to your coverage command line. By default, it will measure line (statement) coverage. It can also measure branch coverage. It can tell you what tests ran which lines.

TestNG is a testing framework inspired from JUnit and NUnit but introducing some new functionalities that make it more powerful and easier to use, such as annotations, or the possibility to run your tests in arbitrarily big thread pools with various policies available (all methods in their own thread, one thread per test class, etc.). You can test that your code is multithread safe, there is flexible test configuration, support for data-driven testing (with @DataProvider), support for parameters, powerful execution model (no more TestSuite). There is a supported by a variety of tools and plug-ins (Eclipse, IDEA, Maven, etc.), it also embeds BeanShell for further flexibility, and there is default JDK functions for runtime and logging (no dependencies), and dependent methods for application server testing. TestNG is designed to cover all categories of tests, unit, functional, end-to-end, integration, etc.