Python unit-testing in multiple threads

Question

I would like to unit-test the functionality of a few classes that mainly do file in- and output. Furthermore I would like to do that on multiple cores (--jobs=4).

The problem is, that the files that are created by the classes often have the same name and they get mixed up in multiple threads. What I do currently is that I run each unit-test in a separate directory like so:

def test(self):
  if os.path.exists("UniqueDir"):
    os.system("rm -rf UniqueDir")
  os.mkdir("UniqueDir")
  os.chdir("UniqueDir")
  #Do the actual testing
  os.chdir("..")
  os.rmdir("UniqueDir")

The downsides are very obvious:

Each test must receive a unique directory name
Each test has this overhead of source which really is not pleasant to look at at all

What approach could I use to 1. separate my tests from one another but 2. do it in a more elegant way?

Any help, suggestion etc. is appreciated!

Cherio Woltan

You do know that threads in Python don't take advantage of multicore processors, right? For that you need multiprocessing (the interface to which is conveniently similar to Threading's) — nmichaels
– nmichaels, Commented Mar 25, 2011 at 14:41
Well I don't know exactly what the unit-test framework does when I set "--jobs=4". But it is something similar to multi threading. At least each test is run on a different core, which might be called multi-core processing...? Anyhow, that was not the issue ;) — Woltan
– Woltan, Commented Mar 25, 2011 at 14:54
you could use test runners such as nose, py.test to run your tests in parallel without the need to modify them. — jfs
– jfs, Commented Mar 25, 2011 at 16:53

Thomas Stachl · Accepted Answer · 2011-03-25 16:54:53Z

7

I would suggest to use the unittest module and build the classes like this:

import unittest
from tempfile import mkdtemp

class Test(unittest.TestCase):

    def setUp(self):
        self.tempdir = mkdtemp()
        os.chdir(self.tempdir)

    def tearDown(self):
        os.rmdir(self.tempdir)

    def testName(self):
        #Do the actual testing
        pass

if __name__ == "__main__":
    unittest.main()

Additionally you could add multiprocessing to create 4 threads.

Edit: removed the os.mkdir because mkdtemp creates a temp directory so it's bogus. Thx Sebastian.

edited Mar 25, 2011 at 16:54

answered Mar 25, 2011 at 16:20

Thomas Stachl

4282 silver badges8 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

jfs Over a year ago

tempfile.mkdtemp() creates a directory, so you should not call os.mkdir() on its result.

Woltan Over a year ago

+1 Nice idea, I'll give that a try first thing Monday morning.

Collectives™ on Stack Overflow

Python unit-testing in multiple threads

1 Answer 1

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related