3

For a specific program I'm working in, we need to evaluate some code, then run a unittest, and then depending on whether or not the test failed, do A or B.

But the usual self.assertEqual(...) seems to display the results (fail, errors, success) instead of saving them somewhere, so I can't access that result. I have been checking the modules of unittest for days but I can't figure out where does the magic happen or if there is somewhere a variable I can call to know the result of the test without having to read the screen (making the program read and try to find the words "error" or "failed" doesn't sound like a good solution).

4
  • If I get it right, you are altering AST during unit test, would you like explain a bit why? Commented Dec 28, 2017 at 16:27
  • I guess pytest hooks can be useful to this scenario. Commented Dec 28, 2017 at 16:29
  • Are you trying to do so-called "fuzzing"? Commented Dec 28, 2017 at 20:50
  • I think it's not exactly fuzzing. But as I understand it (random tests to check for stuff), this might be used for fuzzing. What I'm trying to do is a piece of code which writes a test of a certain type given a function with its inputs and expected output. But I want my program to generate the tests by trial and error. Commented Jan 2, 2018 at 11:47

2 Answers 2

1

After some days of researching, I sent an email to [email protected] and got the perfect solution for my issue. The answer I got was:

I suspect that the reason that you're having trouble getting unittest to do that is that that's not the sort of thing that unittest was written to do. A hint that that's the case seems to me to be that over at the documentation:

https://docs.python.org/3/library/unittest.html

there's a section on the command-line interface but nothing much about using the module as an imported module.

A bit of Googling yields this recipe:

http://code.activestate.com/recipes/578866-python-unittest-obtain-the-results-of-all-the-test/

Which looks as though it might be useful to you but I can't vouch for it and it seems to involve replacing one of the library's files. (Replacing one of the library's files is perfectly reasonable in my opinion. The point of Python's being open-source is that you can hack it for yourself.)

But if I were doing what you're describing, I'd probably write my own testing code. You could steal what you found useful from unittest (kind of the inverse of changing the library in place). Or you might find that your needs are sufficiently simple that a simple file of testing code was sufficient.

If none of that points to a solution, let us know what you get and I'll try to think some more.

Regards, Matt

After changing my result.py module from unittest, I'm able to access the value of the test (True, False, or Error).

Thank you very much, Matt.

P.S. I edited my question so it was more clear and didn't have unnecessary code.

Sign up to request clarification or add additional context in comments.

Comments

0

You can use pdb to debug this issue, in the test simply add these two lines to halt execution and begin debugging.

import pdb pdb.settrace()

Now for good testing practice you want deterministic test results, a test that fails only sometimes is not a good test. I recommend mocking the random function and using data sets that capture the errors you find.

1 Comment

If I'm understanding your answer, you think I have a test which I coded wrong and I want to know where the error is. And it's not. It's a generator of random tests which could be right, wrong, or syntactically incorrect and I want to discard the wrong ones or the syntactically incorrect ones to eventually get to a right test. I'm sorry if I wasn't clear enough, I edited the question.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.