Is it possible to mark a test as "expected to fail" with PHPUnit? This would be useful when performing TDD, and you want to distinguish between genuinely failed tests, and tests that happen to fail because the associated code hasn't been written yet.
Add a comment
|
6 Answers
I think in these cases, it's fairly standard to simply mark the test as skipped. Your tests will still run and the suite will pass, but the test runner will alert you of the skipped tests.
http://phpunit.de/manual/current/en/incomplete-and-skipped-tests.html
3 Comments
mjs
I guess that'll do. I was looking for something like the support for TODO blocks in Perl's Test::More module perldoc.perl.org/Test/More.html#Conditional-tests which will run the tests that are expected to fail, and report if they (unexpectedly) succeeded. (PHPUnit seems to only be able to skip completely.)
syl.fabre
I think that Tom B answer below about markTestIncomplete method is better
Alcalyn
That does not do the trick. A solution should be to mark test as TDD, so it displays
I if the test fails (as not implemented yet), and most importantly, it should fail if the test passes as it means that we forgot to remove the TDD mark. I don't know how to do the trick easily in phpunitThe 'correct' method of handling this is to use $this->markTestIncomplete(). This will mark the test as incomplete. It will come back as passed, but it will display the message provided. See http://www.phpunit.de/manual/3.0/en/incomplete-and-skipped-tests.html for more information.
1 Comment
Matthew Flaschen
The question is about TDD, where you write (ideally complete) tests before the main code. However,
markTestIncomplete is for when you have an "unimplemented test" (phpunit.de/manual/3.7/en/incomplete-and-skipped-tests.html starts by describing an empty test method for an unimplemented test, then explains how that leads to a false success).I really think it's a bad practice, however you can trick PHPUnit this way:
/**
* This test will succeed !!!
* @expectedException PHPUnit_Framework_ExpectationFailedException
*/
public function testSucceed()
{
$this->assertTrue(false);
}
More cleanly:
public function testFailingTest() {
try {
$this->assertTrue(false);
} catch (PHPUnit_Framework_ExpectationFailedException $ex) {
// As expected the assertion failed, silently return
return;
}
// The assertion did not fail, make the test fail
$this->fail('This test did not fail as expected');
}
2 Comments
philippe_b
The code above works, yet I had to catch
PHPUnit_Framework_AssertionFailedError, not PHPUnit_Framework_ExpectationFailedException
aqm
it's good practice if your actually testing a assertion it's self, eg testAssertEqual_fail :D so you can check your custom asserts fail as expected, but for genral use, your doing something wrong
The comment by sixty-nine above is nearly perfect for what I was searching for.
The fail() method is useful for when you set a test for an expected exception and if it did not trigger the exception you want the test to fail.
$this->object->triggerException();
$this->fail('The above statement was expected to trigger and exception.');
Of course the triggerException is replaced by something in your object.
Comments
If you want to have a test fail but know that its failure was expected, you can add a message to the assertion that will output in the results:
public function testExpectedToFail()
{
$this->assertTrue(FALSE, 'I knew this would happen!');
}
In the results:
There was 1 failure:
1) testExpectedToFail(ClassTest)
I knew this would happen!
3 Comments
Ryan Chouinard
This would cause the suite to fail, which doesn't seem to be the desired result.
Clay Hinson
The question mentioned TDD, in which you write failing tests against code that doesn't exist.
Parris Varney
$this->fail('I knew this would happen!'); is shorthand for that :).
In PHPUnit 8.2.5 you can simply expect the thrown assertion exception:
$this->expectException('PHPUnit\Framework\ExpectationFailedException');
$this->assertTrue(false);
