0

The problem appears when test case marked with "async" is failing. App is just crashing in that case, with an unhandled exception of TargetInvocationFail. As far as I understand, the test app is supposed to handle those exceptions, because in case of exception it should just mark respective case as failed. This is exactly what happens with the normal test case (without async / await stuff in it).

I've created an issue report for this as well, see http://phone.codeplex.com/workitem/10751. If you have same problem, please upvote the issue. And if you happen to know some workaround, please let me know here.

EDIT: As Stephen Cleary mentioned in the comment, the issue was caused by the fact that my test case procedure was async void, not async Task. I'd reformulate this question as follows then: why changing return type of test case changes the behavior of the exception handling?

3
  • 1
    You didn't post repro code, either here or in your issue report. If I were to take a guess, I'd say that your unit test method is async void when it should be async Task. Commented Jan 18, 2013 at 14:44
  • You didn't ask any question. Do you have one? Commented Jan 18, 2013 at 14:46
  • @StephenCleary you're absolutely right, making stuff async Task instead of async void fixes this. May I humbly ask you to answer why this happens? I'll edit the question with this info. Commented Jan 18, 2013 at 14:50

2 Answers 2

3

A good general guideline is "avoid async void". One reason for this is the difference in exception handling: async Task methods will place any exceptions on their returned Task, which can be observed when that Task is awaited. async void methods will raise their exceptions directly on the SynchronizationContext that was current at the time of the start of the async void method.

Another thing to keep in mind is that async is primarily a compiler transformation. If you (or anyone else) reflects over an async Task method at runtime, you'll just see a method with a return type of Task; similarly, an async void method just has a return type of void.

So, when the test runner sees a method returning void (not knowing that it is an async void method), it executes it and sees it return without (directly) raising an exception, so it marks it as "passed". Meanwhile, the exception thrown by that async void method is raised directly on the SynchronizationContext (most test runners, including MSTest, provide a thread pool SynchronizationContext), and this exception can cause the test run to report a non-specific error - or it could possibly be ignored if the test run completes quickly enough.

Modern test runners (including MSTest as of VS2012) understand async Task methods by detecting the return type of Task, and will wait for the Task to complete before considering the test method "finished" and marking it as passed (or failed, if the returned Task contains an exception).

I have examples of this behavior in MSTest on my blog (including screenshots showing both outputs of the race condition), but note that those blog entries are almost a year old and talk about async unit testing with VS2010. MSTest was updated with VS2012 so that it has reasonable behavior for async Task methods.

Sign up to request clarification or add additional context in comments.

Comments

0

The main problem we have had with the SL/WP toolkit crashing has been caused by either Asserting or calling TestComplete after a test failed, or calling TestComplete twice. Example: Let's say you mark a test with a timeout attribute and you're waiting for a webrequest that takes longer than the timeout. If the webrequest hasn't failed but was just slow, it will still come back and call TestComplete() after the unit test framework considers it done. I don't know a way to test if the test has timed out so I can write my test to guard against that so I recommend not using it in some of these scenarios.

Another scenario is that you wait for two events, for instance a "completed" and a "failed" event and you do different things if either one of them is raised, but they both call TestComplete: Let's say you code accidentially raises both events => You just called test complete and your code failed. Third scenario is you have a shared object, let's say a WebClient instance that you don't recreate in each test: Now the first test listens for DownloadString and does it's thing. The next test also starts listening for DownloadString when it runs, but since the old one didn't unhook from it, it's DownloadStringCompleted event handler will run again, causing the unit tests to crash.

As Stephen mentions, you can't return Task in the SL and WP unit test frameworks. I've managed to hack that into the Silverlight unit test runner, which you can read about here: http://www.sharpgis.net/post/2012/12/21/Hacking-the-Silverlight-Unit-Tests-to-support-returning-Task.aspx I'd love to do that for Windows Phone Unit Tests as well, but the source code for that is not available, but please go and vote for the release of this here: http://phone.codeplex.com/workitem/10642

Also please vote for adding support for Task to the unit test framework here: http://phone.codeplex.com/workitem/10727 (WinPhone) and here: http://silverlight.codeplex.com/workitem/11457 (Silverlight)

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.