0

I have a XAML build definition running my MSTest unit tests and it takes a long time to execute all of them. I have 4 build agents(VMs with 1 core each) setup in the build environment. I am using Microsoft.TeamFoundation.Build.Activities.RunAgileTestRunner activity in the XAML which is inside a "Run on Agent"(Microsoft.TeamFoundation.Build.Workflow.Activities.AgentScope) activity.

Currently, the unit tests execute on one of the agent. I want to be able to distribute the unit tests to other available agents. Is there a way to do that through TFS configuration or by changing the build definition.

As a workaround, I can split up the unit test project into multiple projects and run them separately on the available agents however I won't be able to see the build summary in one place.

What would be best possible solution for this?

Improve this question
6
  • How long do your tests take to run? Can you split the tests into during and post build? Commented Nov 21, 2017 at 3:04
  • 22 hours(I know, I know, these are more of integration tests than unit tests but that's a different story). Can you give more information on how to split the tests into during and post build? Commented Nov 21, 2017 at 3:34
  • You should work on trimming those down/making them run faster before doing anything else. After that you should consider moving them to post deployment testing. Commented Nov 21, 2017 at 3:37
  • Yes, that is the plan. However, I have my other 3 build agents sitting idle while the first build agent runs those tests. It makes logical sense to split the test execution among the build agents to speed up the execution with minimum changes to the underlying tests. Commented Nov 21, 2017 at 4:03
  • Except that's not exactly what those agents are for, though I understand the desire to use all the "threads" available. The problem is that if you want to be able to fail/rollback based on the tests you need them to be inline with the build/deployment that they are tied to. Commented Nov 21, 2017 at 4:06

1 Answer 1

Reset to default
1

Just check this article : How to run automated tests on different machines in parallel?

The Recommended option:

  • Create multiple environments with each environment consisting of single machine (read single agent).

  • Divide all your test cases onto different sets and queue one set on one environment and other set on other environment and so on.

  • With this each set of test will run in parallel and on different machine.

Also reference below articles:

Besides, If upgrading to VNext build is an option, then you can easily achieve that by following the steps mentioned in below articles:

Improve this answer
Sign up to request clarification or add additional context in comments.

2 Comments

As I have my tests split up in multiple assemblies, I tried running each set in a ParallelForEach as suggested in one of the links above. Now, the build errors out with the message "Could not execute the tests as there is already an active test run." Looks like TFS does not support parallel test execution on the same agent, any thoughts?
I tried running MSTest test runner instead of the RunAgileTestRunner. I was able to run multiple MSTest sessions in a ParallelForEach loop. Now, however the individual tests are taking longer to execute and the overall time taken has increased instead of decreasing. Very strange! Any thoughts or help?

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.