Run Azure DevOps test case with REST APIs

Question

I have a requirement to run automated API test (associated to a test case) in Azure Devops.

I can run it manually by selecting - build and release.

Since I'm running the testcases from Azure Devops yaml pipeline on a Linux agent, I cannot use VSTEST@2 task to input - TestPlan , TestSuite id's.

The only option I found is to run via, APIs - and call the API via - bash task.

Here is the script

param (
    [string]$token="",
    [string]$collection="",
    [string]$projectName ="",
    [int] $planIdStatic = ,
    [int] $suiteIdStatic = ,
    [int] $testcaseID = 


)

$base64AuthInfo = [Convert]::ToBase64String([Text.Encoding]::ASCII.GetBytes(("{0}:{1}" -f $user, $token)))
$response = Invoke-RestMethod "https://dev.azure.com/$collection/$projectName/_apis/test/plans?api-version=5.0" -Method 'GET' -Headers @{Authorization = ("Basic {0}" -f $base64AuthInfo)}
foreach( $val in $response.value)
{

#PLAN ID 
Write-Host $val.id
 # $planId = [convert]::ToInt32($val.id)
 [int] $planId = $planIdStatic
  $suites = "https://dev.azure.com/$collection/$projectName/_apis/test/plans/$planId/suites?api-version=5.0"
 $listofSuites = Invoke-RestMethod $suites -Method 'GET' -Headers @{Authorization = ("Basic {0}" -f $base64AuthInfo)}
 #define Suite ID

 [int] $suiteId = $suiteIdStatic
 $suitename = "https://dev.azure.com/$collection/$projectName/_apis/test/plans/$planId/suites/$suiteId/points?api-version=5.0"
 $listofSuites = Invoke-RestMethod $suitename -Method 'GET' -Headers @{Authorization = ("Basic {0}" -f $base64AuthInfo)}


 #Define TestCaseID

 [int] $tcID = $testcaseID

 $tc = "https://dev.azure.com/$collection/$projectName/_apis/test/plans/$planId/suites/$suiteId/points?testCaseId=$tcID&api-version=5.0"
 $testcaseapi = Invoke-RestMethod $tc -Method 'GET' -Headers @{Authorization = ("Basic {0}" -f $base64AuthInfo)}
 
 #Define PointID

[int] $pointID =    $testcaseapi.value[0].id 
 $runName = "Test RUN Setup"

 #PayLoad

 $payload = @"
 
 {
   "name":"$runName",
   "plan":{
      "id":"$planId"
   },
   "pointIds":[
      "$pointID"
   ]
}
"@;

#Initiate the RUN

$tcRun = "https://dev.azure.com/$collection/$projectName/_apis/test/runs?api-version=5.0"
 $testRun = Invoke-RestMethod $tcRun -Method 'POST'  -ContentType "application/json"  -Body $payload -Headers @{Authorization = ("Basic {0}" -f $base64AuthInfo)} 
Write-Host "Test Run Status is ...." 
}

From the blog - https://ghoshasish99.medium.com/integrate-your-test-automation-framework-with-azure-test-plan-a3230ab0f1da

Unfortunately it creates the test run - but it wont run any testcase it just shows as in-progress forever, and that's all - (Note - Test Plan is associated with Build and Release)

I suspect the script needs payload- body with build and release pipeline & release stage (Assume my release has DEV - QA and PROD) stages.

Kevin Lu · Accepted Answer · 2021-05-05 07:42:02Z

In addition to using Rest API to run test run, you also need to use Release API to run release and need to update the running release information to test run.

Here are the following steps:

Step1: You could define a variable in Release Pipeline.

Step2: Add this variable in VSTest Task:

Note: The test run id needs to correspond to the release one-to-one, and then the status of the test run will be updated.

Here is an example:

param (
    [string]$token="",
    [string]$collection="",
    [string]$projectName ="",
    [int] $planIdStatic =947 ,
    [int] $suiteIdStatic =1086 ,
    [int] $testcaseID =79,
    [int] $releasedefinitionid =96,
    [int] $buildid =62903


)

$base64AuthInfo = [Convert]::ToBase64String([Text.Encoding]::ASCII.GetBytes(("{0}:{1}" -f $user, $token)))
$response = Invoke-RestMethod "https://dev.azure.com/$collection/$projectName/_apis/test/plans?api-version=5.0" -Method 'GET' -Headers @{Authorization = ("Basic {0}" -f $base64AuthInfo)}
foreach( $val in $response.value)
{

#PLAN ID 
Write-Host $val.id
 # $planId = [convert]::ToInt32($val.id)
 [int] $planId = $planIdStatic
  $suites = "https://dev.azure.com/$collection/$projectName/_apis/test/plans/$planId/suites?api-version=5.0"
 $listofSuites = Invoke-RestMethod $suites -Method 'GET' -Headers @{Authorization = ("Basic {0}" -f $base64AuthInfo)}
 #define Suite ID

 [int] $suiteId = $suiteIdStatic
 $suitename = "https://dev.azure.com/$collection/$projectName/_apis/test/plans/$planId/suites/$suiteId/points?api-version=5.0"
 $listofSuites = Invoke-RestMethod $suitename -Method 'GET' -Headers @{Authorization = ("Basic {0}" -f $base64AuthInfo)}


 #Define TestCaseID

 [int] $tcID = $testcaseID

 $tc = "https://dev.azure.com/$collection/$projectName/_apis/test/plans/$planId/suites/$suiteId/points?testCaseId=$tcID&api-version=5.0"
 $testcaseapi = Invoke-RestMethod $tc -Method 'GET' -Headers @{Authorization = ("Basic {0}" -f $base64AuthInfo)}
 
 #Define PointID

[int] $pointID =    $testcaseapi.value[0].id 
 $runName = "Test RUN Setup"

 #PayLoad

 $payload = @"
 
 {
   "name":"test",
    "automated":true,
    "pointIds":[$pointID],
    "state":"NotStarted",
    "dtlTestEnvironment":{"id":"vstfs://dummy"},
    "plan":{"id":"$planId"},
    "filter":{"sourceFilter":"*.dll","testCaseFilter":""}
}
"@;

#Initiate the RUN

$tcRun = "https://dev.azure.com/$collection/$projectName/_apis/test/runs?api-version=5.0"
 $testRun = Invoke-RestMethod $tcRun -Method 'POST'  -ContentType "application/json"  -Body $payload -Headers @{Authorization = ("Basic {0}" -f $base64AuthInfo)} 
Write-Host "Test Run Status is ...."

$testrunid = $testRun.id

echo $testrunid


$url = "https://vsrm.dev.azure.com/$collection/$projectName/_apis/Release/definitions/$($releasedefinitionid)?api-version=5.0-preview.3"
Write-Host "URL: $url"
$pipeline = Invoke-RestMethod -Uri $url -Method Get  -Headers @{Authorization = ("Basic {0}" -f $base64AuthInfo)} 

Write-Host "Pipeline = $($pipeline | ConvertTo-Json -Depth 100)"


$pipeline.variables.testrunid.value = "$testrunid"

####****************** update the modified object **************************
$json = @($pipeline) | ConvertTo-Json -Depth 99


$updatedef = Invoke-RestMethod -Uri $url -Method Put -Body $json -ContentType "application/json" -Headers @{Authorization = ("Basic {0}" -f $base64AuthInfo)} 



$releaseRunurl ="https://vsrm.dev.azure.com/$collection/$projectName/_apis/Release/releases?api-version=6.1-preview.8"

 $releasebody = @"
 
 {
     "definitionId": $releasedefinitionid

}
"@;

$ReleaseRun = Invoke-RestMethod $releaseRunurl -Method 'POST'  -ContentType "application/json"  -Body $releasebody -Headers @{Authorization = ("Basic {0}" -f $base64AuthInfo)} 

$Releaseid=$ReleaseRun.id

echo $Releaseid

$ReleaseEnvID=$ReleaseRun.environments.id

echo $ReleaseEnvID

$updateTestrun="https://dev.azure.com/$collection/$projectName/_apis/test/Runs/$($testrunid)?api-version=6.1-preview.3"

$updatebody = @"
 
 {
    "build":
    {
        "id":"$buildid"
    },
    "releaseEnvironmentUri":"vstfs:///ReleaseManagement/Environment/$ReleaseEnvID","releaseUri":"vstfs:///ReleaseManagement/Release/$Releaseid"

}
"@;


$UpdateRun = Invoke-RestMethod $updateTestrun -Method 'PATCH'  -ContentType "application/json"  -Body $updatebody -Headers @{Authorization = ("Basic {0}" -f $base64AuthInfo)} 


 

}

Result:

When the release is completed, the status of the test run will be updated

Thanks@KevinLu, Truly appreciate. But is there a way that I run release pipline without VStest and update the result? My release pipline has dotnet test CMD line which runs on selfhosted linux agent running in azure container instances. And also the plan is to run the Ps script as part of YAML build pipeline.(eg - yaml build -> deploy -> invoke PS script to run the tests which in-turn triggers release pipeline to update the test results) Since VSTest Task doesnt support on an Linux agent, can we use dotnet core cmd line to update the testrunID?
Hi @user2153844. I could understand your reuqirement. But I am afriad that dotnet core test command doesn't support this feature. Please refer to this doc: learn.microsoft.com/en-us/azure/devops/test/… You need to use vstest task to do this.
Thanks @KevinLu, with little workaround able to make it work.

Collectives™ on Stack Overflow

Run Azure DevOps test case with REST APIs

1 Answer 1

3 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

3 Comments

Your Answer

Sign up or log in

Post as a guest

Related