Leapwork Azure DevOps PowerShell based Integration

Leapwork Azure DevOps PowerShell based Integration

Automatic creation of bugs in Azure DevOps Server

An important part of running automated test cases is to monitor, inspect, and react to the results. This is best accomplished in the test management and bug tracking system that we use for our daily work.

In this article, we will review three different scenarios on how to trigger and create/update bugs in the Azure DevOps Server, formally known as Visual Studio Team Services (VSTS) or Team Foundation Server (TFS), based on the results of the test cases running in Leapwork.

Trigger schedules from API and create bugs in Azure DevOps Server

Integrating Leapwork with any third-party system through the REST API can be done in a few simple steps. This enables us to easily set up the automatic creation or updating of bugs in Azure DevOps Server when test automation cases fail in Leapwork.

As part of the integration, we have a Windows Powershell script that runs a pre-defined Leapwork run list, polls for the results until they are available, and then loops through all failed cases and creates bugs in JIRA as appropriate using API's.

#=========================================================

#Add your Information

#Leapwork Information
#Leapwork API Controller url
$leapworkdomain = <controller machine :port defined>

#Leapwork API Access Key
$leapworkAccessKey = "Access Key"
#Leawprok Schedule ID
$leapworkId = "Schedule ID"

#AzureDevOPs Information

#Personal Access Token
$pat = "Your PAT Token"
#AzureDomain Name
$domain = <Azure instance>
#Azure Project Name
$tfsPath = "Project Name"

#=========================================================

function RunScheduleAndGetResults($schedule)
{
    Write-Host "Getting id for schedule '$schedule'."

# Get the id of the schedule.
    $runScheduleId = "";
    $headers = @{}
    $headers.Add("AccessKey",$leapworkAccessKey)
    $runSchedules = Invoke-WebRequest -ContentType "application/json" -Headers $headers "$leapworkdomain/api/v4/schedules" | ConvertFrom-Json
    foreach($runScheduleItem in $runSchedules)
    {
        if ($runScheduleItem.id -eq $schedule) { $runScheduleId = $runScheduleItem.id }
    }
    if ($runScheduleId -eq "") { throw "Could not find schedule '$schedule'." }

Write-Host "Running the schedule."

# Run the schedule now.
    $timestamp = [DateTime]::UtcNow.ToString("ddMMyyyy HHmmss")
    Start-Sleep 1
    $runNow = Invoke-WebRequest -Method PUT -ContentType "application/json" -Headers $headers "$leapworkdomain/api/v4/schedules/$runScheduleId/runNow"
    if ($runNow.StatusCode -ne 200) { throw "Could not run schedule." }
    $runNowResponse=$runNow.Content | ConvertFrom-Json
    $runId=$runNowResponse.RunId
    # Get the result, keep polling every 5 seconds until a new result is returned.
    do
    {
        Start-Sleep -Seconds 5

Write-Host "Polling for run results."

$runResult = Invoke-WebRequest -ContentType "application/json" -Headers $headers "$leapworkdomain/api/v4/run/$runId" | ConvertFrom-Json

}
    while ($runResult.Status -ne 'Finished')

Write-Host "Results received."

return $runResult
}

function CreateOrUpdateBug{
    param([parameter(mandatory)] [string]$azureBugTitle,
          [parameter(mandatory)] [string]$azureBugDesc)

# Create an authorization header using a Personal Access Token.
    $patEncoded = [Convert]::ToBase64String([Text.Encoding]::ASCII.GetBytes(":$($pat)"));
    $header = @{"Authorization" = "Basic $patEncoded"}

# See if a bug with the same title already exists.
    $queryAlreadyExists = '{ "query": "SELECT [System.Id], [System.Title], [System.State] FROM WorkItems ' +
    'WHERE [System.WorkItemType] = ''Bug'' AND [System.Title] = ''' + $azureBugtitle + ''' AND [System.State] <> ''Removed'' ' +
    'ORDER BY [System.Id] DESC" }'

$urlAlreadyExists = "$domain/$tfsPath/_apis/wit/wiql?api-version=6.0"
    $header['Content-Type'] = 'application/json'
    $resultAlreadyExists = Invoke-RestMethod -Headers $header -Method Post -Body $queryAlreadyExists $urlAlreadyExists

# If the bug already exists, update it.
    # MHN Force creation of new bug

if ($resultAlreadyExists.workItems.Count -gt 0)
    {
        Write-Output "Updating test case in Azure: " $azureBugtitle

# Add a note that this bug was updated or re-opened.
        $timestamp = [DateTime]::UtcNow.ToString("dd-MM-yyyy HH:mm:ss")
        $azureBugDesc = "<strong>Updated or re-opened by Leapwork on $timestamp.</strong><br><br>" + $azureBugDesc

# Update bug.
        $queryUpdateBug = '[ { "op" : "replace", "path" : "/fields/System.Title", "value" : "' + $azureBugtitle + '" }, ' +\
        '{ "op" : "add", "path" : "/fields/Microsoft.VSTS.TCM.ReproSteps", "value" : "' + $azureBugDesc + '" }, ' +\
        '{ "op" : "replace", "path" : "/fields/System.State", "value" : "New" } ]'

$urlUpdateBug = "$domain/_apis/wit/workitems/$($resultAlreadyExists.workItems[0].id)?api-version=1.0"
        $header['Content-Type'] = 'application/json-patch+json'

$resultUpdateBug = Invoke-RestMethod "$urlUpdateBug" -Headers $header -Method Patch -Body $queryUpdateBug
        $workitemId = $resultUpdateBug.id
        Write-Output "Workitem Id Updated : $workitemId"
    }
    else
    {
        Write-Output "Created new test case in Azure: " $azureBugtitle

$queryCreateNewBug = '[\
        { "op" : "add", "path" : "/fields/System.Title", "value" : "' + $azureBugtitle + '" }, ' +\
        '{ "op" : "add", "path" : "/fields/Microsoft.VSTS.TCM.ReproSteps", "value" : "' + $azureBugDesc + '" }, ' +\
        '{ "op" : "add", "path" : "/fields/System.State", "value" : "New" } ]'

$urlCreateNewBug = "$domain/$tfsPath/_apis/wit/workitems/`$Bug?api-version=6.0"
        $header['Content-Type'] = 'application/json-patch+json'
        $resultCreateNewBug = Invoke-RestMethod "$urlCreateNewBug" -Headers $header -Method Patch -Body $queryCreateNewBug -Verbose
        $workitemId = $resultCreateNewBug
        Write-Output "Workitem Id Created : $workitemId"
    }
}

# Run the Leapwork schedule "My Test Schedule" and get the results.
$runResult = RunScheduleAndGetResults($leapworkId)

# If there are any failed cases in the results, iterate through them.
if ($runResult.Failed -gt 0)
{
    Write-Host "Found $($runResult.Failed) failed case(s)."
    $headers = @{}
    $headers.Add("AccessKey",$leapworkAccessKey)
    $runId=$runResult.RunId
    $runItemIds = Invoke-WebRequest -ContentType "application/json" -Headers $headers $leapworkdomain/api/v4/run/$runId/runItemIds | ConvertFrom-Json

$rootPath =$runResult.RunFolderPath
    foreach ($runItemId in $runItemIds.RunItemIds)
    { 
        $bugTitle = ""
        $bugDescription = ""

$runItems = Invoke-WebRequest -ContentType "application/json" -Headers $headers $leapworkdomain/api/v4/runItems/$runItemId | ConvertFrom-Json

if($runItems.FlowInfo.Status -eq 'Failed')
        {
            
            # Create a title for the bug.
            $bugTitle = "Leapwork: " + $runItems.FlowInfo.FlowTitle + " (" + $runItems.AgentInfo.AgentTitle + ")"

# Create a description that contains the log messages.
            $newline = "<div>"
            $bugDescription = "Log from Leapwork: $newline $newline"
            Write-Output "$runItemId"

$keyFrames = Invoke-WebRequest -ContentType "application/json" -Headers $headers "$leapworkdomain/api/v4/runItems/$runItemId/keyframes/1" | ConvertFrom-Json

foreach ($keyframe in $keyFrames)
            {
                if ($key.Level -ne 'trace' -or $key.Level -ne '')
                {
                    $keyframeTimestamp = get-date($keyframe.Timestamp.LocalDateTime) -Format "dd-MM-yyyy HH:mm:ss"
                    $bugDescription += "$keyframeTimestamp - $($keyframe.LogMessage) $newline"
                }
            }
            # Add path to video and screenshots.
            $videoPath = Join-Path -Path $rootPath "$($item.Id)\$($item.Id).avi"
            $videoPath = $videoPath.Replace('\', '\\')
            $screenshotsPath = Join-Path -Path $rootPath "$($item.Id)\"
            $screenshotsPath = $screenshotsPath.Replace('\', '\\')
            $bugDescription += "<br>Video: $videoPath $newline"
            $bugDescription += "<br>Screenshots (if any): $screenshotsPath $newline "
            $bugDescription += "<br><a href='LEAPWORK:/flow/$($item.flowId)' target='_blank'>Open Flow in Studio</a>"

# Create or update bug in TFS.

CreateOrUpdateBug -azureBugTitle $bugTitle -azureBugDesc $bugDescription # Create of Update the bug found

}
    }
}
else
{
    Write-Host "No failed cases found."
}

The following changes are required above script:-

  #Run the Leapwork schedule "SmokeRunSchedule" and get the results.
  $runResult = RunScheduleAndGetResults("SmokeRunSchedule")

# Create an authorization header using a Personal Access Token.
$pat = "YOUR PERSONAL ACCESS TOKEN HERE"

$urlAlreadyExists = "https://YOURINSTANCE.visualstudio.com/DefaultCollection/YOURPROJECT/_apis/wit/wiql?api-version=1.0"

After running the PowerShell script, the schedule will be executed and for failed flows, a bug will be logged in Azure DevOps.

Note: The script contains no error handling or logging mechanisms. It is meant to only demonstrate the core functionality of integrating Leapwork with Azure DevOps Server.

After running, cases are created in Azure DevOps Server and can be managed, for instance, through Visual Studio:

The previous script uses PATs to authorize access to Azure DevOps Server.

If we have the Controller installed on our computer, explore the previously mentioned endpoints by going to the following URL: http://localhost:9001/help/index.

Read the full documentation on the Leapwork REST API.

If you have any questions, contact priority support on prioritysupport@leapwork.com.