Accessing pipeline activity status from an Azure Function

Question

I have an Azure Function which triggers a Pipeline and I'm able to poll the pipeline status to check when it completes using: Pipeline.Properties.RuntimeInfo.PipelineState

My pipeline uses several parallel Copy activities and I'd like to be able to access the status of these activities incase they fail. The Azure documentation describes how to access the pipeline activities but you can only get at static properties like name and description but not dynamic properties like Status (like you can for the Pipeline via its RuntimeInfo property).

For completeness, I've accessed the activity list using:

IList<Microsoft.Azure.Management.DataFactories.Models.Activity> activityList = plHandle.Pipeline.Properties.Activities;

Is it possible to check individual activity statuses programmatically?

Paul Andrew · Accepted Answer · 2017-04-20 10:22:48Z

1

Its certainly possible.

I use the ADF PowerShell cmdlets in the Azure module to monitor our data factories.

Maybe do something like the below for what you need with Get-AzureRmDataFactoryActivityWindow command.

Eg:

$ActivityWindows = Get-AzureRmDataFactoryActivityWindow `
    -DataFactoryName $ADFName.DataFactoryName `
    -ResourceGroupName $ResourceGroup `
    | ? {$_.WindowStart -ge $Now} `
    | SELECT ActivityName, ActivityType, WindowState, RunStart, InputDatasets, OutputDatasets `
    | Sort-Object ActivityName

This gives you the activity level details including the status. Being:

Ready
In Progress
Waiting
Failed

... I list them because they differ slightly from what you see in the portal blades.

The datasets are also arrays if you have multiple inputs and outputs for particular activities.

More ADF cmdlets available here: https://learn.microsoft.com/en-gb/powershell/module/azurerm.datafactories/?view=azurermps-3.8.0

Hope this helps

answered Apr 20, 2017 at 10:22

Paul Andrew

3,2512 gold badges21 silver badges37 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Steven Enticknap Over a year ago

Hi, thanks for replying. Using your Powershell example, I have been able to find the following link to the corresponding REST API call: [learn.microsoft.com/en-us/rest/api/datafactory/… as I am trying to achieve this using C#. I post another reply if I get it to work. Thanks.

Steven Enticknap · Accepted Answer · 2017-04-24 12:19:13Z

I've managed to resolve this by accessing the DataSliceRuns (i.e. activities) for the pipeline as follows:

var datasets = client.Datasets.ListAsync(<resourceGroupName>, <DataFactoryName>).Result;

    foreach (var dataset in datasets.Datasets)
    {
        // Check the activity statuses for the pipelines activities.
        var datasliceRunlistResponse = client.DataSliceRuns.List(<resourceGroupName>, <dataFactoryName>,<DataSetName>, new DataSliceRunListParameters()
                                                                        {
                                                                            DataSliceStartTime = PipelineStartTime.ConvertToISO8601DateTimeString()
                                                                        });

        foreach (DataSliceRun run in datasliceRunlistResponse.DataSliceRuns)
        {
            // Do stuff...
        }
    }

Collectives™ on Stack Overflow

Accessing pipeline activity status from an Azure Function

2 Answers 2

1 Comment

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

1 Comment

Comments

Your Answer

Sign up or log in

Post as a guest

Related