Create Pipeline via Azure Powershell and pipelineReturnValue activity is not properly created

Question

I try to run this command in Azure Powershell

Set-AzDataFactoryV2Pipeline -Force -ResourceGroupName MY_DEV_RESOURCE_GRP -DataFactoryName MYADF001 -Name PL_PROCESS_API -DefinitionFile ./PL_PROCESS_API.json

Inside a PL_PROCESS_API.json file is containing this content

...{
            "name": "Activity_Return",
            "type": "SetVariable",
            "dependsOn": [
                {
                    "activity": "Process API data",
                    "dependencyConditions": [
                        "Succeeded"
                    ]
                }
            ],
            "policy": {
                "secureOutput": false,
                "secureInput": false
            },
            "userProperties": [],
            "typeProperties": {
                "variableName": "pipelineReturnValue",
                "value": [
                    {
                        "key": "strCommaSeparatedString",
                        "value": {
                            "type": "Expression",
                            "content": "@activity('Process API data').output.Holiday"
                        }
                    }
                ],
                "setSystemVariable": true
            }
        }...

When I run this command the sections of pipelineReturnValue in type properties in ADF when the command run successfully will be like this

"typeProperties": {
                "variableName": "pipelineReturnValue",
                "value": "[{\"key\":\"strCommaSeparatedString\",\"value\":{\"type\":\"Expression\",\"content\":\"@length(activity('Process API data').output.Holiday)\"}}]"
            }

I would like to know how to fix this?

No error raise up, the exepected output might be the same as the JSON content in Json file for that pipeline. — Surat Suntong
– Surat Suntong, Commented Dec 16, 2024 at 3:52
As you see in the section of typeProperties.value it have to be json array, but it appear in ADF as JSON String. — Surat Suntong
– Surat Suntong, Commented Dec 16, 2024 at 3:58
I have posted the answer, please check if it solves your query or not. — Rakesh Govindula
– Rakesh Govindula, Commented Dec 16, 2024 at 6:44

Rakesh Govindula · Accepted Answer · 2024-12-16 06:43:44Z

I have tried the same PowerShell script with below JSON which is similar as your pipeline from my end.

{
    "name": "test2",
    "properties": {
        "activities": [
            {
                "name": "Activity_Return",
                "type": "SetVariable",
                "dependsOn": [
                    {
                        "activity": "Process API data",
                        "dependencyConditions": [
                            "Succeeded"
                        ]
                    }
                ],
                "policy": {
                    "secureOutput": false,
                    "secureInput": false
                },
                "userProperties": [],
                "typeProperties": {
                    "variableName": "pipelineReturnValue",
                    "value": [
                        {
                            "key": "strCommaSeparatedString",
                            "value": {
                                "type": "Expression",
                                "content": "@activity('Process API data').output.data"
                            }
                        }
                    ],
                    "setSystemVariable": true
                }
            },
            {
                "name": "Process API data",
                "type": "WebActivity",
                "dependsOn": [],
                "policy": {
                    "timeout": "0.12:00:00",
                    "retry": 0,
                    "retryIntervalInSeconds": 30,
                    "secureOutput": false,
                    "secureInput": false
                },
                "userProperties": [],
                "typeProperties": {
                    "method": "GET",
                    "url": "https://reqres.in/api/users?page=2"
                }
            }
        ],
        "variables": {
            "var1": {
                "type": "String"
            }
        },
        "annotations": []
    },
    "type": "Microsoft.DataFactory/factories/pipelines"
}

The pipeline got created with correct expression as shown below.

enter image description here

I have tried by changing some code in the pipeline JSON and the only time I got the similar JSON string for the return variable is when I had set the "setSystemVariable" property to false in the typeProperties of the Activity_Return activity.

This is the activity JSON for your reference:

{
                "name": "Activity_Return",
                "type": "SetVariable",
                "dependsOn": [
                    {
                        "activity": "Process API data",
                        "dependencyConditions": [
                            "Succeeded"
                        ]
                    }
                ],
                "policy": {
                    "secureOutput": false,
                    "secureInput": false
                },
                "userProperties": [],
                "typeProperties": {
                    "variableName": "pipelineReturnValue",
                    "value": [
                        {
                            "key": "strCommaSeparatedString",
                            "value": {
                                "type": "Expression",
                                "content": "@activity('Process API data').output.data"
                            }
                        }
                    ],
                    "setSystemVariable": false 
                }
            }

Similar Result:

enter image description here

When the setSystemVariable property is false, the pipeline variable will not be considered as a pipeline return variable and the expression for that will be considered as a string. So, make sure the setSystemVariable property is set to true in the pipeline JSON.

Actually my setSystemVariable is already set to true but it still occur with this problem, I'm not sure that is because the version of Azure Powershell or not, it is 3.0.0, what's version had you tried?
My Azure Powershell version is 12.4.0. i.imgur.com/E8oIwKk.png. You can check by updating yours to latest. follow this doc.
I tried to use Azure Cloud Shell that have version of Az.DataFactory 1.18.8, it working properly so I will try to upgrade this module.

Collectives™ on Stack Overflow

Create Pipeline via Azure Powershell and pipelineReturnValue activity is not properly created

1 Answer 1

3 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

3 Comments

Your Answer

Sign up or log in

Post as a guest

Related