JSon Parsing in ADF web activitiy

Rather than using a data flow and spinning up an entire spark cluster just to make an API Call, it's probably better to explicitly set the body of the next API using the output value from previous API call directly.

For example:

if your response from the first API call is:

{ "pid": "0", "name": " test_onbirded1", "description": " ON_BOARDED_TRAIT111 ", "traitType": " ON_BOARDED_TRAIT", "dataSourceId": " 122233", "folderId": " 982331", "traitRule": "(lifecyclemainfrequent == 2)", "ADFWebActivityResponseHeaders": { "x-ms-request-id": "b583f633-501e-0089-1270-7961c2000000", "x-ms-version": "2020-08-04", "x-ms-creation-time": "Tue, 13 Jul 2021 13:18:52 GMT", "x-ms-lease-status": "unlocked", "x-ms-lease-state": "available", "x-ms-blob-type": "BlockBlob", "x-ms-server-encrypted": "true", "Accept-Ranges": "bytes", "Date": "Thu, 15 Jul 2021 11:58:36 GMT", "ETag": ""0x8D94787B0F9ED99"", "Server": "Windows-Azure-Blob/1.0;Microsoft-HTTPAPI/2.0", "Content-Length": "208", "Content-Type": "application/octet-stream", "Last-Modified": "Thu, 15 Jul 2021 11:57:16 GMT" }, "effectiveIntegrationRuntime": "DefaultIntegrationRuntime (North Europe)", "executionDuration": 0, "durationInQueue": { "integrationRuntimeQueue": 0 }, "billingReference": { "activityType": "ExternalActivity", "billableDuration": [ { "meterType": "AzureIR", "duration": 0.016666666666666666, "unit": "Hours" } ] } }

then the body of your next API call should be like so:

{ 
    "pid": "@{activity('First API Call').output.pid}",
    "name": "@{activity('First API Call').output.name}",
    "description": "@{activity('First API Call').output.description}",
    "traitType": "@{activity('First API Call').output.traitType}",
    "dataSourceId": "@{activity('First API Call').output.dataSourceId}", 
    "folderId": " @{activity('First API Call').output.folderId}", 
    "traitRule": "@{activity('First API Call').output.traitRule}"
}

As you can see it is using the output of the previous activity directly, without the cost of spinning up a spark cluster.

Use Select as an interim activitym and then use post, if that is possible. https://learn.microsoft.com/en-us/azure/data-factory/data-flow-select

You cannot achieve this in Data Flow.

So, in Copy Data activity, use the column mapping to generate a new file (in sink).

Used this new file as source in the Data Flow.

Step1: Mapping:

Step2: While Mapping delete unnecessary properties as shown in below image:

Step3: Input

Step4: Output