1

I'm calling a REST API endpoint via Azure Data Factory using copy activity. The API returns 404 - not found for valid queries where no data is found. Whether this is the appropriate response or not (I have no control over the API), is there a way to access the API status code (i.e. not Azure error code?) in ADF?

I want the pipeline to return 'success' when a 404 not found is returned, but still fail when other errors occur. I have set up an 'on failure' error handling activity, but I'm getting stuck on how to distinguish between 'valid' errors & 404s due to no data.

Activity output

The error message string contains the status code (& response), which is what I'm after. Ideally I'd like to compare against these without doing a string comparison, especially seeing as the message appears to be constructed using the exact response fields I'm after.

The 404 only occurs for some query parameters & the response is the same via Postman, so I'm relatively confident it is an intended response from the API.

Based on Azure data factory - getting the HTTP Status of a web activity output I'm not sure if this can be done exactly, but thought I'd (a) check & (b) see if anyone can suggest a better approach.

Improve this question

1 Answer 1

Reset to default
0

As your error message gives needed information, check whether the error message is 404 error or not. If it is 404, succeed the pipeline otherwise fail the pipeline.

To do this, use Try Catch block for the ADF activities.

Upon the failure of the copy activity, create a string type set variable activity and use below expression for that.

@activity('Copy data1').output.errors[0].message

enter image description here

The error message of the copy activity will be stored in this variable. Use an if activity and check whether status code 404 NotFound exists in this or not using below expression.

@contains(variables('msg'),'status code 404 NotFound')

enter image description here

If its true, the pipeline run should succeed. So, add one set variable activity in the True activities of if activity for clear pipeline run or leave True activities as empty.

enter image description here

If it is not 404 error, then we need to fail the pipeline run. So, use a Fail activity inside the False activities of if activity and give the error message and error code like below.

@activity('Copy data1').output.errors[0].Code

enter image description here

If the copy activity fails with other errors, the Fail activity will fail the pipeline run with copy activity error code and error message.

If copy activity fails with 404 error, then pipeline run will succeed.

enter image description here

In case, if you also want to handle the success of the copy activity, check this Documentation to know about the error handling approaches.

Improve this answer
Sign up to request clarification or add additional context in comments.

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.