0

I am trying to automate data loading from CSV file into Azure SQL. I have tried following this guide https://marczak.io/posts/azure-loading-csv-to-sql/ which works perfectly for smaller files but when I try use larger files I get this error: "status": 504, "message": "Execution Timeout Expired. The timeout period elapsed prior to completion of the operation or the server is not responding.\r\nclientRequestId: 347b83c3-0fd9-414f-8795-ce4843d85ebd"

Note: the csv file I am testing is the same as in the example just with extra rows.

I am only very new to azure so any help would be great

Improve this question

1 Answer 1

Reset to default
2

You can try to use Data Factory to help you lode the csv file to Azure SQL database.

Please reference this tutorial: Copy data from Azure Blob storage to a SQL database by using Azure Data Factory:

In this tutorial, you create a data factory by using the Azure Data Factory user interface (UI). The pipeline in this data factory copies data from Azure Blob storage to a SQL database. The configuration pattern in this tutorial applies to copying from a file-based data store to a relational data store.

Data Factory has the feature to Trigger the pipeline on a schedule:

In this schedule, you create a schedule trigger for the pipeline. The trigger runs the pipeline on the specified schedule, such as hourly or daily.

This feature can help you automate data loading from CSV file into Azure SQL.

Hope this helps.

Improve this answer
Sign up to request clarification or add additional context in comments.

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.