1

Environment:

MS Azure: Blob Container, multiple csv files saved in a folder. This is my source. Azure Sql Database. This is my target

Goal: Use Azure Data Factory and build a pipeline to "copy" all files from the container and store them in their respective tables in the Azure Sql database by automatically creating those tables.

How do I do that? I tried following this but I just end up having tables incorrectly created in the database, where table is created with a single column having same name as the table name.

I believe I followed the instructions from that link pretty must as they are.

Improve this question
1
  • You said that the table name is included in the column? Commented Jun 24, 2021 at 2:06

1 Answer 1

Reset to default
2

My CSV file is as follows, one column contains the table name.
enter image description here

The previous steps will not be repeated,it is the same as the link.

  1. At Step3 inside the Foreach activity, we should add a Lookup activity to query the table name from the source dataset.
    enter image description here

  2. We can declare a String type variable tableName pervious, then set the value via expression @activity('Lookup1').output.firstRow.tableName. enter image description here

  3. At sink setting of the Copy activity, we can key in @variables('tableName'). enter image description here

  4. ADF will auto create the table for us. enter image description here

  5. The debug result is as follows:
    enter image description here

Improve this answer
Sign up to request clarification or add additional context in comments.

3 Comments

I feel really stupid. Your original solution in the other post works perfectly. Turns out my source files had table name in the first row, which is why I was getting a single column similar to the table name. I apologize. I should delete this post. But I truly thank you for your response and your solution in the original post. I want to reach out directly but I couldn't find a way and ended up creating a new thread. Again thank you and also apologies!!!
Good to hear that you got it working. Under the assumptions in my answer, it works well, hope you can accept(mark) it as answer. This can be beneficial to other community members. Thank you! :)
I will do that. Thank you :)

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.