0

I have a copy activity which takes the output of a procedure and writes it to a temp CSV file. I needed to have headers in double quotation mark so after that I have a Data Flow task that takes the temp file and adds the quote all in the sink settings. Yet the output is not what is expected. It looks like the last column is missing in some of the records due to comma in the data.

Is there a way to use only copy activity but still have the column names in double quotes?

Improve this question
3
  • we need more details , can you give us more example data? Like source and output, which column missed. Commented Mar 30, 2021 at 13:11
  • I found the root cause. It was due to a comma in one of the columns but I don't know how to overcome it. I need to have everything separated by comma and in double quotation marks including the header Commented Mar 30, 2021 at 13:15
  • Hi @Joanna, if you set comma as the delimiter, data factory will consider the first row as the schema according the delimiter numbers. If your data value which contains the comma, then you will miss the column. And just for now in Data Factory, we can't solve this issue. Commented Mar 31, 2021 at 1:40

1 Answer 1

Reset to default
1

When we set the column delimiter, data factory will consider the first row as the schema according the delimiter number. If your data which has the value which same with the column delimiter, then you will miss some columns.

Just for now in Data Factory, we can't solve it. The only way is that please se the different column delimiter, for example the '|': enter image description here Output example:

enter image description here

And we also can't make the header wrapped by double quote for the output .csv file. It's not supported in Data Factory.

HTH.

Improve this answer
Sign up to request clarification or add additional context in comments.

1 Comment

I found a workaround for the header double quote. I changed the column name to contain the quotation mark and it works.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.