1

I have an Azure data factory (DF) pipeline that consists a Copy activity. The Copy activity uses HTTP connector as source to invoke a REST end-point and returns csv stream that sinks with Azure SQL Database table.

The Copy fails when CSV contains strings (such as 40f52caf-e616-4321-8ea3-12ea3cbc54e9) which are mapped to an uniqueIdentifier field in target table with error message The given value of type String from the data source cannot be converted to type uniqueidentifier of the specified target column.

I have tried to wrapped the source string with {} such as {40f52caf-e616-4321-8ea3-12ea3cbc54e9} with no success.

The Copy activity will work if I modified the target table field from uniqueIdentifier to nvarchar(100).

0

2 Answers 2

3

I reproduce your issue on my side.

enter image description here

The reason is data types of source and sink are dismatch.You could check the Data type mapping for SQL server.

enter image description here

Your source data type is string which is mapped to nvarchar or varchar, and uniqueidentifier in sql database needs GUID type in azure data factory.

So,please configure sql server stored procedure in your sql server sink as a workaround.

Please follow the steps from this doc:

Step 1: Configure your Sink dataset:

enter image description here

Step 2: Configure Sink section in copy activity as follows:

enter image description here

Step 3: In your database, define the table type with the same name as sqlWriterTableType. Notice that the schema of the table type should be same as the schema returned by your input data.

    CREATE TYPE [dbo].[CsvType] AS TABLE(
    [ID] [varchar](256) NOT NULL
)

Step 4: In your database, define the stored procedure with the same name as SqlWriterStoredProcedureName. It handles input data from your specified source, and merge into the output table. Notice that the parameter name of the stored procedure should be the same as the "tableName" defined in dataset.

Create PROCEDURE convertCsv @ctest [dbo].[CsvType] READONLY
AS
BEGIN
  MERGE [dbo].[adf] AS target
  USING @ctest AS source
  ON (1=1)
  WHEN NOT MATCHED THEN
      INSERT (id)
      VALUES (convert(uniqueidentifier,source.ID));
END

Output:

enter image description here

Hope it helps you.Any concern,please free feel to let me know.

Sign up to request clarification or add additional context in comments.

Comments

1

There is a way to fix guid conversion into uniqueidentifier SQL column type properly via JSON configuration. Edit the Copy Activity via Code {} button in top right toolbar.

Put:

    "translator": {
        "type": "TabularTranslator",
        "typeConversion": true
    }

into typeProperties block of the Copy activity. This will also work if Mapping schema is unspecified / dynamic.

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.