Passing properties to U-SQL activity through Azure Data Factory Pipeline through parameters?

Question

I want to pass the slice start/end time properties of a pipeline to a U-SQL activity through the "parameters" property of the activity/pipeline.

The purpose is to generate dynamic file names based on the date the slice starts and orchestrate the process to run daily and process a file of that date.

Is that doable?

Community · Accepted Answer · 2016-07-05 11:54:38Z

2

That should be possible. Here is an example of how I am using the slice date to dynamically define folder structure.

"typeProperties": {
                "scriptPath": "script.usql",
                "scriptLinkedService": "LinkedService_AS_Storage",
                "degreeOfParallelism": 3,
                "priority": 100,
                "parameters": {
                    "in": "$$Text.Format('/RawData/{0:yyyy}/{0:MM}/{0:dd}/In.csv',SliceStart)",
                    "out": "$$Text.Format('/TempData/{0:yyyy}/{0:MM}/{0:dd}/Out.csv',SliceStart)"
                }
            }

Inside the USQL script you would just reference the @in or @out parameters. Be careful, U-SQL has case-sensitive syntax. They are actually added to the top of the script file when azure data factory gets it from storage to run.

Hope this helps.

edited Jul 5, 2016 at 11:54

CommunityBot

11 silver badge

answered May 17, 2016 at 13:31

JustLogic

1,7382 gold badges12 silver badges24 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Tayyab Anwar Over a year ago

Yes, this is exactly how it works. Thank you for your help!

Collectives™ on Stack Overflow

Passing properties to U-SQL activity through Azure Data Factory Pipeline through parameters?

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related