How to implement Loops in U-SQL

Question

Is is possible to implement Loops (while/for) in U-SQL without using C#. If no, can anyone share the c# syntax to implement loops in u-sql.

I am extracting files from a particular date to a date, but right now I am extracting this by writing file path manually.

DROP VIEW IF EXISTS dbo.ReadingConsolidated;  
CREATE VIEW IF NOT EXISTS dbo.ReadingConsolidated 
AS

EXTRACT     
        ControllerID int?,          
        sensorID int?,
        MeasureDate DateTime,
        Value float

FROM  
"adl://datalake.azuredatalakestore.net/2015/7/1/Reading.csv",
"adl://datalake.azuredatalakestore.net/2015/7/2/Reading.csv",
"adl://datalake.azuredatalakestore.net/2015/7/3/Reading.csv",
"adl://datalake.azuredatalakestore.net/2015/7/4/Reading.csv",
"adl://datalake.azuredatalakestore.net/2015/7/5/Reading.csv",
"adl://datalake.azuredatalakestore.net/2015/7/6/Reading.csv",
"adl://datalake.azuredatalakestore.net/2015/7/7/Reading.csv"

Note: these files are present in different folder.

Can above thing is possible using loop?

Alexandre Gattiker · Accepted Answer · 2016-12-08 10:06:43Z

2

The proper way to do this is to use virtual columns, then rely on partition elimination so that only files matching predicates will actually be read (you can confirm that in the job graph).

CREATE VIEW IF NOT EXISTS dbo.ReadingConsolidated 
AS

EXTRACT     
        ControllerID int?,          
        ParameterID int?,
        MeasureDate DateTime,
        Value float,
        date DateTime
    FROM  
    "adl://datalake.azuredatalakestore.net/{date:yyyy}/{date:M}/{date:d}/Reading.csv";


@res =
    SELECT * FROM dbo.ReadingConsolidated
    WHERE date BETWEEN DateTime.Parse("2015/07/01") AND DateTime.Parse("2016/07/07");

answered Dec 8, 2016 at 10:06

Alexandre Gattiker

7493 silver badges5 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

How to implement Loops in U-SQL

1 Answer 1

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related