Azure SQL DW to Azure SQL DW using Polybase

Question

I know you can use polybase using external table to load large volume of data from Blob Storage to Azure SQL DW. But is there any possibility that we can import the data from SQL DW to another SQL DW using polybase directly? Or is there some other way? There must be some way to avoid control node in both SQL DW.

wBob · Accepted Answer · 2017-03-09 10:37:23Z

8

You might be better off using Azure Data Factory to move data between two Azure SQL Data Warehouses. It would make light work of moving the data, but beware any data movement costs, particularly moving across region. Start here. Check the 'Use Polybase' checkbox.

If you do just want to use Polybase and Blob Storage, then you would have to:

first export the data from the source system internal tables to blob store using CETAS.
in the target system create external tables over the files in blob store
n the target system import the data from the external tables into the database using CTAS.

answered Mar 9, 2017 at 10:37

wBob

14.4k3 gold badges26 silver badges43 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

HimalayanNinja Over a year ago

Thank you. I think CETAS->Blob->CTAS is way to go. I couldn't find any better solution

wBob Over a year ago

Consider marking that as an answer if you have found it helpful.

Sascha Dittmann · Accepted Answer · 2017-03-07 19:29:24Z

2

As far as I know, you have to use PolyBase and either Blob Storage or Data Lake Store to get the maximum throughput (bypass the control node)

answered Mar 7, 2017 at 19:29

Sascha Dittmann

6364 silver badges9 bronze badges

Comments

hiroki · Accepted Answer · 2017-03-14 16:27:10Z

2

You can create a new SQL DW from a geo-backup which should be a complete copy of the SQL DW with a 24 hour SLA. First click on create new SQL DW and select backup as an option as opposed to blank or sample.

answered Mar 14, 2017 at 16:27

hiroki

4072 silver badges15 bronze badges

2 Comments

wBob Over a year ago

This is potentially a good option @hirokibutterfield if you want to duplicate an entire warehouse. I think there are some constraints around where you can restore from, for example can you only restore from between paired data centers?

hiroki Over a year ago

You can restore into any data center but the costs and time to restore vary. The paired region is the fastest and cheapest, followed by the original region, and slowest and most expensive is any other data center as you incur egress charges.

Collectives™ on Stack Overflow

Azure SQL DW to Azure SQL DW using Polybase

3 Answers 3

2 Comments

Comments

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

2 Comments

Comments

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related