ADF Pipeline - Shall we create separate Linked service for different tasks of same nature running in parallel

Question

I've 4 data-bricks notebook activities to run in parallel to do the same task but for different parameters (variables). Do I need to create separate Linked service for each notebook connection when same Databricks workspace is being used but notebooks are different?

Here inside each If activity I have one databricks activity.

Q2. Will that affect the performance if I use the same linked service?

I've made 4 different linked service with different names but it seems to be a bad idea.

Nandan · Accepted Answer · 2023-12-18 17:31:55Z

0

you should use the same linked service. Linked service is just a connection to your source and sink and the actual transformations are happening there irrespective of from where you call. So ideally there won't be any change in performance if you use same linked service or multiple linked service since the engine being triggered is the same.

answered Dec 18, 2023 at 17:31

Nandan

5,2252 gold badges12 silver badges28 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

ADF Pipeline - Shall we create separate Linked service for different tasks of same nature running in parallel

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related