Using Data Lake Analytics Linked Service Service Principal Authentication in Data Factory

Question

Trying to add a compute node to ADF (v1), using Azure Data Lake Analytics Linked Service to make some simple u-sql jobs. Data Lake Store Linked Service is already in place and working with service principle authentication, but I can't deploy ADLA linked service with servicePrincipalId and Key.

As per this article, ADLA Linked Service supports service principle authentication for some time now: https://learn.microsoft.com/en-us/azure/data-factory/v1/data-factory-usql-activity#azure-data-lake-analytics-linked-service

The intellisense tells me the parameters are not valid and insists on user credential. Here's a screenshot:

ADF Portal Editor ADLA-ls

(Same warnings in visual studio data factory project btw.)

When I deploy with SPN anyway, I get the error "Entity provisioning failed: Internal Server Error".

Anyone successfully use service principle auth with ADLA linked service?

I have the same issue. SP auth is only accepted for data lake storage in ADFv1. I currently have an open support ticket with Microsoft about it. — Paul Andrew
– Paul Andrew, Commented Jan 8, 2018 at 15:09
@PaulAndrew, are you using User Credential auth in the meantime, or do you have any different workaround? — grayincarnation
– grayincarnation, Commented Jan 8, 2018 at 15:36

Simon Zeinstra · Accepted Answer · 2018-01-10 11:34:57Z

1

It works for me using the following settings:

{
  "name": "datalakeanalyticsinstanceLinkedService",
  "properties": {
    "type": "AzureDataLakeAnalytics",
    "typeProperties": {
      "accountName": "datalakeanalyticsinstance",
      "servicePrincipalId": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxx",
      "servicePrincipalKey": "abcdefghijklmopqrstuvwxyz123456780",
      "tenant": "mytenant.onmicrosoft.com"
    }
  }
}

Also make sure you've run the "Add User Wizard" from Data Lake Analytics. Using this wizard I added the service principal as an Owner with R+W permissions.

Edit: This blog post was just published with many details on Serivce Principal authentication

edited Jan 10, 2018 at 11:34

answered Jan 8, 2018 at 15:07

Simon Zeinstra

82510 silver badges19 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

grayincarnation Over a year ago

do you know if the wizard is any different from adding users via Access Control (IAM)?

Simon Zeinstra Over a year ago

Yes, for example, the wizard also adds permissions on de data lake store file system. I've also updated the above answer with a blog post, explaining Service Principal authentication with Data Lake Analytics / Data Lake Store. When you run the wizard it shows you a list of actions that it perfoms.

Simon Zeinstra Over a year ago

I cannot comment on the discussion above, but I have SP Authentication running on both V1 & V2 of the datafactory.

grayincarnation · Accepted Answer · 2018-01-24 13:26:16Z

1

After a couple of weeks of frustration, suddenly this issue resolved itself.

Adding SP via the "Add Users Wizard" from ADLA is certainly a must for any data lake related pipelines, as @Simon suggested. Other than that @Paul's blog clearly explains the process.

The fact that my pipeline suddenly started working at exactly midnight and hasn't failed since, makes me think ADF has recently got some bugs fixed. So until ADFv2 is fully GA, I suggest deploying the linked services over and over again, or better yet (and more frustrating) deploying new pipelines and services from scratch.

answered Jan 24, 2018 at 13:26

grayincarnation

651 silver badge11 bronze badges

Collectives™ on Stack Overflow

Using Data Lake Analytics Linked Service Service Principal Authentication in Data Factory

2 Answers 2

3 Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

3 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related