Deploying helm charts via Terraform Helm provider and Azure DevOps while fetching the helm charts from ACR

Question

I am trying to deploy the helm charts from ACR to an AKS cluster using Terraform helm provider and Azure DevOps container job but it fails while fetching the helm chart from ACR. Please let me know what is going wrong.

helm provider tf module:

data "helm_repository" "cluster_rbac_helm_chart_repo" {
  name = "mcp-rbac-cluster"
  url  = "https://mcpshareddcr.azurecr.io"
}
# Deploy Cluster RBAC helm chart onto the cluster
resource "helm_release" "cluster_rbac_helm_chart_release" {
  name  = "mcp-rbac-cluster"
  repository = data.helm_repository.cluster_rbac_helm_chart_repo.metadata[0].name
  chart = "mcp-rbac-cluster"
}

provider:

  version                    = "=1.36.0"
  tenant_id                  = var.ARM_TENANT_ID
  subscription_id            = var.ARM_SUBSCRIPTION_ID
  client_id                  = var.ARM_CLIENT_ID
  client_secret              = var.ARM_CLIENT_SECRET
  skip_provider_registration = true
}

data "azurerm_kubernetes_cluster" "aks_cluster" {
  name                = var.aks_cluster
  resource_group_name = var.resource_group_aks
}

locals {
  kubeconfig_path = "/tmp/kubeconfig"
}

resource "local_file" "kubeconfig" {
  filename = local.kubeconfig_path
  content  = data.azurerm_kubernetes_cluster.aks_cluster.kube_admin_config_raw
}

provider "helm" {
  home = "resources/.helm"
  kubernetes {
    load_config_file = true
    config_path = local.kubeconfig_path
  }
}

module "aks_resources" {
  source = "./modules/helm/aks-resources"
}

error: Error: Looks like "" is not a valid chart repository or cannot be reached: Failed to fetch /index.yaml : 404 Not Found

At a glance I'd say that data.helm_repository.cluster_rbac_helm_chart_repo.metadata[0].name should be data.helm_repository.cluster_rbac_helm_chart_repo.metadata.0.name but not 100% sure. Would be useful if you could show the full error you are getting as well instead of the extract in your question currently. — ydaetskcoR
– ydaetskcoR, Commented Jan 2, 2020 at 15:26

Dharman · Accepted Answer · 2020-11-10 03:10:44Z

5

Until now, Helm still doesn't support directly installing chart from an OCI registry.

The recommended steps are:

helm chart remove mycontainerregistry.azurecr.io/helm/hello-world:v1
helm chart pull mycontainerregistry.azurecr.io/helm/hello-world:v1
helm chart export mycontainerregistry.azurecr.io/helm/hello-world:v1 --destination ./install
cd install & helm install myhelmtest ./hello-world

So my solution is:

resource "null_resource" "download_chart" {
  provisioner "local-exec" {
    command = <<-EOT
      export HELM_EXPERIMENTAL_OCI=1
      helm registry login mycontainerregistry.azurecr.io --username someuser --password somepass
      helm chart remove mycontainerregistry.azurecr.io/helm/hello-world:v1
      helm chart pull mycontainerregistry.azurecr.io/helm/hello-world:v1
      helm chart export mycontainerregistry.azurecr.io/helm/hello-world:v1 --destination ./install
    EOT
  }
}

resource "helm_release" "chart" {
  name             = "hello_world"
  repository       = "./install"
  chart            = "hello-world"
  version          = "v1"

  depends_on = [null_resource.download_chart]
}

Not perfect but works.

edited Nov 10, 2020 at 3:10

Dharman♦

33.9k27 gold badges106 silver badges157 bronze badges

answered Nov 10, 2020 at 3:05

Jason Li

1,5851 gold badge12 silver badges20 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Mr-DC Over a year ago

Note: The Terraform Helm provider >=v2.5.0 does now support OCI registries

Robert Cutajar · Accepted Answer · 2021-03-11 13:21:47Z

2

The problem is that you use the wrong url in the Terraform helm_repository. The right url for ACR looks like this:

https://acrName.azurecr.io/helm/v1/repo

And the ACR is a private registry, so it means you need to add the username and password for it. Finally, your Terraform code should like this with version 2.0+ of helm provider:

resource "helm_release" "my-chart" {
  name  = "my-chart"
  chart = "my/chart"
  repository  = "https://${var.acr_name}.azurecr.io/helm/v1/repo"
  repository_username = var.acr_user_name
  repository_password = var.acr_user_password
}

Or with 1.x helm provider:

data "helm_repository" "cluster_rbac_helm_chart_repo" {
  name = "mcp-rbac-cluster"
  url  = "https://mcpshareddcr.azurecr.io/helm/v1/repo"

  username = "xxxxx"
  password = "xxxxx"
}

# Deploy Cluster RBAC helm chart onto the cluster
resource "helm_release" "cluster_rbac_helm_chart_release" {
  name  = "mcp-rbac-cluster"
  repository = data.helm_repository.cluster_rbac_helm_chart_repo.metadata[0].name
  chart = "mcp-rbac-cluster"
}

Update

Here is the screenshot that it works well and deploy the charts in the AKS:

edited Mar 11, 2021 at 13:21

Robert Cutajar

3,8011 gold badge37 silver badges47 bronze badges

answered Jan 3, 2020 at 6:13

Charles Xu

31.8k2 gold badges26 silver badges42 bronze badges

12 Comments

user3616775 Over a year ago

Thanks for the response. I did put the URL as above and it reached the ACR and failed with

Error: Looks like "***/helm/v1/repo" is not a valid chart repository or cannot be reached: Failed to fetch ***/helm/v1/repo/index.yaml : 401 Unauthorized

In my case I can't provide the username and password directly in the terraform file since I am using the service principal via the azure devops pipeline so is there any mean i can pass the username and password from the pipeline to the tf file rather then writing it in file ?

Charles Xu Over a year ago

@user3616775 Do you read the answer carefully? I said you need to add the username and password of your ACR.

user3616775 Over a year ago

yes i read that but i can't pass it directly ``` - bash: 'terraform init -input=false' env: TF_VAR_ARM_ACCESS_KEY: $(storage_access_key) TF_VAR_ARM_CLIENT_ID: $(AZURE_CLIENT_ID) TF_VAR_ARM_CLIENT_SECRET: $(AZURE_CLIENT_SECRET) TF_VAR_ARM_SUBSCRIPTION_ID: $(AZURE_SUBSCRIPTION_ID) TF_VAR_ARM_TENANT_ID: $(AZURE_TENANT_ID) TF_LOG: DEBUG displayName: 'terraform init' ```

Charles Xu Over a year ago

@user3616775 Do you set the credential of your ACR in the Terraform? It's not the sp credential, it's the ACR's.

Charles Xu Over a year ago

@user3616775 Do you solve the problem? If the answer works for you, please accept it.

|

Patrick Picard · Accepted Answer · 2021-04-08 14:00:07Z

Small enhancement to the above solution. Include a trigger to force a download of the chart every time. Otherwise, it expects that you always maintain the local copy of the chart post the first deployment

resource "null_resource" "download_chart" {
  triggers = {
    always_run = timestamp()
  }

  provisioner "local-exec" {
    command = <<-EOT
      export HELM_EXPERIMENTAL_OCI=1
      helm registry login ${var.registry_fqdn} --username ${var.acr_client_id} --password ${var.acr_client_secret}
      helm chart remove ${var.registry_fqdn}/helm/${var.chart_name}:${var.chart_tag}
      helm chart pull ${var.registry_fqdn}/helm/${var.chart_name}:${var.chart_tag}
      helm chart export ${var.registry_fqdn}/helm/${var.chart_name}:${var.chart_tag} --destination ./install
    EOT
  }
}

Collectives™ on Stack Overflow

Deploying helm charts via Terraform Helm provider and Azure DevOps while fetching the helm charts from ACR

3 Answers 3

1 Comment

12 Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

1 Comment

12 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related