Dynamically refer to a Terraform data resource by variable containing its name

Question

I'm trying to concatenate a declared variable within the Terraform data assignation to build a dynamic call.

Having the following code:

# Policy 1
data "aws_iam_policy_document" "1_s3_access_policy" {
   statement {
     effect = "Allow"
     actions = [
      "s3:GetObject",
      "s3:PutObject",
      "s3:DeleteObject",
     ]
     resources = [
      "arn:aws:s3:::1_s3_access_policy/*",
      "arn:aws:s3:::1_s3_access_policy",
     ]
     principals {
      type        = "AWS"
      identifiers = ["arn:aws:iam::67435677645:user/d2c-user-us-west-1"]
     }
   }
 }
# policy 2
data "aws_iam_policy_document" "2_s3_access_policy" {
    statement {
     effect = "Allow"
     actions = [
      "s3:GetObject",
      "s3:PutObject",
      "s3:DeleteObject",
     ]
     resources = [
      "arn:aws:s3:::2_s3_access_policy/*",
      "arn:aws:s3:::2_s3_access_policy",
     ]
     principals {
      type        = "AWS"
      identifiers = ["arn:aws:iam::67435677645:user/d2c-user-us-west-1"]
     }
   }
 }
# Policy 3
...
variable "s3_bucket_names" {
      type    = list(any)
      default = ["1_s3_access_policy", "2_s3_access_policy", "3_s3_access_policy"]
}

module "platform-cloud" {
      source              = "./module"
      count               = length(var.s3_bucket_names) //count will be 3
      bucket_name         = var.s3_bucket_names[count.index]
      sse_algorithm       = "aws:kms"
      iam_policy_document =  data.aws_iam_policy_document.${var.s3_bucket_names[count.index]}.json
}

It's failing with error:

Error: Invalid character

on main.tf line 10, in module "platform-cloud":

iam_policy_document = data.aws_iam_policy_document.${var.s3_bucket_names[count.index]}.json

This character is not used within the language.

Is there a way to concatenate a variable within terraform data call?

Martin Atkins · Accepted Answer · 2021-10-16 16:07:29Z

4

Terraform cannot dynamically look up a resource in the way you are attempting because that would mean that the resource dependency would not be decided until evaluation time, but Terraform needs to know the correct dependency order before evaluating any expressions.

However, you can meet your use-case of dynamically selecting a resource with the extra step of creating a mapping using the keys that you'll then use to select each one:

locals {
  bucket_policies = {
    s3_access_policy_1 = data.aws_iam_policy_document.s3_access_policy_1
    s3_access_policy_2 = data.aws_iam_policy_document.s3_access_policy_2
    s3_access_policy_3 = data.aws_iam_policy_document.s3_access_policy_3
  }
}

module "platform-cloud" {
  source = "./module"
  count  = length(var.s3_bucket_names) //count will be 3

  bucket_name         = var.s3_bucket_names[count.index]
  sse_algorithm       = "aws:kms"
  iam_policy_document = local.bucket_policies[var.s3_bucket_names[count.index]].json
}

I have changed the names of your data resources to be like s3_access_policy_1 instead of 1_s3_access_policy because Terraform does not allow resource names to start with digits. Terraform isn't reporting that error message yet because you have a syntax error, but resolving the syntax error would've then exposed the naming error.

Notice that now module.platform-cloud's iam_policy_document refers to the whole of local.bucket_policies, which in turn depends on all three of the data resources. Terraform therefore understands that it must evaluate all three of those data resources before evaluating that module argument, which produces a correct evaluation order.

Although not directly related to your question, I'd recommend reading When to use for_each instead of count to decide whether it might be better to use for_each in your module "platform-cloud" block, instead of count.

edited Oct 16, 2021 at 16:07

answered Oct 16, 2021 at 1:18

Martin Atkins

76.4k9 gold badges152 silver badges168 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Sallyerik Over a year ago

Great approach!! just to need to modify data.aws_iam_policy_document instead of data.aws_iam_policy and it works just fine. Probably I mess a bit naming my buckets and policies code examples with digits but in real case the bucket names are different so just need to assign the correct name in the local bucket policies variable. Thanks mate!

Marcin · Accepted Answer · 2021-10-15 00:08:25Z

1

Normally you would do this as follows:

iam_policy_document =  data.aws_iam_policy_document[var.s3_bucket_names[count.index]].json

Exact details would depend on how data.aws_iam_policy_document is defined. But sadly such information is not given in your question.

answered Oct 15, 2021 at 0:08

Marcin

241k16 gold badges315 silver badges368 bronze badges

6 Comments

jordanm Over a year ago

Syntax is correct, but count is going to be invalid here. Agree is not enough info for a "complete" answer.

Sallyerik Over a year ago

I've edited and added my code with the data content. I've tried @Marcin comment above but it still failing.

Sallyerik Over a year ago

iam_policy_document = data.aws_iam_policy_document[var.s3_bucket_names[count.index]].json  The "data" object must be followed by two attribute names: the data source type and the resource name.

Marcin Over a year ago

@Sallyerik You can't do what you want. You have to fully refactor your data.aws_iam_policy_document definitions. i would suggest making new question dedicated to this new problem.

Sallyerik Over a year ago

count works fine, with the var just wanted to point out the resource name of the data object. I think the content of the data object shouldn't add much value, does it?

|

Collectives™ on Stack Overflow

Dynamically refer to a Terraform data resource by variable containing its name

2 Answers 2

1 Comment

6 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

1 Comment

6 Comments

Your Answer

Sign up or log in

Post as a guest

Related