Connection refused creating s3 bucket AWS Localstack with Terraform using docker compose

Question

I have this issue regarding creating s3 bucket using AWS localstack, Terraform using docker-compose. I am using docker-compose with TF to create a bucket and creating the bucket is given connection refused. The http://localhost:4566/health on the browser is running and return the services for the AWS localstack.

Error :

Error: creating Amazon S3 (Simple Storage) Bucket (bucket-c): RequestError: send request failed
terraform  | │ caused by: Put "http://localhost:4566/bucket-c": dial tcp 127.0.0.1:4566: connect: connection refused
terraform  | │ 
terraform  | │   with aws_s3_bucket.bucket["bucket-c"],
terraform  | │   on main.tf line 1, in resource "aws_s3_bucket" "bucket":
  | │    1: resource "aws_s3_bucket" "bucket" {

The docker compose :

version: '3'
services:
  localstack:
      image: localstack/localstack
      container_name: localstack 
      network_mode: bridge       
      ports:
        - "127.0.0.1:4566:4566"            # LocalStack Gateway
        - "127.0.0.1:4510-4559:4510-4559"  # external services port range
        - "127.0.0.1:443:443"              # LocalStack HTTPS Gateway
      networks:
        - test-net
      hostname: localstack
      environment:
        - AWS_DEFAULT_REGION=eu-west-2
        - AWS_ACCESS_KEY_ID=test
        - AWS_SECRET_ACCESS_KEY=test
        - DEBUG=${DEBUG:-1}
        - DOCKER_HOST=unix:///var/run/docker.sock
        - LS_LOG=WARN
        - HOSTNAME_EXTERNAL=localstack
        - SERVICES=s3,kafka
        - HOST_TMP_FOLDER=${TMPDIR}
        - DATA_DIR=/tmp/localstack/data
        - PERSISTENCE=${PERSISTENCE-}
      volumes:
          - "${TMPDIR:-/tmp}/localstack:/var/lib/localstack"
          - "/Users/devops/.rd/docker.sock:/var/run/docker.sock"
          - "localstack_data:/tmp/localstack"
  terraform:
    image: hashicorp/terraform:latest  # Use an official Terraform image
    working_dir: /app
    volumes:
      - ./terraform:/app
    depends_on:
      - localstack
    environment:
      AWS_ACCESS_KEY_ID: test
      AWS_SECRET_ACCESS_KEY: test
      AWS_DEFAULT_REGION: eu-west-2
    entrypoint: ["sh", "/app/run-terraform.sh"]

volumes:
  kafka-data:
  localstack_data:

The tf for the s3 bucket is :

provider "aws" {
  region   = "eu-west-2"
  access_key                = "test"
  secret_key                  = "test"
  skip_credentials_validation = true
  skip_metadata_api_check     = true
  skip_requesting_account_id  = true
  s3_use_path_style = true

 endpoints {

    s3 = "http://localhost:4566"

  }
}
    resource "aws_s3_bucket" "bucket" {
      for_each = toset(["bucket-a", "bucket-b", "bucket-c"])
      bucket   = each.key
    }
    resource "aws_s3_bucket_acl" "s3_bucket" {
      for_each = aws_s3_bucket.bucket
      bucket   = aws_s3_bucket.bucket[each.key].id
      acl      = "public-read-write"
    }
    
    resource "aws_s3_bucket_policy" "s3_bucket_policy" {
      for_each = aws_s3_bucket.bucket
      bucket = each.value.id
      policy = jsonencode({
        Version = "2012-10-17",
        Statement = [
          {
            Sid       = "PublicReadGetObject",
            Effect    = "Allow",
            Principal = "*",
            Action    = "s3:GetObject",
            Resource  = [
              "${each.value.arn}/*",
            ],
          },
        ],
      })
    }

Anca G · Accepted Answer · 2023-08-06 20:03:51Z

4

There are two things that you missed here: make sure your terraform container is on the defined network, test-net, and also the S3 endpoint should point to "http://localstack:4566" (the LocalStack container). I know the docs show a configuration for localhost, but those were meant to run on your machine and not in a container. Hope this helps.

answered Aug 6, 2023 at 20:03

Anca G

2241 silver badge3 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

user5820327 Over a year ago

I have removed the test-net though and used the as endpoint localstack:4566. the new error comes up Put "localstack:4566/bucket-c": dial tcp: lookup localstack on 127.0.0.11:53: no such host

user5820327 Over a year ago

It is resolved and would accept the answer as using s3 endpoint with localstack:4566 helps alot. I need to add networks to the localstack, terraform services and then put the networks again at the end in this format terraform: image: hashicorp/terraform:latest networks: - mynetwork localstack: image: localstack/localstack networks: - mynetwork networks: mynetwork:

E_K Over a year ago

I'm not sure why this error only happens with s3 and not other resources. I'm using tflocal which automatically uses "http://localstack:4566"

E_K Over a year ago

for the new error Put "localstack:4566/bucket-c": dial tcp: lookup localstack on 127.0.0.11:53: no such host. I only had to set S3_HOSTNAME=localstack

Collectives™ on Stack Overflow

Connection refused creating s3 bucket AWS Localstack with Terraform using docker compose

1 Answer 1

4 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

4 Comments

Your Answer

Sign up or log in

Post as a guest

Related