504 timeout accessing S3 from Lambda with boto3

Question

This is not an issue with the size of the files, the files are very small. It works perfectly where running locally, but not in the Lambda.

I attached a bucket-access policy to Lambda role mylambdarole:

which looks like this:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "s3:PutObject",
                "s3:PutObjectAcl",
                "s3:GetObject",
                "s3:GetObjectAcl",
                "s3:AbortMultipartUpload"
            ],
            "Resource": [
                "arn:aws:s3:::my-bucket",
                "arn:aws:s3:::my-bucket/*"
            ]
        }
    ]
}

I also added a permission policy to my-bucket:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "Lambda permissions",
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::XXX843121XXX:role/mylambdarole"
            },
            "Action": "s3:*",
            "Resource": [
                "arn:aws:s3:::my-bucket",
                "arn:aws:s3:::my-bucket/*"
            ]
        }
    ]
}

Lambda is in a VPC, but I think this shouldn't be a problem as I have a route to internet gateway:

The endpoint uses boto3, which for example loads data from the bucket:

def get_user_data_from_bucket(user_id: str):
    s3_client = boto3.client("s3", aws_access_key_id=ACCESS_KEY, aws_secret_access_key=SECRET_KEY, region_name=REGION)
    try:
        data = s3_client.get_object(Bucket=BUCKET_NAME, Key=f"{user_id}/user_data.json")
        return json.loads(data["Body"].read())
    except ClientError as e:
        if e.response["Error"]["Code"] == "NoSuchKey":
            return {}
        raise e

I also tried adding config=Config(s3={'addressing_style':'path'}) to the client, but it didn't solve the issue.

As I said, it works when running the server locally. However, when the endpoint for downloading data is called, there is no error response, only after some time I get 504: Endpoint request timed out. Accessing RDS from the same Lambda function works just fine.

Why are you using aws_access_key_id & aws_secret_access_key when using a Lambda execution role? — Ermiya Eskandary
– Ermiya Eskandary, Commented Mar 1, 2022 at 12:42
Well I need it to use it locally and I assume the lambda just sets those environment variables. — KarloSpacapan
– KarloSpacapan, Commented Mar 1, 2022 at 12:48

jarmod · Accepted Answer · 2022-03-01 15:50:02Z

2

You configured your Lambda function to connect to a VPC public subnet. Lambda functions connected to a public subnet cannot access the internet. Even though it routes to the IGW, the Lambda does not have a public IP, and so its outbound packets are dropped at the IGW. More details here.

Re-configure the Lambda function to connect to a private subnet.

Also, there's no need for that S3 bucket policy (use the IAM role) or the manual credentials (use the IAM role).

edited Mar 1, 2022 at 15:50

answered Mar 1, 2022 at 14:43

jarmod

79.8k18 gold badges132 silver badges137 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

KarloSpacapan Over a year ago

Thank you. Based on the answer you linked I looked at VPC endpoints and followed stackoverflow.com/a/57742313/13504606, which solved the issue.

Collectives™ on Stack Overflow

504 timeout accessing S3 from Lambda with boto3

1 Answer 1

1 Comment

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Linked

Related