AWS Lambda S3 Event Trigger - Object Key Has Pluses Instead of Spaces

Question

I have an AWS Lambda function set up as an event source for an S3 bucket to process newly uploaded PDF files. However, I've noticed that when a file with spaces in its name is uploaded, the event data I receive in the Lambda function has pluses (+) instead of spaces in the key property of the S3 event object.

For example, if I upload a file named "World War.pdf" to my S3 bucket, the Lambda event object shows the key as "public/World+War.pdf":

{
    "s3SchemaVersion": "1.0",
    "configurationId": "arn:aws:cloudformation:eu-west-1:12345673333312:stack/my-lambda-stack/...",
    "bucket": {
        "name": "mybucket",
        "ownerIdentity": { "principalId": "EXAMPLE" },
        "arn": "arn:aws:s3:::mybucket"
    },
    "object": {
        "key": "public/World+War.pdf",
        "size": 40656,
        "eTag": "123456789abcdef",
        "sequencer": "1234567890ABCDEF"
    }
}

This causes issues when I try to use this key to access the file in my S3 bucket, as the actual file name contains spaces and not pluses.

What I've tried:

I have considered using decodeURIComponent to convert the + back to spaces, but I'm unsure if this is a reliable solution, or if I'm supposed to handle these keys differently.

const key = decodeURIComponent(event.Records[0].s3.object.key);
I have also looked into the S3 event notification configurations to see if there's a way to receive the key without URL encoding, but found nothing in the AWS documentation.

Questions:

Why does AWS encode the S3 object key in the event, and is decodeURIComponent the recommended approach to decode it?
Is there a configuration in AWS S3 or Lambda that I can set to receive the object key with spaces instead of pluses?

CloudWatch Error Message:

Error processing and ingesting PDF: public/World+War.pdf. Error: NoSuchKey: The specified key does not exist.

Any help or guidance on this would be greatly appreciated!

Yes it's being encoded. Why do you need to convert back to spaces? If your Lambda function just needs to read the object in S3, then the object key with + will work. — Mark B
– Mark B, Commented Nov 9, 2023 at 13:53
Hi Mark, it's because I sent the file name to an SQS queue, and when the queue consumer tries to obtain the file name, it cannot due to the +. — Lloukas
– Lloukas, Commented Nov 9, 2023 at 14:17
"when the queue consumer tries to obtain the file name, it cannot due to the +" That doesn't make much sense. What is the actual error? — Mark B
– Mark B, Commented Nov 9, 2023 at 15:13
Error processing and ingesting PDF: public/World+War.pdf. Error: NoSuchKey: The specified key does not exist. — Lloukas
– Lloukas, Commented Nov 9, 2023 at 16:32

Jahith Hussain · Accepted Answer · 2023-11-09 13:58:42Z

2

I hope you can find a solution at this link. I had a similar issue and found a solution through this link. AWS: how to fix S3 event replacing space with '+' sign in object key names in json

answered Nov 9, 2023 at 13:58

Jahith Hussain

192 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Lloukas · Accepted Answer · 2023-11-10 09:18:30Z

1

The solution for this question was two fold:

Define the variables before using it
Decoding it.

Final solution:

const file = decodeURIComponent(event.Records[0].s3.object.key.replace(/\+/g, " "));

edited Nov 10, 2023 at 9:18

answered Nov 9, 2023 at 14:48

Lloukas

2413 silver badges16 bronze badges

1 Comment

Michael - sqlbot Over a year ago

This is the result of a very early design decision in S3 that can't be fixed because so much code accommodates the broken behavior. Object keys have spaces replaced with + like you would find in a query string. It's very much incorrect to encode this way in the path, but that's now it was done in S3.

Collectives™ on Stack Overflow

AWS Lambda S3 Event Trigger - Object Key Has Pluses Instead of Spaces

2 Answers 2

Comments

1 Comment

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

1 Comment

Your Answer

Sign up or log in

Post as a guest

Linked

Related