AWS Lambda Python 3 handle POST file

Question

I'm working with Python 3.8 on AWS Lambda, and I would like to handle posted file, like requests.files with Django. But it's impossible with AWS Lambda. I want to put this file in the S3 with :

s3.put_object(
    Body=fileAsString,
    Bucket=MY_BUCKET,
    Key='my-file.jpg'
)

When I return directly the received event, with the file :

First try : with cgi.parse_multipart : I my handler :

c_type, c_data = parse_header(event['headers']['Content-Type'])
boundary = c_data['boundary'].encode('latin1')
body = event['body'].encode('utf8')

fp = BytesIO(body)
pdict = {
    'boundary': boundary,
    'CONTENT-LENGTH': str(len(body))
}
form_data = parse_multipart(fp, pdict)
fileBytes: bytes = form_data['file'][0]
return ({'statusCode': 200, 'body': json.dumps(str(fileBytes))}

I receive : I also tried with form_data['file'][0].decode('utf-8') but I receive : and I have always the "?"

I should get this, because it's the original image, opened in edition :

Second try : I follow this tutorial : https://medium.com/swlh/upload-binary-files-to-s3-using-aws-api-gateway-with-aws-lambda-2b4ba8c70b8e

So, I tried :

file_content = base64.b64decode(event['body'])

but I get :

string argument should contain only ASCII characters

(the same with validate=False) and when I try to load body as dict with :

body2 = json.loads(event['body'])

I get :

loads failed : Expecting value: line 1 column 1 (char 0)

I tried to add "image/png" and "image/*" to "Binary Media Types" in the settings of the API, but no change.

Any idea ?

Did you found the solution ?

Alex Montoya
– Alex Montoya

2020-04-17 18:54:41 +00:00
Commented Apr 17, 2020 at 18:54 — Alex Montoya
– Alex Montoya, Commented Apr 17, 2020 at 18:54
No, I gave up...

Jérémie RPK
– Jérémie RPK

2020-05-19 12:44:23 +00:00
Commented May 19, 2020 at 12:44 — Jérémie RPK
– Jérémie RPK, Commented May 19, 2020 at 12:44

redPanda · Accepted Answer · 2020-10-30 13:31:28Z

4

This is an old post but I was going crazy trying to solve it.

It turns out that the solution for me was to add ALL binary types to the API Gateway as so:

My issue was that my code was working for a .png file but not a .jpg file. Once the correct MIME type was added "image/jpeg" all worked OK and the Error

string argument should contain only ASCII characters

went away. NOTE: that the API needs to be re-deployed after this change (which is not instantaneous)

Bottom line: this code is my working code:

import json
import boto3
import base64
  
s3 = boto3.client('s3')

def lambda_handler(event, context):
if event['httpMethod'] == 'POST' : 
    fileName = event['multiValueQueryStringParameters']['fileName']
    bodyData = event['body']
    decodedFile = base64.b64decode(bodyData)
    s3.put_object(Bucket=<BUCKET_NAME>, Key=fileName[0], Body=decodedFile)
    return {'statusCode': 200, 
        'body': json.dumps({'message': 'file saved successfully'}), 
        'headers': {'Access-Control-Allow-Origin': '*'}}

answered Oct 30, 2020 at 13:31

redPanda

7957 silver badges17 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Sumant Agnihotri Over a year ago

To add here, in my case the POST request Content-Type header was set to form-data. So adding multipart/form-datato Binary media type, resolved my issue. However, all the other form data is lost when the gateway converts into binary.

msc · Accepted Answer · 2020-06-17 07:16:14Z

When you use requests.files, it will upload images as multipart. It's a standard way to upload multiple content-type files at once but you'll need to parse it when receive. If your application is not sensitive with content-type, just use base64 enc/dec to post and receive like below pseudo code:

Uploader:

import json
import requests
import base64

with open("/path/to/your/file/image.png", "rb") as image_file:
        encoded_string = base64.b64encode(image_file.read())

    payload = {
                'name': 'image.png',
                'file': encoded_string.decode('utf-8')
                }

    url = 'https://api_url/upload'
    r = requests.post(url, data = json.dumps(payload), headers = {})

Lambda side:

import json
import boto3
import base64

def upload(event, context):
    body = json.loads(event['body'])

    s3 = boto3.client("s3", region_name='us-east-1')

    decodedFile = base64.b64decode(body['file'])
    rs = s3.put_object(
                        Bucket='bucket_name',
                        Key=body['name'],
                        Body=decodedFile
                        )

    response = {
        "statusCode": 200,
    }

    return response

user1773219 · Accepted Answer · 2021-02-22 01:36:35Z

0

upload_file = request.files['file']
img = re.sub(b"\xef\xbf\xbd", b'', upload_file.stream.read())

when i upload to the flask, the AWS Lambda add UTF-8 BOM,this is just a trick to cleanup utf-8 bom bytes, it works fine for me.

split-function-add-xef-xbb-xbf-n-to-my-list

edited Feb 22, 2021 at 1:36

answered Feb 20, 2021 at 3:50

user1773219

212 bronze badges

1 Comment

hivert Over a year ago

Can you add some explanations ?

Sebgenie · Accepted Answer · 2023-11-17 10:26:10Z

0

The solution for me was to do:

event['body'] = str(event['body']).encode('utf-8)

answered Nov 17, 2023 at 10:26

Sebgenie

1

Comments

Awwal15 · Accepted Answer · 2024-08-24 18:36:02Z

0

I solved this easily just by adding multipart/form-data to Binary media type an i handled the file extensions in my code to disallow some file types

answered Aug 24, 2024 at 18:36

Awwal15

113 bronze badges

Collectives™ on Stack Overflow

AWS Lambda Python 3 handle POST file

5 Answers 5

1 Comment

Comments

1 Comment

Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

5 Answers 5

1 Comment

Comments

1 Comment

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related