Save image data from a iterator object to AWS S3 in python

Question

I am calling an API that returns a iterator object containing image data. I'd like to iterate over them and upload to s3. I could either open them into .png or .jpeg before or after dumping / uploading them to s3.

import boto3

# Download / open photo
img_obj = gmaps.places_photo(ph, max_width = 500, max_height = 400)
            
print(img_obj)

<generator object Response.iter_content.<locals>.generate at 0x7ffa2dsa7820>
            
s3 = boto3.client('s3')
with open('output_image_{}.png'.format(c), 'w') as data:
    for chunk in img_obj:
       s3.upload_fileobj(data,'mybucket','img/{}'.format(chunk))

Error:

Input <_io.BufferedWriter name='output_image_1.png'> of type: <class '_io.BufferedWriter'> is not supported.

On local machine, I am able to write the file:

with open("output_image_{}.png".format(c), "w") as fp:
    for chunk in img_obj:
       fp.write(chunk)

I'd like to directly save the img_obj on AWS S3.

@jarmod any thoughts?

kms
– kms

2021-08-25 03:56:39 +00:00
Commented Aug 25, 2021 at 3:56 — kms
– kms, Commented Aug 25, 2021 at 3:56

Roxy · Accepted Answer · 2021-08-27 07:38:51Z

1

This can be done by using the s3fs to save the image in png format.

I could recreate and achieve the sample using the below code:

from PIL import Image
import s3fs

s3=s3fs.S3FileSystem(client_kwargs=<aws_credentials>)


img_obj = Image.open('sample_file.png')
img_obj.save(s3.open('s3:///<bucket_name>/sample_file.png', 'wb'), 'PNG')

answered Aug 27, 2021 at 7:38

Roxy

1,0439 silver badges20 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Save image data from a iterator object to AWS S3 in python

1 Answer 1

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related