Run shell command on AWS S3 file upload using AWS Lambda

Question

My application involves creating a file in S3 bucket on every successful request completion. I want to run a shell script as soon as file is uploaded. Shell script can be something like mv file1 file2 (It won't be rename, I have something to do with zipping file and fixing it but it will be a shell command and nothing else like zip -FFv final.zip --out fixed.zip).

Is it possible through AWS-Lambda?

If Yes, will it involve copying file to some other location, perform the operation and upload back to S3 or it will be directly executed on S3 itself as My files are of large sizes (20 GB or so)

I am very new to Lambda, I know we can set triggers on aws but needed to know the details. — R.B.
– R.B., Commented Feb 5, 2020 at 2:00
How often would this process be executing? If it is relatively frequently, then running an Amazon EC2 instance would be worthwhile. It would not have the disk space limitations of an AWS Lambda function. — John Rotenstein
– John Rotenstein, Commented Feb 5, 2020 at 8:25

Jason Wadsworth · Accepted Answer · 2020-02-05 02:05:39Z

1

It's really going to depend on what you want to do with the file. A large cannot really be processed with a Lambda function because of the memory and disk limitations in Lambda (512MB disk space and max of ~3GB memory). There are some S3 functions that can be done that don't require downloading the file. As for doing it as a shell command, it's doable, but not as easily as writing code (though PowerShell is natively supported).

answered Feb 5, 2020 at 2:05

Jason Wadsworth

8,99026 silver badges38 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Run shell command on AWS S3 file upload using AWS Lambda

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related