How to run python file in AWS S3 bucket from EC2?

Question

I tried run python file from AWS S3 storage like

python s3://test-bucket/test/py_s3_test.py

I'm getting Error :

python: can't open file 's3://test-bucket/test/py_s3_test.py': [Errno 2] No such file or directory

Is there anyway to run python file resides in AWS S3.

Thank you.

You will need to download it from S3 first. If you can use the http interface to S3, you could do something like curl https://s3_url/ | python. — jordanm
– jordanm, Commented Jun 17, 2019 at 12:51
As @jordanm pointed out, you can't directly stream an s3 file with python. On top of his answer, you can also try aws-cli for such operations ! — Exho
– Exho, Commented Jun 17, 2019 at 12:54

Red Boy · Accepted Answer · 2019-06-17 13:19:05Z

1

Try this one, it will work.

aws s3 cp s3://yourbucket/path/to/file/hello.py - | python

Explanation: Its downloading the file from S3 and then passing stream to python for execution.

Alternatively, you could split it into multiple steps as well like download the file, save it to any local file and execute the locally saved file.

Hope it helps!

answered Jun 17, 2019 at 13:19

Red Boy

5,7893 gold badges35 silver badges49 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Red Boy Over a year ago

Though I'm still stretching my head to understand, why you need to download the file stored in S3 and execute? Are you looking for something like AWS Lambda?

abhimanyu Over a year ago

yes, Work for me. Created batch file to copy , execute and delete.

Collectives™ on Stack Overflow

How to run python file in AWS S3 bucket from EC2?

1 Answer 1

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related