Retrieve file from S3 with curl from EC2 using IAM role

Question

I created an IAM role and assigned it to my EC2 instance. I created an IAM role policy and linked it to my S3 bucket. I can now list the files on my S3 bucket using the aws-cli on my EC2 instance.

[[email protected] ~]# aws --region eu-west-1 s3 ls s3://s3-bucket-name
2017-05-02 02:37:39        137 index.html

Now if I try to curl that file I get an access denied error:

[[email protected] ~]# curl https://s3-eu-west-1.amazonaws.com/s3-bucket-name/index.html
<?xml version="1.0" encoding="UTF-8"?>
<Error><Code>AccessDenied</Code><Message>Access Denied</Message><RequestId>782176F442A4995A</RequestId><HostId>qZWZTXZdlgtD4BiVYfp5+Mj/gU3UXRiYQmsSyaAhwED1JtzUWs9QKE99RmsQ6w0uDB

Is there a way I could make this curl request work without relaxing the permissions on the S3 bucket? Something like authentication, headers etc etc.

curl `aws --region eu-west-1 s3 presign s3://s3-bucket-name/index.html` — sajal
– sajal, Commented May 2, 2017 at 3:39

John Rotenstein · Accepted Answer · 2017-05-02 03:36:58Z

7

GET requests to Amazon S3 require an authorization string that uses AWS Signature v4.

See: GET Object

Alternatively, you could create a Pre-Signed URL which has extra parameters and grants time-limited access to an Amazon S3 object. You can generate a Pre-Signed URL with a couple of lines of code and then provide it to users (or use it in an HTML page) to grant temporary access to private objects stored in Amazon S3.

Of course, the easiest way to retrieve data from Amazon S3 is by using API calls or the AWS Command-Line Interface (CLI).

answered May 2, 2017 at 3:36

John Rotenstein

273k28 gold badges456 silver badges541 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

kevlarr Over a year ago

Presigned url is a great strategy, especially if you're using a library that doesn't let you add headers to the curl command.

Collectives™ on Stack Overflow

Retrieve file from S3 with curl from EC2 using IAM role

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related