Creating presigned url for a S3 folder in python

Question

I'm trying to generate a presigned url to a S3 folder (which itself contains more folders/files) and distribute it between my clients so they can download its content. i.e. by clicking the link, the users will download the folder to their local disk.

However, I keep getting a "no such key" error in an XML dialogue.

I'm using client.generate_presigned_url() from boto3 sdk

def create_presigned_url(bucket, object):
    try:
        url = s3_client.generate_presigned_url(
            'get_object',
            Params={
                'Bucket': bucket,
                'Key': object
            },
            ExpiresIn=240,
            HttpMethod='GET'
        )
    except ClientError as e:
        print(e)
        return None
    return url

this is the error message:

This XML file does not appear to have any style information associated with it. The document tree is shown below.
<Error>
   <Code>NoSuchKey</Code>
      <Message>The specified key does not exist.</Message>
         <Key>output/BARNES/070419/APR19BAR/</Key>
         <RequestId>E6BE736FE945FA22</RequestId>
         <HostId>
      hk3+d+***********************************************************+EO2CZmo=
          </HostId>
</Error>

Your object variable has to be a key to a single S3 object. As in the answer below a "folder" is not really an object in S3, its a mere part of a key to your actual file. — Tommy
– Tommy, Commented Jun 25, 2019 at 10:39

Ninad Gaikwad · Accepted Answer · 2019-06-25 10:29:26Z

8

S3 has no concept of "folders". What you are effectively trying to do here is create a presigned url for multiple keys which is also not possible. If you absolutely have to share single url for multiple files, you'll need zip them into a single object and then share key of that object using presigned url.

answered Jun 25, 2019 at 10:29

Ninad Gaikwad

4,5202 gold badges16 silver badges26 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Yotam Raz Over a year ago

Thanks Ninad, That option already crossed my mine. Although it sounds promising, I don't want to hold two copies of the same data.

Ninad Gaikwad Over a year ago

A presigned URL is only applicable to a single object. So you'll have to try something else if you want users to be able to download multiple objects.

valex · Accepted Answer · 2020-04-30 13:27:31Z

4

import boto3

s3_client = boto3.client(
    's3',
    region_name='your_region_name',
    aws_access_key_id='your_aws_access_key_id',
    aws_secret_access_key='your_aws_access_key_id',
)

# Just specify folder_name:
url = s3_client.generate_presigned_url(
    ClientMethod='put_object',
    Params={'Bucket': 'your_bucket_name', 'Key': 'folder_name/file_name.txt',},
    ExpiresIn=60,
)

answered Apr 30, 2020 at 13:27

valex

5,8972 gold badges40 silver badges41 bronze badges

Comments

Peter Csala · Accepted Answer · 2020-07-15 08:21:11Z

The generated URL will expire after a week.

Since there is a restriction by AWS: you can keep a link at most one week.
That is approximately 600000 seconds (See ExpiresIn).

def create_presigned_url(bucket, object):
    try:
        bucket_name = bucket
        ACCESS_KEY = "access_key"
        SECRET_KEY = "secret_key"
        key = object
        location = boto3.client('s3', aws_access_key_id=ACCESS_KEY,aws_secret_access_key=SECRET_KEY).get_bucket_location(Bucket=bucket_name)['LocationConstraint']
        s3_client = boto3.client(
                's3',
                region_name=location,
                aws_access_key_id=ACCESS_KEY,
                aws_secret_access_key=SECRET_KEY,
            )
        url = s3_client.generate_presigned_url(
                ClientMethod='get_object',
                Params={'Bucket': bucket_name, 'Key': key, },
                ExpiresIn=600000,
            )
    except ClientError as e:
        print(e)
        return None
    return url

Collectives™ on Stack Overflow

Creating presigned url for a S3 folder in python

3 Answers 3

2 Comments

Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

2 Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related