Files served unbearably slow from amazon s3

Question

I have a django app on heroku which serves the static files from amazon s3 bucket. I use boto library and followed the guide on the website. What can I do to speed up the file transfers?

Some of the code:

DEFAULT_FILE_STORAGE = 'storages.backends.s3boto.S3BotoStorage'
AWS_ACCESS_KEY_ID = 'xxxx'
AWS_SECRET_ACCESS_KEY = 'xxxx'
AWS_STORAGE_BUCKET_NAME = 'boxitwebservicebucket'
STATICFILES_STORAGE = 'storages.backends.s3boto.S3BotoStorage'
STATIC_URL = 'http://' + AWS_STORAGE_BUCKET_NAME + '.s3.amazonaws.com/'

the view

class GetFileContent(View):
    def get(self,request, userName, pk):
        user = User.objects.filter(username = userName)
        filtered = Contentfile.objects.filter(pk = pk, published=True, file_owner = user)
        data = filtered[0].content
        filename = filtered[0].title + "." + filtered[0].file_type
        response = HttpResponse(data, content_type='application/force-download')
        response['Content-Disposition'] = 'attachment; filename="{}"'.format(filename)
        return response
        pass

I suspect django is serving the file even though it sits on the s3 server, How can I direct the user to the s3 link directly?

I would recommend serving files from S3 via CloudFront instead. — Brandon Taylor
– Brandon Taylor, Commented Dec 20, 2013 at 12:51
Currently, Im in demo stages and I would like to keep everything free. However Im not sure that's the problem, I'm beginning to suspect that actually even though I'm using amazon s3 storage, Django is serving the files instead of amazon because when I look at my network traffic, I can't see any links leading to amazon. — Yoav Schwartz
– Yoav Schwartz, Commented Dec 20, 2013 at 13:07
CloudFront would only cost you $0.12 for the first 10 TB out per month. That's pretty affordable :). If your network traffic says that your files are being served by Django, that's a different issue altogether. — Brandon Taylor
– Brandon Taylor, Commented Dec 20, 2013 at 13:10
Yeah, It's the first time I'm trying to serve files using Django, and I can find a lot of documentation on how to set up the static files for the site to be served with django, but non about how to serve downloadable file, I would have guessed it would be more common of a problem. — Yoav Schwartz
– Yoav Schwartz, Commented Dec 20, 2013 at 13:12
If you look here at django-storages, you will see that the url function accepts a header param. bitbucket.org/david/django-storages/src/… You should try to generate a url for that file using django storages, making sure to pass the Content-Disposition 'attachment' in the headers, and then return a 302 redirect to that url. If all goes well the file will be served from Amazon with the right header, forcing the browser to download instead of open the url. — Pierre Drescher
– Pierre Drescher, Commented Jan 7, 2014 at 21:02

Mathieu Steele · Accepted Answer · 2014-01-10 03:14:23Z

Here is how I do it - It does not feel slow to me:

models.py:

class Document(models.Model):
    id = UUIDField(primary_key = True)
    extension = models.CharField(max_length = 5)
    created_on = CreationDateTimeField()
    labels = models.ManyToManyField(Label)

    def url(self, bucket):
        url = get_s3_url(bucket, '' + str(self.id) + str(self.extension) + '')
            return 'https' + url[4:]

views.py:

import urllib2


@login_required
def view(request, document_id):
    document = Document.objects.get(id = document_id)

    response_file = urllib2.urlopen(document.url(request.user.profile.aws_documents_bucket_name))

    response = HttpResponse(response_file.read(), mimetype = document.mimetype)
    response['Content-Disposition'] = 'inline; filename=' + str(document.id) + document.extension

    return response

utils.py:

from boto.s3.connection import S3Connection
from boto.s3.key import Key
from django.conf import settings


def get_s3_url(bucket, filename):
    s3 = S3Connection(settings.AWS_ACCESS_KEY_ID, settings.AWS_SECRET_ACCESS_KEY)
    key = s3.create_bucket(bucket).get_key('' + filename + '')
    return key.generate_url(3600, "GET", None, True, True) # This gives an authenticated url available for only a short time period (by design)

My individual users or groups of users have designated buckets referenced in the profile object. AWS Credentials stored in settings.py.

Collectives™ on Stack Overflow

Files served unbearably slow from amazon s3

1 Answer 1

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related