Uploading a File to GAE Blobstore from URL

Question

I need to get a video from a URL, for example:

https://zencoder-temp-storage-us-east-1.s3.amazonaws.com/o/20130722/5aacb76fc3fd47715c0329d1235dcccf/4fc612e92131e159abc761f7d54d86b5.mp4?AWSAccessKeyId=AKIAI456JQ76GBU7FECA&Signature=AGgZb1eMr105RXcoQFp8yFFTfFg%3D&Expires=1374614893

and then use Google App Engine to save it to the blobstore (or possibly Google Cloud Storage if the blobstore can't download straight from a URL) is there a simple way to do this? I already have it set up so that I can get a user to upload a video but I am not sure about doing it this way. Is the URLFetch Library what I'm looking for?

Would it be something like:

class UploadHandler(blobstore_handlers.BlobstoreUploadHandler):
  def post(self):
    #I'm just not sure what to do here, how I can get that 'file' from a URL
    upload_files = self.get_uploads('file')
    blob_info = upload_files[0]

    video = Video(
        title = "some title",
        video_ref = blob_info.key())
    video.put()

Faisal · Accepted Answer · 2013-07-22 22:41:51Z

1

Yeah you will need to use urlfetch then use the google cloud storage client library(https://developers.google.com/appengine/docs/python/googlecloudstorageclient/functions#open) to write the results. You need to consider that urlfetch response size is limited to 32mb so you need to split download into 32mb pieces and also there is a 60 seconds user request limit or 10 minute taskqueue limit or the use backends which doesn't have a timeout limit.

You could also directly upload to blobstore or cloudstorage (https://developers.google.com/appengine/docs/python/blobstore/#Uploading_a_Blob) which is handled by special instances that is designed to handle these uploads.

answered Jul 22, 2013 at 22:41

Faisal

2,27615 silver badges19 bronze badges

Sign up to request clarification or add additional context in comments.

5 Comments

clifgray Over a year ago

What do you mean "directly upload" and what sort of special instance?

Faisal Over a year ago

It seems like your sample code above is for the result of blobstore handler anyway. So this is something that you would use if you post to the form with action=blobstore.create_upload_url('/upload') where /upload is routed to UploadHandler. This has no timeout limits except the blobstore filesize limit.

clifgray Over a year ago

Right, well I know pretty well how to get the user to upload a video, I just need to do it from a different URL

Faisal Over a year ago

Then yeah the only way is download and write with cloud storage client or files api which I believe is deprecated. I hope the cloud storage client supports blobstore.

Aerox Over a year ago

is there a way or a working example on how to use Google Cloud Storage with JAVA/JAVASCRIPT instead of always Python Python Python and another time Python? :-)

Collectives™ on Stack Overflow

Uploading a File to GAE Blobstore from URL

1 Answer 1

5 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

5 Comments

Your Answer

Sign up or log in

Post as a guest

Related