How to download files programatically from Google Storage via python?

Question

I'm currently trying to download some files from Google Storage using the Python API. Obviously this can be done by using gsutil's cp, and indeed I could do it.

sudo gsutil cp gs://???/stats/installs/*.csv .

However, all examples of Google Python API I've found don't cover this subject, I'm even considering if this functionality is covered by the API.

gsutil is written in Python; you can almost certainly look inside to see what it does. Although I actually think it's a wrapper around boto. — Daniel Roseman
– Daniel Roseman, Commented Nov 9, 2016 at 15:42

ivan_pozdeev · Accepted Answer · 2016-11-09 22:59:30Z

4

This is covered by Python Example | Cloud Storage Documentation | Google Cloud Platform (the 1st in Google on "google storage python wrapper"):

To download an object: storage/api/crud_object.py (View on GitHub)

def get_object(bucket, filename, out_file):
    service = create_service()

    # Use get_media instead of get to get the actual contents of the object.
    # http://g.co/dv/resources/api-libraries/documentation/storage/v1/python/latest/storage_v1.objects.html#get_media
    req = service.objects().get_media(bucket=bucket, object=filename)

    downloader = http.MediaIoBaseDownload(out_file, req)

    done = False
    while done is False:
        status, done = downloader.next_chunk()
        print("Download {}%.".format(int(status.progress() * 100)))

    return out_file

edited Nov 9, 2016 at 22:59

answered Nov 9, 2016 at 18:36

ivan_pozdeev

36.6k19 gold badges115 silver badges165 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

Alberto Bonsanto Over a year ago

By the way I was lookign at this site and they show a different approach using a json but the example doesn't work, 'cause oauth2client.client.SignedJwtAssertionCredentials and apiclient.discovery import build can't be found, maybe this approach isn't suported anymore? I want to avoid using gcloud init

ivan_pozdeev Over a year ago

@AlbertoBonsanto that link seems to be about downloading some reports rather than user files.

Alberto Bonsanto Over a year ago

Indeed I want to download those reports

Brandon Yarbrough Over a year ago

Ah, yes, you're right, looks like oauth2client changed how it does this: github.com/google/oauth2client/blob/…

Collectives™ on Stack Overflow

How to download files programatically from Google Storage via python?

1 Answer 1

4 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

4 Comments

Your Answer

Sign up or log in

Post as a guest

Related