Upload folder to google cloud bucket with python

Question

I know that I can upload single files like this:

bucket_name = "my-bucket-name"
bucket = client.get_bucket(bucket_name)

blob_name = "myfile.txt"
blob = bucket.blob(blob_name)

blob.upload_from_filename(blob_name)

How can I do the same with a folder? Is there something like blob.upload_from_foldername? I tried the same code with replacing myfile.txt with myfoldername but it did not work.

FileNotFoundError: [Errno 2] No such file or directory: 'myfoldername'

This is the folder structure:

I assume something is wrong with the path but I am not sure what. I am executing the code from Untitled.ipynb. Works with myfile.txt but not with myfoldername.

I do not want to use a command line function.

TQA · Accepted Answer · 2021-10-29 15:09:43Z

2

You can't upload an empty folder or directory in Google Cloud Storage but you can create empty folder in Cloud Storage using the client:

from google.cloud import storage

def create_newfolder(bucket_name, destination_folder_name):
    storage_client = storage.Client()
    bucket = storage_client.get_bucket(bucket_name)
    blob = bucket.blob(destination_folder_name)

    blob.upload_from_string('')

    print('Created {} .'.format(destination_folder_name))

And if you are trying to upload a whole directory, you can use the codes below:

import glob
import os 
from google.cloud import storage

client = storage.Client()
def upload_from_directory(directory_path: str, destination_bucket_name: str, destination_blob_name: str):
    rel_paths = glob.glob(directory_path + '/**', recursive=True)
    bucket = client.get_bucket(destination_bucket_name)
    for local_file in rel_paths:
        remote_path = f'{destination_blob_name}/{"/".join(local_file.split(os.sep)[1:])}'
        if os.path.isfile(local_file):
            blob = bucket.blob(remote_path)
            blob.upload_from_filename(local_file)

edited Oct 29, 2021 at 15:09

TQA

2671 gold badge3 silver badges12 bronze badges

answered Apr 15, 2021 at 7:34

JM Gelilio

3,8061 gold badge15 silver badges25 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Exa Over a year ago

thanks! destination_blob_name would be the name I want the directory to have in the bucket?

Collectives™ on Stack Overflow

Upload folder to google cloud bucket with python

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related