How to read json file in cloud storage using cloud functions - python

Question

The need is to read the json file data present in GCS bucket using cloud functions (python). I have uploaded the required json data to bucket and try running the cloud function.

The input json file data :

{"count":15,"data":[{"P_ID":21.0,"P_NAME":"MMME","TZ":"PST","DATE_MODIFIED":"2005-10-14 00:00:00"}]}

here is the code:

    from google.cloud import storage
    import base64
    import json
    import os
    
def hello_gcs(event, context):
    """
    Triggered by a change to a Cloud Storage bucket.
    Args:
         event (dict): Event payload.
         context (google.cloud.functions.Context): Metadata for the event.
    """
    print("in CF")  
    print("event", event)

    file_name = event['name']
    bucket_name = event['bucket']
    client = storage.Client()
    bucket = client.get_bucket(bucket_name)
    file_blob = storage.Blob(file_name, bucket)
    download_data = file_blob.download_as_string().decode()
    jsondata = {}
    jsondata = download_data
    print("download_data : ", download_data)
    print("jsondata := ", jsondata)
    print(jsondata['count'])

Please correct me. I am not getting the data displayed in the code. this is just to test the cloud function once this is working I need to implement additional features.

Sekhar · Accepted Answer · 2022-04-19 10:45:40Z

here is the working code. Converted the download file to json object and accessed the fields.

from google.cloud import storage
import base64
import json
import os

def hello_gcs(event, context):
    """
    Triggered by a change to a Cloud Storage bucket.
    Args:
         event (dict): Event payload.
         context (google.cloud.functions.Context): Metadata for the event.
    """
    print("in CF")  
    print("event", event)

    file_name = event['name']
    bucket_name = event['bucket']
    client = storage.Client()
    bucket = client.get_bucket(bucket_name)
    file_blob = storage.Blob(file_name, bucket)
    download_data = file_blob.download_as_string().decode()
    print("download_data : ", download_data)
    """
    jsondata = {}
    jsondata = download_data
    """
    jsondata = {}
    #convert string to  object
    jsondata = json.loads(download_data)
    print("jsondata => ", jsondata)
    print(jsondata['count'])

guillaume blaquiere · Accepted Answer · 2022-04-18 19:20:58Z

0

The function that you use if triggered on an event, when an object is written to Cloud Storage for instance. But you only have the event, not the content. With the event data, you have all that you need for downloading the file and to process it.

answered Apr 18, 2022 at 19:20

guillaume blaquiere

76.6k3 gold badges65 silver badges102 bronze badges

2 Comments

Sekhar Over a year ago

Hi, as you said the trigger is based on the file creation in the storage bucket. In my code I am downloading it but not able to see the data (on print). can you please suggest if my code is right or anything to be added.

guillaume blaquiere Over a year ago

Why do you decode your string? Did you store the content in b64? If not, remove the decode and print the data. It should work.

Collectives™ on Stack Overflow

How to read json file in cloud storage using cloud functions - python

2 Answers 2

Comments

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related