Cloud Function to trigger a Dataflow job in Go

Question

I have a Dataflow job written using the Python SDK. But I want to trigger this Dataflow Job using Cloud Functions which should written in Go. I found a thread similar to this which has the function written in Python Trigger Dataflow with Python

My flow would be like below:

PubSub Notification to trigger Cloud Functions
Cloud Functions written in Go to start a DataFlow Job written in Python

I can make the Dataflow SDK as a template. Any pointers on how I can trigger the Python Dataflow templates from Go? Or is there a better way for it?

Many thanks!

Source code · Accepted Answer · 2023-02-09 01:07:36Z

3

check "google.golang.org/api/dataflow/v1b3" package

I don't have a way to test it, but I think it looks like a code like this

ctx := context.Background()
dataflowService, _ := dataflow.NewService(ctx)
jobService := dataflow.NewProjectsJobsService(dataflowService)
jobService.Get("project ", "job id").Do()

answered Feb 9, 2023 at 1:07

Source code

3412 silver badges4 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

Bruno Volpato Over a year ago

This looks good. More specifically for templates though: pkg.go.dev/google.golang.org/api/dataflow/… API Reference: cloud.google.com/dataflow/docs/reference/rest/v1b3/…

Ashok KS Over a year ago

Thanks for the answer. Can you please let me know how to pass the template to this?

Ashok KS Over a year ago

` dataflow = build('dataflow', 'v1b3') request = dataflow.projects().templates().launch( projectId=project, gcsPath=template, body={ 'jobName': job, 'parameters': parameters, } ) response = request.execute() ` I could find for Python. Is there something similar using Go?

Collectives™ on Stack Overflow

Cloud Function to trigger a Dataflow job in Go

1 Answer 1

3 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

3 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related