Export query results from BigQuery to Postgres

Question

I am trying to export the results of a query in BigQuery, and get the data into Postgres. The data may be as much 250 million record, ~26Gb.

Option 1:

Save query results to a temp table Export table to csv(s) Bulk upsert to postgres

(This will be slow)

Option 2:

Somehow get the two DBs to speak directly I don't know if this is possible

Thank you for any information!!!

David · Accepted Answer · 2018-10-10 19:16:36Z

5

This BigQuery Foreign Data Wrapper for postgreSQL allows you to query BigQuery directly from within PostgreSQL. Using this and a SELECT INTO statement should allow you to efficiently and directly copy from BigQuery to PostgreSQL.

answered Oct 10, 2018 at 19:16

David

9,7811 gold badge22 silver badges58 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Eli Byers Over a year ago

Thank you for the recommendation! I will mark this as the answer when I get it working.

Felipe Hoffa Over a year ago

This might work for queries that reduce data from the original 26GB to an understandable result - but for 26GB of data I'll recommend to export to CSV and re-import.

Collectives™ on Stack Overflow

Export query results from BigQuery to Postgres

1 Answer 1

2 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related