RowIterator Object Not Callable while using BigQuery + Pandas in Python

Question

I am working on putting my Google BigQuery data into a pandas dataframe. I am successfully able to run the below code and print the result set.

from google.cloud import bigquery
from google.oauth2 import service_account
from pandas.io import gbq

credentials = service_account.Credentials.from_service_account_file('My Project Credentials.json')
project_id = 'essential-cairn-253818'
client = bigquery.Client(credentials= credentials,project=project_id)
query = client.query("""
SELECT  device.model as model
FROM     `my-table-name`
LIMIT 100
""")

results = query.result()
for row in results:
  print("{}".format(row.model))

However, I would like to use the pandas.io.gbq.read_gbq() functionality to put this into a dataframe. I add the next rows of code and I get stuck.

query2 = """
SELECT  device.model as model
FROM     `my-table-name`
LIMIT 100
"""
results_df = gbq.read_gbq(query2, project_id=project_id, private_key='My Project Credentials.json', dialect = 'standard')

This produces the error:

TypeError: 'RowIterator' object is not callable

I'm not sure where I'm going wrong. I am following the question seen here: Live data from BigQuery into a Python DataFrame

Can anyone point me in the right direction?

Consider using the pandas-gbq package. pandas-gbq.readthedocs.io/en/latest . Your other option is loop through results (as you have done) and append each row to a list, then convert the list to a pandas df — rtenha
– rtenha, Commented Sep 24, 2019 at 20:48

Matias Hermida · Accepted Answer · 2019-09-24 20:32:35Z

0

I'm new too.

You could try pandas.read_gbq, seems a better choice. Documentation: https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.read_gbq.html

Also this other method seems less problematic: How to integrate Bigquery & pandas

answered Sep 24, 2019 at 20:32

Matias Hermida

731 gold badge1 silver badge8 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

JunkLatte · Accepted Answer · 2019-10-01 19:50:00Z

0

This was fixed with closing Spyder and restarting Python... something was going on unrelated to the code in this query. Very odd, but working now!

answered Oct 1, 2019 at 19:50

JunkLatte

411 silver badge6 bronze badges

Comments

groenhen · Accepted Answer · 2020-03-24 07:51:29Z

0

Why don't you use the official method query_job.result().to_dataframe()?

My code is:

results = query_job.result()
df = results.to_dataframe()
df.to_csv('demo1.csv',index=False, encoding='utf-8', sep = ',')

edited Mar 24, 2020 at 7:51

groenhen

3,02725 gold badges51 silver badges69 bronze badges

answered Mar 24, 2020 at 7:28

jackie zhang

1

2 Comments

Yunnosch Over a year ago

I recommend against answers in the shape of a rhetoric question, it risks being misunderstood as a question where an answer should be.

Musebe Ivan Over a year ago

query_job.to_dataframe() without the .results() part seems to give the same output as well

Collectives™ on Stack Overflow

RowIterator Object Not Callable while using BigQuery + Pandas in Python

3 Answers 3

Comments

Comments

2 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

Comments

Comments

2 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related