BigQuery: Error 500 on streaming insert via Java API

Question

I'm trying to insert a row using this example code:

https://developers.google.com/bigquery/streaming-data-into-bigquery#streaminginsertexamples

TableRow row = new TableRow();
row.set("Col1", 50);
row.set("Col2", 50);
TableDataInsertAllRequest.Rows rows = new TableDataInsertAllRequest.Rows();
rows.setInsertId("" + System.currentTimeMillis());
rows.setJson(row);
List rowList = new ArrayList();
rowList.add(rows);
TableDataInsertAllRequest content = new TableDataInsertAllRequest().setRows(rowList);
try
{
  TableDataInsertAllResponse response =
      bigquery
      .tabledata()
      .insertAll("<myprojectid>", "6bc2cf53x8684x42a4x8149x9ff20fa8beed", "TestTable", content)
      .execute();

  System.out.println("Response: " + response);
}
catch (IOException e)
{
  // handle
}

Schema is this:

Col1 INTEGER NULLABLE
Col2 INTEGER NULLABLE

I'm receiving an error:

com.google.api.client.googleapis.json.GoogleJsonResponseException: 500
{
  "code" : 500,
  "errors" : [ {
    "domain" : "global",
    "message" : "Unexpected. Please try again.",
    "reason" : "internalError"
  } ],
  "message" : "Unexpected. Please try again."
}

Project exists, dataset exists, table exists, data types match the column spec, and I'm using the example code. Jordan said on this question [1] that 500 means a BigQuery bug. I don't mind working around it (if that's it) but I have no idea how to. What can I try? I've hardcoded simple values for each column, I've tried a different table, I've tried a different, brand-new project/dataset/table set. Same result.

This is both on the dev or GAE environment.

If it helps:

Project number: 1066943214796 (note I'm not using that, I'm using the string id in the call), datasetid and table now in the code above.

[1] HTTP error 500 when requesting google big query API using service account

Can you paste your code? I copy pasted the sample code, filled in my values, everything is working as expected. I want to check what might be different. — Felipe Hoffa
– Felipe Hoffa, Commented Oct 9, 2013 at 2:35

Felipe Hoffa · Accepted Answer · 2013-10-09 15:02:59Z

3

+50

I was able to reproduce and identify the problem!

I get the exact same error, unless I rename the dataset. After I changed "6bc2cf53x8684x42a4x8149x9ff20fa8beed" to an existing, simpler string, it all worked.

Can you try a different dataset name? I'll file a bug to discover why this is happening in the meantime.

answered Oct 9, 2013 at 15:02

Felipe Hoffa

59.8k23 gold badges185 silver badges363 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

Richard Watson Over a year ago

You got it! Thank you so much - it's been bugging me tremendously and it's such a relief to find there's a reason. Of course it was the one variable I didn't think to change. The original is derived from a guid and I think I tried at least two different ones.

Felipe Hoffa Over a year ago

Happy to help! I'm glad you chose not to anonymize the dataset name :)

Richard Watson Over a year ago

Btw, I think it's dataset or table id's that start with a number. I prefixed a letter and the same id works, for both table and dataset id's.

Siddartha Naidu · Accepted Answer · 2013-10-02 19:59:59Z

3

Have you deleted and re-created the tables you have been trying to insert data into?

We (BigQuery team) have been investigating the error and believe it is caused by cache inconsistency issues in the system. The cache can end up in this state if:

1) a table is created 2) some rows are streamed to the table 3) table is deleted and recreated 4) more rows are streamed to the table -- internalError

It would be useful to know if this matches the sequence of operations you were attempting.

The cache itself invalidates itself in ~30min at which point the operations should succeed. We are working on a fix but a temporary work around is to avoid reusing a table name.

When the issue is fixed I will update this question.

answered Oct 2, 2013 at 19:59

Siddartha Naidu

861 bronze badge

5 Comments

Richard Watson Over a year ago

I don't think it's that - I haven't managed to stream anything to it yet. I did upload a CSV file to check if there was anything specific to the table that prevented upload.

Richard Watson Over a year ago

I also tried two other tables just in case and neither worked, so it must be something I'm doing to trigger the alleged bug. Both the dataset and table were created programatically via the Java API.

Richard Watson Over a year ago

I've created a new project, new dataset, new table, still gives the same error. Bounty added - I'd really like this fixed.

Felipe Hoffa Over a year ago

Please paste your code! I copy pasted from the link you gave, and it worked.

Nicholas Mayne Over a year ago

I have just run in to this, was it ever fixed? I dont get an error, just missing data, but the process is exactly how you mention. but with the C# sdk

Collectives™ on Stack Overflow

BigQuery: Error 500 on streaming insert via Java API

2 Answers 2

3 Comments

5 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

3 Comments

5 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related