Google BigQuery insert using POST

Question

I'm trying to insert data to BigQuery table using POST request. My application create body to request in specified format:

--xxx
Content-Type: application/json; charset=UTF-8

{
   "configuration": {
       "load": {
           "sourceFormat": "NEWLINE_DELIMITED_JSON"
       },
       "destinationTable": {
           "projectId": "some-id",
           "datasetId": "dataset-id",
           "tableId": "cards"
       }
   }
}
--xxx
Content-Type: application/octet-stream

{"board_id":1,"version":2,"card_id":1,"title":"Tytul kartki 1"}
--xxx--

but when i send this data using:

credentials = SignedJwtAssertionCredentials(
        SERVICE_ACCOUNT_EMAIL,
        key,
        scope='https://www.googleapis.com/auth/bigquery')
self.http = credentials.authorize(httplib2.Http())
headers = {'Content-Type': 'multipart/related; boundary=xxx'}
resp, content = self.http.request(url, method="POST",
                                       body=output,
                                       headers=headers)

the response from server is:

Status: {'date': 'Thu, 25 Jul 2013 12:49:06 GMT', 'status': '400', 'content-length': '205', 'content-type': 'application/json', 'server': 'HTTP Upload Server Built on Jul 12 2013 17:12:36 (1373674356)'}
Content: {
 "error": {
  "errors": [
   {
    "domain": "global",
    "reason": "required",
    "message": "Required parameter is missing"
   }
  ],
  "code": 400,
  "message": "Required parameter is missing"
 }
}

I have no idea what parameter is missing. Only parameter which is required in documentation is sourceUris but i want to load data from request body not from GS.

I think you are missing setting the schema?

Felipe Hoffa
– Felipe Hoffa

2013-07-25 18:43:15 +00:00
Commented Jul 25, 2013 at 18:43 — Felipe Hoffa
– Felipe Hoffa, Commented Jul 25, 2013 at 18:43

RemeR · Accepted Answer · 2013-07-25 19:24:04Z

1

I think that the problem is you are missing the schema configuration:

'configuration': {
  'load': {
    'sourceFormat': <required for JSON files>',
    'schema': {
      'fields': [
        {'name':'f1', 'type':'STRING'},
        {'name':'f2', type:'INTEGER'}
      ]
    },
    'destinationTable': {
      'projectId': 'projectId',
      'datasetId': 'datasetId',
      'tableId': 'tableId'
    }
  }
}

This link maybe help you: https://developers.google.com/bigquery/loading-data-into-bigquery

answered Jul 25, 2013 at 19:24

RemeR

6578 silver badges21 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Lubosz Kośnik Over a year ago

I didn't wrote the schema because it's optional parameter when we use data in JSON format specification

JKWA · Accepted Answer · 2013-08-07 05:41:41Z

0

Check out the example on their loading data page. Note the project id is located in two places, both jobData and destinationTable. This might be the cause of the missing parameter error.

answered Aug 7, 2013 at 5:41

JKWA

3232 silver badges9 bronze badges

Comments

vytauts · Accepted Answer · 2014-11-10 10:07:14Z

0

The problem is that "destinationTable" must be inside "load" object:

{
   "configuration": {
       "load": {
           "sourceFormat": "NEWLINE_DELIMITED_JSON"
           "destinationTable": {
               "projectId": "some-id",
               "datasetId": "dataset-id",
               "tableId": "cards"
           }
       }
   }
}

It can be seen in Loading Data Into BigQuery page.

answered Nov 10, 2014 at 10:07

vytauts

11 bronze badge

Collectives™ on Stack Overflow

Google BigQuery insert using POST

3 Answers 3

1 Comment

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

1 Comment

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related