Python:error inserting json documents in mongodb collection

Question

I'm new to pymongo and mongodb and I have a json file with 157 documents that I want to insert in a collection called Students . I load the json file line by line and import each document inside a list called data . Then I try using the insert_one() method in a loop to insert each document inside the Students collection and I get many errors bellow The format of the json file is :

 //example of an entry 
{"_id":{"$oid":"5e99cb577a781a4aac69da3c"},"name":"Tanner Wilson","email":"[email protected]","yearOfBirth":{"$numberInt":"1962"},"address":[{"street":"Halsey Street","city":"Greenwich","postcode":{"$numberInt":"13832"}}]}

My code :

import pymongo
from pymongo import MongoClient
import json 

client = MongoClient('localhost:27017')

db  = client['InfoSys']
collection=db['Students']
data=[]

with open('students.json') as f:
   for json_data in f:
       element =json.loads(json_data)
       data.append(element)

'''print(data)''' // the json data can be printed perfectly 

for k in data :
   collection.insert_one(k) //this is where the error happens

And I get the errors bellow : File "C:\Users\User\Anaconda3\lib\site-packages\pymongo\collection.py", line 698, in insert_one session=session), File "C:\Users\User\Anaconda3\lib\site-packages\pymongo\collection.py", line 612, in _insert bypass_doc_val, session) File "C:\Users\User\Anaconda3\lib\site-packages\pymongo\collection.py", line 600, in _insert_one acknowledged, _insert_command, session) File "C:\Users\User\Anaconda3\lib\site-packages\pymongo\mongo_client.py", line 1491, in _retryable_write return self._retry_with_session(retryable, func, s, None) File "C:\Users\User\Anaconda3\lib\site-packages\pymongo\mongo_client.py", line 1384, in _retry_with_session return func(session, sock_info, retryable) File "C:\Users\User\Anaconda3\lib\site-packages\pymongo\collection.py", line 595, in _insert_command retryable_write=retryable_write) File "C:\Users\User\Anaconda3\lib\site-packages\pymongo\pool.py", line 618, in command self._raise_connection_failure(error) File "C:\Users\User\Anaconda3\lib\site-packages\pymongo\pool.py", line 613, in command user_fields=user_fields) File "C:\Users\User\Anaconda3\lib\site-packages\pymongo\network.py", line 129, in command codec_options, ctx=compression_ctx) File "C:\Users\User\Anaconda3\lib\site-packages\pymongo\message.py", line 707, in _op_msg flags, command, identifier, docs, check_keys, opts) bson.errors.InvalidDocument: key '$oid' must not start with '$'

I would appreciate your help with guiding me to solve this issue .

Belly Buster · Accepted Answer · 2020-05-02 15:27:16Z

1

Assuming you have data in a string format, you need to use loads() from the bson.json_util library; e.g.

from pymongo import MongoClient
import bson.json_util

client = MongoClient()

db  = client['InfoSys']
collection=db['Students']
data='{"_id":{"$oid":"5e99cb577a781a4aac69da3c"},"name":"Tanner Wilson","email":"[email protected]","yearOfBirth":{"$numberInt":"1962"},"address":[{"street":"Halsey Street","city":"Greenwich","postcode":{"$numberInt":"13832"}}]}'
collection.insert_one(bson.json_util.loads(data))
print(collection.find_one())

result:

{'_id': ObjectId('5e99cb577a781a4aac69da3c'), 'name': 'Tanner Wilson', 'email': '[email protected]', 'yearOfBirth': 1962, 'address': [{'street': 'Halsey Street', 'city': 'Greenwich', 'postcode': 13832}]}

answered May 2, 2020 at 15:27

Belly Buster

8,9142 gold badges12 silver badges25 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

Vasilis Skentos Over a year ago

I now receive errors in 2 lines . I get : ` File "C:\Users\User\Anaconda3\lib\site-packages\bson\json_util.py", line 410, in loads return json.loads(s, *args, **kwargs)` and File "C:\Users\User\Anaconda3\lib\json_init_.py", line 341, in loads raise TypeError(f'the JSON object must be str, bytes or bytearray, ' TypeError: the JSON object must be str, bytes or bytearray, not list

Belly Buster Over a year ago

My example was for a single record. Did you get that working? Once you have that working you need to iterate your list to load each record.

Vasilis Skentos Over a year ago

Yes i iterated through the list using a for loop . Specifically i wrote for k in data: collection.insert_one(bson.json_util.loads(k)) . The new errors are: in loads raise TypeError(f'the JSON object must be str, bytes or bytearray, ' TypeError: the JSON object must be str, bytes or bytearray, not dict and the second error in loads return json.loads(s, *args, **kwargs)

anton broos Over a year ago

I spend hours on this problem. Fixed it by changing json.loads() into bson.json_util.loads()

Collectives™ on Stack Overflow

Python:error inserting json documents in mongodb collection

1 Answer 1

4 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

4 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related