How to serialize ObjectId with jsonpickle in flask pymongo

Question

I'm using Flask with MongoDB as a database and pymongo as a driver. I have a trouble serializing object into JSON with json.dumps(), json.loads(), json_util.dumps() and many more as it always says TypeError: Object of type 'EachRoom' is not JSON serializable. I found a library called jsonpickle to serialize object into JSON. It's been done well as expected except for the ObjectId fields. It shows as a bytes type when i check the datatype. This is my code:

class EachRoom:
    def __init__(self, id, rooms):
        self.id =  id
        self.rooms = rooms

class Room(object):
    @classmethod
    def return_all_by_user(cls):
        cursor_user = mongo.db.rooms.distinct('user_id')
        users = []

        for row in cursor_user:
            print(str(row))
            cursor_room = mongo.db.rooms.find({'user_id': ObjectId(row)})

            each = EachRoom(ObjectId(row), cursor_room)
            users.append(each)
            print(cursor_room)
        print(users)
        user = jsonpickle.encode(users, unpicklable=False)
        res_user = jsonpickle.decode(user)
        print(res_user)

        if cursor_room:
            return Response(
                json_util.dumps({'message': 'success', 'user': res_user}, json_options=json_util.RELAXED_JSON_OPTIONS),
                mimetype='application/json'
            )
        else:
            return {'message': 'Document rooms is empty'}, 404

And here's the response:

{
    "message": "success",
    "user": [
        {
            "id": {
                "$binary": {
                    "base64": "Xl/WQvzhBgBTGeiE",
                    "subType": "00"
                }
            },
            "rooms": [
                {
                    "_id": {
                        "$binary": {
                            "base64": "XngkNuL9Y6AIX04v",
                            "subType": "00"
                        }
                    },
                    "created_at": "2020-03-23 09:51:34.789000",
                    "name": "Ruang Makan",
                    "updated_at": "2020-03-23 09:51:34.789000",
                    "user_id": {
                        "$binary": {
                            "base64": "Xl/WQvzhBgBTGeiE",
                            "subType": "00"
                        }
                    }
                },
                {
                    "_id": {
                        "$binary": {
                            "base64": "XngkROL9Y6AIX04w",
                            "subType": "00"
                        }
                    },
                    "created_at": "2020-03-23 09:51:48.834000",
                    "name": "Kamar Mandi",
                    "updated_at": "2020-03-23 09:51:48.834000",
                    "user_id": {
                        "$binary": {
                            "base64": "Xl/WQvzhBgBTGeiE",
                            "subType": "00"
                        }
                    }
                },
                {
                    "_id": {
                        "$binary": {
                            "base64": "XngkTOL9Y6AIX04x",
                            "subType": "00"
                        }
                    },
                    "created_at": "2020-03-23 09:51:56.220000",
                    "name": "Kamar Tidur Utama",
                    "updated_at": "2020-03-23 09:51:56.220000",
                    "user_id": {
                        "$binary": {
                            "base64": "Xl/WQvzhBgBTGeiE",
                            "subType": "00"
                        }
                    }
                }
            ]
        },
        {
            "id": {
                "$binary": {
                    "base64": "XnYgKnLe9u63KLbG",
                    "subType": "00"
                }
            },
            "rooms": [
                {
                    "_id": {
                        "$binary": {
                            "base64": "Xnq+2G5/pNeS7ynf",
                            "subType": "00"
                        }
                    },
                    "created_at": "2020-03-25 09:15:52.819000",
                    "name": "Kamar Tidur Eren",
                    "updated_at": "2020-03-25 09:15:52.819000",
                    "user_id": {
                        "$binary": {
                            "base64": "XnYgKnLe9u63KLbG",
                            "subType": "00"
                        }
                    }
                },
                {
                    "_id": {
                        "$binary": {
                            "base64": "Xnq+6W5/pNeS7yng",
                            "subType": "00"
                        }
                    },
                    "created_at": "2020-03-25 09:16:09.683000",
                    "name": "Kamar Tidur Utama",
                    "updated_at": "2020-03-25 09:16:09.683000",
                    "user_id": {
                        "$binary": {
                            "base64": "XnYgKnLe9u63KLbG",
                            "subType": "00"
                        }
                    }
                },
                {
                    "_id": {
                        "$binary": {
                            "base64": "Xnq+8G5/pNeS7ynh",
                            "subType": "00"
                        }
                    },
                    "created_at": "2020-03-25 09:16:16.262000",
                    "name": "Kamar Mandi",
                    "updated_at": "2020-03-25 09:16:16.262000",
                    "user_id": {
                        "$binary": {
                            "base64": "XnYgKnLe9u63KLbG",
                            "subType": "00"
                        }
                    }
                }
            ]
        }
    ]
}

The only thing that goes wrong is the id, user_id, and _id as the ObjectId type. Need advice please..

Check this JSON serializing Mongodb. Maybe you are interested in MongoEngine which fits better your code. MongoEngine has method to_json which serializes correctly BSON objects (ObjectId, DbRef, etc...) — Valijon
– Valijon, Commented Mar 25, 2020 at 15:51
@Valijon thanks, but according to this answer that he doesn't recommend mongoengine as it is no longer maintained, too buggy, and has a lot of unexpected behavior. Should i? — Vicky Sultan
– Vicky Sultan, Commented Mar 25, 2020 at 15:59
It's a personal opinion, no proof provided. It's one of officially recognised MongoDB driver and mantained actively by community — Valijon
– Valijon, Commented Mar 25, 2020 at 17:06

Belly Buster · Accepted Answer · 2020-03-25 17:06:44Z

1

Here's three tips in one:

To load a class object into MongoDB, use the class's built-in __dict__ property.
To turn your BSON into JSON use dumps from the bson.json_util module
Finally avoid using id as a variable; it's a python function.

Here's a small sample to try:

from pymongo import MongoClient
from bson import ObjectId
from bson.json_util import dumps

db = MongoClient()['mydatabase']

class EachRoom:
    def __init__(self, room_id, rooms):
        self.id = room_id
        self.rooms = rooms

room1 = EachRoom(ObjectId("123456789012345678901234"), '1')
db.rooms.insert_one(room1.__dict__)
print(dumps(db.rooms.find_one()))

Gives:

{"_id": {"$oid": "5e7b8d7ebf681e2e5ecb6059"}, "id": {"$oid": "123456789012345678901234"}, "rooms": "1"}

answered Mar 25, 2020 at 17:06

Belly Buster

8,9142 gold badges12 silver badges25 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Vicky Sultan Over a year ago

Wow, i was helped by your __dict__ hint, although the rest are irrelevant for my code. Thank you for the tips!

Collectives™ on Stack Overflow

How to serialize ObjectId with jsonpickle in flask pymongo

1 Answer 1

1 Comment

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Linked

Related