17

I'm looking for MondoDB Python ODM/ORM that takes the best from two worlds: ODM/ORM & ultra fast direct dictionary read.

In other words package shall comply with following requirements:

  1. Allows to define and enforce schema.
  2. Allows to validate fields.
  3. Allows to read objects directly from mongodb (no ODM/ORM overhead).
    • Collections/objects returned directly by pymongo can be accessed using ODM/ORM layer (w/o extra queries).
    • I would imagine some kind of lazy field added by pymongo driver to objects that provides access to ORM juice (pymongo allows for such extensions).
    • Imagine use case:
      • For fast read we go directly to driver,
      • For data entry we use full ODM/ORM functionality
  4. Geofields support
  5. GridFS support of normal files and images
  6. DBRef support
  7. Does not enforce any hidden, framework specific fields
  8. Will work with Flask :)
  9. Has forms framework.
    • Forms cover sublists/subdicts
    • Backbone based forms would be just awesome
  10. Creates backbone models, collections, validators based of python definition

I know that I'm asking for much but wouldn't it be awesome to have something like this :)

In fact question could be rephrased into: "Which of existing Python Mongodb ODM/ORMs (MongoKit, MongoEngine) could be easily extended this way."

1
  • I just heard about pyodm. It could be worth a look. Commented Aug 2, 2018 at 10:16

5 Answers 5

10

1st of all I'd love to have it as well.

Based on my research

Ming is a very promising mongo ODM driver: http://merciless.sourceforge.net/

MongoEngine is too big to adapt it for your requirements/usecase. Same for MongoKit.

There is some hope in micromongo: http://packages.python.org/micromongo

Could be a good starting point.

BTW: Hmmm... I don't know what you're writing but having forms and validators covered you can easily build admin interface and end-up having next generation Django. Good luck!

Sign up to request clarification or add additional context in comments.

1 Comment

Apparently, micromongo has not moved since 2012. However, there's a new ODM coming: umongo. It is not as full-featured as required in the question, but it is meant to be simpler than MongoEngine, so you can build upon it more easily.
4

Well if you take the theory of an ORM what you need is not an ORM, since MongoDB is not a relational database, so there's no mapping to relational entities to do.

I think you must to check the answer of this post to check wether the recommendations made are what you need, I personally use MongoDB + Python directly, because of the "nature" of both Mongo and Python.

Looking around I found this library that could help you mongoEngine. Is like Django ORM. You should check it.

2 Comments

Not digging into details mongo supports references, so we have a very basic subset of relational database. For sake of correctness we can refer by ODM/ORM. Direct access to pymongo is pretty ok. I've been de eloping that way for a while. MongoEngine is too heavy. Same for MongoKit. They would be ok if only I could find a way to use them on already loaded list. BTW: packages.python.org/micromongo is not bad although still not the thing I'm looking for.
Well, when you use a ORM or ODM you know that will not be as light as using directly the db connection. So, my recommendation for you is to use directly from Python, how you said is pretty good and just works.
4

umongo was born long after that question was asked. Although relatively young, it is gaining maturity. Like micromongo, it aims at being simple (the "u" stands for µ/micro), so it does not provide the numerous features requested in the question, but it is meant to be a sane base to build upon.

It uses Marshmallow internally for model schema/validation, so you can use it happily with other libraries using Marshmallow. I use it in a Flask application that provides a REST API using Marshmallow to parse inputs/outputs. This limits the amount of duplication between DB and API schema. The application uses other libraries from the Marshmallow environment (webargs/apispec).

Since you have direct access to collections, you can use plain pymongo queries.

Also, it is compatible not only with pymongo but also with async drivers.

Comments

2

Take a look at pymongoext https://github.com/musyoka-morris/pymongoext which is highly inspired by mongoose.

Key features are:

  • No custom-query language or API to learn (If you know how to use pymongo, you already know how to use pymongoext)
  • object-like results instead of dict-like. These objects extend dict and thus you can use either syntax (i.e. foo.bar or foo['bar']).
  • custom data manipulators, to transform documents before saving and after retrieval. (Inspired by mongoose virtuals)
  • schema validation (which uses MongoDB JSON Schema validation)
  • nested and complex schema declaration
  • required fields validation
  • default values
  • custom validators
  • operator for validation (OneOf, AllOf, AnyOf, Not)
  • indexes management (Pymongoext transparently and intelligently handles the maintenance of MongoDB indexes)

Example usage:

from datetime import datetime
from pymongo import MongoClient, IndexModel
from pymongoext import *


class User(Model):
    @classmethod
    def db(cls):
        return MongoClient()['my_database_name']

    __schema__ = DictField(dict(
        email=StringField(required=True),
        name=StringField(required=True),
        yob=IntField(minimum=1900, maximum=2019)
    ))

    __indexes__ = [IndexModel('email', unique=True), 'name']

    class AgeManipulator(Manipulator):
        def transform_outgoing(self, doc, model):
            doc['age'] = datetime.now().year - doc['yob']
            return doc


# Create a user
>>> User.insert_one({'email': '[email protected]', 'name': 'Jane Doe', 'yob': 1990})

# Fetch one user
>>> user = User.find_one()

# Print the users age
>>> print(user.age)

Comments

0

Please take a look at my library - Butty - lightweight async Pydantic based ODM, it uses similar to well known ODM Beanie approach to use Pydantic as a base for documents, but different way of document linking, allowing linked documents to be plain pydantic classes instead of wraping it to Links[], allowing more support from type tooling.

import asyncio
from typing import Any

from motor.motor_asyncio import AsyncIOMotorClient

from butty import Engine, F
from butty.utility.serialid_document import SerialIDDocument


class Department(SerialIDDocument):
    name: str


class User(SerialIDDocument):
    department: Department
    name: str


async def main() -> None:
    motor: AsyncIOMotorClient[Any] = AsyncIOMotorClient("localhost")
    await motor.drop_database("butty_test")
    await Engine(motor["butty_test"], link_name_format=lambda f: f.alias + "_id").bind().init()

    it_department = await Department(name="IT").save()
    sales_department = await Department(name="Sales").save()

    vasya = await User(name="Vasya Pupkin", department=it_department).save()
    frosya = await User(name="Frosya Taburetkina", department=it_department).save()
    vova = await User(name="Vova Kastryulkin", department=sales_department).save()

    assert await User.find(F(User.department.name) == "IT", sort={User.name: 1}) == [frosya, vasya]
    assert await User.find(F(User.department.id) == sales_department.id) == [vova]
    assert await User.find(F(User.name) % "Pupkin") == [vasya]

    vasya_raw = await User.__collection__.find_one({"id": vasya.id})
    assert vasya_raw is not None
    del vasya_raw["_id"]
    assert vasya_raw == {
        "id": 1,
        "name": "Vasya Pupkin",
        "department_id": 1,
    }


asyncio.run(main())

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.