Creating Pydantic Model for large nested Parent, Children complex JSON file

Question

I have a JSON file that has a number of objects that have parent, child relationships. This is what the JSON looks like at a high-level.

{
    "objectType1": {
        "objectType1ID1": {
            "parent": {},
            "children": [{"id":"objectType2ID2"}],
            "properties": {}
        },
        "objectType1ID2": {
            "parent": {},
            "children": [{}],
            "properties": {}
        }
    },
    "objectType2": {
        "objectType2ID1": {
            "parent": {},
            "children": [],
            "properties": {}
        },
        "objectType2ID2": {
            "parent": {"id":"objectType1ID1"},
            "children": [],
            "properties": {}
        }
    }
}

The first level keys contain the various types of objects and then within each type there are any number of objects of that type that contain parent, children (list), and properties keys. Each object can have at most one parent object or many children objects in a list.

I am trying to build a Pydantic model for this structure. Below is what I have so far but I am new to Pydantic and I am stuck.

from pydantic import BaseModel
from typing import List, Optional

class ObjectParent(BaseModel):
    id: Optional[str]

class ObjectChild(BaseModel):
    id: Optional[str]

class Properties(BaseModel):
    name: Optional[str]
    description: Optional[str]


class Object(BaseModel):
    id: str
    parent: Optional['ObjectParent'] = None
    children: List['ObjectChild'] = []
    properties: Properties

Can you further describe why are you stuck? Your code does not seem to parse any JSON and you are not showing us any specific error. — Hernán Alarcón
– Hernán Alarcón, Commented Dec 16, 2022 at 3:05
it seems the json is already describing a relationship, you need to parse them, preferably in a root_validator to the structure you need. you can think of pydantic is a complex storage validation tool. — Omer Ben Haim
– Omer Ben Haim, Commented Dec 19, 2022 at 7:18
Nit: Don't use Object as a classname, it may cause overriding of default behavior, perhaps use JsonObject or something more specific instead? — Yaakov Bressler
– Yaakov Bressler, Commented Jan 10, 2023 at 14:49
The dynamic keys (parent root key, ex objectType1) is a challenge. Definitely doable, but tricky. — Yaakov Bressler
– Yaakov Bressler, Commented Jan 10, 2023 at 15:00

Yaakov Bressler · Accepted Answer · 2023-01-13 17:59:42Z

I took at a stab at solving this, some advanced usage of pydantic is required.

TLDR

Let's cut to the chase -> here's the functionality you want to utilize:

from typing import Dict
from pydantic import BaseModel

class JsonObject(BaseModel):
    __root__: Dict[str, Dict[str, int]]

# Allows instantiating by parsing an object:
my_obj = JsonObject.parse_obj({
    "a": {"one": 1, "two": 2},
    "b": {"three": 3},
    "c": {}
})

>> my_obj.__root__
{'a': {'one': 1, 'two': 2}, 'b': {'three': 3}, 'c': {}}

Now let's dive into it

Let's try to model a single "root" object, at the depth of your nested structure:

"objectType1ID1": {
  "parent": {},
  "children": [
   {
    "id": "objectType2ID2"
   }
  ],
  "properties": {}
 }

The following pydantic models should work:

class AssociatedObject(BaseModel):
    """
    Represents an associated object, whether
    that is a parent or a child.
    """
    id: Optional[str]


class ObjectProperties(BaseModel):
    name: Optional[str]
    description: Optional[str]


class BaseObject(BaseModel):
    """
    Represents an object at the furthest depth
    of the nested structure
    """
    parent: AssociatedObject = AssociatedObject()
    children: List[AssociatedObject] = [AssociatedObject()]
    properties: ObjectProperties = ObjectProperties()

If we instantiate with empty values, we should get what is represented in your json object:

>> BaseObject().dict(exclude_none=True)
{'parent': {}, 'children': [{}], 'properties': {}}

Now for the hard part – how to define the parent keys?

Create a class which can store our object and its type name (objectType1ID1) as a dynamic dictionary:

from typing import Dict

class JsonObject(BaseModel):
    __root__: Dict[str, Dict[str, BaseObject]]

If we instantiate, we see it works as expected:

json_object = JsonObject(__root__=
    {
        "objectType1": {
            "objectType1ID1": BaseObject(parent=AssociatedObject(id="objectType2ID2")),
            "objectType1ID2": BaseObject()
        },
        "objectType2": {
            "objectType2ID1": BaseObject(),
            "objectType2ID2": BaseObject(parent=AssociatedObject(id="objectType1ID1"))
        }
    }
)

parent_object.dict(exclude_none=True)['__root__']

{
 "objectType1": {
  "objectType1ID1": {
   "parent": {
    "id": "objectType2ID2"
   },
   "children": [
    {}
   ],
   "properties": {}
  },
  "objectType1ID2": {
   "parent": {},
   "children": [
    {}
   ],
   "properties": {}
  }
 },
 "objectType2": {
  "objectType2ID1": {
   "parent": {},
   "children": [
    {}
   ],
   "properties": {}
  },
  "objectType2ID2": {
   "parent": {
    "id": "objectType1ID1"
   },
   "children": [
    {}
   ],
   "properties": {}
  }
 }
}

Finally: You can now parse your initial nested object successfully:

# Object provided in original Q
my_initial_object = {
    "objectType1": {
        "objectType1ID1": {
            "parent": {},
            "children": [{"id":"objectType2ID2"}],
            "properties": {}
        },
        "objectType1ID2": {
            "parent": {},
            "children": [{}],
            "properties": {}
        }
    },
    "objectType2": {
        "objectType2ID1": {
            "parent": {},
            "children": [],
            "properties": {}
        },
        "objectType2ID2": {
            "parent": {"id":"objectType1ID1"},
            "children": [],
            "properties": {}
        }
    }
}

# This works
my_json_object = JsonObject.parse_obj(my_initial_object)

Collectives™ on Stack Overflow

Creating Pydantic Model for large nested Parent, Children complex JSON file

1 Answer 1

TLDR

Now let's dive into it

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

TLDR

Now let's dive into it

Comments

Your Answer

Sign up or log in

Post as a guest

Related