Python: Use nested dict to create class of nested dataclass

Question

In the basic case, one can easily map a dictionary to the parameters. The below shows the basic example.

def func1(x: int, y: int):
    return x+y

input = {
    "x": 1,
    "y": 2,
}

## This Works
sum = func1(**input)
# sum = 3

Does Python provide any other types of shortcuts which would enable this type of behavior for nested classes?

from dataclasses import dataclass

@dataclass
class X:
  x: int


@dataclass
class Y:
  y: int


def func2(x: X, y: Y):
    return x.x + y.y


input_2 = {
    "X": {
        "x": 1,
    },
    "Y": {
        "y": 1,
    },
}

sum = func2(**input_2)
# TypeError: func2() got an unexpected keyword argument 'X'

I have tried other approach's. This is an example fo something that works, but is not very general.

sum = func2(X(input_2[X][x]),Y(input_2[Y][y])

Also failed with pydantic

from pydantic import BaseModel

class X(BaseModel):
  x: int


class Y(BaseModel):
  y: int


def func2(x: X, y: Y):
    return x.x + y.y


input_2 = {
    "X": {
        "x": 1,
    },
    "Y": {
        "y": 1,
    },
}
sum = func2(**input_2)

What about something like this? geeksforgeeks.org/how-to-change-a-dictionary-into-a-class — dekuShrub
– dekuShrub, Commented Mar 7, 2022 at 23:46
You get the TypeError because the keys of the dictionary must map to the names of the arguments of the function. Since you have a key "X", python expects an argument named X. Instead you have one named x with type X. I don't know of any way to map the elements of the dictionary to arguments based on the type annotation. — Code-Apprentice
– Code-Apprentice, Commented Mar 8, 2022 at 0:39
The problem with this is that it assumes every argument has a different type. That is usually a very bad assumption. — user2357112
– user2357112, Commented Mar 8, 2022 at 0:47

Wizard.Ritvik · Accepted Answer · 2022-03-08 17:10:41Z

You can use a decorator to convert each dict argument for a function parameter to its annotated type, assuming the type is a dataclass or a BaseModel in this case.

An example with the dataclass-wizard - which should also support a nested dataclass model:

import functools

from dataclasses import dataclass, is_dataclass
from dataclass_wizard import fromdict


def transform_dict_to_obj(f):
    name_to_tp = {name: tp for name, tp in f.__annotations__.items()
                  if is_dataclass(tp)}

    @functools.wraps(f)
    def new_func(**kwargs):
        for name, tp in name_to_tp.items():
            if name in kwargs:
                kwargs[name] = fromdict(tp, kwargs[name])

        return f(**kwargs)

    return new_func


@dataclass
class X:
  x: int


@dataclass
class Y:
  y: int


@transform_dict_to_obj
def func2(*, x: X, y: Y) -> str:
    return x.x + y.y


input_2 = {
    "x": {
        "x": 1,
    },
    "y": {
        "y": 1,
    },
}

sum = func2(**input_2)

print('Sum:', sum)
assert sum == 2  # OK

Similarly, with pydantic:

import functools

from pydantic import BaseModel

class X(BaseModel):
  x: int


class Y(BaseModel):
  y: int


def transform_dict_to_obj(f):
    name_to_from_dict = {name: tp.parse_obj
                         for name, tp in f.__annotations__.items()
                         if issubclass(tp, BaseModel)}

    @functools.wraps(f)
    def new_func(**kwargs):
        for name, from_dict in name_to_from_dict.items():
            if name in kwargs:
                kwargs[name] = from_dict(kwargs[name])

        return f(**kwargs)

    return new_func


@transform_dict_to_obj
def func2(*, x: X, y: Y) -> str:
    return x.x + y.y


input_2 = {
    "x": {
        "x": 1,
    },
    "y": {
        "y": 1,
    },
}

sum = func2(**input_2)

print('Sum:', sum)
assert sum == 2  # OK

For a slightly more optimized version, instead of using a for loop in the decorator each time, you can only add the logic you need to run, and then generate the new function using dataclasses._create_fn() or similar:

from dataclasses import dataclass, is_dataclass, _create_fn
from dataclass_wizard import fromdict


def transform_dict_to_obj_optimized(f):
    args = []
    body_lines = []
    return_type = f.__annotations__.pop('return', None)

    for name, tp in f.__annotations__.items():
        type_name = tp.__qualname__
        args.append(name)

        if is_dataclass(tp):
            body_lines.append(f'if {name}:')
            body_lines.append(f' {name} = fromdict({type_name}, {name})')

    body_lines.append(f'return original_fn({",".join(args)})')

    return _create_fn(f.__name__, args, body_lines,
                      return_type=return_type,
                      locals={'original_fn': f},
                      globals=globals())


@dataclass
class X:
  x: int


@dataclass
class Y:
  y: int


@transform_dict_to_obj_optimized
def func2(x: X, y: Y) -> int:
    return x.x + y.y


input_2 = {
    "x": {
        "x": 1,
    },
    "y": {
        "y": 1,
    },
}

sum = func2(**input_2)

print('Sum:', sum)
assert sum == 2  # OK

Toan T · Accepted Answer · 2022-03-08 00:45:43Z

1

I think create a new class that includes X and Y, assume C can work for your case

from pydantic import BaseModel

class X(BaseModel):
  x: int


class Y(BaseModel):
  y: int


class C(X, Y):
    pass

def func2(c: C):
    x = c.x
    y = c.y
    return x + y


input_2 = C(**{
    "x": 1,
    "y": 1,
})
sum = func2(input_2)
print(sum)

answered Mar 8, 2022 at 0:45

Toan T

5771 gold badge5 silver badges18 bronze badges

Comments

Dan Ciborowski - MSFT · Accepted Answer · 2022-03-08 04:05:03Z

0

@dataclass
class Math:
"""Collection of Configurations and Data Loading Utilities for PlayFab Churn Featurization"""
    x: X
    y: Y

    @classmethod
    def load(cls, config_json):
        return Math(
            x=X(**config_json['x']),
            y=Y(**config_json['y']),
        )

I want around the constructor and gave myself a different way out. I still get the benefits all of my nested dataclasses, maintain backwards compatibility with my old constructor, and still do not need a init method.

edited Mar 8, 2022 at 4:05

answered Mar 8, 2022 at 0:26

Dan Ciborowski - MSFT

7,31710 gold badges61 silver badges90 bronze badges

1 Comment

Code-Apprentice Over a year ago

Is this answering your own question? I don't understand the connection since you attempted answer shows completely different classes and functions than you rquestion.

Collectives™ on Stack Overflow

Python: Use nested dict to create class of nested dataclass

3 Answers 3

Comments

Comments

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

Comments

Comments

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related