Detect changes to a nested dictionary with Python

Question

I've seen a few similar questions on SO regarding detecting changes to a dictionary and calling a function when the dictionary changes, such as:

These examples use variations of the Observer pattern or overloading __setitem__, but all these examples don't detect changes on nested dictionary values.

For example, if I have:

my_dict = {'a': {'b': 1}}
my_dict['a']['b'] = 2

The assignment of 2 to the element ['a']['b'] will not be detected.

I'm wondering if there is an elegant way of detecting changes not only to the base elements of a dictionary but all the child elements of a nested dictionary as well.

What does it mean to detect changes? How are the changes being made without your knowledge? I assume there are some broader aspects that you haven't touched on here. — roganjosh
– roganjosh, Commented Dec 21, 2018 at 9:33
my_dict['a']... is getting, not setting, an item, then ...['b'] = 2 is setting on the child. The child also has to have the custom behaviour. — jonrsharpe
– jonrsharpe, Commented Dec 21, 2018 at 9:36
If you have a look at the example links provided, you will see what I mean. I think it's pretty clear. For example, when my_dict['a']['b'] = 2 is assigned, I would like to be able to detect that the value ['a']['b'] in the nested dictionary has changed and run a specific function based on what has changed. — Willem van Ketwich
– Willem van Ketwich, Commented Dec 21, 2018 at 9:36
Well, at the moment, you just give an example using a regular dictionary. Presumably you have subclassed it — roganjosh
– roganjosh, Commented Dec 21, 2018 at 9:39
As given in the second link, why don't we recursively convert all nested dicts to custom MyDicts subclassing dict? — Deepak Saini
– Deepak Saini, Commented Dec 21, 2018 at 9:43

ma3oun · Accepted Answer · 2018-12-21 10:54:25Z

7

Building on the answer given in here, just do the following:

class MyDict(dict):
    def __setitem__(self, item, value):
        print("You are changing the value of {} to {}!!".format(item, value))
        super(MyDict, self).__setitem__(item, value)

and then:

my_dict = MyDict({'a': MyDict({'b': 1})})

my_dict['a']['b'] = 2

You are changing the value of b to 2!!

my_dict['a'] = 5

You are changing the value of a to 5!!

If you want to avoid manual calls to MyDict at each nesting level, one way of doing it, is to fully overload the dict class. For example:

class MyDict(dict):
    def __init__(self,initialDict):
        for k,v in initialDict.items():
          if isinstance(v,dict):
            initialDict[k] = MyDict(v)
        super().__init__(initialDict)

    def __setitem__(self, item, value):
        if isinstance(value,dict):
          _value = MyDict(value)
        else:
          _value = value
        print("You are changing the value of {} to {}!!".format(item, _value))
        super().__setitem__(item, _value)

You can then do the following:

# Simple initialization using a normal dict synthax
my_dict = MyDict({'a': {'b': 1}})

# update example
my_dict['c'] = {'d':{'e':4}}

You are changing the value of c to {'d': {'e': 4}}!!

my_dict['a']['b'] = 2
my_dict['c']['d']['e'] = 6

You are changing the value of b to 2!!

You are changing the value of e to 6!!

edited Dec 21, 2018 at 10:54

answered Dec 21, 2018 at 9:45

ma3oun

3,8201 gold badge24 silver badges36 bronze badges

Sign up to request clarification or add additional context in comments.

5 Comments

Corentin Limier Over a year ago

Nice one, maybe it would be interesting to convert directly the nested dicts to MyDict objects (using isinstance on the value)

Willem van Ketwich Over a year ago

thanks for your answer, ma3oun - this is one way of doing it, however what Corentin Limier describes would be a more 'elegant' solution imho.

ma3oun Over a year ago

See the edit: Solution without modifying the built dictionary. However, you would need to add the other dict constructors for a complete solution including the use of kwargs

Willem van Ketwich Over a year ago

thanks, this is the best solution so far. it would be interesting to see if we could identify the full context of the change. e.g You are changing the value of ['c']['d']['e'] to 6!!.

ma3oun Over a year ago

It can be done easily by adding a parent attribute in the constructor. You initialize it to None for the root dict and fill it with the key when setting a child dict item (you also need to set it in __init__ for all dict values.

Deepak Saini · Accepted Answer · 2018-12-21 10:17:22Z

1

Complete solution borrowing from the this link(the second one given by OP)

class MyDict(dict):
    def __setitem__(self, item, value):
        print("You are changing the value of {key} to {value}!!".format(key=item, value=value))
        super(MyDict, self).__setitem__(item, convert_to_MyDict_nested(value))

def convert_to_MyDict_nested(d):
    if not(isinstance(d, dict)):
        return d
    for k, v in d.items():
        if isinstance(v, dict):
            d[k] = convert_to_MyDict_nested(v)
    return MyDict(d)

So that if

d = {'a': {'b': 1}}

then,

d = convert_to_MyDict_nested(d)
d['a']['b'] = 2  # prints You are changing the value of b to 2!!
d['a']= 5  # prints You are changing the value of a to 5!!

Also, edited according to comment by OP. So,

d["c"] = {"e" : 7}  # prints You are changing the value of c to {'e': 7}!!
d["c"]["e"] = 9  # prints You are changing the value of e to 9!!

edited Dec 21, 2018 at 10:17

answered Dec 21, 2018 at 10:05

Deepak Saini

2,9201 gold badge22 silver badges26 bronze badges

2 Comments

Willem van Ketwich Over a year ago

Is there a way to do this internally to the class MyDict? The reason I ask is if we set d['a'] = {'b': 6} then do d['a']['b'] = 7, the second assignment will not be captured as it isn't of type MyDict.

Deepak Saini Over a year ago

yeah, we change to super(MyDict, self).__setitem__(item, convert_to_MyDict_nested(value)) so that, all dicts ever put inside the initial MyDict and MyDicts.

Lie Ryan · Accepted Answer · 2018-12-21 10:30:04Z

1

You can write an adapter class that automatically wraps values in itself, like this (untested, but I think it illustrates the point):

class DictChangeListenerAdapter:
    def __init__(self, original, listener):
        self._original = original
        self._listener = listener

    def __getitem__(self, key):
        value = self._original.__getitem__(key)
        if isinstance(value, dict):
            return DictChangeListenerAdapter(self._original[key], self._listener)
        else:
            return value

    def __setitem__(self, key, value):
        self._original.__setitem__(key, value)
        self._listener(self, key, value)

Note that this is going to cause access to the wrapped items to be much more expensive, use with care.

edited Dec 21, 2018 at 10:30

answered Dec 21, 2018 at 10:24

Lie Ryan

65.3k14 gold badges103 silver badges151 bronze badges

2 Comments

Willem van Ketwich Over a year ago

thanks for this, although printing a parent dict doesn't give a dict but the adapter object.. ? <__main__.DictChangeListenerAdapter object at 0x10dd867f0>

Lie Ryan Over a year ago

You'd need to unwrap it first before printing. Or you can implement __repr__ and forward it to the underlying dict.

Collectives™ on Stack Overflow

Detect changes to a nested dictionary with Python

3 Answers 3

5 Comments

2 Comments

2 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

5 Comments

2 Comments

2 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related