Fill missing keys by comparing example json in python

Question

I want merge two dictionaries first over second, meaning the first dictionary doesn't lose its previous values and only the missing key values are added into first.

Original =  {key1: { key2:[{ key3: y1 , key5: z1 , key6 [{key7: n1}]}]}}

Example = {key1: { key2:[{ key3: None , key4: None ,key5: None , key6 [{key7: None,key8:None}]}]}}

def update2(final, first):
 for k, v in first.items():
    if isinstance(v, str):
        final[k] = v
    elif k in final:
        if isinstance(final[k], dict):
            final[k].update2(first[k])
        else:
            final[k] = first[k]
    else:
        final[k] = first[k]

    final_json = update2(Example['key1'], Original['key1'])
    print(final_json)

#returns only Original not appended output

    def update3(right,left):
        d = dict()
        for key,val in left.items():
         d[key]=[val]
        for key,val in right.items():
         if key not in d.keys():
          d[key].append(val)
         else:
          d[key]=[val]

    final_json = update3(Example['key1'], Original['key1'])
    print(final_json) #returns None

Expected output:

{key1: { key2:[{ key3: y1 , key4: None ,key5: z1 , key6 [{key7: n1,key8:None}]}]}}

I have referred many stackoverlfow posts but nothing is working since it has to be iterated at multiple levels.

Set default values according to JSON schema automatically

how to merge two dictionaries based on key but include missing key values in python?

Merge two dictionaries and persist the values of first dictionaries

My goal is to add default values for missing keys from example file.I am beginner to Python.

Could you use valid python syntax?

Olvin Roght
– Olvin Roght

2022-06-22 09:17:38 +00:00
Commented Jun 22, 2022 at 9:17 — Olvin Roght
– Olvin Roght, Commented Jun 22, 2022 at 9:17

maya · Accepted Answer · 2022-06-22 09:44:09Z

1

Try processing it recursively, in the following steps.

determine if the key is present, if not, assign the value directly, if so go to the next step
determine the type of the value

Dictionary: recursive call
List: iterate through the contents and make a recursive call for each item
Other: assign directly if the original dictionary is not None

like this:

def update(orignal, addition):
    for k, v in addition.items():
        if k not in orignal:
            orignal[k] = v
        else:
            if isinstance(v, dict):
                update(orignal[k], v)
            elif isinstance(v, list):
                for i in range(len(v)):
                    update(orignal[k][i], v[i])
            else:
                if not orignal[k]:
                    orignal[k] = v




Original =  {
    "key1": {
        "key2":[
            { "key3": "y1" ,
              "key5": "z1" ,
              "key6": [
                {"key7": "n1"}
              ]
            }
        ]
    }
}

Example = {
    "key1": {
        "key2":[
            {
                "key3": None ,
                "key4": None ,
                "key5": None ,
                "key6": [
                    {"key7": None,"key8":None}
                ]
            }
        ]
    }
}
print("Original: ", Original)
print("Addition: ", Example)
update(Original, Example)
print("Merge: ", Original)

# Original:  {'key1': {'key2': [{'key3': 'y1', 'key5': 'z1', 'key6': [{'key7': 'n1'}]}]}}
# Addition:  {'key1': {'key2': [{'key3': None, 'key4': None, 'key5': None, 'key6': [{'key7': None, 'key8': None}]}]}}
# Merge:  {'key1': {'key2': [{'key3': 'y1', 'key5': 'z1', 'key6': [{'key7': 'n1', 'key8': None}], 'key4': None}]}}

answered Jun 22, 2022 at 9:44

maya

1,0901 gold badge5 silver badges9 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

learner Over a year ago

Thanks for the quick answer.I have tried this but I got error AttributeError: 'str' object has no attribute 'items' on for k, v in addition.items(): So I have parsed with json.loads and tried but got TypeError: the JSON object must be str, bytes or bytearray, not dict

learner Over a year ago

Somehow I managed to get expected output but it is not working if the json has more than 1 record. It adds missing columns only in one record and remaining records are not added with missing keys. Original: {'key1': {'key2': [{'key3': 'y1', 'key5': 'z1', 'key6': [{'key7': 'n1'}]}, {'key3': 'y1', 'key5': 'z1', 'key6': [{'key7': 'n1'}]}]}} Merge: {'key1': {'key2': [{'key3': 'y1', 'key5': 'z1', 'key6': [{'key7': 'n1', 'key8': None}], 'key4': None}, {'key3': 'y1', 'key5': 'z1', 'key6': [{'key7': 'n1'}]}]}}# in second part key4 and key 8 are not added

maya Over a year ago

I tried testing with your example parameters and the results were fine (as I expected them to be), the problem may be with the following： 1. make sure that each parameter is of the correct type, load dictionary, list data structure 2. for keys in the dictionary, e.g. the following format: {"key1": [{"k": 2}, {"v": 3}]}, if there is more than one in the list, how to match the appended pattern you have not clarified, suggest posting your test case or the expected result

learner Over a year ago

Its because of second problem you have mentioned. Expected Output:{'key1': {'key2': [{'key3': 'y1', key4='None','key5': 'z1', 'key6': [{'key7': 'n1', 'key8': None}]}, {'key3': 'y2',key4=None, 'key5': 'z2', 'key6': [{'key7': 'n2',key8='None'}]}]}} From Key3 till Key7,the same dictionary is repeating so I need to add missing keys in this dictionary as well. Actual Output:{'key1': {'key2': [{'key3': 'y1', 'key5': 'z1', 'key6': [{'key7': 'n1', 'key8': None}], 'key4': None}, {'key3': 'y1', 'key5': 'z1', 'key6': [{'key7': 'n1'}]}]}}

Alexandre Novius · Accepted Answer · 2022-06-22 10:11:38Z

0

I assumed that there was no list and that it was not desired that a key present in Original but not present in Example be added to Example. I'm not sure this is the answer that you are looking for, but here is a proposal.

def f(base: dict, data: dict):
    keys = base.keys() & data.keys()
    for key in keys:
        if isinstance(base[key], dict):
            f(base[key], data[key])            
        elif base[key] is None:
            base[key] = data[key]

python

>>> data = {"key1": {"key2": {"key3": "y1", "key5": "z1", "key6": {"key7": "n1"}}}}
>>> base = {"key1": {"key2": {"key3": "a1", "key4": None, "key5": None, "key6": {"key7": None, "key8": None}}}}
>>> f(base, data)
>>> print(base)
{'key1': {'key2': {'key3': 'a1', 'key4': None, 'key5': 'z1', 'key6': {'key7': 'n1', 'key8': None}}}}

edited Jun 22, 2022 at 10:11

answered Jun 22, 2022 at 10:06

Alexandre Novius

1828 bronze badges

1 Comment

learner Over a year ago

a key present in Example but not present in Original be added to Original. But anyway this will not work this I had list inside the json and it just check first key 1,since it present in both it gives Original as it is before not the updated one

Collectives™ on Stack Overflow

Fill missing keys by comparing example json in python

2 Answers 2

4 Comments

1 Comment

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

4 Comments

1 Comment

Your Answer

Sign up or log in

Post as a guest

Linked

Related