Json in Python: Receive/Check duplicate key error

Question

The json module of python acts a little of the specification when having duplicate keys in a map:

import json
>>> json.loads('{"a": "First", "a": "Second"}')
{u'a': u'Second'}

I know that this behaviour is specified in the documentation:

The RFC specifies that the names within a JSON object should be unique, but does not specify how repeated names in JSON objects should be handled. By default, this module does not raise an exception; instead, it ignores all but the last name-value pair for a given name:

For my current project, I absolutely need to make sure that no duplicate keys are present in the file and receive an error/exception if this is the case? How can this be accomplished?

I'm still stuck on Python 2.7, so a solution which also works with older versions would help me most.

Amber · Accepted Answer · 2013-04-23 14:41:26Z

13

Well, you could try using the JSONDecoder class and specifying a custom object_pairs_hook, which will receive the duplicates before they would get deduped.

import json

def dupe_checking_hook(pairs):
    result = dict()
    for key,val in pairs:
        if key in result:
            raise KeyError("Duplicate key specified: %s" % key)
        result[key] = val
    return result

decoder = json.JSONDecoder(object_pairs_hook=dupe_checking_hook)

# Raises a KeyError
some_json = decoder.decode('''{"a":"hi","a":"bye"}''')

# works
some_json = decoder.decode('''{"a":"hi","b":"bye"}''')

edited Apr 23, 2013 at 14:41

answered Apr 23, 2013 at 14:35

Amber

532k89 gold badges643 silver badges558 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

theomega Over a year ago

Thanks! Are there any possibilites to generated some context on the error message? For example the name of the map (if any) which contains the error?

Amber Over a year ago

Not easily, because the hook only gets the key-value pairs. (You could dump out the entire list, which would probably give you a good guess as to what it was based on the collection of values, but also might be rather large.)

Shrikant Nadiger · Accepted Answer · 2018-09-07 08:54:35Z

0

The below code will take any json that has repeated keys and put them into an array. for example takes this json string '''{"a":"hi","a":"bye"}''' and gives {"a":['hi','bye']} as output

import json
def dupe_checking_hook(pairs):
    result = dict()
    for key,val in pairs:
        if key in result:
            if(type(result[key]) == dict):
                temp = []
                temp.append(result[key])
                temp.append(val)
                result[key] = temp
            else:
                result[key].append(val)
        else:
            result[key] = val
    return result

decoder = json.JSONDecoder(object_pairs_hook=dupe_checking_hook)
#it will not raise error
some_json = decoder.decode('''{"a":"hi","a":"bye"}''')

edited Sep 7, 2018 at 8:54

answered Sep 4, 2018 at 5:44

Shrikant Nadiger

11 bronze badge

1 Comment

Baptiste Mille-Mathias Over a year ago

Please add some explanation to your code. and how it solves the question. Thanks

Collectives™ on Stack Overflow

Json in Python: Receive/Check duplicate key error

2 Answers 2

2 Comments

1 Comment

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

2 Comments

1 Comment

Your Answer

Sign up or log in

Post as a guest

Linked

Related