Python JSON parse duplicate records

Question

In my JSON file I have sequence of same-named records

"ability_upgrades":{
    "ability":5155,
    "time":1226
},
"ability_upgrades":{
    "ability":5155,
    "time":1426
},
"ability_upgrades":{
    "ability":5155,
    "time":1497
},
"ability_upgrades":{
    "ability":5157,
    "time":1543
},

If I will use json.loads it will save only last of that records.
What I have to do if I want a list of ability_upgrades?

Are you in any position to change the file format? Because this JSON is horribly broken, and broken JSON is as much of a bug as any TypeError or kernel panic. It's not clear whether you're asking about how to format your JSON properly or how to force your code to read it despite the errors. — user2357112
– user2357112, Commented May 23, 2015 at 0:21
@user2357112 Yes, I know. It is not real JSON ofc. It is output of dota2 game log. I just make it JSON-like with some regex manipulations to easy-parse. — sashaaero
– sashaaero, Commented May 23, 2015 at 0:41
Then why not generate something valid, like a list of objects instead of an object with the same key over and over, or even a stream of separate objects? — abarnert
– abarnert, Commented May 23, 2015 at 1:08

zw324 · Accepted Answer · 2015-05-23 00:29:44Z

5

The documentation says a parameter called object_pairs_hook in json.loads can be used for handling duplicated records:

>>> json.loads(s, object_pairs_hook=(lambda x:  x))
[(u'ability_upgrades', [(u'ability', 5155), (u'time', 1226)]), (u'ability_upgrades', [(u'ability', 5155), (u'time', 1426)]), (u'ability_upgrades', [(u'ability', 5155), (u'time', 1497)]), (u'ability_upgrades', [(u'ability', 5157), (u'time', 1543)])]

How that function is defined is totally depending on what you want it to be. In the example above, since that hook gets as parameters a list of pairs, the identity function simply gives the list back.

E.g., if you define:

def collectAbilityUpgrades(s):
    d = {}
    for k, v in s:
        if k not in d:
            d[k] = []
        d[k].append(v)
    return d

Then:

>>> json.loads(s, object_pairs_hook=collectAbilityUpgrades)
{u'ability_upgrades': [{u'ability': [5155], u'time': [1226]}, {u'ability': [5155], u'time': [1426]}, {u'ability': [5155], u'time': [1497]}, {u'ability': [5157], u'time': [1543]}]}

Which probably is similar to what you want.

edited May 23, 2015 at 0:29

answered May 23, 2015 at 0:22

zw324

27.2k16 gold badges88 silver badges119 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

martineau Over a year ago

Reading the fine documentation...gosh what a novel concept! Good implementation of a suitable hook, too. +1

Collectives™ on Stack Overflow

Python JSON parse duplicate records

1 Answer 1

1 Comment

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Linked

Related