remove duplicate array in multiple jsons using python [duplicate]

Question

I have multiple JSON records like

{
    "Stat": "DEN",
    "Change": [{
            "From": "",
            "To": "DEN",
            "changeTimestamp": "20200325000000"
        },
        {
            "From": "",
            "To": "DEN",
            "changeTimestamp": "20200325000000"
        },
    ],
    "Date": 20200401
}

since Changes array has duplicates need to eliminate them it should be like

{
    "Stat": "DEN",
    "Change": [{
            "From": "",
            "To": "DEN",
            "changeTimestamp": "20200325000000"
        }
    ],
    "Date": 20200401
}

since it is an array am unable to use list, I have a code like

doc['Changes'] = list(set(doc['Changes'])) if doc['Changes'] else []

It is working for items that are on the list but I came to know its not a list it's an array it won't work, can I get help for this, please

It is not a list like. "Changes" : [1,2,3,4]. It is array like above — srmu
– srmu, Commented Dec 14, 2020 at 11:49
It is not a list like. "Changes" : [1,2,3,4]. It is array like above — srmu
– srmu, Commented Dec 14, 2020 at 11:52
What do you mean by that? Perhaps you should clarify yourself with Python nomenclature. type([1, 2, 3, 4]) gives list. — norok2
– norok2, Commented Dec 14, 2020 at 11:57

mhhabib · Accepted Answer · 2020-12-14 11:23:37Z

1

You can remove duplicate keys by dictionary comprehension since the dictionary does not allow duplicate keys. I have set the standard for To because you have sent it two times at the same receiver.

{each['To']: each for each in doc['Changes']}.values()

answered Dec 14, 2020 at 11:23

mhhabib

3,1471 gold badge19 silver badges35 bronze badges

Sign up to request clarification or add additional context in comments.

8 Comments

srmu Over a year ago

If there are duplicates it need to correct to make one,if there are only single it should not get distrub....this will work right?

mhhabib Over a year ago

Yes it will work

srmu Over a year ago

Ok so it should be like doc['Changes'] ={each['To']: each for each in doc['Changes']}.values(). Am I correct?

mhhabib Over a year ago

Yes you're correct

srmu Over a year ago

yes thanks so much one final question if i have json like Changes has 2 unique values(out of three 2 are duplicates) { "Status": "CLO", "Changes": [{ "From": "", "To": "ACT", "changeTimestamp": "20200325000000" }, { "From": "", "To": "ACT", "changeTimestamp": "20200325000000" }, { "From": "DEC", "To": "ACT", "changeTimestamp": "20200325000000" } ], "Date": 20200401 }

|

Collectives™ on Stack Overflow

remove duplicate array in multiple jsons using python [duplicate]

1 Answer 1

8 Comments

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

8 Comments

Linked

Related