removing json items from array if value is duplicate python

Question

I am incredibly new to python.

I have an array full of json objects. Some of the json objects contain duplicated values. The array looks like this:

 [{"id":"1","name":"Paul","age":"21"},
  {"id":"2","name":"Peter","age":"22"},
  {"id":"3","name":"Paul","age":"23"}]

What I am trying to do is to remove an item if the name is the same as another json object, and leave the first one in the array.

So in this case I should be left with

  [{"id":"1"."name":"Paul","age":"21"},
  {"id":"2","name":"Peter","age":"22"}]

The code I currently have can be seen below and is largely based on this answer:

import json
ds = json.loads('python.json') #this file contains the json
unique_stuff = { each['name'] : each for each in ds }.values()

all_ids = [ each['name'] for each in ds ]
unique_stuff = [ ds[ all_ids.index(text) ] for text in set(texts) ]

print unique_stuff

I am not even sure that this line is working ds = json.loads('python.json') #this file contains the json as when I try and print ds nothing shows up in the console.

Please show us your non-working code so we can help you fix it. — PM 2Ring
– PM 2Ring, Commented Mar 28, 2016 at 7:57

Abhijit · Accepted Answer · 2016-03-28 08:05:57Z

4

You might have overdone in your approach. I might tend to rewrite the list as a dictionary with "name" as a key and then fetch the values

ds = [{"id":"1","name":"Paul","age":"21"},
  {"id":"2","name":"Peter","age":"22"},
  {"id":"3","name":"Paul","age":"23"}]

{elem["name"]:elem for elem in ds}.values()
Out[2]: 
[{'age': '23', 'id': '3', 'name': 'Paul'},
 {'age': '22', 'id': '2', 'name': 'Peter'}]

Off-course the items within the dictionary and the list may not be ordered, but I do not see much of a concern. If it is, let us know and we can think over it.

answered Mar 28, 2016 at 8:05

Abhijit

64k20 gold badges143 silver badges209 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

Paul Fitzgerald Over a year ago

thanks I will try that now. I am also trying to figure out how to get the json data into the python file I am writing my code in

Abhijit Over a year ago

@PaulFitzgerald: I thought you already got that working? ds = json.loads('python.json')

Paul Fitzgerald Over a year ago

I did too, but i dont think it is working as when i do print ds nothing shows in the terminal when I run the program, so I dont think its working. Can you see anything wrong with this code? or is it a python thing that wont show in the terminal?

PM 2Ring Over a year ago

@PaulFitzgerald: json.loads expects its string argument to contain JSON data, not a file name. See pkacprzak's answer for the correct way to load JSON from a file.

gboffi · Accepted Answer · 2016-03-28 08:11:23Z

3

If you need to keep the first instance of "Paul" in your data a dictionary comprehension gives you the opposite result.

A simple solution could be as following

new = []
seen = set()
for record in old:
    name = record['name']
    if name not in seen:
        seen.add(name)
        new.append(record)
del seen

answered Mar 28, 2016 at 8:11

gboffi

25.4k10 gold badges62 silver badges98 bronze badges

1 Comment

Paul Fitzgerald Over a year ago

Thanks, nice solution

pkacprzak · Accepted Answer · 2016-03-28 08:11:36Z

1

First of all, your json snippet has invalid format - there are dot instead of commas separating some keys.

You can solve your problem using a dictionary with names as keys:

import json

with open('python.json') as fp:
    ds = json.load(fp) #this file contains the json

    mem = {}

    for record in ds:
        name = record["name"]
        if name not in mem:
            mem[name] = record

    print mem.values()

answered Mar 28, 2016 at 8:11

pkacprzak

5,6131 gold badge20 silver badges37 bronze badges

Collectives™ on Stack Overflow

removing json items from array if value is duplicate python

3 Answers 3

4 Comments

1 Comment

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

4 Comments

1 Comment

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related