python remove duplicate element from list

Question

In python i am getting a list in a variable like :

[ {'store_id': '321', 'first_name': 'A', 'name': 'A'}, 
{'second_id': '322', 'first_name': 'B', 'name': 'B', },
{'second_id': '323', 'second_name': 'c', 'name': 'c', },
{'second_id': '324', 'second_name': 'A', 'name': 'A', },
]

what i actually want is i want a list without duplicating the name . if it occur once then i want to remove t and create a new list with distinct data.i am stuck here i want all data in new list. how can i remove the duplicate data from the list .

in my case i want a ne list like :

Either

{'second_id': '322', 'first_name': 'B', 'name': 'B', },
{'second_id': '323', 'second_name': 'c', 'name': 'c', },
{'second_id': '324', 'second_name': 'A', 'name': 'A', },
]

Or

[ {'store_id': '321', 'first_name': 'A', 'name': 'A'}, 
{'second_id': '322', 'first_name': 'B', 'name': 'B', },
{'second_id': '323', 'second_name': 'c', 'name': 'c', },

]

And the code after that i am getting this is given below:

result = {}
    data = request.POST
    teamName = []

    First = Test.objects.filter(d=data.get('id')).values(
        'first_id','first_name').annotate(id=F('first_id'),name=F('first_name')).distinct()
    Second = Test.objects.filter(id=data.get('id')).values(
        'second_id','second_name').annotate(id=F('second_id'),name=F('second_name')).distinct()
    combined_results = list(chain(First, Second))


    for team in combined_results:
        team['text'] = team['name']
        team['id'] = team['id']
        teamName.append(team)

    if not combined_results:
        result['status'] = False
        result['data'] = ['Data not found']
    else:
        result['status'] = True
        result['data'] = teamName

    return JsonResponse(result)

@chess_lover_6 everythin is fine i am merging two queryset in a single list thats why i am getting this list that is fine . the thing is that i aslo want to remove duplicate through the loop — Pankhuri
– Pankhuri, Commented Mar 10, 2021 at 3:53
Create an empty set (1) and an empty list (2). Traverse the original list. For each element, check whether the name is in set (1). If it is, continue to the next element. If it isn’t, append the element to list (2) and add the name to the set (1). — crcvd
– crcvd, Commented Mar 10, 2021 at 3:55
Does this answer your question? removing json items from array if value is duplicate python — Skylar Liang
– Skylar Liang, Commented Mar 10, 2021 at 3:58
@Pankhuri instead of doing all this normalize your database schema. You will have to keep making weird workarounds unless you normalize your database schema. — Abdul Aziz Barkat
– Abdul Aziz Barkat, Commented Mar 10, 2021 at 4:15

Shiva · Accepted Answer · 2021-03-10 04:05:04Z

3

This should give you the second form

names = set()
newList = []
for d in mylist:
    if d['name'] in names:
        continue
    else:
        newList.append(d)
        names.add(d['name'])

print(newList)

Output:

[{'store_id': '321', 'first_name': 'A', 'name': 'A'},
 {'second_id': '322', 'first_name': 'B', 'name': 'B'},
 {'second_id': '323', 'second_name': 'c', 'name': 'c'}]

EDIT:
If you want the first form, you will have to sort your original list in descending order of store_id/second_id using:

mylist = sorted(mylist, key=lambda x: x.get('store_id') or x.get('second_id'), reverse=True)

and then filter the list as earlier.

edited Mar 10, 2021 at 4:05

answered Mar 10, 2021 at 3:55

Shiva

2,86826 silver badges40 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

Pankhuri Over a year ago

No its not working returning the same result

Shiva Over a year ago

Can you show your code? May be you are printing or using the original list instead of the new one?

Pankhuri Over a year ago

i have updated the question please see above in the question what actually i amd doing ..

JacksonPro · Accepted Answer · 2021-03-10 04:02:14Z

0

using list comprehension:

lst = [ {'store_id': '321', 'first_name': 'A', 'name': 'A'}, 
{'second_id': '322', 'first_name': 'B', 'name': 'B', },
{'second_id': '323', 'second_name': 'c', 'name': 'c', },
{'second_id': '324', 'second_name': 'A', 'name': 'A', },
]

uniqueLst = []
lst2 = []
[[lst2.append(x), uniqueLst.append(x['name'])] for x in lst if x['name'] not in uniqueLst]

print(lst2)

answered Mar 10, 2021 at 4:02

JacksonPro

3,2752 gold badges12 silver badges34 bronze badges

1 Comment

Pankhuri Over a year ago

i have updated the question please see above in the question what actually i amd doing ..

Usman Ghani Mughal · Accepted Answer · 2021-03-10 04:34:36Z

I have tried to solve your issue, It may not be the best approach but it does the job.

lets suppose you have this list

orignal_list = [
        {'store_id': '321', 'first_name': 'A', 'name': 'A'},
        {'second_id': '322', 'first_name': 'B', 'name': 'B', },
        {'second_id': '323', 'second_name': 'c', 'name': 'c', },
        {'second_id': '324', 'second_name': 'A', 'name': 'A', },
      ]

so we can make a function which can take original list and return a filtered list.

def filter_list(original_list):
    # temp list which will contains the filtered records 
    temp_list = list()

    # loop over original_list
    for original_item in original_list:
        # get values of dict expect id because every element have different id
        original_values = [*original_item.values()][1:]
        flag = True
        # loop over temp_list
        for temp_items in temp_list:
            # get values expect the id
            temp_values = [*temp_items.values()][1:]
            # check if sample/record/item already exists
            if original_values == temp_values:
                flag = False

        if flag:
            temp_list.append(original_item)

    return temp_list

I hope this will solve your problem I have tried to explain the code using comments, let me know if you did not understand any thing.

Chr_8580 · Accepted Answer · 2021-09-27 01:07:46Z

0

I did something similar with sets. Since A is a set of elements:

for i in A:
    if i not in B:
        B.append(i)

edited Sep 27, 2021 at 1:07

answered Sep 27, 2021 at 0:32

Chr_8580

815 bronze badges

1 Comment

Community Over a year ago

Your answer could be improved with additional supporting information. Please edit to add further details, such as citations or documentation, so that others can confirm that your answer is correct. You can find more information on how to write good answers in the help center.

Collectives™ on Stack Overflow

python remove duplicate element from list

4 Answers 4

3 Comments

1 Comment

Comments

1 Comment

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

4 Answers 4

3 Comments

1 Comment

Comments

1 Comment

Your Answer

Sign up or log in

Post as a guest

Linked

Related