Removing duplicates from a list but returning the same list

Question

I need to remove the duplicates from a list but return the same list. So options like:

return list(set(list))

will not work for me, as it creates a new list instead.

def remove_extras(lst):
  for i in lst:
    if lst.count(i)>1:
        lst.remove(i)
  return lst

Here is my code, it works for some cases, but I dont get why it does not work for remove_extras([1,1,1,1]), as it returns [1,1] when the count for 1 should be >1.

First of all, you should never (until absolutely necessary) modify the object that you are iterating over in a loop. — Kaushik NP
– Kaushik NP, Commented Mar 10, 2017 at 2:51

niemmi · Accepted Answer · 2017-03-10 03:00:20Z

2

You can use slice assignment to replace the contents of the list after you have created a new list. In case order of the result doesn't matter you can use set:

def remove_duplicates(l):
    l[:] = set(l)

l = [1, 2, 1, 3, 2, 1]
remove_duplicates(l)
print(l)

Output:

[1, 2, 3]

edited Mar 10, 2017 at 3:00

answered Mar 10, 2017 at 2:55

niemmi

17.3k7 gold badges38 silver badges42 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

ShadowRanger Over a year ago

Ah, beat me to it. FYI, the list wrapping isn't needed: l[:] = set(l) will work just fine (internally, I believe it listifies, but it's cheaper than explicit conversion).

niemmi Over a year ago

@ShadowRanger Thanks for the comment, updated the answer. I thought slice assignment requires a list but any iterable seems to work.

Kaushik NP · Accepted Answer · 2017-03-10 03:18:12Z

0

You can achieve this using OrderedDict which removes the duplicates while maintaining order of the list.

>>> from collections import OrderedDict
>>> itemList = [1, 2, 0, 1, 3, 2]
>>> itemList[:]=OrderedDict.fromkeys(itemList)
>>> itemList
[1, 2, 0, 3]

This has a Runtime : O(N)

edited Mar 10, 2017 at 3:18

answered Mar 10, 2017 at 3:01

Kaushik NP

6,81510 gold badges33 silver badges60 bronze badges

4 Comments

ShadowRanger Over a year ago

Without slice assigning back to itemList, this doesn't actually answer the question. You want itemList[:] = OrderedDict.fromkeys(itemList).

Kaushik NP Over a year ago

@ShadowRanger, ofcourse. I just showed how it works. Assigning it back is the easy part after all.

ShadowRanger Over a year ago

It was the only actual question the OP had; it's fine to demonstrate a better way to uniquify, but you do need to answer the question to (how to dedup in place so the original list is deduped).

Kaushik NP Over a year ago

I get your point @ShadowRanger. Have updated the answer. My usual ideology is to guide others, and point them in the right direction. That according to me should be enough and helps the most :D

Collectives™ on Stack Overflow

Removing duplicates from a list but returning the same list

2 Answers 2

2 Comments

4 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

2 Comments

4 Comments

Your Answer

Sign up or log in

Post as a guest

Related