Fastest or most idiomatic way to remove object from list of objects in Python

Question

Suppose I have a list L of unknown objects, O1 to On, and I want to remove another object reference M which may refer to one of the objects in L, I've managed to do it using:

L = [ O1, O2, ... On]

...

L = [ j for j in L if j not in [ M ] ]

which is lovely and idiomatic... but I'm having to do it a lot, and I wonder if there's not another more idiomatic way, or if there is not a faster way.

The important point is that the list of objects is unknown, and may or may not include the object to be excluded. I want to avoid extending or subclassing the objects where possible.

List comprehension will be the fastest way (list comprehensions are generally faster in Python + it will be with O(n) complexity). On a side note, if M is a single object why would you want to do j not in [M]? j == M will definitely be a little faster as direct comparisons are always faster. — Muhammad Tahir
– Muhammad Tahir, Commented Apr 29, 2016 at 7:24
try:L.remove(M) except ValueError:pass? However the remove method only removes the first element equal to M. — Bakuriu
– Bakuriu, Commented Apr 29, 2016 at 7:24
The problem here is that if your objects are unknown, you don't know whether they are hashable. This is why you can't use a set and end up with quadratic runtime. In practice, having a list of objects with completely unknown properties seems weird. Where and why are you getting this list? To me this looks like a problem upstream, which needs to be fixed upstream if you want to avoid quadratic runtime for the filtering. — timgeb
– timgeb, Commented Apr 29, 2016 at 7:51
I agree @timgeb - but sadly, I'm just they guy downstream from the sewage-works... — Dycey
– Dycey, Commented Apr 29, 2016 at 7:57

Muhammad Tahir · Accepted Answer · 2016-04-29 07:32:38Z

5

list.remove seems to be the fastest way, with list comprehension as the second fastest and filter at last.

Here are the timeit results

In: python -m timeit '[x for x in [1,2,3,4,5] if x not in [4]]'
Out: 1000000 loops, best of 3: 0.285 usec per loop

In: python -m timeit '[x for x in [1,2,3,4,5] if x != 4]'
Out: 1000000 loops, best of 3: 0.254 usec per loop

In: python -m timeit 'filter(lambda x: x not in [4], [1,2,3,4,5])'
Out: 1000000 loops, best of 3: 0.577 usec per loop

In: python -m timeit 'filter(lambda x: x != 4, [1,2,3,4,5])'
Out: 1000000 loops, best of 3: 0.569 usec per loop

In: python -m timeit '[1,2,3,4,5].remove(4)'
Out: 10000000 loops, best of 3: 0.132 usec per loop

answered Apr 29, 2016 at 7:32

Muhammad Tahir

5,2221 gold badge25 silver badges40 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

Serge Ballesta Over a year ago

Your results for remove are too optimistic, because you should add a try except to catch a possible ValueError. But I agree with you it's still the best way - provided OP want to change the original list because other methods builds a copy..

Dycey Over a year ago

Agreed about the Try. Thanks @Muhammad

Muhammad Tahir Over a year ago

@SergeBallesta you are right, in case of try except + no exception it was still faster than list comprehension but for try except + raised exception it was slower than list comprehension.

Francesco · Accepted Answer · 2016-04-29 07:27:46Z

2

What about the built in filter function?

>>> l = [1,2,3,4,5]
>>> f = [4]
>>> filter(lambda x: x not in f, l)
[1, 2, 3, 5]

or in python3

>>> list(filter(lambda x: x not in f, l))
[1, 2, 3, 5]

edited Apr 29, 2016 at 7:27

answered Apr 29, 2016 at 7:23

Francesco

4,3002 gold badges25 silver badges31 bronze badges

2 Comments

Bakuriu Over a year ago

Note: this does not return a list in python3+.

Francesco Over a year ago

added the python3 version

piRSquared · Accepted Answer · 2016-04-29 07:43:56Z

2

Use try/except wrapped in a recursive function recursion takes care of potential multiple M's

def tremove(L, M):
    try:
        L.remove(M)
        return tremove(L, M)
    except:
        return L

tremove(L, M)

edited Apr 29, 2016 at 7:43

answered Apr 29, 2016 at 7:27

piRSquared

296k68 gold badges509 silver badges654 bronze badges

Comments

MSeifert · Accepted Answer · 2016-04-29 07:47:59Z

2

If there are multiple occurences of your value then probably a while-loop with remove is needed:

L = [1,2,3,4,5]
while True:
    try:
        L.remove(4)
    except:
        break

It is a bit slower (due to the exception handling and multiple iterations over the list) than the list comprehension:

[ j for j in L if j != 4 ]

but both do work fine. If you want to exclude multiple values then you should use the list-comprehension:

M = [1, 4]
[ j for j in L if j not in M ]

because the try / except will be nested and the list comprehension only needs to traverse the list once.

answered Apr 29, 2016 at 7:47

MSeifert

154k41 gold badges356 silver badges378 bronze badges

Comments

timgeb · Accepted Answer · 2016-04-29 08:10:43Z

Here's an idea that let's you make O(1) containment checks for hashable items. It should be dramatically faster for long lists M with lots of hashables.

class MixedBag(object):
    def __init__(self, *args):
        self.hashed = set()
        self.nothashed = []

        for x in args:
            self.add(x)

    def add(self, x):
        try:
            self.hashed.add(x)
        except TypeError:
            self.nothashed.append(x)

    def __contains__(self, x):
        try:
            return x in self.hashed
        except TypeError:
            return x in self.nothashed


L = [[1,2,3], 4, '5', {6}]
M = [[1,2,3], '5', {4}]

mix = MixedBag(*M)
L = [x for x in L if x not in mix]
print(L) # [4, set([6])]

Collectives™ on Stack Overflow

Fastest or most idiomatic way to remove object from list of objects in Python

5 Answers 5

3 Comments

2 Comments

Comments

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

5 Answers 5

3 Comments

2 Comments

Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related