Finding repeated values in multiple lists

Question

I am trying to find if any of the sublists in list1 has a repeated value, so i need to be told if a number in list1[0] is the same number in list[1] (which 20 is repeated)

the numbers represent coords and the coords of each item in list1 cannot over lap, if they do then i have a module that reruns a make a new list1 untill no coords are the smae

please help

    list1 = [[7, 20], [20, 31, 32], [66, 67, 68],[7, 8, 9, 2],
             [83, 84, 20, 86, 87], [144, 145, 146, 147, 148, 149]]

    x=0
    while x != 169:
        if list1.count(x) > 0:
        print ("repeat found")
    else:
        print ("no repeat found")
    x+=1

By "repeated value" do you mean that a value in one sublist is in another sublist? Or do you mean that a value appears more than once in a single sublist? — Steven Rumbalski
– Steven Rumbalski, Commented Jun 8, 2013 at 22:33
Can you add this remark to the question. This is totally different from what write there. — Mike Müller
– Mike Müller, Commented Jun 8, 2013 at 22:44
Do you need to know where the over lap occurs or just detect it? — dansalmo
– dansalmo, Commented Jun 8, 2013 at 22:51
Just add some example input with and without repeats to really make clear what "repeat" means. — Mike Müller
– Mike Müller, Commented Jun 8, 2013 at 23:01

Community · Accepted Answer · 2017-05-23 12:28:58Z

3

How about something like:

is_dup = sum(1 for l in list1 if len(set(l)) < len(l))
if is_dup > 0:
  print ("repeat found")
else:
  print ("no repeat found")

Another example using any:

any(len(set(l)) < len(l) for l in list1)

To check if only one item is repeated in all of the lists I would chain them and check. Credit to this answer for flattening a list of lists.

flattened = sum(list1, [])
if len(flattened) > len(set(flattened)):
  print ("dups")
else:
  print ("no dups")

I guess the proper way to flatten lists is to use itertools.chain which can be used as such:

flattened = list(itertools.chain(*list1))

This can replace the sum call I used above if that seems like a hack.

edited May 23, 2017 at 12:28

CommunityBot

11 silver badge

answered Jun 8, 2013 at 22:28

squiguy

33.7k8 gold badges63 silver badges67 bronze badges

Sign up to request clarification or add additional context in comments.

9 Comments

squiguy Over a year ago

@JohnMassee This will work if all you care about is checking for duplicates. If you need to know which ones do overlap you might consider using a dictionary type.

Mike Müller Over a year ago

sum(1 for l in list1 if len(set(l)) < len(l)) gives 0 which is not the result the updated question asks for. The 20 is considered a repeat.

jfs Over a year ago

@Mike: len(flattened) > len(set(flattened)) returns the same answer as yours has_duplicates()

i love crysis Over a year ago

the code using flattened checks between sublists. it outputted dubs with list1 and no dubs when i changed 20 to a diffent number

Mike Müller Over a year ago

Ok. Did not read so far, just concentrated on the first version. Sorry for the stir-up. All is fine. :)

|

Mike Müller · Accepted Answer · 2013-06-08 23:54:34Z

2

Solution for the updated question

def has_duplicates(iterable):
    """Searching for duplicates in sub iterables.

    This approach can be faster than whole-container solutions
    with flattening if duplicates in large iterables are found 
    early.
    """
    seen = set()
    for sub_list in iterable:
        for item in sub_list:
            if item in seen:
                return True
            seen.add(item)
    return False


>>> has_duplicates(list1)
True
>>> has_duplicates([[1, 2], [4, 5]])
False
>>> has_duplicates([[1, 2], [4, 5, 1]])
True

Lookup in a set is fast. Don't use a list for seen if you want it to be fast.

Solution for the original version of the question

If the length of the list is larger than the length of the set made form this list there must be repeated items because a set can only have unique elements:

>>> L = [[1, 1, 2], [1, 2, 3], [4, 4, 4]]
>>> [len(item) - len(set(item)) for item in L]
[1, 0, 2]

This is the key here

>>> {1, 2, 3, 1, 2, 1}
set([1, 2, 3])

EDIT

If your are not interested in the number of repeats for each sub list. This would be more efficient because its stops after the first number greater than 0:

>>> any(len(item) - len(set(item)) for item in L)
True

Thanks to @mata for pointing this out.

edited Jun 8, 2013 at 23:54

answered Jun 8, 2013 at 22:29

Mike Müller

86k21 gold badges174 silver badges165 bronze badges

2 Comments

mata Over a year ago

any(len(item) - len(set(item)) for item in L) would do if your're just interested wheater there's a match. It has the advantage that any only tries until a match has been found and then returns.

i love crysis Over a year ago

all i need this thing to do is identify weather or not a number is repeated in the whole of list1

perreal · Accepted Answer · 2013-06-08 22:41:08Z

1

from collections import Counter
list1=[[7, 20], [20, 31, 32], [66, 67, 68],
        [7, 8, 9, 2], [83, 84, 20, 86, 87],
        [144,144, 145, 146, 147, 148, 149]]
for i,l in enumerate(list1):
    for r in [x for x,y in Counter(x for x in l).items() if y > 1]:
        print 'at list ', i, ' item ', r , ' repeats'

and this one gives globally repeated values:

expl=sorted([x for l in list1 for x in l])
print [x for x,y in zip(expl, expl[1:]) if x==y]

edited Jun 8, 2013 at 22:41

answered Jun 8, 2013 at 22:36

perreal

98.7k23 gold badges159 silver badges187 bronze badges

Comments

mfaerevaag · Accepted Answer · 2013-06-08 22:31:17Z

0

For Python 2.7+, you should try a Counter:

import collections

list = [1, 2, 3, 2, 1]
count = collections.Counter(list)

Then count would be like:

Counter({1: 2, 2: 2, 3:1})

Collectives™ on Stack Overflow

Finding repeated values in multiple lists

4 Answers 4

9 Comments

Solution for the updated question

Solution for the original version of the question

EDIT

2 Comments

Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

4 Answers 4

9 Comments

Solution for the updated question

Solution for the original version of the question

EDIT

2 Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related