remove duplicates from a multidimensional list (python)

Question

I'm trying to remove duplicate items from a multidimensional list. My goal is to remove items that are the same across the lists.

For example: List 2, List 6 and List 7 contains the car Bentley. I want to remove that car from 2 of the lists.

How do I accomplish this?

The code below only works if I pass in a single list containing duplicate entries, but I need to deduplicate a multidimensional list.

cars = [
     ["Acura", "Alfa Romeo", "Aston Martin", "Audi", "Aston Martin"],
     ["Bentley", "BMW", "Bugatti", "Buick"],
     ["Cadillac", "Chrysler", "Citroen"],
     ["Dodge", "Ferrari", "Fiat", "Ford"],
     ["Geely", "Honda", "Hyundai", "Infiniti"],
     ["Alfa Romeo", "Bentley", "Hyundai", "Lamborghini"],
     ["Koenigsegg", "Bentley", "Maserati", "Lamborghini"]
    ]

def remove(duplicate):
  final_list = []
  for num in duplicate:
    if num not in final_list:
        final_list.append(num)
  return final_list


print (remove(cars))

returns:
[
 ['Acura', 'Alfa Romeo', 'Aston Martin', 'Audi','Aston Martin']
 ['Bentley', 'BMW', 'Bugatti', 'Buick'], 
 ['Cadillac', 'Chrysler', 'Citroen'], 
 ['Dodge', 'Ferrari', 'Fiat', 'Ford'], 
 ['Geely', 'Honda', 'Hyundai', 'Infiniti'], 
 ['Alfa Romeo', 'Bentley', 'Hyundai', 'Lamborghini'
 ['Koenigsegg', 'Bentley', 'Maserati', 'Lamborghini']
]

My desired output after deduplication is shown below. No list within this multidimensional list contains a duplicate entry.

 [
  ['Acura', 'Alfa Romeo', 'Aston Martin', 'Audi']
  ['Bentley', 'BMW', 'Bugatti', 'Buick'], 
  ['Cadillac', 'Chrysler', 'Citroen'], 
  ['Dodge', 'Ferrari', 'Fiat', 'Ford'], 
  ['Geely', 'Honda', 'Hyundai', 'Infiniti'], 
  ['Bentley', 'Hyundai', 'Lamborghini'
  ['Koenigsegg', 'Maserati']
 ]

Which one do you want to remove? Would you be OK with the output being a single list? — Mad Physicist
– Mad Physicist, Commented Aug 27, 2018 at 16:12
Fix your indentation. Right now, your code can not be pasted into an editor and executed without a syntax error. — Mad Physicist
– Mad Physicist, Commented Aug 27, 2018 at 16:13

Dani Mesejo · Accepted Answer · 2018-08-27 16:21:13Z

2

You can do it like this:

def remove(duplicate):
    final_list = []
    found = set([])
    for num in duplicate:
        lst = []
        for element in num:
            if element not in found:
                found.add(element)
                lst.append(element)
        final_list.append(lst)
    return final_list

Output

[
    ['Acura', 'Alfa Romeo', 'Aston Martin', 'Audi'],
    ['Bentley', 'BMW', 'Bugatti', 'Buick'],
    ['Cadillac', 'Chrysler', 'Citroen'],
    ['Dodge', 'Ferrari', 'Fiat', 'Ford'],
    ['Geely', 'Honda', 'Hyundai', 'Infiniti'],
    ['Lamborghini'],
    ['Koenigsegg', 'Maserati']
]

answered Aug 27, 2018 at 16:21

Dani Mesejo

62.2k6 gold badges57 silver badges86 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Life is complex Over a year ago

That was exactly what I needed. Thanks for the quick response!!

ozata · Accepted Answer · 2018-08-27 16:31:50Z

0

You can use memoization with such purpose. Store the first occurrence of every element in a list lets say list F. If an element is not in F, then it is unique. Store the unique element in F and and repeat the process.

answered Aug 27, 2018 at 16:31

ozata

5621 gold badge5 silver badges16 bronze badges

Collectives™ on Stack Overflow

remove duplicates from a multidimensional list (python)

2 Answers 2

1 Comment

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

1 Comment

Comments

Your Answer

Sign up or log in

Post as a guest

Related