Compare items in list with nested for-loop

Question

I have a list of URLs in an open CSV which I have ordered alphabetically, and now I would like to iterate through the list and check for duplicate URLs. In a second step, the duplicate should then be removed from the list, but I am currently stuck on the checking part which I have tried to solve with a nested for-loop as follows:

for i in short_urls:
    first_url = i
    for s in short_urls:
        second_url = s
    if i == s:
       print "duplicate"
    else:
       print "all good"

The print statements will obviously be replaced once the nested for-loop is working. Currently, the list contains a few duplicates, but my nested loop does not seem to work correctly as it does not recognise any of the duplicates.

My question is: are there better ways to do perform this exercise, and what is the problem with the current nested for-loop?

Many thanks :)

Indentation. If something is not intended to the same level as your loop, it is not inside the loop. — SethMMorton
– SethMMorton, Commented May 21, 2014 at 20:51

huu · Accepted Answer · 2014-05-21 21:07:40Z

2

By construction, your method is faulty, even if you indent the if/else block correctly. For instance, imagine if you had [1, 2, 3] as short_urls for the sake of argument. The outer for loop will pick out 1 to compare to the list against. It will think it's finding a duplicate when in the inner for loop it encounters the first element, a 1 as well. Essentially, every element will be tagged as a duplicate and if you plan on removing duplicates, you'll end up with an empty list.

The better solution is to call set(short_urls) to get a set of your urls with the duplicates removed. If you want a list (as opposed to a set) of urls with the duplicates removed, you can convert the set back into a list with list(set(short_urls)).

In other words:

short_urls = ['google.com', 'twitter.com', 'google.com']
duplicates_removed_list = list(set(short_urls))
print duplicates_removed_list # Prints ['google.com', 'twitter.com']

edited May 21, 2014 at 21:07

answered May 21, 2014 at 20:52

huu

7,5602 gold badges36 silver badges49 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Matthias Over a year ago

I already tried your first suggestion; it leads to the loop running infinitely. How would you exactly call the set(short_urls)? Not sure I fully understand it yet.

huu Over a year ago

If short_urls is a list, you can literally just do set(short_urls) and you'll get a set of the original list. A set is a collection with no duplicates.

Lorenzo Baracchi · Accepted Answer · 2014-05-21 20:59:20Z

0

if i == s:

is not inside the second for loop. You missed an indentation

for i in short_urls:
    first_url = i
    for s in short_urls:
        second_url = s
        if i == s:
           print "duplicate"
        else:
           print "all good"

EDIT: Also you are comparing every element of an array with every element of the same array. This means compare the element at position 0 with the element at postion 0, which is obviously the same. What you need to do is starting the second for at the position after that reached in the first for.

edited May 21, 2014 at 20:59

answered May 21, 2014 at 20:52

Lorenzo Baracchi

2,0181 gold badge14 silver badges19 bronze badges

1 Comment

Matthias Over a year ago

See my comment on the previous answer: doing this leads to an infinite loop for some reason.

Collectives™ on Stack Overflow

Compare items in list with nested for-loop

2 Answers 2

2 Comments

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

2 Comments

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related