What is an easy way to remove duplicates from only part of the string in Python?

Question

I have a list of strings that goes like this:

I would like to remove all duplicates where second 2 numbers are the same. So after running it through program I would get something like this:

But something like

would also be correct.

Please update your question with the code you have tried.

quamrana
– quamrana

2020-06-02 09:44:19 +00:00
Commented Jun 2, 2020 at 9:44 — quamrana
– quamrana, Commented Jun 2, 2020 at 9:44

One Lyner · Accepted Answer · 2020-06-02 12:19:01Z

1

Here is a nice and fast trick you can use (assuming l is your list):

list({ s.split(';', 1)[1] : s for s in l }.values())

No need to import anything, and fast as can be.

In general you can define:

def custom_unique(L, keyfunc):
    return list({ keyfunc(li): li for li in L }.values())

edited Jun 2, 2020 at 12:19

answered Jun 2, 2020 at 12:08

One Lyner

2,0341 gold badge8 silver badges8 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

vanjavk Over a year ago

Oh, this solution is nice. If I understand correctly it generates a dictionary with the second 2 numbers as key and whole string as value. so whenever the same 2 numbers appear first one gets overwritten and no duplicates are made.

One Lyner Over a year ago

@vanjavk yes, that's it

Tom Ron · Accepted Answer · 2020-06-02 10:13:33Z

1

You can group the items by this key and then use the first item in each group (assuming l is your list).

import itertools
keyfunc = lambda x: x.split(";", 1)[1]
[next(g) for k, g in itertools.groupby(sorted(l, key=keyfunc), keyfunc)]

edited Jun 2, 2020 at 10:13

answered Jun 2, 2020 at 9:49

Tom Ron

6,2513 gold badges28 silver badges44 bronze badges

4 Comments

vanjavk Over a year ago

Just need to change to: x.split(";")[1]+";"+x.split(";")[2]and this solution works!

Tom Ron Over a year ago

I fixed my answer and it is now slightly more efficient than your suggestion.

Tom Ron Over a year ago

@OneLyner, please describe where it fails, s\he asked for the second 2 numbers to be the same

One Lyner Over a year ago

@TomRon sorry I misread :) You might be interested in my answer.

snatchysquid · Accepted Answer · 2020-06-02 09:49:08Z

0

Here is a code on the few first items, just switch my list with yours:

x = [
'7;213;164',
'8;213;164',
'9;145;112',
'10;145;112',
'11;145;112',
]
new_list = []
for i in x:
    check = True
    s_part = i[i.find(';'):]
    for j in new_list:
        if s_part in j:
            check = False
    if check == True:
        new_list.append(i)

print(new_list)

Output:

['7;213;164', '9;145;112']

answered Jun 2, 2020 at 9:49

snatchysquid

1,39212 silver badges25 bronze badges

Collectives™ on Stack Overflow

What is an easy way to remove duplicates from only part of the string in Python?

3 Answers 3

2 Comments

4 Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

2 Comments

4 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related