removing duplicates in a list with linq

Question

Suppose you have a list of MyObject like this:

public class MyObject
{
  public int ObjectID {get;set;}
  public string Prop1 {get;set;}
}

How do you remove duplicates from a list where there could be multiple instance of objects with the same ObjectID.

Thanks.

Why should there ever be objects with the same ID (usually that is kept distinct for exactly this reason...)? I suppose you could create a list of the IDs and stings (as a composite string) and then do a .Distinct() but I think there is something more fundamentally wrong if IDs are non-unique. — soandos
– soandos, Commented May 11, 2011 at 19:45
@soandos: He might have a way to get the same object in the list twice on accident. — Jay Sullivan
– Jay Sullivan, Commented May 11, 2011 at 19:50
But if that something going wrong is in an external source then removing duplicates is necessary. — dragoncmd
– dragoncmd, Commented May 12, 2016 at 18:53

BrokenGlass · Accepted Answer · 2017-03-13 22:32:10Z

50

You can use GroupBy() and select the first item of each group to achieve what you want - assuming you want to pick one item for each distinct ObjectId property:

var distinctList = myList.GroupBy(x => x.ObjectID)
                         .Select(g => g.First())
                         .ToList();

Alternatively there is also DistinctBy() in the MoreLinq project that would allow for a more concise syntax (but would add a dependency to your project):

var distinctList = myList.DistinctBy( x => x.ObjectID).ToList();

edited Mar 13, 2017 at 22:32

answered May 11, 2011 at 19:43

BrokenGlass

161k28 gold badges297 silver badges339 bronze badges

Sign up to request clarification or add additional context in comments.

8 Comments

frenchie Over a year ago

Ok, cool, this is it. Would myList = myList.... also work? Just looking to avoid creating a new list.

BrokenGlass Over a year ago

@frenchie: Yes of course, you can reassign to the same list variable

Zebi Over a year ago

You can not avoid creating a new list when using linq. If you want to use the same list you have to modify it yourself using list.Remove(item) or list.RemoveAt(index).

naz786 Over a year ago

I tried this, but it is not working for my list of objects. I still get duplicates returned. Please see my question: stackoverflow.com/questions/42316343/…

rollsch Over a year ago

Any downsides to using something like MoreLinq?

|

Kristof Claes · Accepted Answer · 2011-05-11 19:53:34Z

12

You can do this using the Distinct() method. But since that method uses the default equality comparer, your class needs to implement IEquatable<MyObject> like this:

public class MyObject : IEquatable<MyObject>
{
    public int ObjectID {get;set;}
    public string Prop1 {get;set;}

    public bool Equals(MyObject other)
    {
        if (other == null) return false;
        else return this.ObjectID.Equals(other.ObjectID); 
    }

    public override int GetHashCode()
    {
        return this.ObjectID.GetHashCode();
    }
}

Now you can use the Distinct() method:

List<MyObject> myList = new List<MyObject>();
myList.Add(new MyObject { ObjectID = 1, Prop1 = "Something" });
myList.Add(new MyObject { ObjectID = 2, Prop1 = "Another thing" });
myList.Add(new MyObject { ObjectID = 3, Prop1 = "Yet another thing" });
myList.Add(new MyObject { ObjectID = 1, Prop1 = "Something" });

var duplicatesRemoved = myList.Distinct().ToList();

answered May 11, 2011 at 19:53

Kristof Claes

11k3 gold badges33 silver badges42 bronze badges

5 Comments

Juan Gomez Over a year ago

That would make the code look so much elegant! and easy to read. :-)

frenchie Over a year ago

Why is this better than the 3-line answer provided above? What does it do more?

Zebi Over a year ago

It enables your objects to be compared. This will be needed for some other list features, too and objects wich have an ID (entities) should be comparable by their id.

Kristof Claes Over a year ago

@frenchie: I believe it is more elegant and readable than the GroupBy solution and unlike the DistinctBy solution, it works without adding extra libraries.

naz786 Over a year ago

When I use this method and debug the list using a foreach loop, i still get duplicates. Please see my question: stackoverflow.com/questions/42316343/…

DigitalNomad · Accepted Answer · 2011-05-11 19:54:28Z

3

You could create a custom object comparer by implementing the IEqualityComparer interface:

public class MyObject
{
    public int Number { get; set; }
}

public class MyObjectComparer : IEqualityComparer<MyObject>
{
    public bool Equals(MyObject x, MyObject y)
    {
        return x.Id == y.Id;
    }

    public int GetHashCode(MyObject obj)
    {
        return obj.Id.GetHashCode();
    }
}

Then simply:

myList.Distinct(new MyObjectComparer())

answered May 11, 2011 at 19:54

DigitalNomad

4281 gold badge3 silver badges18 bronze badges

1 Comment

naz786 Over a year ago

Tried this, but I still get the duplicated list returned.

Collectives™ on Stack Overflow

removing duplicates in a list with linq

3 Answers 3

8 Comments

5 Comments

1 Comment

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

8 Comments

5 Comments

1 Comment

Your Answer

Sign up or log in

Post as a guest

Linked

Related