21

I have one Arraylist of String and I have added Some Duplicate Value in that. and i just wanna remove that Duplicate value So how to remove it.

Here Example I got one Idea.

List<String> list = new ArrayList<String>();
        list.add("Krishna");
        list.add("Krishna");
        list.add("Kishan");
        list.add("Krishn");
        list.add("Aryan");
        list.add("Harm");

        System.out.println("List"+list);

        for (int i = 1; i < list.size(); i++) {
            String a1 = list.get(i);
            String a2 = list.get(i-1);
            if (a1.equals(a2)) {
                list.remove(a1);
            }
        }

        System.out.println("List after short"+list);

But is there any Sufficient way remove that Duplicate form list. with out using For loop ? And ya i can do it by using HashSet or some other way but using array list only. would like to have your suggestion for that. thank you for your answer in advance.

Improve this question
2
  • Is there a reason for not using for loops, or hash sets. Commented Feb 24, 2014 at 10:44
  • 1
    Well, you can rewrite the for-loop to use a while-loop or recursion instead, but I suspect that's not what you have in mind. A bit of context (explain why you want to do something the way you want to do it) often helps with the explanation - as it stands, I could make a few guesses as to what you want, but that's all they'd be - guesses. And your code would only remove duplicates that next to each other - is this what you want? Commented Feb 24, 2014 at 10:53

18 Answers 18

Reset to default
64

You can create a LinkedHashSet from the list. The LinkedHashSet will contain each element only once, and in the same order as the List. Then create a new List from this LinkedHashSet. So effectively, it's a one-liner:

list = new ArrayList<String>(new LinkedHashSet<String>(list))

Any approach that involves List#contains or List#remove will probably decrease the asymptotic running time from O(n) (as in the above example) to O(n^2).

EDIT For the requirement mentioned in the comment: If you want to remove duplicate elements, but consider the Strings as equal ignoring the case, then you could do something like this:

Set<String> toRetain = new TreeSet<String>(String.CASE_INSENSITIVE_ORDER);
toRetain.addAll(list);
Set<String> set = new LinkedHashSet<String>(list);
set.retainAll(new LinkedHashSet<String>(toRetain));
list = new ArrayList<String>(set);

It will have a running time of O(n*logn), which is still better than many other options. Note that this looks a little bit more complicated than it might have to be: I assumed that the order of the elements in the list may not be changed. If the order of the elements in the list does not matter, you can simply do

Set<String> set = new TreeSet<String>(String.CASE_INSENSITIVE_ORDER);
set.addAll(list);
list = new ArrayList<String>(set);
Improve this answer
Sign up to request clarification or add additional context in comments.

6 Comments

+1 for right ans. but still have one question that if i want to ignore case sensitivity then what to do?
@Krishna If you want to ignore the case of the strings, then you should mention this already in the question. However, I have inserted an EDIT covering this case as well.
Great Dude its absolutely right answer heads off. :)
is that possible for object ? i mean i have ArrayList<ModelGroup> this type of arraylist and i want to remove duplicate value from it
@Vaishali Then, the ModelGroup class must implement the hashCode and equals methods properly. There are many resource about this on the web.
|
12

if you want to use only arraylist then I am worried there is no better way which will create a huge performance benefit. But by only using arraylist i would check before adding into the list like following

void addToList(String s){
  if(!yourList.contains(s))
       yourList.add(s);
}

In this cases using a Set is suitable.

Improve this answer

2 Comments

yes that is right that can convert it to the list but still not satisfying answer. even +1 for correct
I think the cruciual question here is whether the removal should be considered as a one-time "cleanup" operation, or whether it is feasible to make sure that each element is contained only once. For example, inserting n elements with your approach would have a running time of O(n^2) ...
10

You can make use of Google Guava utilities, as shown below

 list = ImmutableSet.copyOf(list).asList();

This is probably the most efficient way of eliminating the duplicates from the list and interestingly, it preserves the iteration order as well.

UPDATE

But, in case, you don't want to involve Guava then duplicates can be removed as shown below.

ArrayList<String> list = new ArrayList<String>();
    list.add("Krishna");
    list.add("Krishna");
    list.add("Kishan");
    list.add("Krishn");
    list.add("Aryan");
    list.add("Harm");

System.out.println("List"+list);
HashSet hs = new HashSet();
hs.addAll(list);
list.clear();
list.addAll(hs);

But, of course, this will destroys the iteration order of the elements in the ArrayList.

Shishir

Improve this answer

5 Comments

your ans is actually right but adding jar file for just one use that not sound better anyways +1 for ans.
@Krishna: The Guava library for Java is very useful & powerful. Its not about adding a new JAR to accomplish a small task, its all about how smartly & most efficiently can a task can be accomplished with the least number of code.
This stackoverflow question lists few of benefits of Guava. stackoverflow.com/questions/3759440/…
Well as u said it will destroys the iteration order.
What about mutable version of this? What if I want to add data to list after removing duplicates? ImmutableSet is throwing an exception
6

Java 8 stream function

You could use the distinct function like above to get the distinct elements of the list, 

stringList.stream().distinct();

From the documentation,

Returns a stream consisting of the distinct elements (according to Object.equals(Object)) of this stream.

Another way, if you do not wish to use the equals method is by using the collect function like this, 

stringList.stream()  
    .collect(Collectors.toCollection(() -> 
        new TreeSet<String>((p1, p2) -> p1.compareTo(p2)) 
));

From the documentation,

Performs a mutable reduction operation on the elements of this stream using a Collector.

Hope that helps.

Improve this answer

2 Comments

IS it just if we use Java 8?
There is much more that you can do using functional-style operations on streams of elements. You can read this and this or you can google..
5

Simple function for removing duplicates from list

private void removeDuplicates(List<?> list)
{
    int count = list.size();

    for (int i = 0; i < count; i++) 
    {
        for (int j = i + 1; j < count; j++) 
        {
            if (list.get(i).equals(list.get(j)))
            {
                list.remove(j--);
                count--;
            }
        }
    }
}

Example:
Input: [1, 2, 2, 3, 1, 3, 3, 2, 3, 1, 2, 3, 3, 4, 4, 4, 1]
Output: [1, 2, 3, 4]

Improve this answer

Comments

4 
List<String> list = new ArrayList<String>();
        list.add("Krishna");
        list.add("Krishna");
        list.add("Kishan");
        list.add("Krishn");
        list.add("Aryan");
        list.add("Harm");

HashSet<String> hs=new HashSet<>(list);

System.out.println("=========With Duplicate Element========");
System.out.println(list);
System.out.println("=========Removed Duplicate Element========");
System.out.println(hs);
Improve this answer

1 Comment

its right dude but just change Krishna to krishna then its giving both the ans. but i want to ignore case sensitivity. now try it.
2

I don't think the list = new ArrayList<String>(new LinkedHashSet<String>(list)) is not the best way , since we are using the LinkedHashset(We could use directly LinkedHashset instead of ArrayList),

Solution:

import java.util.ArrayList;
public class Arrays extends ArrayList{

@Override
public boolean add(Object e) {
    if(!contains(e)){
        return super.add(e);
    }else{
        return false;
    }
}

public static void main(String[] args) {
    Arrays element=new Arrays();
    element.add(1);
    element.add(2);
    element.add(2);
    element.add(3);

    System.out.println(element);
}
}

Output: [1, 2, 3]

Here I am extending the ArrayList , as I am using the it with some changes by overriding the add method.

Improve this answer

1 Comment

i got ans already have ans ManojKumar any ways for your try +1 :)
2 
     public List<Contact> removeDuplicates(List<Contact> list) {
    // Set set1 = new LinkedHashSet(list);
    Set set = new TreeSet(new Comparator() {
        @Override
        public int compare(Object o1, Object o2) {
                 if(((Contact)o1).getId().equalsIgnoreCase(((Contact)2).getId()) ) {
                return 0;
            }
            return 1;
        }
    });
    set.addAll(list);
    final List newList = new ArrayList(set);
    return newList;
}
Improve this answer

Comments

1

This will be the best way

    List<String> list = new ArrayList<String>();
    list.add("Krishna");
    list.add("Krishna");
    list.add("Kishan");
    list.add("Krishn");
    list.add("Aryan");
    list.add("Harm");

    Set<String> set=new HashSet<>(list);
Improve this answer

3 Comments

He did just say he only wanted to use ArrayList.
"And ya i can do it by using HashSet or some other way but using array list only."
Moreover this solution will not preserve insertion order; you'd need a LinkedHashSet.
1

It is better to use HastSet

1-a) A HashSet holds a set of objects, but in a way that it allows you to easily and quickly determine whether an object is already in the set or not. It does so by internally managing an array and storing the object using an index which is calculated from the hashcode of the object. Take a look here

1-b) HashSet is an unordered collection containing unique elements. It has the standard collection operations Add, Remove, Contains, but since it uses a hash-based implementation, these operation are O(1). (As opposed to List for example, which is O(n) for Contains and Remove.) HashSet also provides standard set operations such as union, intersection, and symmetric difference.Take a look here

2) There are different implementations of Sets. Some make insertion and lookup operations super fast by hashing elements. However that means that the order in which the elements were added is lost. Other implementations preserve the added order at the cost of slower running times.

The HashSet class in C# goes for the first approach, thus not preserving the order of elements. It is much faster than a regular List. Some basic benchmarks showed that HashSet is decently faster when dealing with primary types (int, double, bool, etc.). It is a lot faster when working with class objects. So that point is that HashSet is fast.

The only catch of HashSet is that there is no access by indices. To access elements you can either use an enumerator or use the built-in function to convert the HashSet into a List and iterate through that.Take a look here

Improve this answer

4 Comments

But HashSet doesn't preserve order, and the OP doesn't want to use HashSet.
use Immutable Set Copy. It preserves the order http://google-collections.googlecode.com/svn/trunk/javadoc/com/google/common/collect/ImmutableSet.html
well ya but that would load more and ya that is actually right but what about tree set if i convert it to treeset that will give me same orderwise and even duplicate not allowed. :)
Yes, Treeset guarantees log(n) time cost for the basic operations (add, remove and contains) guarantees that elements of set will be sorted (ascending, natural, or the one specified by you via it's constructor) doesn't offer any tuning parameters for iteration performance offers a few handy methods to deal with the ordered set like first(), last(), headSet(), and tailSet() etc
1

Without a loop, No! Since ArrayList is indexed by order rather than by key, you can not found the target element without iterate the whole list.

A good practice of programming is to choose proper data structure to suit your scenario. So if Set suits your scenario the most, the discussion of implementing it with List and trying to find the fastest way of using an improper data structure makes no sense.

Improve this answer

2 Comments

may be u dont have idea. just go threw the linkedHashset or treeset and u will have the ans. i said no use of for loop i did not says no use of conversion. :)
But you said using array list only. This constraint means going to other data structure didn't meet your requirement. :) @Krishna
1 
public static void main(String[] args) {
    @SuppressWarnings("serial")
    List<Object> lst = new ArrayList<Object>() {
        @Override
        public boolean add(Object e) {
            if(!contains(e))
            return super.add(e);
            else
            return false;
        }
    };
    lst.add("ABC");
    lst.add("ABC");
    lst.add("ABCD");
    lst.add("ABCD");
    lst.add("ABCE");
    System.out.println(lst);

}

This is the better way

Improve this answer

Comments

1

list = list.stream().distinct().collect(Collectors.toList());
This could be one of the solutions using Java8 Stream API. Hope this helps.

Improve this answer

Comments

1 
 public void removeDuplicates() {
    ArrayList<Object> al = new ArrayList<Object>();
    al.add("java");
    al.add('a');
    al.add('b');
    al.add('a');
    al.add("java");
    al.add(10.3);
    al.add('c');
    al.add(14);
    al.add("java");
    al.add(12);

    System.out.println("Before Remove Duplicate elements:" + al);
    for (int i = 0; i < al.size(); i++) {
        for (int j = i + 1; j < al.size(); j++) {
            if (al.get(i).equals(al.get(j))) {
                al.remove(j);
                j--;
            }
        }
    }
    System.out.println("After Removing duplicate elements:" + al);
}

Before Remove Duplicate elements:

[java, a, b, a, java, 10.3, c, 14, java, 12]

After Removing duplicate elements:

[java, a, b, 10.3, c, 14, 12]
Improve this answer

Comments

0

Using java 8:

public static <T> List<T> removeDuplicates(List<T> list) {
    return list.stream().collect(Collectors.toSet()).stream().collect(Collectors.toList());
}
Improve this answer

Comments

0

In case you just need to remove the duplicates using only ArrayList, no other Collection classes, then:- 

//list is the original arraylist containing the duplicates as well
List<String> uniqueList = new ArrayList<String>();
    for(int i=0;i<list.size();i++) {
        if(!uniqueList.contains(list.get(i)))
            uniqueList.add(list.get(i));
    }

Hope this helps!

Improve this answer

Comments

0 
private static void removeDuplicates(List<Integer> list)
{
    Collections.sort(list);
    int count = list.size();
    for (int i = 0; i < count; i++) 
    {
        if(i+1<count && list.get(i)==list.get(i+1)){
            list.remove(i);
            i--;
            count--;
        }
    }
}
Improve this answer

Comments

0 
public static List<String> removeDuplicateElements(List<String> array){
    List<String> temp = new ArrayList<String>();
    List<Integer> count = new ArrayList<Integer>();
    for (int i=0; i<array.size()-2; i++){
        for (int j=i+1;j<array.size()-1;j++)
            {
                if (array.get(i).compareTo(array.get(j))==0) {
                    count.add(i);
                    int kk = i;
                }
            }
        }
        for (int i = count.size()+1;i>0;i--) {
            array.remove(i);
        }
        return array;
    }
}
Improve this answer

Comments

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.