I have an ArrayList<String>, and I want to remove repeated strings from it. How can I do this?
40 Answers
The @jonathan-stafford solution is OK. But this don't preserve the list order.
If you want preserve the list order you have to use this:
public static <T> void removeDuplicate(List <T> list) {
Set <T> set = new HashSet <T>();
List <T> newList = new ArrayList <T>();
for (Iterator <T>iter = list.iterator(); iter.hasNext(); ) {
Object element = iter.next();
if (set.add((T) element))
newList.add((T) element);
}
list.clear();
list.addAll(newList);
}
It's only to complete the answer. Very good!
1 Comment
T element = iter.next();, you don’t need the unchecked type casts. Or you use for(T element: list) … instead of dealing with an Iterator manually.Here is my answer without using any other data structure like set or hashmap etc.
public static <T> ArrayList<T> uniquefy(ArrayList<T> myList) {
ArrayList <T> uniqueArrayList = new ArrayList<T>();
for (int i = 0; i < myList.size(); i++){
if (!uniqueArrayList.contains(myList.get(i))){
uniqueArrayList.add(myList.get(i));
}
}
return uniqueArrayList;
}
1 Comment
ArrayList. Which will be very inefficient for larger lists.Time Complexity : O(n) : Without Set
private static void removeDup(ArrayList<String> listWithDuplicateElements) {
System.out.println(" Original Duplicate List :" + listWithDuplicateElements);
List<String> listWithoutDuplicateElements = new ArrayList<>(listWithDuplicateElements.size());
listWithDuplicateElements.stream().forEach(str -> {
if (listWithoutDuplicateElements.indexOf(str) == -1) {
listWithoutDuplicateElements.add(str);
}
});
System.out.println(" Without Duplicate List :" + listWithoutDuplicateElements);
}
1 Comment
indexOf on an ArrayList within a forEach is O(n²).This is the right one (if you are concerned about the overhead of HashSet.
public static ArrayList<String> removeDuplicates (ArrayList<String> arrayList){
if (arrayList.isEmpty()) return null; //return what makes sense for your app
Collections.sort(arrayList, String.CASE_INSENSITIVE_ORDER);
//remove duplicates
ArrayList <String> arrayList_mod = new ArrayList<>();
arrayList_mod.add(arrayList.get(0));
for (int i=1; i<arrayList.size(); i++){
if (!arrayList.get(i).equals(arrayList.get(i-1))) arrayList_mod.add(arrayList.get(i));
}
return arrayList_mod;
}
1 Comment
Collectiors.sort…Set<String> strSet = strList.stream().collect(Collectors.toSet());
Is the easiest way to remove your duplicates.
1 Comment
Set<String> strSet = new HashSet<>(strList);If you want your list to automatically ignore duplicates and preserve its order, you could create a HashList(a HashMap embedded List).
public static class HashList<T> extends ArrayList<T>{
private HashMap <T,T> hashMap;
public HashList(){
hashMap=new HashMap<>();
}
@Override
public boolean add(T t){
if(hashMap.get(t)==null){
hashMap.put(t,t);
return super.add(t);
}else return false;
}
@Override
public boolean addAll(Collection<? extends T> c){
HashList<T> addup=(HashList<T>)c;
for(int i=0;i<addup.size();i++){
add(addup.get(i));
}return true;
}
}
Usage Example:
HashList<String> hashlist=new HashList<>();
hashList.add("hello");
hashList.add("hello");
System.out.println(" HashList: "+hashlist);
2 Comments
ListIterator<String> i = hashList.listIterator(); i.add("hello"); i.add("hello");. Alternatively, you can use hashList.add("a"); hashList.add("b"); hashList.replaceAll(x -> "hello");. And there might be more ways to counteract this approach in the future. The takeaway is that you shouldn’t try to enforce new contracts via subclassing with classes not designed for that. You will find more about this topic when you search for the “Prefer composition over inheritance” OOP rule.Would something like this work better ?
public static void removeDuplicates(ArrayList<String> list) {
Arraylist<Object> ar = new Arraylist<Object>();
Arraylist<Object> tempAR = new Arraylist<Object>();
while (list.size()>0){
ar.add(list(0));
list.removeall(Collections.singleton(list(0)));
}
list.addAll(ar);
}
That should maintain the order and also not be quadratic in run time.
1 Comment
Here is a solution that works with any object:
public static <T> List<T> clearDuplicates(List<T> messages,Comparator<T> comparator) {
List<T> results = new ArrayList<T>();
for (T m1 : messages) {
boolean found = false;
for (T m2 : results) {
if (comparator.compare(m1,m2)==0) {
found=true;
break;
}
}
if (!found) {
results.add(m1);
}
}
return results;
}
Comments
Kotlin
val list = listOf('a', 'A', 'b', 'B', 'A', 'a')
println(list.distinct()) // [a, A, b, B]
println(list.distinctBy { it.uppercaseChar() }) // [a, b]
from here kotlinlang
Comments
In Java, List permits ordered access of their elements. They can have duplicates because their lookup key is the position not some hash code, every element can be modified while they remain in the list where as Set represents a collection of unique elements and while elements are in set, they must not be modified.While there is no restriction preventing you from modifying elements in a set, if an element is modified, then it could become forever lost in the set.
public static void main(String[] args) {
List<String> l = new ArrayList<String>();
l.add("A");
l.add("B");
l.add("C");
l.add("A");
System.out.println("Before removing duplicates: ");
for (String s : l) {
System.out.println(s);
}
Set<String> set = new HashSet<String>(l);
List<String> newlist = new ArrayList<String>(set);
System.out.println("after removing duplicates: ");
for (String s : newlist) {
System.out.println(s);
}
}
for reference, refer this link How to remove duplicates from ArrayList