You might not care, because you think that the Set would retain its integrity if you do something like that. But it won't.
You either need Collections.newSetFromMap(new ConcurrentHashMap<E, Boolean>()); or to use some syncronization. Removal isn't the only problem here. You could also end up with duplicates in the Set or who knows what else if you just use a HashSet.
Given a proper Set implementation designed to handle threading issues, then one thread will remove the item and the other will find the item not there, which can be determined by the return value of the remove method on the Set.
Note that if you are iterating over the set, you have to use some external synchronization if you are removing elements during the iteration.
ConcurrentModificationExceptionis thrown in my case. Instead of solving it withCollections.synchronized*, I now replacejava.util.Setinterface withjava.util.concurrent.ConcurrentLinkedQueue, in case of much overhead---of course, I have to care about duplicate elements of it.Collections.synchronizedSet()as mentioned below, orCollections.newSetFromMap(new ConcurrentHashMap()).