In the first case you get a concurrentModificationException because there's a modification count associated with iteration in terms of internal implementation. If the modification count changes during iteration a concurrentModificaitonException is thrown.
The solution is to use iterator.remove() instead of removing an element directly from the map.
In the second case, you are iterating not the map but a different collection while removing an element from the map. In this case, the modification count never changes during iteration because you are iterating a different collection.
Also, in a multithreaded environment always use synchronized on a collection that represents shared mutable state in a class before iterating it otherwise you can get a concurrentModificationException .
In a multithreaded environment, your second solution would not be correct because you haven't synchronized the statement where you transfer the keyset of the original map to a new collection. So there's potential to get a concurrentModificationException. Use a ConcurrentHashMap in a multithreaded environment while knowing that not every operation or set of operations on a ConcurrentHashMap is threadsafe by default.
Iteratorand if you have anIteratorfor some type ofSet, then accesses to theSetare invalid. I'm not sure if I'm remembering this correctly