It's safe. Map has no idea what the types are of the keys and values it contains. As far as its implementation is concerned, they are both Object, and Object does not expose any relevant state.
When you write oldValue.anything you're 'following the pointer' (in javaspeak, 'dereferencing'). From the point of view of the hashmap, you're not changing anything. Or rather, the things you might change cannot possibly affect the map.
Ordinarily you'd then think that oldValue = x would be a problem; that's not 'dereferencing', that's modifying the pointer directly. Except that's not true either: Java is pass-by-value, so that oldValue is a copy of the pointer the hashmap internally uses. You can change it all day, the map cannot witness any of these changes thus by definition that cannot break anything.
Finally, return oldValue; - that does have an effect, and the spec indicates what that is. What you're doing fits within that spec, and specs are guaranteed future proof (or rather, OpenJDK's backwards compatibility promises aren't absolute, but if OpenJDK really thinks its a good idea to modify how maps work, that would break all existing code anyway, you can't prepare for such a thing).
Thus, we end up at the answer: What you're doing is guaranteed safe. Not becauser the spec suggests it, no, because java itself implies no imaginable future change to any map impl could possibly be affected.