In terms of actual syntax and compilation, the release notes for the different versions of Java are your best guide here. Of note would be Java 5's introduction of certain keywords, like assert and enum to the language which would actually cause compilation errors in your code base.
Aside from that, provided that the code actually did compile successfully (great, you didn't use any of the keywords!), then you have to take into account that behaviors between various classes have changed. They're available in the release notes too, and you should peruse them should you wish to avoid any breakage with your legacy code.
So in a nutshell, yes, your code could break if you use a higher version which relies on behaviors that have changed in newer versions.
There's also the matter of being sure that if you do compile your code with the newer JDK version, you must be sure that any JVM that deploys it uses at least that version or you will not be able to successfully run it.