node.js is an implementation of ECMAScript plus a number of additional libraries that let you do things like access the file system, do native TCP networking, etc...
The Javascript in a browser is an implementation of ECMAScript plus a number of host objects in the browser that let you manipulate things in the browser (such as the displayed document).
Only pure Javascript that does not use any of the browser host objects (e.g. does not access the DOM and does not use any browser-specific APIs) could run in both places.
Could I, for example, just get a javascript game I wrote, copy paste
the code in the appropriate place in node.js, and it would run
perfectly?
No. Assuming your game has a user interface, there is no browser-like user interface in node.js.
Or would I need to make changes to my javascript code (e.g. syntax,
libraries (suppose the code uses Jquery and box2dWeb))?
It depends upon what non pure ECMAScript things your current code uses. All of those non pure ECMAScript things (like DOM objects in the browser) would have to be modified and replaced with something else.
What some people do not realize is that ECMAScript (which is what is in common between browser Javascript and node.js) is just the language itself and does not include things like the window object, the document object, methods to create or find DOM elements, etc... Those are "host" methods or objects which are specific to the browser and not specific to ECMAScript.