I'm recompiling kernel testing performance and other stuff, in general all is going well except for the nvidia driver, sometimes I modify the kernel just a little, like adding or removing drivers for webcam support and then I go and re-install kernel and modules again, and whenever I do that I need to rebuilt the nvidia module, and reinstall all the libraries for X and OpenGL and cuda and so on.

Is there a way to avoid this and just re-use the previously compiled nvidia driver, or I have to rebuild the module if the kernel is recompiled ?