Pretty much all resources relating to game programming, especially 3D open-world games, talk about how you have to constantly be unloading and reloading assets to and from disk, system memory, and video memory. I can understand this on consoles, as they have very simple memory management schemes that cannot deal with any potential overflow.

However, on the PC, the situation is very different. There is virtual memory provided by the OS for system memory and the graphics driver handles swapping back and forth from the CPU to the GPU. So why is it that it's not sufficient to load everything at once and only have to deal with prefetching and possibly making sure that there is no blocking while a required asset is being swapped?