You're most likely using libpq.dll, the main PostgreSQL client library. You can determine which libraries it requires using Dependency Walker, but from memory it just needs the openssl DLLs that're in the same directory as it in the PostgreSQL bin dir, and maybe zlib. I don't have a handy Windows box to check with right now. libpq and its direct dependencies from the PostgreSQL bin folder should just be bundled with your application in the same directory as your application executable. They won't be used by anything except your app and won't cause conflicts with software installed elsewhere on the system.
There's one additional dependency I suspect may be biting you: You need the appropriate version of the Microsoft Visual Studio C++ Redistributable installed on the target machines. This is installed automatically by the PostgreSQL installer. If you're bundling libpq in an application your installer must run the redist installer as well. The required version depends on exactly which PostgreSQL version you're using (which you neglected to mention); it can be determined with dependency walker.