The role of an X11 window manager is quite complex. ReadFirst, learn more about X core protocol and X architecture. Then read EWMH if you need to understand itthe conventional roles of WMs (also known and respected by X11 toolkits like GTK, Qt, etc...).
Even single-application but multiple-windows (e.g. popups) programs practically need some WM
Then you could choose, or configure, or perhaps patch, some window manager to suite your particular needs. Perhaps awesome, since it is scriptable in Lua, might please you (and you could find other scriptable window managers, or patch some existing one...). Some window managers are using only or mostly the keyboard (e.g. ratpoison, xmonad, etc...)
(writing your own EWMH-compliant window manager would take you too much time)