As far as I know there are really only two advantages of using vanilla javascript versus a library such as JQuery, MooTools, etc.
- Libraries have a payload that eats bandwidth. But as people already have pointed out in the other answers you can limit this with gzipping and caching. If you only want a subset of jQuery you can do with SizzleJS and with MooTools you have the option of selecting the feature sets that you want in the same way as what Modernizr does.
- Libraries are big and takes some time to learn. Then again, this is a one-time investment for developers... and it looks nice on your resumé to know javascript libraries.
- (BONUS) Libraries are not a silver bullet so if you like to reinvent the wheel then this is definitely way to go.
It is worth pointing out why you want to use a javascript library, for which there are plenty:
- You don't need to write your own framework to support your development. If you're curious how things work you can check out the code since it is open source.
- The libraries solves browser compatibility. Both DOM and javascript have some differences between browsers. Trust me, this is a huge time sink if you have to hack fixes yourself.
- It is de facto internet standard to use javascript libraries, most of them are well documented by now and most web developers (who know javascript) know how to use them.
- You don't really give up javascript when using a library. You still need to know Javascript with it's types, objects, how closures work, etc.
- Most libraries are modularized and it doesn't take a long time to write plugins or use require and the AMD pattern.
- CSS selecting elements from the DOM is a huge help.
- (BONUS) You can use them with CoffeeScript as well.
I used to work at a web shop that was adamant on using vanilla javascript because jQuery was big and scary. This decision, mostly influenced by a lone "javascript developer", was the source of many browser bugs and slow development. Trying and trying to get into his codebase was a hair-pulling experience. Writing your own framework might seem like a good idea, but if you want to hire new developers then they can't quickly hop in and help out. Then there is also the issue of the bus factor to consider.
As I said I used to work there... there were greener pastures elsewhere. :^)