Refactoring
I did not see the word "refactoring" mentioned even once so far. So, here we go:
Feel free to implement a new feature as directdirectly as possible. If you have only a single, simple, class, you likely do not need an interface, a superclass, a factory etc. for it.
If and when you notice that you expand the class in a way that it grows too fat, then is the time to rip it apart. At that time it makes great sense to think about how you actually should do that.
Patterns are a mind tool
Patterns, or more specifically the book "Design Patterns" by the gang of four, are great, amongst othersother reasons, because they build a language for developers to think and talk in. It is easy to say "observer", "factory" or "facade" and everyone knows exactly what it means, right away.
So my opinion would be that every developer should have a passing knowledge about at least the patterns in the original book, simply to be able to talk about OO concepts without always having to explain the basics. Should you actually use the patterns everytime a possibility to do so appears? Most likely not.
Libraries
Libraries are likely the one area where it may be in order to err on the side of too many pattern-based choices instead of too little. Changing something from a "fat" class to something with more pattern-derived (usually that means more and smaller classes) will radically change the interface; and that is the one thing you do not usually want to change in a library, because it is the only thing that is of real interest to the user of your library. They wouldn't care less about how you deal with your functionality internally, but they do very much care if they constantly have to change their program when you do a new release with a new API.