If you're providing a header-only library, then you have no choice, and I'll not consider that further.
If the methods are necessarily empty, then defining in the header is usually fine (though it can be visual clutter; that can be avoided by defining outside the class declaration).
If the implementation may change due to future requirements, then be aware that code compiled against the earlier version will require re-compilation to get the new implementation. Unlikely to be a problem within a single build tree, but if you're providing a shared library, you'll want to avoid that, as it can introduce hard-to-diagnose problems.
If the methods are constructor, destructor, or copy/move assignment, then explicitly declaring = default may be what you want instead. There is a substantive difference between defining any an empty method and declaring as defaulted.