A compiler that is written in the language it compiles, and that is capable of compiling itself, is called "self-hosting". There are a couple of advantages to a self-hosting compiler:

1. When working on a compiler, you need to know three languages: the language you are compiling (the source language), the language you are compiling to (the target language), and the language you are writing the compiler in (the implementation language). Self-hosting allows you to get rid of one of them. This increases the amount of people able to work in the compiler by lessening the amount of knowledge a potential contributor needs to possess.
2. Production-grade industrial-strength high-performance compilers are large, complex, resource-intensive programs. They are a good test for your language (can your language's abstraction features handle such a large and complex project?) and your compiler (if the compiler can compile itself, then it probably also can compile other large, complex programs).
3. If your compiler is very simple, the code of the compiler can serve as a specification of the language's behavior. (In general, production-grade compilers aren't simple and simple compilers aren't production-grade, though. Also, this shouldn't be your only specification, otherwise you'll never be able to tell whether or not your compiler is correct.)
4. Self-hosting is considered to be an important milestone for a language.

There are also some disadvantages:

1. The complex bootstrap process.
2. If the compiler writer is also the language designer, there is the danger that he will add features that that he can use while writing the compiler, and leave out features that are hard to write a compiler for, thus ending up with a language that is only good for writing compilers and nothing else. (That's not necessarily a bad thing if you are designing a language for writing compilers.)

In one of his articles, Prof. Niklaus Wirth gave a nice example of the latter: when designing the Oberon language, he wrote the compiler at the same time he was designing the language. The system he was writing on, only had an obscure proprietary dialect of Fortran. After same time, he realized that he had subconsciously left out or changed features that would make it easier to write programs in Oberon because he couldn't think of a nice way to implement them in the obscure Fortran dialect. So, he threw away the compiler, re-examined his design and started a new compiler in Oberon itself. Oberon was intended to be a systems programming language, so writing a compiler and a standard library in it was a natural choice.

Now, the question is, how did he solve the bootstrap problem? Well, he was a professor, after all: he handed out portions of the compiler to his students, to manually translate by hand into Fortran.