Tokenized output of C source code

Question

I want to look at the tokenized output my c-source code. The cpp processor first process the cpp-directives and then it tokenizes the c source code. Then the this tokenized output is parsed.

After that assembler does the job and process continues.

I have written my tokenizer using flex. I just want to see how gcc compiler tokenizes c source code.

Correct me if I am wrong.

It's not really clear what you are asking; can you rephrase? — Jost
– Jost, Commented Feb 7, 2015 at 11:46
In the documentation of the GNU preprocessor, it is written that the c source code is tokenized. Ex 2+3 is tokenized as Int(2) PLUS Int(3) — Rishit Sanmukhani
– Rishit Sanmukhani, Commented Feb 7, 2015 at 11:48

Thomas Padron-McCarthy · Accepted Answer · 2015-02-07 15:57:47Z

3

GCC can output the result of the preprocessing step, and there is another option (intended for debugging the preprocessor) to show you details of the tokens:

gcc  -E  -fdebug-cpp  yourinpufile.c

answered Feb 7, 2015 at 15:57

Thomas Padron-McCarthy

27.7k8 gold badges55 silver badges77 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Rishit Sanmukhani Over a year ago

I believe that the code of the c-tokenizer must have been written in flex, as flex and bison are used in GCC. Can I get the source code of the c-tokenizer? I want to understand how does gcc compiler works.

Collectives™ on Stack Overflow

Tokenized output of C source code

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related