Creating my own html parser

Question

I know this post, I've already read it but still I'd like to learn what language does an html parser (may) use? I mean, does it parse the whole source with a regex or it uses a normal programming language such as c# or python?

Apart from the question above can you also brief me on from where I should start to create my own parser? (I'd like to create an html parser for my personal needs :)

You can use any Turing complete language. Regular expressions (at least those of formal language theory) aren’t. But most regular expression libraries and implementations are far more capable (see for example Can extended regex implementations parse HTML?). — Gumbo
– Gumbo, Commented Jul 29, 2011 at 18:12
Be sure to read this: stackoverflow.com/questions/1732348/… A masterpiece of StackOverflow. — Iterator
– Iterator, Commented Jul 29, 2011 at 18:12
@Gumbo: technically, you can use a pushdown automata. You don't need Turing completeness :D. And to Shaokan: the fact that HTML has a Context-Free Grammar makes any traditional programming language quite suitable. There are a variety of tools for building such parsers. I like Antlr with Java (or C# or python). If you want to build such a parser completely by hand, you should consult any reference on compiler implementation. Parsing CFGs is almost always well-discussed in compiler books. — ccoakley
– ccoakley, Commented Jul 29, 2011 at 20:39

Iterator · Accepted Answer · 2011-07-29 18:15:06Z

2

Python, Java, and Perl are all fine languages for learning to write an HTML parser. Perl is very pleasant for regular expressions, but that's not what you need for a parser. It is a bit more pleasant to write OO programs in Python or Java. C/C++/C#, etc., are also common, for very fast parsers. However, as a learning exercise, I recommend Python or Java, so that you can compare your work with standard parsers.

answered Jul 29, 2011 at 18:15

Iterator

20.6k12 gold badges80 silver badges113 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

mbq · Accepted Answer · 2011-08-01 17:22:12Z

1

The standard way is to use some Yacc/Lex duet; second makes a code that splits the code into tokens, first builds a code that converts a token stream into some desired structure.

There is also some more tempting option, Ragel. Here you just write a big regexp-like structure capable of matching entire file and define a hooks that will fire when a certain sub-pattern was matched.

answered Aug 1, 2011 at 17:22

mbq

18.7k6 gold badges52 silver badges72 bronze badges

Collectives™ on Stack Overflow

Creating my own html parser

2 Answers 2

Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related