Parse URL without DNS queries in Java

Question

I'm parsing squid logs with Java. It seemed appropriate to use URL class. This class, however, makes a DNS request, which indefinitely slows down parsing. Are there other easy ways to extract hostname and port from an url?

Conditions

url schema might be ommited in squid logs
an absent (default) port should be derived for ftp, http, https protocols

Log example:

1288763851.129    295 10.10.100.10 TCP_MISS/200 435 GET http://win.mail.ru/cgi-bin/checknew? - DIRECT/217.69.128.52 text/plain
1288763881.110    275 10.10.100.10 TCP_MISS/200 434 GET http://win.mail.ru/cgi-bin/checknew? - DIRECT/217.69.128.52 text/plain
1288763883.093  60001 10.10.102.202 TCP_MISS/503 0 CONNECT www.update.microsoft.com:443 - DIRECT/- -
1288763884.301      0 10.10.102.202 NONE/400 3506 GET / - NONE/- text/html
1288763911.194    359 10.10.100.10 TCP_MISS/200 435 GET http://win.mail.ru/cgi-bin/checknew? - DIRECT/217.69.128.52 text/plain
1288763941.097    264 10.10.100.10 TCP_MISS/200 434 GET http://win.mail.ru/cgi-bin/checknew? - DIRECT/217.69.128.52 text/plain
1288763944.094  59777 10.10.102.202 TCP_MISS/503 0 CONNECT www.update.microsoft.com:443 - DIRECT/- -
1288763971.123    289 10.10.100.10 TCP_MISS/200 434 GET http://win.mail.ru/cgi-bin/checknew? - DIRECT/217.69.128.52 text/plain
1288764002.257   1421 10.10.100.10 TCP_MISS/200 435 GET http://win.mail.ru/cgi-bin/checknew? - DIRECT/217.69.128.52 text/plain

EDIT: I had to write my own class parser for this task. The idea is to use InetAddress if thestring has an IP or simple string for hostnames.

I wrote galimatias, a Java URL parsing library that you can use for the job. Once it parses the URL, you can get the host and check if it's a domain, a IPv4 or IPv6 address. It's still in early stages but it's quite solid for this use case. — smola
– smola, Commented Jan 2, 2014 at 0:17

Bruno · Accepted Answer · 2010-11-03 16:55:00Z

1

You could try Restlet's Reference class.

answered Nov 3, 2010 at 16:55

Bruno

123k33 gold badges278 silver badges389 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Basilevs Over a year ago

There is no restlet keyword in my debian distribution. I need more common solution.

Bruno Over a year ago

If you program in Java, most libraries won't be bundled out of the box with distributions. If you're after an easy installation, you could consider a build/distribution system such as Maven (Restlet has its own Maven repository, which you could easily configure in your project).

user207421 · Accepted Answer · 2010-11-04 00:07:33Z

1

Use the java.net.URI class.

answered Nov 4, 2010 at 0:07

user207421

312k45 gold badges324 silver badges493 bronze badges

3 Comments

user207421 Over a year ago

I'm not surprised. It does parse "update.microsoft.com:443", which is about the only way you can use that string in Java.

Basilevs Over a year ago

Do you mean extra quotes are needed?

user207421 Over a year ago

I mean a protocol is needed. SO turned that into a link so you can't see what I actually typed.

Collectives™ on Stack Overflow

Parse URL without DNS queries in Java

2 Answers 2

2 Comments

3 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

2 Comments

3 Comments

Your Answer

Sign up or log in

Post as a guest

Related