How to extract contents from .html.gz web page using Java?

Question

I was trying to scrape the data from a web page using Java Servlet, but I found out that the page is compressed. So when I make a URLConnection, it invokes to download the zipped file.

Can anyone help me with this? Actually, I would be visiting 1000s of pages like these, parse the table data using DOM and populate the database to make a query for some of the text words, and display the results. So I was wondering if this could make the process too slow.

Is there a way to do this without downloading the file? Any suggestions would be greatly appreciated. Thanks.

try{

        URL url = new URL("example.html.gz");
        URLConnection conn = url.openConnection();

         //FileInputStream instream= new FileInputStream(???What do I enter???);
         //GZIPInputStream ginstream =new GZIPInputStream(instream);
        conn.setAllowUserInteraction(false);
        InputStream urlStream = url.openStream();
        BufferedReader buffer = new BufferedReader(new InputStreamReader(urlStream));

        String t = buffer.readLine();
        while(t!=null){
            temp = temp + t ;
            t = buffer.readLine();
        }

After a lot of searching I finally Found answer here: [here][1] [1]: stackoverflow.com/questions/11093153/… — Crocode
– Crocode, Commented May 26, 2013 at 0:53

Chris · Accepted Answer · 2013-05-26 09:13:20Z

2

Can you try this:

GZIPInputStream ginstream =new GZIPInputStream(conn.getInputStream());

The rest is same as your code.

edited May 26, 2013 at 9:13

answered May 24, 2013 at 21:33

Chris

5,70410 gold badges42 silver badges59 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Crocode Over a year ago

The constructor FileInputStream(InputStream) is undefined.

Collectives™ on Stack Overflow

How to extract contents from .html.gz web page using Java?

1 Answer 1

1 Comment

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Linked

Related