Java HttpURLConnection avoid redirect and go to original page

Question

I'm trying to get data from an urlA but I kept being redirected from the urlA to urlB.

I receive the response code 301 (moved permanentely) but if I use any browser (e.g. Chrome, Firefox or even Internet explorer) I can still go to urlA without being redirected. So urlA still exists (the browser does not load it from any kind of cache) and does not redirect the user automatically to urlB if you use a web browser.

How can I force my Java programm using HttpURLConnection to go to the original urlA that still exists?

private StringBuffer getHTMLCode(String urlA) throws IOException {
    URL url = new URL(urlA);
    final String userAgent = "Mozilla/5.0";

    HttpURLConnection con = (HttpURLConnection) url.openConnection();
    con.setInstanceFollowRedirects(false);  // NO REDIRECT, if I set TRUE I will be redirected to urlB
    con.setRequestMethod("GET");
    con.setRequestProperty("User-Agent", userAgent);

    int responseCode = con.getResponseCode(); 
    System.out.println("\nSending 'GET' request to URL : " + url); // shows my original urlA
    System.out.println("Response Code : " + responseCode);  // <-- 301 moved permanently        

    BufferedReader in = new BufferedReader(new InputStreamReader(con.getInputStream()));
    StringBuffer htmlCode = new StringBuffer();
    String inputLine;    

    while ((inputLine = in.readLine()) != null) {
        htmlCode.append(inputLine);
    }

    System.out.println(htmlCode);  // <head><title>Document Moved</title></head><body><h1>Object Moved</h1>This document may be found <a HREF="urlB">here</a></body>

    in.close();     
    return htmlCode;
}

If you got 301 you weren't redirected. Unclear what you're asking. — user207421
– user207421, Commented Jan 29, 2016 at 18:05
Ok, but nevertheless, I don't get the original html code from urlA but a instead htmlCode contains different html with a text saying "This document may be found..." (see the last sysout in my code above). That's why I thought I'm being redirected. My question is: how can I get the html code of urlA? — SeanKw
– SeanKw, Commented Jan 30, 2016 at 22:19

Danger · Accepted Answer · 2018-12-12 21:48:53Z

Possibly the code you are looking for is this, which uses setInstanceFollowRedirects

private StringBuffer getHTMLCode(String urlA) throws IOException {
    URL url = new URL(urlA);
    final String userAgent = "Mozilla/5.0";

    HttpURLConnection con = (HttpURLConnection) url.openConnection();
    con.setInstanceFollowRedirects(false);  // NO REDIRECT, if I set TRUE I will be redirected to urlB
    con.setRequestMethod("GET");
    con.setRequestProperty("User-Agent", userAgent);
    con.setInstanceFollowRedirects(false);

    int responseCode = con.getResponseCode(); 
    System.out.println("\nSending 'GET' request to URL : " + url); // shows my original urlA
    System.out.println("Response Code : " + responseCode);  // <-- 301 moved permanently        

    BufferedReader in = new BufferedReader(new InputStreamReader(con.getInputStream()));
    StringBuffer htmlCode = new StringBuffer();
    String inputLine;    

    while ((inputLine = in.readLine()) != null) {
        htmlCode.append(inputLine);
    }

    System.out.println(htmlCode);  // <head><title>Document Moved</title></head><body><h1>Object Moved</h1>This document may be found <a HREF="urlB">here</a></body>

    in.close();     
    return htmlCode;
}

Collectives™ on Stack Overflow

Java HttpURLConnection avoid redirect and go to original page

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related