8

I'm trying to GET a url using HTTPUrlConnection, however I'm always getting a 500 code, but when I try to access that same url from the browser or using curl, it works fine!

This is the code

try{
    URL url = new URL("theurl"); 
    HttpURLConnection httpcon = (HttpURLConnection) url.openConnection();
    httpcon.setRequestProperty("Accept", "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8");
    httpcon.setRequestProperty("User-Agent", "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:14.0) Gecko/20100101 Firefox/14.0.1");
    System.out.println(httpcon.getHeaderFields());
    }catch (Exception e) {
        System.out.println("exception "+e);
    }

When I print the headerfields, it shows the 500 code.. when I change the URL to something else like google.com , it works fine. But I don't understand why it doesn't work here but it works fine on the browser and with curl.

Any help would be highly appreciated..

Thank you,

4
  • 500 is for internal server error Commented Jul 30, 2012 at 12:50
  • whats theurl that you have tried ? Commented Jul 30, 2012 at 13:15
  • @sunil I was trying rassd.com/1-23544.htm Commented Jul 30, 2012 at 13:31
  • The error is on the server, you can't solve it from the client. Commented Mar 4, 2024 at 16:02

8 Answers 8

9

The status code 500 suggests that the code at web server have been crashed .Use HttpURLConnection#getErrorStream() to get more idea of the error. Refer Http Status Code 500

Sign up to request clarification or add additional context in comments.

Comments

8

This is mostly happening because of encoding. If you are using browser OK, but getting 500 ( internal server error ) in your program,it is because the browsers have a highly sophisticated code regarding charsets and content-types.

Here is my code and it works in the case of ISO8859_1 as charset and english language.

public void sendPost(String Url, String params) throws Exception {


    String url=Url;
    URL obj = new URL(url);
    HttpsURLConnection con = (HttpsURLConnection) obj.openConnection();

    con.setRequestProperty("Acceptcharset", "en-us");
    con.setRequestProperty("Accept-Language", "en-US,en;q=0.5");
    con.setRequestProperty("charset", "EN-US");
    con.setRequestProperty("Accept", "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8");
    String urlParameters=params;
    // Send post request
    con.setDoOutput(true);
    con.setDoInput(true);
    con.connect();
    //con.

    DataOutputStream wr = new DataOutputStream(con.getOutputStream());
    wr.writeBytes(urlParameters);
    wr.flush();
    wr.close();

    int responseCode = con.getResponseCode();
    System.out.println("\nSending 'POST' request to URL : " + url);
    System.out.println("Post parameters : " + urlParameters);
    System.out.println("Response Code : " + responseCode);

    BufferedReader in = new BufferedReader(
            new InputStreamReader(con.getInputStream()));
    String inputLine;
    StringBuffer response = new StringBuffer();

    while ((inputLine = in.readLine()) != null) {
        response.append(inputLine);
    }
    in.close();

    //print result
    System.out.println(response.toString());
    this.response=response.toString();
    con.disconnect();

}

and in the main program , call it like this:

myclassname.sendPost("https://change.this2webaddress.desphilboy.com/websitealias/orwebpath/someaction","paramname="+URLEncoder.encode(urlparam,"ISO8859_1"))

Comments

2

I ran into the problem of "URL works in browser, but when I do http-get in java I get a 500 Error".

In my case the problem was that the regular http-get ended up in an infinite redirect loop between /default.aspx and /login.aspx

        URL oUrl = new URL(url);
        HttpURLConnection con = (HttpURLConnection) oUrl.openConnection();
        con.setRequestMethod("GET");
        ...
        int responseCode = con.getResponseCode();

What was happening was: The server serves up a three-part cookie and con.getResponseCode() only used one of the parts. The cookie data in the header looked like this:

header.key = null
     value = HTTP/1.1 302 Found
...
header.key = Location
     value = /default.aspx
header.key = Set-Cookie
     value = WebCom-lbal=qxmgueUmKZvx8zjxPftC/bHT/g/rUrJXyOoX3YKnYJxEHwILnR13ojZmkkocFI7ZzU0aX9pVtJ93yNg=; path=/
     value = USE_RESPONSIVE_GUI=1; expires=Wed, 17-Apr-2115 18:22:11 GMT; path=/
     value = ASP.NET_SessionId=bf0bxkfawdwfr10ipmvviq3d; path=/; HttpOnly
...

So the server when receiving only a third of the needed data got confused: You're logged in! No wait, you have to login. No, you're logged in, ...

To work around the infinite redirect-loop I had to manually look for re-directs and manually parse through the header for "Set-cookie" entries.

            con = (HttpURLConnection) oUrl.openConnection();
            con.setRequestMethod("GET");
            ...
            log.debug("Disable auto-redirect. We have to look at each redirect manually");
            con.setInstanceFollowRedirects(false);
            ....
            int responseCode = con.getResponseCode();

With this code the parsing of the cookie, if we get a redirect in the responseCode:

private String getNewCookiesIfAny(String origCookies, HttpURLConnection con) {
    String result = null;
    String key;
    Set<Map.Entry<String, List<String>>> allHeaders = con.getHeaderFields().entrySet();
    for (Map.Entry<String, List<String>> header : allHeaders) {
        key = header.getKey();

        if (key != null && key.equalsIgnoreCase(HttpHeaders.SET_COOKIE)) {
            // get the cookie if need, for login
            List<String> values = header.getValue();
            for (String value : values) {
                if (result == null || result.isEmpty()) {
                    result = value;
                } else {
                    result = result + "; " + value;
                }
            }
        }
    }
    if (result == null) {
        log.debug("Reuse the original cookie");
        result = origCookies;
    }
    return result;
}

Comments

1

Make sure that your connection allows following redirects - this is one of the possible reasons for difference in behaviour between your connection and the browser (allows redirect by default).

It should be returning code 3xx, but there maybe something else somewhere that changes it to 500 for your connection.

4 Comments

emm.. I don't think this is the problem, because im not redirected when i try it in the browser.. but thanks for the hint.
Than read further error details from error stream as suggested by Muse. If it is not redirect than the only other possibility I can think of is that the server is expecting something else (cookie, other type of headers) in your request. I would copy ALL headers passed by your browser in the successful request (it should result in successful HttpURLConnection call) and than work my way backward removing one header at a time.
OK I think I know what's wrong.. The page is actually down but from the browser I think it used to get the cached version.. could that be it? because that page is not at all working.. but when i try another page under the same host.. it works!
OK, that happens. For testing purposes I always have my browsers settings to ignore cache and re-fetch data from the server every time.
1

I faced the same issue, and our issue was there was a special symbol in one of the parameter values. We fixed it by using URLEncoder.encode(String, String)

Comments

1

Check the parameter

httpURLConnection.setDoOutput(false);

Only for GET Method and set to true on POST, this save me lot of time!!!

Comments

0

In my case it turned out that the server always returns HTTP/1.1 500 (in Browser as in Java) for the page I wanted to access, but successfully delivers the webpage content nonetheless.

A human accessing the specific page via Browser just doesn't notice, since he will see the page and no error message, in Java I had to read the error stream instead of the input stream (thanks @Muse).

I have no idea why, though. Might be some obscure way to keep Crawlers out.

Comments

0

This is an old question, but I have had same issue and solved it this way.

This might help other is same situation.

In my case I was developing system on local environment, and every thing worked fine when I checked my Rest Api from browser but I got all the time thrown HTTP error 500 in my Android system.

The problem is when you work on Android, it works on VM (Virtual Machine), that said it means your local computer firewall might preventing your Virtual Machine accessing the local URL (IP) address.

You need just to allow that in your computer firewall. The same thing apply if you trying to access system from out side your network.

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.