Java how to find out if a URL is http or https?

Question

I am writing a web crawler tool in Java. When I type the website name, how can I make it so that it connects to that site in http or https without me defining the protocol?

try {
   Jsoup.connect("google.com").get();
} catch (IOException ex) {
   Logger.getLogger(LinkGUI.class.getName()).log(Level.SEVERE, null, ex);
}

But I get the error:

java.lang.IllegalArgumentException: Malformed URL: google.com

What can I do? Are there any classes or libraries that do this?

What I'm trying to do is I have a list of 165 Courses, each with 65 - 71 html pages with links all throughout them. I am writing a Java program to test if the link is broken or not.

No, there is no such class in the JDK and I doubt any library allows that. You should be clear about what you want. HTTP and HTTPS are very different. — Sotirios Delimanolis
– Sotirios Delimanolis, Commented Mar 27, 2014 at 0:09
I don't know you usecase, but try using http, it should be OK.. Most sites implement url redirection. I agree with the above comment though. — hellboy
– hellboy, Commented Mar 27, 2014 at 0:15
What I'm trying to do is I have a list of 165 Courses, each with 65 - 71 html pages with links all throughout them. I am writing a Java program to test if the link is broken or not. — Chase W.
– Chase W., Commented Mar 27, 2014 at 0:35

cybersam · Accepted Answer · 2014-03-27 01:15:08Z

1

You can write your own simple method to try both protocols, like:

static boolean usesHttps(final String urlWithoutProtocol) throws IOException {
    try {
        Jsoup.connect("http://" + urlWithoutProtocol).get();
        return false;
    } catch (final IOException e) {
        Jsoup.connect("https://" + urlWithoutProtocol).get();
        return true;
    }
}

Then, your original code can be:

try {
    boolean shouldUseHttps = usesHttps("google.com");
} catch (final IOException ex) {
    Logger.getLogger(LinkGUI.class.getName()).log(Level.SEVERE, null, ex);
}

Note: you should only use the usesHttps() method once per URL, to figure out which protocol to use. After you know that, you should connect using Jsoup.connect() directly. This will be more efficient.

edited Mar 27, 2014 at 1:15

answered Mar 27, 2014 at 0:55

cybersam

67.3k6 gold badges59 silver badges81 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

no id Over a year ago

I would not suggest such a method for every day use, since raising an exception is a costly operation. It is better to detect access mode once and a next time connect using protocol detected.

Collectives™ on Stack Overflow

Java how to find out if a URL is http or https?

1 Answer 1

1 Comment

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Linked

Related