Java HTTP POST and GET converted from CURL

Question

Hey all I am trying my best to convert this working CURL script to Java:

@ECHO off
set COOKIES=.\cookies.txt
set USER=myUN
set PASSWORD="myPW"
set HOST=https://zzzzzzz.zzzzz.zz.zz:9443

cls

ECHO "Get the session cookie-------------------------------------"
    set out1 = curl -k -c %cookies% "%host%/qm/authenticated/identity" > nul
ECHO "-----------------------------------------------------------"

ECHO "Use the session cookie to logon----------------------------"
    curl -k -L -b %COOKIES% -c %COOKIES% -d j_username=%USER% -d j_password=%PASSWORD% %host%/qm/authenticated/j_security_check > nul
ECHO "-----------------------------------------------------------"

ECHO "Use the cookie to get the catalog--------------------------"
    curl -k -L -b %COOKIES% -H "Accept: application/xml" %host%/qm/process/project-areas/_zzzzzzzzzzzzzzzzzzzz/members
ECHO "-----------------------------------------------------------"

where:

               -k: Allow insecure server connections when using SSL
    -c <filename>: Write cookies to <filename> after operation
               -L: Follow redirects
        -b <data>: Send cookies from string/file
-H <header/@file>: Pass custom header(s) to server
        -d <data>: HTTP POST data

I've searched around for java code that includes an example of creating a cookie but I have yet to find something that would help me with my code above.

The closest I could find and modify the POST call is this:

CookieStore cookieStore = new BasicCookieStore();
String USER     = "myUN";
String PASSWORD = "myPW";
String HOST     = "https://zzzzzzz.zzzzz.zz.zz:9443";

// CookieSpecs.STANDARD is the RFC 6265 compliant policy
RequestConfig requestConfig = RequestConfig
                              .custom()
                              .setCookieSpec(CookieSpecs.STANDARD)
                              .build();

// automatically follow redirects
CloseableHttpClient client = HttpClients
                            .custom()
                            .setRedirectStrategy(new LaxRedirectStrategy())
                            .setDefaultRequestConfig(requestConfig)
                            .setDefaultCookieStore(cookieStore)
                            .build();
HttpPost postIT                  = new HttpPost(HOST + "/qm/authenticated/identity");
List<NameValuePair> urlParams    = new ArrayList<>();

urlParams.add(new BasicNameValuePair("j_username", USER));
urlParams.add(new BasicNameValuePair("j_password", PASSWORD));
postIT.setEntity(new UrlEncodedFormEntity(urlParams));

HttpResponse mmmCookie = client.execute(postIT);
// ... we have our cookie!

For postIT the value is:

POST https://zzzzz.zzz.zzzz.zzzz:9443/qm/authenticated/identity HTTP/1.1

However, it has an error of:

error: null

Not sure why the error is null since postIT has data? So not only am I not able to run the modified code I did, I'm still wondering how I go about calling a GET command using the cookie even if the code above worked.

So, in a nutshell:

Fix httpPost error.
How to send cookie on other GET methods.

Help would be great!

update for VGR:

public static void main(String[] args) throws IOException {
    Path currentRelativePath = Paths.get("").toAbsolutePath();
    PermissiveTrustManager blah = new PermissiveTrustManager();

    blah.readMembers("https://zzzzzz.zzzz.zz.zzz:9443", "zzzzzz", "zzzzzzz", currentRelativePath);
}

The error is on this line:

check(tm -> tm.checkServerTrusted(certChain, authType, socket), socket);

certChain, authType and socket all have data in them.

Where did error: null come from? How did you print or view it? — VGR
– VGR, Commented Mar 9, 2020 at 19:04
@VGR The error came from HttpResponse mmmCookie = client.execute(postIT); — StealthRT
– StealthRT, Commented Mar 9, 2020 at 19:28
Are you saying that calling client.execute(postIT) caused error: null to appear on the screen somewhere? — VGR
– VGR, Commented Mar 9, 2020 at 20:21

VGR · Accepted Answer · 2020-03-11 18:07:56Z

Since we don’t know what’s causing your cryptic error, I would suggest abandoning the third party library, and using the java.net package:

public void readMembers(String schemeAndAuthority,
                        String username,
                        String password,
                        Path membersFileToWrite)
throws IOException {

    URI baseURI = URI.create(schemeAndAuthority);

    CookieHandler oldCookieHandler = CookieHandler.getDefault();
    boolean oldFollowRedirects = HttpURLConnection.getFollowRedirects();

    CookieHandler.setDefault(new CookieManager());
    HttpURLConnection.setFollowRedirects(true);

    try {
        HttpURLConnection connection;

        URI authURI = baseURI.resolve("/qm/authenticated/identity");
        connection = (HttpURLConnection) authURI.toURL().openConnection();
        connection.getResponseCode();

        URI securityURI = baseURI.resolve(
            "/qm/authenticated/j_security_check");
        String postData = 
            "j_username=" + URLEncoder.encode(username, "UTF-8") + "&" +
            "j_password=" + URLEncoder.encode(password, "UTF-8");
        connection = (HttpURLConnection)
            securityURI.toURL().openConnection();
        connection.setRequestMethod("POST");
        connection.setRequestProperty("Content-Type",
            "application/x-www-form-urlencoded");
        connection.setDoOutput(true);
        try (OutputStream postBody = connection.getOutputStream()) {
            postBody.write(postData.getBytes(StandardCharsets.UTF_8));
        }
        connection.getResponseCode();

        URI catalogURI = baseURI.resolve(
            "/qm/process/project-areas/_zzzzzzzzzzzzzzzzzzzz/members");
        connection = (HttpURLConnection)
            catalogURI.toURL().openConnection();
        connection.setRequestProperty("Accept", "application/xml");
        try (InputStream responseBody = connection.getInputStream()) {
            Files.copy(responseBody, membersFileToWrite);
        }
    } finally {
        CookieHandler.setDefault(oldCookieHandler);
        HttpURLConnection.setFollowRedirects(oldFollowRedirects);
    }
}

However, the above code does not provide the equivalent of curl’s -k option. For that, we have to create a custom SSLContext, initialized with a TrustManager that allows all certificates:

static class PermissiveTrustManager
extends X509ExtendedTrustManager {
    private final X509ExtendedTrustManager[] realTrustManagers;

    private interface Checker {
        void checkWith(X509ExtendedTrustManager realTrustManager)
        throws CertificateException;
    }

    PermissiveTrustManager() {
        TrustManagerFactory factory;
        try {
            factory = TrustManagerFactory.getInstance(
                TrustManagerFactory.getDefaultAlgorithm());
            factory.init((KeyStore) null);
        } catch (GeneralSecurityException e) {
            // We should not be able to get here.
            throw new RuntimeException(e);
        }

        TrustManager[] allTrustManagers = factory.getTrustManagers();
        realTrustManagers = Arrays.stream(allTrustManagers)
            .filter(tm -> tm instanceof X509ExtendedTrustManager)
            .map(tm -> (X509ExtendedTrustManager) tm)
            .toArray(X509ExtendedTrustManager[]::new);

    }

    private void check(Checker checker) {
        try {
            for (X509ExtendedTrustManager realTrustManager : realTrustManagers) {
                checker.checkWith(realTrustManager);
            }
        } catch (CertificateException e) {
            System.err.println("Ignoring invalid certificate");
            e.printStackTrace();
        }
    }

    private void check(Checker checker,
                       Socket socket) {
        try {
            for (X509ExtendedTrustManager realTrustManager : realTrustManagers) {
                checker.checkWith(realTrustManager);
            }
        } catch (CertificateException e) {
            System.err.println("Ignoring invalid certificate for " + 
                socket.getRemoteSocketAddress());
            e.printStackTrace();
        }
    }

    @Override
    public void checkClientTrusted(X509Certificate[] certChain,
                                   String authType,
                                   Socket socket) {
        check(tm -> tm.checkClientTrusted(certChain, authType, socket), socket);
    }

    @Override
    public void checkClientTrusted(X509Certificate[] certChain,
                                   String authType,
                                   SSLEngine engine) {
        check(tm -> tm.checkClientTrusted(certChain, authType, engine));
    }

    @Override
    public void checkServerTrusted(X509Certificate[] certChain,
                                   String authType,
                                   Socket socket) {
        check(tm -> tm.checkServerTrusted(certChain, authType, socket), socket);
    }

    @Override
    public void checkServerTrusted(X509Certificate[] certChain,
                                   String authType,
                                   SSLEngine engine) {
        check(tm -> tm.checkServerTrusted(certChain, authType, engine));
    }

    @Override
    public void checkClientTrusted(X509Certificate[] certChain,
                                   String authType) {
        check(tm -> tm.checkClientTrusted(certChain, authType));
    }

    @Override
    public void checkServerTrusted(X509Certificate[] certChain,
                                   String authType) {
        check(tm -> tm.checkServerTrusted(certChain, authType));
    }

    @Override
    public X509Certificate[] getAcceptedIssuers() {
        return new X509Certificate[0];
    }
}

private static SSLContext createPermissiveSSLContext()
throws IOException {

    TrustManager[] trustManagers = { new PermissiveTrustManager() };

    try {
        SSLContext context = SSLContext.getInstance("TLS");
        context.init(null, trustManagers, null);
        return context;
    } catch (GeneralSecurityException e) {
        throw new IOException(e);
    }
}

public void readMembers(String schemeAndAuthority,
                        String username,
                        String password,
                        Path membersFileToWrite)
throws IOException {

    URI baseURI = URI.create(schemeAndAuthority);

    CookieHandler oldCookieHandler = CookieHandler.getDefault();
    boolean oldFollowRedirects = HttpURLConnection.getFollowRedirects();
    SSLContext oldSSLContext = null;
    try {
        oldSSLContext = SSLContext.getDefault();
    } catch (GeneralSecurityException e) {
        e.printStackTrace();
    }

    CookieHandler.setDefault(new CookieManager());
    HttpURLConnection.setFollowRedirects(true);
    SSLContext.setDefault(createPermissiveSSLContext());

    try {
        HttpURLConnection connection;

        URI authURI = baseURI.resolve("/qm/authenticated/identity");
        connection = (HttpURLConnection) authURI.toURL().openConnection();
        connection.getResponseCode();

        URI securityURI = baseURI.resolve(
            "/qm/authenticated/j_security_check");
        String postData = 
            "j_username=" + URLEncoder.encode(username, "UTF-8") + "&" +
            "j_password=" + URLEncoder.encode(password, "UTF-8");
        connection = (HttpURLConnection)
            securityURI.toURL().openConnection();
        connection.setRequestMethod("POST");
        connection.setRequestProperty("Content-Type",
            "application/x-www-form-urlencoded");
        connection.setDoOutput(true);
        try (OutputStream postBody = connection.getOutputStream()) {
            postBody.write(postData.getBytes(StandardCharsets.UTF_8));
        }
        connection.getResponseCode();

        URI catalogURI = baseURI.resolve(
            "/qm/process/project-areas/_zzzzzzzzzzzzzzzzzzzz/members");
        connection = (HttpURLConnection)
            catalogURI.toURL().openConnection();
        connection.setRequestProperty("Accept", "application/xml");
        try (InputStream responseBody = connection.getInputStream()) {
            Files.copy(responseBody, membersFileToWrite);
        }
    } finally {
        CookieHandler.setDefault(oldCookieHandler);
        HttpURLConnection.setFollowRedirects(oldFollowRedirects);
        if (oldSSLContext != null) {
            SSLContext.setDefault(oldSSLContext);
        }
    }
}

Obviously, I have no way to test this.

You could call the method like this:

new CatalogRetriever().readMembers(
    "https://zzzzzzz.zzzzz.zz.zz:9443", "myUN", "myPW",
    Paths.get("members"));

As described in the URI docs, the scheme is the http: or https: part. The authority is // followed by a hostname/port (and optional user/password, depending on the protocol).

What should schemeAndAuthority be? And I am guessing that readMembers is what I need to be calling in the main function?
I'm getting the error PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
Try removing all the e.printStackTrace(); lines. The PermissiveTrustManager class deliberately swallows certificate exceptions. Also Paths.get("").toAbsolutePath() will not work; you need to pass the Path of a file (which doesn’t need to exist), not a directory.
Okay i did that and its now saying cannot write to a URLConnection if doOutput=false - call setDoOutput(true) The line is this: try (OutputStream postBody = connection.getOutputStream()) {

Collectives™ on Stack Overflow

Java HTTP POST and GET converted from CURL

1 Answer 1

9 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

9 Comments

Your Answer

Sign up or log in

Post as a guest

Related