How to generate the proper RestTemplate usage from curl?

Question

I have a working curl command that is below:

curl -v  -H "Content-Type:application/octet-stream" \
         -H "x-amz-server-side-encryption:aws:kms" \
         -H "x-amz-server-side-encryption-aws-kms-key-id:abcdef81-abcd-4c85-b1d8-ee540d0a5f5d" \
         --upload-file /Users/fd/Downloads/video.mp4 \
         'https://video-uploads-prod.s3-accelerate.amazonaws.com/ABCDEAQGZHEhM55fvvA/ads-aws_userUploadedVideo?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Date=20200106T165718Z&X-Amz-SignedHeaders=content-type%3Bhost%3Bx-amz-server-side-encryption%3Bx-amz-server-side-encryption-aws-kms-key-id&X-Amz-Expires=86400&X-Amz-Credential=ABCDEFHLWTCWZ2MUPPBQ%2F20200106%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Signature=037949abcd1234b063c75d3d505dd9120dd3fa9250c1ababa152e91fee123ca0'

The curl is working properly:

* We are completely uploaded and fine
< HTTP/1.1 200 OK

However, when I try to use RestTemplate (i'm using spring boot 1.5.6) I'm not able to make it work. The code I use is:

byte[] media = //video in mp4//;
String uploadUrl = "https://video-uploads-prod.s3-accelerate.amazonaws.com/ABCDEAQGZHEhM55fvvA/ads-aws_userUploadedVideo?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Date=20200106T165718Z&X-Amz-SignedHeaders=content-type%3Bhost%3Bx-amz-server-side-encryption%3Bx-amz-server-side-encryption-aws-kms-key-id&X-Amz-Expires=86400&X-Amz-Credential=ABCDEFHLWTCWZ2MUPPBQ%2F20200106%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Signature=037949abcd1234b063c75d3d505dd9120dd3fa9250c1ababa152e91fee123ca0";

RestTemplate restTemplate = new RestTemplate();

HttpHeaders headers = new HttpHeaders();
headers.setContentType(MediaType.APPLICATION_OCTET_STREAM);
headers.set("x-amz-server-side-encryption", encryption);
headers.set("x-amz-server-side-encryption-aws-kms-key-id", awsKmsKeyId);

HttpEntity entity = new HttpEntity<>(media, headers);

ResponseEntity<String> respEntity = restTemplate.exchange(uploadUrl, HttpMethod.POST, entity, String.class);

The error I get from AWS is:

<Error>
   <Code>AuthorizationQueryParametersError</Code>
   <Message>Error parsing the X-Amz-Credential parameter; the Credential is mal-formed; expecting "&lt;YOUR-AKID&gt;/YYYYMMDD/REGION/SERVICE/aws4_request".</Message>
   <RequestId>51FF099744C43804</RequestId>
   <HostId>FOlLws+txYMP0hKEg7aDjQeeARdn7bJN+lw7q/aGA48hRnr1YEsJrVmRi6oEz+mkpHlTIax5MkI=</HostId>
</Error>

My suspicion is that RestTemplate is changing the encoding of the URL. Is there anyway to replicate exactly the same as curl with RestTemplate?

Did you try to encode URL? url-encode-decode.com try it and let me know if it works... — Jonathan JOhx
– Jonathan JOhx, Commented Jan 6, 2020 at 23:00
@Oneguy I confirmed there is a mess with the encoding for sure. For some reason, curl works perfectly well, but the RestTemplate does not. Even with the previously decoded url. — Federico Piazza
– Federico Piazza, Commented Jan 6, 2020 at 23:10

ddewaele · Accepted Answer · 2020-07-15 12:22:03Z

3

You can pass a URI as opposed to a String in your RestTemplate and that should get rid of your error. Had the same thing happening here.

URI uri = URI.create(urlAsString);
Map response = new RestTemplate().exchange(uri, GET, new HttpEntity(headers), Map.class).getBody();

It seems like slashes are being hex-escaped when using a String so %2F becomes %252F. Passing in a URI directly avoids that.

edited Jul 15, 2020 at 12:22

answered Jul 14, 2020 at 22:36

ddewaele

22.6k10 gold badges75 silver badges86 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Anton Hlinisty · Accepted Answer · 2020-01-06 23:05:15Z

0

I haven't test the following approach but what if you pass an expanded url to a restTemplate instance?

String uploadUrl = "...{your_params_in_placeholders}";
URI expanded = new UriTemplate(url).expand(uploadUrl, <param_values>);
url = URLDecoder.decode(expanded.toString(), "UTF-8");
restTemplate.exchange(url, HttpMethod.POST, entity, String.class);

If it doesn't work you can try to pay some attention to the encoded values in your URL(I mean "%2F" and so on)

answered Jan 6, 2020 at 23:05

Anton Hlinisty

1,4671 gold badge22 silver badges36 bronze badges

1 Comment

Federico Piazza Over a year ago

I have tried using UriComponentsBuilder and it helped avoiding the error I posted in my question, but still have some issues since I get a response saying "signature does not match". This means that in some place there still an encoding

Federico Piazza · Accepted Answer · 2020-01-08 17:00:33Z

I could make it work after struggling a lot with this. The code is pretty ugly but made the trick to work. Post for others in case you need it. Anyway, this is not answering my question I want get the exact RestTemplate configuration out of a curl command.

I had to manually decode each query param with URLDecoder.decode and then built the URI with UriComponentsBuilder.fromHttpUrl and added the previously decoded params

byte[] media = //video in mp4//;
String uploadUrl = "https://video-uploads-prod.s3-accelerate.amazonaws.com/ABCDEAQGZHEhM55fvvA/ads-aws_userUploadedVideo?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Date=20200106T165718Z&X-Amz-SignedHeaders=content-type%3Bhost%3Bx-amz-server-side-encryption%3Bx-amz-server-side-encryption-aws-kms-key-id&X-Amz-Expires=86400&X-Amz-Credential=ABCDEFHLWTCWZ2MUPPBQ%2F20200106%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Signature=037949abcd1234b063c75d3d505dd9120dd3fa9250c1ababa152e91fee123ca0";

RestTemplate restTemplate = new RestTemplate();

HttpHeaders headers = new HttpHeaders();
headers.setContentType(MediaType.APPLICATION_OCTET_STREAM);
headers.set("x-amz-server-side-encryption", encryption);
headers.set("x-amz-server-side-encryption-aws-kms-key-id", awsKmsKeyId);

HttpEntity entity = new HttpEntity<>(media, headers);

String url = uploadUrl.split("\\?")[0];
String[] urlParams = uploadUrl.split("\\?")[1].split("&");

MultiValueMap<String, String> params = new LinkedMultiValueMap<String, String>();

for (String p : urlParams) {
    String key = p.split("=")[0];
    String value = URLDecoder.decode(p.split("=")[1], StandardCharsets.UTF_8.toString());
    params.put(key, Collections.singletonList(value));
}

String uri = UriComponentsBuilder.fromHttpUrl(url)
        .queryParams(params)
        .build()
        .toUriString();

ResponseEntity<String> respEntity = restTemplate.exchange(uri, HttpMethod.PUT, entity, String.class);

Collectives™ on Stack Overflow

How to generate the proper RestTemplate usage from curl?

3 Answers 3

Comments

1 Comment

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

Comments

1 Comment

Comments

Your Answer

Sign up or log in

Post as a guest

Related