1

I'm accessing a REST service for data query and download. This is the very first call that performs the authentication. The response is a json structure that contains an authentication token.

When I do this call with curl ...

$ curl -v -X POST ${AUTH_URL} \
   -H 'Content-Type: application/x-www-form-urlencoded' \
   -d 'apikey='${API_KEY}'&grant_type=api_key&client_id=IDP'

... I get the following response.

First the headers:

< server: IIS
< date: Thu, 25 Feb 2021 17:59:34 GMT
< content-type: application/json
< content-length: 1500
< x-content-type-options: nosniff
< x-xss-protection: 1; mode=block
< strict-transport-security: max-age=31536000; includeSubdomains;
< cache-control: no-store
< set-cookie: KC_RESTART=; Version=1; Expires=Thu, 01-Jan-1970 00:00:10 GMT; Max-Age=0; Path=/auth/realms/IDP/; HttpOnly
< pragma: no-cache
< x-frame-options: SAMEORIGIN
< referrer-policy: no-referrer
< vary: Origin
< via: 1.1 google
< alt-svc: clear

And the content:

{"access_token":"eyJhbGciOiJSUzI1NiIsInR5cCIgOiAiSldUIiwia2lkIiA6ICIxMThSRENzZTlqUWR4UVRnSkt2ZXlvSHBaaWE4R0pIVEU5RjJPSmE1M3N3In0.eyJleHAiOjE2MTQyOTAzNzQsImlhdCI6MTYxNDI3NTk3NCwianRpIjoiNzBkMmMwZGMtZWY3Yy00NDM5LWJiNmUtNmQ4MDEzZGU2YTU0IiwiaXNzIjoiaHR0cHM6Ly9hdXRoZW50aWNhdGUuZm91bmRhdGlvbi5hcGkub25lYXRsYXMuYWlyYnVzLmNvbS9hdXRoL3JlYWxtcy9JRFAiLCJhdWQiOiJJRFAiLCJzdWIiOiJmNmI0ZjVkNC0zMzZiLTRlMTctODc3Ni1jNjA1ZTczNTRjYmIiLCJ0eXAiOiJCZWFyZXIiLCJhenAiOiJJRFAiLCJzZXNzaW9uX3N0YXRlIjoiODJkYWM4MjMtMTViOS00MmVlLWE2YzEtZjg2ZDQ2MzY1ZjAzIiwiYWNyIjoiMSIsInNjb3BlIjoiIiwibmJmIjowLCJyb2xlIjoie1wiZ2VvLmlkcC5ub3RpZnlcIjpbXCJ1c2VyXCJdLFwiZ2VvLmFwcC5vYWRcIjpbXCJ1c2VyXCJdLFwiZ2VvLmlkcC5kYXRhc3RvcmVcIjpbXCJ1c2VyXCJdLFwiZ2VvLmFwcC53b3JrYmVuY2hcIjpbXCJ1c2VyXCJdfSIsInJvbGVzIjp7Imdlby5pZHAubm90aWZ5IjpbInVzZXIiXSwiZ2VvLmFwcC5vYWQiOlsidXNlciJdLCJnZW8uaWRwLmRhdGFzdG9yZSI6WyJ1c2VyIl0sImdlby5hcHAud29ya2JlbmNoIjpbInVzZXIiXX0sInN1aWQiOiIxMTg2NTEzNTQ2IiwidXVpZCI6Ijc5Yjg0MTZlLTY0NDYtNGMwYy1hODg1LWY1MzE2ZGMzOWMyZSIsImxvYSI6MTAwfQ.5W_E4fkhirbJZNAJ_TwMbLhcKdmnHBXOjvLUr4vW-DBRvSFfQrpdlDHLMIVI4B7bZ-OU_FVnH__i_diKYJFRH4l3Zqy8maa1pyj_WhZJksqBB69ehv8xx_3qtuJCZ0z0hln0FzmyG_Ep_uaru3gK_h33SuFxjdKr4F5XocyrYpGE-ewm-mBLj4DOBnZSJ4HgV0BG02LJIPIU8BybTmvgV-4mW3LXOVKDUJMmP4TF_ZEUzNz4a1vhoW4VIOvaNkk_8v8m_R4zjNOGmd_4jWEywORBZ1ofqvn72usY7TWEVpGBxR-rKYgzWXrdeBE4_l61MT420rBID9dbI2zRgEyVIQ","expires_in":14400,"refresh_expires_in":0,"token_type":"bearer","not-before-policy":0,"session_state":"82dac823-15b9-42ee-a6c1-f86d46365f03","scope":""}

Notice that the length of the content is 1500. That is also what the content-length: 1500 header says. When I do the same test using Python, I get the same result: a 1500 characters result.

But when I do the same test using Oracle UTL_HTTP, the result is only 1453 characters. Here is debugging from my PL/SQL code:

% resp.status_code=200
% resp.reason_phrase=OK
% resp.http_version=HTTP/1.0
% resp.get_headers
% .. Server: IIS
% .. Date: Thu, 25 Feb 2021 17:50:51 GMT
% .. Content-Type: application/json
% .. Content-Length: 1453
% .. X-Content-Type-Options: nosniff
% .. X-XSS-Protection: 1; mode=block
% .. Strict-Transport-Security: max-age=31536000; includeSubdomains;
% .. Cache-Control: no-store
% .. Set-Cookie: KC_RESTART=; Version=1; Expires=Thu, 01-Jan-1970 00:00:10 GMT;
Max-Age=0; Path=/auth/realms/IDP/; HttpOnly
% .. Pragma: no-cache
% .. X-Frame-Options: SAMEORIGIN
% .. Referrer-Policy: no-referrer
% .. Vary: Origin
% .. Via: 1.1 google
% .. Alt-Svc: clear
% Response:
{"access_token":"eyJhbGciOiJSUzI1NiIsInR5cCIgOiAiSldUIiwia2lkIiA6ICIxMThSRENzZT
lqUWR4UVRnSkt2ZXlvSHBaaWE4R0pIVEU5RjJPSmE1M3N3In0.eyJleHAiOjE2MTQyODk4NTEsImlhd
CI6MTYxNDI3NTQ1MSwianRpIjoiMDQ4ZjhlMjctZTgwZi00MjIyLWFmNDAtMjZlNDdmYTFhMDg0Iiwi
aXNzIjoiaHR0cHM6Ly8zNS4xOTAuNTkuNzkvYXV0aC9yZWFsbXMvSURQIiwiYXVkIjoiSURQIiwic3V
iIjoiZjZiNGY1ZDQtMzM2Yi00ZTE3LTg3NzYtYzYwNWU3MzU0Y2JiIiwidHlwIjoiQmVhcmVyIiwiYX
pwIjoiSURQIiwic2Vzc2lvbl9zdGF0ZSI6ImYzYmZlOGJiLTFiZGYtNDQ5OS04NDQwLWIxODk5OGYwY
jg5NiIsImFjciI6IjEiLCJzY29wZSI6IiIsIm5iZiI6MCwicm9sZSI6IntcImdlby5pZHAubm90aWZ5
XCI6W1widXNlclwiXSxcImdlby5hcHAub2FkXCI6W1widXNlclwiXSxcImdlby5pZHAuZGF0YXN0b3J
lXCI6W1widXNlclwiXSxcImdlby5hcHAud29ya2JlbmNoXCI6W1widXNlclwiXX0iLCJyb2xlcyI6ey
JnZW8uaWRwLm5vdGlmeSI6WyJ1c2VyIl0sImdlby5hcHAub2FkIjpbInVzZXIiXSwiZ2VvLmlkcC5kY
XRhc3RvcmUiOlsidXNlciJdLCJnZW8uYXBwLndvcmtiZW5jaCI6WyJ1c2VyIl19LCJzdWlkIjoiMTE4
NjUxMzU0NiIsInV1aWQiOiI3OWI4NDE2ZS02NDQ2LTRjMGMtYTg4NS1mNTMxNmRjMzljMmUiLCJsb2E
iOjEwMH0.XHwxx3TzNNwgzVMv18Jav4bqXW9Q4n2bP_HV1iy0K4VPH-w84tXsHjXfH_f05Ynn2CXqv-
rdHds6KtuZaI1aypNnIvNvmbUiNHd6M1geLY4w8Yy9rg9-WFjYiFXbLTP7vvUAMSHueJmeT6WvzAsUT
Z7IQdp0w5aLDQ6ElV8pX1khBMCC7uXedRRDK-UC1MlJBrWtbhIMu5MaqpdpPeBcBMCvmqUBFTFfW6dQ
Ko01jeDjxePz_gZ2wdyU8fkV8UNTzkS3i6PYUkcxi3pmEC5r93JSNGVRUsZ53y5IjcaJK4aRXvvZQzV
iOitsbu8Pfciii2E_NDlk3qYgSqlxVrmzNA","expires_in":14400,"refresh_expires_in":0,
"token_type":"bearer","not-before-policy":0,"session_state":"f3bfe8bb-1bdf-4499
-8440-b18998f0b896","scope":""}

Notice the content length is now 1453 characters. The difference is with the token information inside the JSON response. It should be 1330 characters, but is only 1283 characters. The rest of the JSON document is the same. And the returned token is not valid for further use.

I can't find any explanation as to why the response is shorter when requested from UTL_HTTP. I first thought it had something to do with character set encoding. I have everything set to UTF-8. The token is returned in a base64 encoding.

Here is the code I use (I did not include the debugging code):

-- Setup the http request type and add the content
http_req := utl_http.begin_request(url, 'POST', 'HTTP/1.0');
utl_http.set_header(http_req, 'Content-Type', content_type);
utl_http.set_body_charset(http_req,'UTF-8');
utl_http.set_header(http_req, 'Content-Length', length(post_content));
utl_http.write_text(http_req, post_content);

-- Call the REST endpoint and fetch the http response
http_resp := utl_http.get_response(http_req);
utl_http.set_body_charset(http_resp,'UTF-8');

-- Read the response content
begin
  json_response := '';
  i := 0;
  loop
    utl_http.read_line(http_resp, response_line, false);
    json_response := json_response || response_line;
    i := i + 1;
  end loop;
exception
  when utl_http.end_of_body then
    utl_http.end_response(http_resp);
end;

I have been staring at this problem for hours. I tried various things, like setting or not setting explicit character set encoding, all to not effect. I can't see what I'm doing wrong and why Oracle would do anything with the response. I could imagine it truncating it for some reason - but why would characters be removed from inside the response ?

3
  • There should be some missing code of transferring buffer response_line to json json_response Commented Feb 25, 2021 at 21:13
  • Yes, indeed. I missed it from the cut and paste into the question. It is there however. I edited the question to include it in the code. Commented Feb 26, 2021 at 9:37
  • 1
    Found the explanation: I was using HTTP/1.0 as protocol version. Once I removed that from (or switched to HTTP/1.1) everything worked fine. Commented Feb 26, 2021 at 14:46

1 Answer 1

2

I'm slightly ashamed. The answer is simple: in the begin_request() call I was explicitly specifying the HTTP 1.0 protocol:

http_req := utl_http.begin_request(url, 'POST', 'HTTP/1.0');

Once I got rid of that (which means Oracle uses the HTTP 1.1 protocol):

http_req := utl_http.begin_request(url, 'POST');

Then everything started working correctly: I now get the full and entire response from the server.

There is still an oddity, in that when I do curl --http1.0 to also force use of HTTP 1.0, I still get the right answer. So I assume that there is something inside the Oracle implementation that causes trouble when using HTTP 1.0 on some workloads.

Sign up to request clarification or add additional context in comments.

2 Comments

It is really strange because Content-Length is determined at the server side
Yes, it is strange. I was expecting that a raw curl --http1.0 would have the same effect, but that returns the right answer ... Really odd.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.