Python GET HTTP request taking multiple attempts to succeed

Question

I have this line for code where I am sending a get() request to a http server:

json_data = requests.get(
        "http://"+ <host_ip> + <http_resource_path>,
        auth=HttpNtlmAuth('<Username>', '<Passwrd>'), headers={'host': host_DNS_name}).json()

Now when I execute this code it succeeds but I notice it's taking 3 attempts to succeed(the code hits the url 3 times and succeeds in the 3rd attempt with frist 2 resulting in 401 error). This is what I see on my console when I execute this code:

http://<host-IP:80 "GET <http_resource_path> HTTP/1.1" 401 58
http://<host-IP:80 "GET <http_resource_path> HTTP/1.1" 401 341
http://<host-IP:80 "GET <http_resource_path> HTTP/1.1" 200 74

As you can see with same URL, credentials and port number(it's 80) every time, it's taking 3 attempts to succeed(first 2 attempts are resulting in unauthorized access but the 3rd one succeeds with the same parameters). This multiple attemptis causing a latency of about 15 seconds(between 1st and 3rd attempt). Now I am unable to understand why is taking 3 attempts to succeed when the parameters are not changed? Is there something in my code that I can do to resolve this issue and/or debug or is the issue in the server I am trying to access?

It seems hard to know if the issue is in the server since you don't give the url, how can anyone test it to ensure it's your code VS the server? — G_M
– G_M, Commented Feb 26, 2018 at 23:22
@DeliriousLettuce I don't have access to the server code and the reason I am not giving url is because it's an intranet url accessible only from VPC. Is there a way I can add something to my python code that will help me debug what's going on at the server and why is it denying first 2 attempts? — user2966197
– user2966197, Commented Feb 26, 2018 at 23:32
Wild guess: probably has something to do with HTTP Digest authentication. In Digest’s case, there will be at least two requests: first one (401) to fetch the nonce used for encrypting auth credentials, another one to actually authenticate. I’m not sure where the third request is coming from. — Linas Valiukas
– Linas Valiukas, Commented Feb 26, 2018 at 23:32
@Linas is there a way or workaround to test that out and if it's that's the case then resolve it so that it get's succeeded in one try? — user2966197
– user2966197, Commented Feb 26, 2018 at 23:39
Sorry, my previous comment was wrong, I’ve just noticed that you’re using NLTM auth. I’ll post a proper answer. — Linas Valiukas
– Linas Valiukas, Commented Feb 26, 2018 at 23:48

Linas Valiukas · Accepted Answer · 2018-02-26 23:53:23Z

1

You’re using HttpNltmAuth which I’m not familiar with, but here it says that three roundtrips is how it’s designed to work:

https://www.innovation.ch/personal/ronald/ntlm.html

If you want less roundtrips, switch to HTTP digest access auth or HTTP basic access auth on the server-side (please take security considerations in mind too, especially with HTTP basic).

answered Feb 26, 2018 at 23:53

Linas Valiukas

1,4061 gold badge15 silver badges23 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

bigblind · Accepted Answer · 2018-02-26 23:25:51Z

0

The server is returning a 401 response, and your question isn't saying anything about the server you're connecting to, so we can't really reason about why it might be. It might be because you're hitting some rate-limiting code, but the server might be disallowing the first 2 requests for many reasons.

If you have control over the server's code, look at the source code and look at the code that handles the request you're making. Also look at code that applies to all requests, like middleware.

If you don't have access to the source code, you'll have to ask the owner o the site why this might be.

answered Feb 26, 2018 at 23:25

bigblind

12.9k14 gold badges72 silver badges132 bronze badges

1 Comment

user2966197 Over a year ago

I don't have access to the server code. Is there a way I can add something to my python code that will help me debug what's going on at the server and why is it denying first 2 attempts?

Collectives™ on Stack Overflow

Python GET HTTP request taking multiple attempts to succeed

2 Answers 2

Comments

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related