Python requests causing error on certain urls

Question

For some reason when I try to get and process the following url with python-requests I receive an error causing my program to fail. Other similar urls seems to work fine

import requests

test = requests.get('http://t.co/Ilvvq1cKjK')
print test.url, test.status_code

What could be causing this URL to fail instead of just producing a 404 status code?

Isn't HTTPConnectionPool(host='nasa.gov', port=80): Max retries exceeded with url: /aeronautics enough info.? The actual connection to the server is being refused so you can't get a 404 from it... — Jon Clements
– Jon Clements, Commented Sep 22, 2014 at 1:56
That is certainly enough info if I was just processing a single web request, however I am running multiple urls through requests.get() and this one causes the program to terminate is there a way around that? — John
– John, Commented Sep 22, 2014 at 1:59

Jon Clements · Accepted Answer · 2014-09-22 02:13:50Z

2

The requests library has an exception hierarchy as listed here

So wrap your GET request in a try/except block:

import requests

try:
    test = requests.get('http://t.co/Ilvvq1cKjK')
    print test.url, test.status_code
except requests.exceptions.ConnectionError as e:
    print e.request.url, "*connection failed*"

That way you end up with similar behaviour to what you're doing now (so you get the redirected url), but cater for not being able to connect rather than print the status code.

edited Sep 22, 2014 at 2:13

answered Sep 22, 2014 at 2:04

Jon Clements

143k34 gold badges254 silver badges288 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Python requests causing error on certain urls

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related