How do I get HTTP header info without authentication using python?

Question

I'm trying to write a small program that will simply display the header information of a website. Here is the code:

import urllib2

url = 'http://some.ip.add.ress/'

request = urllib2.Request(url)

try:
    html = urllib2.urlopen(request)
except urllib2.URLError, e:
    print e.code
else:
    print html.info()

If 'some.ip.add.ress' is google.com then the header information is returned without a problem. However if it's an ip address that requires basic authentication before access then it returns a 401. Is there a way to get header (or any other) information without authentication?

I've worked it out.

After try has failed due to unauthorized access the following modification will print the header information:

print e.info()

instead of:

print e.code()

Thanks for looking :)

It's ok i've worked it out. print e.info() on error Still learning python.... — cookertron
– cookertron, Commented Apr 17, 2012 at 15:53

vartec · Accepted Answer · 2012-04-17 16:03:38Z

1

If you want just the headers, instead of using urllib2, you should go lower level and use httplib

import httplib
conn = httplib.HTTPConnection(host)
conn.request("HEAD", path)
print conn.getresponse().getheaders()

answered Apr 17, 2012 at 16:03

vartec

135k38 gold badges227 silver badges248 bronze badges

Sign up to request clarification or add additional context in comments.

5 Comments

cookertron Over a year ago

I tired that before and received a 401. Your method works great! Thanks for that

cookertron Over a year ago

I seem to be having problems again, the python console is returning [Errno 104] Connection reset by peer and i'm not sure what this means. I'm the peer but i'm not resetting anything, am i!?

vartec Over a year ago

Most likely means, that the other end is not accepting HTTP connections.

cookertron Over a year ago

You'd think so but i can access and log in via my chrome browser just fine. Also my orginal code (at the top) works fine and returns the headers no problem! Straaaange

Piotr Dobrogost Over a year ago

Instead of going low level I think it would be better to show how to do HEAD request using urllib2.

Community · Accepted Answer · 2017-05-23 11:55:53Z

0

If all you want are HTTP headers then you should make HEAD not GET request. You can see how to do this by reading Python - HEAD request with urllib2.

edited May 23, 2017 at 11:55

CommunityBot

11 silver badge

answered Apr 17, 2012 at 18:33

Piotr Dobrogost

42.7k47 gold badges243 silver badges379 bronze badges

Collectives™ on Stack Overflow

How do I get HTTP header info without authentication using python?

2 Answers 2

5 Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

5 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related