There is some way to send http requests in python with specific http version protocol.I think that, with httplib or urllib, it is not possible.
For example: GET / HTTP/6.9
Thanks in advance.
4
-
Why do you want to do this? What do you hope to accomplish by sending an illegal version number?Kevin– Kevin2014-11-11 21:12:59 +00:00Commented Nov 11, 2014 at 21:12
-
HTTP fingerprinting, to obtain what version and model of web server is running an toe.jomag– jomag2014-11-11 21:21:56 +00:00Commented Nov 11, 2014 at 21:21
-
1Is there a reason it has to be "with httplib or urllib"? (Especially since the latter is deprecated?)abarnert– abarnert2014-11-11 21:30:45 +00:00Commented Nov 11, 2014 at 21:30
-
No, I had thought to do this with sockets, but someone said me "don't reinvent the wheel"jomag– jomag2014-11-11 21:52:11 +00:00Commented Nov 11, 2014 at 21:52
Add a comment
|
3 Answers
The simple answer to your question is: You're right, neither httplib nor urllib has public, built-in functionality to do this. (Also, you really shouldn't be using urllib for most things—in particular, for urlopen.)
Of course you can always rely on implementation details of those modules, as in Lukas Graf's answer.
Or, alternatively, you could fork one of those modules and modify it, which guarantees that your code will work on other Python 2.x implementations.*. Note that httplib is one of those modules that has a link to the source up at the top, which means it's mean to server as example code, not just as a black-box library.
Or you could just reimplement the lowest-level function that needs to be hooked but that's publicly documented. For httplib, I believe that's httplib.HTTPConnection.putrequest, which is a few hundred lines long.
Or you could pick a different library that has more hooks in it, so you have less to hook.
But really, if you're trying to craft a custom request to manually fingerprint the results, why are you using an HTTP library at all? Why not just do this?
msg = 'GET / HTTP/6.9\r\n\r\n'
s = socket.create_connection((host, 80))
with closing(s):
s.send(msg)
buf = ''.join(iter(partial(s.recv, 4096), ''))
* That's not much of a benefit, given that there will never be a 2.8, all of the existing major 2.7 implementations share the same source for this module, and it's not likely any new 2.x implementation will be any different. And if you go to 3.x, httplib has been reorganized and renamed, while urllib has been removed entirely, so you'll already have bigger changes to worry about.
Comments
You can do it easily enough by subclassing httplib.HTTPConnection and redefining the class attribute _http_vsn_str:
from httplib import HTTPConnection
class MyHTTPConnection(HTTPConnection):
_http_vsn_str = '6.9'
conn = MyHTTPConnection("www.stackoverflow.com")
conn.request("GET", "/")
response = conn.getresponse()
print "Status: {} {}".format(response.status, response.reason)
print "Headers: {}".format(response.getheaders())
print "Body: {}".format(response.read())
Of course this will result in a 400 Bad Request for most servers:
Status: 400 Bad Request
Headers: [('date', 'Tue, 11 Nov 2014 21:21:12 GMT'), ('connection', 'close'), ('content-type', 'text/html; charset=us-ascii'), ('content-length', '311')]
Body: <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN""http://www.w3.org/TR/html4/strict.dtd">
<HTML><HEAD><TITLE>Bad Request</TITLE>
<META HTTP-EQUIV="Content-Type" Content="text/html; charset=us-ascii"></HEAD>
<BODY><h2>Bad Request</h2>
<hr><p>HTTP Error 400. The request is badly formed.</p>
</BODY></HTML>
7 Comments
Kevin
I don't like this. That attribute's existence and purpose is not contractually guaranteed. This could break in a future version of Python.
Lukas Graf
@Kevin well - you're changing the HTTP version identifier to
6.9 while still talking HTTP 1.1. No public API is going to ever support something this.Kevin
True... I suppose I'm really concerned that OP probably should not be trying to do exactly this in the first place. Perhaps they should be using an entirely different technique.
abarnert
@Kevin: Then the answer is, "No, you can't do this with the
httplib in Python 2.7". But you can fork your own copy of httplib and modify or subclass that, in which case it is guaranteedLukas Graf
Yes, if HTTP fingerprinting is the purpose, this probably should be done on just a TCP socket level.
|
