python socket how to properly redirect http/s requests using the same socket connection?

Question

I've got here a code that sends an HTTPS request. My problem is handling redirection requests using the same socket connection.

I know that the requests module can handle this redirects very well but this code is for a proxy server that I'm developing.

Any help would be appreciated. Thanks!

import socket, ssl
from ssl import SSLContext

HOST = "www.facebook.com"
PORT = 443

ContextoSSL = SSLContext(protocol=ssl.PROTOCOL_SSLv23)
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sslSocket = ContextoSSL.wrap_socket(sock, server_hostname=HOST)
sslSocket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
sslSocket.connect((HOST, PORT))
sslSocket.do_handshake()

der = sslSocket.getpeercert(binary_form=True)
pem_data = ssl.DER_cert_to_PEM_cert(der)
#print(pem_data) # print certificate



'''1st request'''
headers = \
    "GET / HTTP/1.1\r\n"\
    "Host: www.facebook.com\r\n"\
    "User-Agent: python-requests/2.22.0\r\n"\
    "Accept-Encoding: gzip, deflate\r\nAccept: */*\r\n"\
    "Connection: keep-alive\r\n\r\n"
print("\n\n" + headers)

sslSocket.send(headers.encode()) # send request
response = sslSocket.recv(9999)
print(response) # print receive response




'''2nd request''' # on this redirect with cookie set, response should be 200 OK 
cookie, location = "", ""
for line in response.decode().splitlines():
    if "Set-Cookie" in line:
        cookie = line.replace("Set-Cookie: ", "").split(";")[0]

    if "Location" in line:
        location = line.replace("Location: ", "").split("/")[3]

print(cookie, location)

headers = \
    f"GET /{location} HTTP/1.1\r\n"\
    "Host: www.facebook.com\r\n"\
    "User-Agent: python-requests/2.22.0\r\n"\
    "Accept-Encoding: gzip, deflate\r\nAccept: */*\r\n"\
    "Connection: keep-alive\r\n"\
    f"Cookie: {cookie}\r\n\r\n"
print("\n\n" + headers)

sslSocket.send(headers.encode()) # send request
response = sslSocket.recv(9999)
print(response) # print received response

Steffen Ullrich · Accepted Answer · 2019-11-24 06:27:30Z

1

To handle a redirect you must first get the new location:

first properly read the response as specified in the HTTP standard, i.e. read the full body based on the length declared in the response
parse the response
check for a response code which indicates a redirect
in case of a redirect extract the new location from the Location field in the response header

Once you have the new location you can issue the new request for this location. If the new location is for the same domain and if both request and response indicated that the TCP connection can be reused you can try to issue the new request on the same TCP connection. But you need to handle the case that the server might close the connection anyway since this is explicitly allowed.

In all other cases you have to create a new TCP connection for the new request.

Note that showing you how you exactly can code this would be too broad. There is a reason HTTP libraries exist which you'd better use for this purpose instead of implementing all the complexity yourself.

answered Nov 24, 2019 at 6:27

Steffen Ullrich

125k11 gold badges155 silver badges194 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

markmark123 Over a year ago

I agree with you, I need to somehow implement the libraries in order to make things easier. Thanks for the answer

Collectives™ on Stack Overflow

python socket how to properly redirect http/s requests using the same socket connection?

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related