Python socket and GET method of HTTP protocol?

Question

I'm having a problem using the GET method of HTTP protocol, using socket library in python 3.8.2. My code:

import socket

mysocket=socket.socket(socket.AF_INET, socket.SOCK_STREAM)
mysocket.connect(('www.w3.org',80))

msg = 'GET https://www.w3.org/Status.html HTTP/1.1\r\n\r\n'
mysocket.sendall(msg.encode())

while True:
    data = mysocket.recv(1024)
    if (len(data)<1):
        break
    data.rstrip()
    print(data.decode())

mysocket.close()

It is supposed to prints the html code of w3.org/Status.html in the console but instead it returns a Error 400 Bad Request, here:

$ python web_navigator.py

HTTP/1.1 400 Bad Request
date: Thu, 23 Apr 2020 02:13:59 GMT
last-modified: Thu, 26 Mar 2020 19:01:07 GMT
etag: "3f4-5a1c69b70a2c0"
accept-ranges: bytes
content-length: 1012
vary: upgrade-insecure-requests
content-type: text/html; charset=iso-8859-1

after this, shows a html page with the error 400

Can anyone help me with this?

georgexsh · Accepted Answer · 2020-04-23 03:33:11Z

2

Host header is mandatory for HTTP/1.1, and, as you are connecting 80 port, scheme should be http, so simply change your message to:

msg = 'GET http://www.w3.org/Status.html HTTP/1.1\r\nHost: www.w3.org\r\n\r\n'

edited Apr 23, 2020 at 3:33

answered Apr 23, 2020 at 3:16

georgexsh

16.8k3 gold badges45 silver badges71 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

Jonathon Reinhart Over a year ago

It's not just the w3.org server, it's that HTTP/1.1 requires the Host header. See serverfault.com/questions/163511/….

georgexsh Over a year ago

@JonathonReinhart thanks very much for rescuing me from forgetting this import knowledge

Steffen Ullrich Over a year ago

While based on the HTTP/1.1 specification a fully specified URL with method and domain might be fine, in practice many servers expect only the path here and the full URL is only used for proxy requests. Thus it should be GET /Status.html ... instead.

georgexsh Over a year ago

@SteffenUllrich thanks for the completion, the http server of w3.org could handle absolute form resource well.

Collectives™ on Stack Overflow

Python socket and GET method of HTTP protocol?

1 Answer 1

4 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

4 Comments

Your Answer

Sign up or log in

Post as a guest

Related