Can you get Request Headers with python request library?

Question

I am using the python requests library to get all the headers from a website, however requests only seems to be getting the Response Headers and i also need the Request Headers.

Is there a way to get the Request Headers within the requests library or should i use a differant library to get the headers?

my code:

import requests

r = requests.get("https://google.com", allow_redirects = False)

for key in r.headers:
    print(key, ": ", r.headers[key])

output:

Location :  https://www.google.com/
Content-Type :  text/html; charset=UTF-8
Date :  Wed, 19 Feb 2020 13:08:27 GMT
Expires :  Fri, 20 Mar 2020 13:08:27 GMT
Cache-Control :  public, max-age=2592000
Server :  gws
Content-Length :  220
X-XSS-Protection :  0
X-Frame-Options :  SAMEORIGIN
Alt-Svc :  quic=":443"; ma=2592000; v="46,43",h3-Q050=":443"; ma=2592000,h3-Q049=":443"; ma=2592000,h3-Q048=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000

Dont you define the request headers in your request to google? — Jakob F
– Jakob F, Commented Feb 19, 2020 at 13:18
The request header are the headers that you set. Why would you need to get them? — Klaus D.
– Klaus D., Commented Feb 19, 2020 at 13:18
I am scraping websites for information such as headers and some of the information such as upgrade-insecure-requests is held inside the request headers. — ConceptOfVoltage
– ConceptOfVoltage, Commented Feb 19, 2020 at 13:24

smaftoul · Accepted Answer · 2020-02-19 13:21:03Z

9

The response object contains a request object that is the request which produced the reponse.

This requests.models.PreparedRequest object is accessible through the request property of the response object, its header are in the property headersof the request object.

See this example:

>>> import requests
>>> r = requests.get("http://google.com")
>>> r.request.headers
{'Connection': 'keep-alive', 'User-Agent': 'python-requests/2.22.0', 'Accept': '*/*', 'Accept-Encoding': 'gzip, deflate'}

answered Feb 19, 2020 at 13:21

smaftoul

2,87122 silver badges17 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

ConceptOfVoltage Over a year ago

Thanks for the answer however this doesnt get all of the Request Headers. if you view the headers through chrome you get some more such as:

authority, method, path, schem,  accept, accept-encoding, accept-language, cache-control, sec-fetch-mode, sec-fetch-site, sec-fetch-user, upgrade-insecure-requests, user-agent, x-client-data

smaftoul Over a year ago

You cannot see the headers produced by python's request library in chrome, you can only inspect request's headers produced by chrome.

smaftoul Over a year ago

You can verify by launching your server that dumps request headers and compare that with the returned dict, it's the same. (gist.github.com/phrawzty/… is an example of such a server)

jave.web Over a year ago

If you need to json.dumps them in a pretty way, you will find out that they are not serializable as-they-are, but, you can convert it to dict like json.dumps(dict(r.request.headers), indent=2)

Collectives™ on Stack Overflow

Can you get Request Headers with python request library?

1 Answer 1

4 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

4 Comments

Your Answer

Sign up or log in

Post as a guest

Related