Capturing Network Traffic Headers for a Specific File using Python

Question

I want to capture headers for a specific name from the network traffic. I saw other answers on stackoverflow that talks about using browser mob and all. But what should I do to capture specific header. For example in the image I have uploaded

,

let's consider the file with Name = id always appears in a link and I want to capture the header (specifically the url) of this file. This

As soon as a file with Name = id appears I should get its Request URL. What should I do for that and also I want to capture the cookie as well.

Any help would me much appreciated. Thank you.

You'll need to write code, which you do not appear to have even attempted to do. — Scott Hunter
– Scott Hunter, Commented Feb 7, 2022 at 20:38
You want to make a Python script to monitor the user's browser activity? — Jack Deeth
– Jack Deeth, Commented Feb 7, 2022 at 20:39
You say you want to capture the header of a request - where is the request coming from and how do you expect to access it? If you plan to make the request from your script, you would already know what's in the header, you made it. If you want the header as it was constructed in a browser, you need to insert yourself between the page and the server in the browser, which won't be possible with Python. If you want to listen for outgoing requests outside the browser, you need some sort of sniffer and that will be exceedingly hard to build. Are you perhaps after the header of a response? — Grismar
– Grismar, Commented Feb 7, 2022 at 20:46
Perhaps you should explain what you're actually trying to do and why you need this header - this may be an XY problem. — Grismar
– Grismar, Commented Feb 7, 2022 at 20:47
I am not using it to monitor someone's browser activity. I'll try to explain. It's a video streaming website and it requests the videos from it's server in segments. So i am trying to get the request link of the first segment, for other segment links I can just modify the first one. The link should not be of first segment necessarily, it can be of any segment but if it's first then I will not have to wait long for download to start. — Martin
– Martin, Commented Feb 8, 2022 at 6:40

Handsome Hunk · Accepted Answer · 2022-03-04 14:59:28Z

Okay if you want to capture the request url of a file and as you said you know the format of the url and you can identify it from many urls then for that you can try this function:

def get_url(args):
    network_logs = driver.execute_script("var performance = window.performance || window.mozPerformance || "
                                         "window.msPerformance || window.webkitPerformance || {}; var network = "
                                         "performance.getEntries() || {}; return network;")
    for network_log in network_logs:
        name = network_log.get("name", "")
        if "your_url_identifier" in name:
            return name
    return False

You can call this method directly after

browser.get(url)

and it will return you the request url which has the string your_url_identifier in it. Something like this:

browser.get(url)
file_link = WebDriverWait(browser, 1000).until(get_url)

This get_url method will keep running until either the desired url is captured or the time (1000 seconds in this example) is over.

Try and let me know if it works.

Collectives™ on Stack Overflow

Capturing Network Traffic Headers for a Specific File using Python

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related