BrowserMob Proxy Python - How to get response body?

Question

I need to get the response body content for a POST request using Selenium Chrome driver and browsermob proxy. Currently this content is not included in my file HAR output when i read it although i can see the response in the browser network traffic. How can i make it so response traffic is captured? (sorry new to programming and can't see much python documentation for BMP)

    server.start()
    proxy = server.create_proxy()
    chrome_options = webdriver.ChromeOptions()
    chrome_options.add_argument("--proxy-server={0}".format(proxy.proxy)) 
    driver = webdriver.Chrome(executable_path="chromedriver.exe", chrome_options=chrome_options)

    proxy.new_har("req", options={'captureHeaders': True,'captureContent':True})
    driver.get('https://www.example.com/something')


    result_har = json.dumps(proxy.har, ensure_ascii=False)
    with open("haroutput.har", "w") as harfile:
        harfile.write(result_har)

    server.stop()
    driver.quit()

Nick Vee · Accepted Answer · 2020-04-26 17:25:55Z

8

You can get request and response by key with the identical name in proxy.har['log']['entries'] . Response's content is under entry['response']['content']

But before you have to add 'captureContent':True into the option dict of proxy.new_har call.

Example:

from browsermobproxy import Server

server = Server("./bin/browsermob-proxy")

server.start()
proxy = server.create_proxy()

from selenium import webdriver
co = webdriver.ChromeOptions()
co.add_argument('--proxy-server={host}:{port}'.format(host='localhost', port=proxy.port))

driver = webdriver.Chrome(executable_path="chromedriver", chrome_options=co)

proxy.new_har('req',options={'captureHeaders': True,'captureContent':True})

driver.get(url)
proxy.har  # returns a HAR

for ent in proxy.har['log']['entries']:
    _url = ent['request']['url']
    _response = ent['response']
    _content = _response['content']['text']

edited Apr 26, 2020 at 17:25

Nick Vee

1,1752 gold badges12 silver badges27 bronze badges

answered Jul 12, 2018 at 3:15

maxximus

812 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

cglacet Over a year ago

That doesn't seem to be working, entries is a list of dictionary with keys like "method", "url", "cookies", "queryString", etc. But it seems like there is no content (even with 'captureContent':True)

ajgriese Over a year ago

This worked for me. I was able to find the response data in proxy.har['log']['entries'][0]['response']['content']['text']

cglacet Over a year ago

That's strange, did you manage to have it with the firefox driver too?

Nick Vee Over a year ago

@cglacet "method", "url", "cookies", "queryString"... are keys of the request object, but this is a topic about response body

Collectives™ on Stack Overflow

BrowserMob Proxy Python - How to get response body?

1 Answer 1

4 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

4 Comments

Your Answer

Sign up or log in

Post as a guest

Related