How can you use a proxy with authentication with Watir?
From the documentation
proxy = {
http: 'my.proxy.com:8080',
ssl: 'my.proxy.com:8080'
}
chrome_browser = Watir::Browser.new :chrome, proxy: proxy
My proxy is structured like ip:port:username:password and doesn't appear to be working. How can I use this proxy with Watir?
For posterity:
You can use https://github.com/SebastianJ/proxy-chain-rb to setup a proxy on localhost without authentication that redirects to your actual proxy.
real_proxy = "http://username:password@host:port"
server = ProxyChainRb::Server.new
generated_proxy = server.start(real_proxy)
proxy = {
http: generated_proxy,
ssl: generated_proxy
}
Watir::Browser.new(:chrome, proxy: proxy)
* do your scraping *
server.stop
If running headless mode is not a requirement, you can also use an extension.
Create the zip file described in Selenium using Python: enter/provide http proxy password for firefox (obviously, with your own proxy information).
Create a .crx file from the zip. You can do this in Chrome -> Settings -> Extensions -> toggle Developer mode -> Pack extension
options = { extensions: ['path of .crx file'] }
browser = Watir::Browser.new :chrome, options: options
Do note that this will not work in headless Chrome.
...http: 'http://username:[email protected]:8080'...ssl! @Kiril S. doesn't work. I'm trying this as well and have problems with it.