1

I'm coding on a high scalable web harvester running on top of Eventmachine. All works fine and fast. Recently I'm trying to fire the requests through a bunch of proxies which also works fine, e.g.:

EventMachine.run do
  connect_opts = { :proxy => { :host => '11.12.13.14', :port => 3128 } }
  request_opts = { :proxy => { :authorization => ['jdoe', 'mysecretpass'] } }
  req = EventMachine::HttpRequest.new('http://www.example.com/', connect_opts).get request_opts
  req.callback { }
end

I'm iterating over hundreds of proxies and firing several hundred requests per second. What I need now is to know which proxy was used for which request to store this metadata in a db. How would you fetch this information from the req object?

I hoped there's is some method like req.proxy_foo (equivalent to req.response, req.response_header etc.) to get this information but didn't find any appropriate way yet.

Improve this question
Related questions
2
Help with HTTP Intercepting Proxy in Ruby?
2
accessing a website through a proxy using Net::HTTP proxy in ruby
2
EventMachine::HttpRequest and Keep-alive connection
Load 7 more related questions Show fewer related questions

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.