3

Why urlopen from urllib3 does not retry when it hits a ReadTimeoutError ? Am I missing something ? I can get around this by using my own retry code, but I would prefer to use the one from the module.

I've even tried to explicitely put retries=3 as an argument of HTTPSConnectionPool.request() but it does not help...

Here's my code:

conn = urllib3.HTTPConnectionPool(host, port=port)
return conn.request(method, api_url, data, retries=3, 
                    timeout=urllib3.Timeout(connect=2, read=3))

Thanks !

EDIT:

Now, the response with some debug info (it's a log from openERP):

2014-06-18 08:36:42,791 5585 DEBUG test 
openerp.addons.myaddon.xml_request.xml_request: args prepared
2014-06-18 08:36:42,791 DEBUG Added an stderr logging handler to logger: 
urllib3
2014-06-18 08:36:42,791 5585 DEBUG test urllib3: Added an stderr logging 
handler to logger: urllib3
2014-06-18 08:36:42,792 INFO Starting new HTTPS connection (1): 
some.host.com
2014-06-18 08:36:42,792 5585 INFO test urllib3.connectionpool: Starting new 
HTTPS connection (1): some.host.com
2014-06-18 08:36:43,699 DEBUG Setting read timeout to 10
2014-06-18 08:36:43,699 5585 DEBUG test urllib3.connectionpool: Setting read 
timeout to 10
2014-06-18 08:36:53,722 5585 ERROR test openerp.osv.osv: Uncaught exception
Traceback (most recent call last):
  File "/home/openerp/server/openerp/osv/osv.py", line 132, in wrapper
    return f(self, dbname, *args, **kwargs)
  File "/home/openerp/server/openerp/osv/osv.py", line 199, in execute
    res = self.execute_cr(cr, uid, obj, method, *args, **kw)
  File "/home/openerp/server/openerp/osv/osv.py", line 187, in execute_cr
    return getattr(object, method)(cr, uid, *args, **kw)
  File "/home/openerp/custom_addons/myaddon/xml_request/xml_request.py", 
  line 260, in button_test
    self.request(cr, uid, ids[0], data, context=context)
  File "/home/openerp/custom_addons/myaddon/xml_request/xml_request.py", 
  line 234, in request
    config_rec.xml_port, 'POST', api_url, vals)
  File "/home/openerp/custom_addons/myaddon/xml_request/xml_request.py", 
  line 208, in connAndReq
    timeout=urllib3.Timeout(connect=5, read=10))
  File "/usr/lib/python2.7/dist-packages/urllib3/request.py", line 79, in 
  request
    **urlopen_kw)
  File "/usr/lib/python2.7/dist-packages/urllib3/request.py", line 142, in 
  request_encode_body
    **urlopen_kw)
  File "/usr/lib/python2.7/dist-packages/urllib3/connectionpool.py", line 
  574, in urlopen
    raise ReadTimeoutError(self, url, "Read timed out.")
ReadTimeoutError: HTTPSConnectionPool(host=u'some.host.com', port=443): Read 
timed out.

As you can see, the ReadTimeoutError is thrown just after the expiration time, with no retries.

5
  • Are you sure that it's not retrying three times and then failing? Try adding a logger to see more detailed operations: urllib3.add_stderr_logger(); If it's not, then it might be a bug. Relatedly, we have a PR for adding more fine-grained retry controls here, any feedback is appreciated: github.com/shazow/urllib3/pull/326 Commented Jun 17, 2014 at 19:18
  • @shazow Thanks, I've added that, and I can see more things now ^^ I'll come back when I've done my tests. Commented Jun 18, 2014 at 8:33
  • Actually I suspect this is related to github.com/shazow/urllib3/pull/399 which has been fixed. Could you give it a try with urllib3 from master? If it's still broken, then please open an issue. :) Commented Jun 19, 2014 at 2:08
  • Can you please make your last comment an answer ? It works with the master branch. Side note: thanks for this great library, didn't realised you were the main creator. Commented Jun 19, 2014 at 7:50
  • Done. :) You're very welcome! I'm planning to push a new release next week or so, hang tight. Commented Jun 19, 2014 at 23:24

1 Answer 1

4

Looks like this is a bug in urllib3 v1.8.2.

It should be fixed here, available in the master branch.

New release is coming soon. Sorry for the inconvenience. :)

Sign up to request clarification or add additional context in comments.

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.