How to properly handle Redis connections with Python / Python RQ?

Question

What is the best pattern to handle Redis connections (both for interacting with Redis directly and indirectly through Python-RQ)?

Generally, database connections need to be closed / returned to a pool when done, but I don't see how to do that with redis-py. That makes me wonder if I'm doing it the wrong way.

Also, I have seen some performance dips when enqueuing jobs to RQ, which I've been told might relate to poor connection usage / reuse.

Basically, I'm interested in knowing the correct pattern, so I can either verify or correct what we have in our application.

Thanks a lot! If there's more information that would be helpful, let me know.

mmenafra · Accepted Answer · 2013-03-15 13:20:46Z

5

Behind the scenes, redis-py uses a connection pool to manage connections to a Redis server. By default, each Redis instance you create will in turn create its own connection pool. You can override this behavior and use an existing connection pool by passing an already created connection pool instance to the connection_pool argument of the Redis class. You may choose to do this in order to implement client side sharding or have finer grain control of how connections are managed

pool = redis.ConnectionPool(host='localhost', port=6379, db=0)
r = redis.Redis(connection_pool=pool)

answered Mar 15, 2013 at 13:20

mmenafra

1106 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

Juan Carlos Coto Over a year ago

Great! However, it still isn't very clear to me if the connections returned by that connection pool should be closed at any point or if it's handled automatically. Any idea?

mmenafra Over a year ago

am not doing any closing of the connections nor did I read about it and am not seeing any performance issues yet. Maybe there is a a proper way of handling the connections

Ronnie Over a year ago

@JuanCarlosCoto: its been years now since you posted this thread, I believe you would be having tons of experience now. Can you share the best practices to follow while handling multiple connections?

Collectives™ on Stack Overflow

How to properly handle Redis connections with Python / Python RQ?

1 Answer 1

3 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

3 Comments

Your Answer

Sign up or log in

Post as a guest

Related