Managing connection to redis from Python

Question

I'm using redis-py in my python application to store simple variables or lists of variables in a Redis database, so I thought it would be better to create a connection to the redis server every time I need to save or retrieve a variable as this is not done very often and I don't want to have a permanent connection that timeout.

After reading through some basic tutorials, I created the connections using the Redis class, but have not found a way to close the connection, as this is the first time I'm using Redis. I'm not sure if I'm using the best approach for managing the connections so I would like some advice for this. This is how I'm setting or getting a variable now:

import redis

def getVariable(variable_name):
    my_server = redis.Redis("10.0.0.1")
    response = my_server.get(variable_name)
    return response

def setVariable(variable_name, variable_value):
    my_server = redis.Redis("10.0.0.1")
    my_server.set(variable_name, variable_value)

I basically use this code to store the last connection time or to get an average of requests per second done to my app and stuff like that.

Thanks for your advice.

Didier Spezia · Accepted Answer · 2012-10-19 11:41:17Z

81

Python uses a reference counter mechanism to deal with objects, so at the end of the blocks, the my_server object will be automatically destroyed and the connection closed. You do not need to close it explicitly.

Now this is not how you are supposed to manage Redis connections. Connecting/disconnecting for each operation is too expensive, so it is much better to maintain the connection opened. With redis-py it can be done by declaring a pool of connections:

import redis

POOL = redis.ConnectionPool(host='10.0.0.1', port=6379, db=0)

def getVariable(variable_name):
    my_server = redis.Redis(connection_pool=POOL)
    response = my_server.get(variable_name)
    return response

def setVariable(variable_name, variable_value):
    my_server = redis.Redis(connection_pool=POOL)
    my_server.set(variable_name, variable_value)

Please note connection pool management is mostly automatic and done within redis-py.

answered Oct 19, 2012 at 11:41

Didier Spezia

73.7k12 gold badges194 silver badges157 bronze badges

Sign up to request clarification or add additional context in comments.

6 Comments

jeruki Over a year ago

thanks i didnt know about that, but if i create the connection pool do i have a method to close it any way when im out of this module or is it not necesary?

Didier Spezia Over a year ago

Not necessary. Python will close the connection when the last reference to the POOL object will be destroyed (here at the end of the script).

Guillaume Gendre Over a year ago

I'm connecting to redis with an URL from my server environnement. I use redis.from_url(ENV['MYSERVER_REDIS_URL'],0). I didn't find any ConnectionPool.from_url() method. Do you know how to create a connection pool from an URL ?

pors Over a year ago

This might be because of an older version of redis.py that this answer refers to, but currently redis.py will set up a connection pool for you. No need to explicitely do that.

dman Over a year ago

Some on #redis feel that the cost of maintaining a connection pool is more costly than spawning new connections if the use is not that often.

|

haren · Accepted Answer · 2015-05-19 13:28:39Z

4

@sg1990 what if you have 10.000 users requiring redis at the same time? They cannot share a single connection and you've just created yourself a bottleneck.

With a pool of connections you can create an arbitrary number of connections and simply use get_connection() and release(), from redis-py docs.

A connection per user is a huge overkill, since every connection needs to maintain an open socket. This way you'd automatically decrease a number of e.g. concurrent websocket users that your machine can handle by half.

answered May 19, 2015 at 13:28

haren

1,6952 gold badges13 silver badges19 bronze badges

2 Comments

r_black Over a year ago

AFAIK connection pool is created automatically either you specify it explicitly or not. That is true for what I can see from source github.com/andymccurdy/redis-py/blob/master/redis/…

Shiplu Mokaddim Over a year ago

@r_black, you are correct. It means, if you create redis.Redis multiple times, it'll create multiple connection pools. If you provide connection_pool then it uses that, so you have a single connection.

StarGit · Accepted Answer · 2021-02-13 20:58:36Z

-2

you can use this to create two databases in redis:

    r1  = redis.StrictRedis(host="localhost", port=6379, db=0, decode_responses=True)
    r2  = redis.StrictRedis(host="localhost", port=6379, db=1, decode_responses=True)

answered Feb 13, 2021 at 20:58

StarGit

3515 bronze badges

Collectives™ on Stack Overflow

Managing connection to redis from Python

3 Answers 3

6 Comments

2 Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

6 Comments

2 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related