context-managed resource on module level

Question

I am looking for a pattern where I have multiple functions that need access to a resource that is context-managed.

In particular, I am using fastAPI and want to re-use aiopg (async psycopg2) connections.

This is the basic layout:

@app.get("/hello")
def hello():
    async with aiopg.connect(...) as conn:
        async with conn.cursor(...):
            return cursor.execute(...)

Now I want to avoid a connection per route. I could think of an object outside, and in the route I either access the conn property or await creation (and store back) and then just use the with on the cursor() method.

class Pg():
    async def conn(self):
        if not self._conn:
            self._conn = await aiopg.connect(...)
        return self._conn

myPg = Pg()

@app.get("/hello")
def hello():
    conn = await myPg.conn()
    async with conn.cursor(...):
        return cursor.execute(...)

However, I then would lose the ability to automatically close the connection.

I think that I miss something really obvious here and hope that someone could guide me how to implement this properly.

Thanks!

Tryph · Accepted Answer · 2020-08-27 09:38:16Z

3

aiopg provides a Pool class that can manage connections.

just create a pool instance at the module level:

pool = await aiopg.create_pool('<your connection string>')

Then you can use the Pool.acquire context-manager to get a connection:

async with pool.acquire() as conn:
    ...

If connections already exist in the pool, they will be re-used.

edited Aug 27, 2020 at 9:38

answered Aug 27, 2020 at 9:30

Tryph

6,27935 silver badges51 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

user4344 Over a year ago

First of all, thanks for the Pool hint, this may be a solution. But assuming that a pool would need to be close()d or the like, this would bear the same sort of problem, wouldn't it?

Tryph Over a year ago

@user4344 whats does mean "automatically close the connection"? close it after a period of activity? after the end of a particular process? anyway Pool will indeed probably not handle this for you. But I don't think it is a problem to let the connections live for the whole application runtime, they will be closed when the application quits (with explicit pool.close() or, I guess, by the pool.__del__ invocation when the pool is destroyed)

user4344 Over a year ago

I mean: If I was able to open the connection or the connection pool with a context manager, then it would be closed automatically, like in my first example where the connection is closed. But with what I was posting in the example, there is no "block" that will be left call the context manager __leave__() method. But as you say: it most probably won't be a problem to not explicitely close() the connection or the pool. I'll go with your suggestion to use the pool!

Collectives™ on Stack Overflow

context-managed resource on module level

1 Answer 1

3 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

3 Comments

Your Answer

Sign up or log in

Post as a guest

Related