Handle multiple connections in Flask API

Question

I'm writing a simple internal REST API for our solution using Flask, serving JSON objects through get calls (including authentication). We have multiple backends to fetch data from. From what I understand these should be connected to in a function decorated with @app.before_request and assigned to the g global for use in the specific route being requested. It's not a pattern I'm used to.

Here is a toy example of what I'm doing:

@app.before_request
def before_request():
    g.some_conn_a = create_connection('a')
    g.some_conn_b = create_connection('b')
    g.some_client = create_client()


@app.route('/get_some_data')
@requires_auth
def get_some_data():
    # Fetch something from all connections in g
    payload = ... # Construct payload using above connections
    return jsonify(payload)


@app.route('/get_some_other_data')
@requires_auth
def get_some_other_data():
    # Fetch something from maybe just g.some_conn_b
    payload = ... # Construct payload using g.some_conn_b
    return jsonify(payload)

This seems wasteful to me if the user makes a request for data residing in only one or two of these connections/clients, like in the get_some_other_data route example.

I'm considering just making the connections/clients in the route functions instead, or load it lazily. What's the "correct" way? I hope it isn't to make a new module, that seems extreme for what I'm doing.

brennan · Accepted Answer · 2017-04-04 03:59:12Z

2

Riffing on the Flask docs Database Connections example you could modify get_db() to accept an argument for each of your multiple connections.

def get_db(conn):
    """Open specificied connection if none yet for the current app context. """
    if conn == 'some_conn_a':
        if not hasattr(g, 'some_conn_a'):
            g.some_conn_a = create_connection('a')
        db = g.some_conn_a
    elif conn == 'some_conn_b':
        if not hasattr(g, 'some_conn_b'):
            g.some_conn_b = create_connection('b')
        db = g.some_conn_b 
    elif conn == 'some_client':
        if not hasattr(g, 'some_client'):
            g.some_client = create_client()
        db = g.some_client
    else:
        raise Exception("Unknown connection: %s" % conn)

    return db

@app.teardown_appcontext
def close_db(error):
    """Closes the db connections. """
    if hasattr(g, 'some_conn_a'):
        g.some_conn_a.close()
    if hasattr(g, 'some_conn_b'):
        g.some_conn_b.close()
    if hasattr(g, 'some_client'):
        g.some_client.close()

Then you could query each connection as needed:

@app.route('/get_some_data')
def get_some_data():
    data_a = get_db('some_conn_a').query().something()
    data_b = get_db('some_conn_b').query().something()
    data_c = get_db('some_client').query().something()
    payload = {'a': data_a, 'b': data_b, 'c': data_c}
    return jsonify(payload)

The get_db() pattern is preferred over the before_request pattern for lazy loading database connections. The docs examples for Flask 0.11 and up utilize the get_db() pattern to a larger extent.

edited Apr 4, 2017 at 3:59

answered Apr 3, 2017 at 12:08

brennan

3,50328 silver badges43 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

André C. Andersen Over a year ago

Thank you for the contribution. A question: If the before_request decorator isn't recommended for connections, why is it in the tutorial? flask.pocoo.org/docs/0.10/tutorial/dbcon Ninja edit: Ah. The example is older than your example. So things have changed since I worked with it last.

André C. Andersen Over a year ago

Alright. I've look into it more now. Your recommendation seems to be aligned with my initial thoughts about loading it lazily from the routing function. This also makes more sense to me. The additional step is to assign the connection g as soon as it's created, so that we can close it in teardown_appcontext. I'm fine with this. Thanks.

Collectives™ on Stack Overflow

Handle multiple connections in Flask API

1 Answer 1

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related