Deploying Flask sqlalchemy apps in AWS lambda and API gateway

Question

I am not able to find good resources which can help me understand how can i migrate my Flask and sqlalchemy apps to AWS lambda and API gateway and make it serverless. Like for instance below is a sample code taken from the flask_sqlalchemy documentation :

from flask import Flask
from flask_sqlalchemy import SQLAlchemy

app = Flask(__name__)
app.config['SQLALCHEMY_DATABASE_URI'] = 'sqlite:////tmp/test.db'
db = SQLAlchemy(app)


class User(db.Model):
    id = db.Column(db.Integer, primary_key=True)
    username = db.Column(db.String(80), unique=True, nullable=False)
    email = db.Column(db.String(120), unique=True, nullable=False)

    def __repr__(self):
        return '<User %r>' % self.username

Now how can i migrate this code to AWS lambda . Is it even possible . For instance the line app = Flask(__name__) should not be there right ? If there is no app variable how am i going to initialize db variable ?

Please can someone give me some intro or a link to a good tutorial which will clear these concepts?

Many thanks in advance.

It doesn't answer your question, but you might want to consider using chalice as a Flask-like framework that gives you a solution for Lambda+APIGW apps. — dmulter
– dmulter, Commented Jun 28, 2018 at 16:59

weaver · Accepted Answer · 2019-10-11 15:30:16Z

To use a Flask/sqlalchemy app with Lambda, you need to wrap Flask in the Lambda dispatch model, and make sure sqlalchemy can access its database.

Dispatching Lambda requests to Flask

You can integrate Chalice with Flask like this:

class ChaliceWithFlask(chalice.Chalice):
    """
    Subclasses Chalice to host a Flask app, route and proxy requests to it.
    """
    def __init__(self, flask_app, *args, **kwargs):
        super().__init__(*args, **kwargs)
        self.flask_app = flask_app
        self.trailing_slash_routes = []
        routes = collections.defaultdict(list)
        for rule in self.flask_app.url_map.iter_rules():
            route = re.sub(r"<(.+?)(:.+?)?>", r"{\1}", rule.rule)
            if route.endswith("/"):
                self.trailing_slash_routes.append(route.rstrip("/"))
            routes[route.rstrip("/")] += rule.methods
        for route, methods in routes.items():
            self.route(route, methods=list(set(methods) - {"OPTIONS"}), cors=True)(self.dispatch)

    def dispatch(self, *args, **kwargs):
        uri_params = self.current_request.uri_params or {}
        path = self.current_request.context["resourcePath"].format(**uri_params)
        if self.current_request.context["resourcePath"] in self.trailing_slash_routes:
            if self.current_request.context["path"].endswith("/"):
                path += "/"
            else:
                return chalice.Response(status_code=requests.codes.found, headers={"Location": path + "/"}, body="")
        req_body = self.current_request.raw_body if self.current_request._body is not None else None
        base_url = "https://{}".format(self.current_request.headers["host"])
        query_string = self.current_request.query_params or {}
        with self.flask_app.test_request_context(path=path,
                                                 base_url=base_url,
                                                 query_string=list(query_string.items()),
                                                 method=self.current_request.method,
                                                 headers=list(self.current_request.headers.items()),
                                                 data=req_body,
                                                 environ_base=self.current_request.stage_vars):
            flask_res = self.flask_app.full_dispatch_request()
        res_headers = dict(flask_res.headers)
        res_headers.pop("Content-Length", None)
        res_body = b"".join([c for c in flask_res.response])
        return chalice.Response(status_code=flask_res._status_code, headers=res_headers, body=res_body)

flask_app = flask.Flask(app_name)
# add routes, etc. to your Flask app here
app = ChaliceWithFlask(app_name, flask_app=flask_app)

Connecting sqlalchemy to the database

You could access the database directly, but that means opening the database port to the Internet or placing your Lambda in a VPC (which makes it impossible for the Lambda to be accessible over the Internet). Also, traditional database drivers make assumptions about persistence of their connections that are not satisfied in Lambda.

AWS recently came out with the perfect solution for this - the AWS Aurora RDS Data API. It's basically an AWS-authenticated SQL-over-HTTP tunnel. I wrote a SQLAlchemy adapter for it: sqlalchemy-aurora-data-api. After installing it, you can do:

from sqlalchemy import create_engine

cluster_arn = "arn:aws:rds:us-east-1:123456789012:cluster:my-aurora-serverless-cluster"
secret_arn = "arn:aws:secretsmanager:us-east-1:123456789012:secret:MY_DB_CREDENTIALS"

app = Flask(app_name)
app.config['SQLALCHEMY_DATABASE_URI'] = 'postgresql+auroradataapi://:@/my_db_name'
engine_options=dict(connect_args=dict(aurora_cluster_arn=cluster_arn, secret_arn=secret_arn))
db = flask_sqlalchemy.SQLAlchemy(app, engine_options=engine_options)

This code does not work for me running inside a chalice/lambda function because sqlalchemy could not be installed (there is no whl) and therefore I had to include sqlalchemy manually. How did you get this to work?
See chalice.readthedocs.io/en/latest/topics/… for packaging instructions for packages that don't have wheels available.

Christian Lim · Accepted Answer · 2018-08-16 02:50:41Z

First of all, in AWS Lambda, you don't use your Flask for routing anymore. Instead use AWS API Gateway for routing. An example of the routing is shown below, from https://apievangelist.com/2017/10/23/a-simple-api-with-aws-dynamodb-lambda-and-api-gateway/

As you can see at the right end of the picture, the "Lambda" box shows the name of the Lambda function you have uploaded. For Lambda in Python, see https://docs.aws.amazon.com/lambda/latest/dg/python-programming-model-handler-types.html

Basically, the main thing in Python lambda is the:

def handler_name(event, context): 
   ...
   return some_value

From the event and context you can get everything: path, HTTP method, headers, params, body, etc (like flask.request). You might also need to know there are two ways of doing Lambda the LAMBDA and LAMBDA_PROXY (see the Integration Request box in the first picture).

Short version difference is:

LAMBDA mode will preprocess request body automatically and gives your Lambda function a Python object in event.
LAMBDA_PROXY will give you raw HTTP request, you need to convert the content yourself inside the Lambda function.

As for SQL Alchemy, all you need to do is to zip all the SQL Alchemy library code and its' dependency together with your Lambda function and upload it to the Lambda Console, it works without any modification.

Please note that SQLite will not work in Lambda, as Lambda function has no filesystem access. You should put the data somewhere else, e.g. Amazon RDS (with MySQL, PostgreSQL, whatever you like) and then make sure the Lambda is connected to the same VPC (Amazon internal router) with the RDS database.

Collectives™ on Stack Overflow

Deploying Flask sqlalchemy apps in AWS lambda and API gateway

2 Answers 2

Dispatching Lambda requests to Flask

Connecting sqlalchemy to the database

2 Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Dispatching Lambda requests to Flask

Connecting sqlalchemy to the database

2 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related