AWS Lambda arm64 not supporting SQLAlchemy for Python-3.12

Question

In my case below are the Lambda Runtime configurations.

Runtime - Python3.12 Architecture - arm64

I have attached 2 layers to my lambda function. Each layer has a set of packages as below.

layer1	layer2
SQLAlchemy==2.0.21	country-converter
aws-xray-sdk	urllib3
boto3	psycopg2-binary
botocore	psycopg2
aws-lambda-context==1.1.0
requests
cryptography

My function of the lambda is whenever it receives a request it call a RDS (postgres DB) to fetch the data, process it and put in queue. I am using SQLALchemy to establish the connection with DB. It seems I am facing an issue while running create_engine method and I am not sure what is the reason for that. In my cloudwatch I could see the below error always.

Runtime.ImportModuleError: Unable to import module 'sample.main': No module named 'psycopg2._psycopg

NOTE: All the above lambda, layer and queue creation are made through terraform So, previously I am using Python3.8 and Architecture as X86_64 processor. So, now as a part of recent changes in code I also made changes in Lambda Runtime configurations to Python3.12 and arm64.

I have tried for different versions of the package as well as tried for Python3.11 version I am ending up getting the same error.

As a part of Cost reduction, and makes code to execute faster we have decided to move to ARM. NOTE: For the same lambda I have used ARM processor with Python-3.8 version, and it could be effectively seen that the time taken for execution was considerably reduced (Tried for more than 10k executions). So, we have decided to move to ARM along with Python version upgrade. — Sivakarthik
– Sivakarthik, Commented Jan 17, 2024 at 10:26

Sivakarthik · Accepted Answer · 2024-08-06 02:14:45Z

0

When downloading SQLAlchemy using pip, our system was running on Python 3.10. We extracted the files and uploaded them to a layer specifying Python version 3.12, which was also used in our Lambda environment. This mismatch caused an error.

Solution: We upgraded our machine to Python 3.12 and tried again. This approach resolved the issue.

If you have any other suggestions or approaches to solve this issue, please share them in the comments. Thank you!

answered Aug 6, 2024 at 2:14

Sivakarthik

32 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

AWS Lambda arm64 not supporting SQLAlchemy for Python-3.12

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related