Not all Python code executing in AWS Lambda function

Question

I have a simple lambda function which prints an event and then attempts to insert a row into a database. It runs with no error, but does not execute all of the code.

event gets printed, but the row never gets inserted into the table. Anything, even a print statement I put after connection doesn't get executed. I'm guessing something is wrong with the connection, but as far as I know I have no way of telling what is wrong. Are there more logs somewhere? In CloudWatch I see at the end it says Task timed out after 3.00 seconds

import boto3
import psycopg2

s3 = boto3.client('s3')


def insert_data(event=None, context=None):
    print(event)

    connection = psycopg2.connect(user="xxxx", password="xxxx",
                                  host="xxxx", port="xx",
                                  database="xxxx")

    cursor = connection.cursor()
    postgres_insert_query = "INSERT INTO dronedata (name,lat,long,other) VALUES ('img2','54','43','from lambda')"
    cursor.execute(postgres_insert_query)
    connection.commit()
    count = cursor.rowcount
    print(count, "Record inserted successfully into mobile table")

@Marcin Yes, I see that event gets printed and the last line it says Task timed out after 3.00 seconds. Does this just mean it is failing to reach the postgres database? Perhaps an connection/networking problem? I would have though I would have gotten an a different error. — franchyze923
– franchyze923, Commented Sep 24, 2020 at 22:47
If increasing the timeout does not help, then it is likely to be a problem communicating with the database. Check the configuration of the security groups on the database to ensure that inbound traffic is permitted. Is the database running on an instance, Amazon RDS or Amazon Redshift? — John Rotenstein
– John Rotenstein, Commented Sep 24, 2020 at 22:52
@JohnRotenstein That was it! I'm using Amazon RDS. When I set it up, I allowed inbound only from my IP. It was working when I was testing on my computer, but not lambda, understandably. Thank you for the help. Can you submit as answer so I can accept it. — franchyze923
– franchyze923, Commented Sep 24, 2020 at 22:56

John Rotenstein · Accepted Answer · 2020-09-24 23:37:00Z

3

The typical security setup is:

A security group on the AWS Lambda function (Lambda-SG) that permits all outbound access (no need for inbound rules)
A security group on the database (either an EC2 instance or Amazon RDS) (DB-SG) that permits inbound access on the appropriate port from Lambda-SG

That is, DB-SG should specifically reference Lambda-SG in its inbound rules.

answered Sep 24, 2020 at 23:37

John Rotenstein

273k28 gold badges456 silver badges541 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Marcin · Accepted Answer · 2020-09-24 22:50:33Z

1

Yes, you have to increase default Timeout from 3 seconds to more:

Timeout – The amount of time that Lambda allows a function to run before stopping it. The default is 3 seconds. The maximum allowed value is 900 seconds.

answered Sep 24, 2020 at 22:50

Marcin

241k16 gold badges315 silver badges368 bronze badges

3 Comments

franchyze923 Over a year ago

I increased it to 30 seconds but still did not work. I don't think it should take that long to insert a row.

jarmod Over a year ago

Secondary problem is connectivity to the DB. This will be either security groups (firewalls) or networking routing.

Marcin Over a year ago

@fpolig01 John already gave answer for the second issue :-)

Mrinal · Accepted Answer · 2020-09-27 19:11:29Z

0

hence psycopg2 is an external lib, please upload that lib along with your code into your Lambda Function. So Issue is, it is not able to connect, that's why you are facing a timeout issue.

answered Sep 27, 2020 at 19:11

Mrinal

1255 bronze badges

3 Comments

franchyze923 Over a year ago

It had to do with my security configuration on the RDS instance. Lambda was unable to connect to it. I already packaged psycopg2 (had to build it on an AMI ec2 instance)

Mrinal Over a year ago

Did you check, you lambda execution role, that it has EC2 access or Not?

Mrinal Over a year ago

Otherwise, you can use Layer Feature in Lambda Function

Collectives™ on Stack Overflow

Not all Python code executing in AWS Lambda function

3 Answers 3

Comments

3 Comments

3 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

Comments

3 Comments

3 Comments

Your Answer

Sign up or log in

Post as a guest

Related