1

I was tried to connect into RDS postgresql database using Lambda function. Lambda function returning timed out error. Here is the source code.

postgres_test.py:

import sys
import logging
import psycopg2

from db_util import make_conn, fetch_data
def lambda_handler(event, context):
    query_cmd = "SELECT COUNT(*) FROM users"
    # print query_cmd

    # get a connection, if a connect cannot be made an exception will be raised here
    conn = make_conn()

    result = fetch_data(conn, query_cmd)
    conn.close()

    return result

db_util.py:

import psycopg2

db_host = "db_host"
db_port = 5432
db_name = "db_name"
db_user = "db_user"
db_pass = "db_pass"
db_table = "db_table"


def make_conn():
    conn = None
    try:
        conn = psycopg2.connect("dbname='%s' user='%s' host='%s' password='%s'" % (db_name, db_user, db_host, db_pass))
    except:
        print ("I am unable to connect to the database")
    return conn


def fetch_data(conn, query):
    result = []
    print ("Now executing: " + query)
    cursor = conn.cursor()
    cursor.execute(query)

    raw = cursor.fetchall()
    for line in raw:
        result.append(line)

    return result

here is the execution results:

Test Event Name
triggers3event

Response
{
  "errorMessage": "2021-10-19T22:55:12.543Z 207db48f-eb82-4bbd-afd5-1836b63874cf Task timed out after 3.00 seconds"
}

Function Logs
START RequestId: 207db48f-eb82-4bbd-afd5-1836b63874cf Version: $LATEST
END RequestId: 207db48f-eb82-4bbd-afd5-1836b63874cf
REPORT RequestId: 207db48f-eb82-4bbd-afd5-1836b63874cf  Duration: 3003.71 ms    Billed     Duration: 3000 ms    Memory Size: 128 MB Max Memory Used: 12 MB  
2021-10-19T22:55:12.543Z 207db48f-eb82-4bbd-afd5-1836b63874cf Task timed out after 3.00 seconds

Request ID
207db48f-eb82-4bbd-afd5-1836b63874cf

Pls help me to fix following issue. I am not sure what was wrong from above codes.

Improve this question

2 Answers 2

Reset to default
1

Usually the reason Lambda function times out is due to configuration of Security Groups.

In order to rectify, please follow these suggestions:

  1. Check whether the Lambda and corresponding db service is in same VPC

  2. Check whether the Security Group is allowing the connection

  3. If you are connecting the db over internet, check the connection. (If you are in VPC, see you are routing via NAT and IG)

Again, reiterating one more time, that many times, timed out issue is to due to network allow/deny, so in case of Lambda, a Security Group issue.

Improve this answer
Sign up to request clarification or add additional context in comments.

2 Comments

Sounds good. The db and lambda are not in same VPC, in this case, how can I access to the db using my lambda function? @Shashikant
Hi one way to connect is using VPC Peering, so that both VPC are connected and your Lambda can access your DB, Here is documentation from AWS about VPC Peering (docs.aws.amazon.com/vpc/latest/peering/what-is-vpc-peering.html) Second method, (It will be little expensive) is, by making DB public and accessing by over internet using NAT and IG in Lambda.
1

It might simply take your code more than 3 seconds to connect and retrieve data, so your Lambda function times out. You have the default setting of 3 seconds, as you can see in the CloudWatch Logs):

Task timed out after 3.00 seconds

Increase the timeout and the configured RAM size for the Lambda (which will give the Lambda function proportionally more CPU and make it run faster) and retry.

If that also fails, then the problem may be that your Lambda function has no network route to the RDS database. Having no route can cause the DB connection attempt to hang and the Lambda function to time out.

From the information you've provided in comments below, it looks like the RDS database is actually running in a different AWS account but is accessible publicly. This suggests that your Lambda function may be configured incorrectly. If you don't need the Lambda function to run in a VPC then don't configure it to run in VPC - see the Lambda function VPC configuration options and remove the subnet assignment(s).

Improve this answer

14 Comments

where can I increase the timeout and RAM size?
They are configured on the Lambda function. If you created it manually in the Lambda console then change the configuration there. If you used a serverless deployment tool, then go back to your Lambda function template, modify it, and re-deploy.
It's slightly annoying that lambda functions seem to default to such a low timeout (3 seconds) however I definitely understand the reasoning behind it. In most realistic scenarios, you probably want to set a timeout of 1-2 minutes, and note that won't otherwise affect how much you are billed (unless your task runs that long of course).
Incorrectly written or configured Lambda functions would then take much longer to fail, costing more money, and possibly making newcomers to the platform more annoyed. That said, the real solution is automation. You can automate the settings you want in your serverless deployment tool config.
@rv.kvetch the original intent of Lambdas is fast and very quickly firing functions. Though they have the capability to be run up to 15 mins with multiple cores, their original design intention is quick firing and low overhead/cost. So the default settings are arranged in this manner. You should aim to move away from using the Console to create resources, and get started with SAM, Cloudformation, or CDK to have far better and more granular control over your resources - and be able to define all these settings at stand up time
|

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.