I'm learning about AWS, and very outside my element, which is typically embedded software. I'm porting this question from Software Engineering after several days of no traction there.
Problem/question: in trying to set up a S3 bucket where file additions trigger a Lambda, how can one debug or trace the sequence of events from file addition to SNS notification to Lambda execution?
I'm trying to set up the following:
- Someone adds something to a S3 bucket
- The file addition triggers a Lambda which adds the file to a git repo (More specifically: the file addition triggers a SNS notification on a topic to which a Lambda is subscribed -- I felt this was "better" than the S3 bucket directly invoking the Lambda because of the decoupling)
I'm using the AWS CLI for everything, and for the immediate future, I am limited to using LocalStack to substitute for "real" AWS.
What I've done so far (simplified):
- Created my bucket (
aws s3 mb my-bucket --endpoint-url=http://localhost:4572) - Created a "hello-world" Lambda with attached role-policy
aws:policy/AWSLambdaFullAccess - Added permission to the Lambda (
aws lambda add-permission --function-name first_lambda --action lambda:InvokeFunction --statement-id sns-invoke-lambda --principal sns.amazonaws.com --endpoint-url=http://localhost:4574) - Created a SNS topic (
aws sns create-topic --name my-topic --endpoint-url=http://localhost:4575) - Subscribed my Lambda to the topic (
aws sns subscribe --topic-arn arn:aws:sns:us-east-1:000000000000:my-topic --protocol lambda --notification-endpoint arn:aws:lambda:us-east-1:000000000000:function:first_lambda --endpoint-url=http://localhost:4575) - Configured a put-bucket-notification-configuration (
aws s3api put-bucket-notification-configuration --bucket my-bucket --notification-configuration file://s3-ObjectCreated_notify.json --endpoint-url=http://localhost:4572) - Verified being able to upload and download to/from my bucket
- Verified being able to manually invoke my Lambda (
aws lambda invoke --function-name first_lambda outfile.txt --endpoint-url=http://localhost:4574)
This is my "hello-world" Lambda function, naively cobbled together from here and here:
import json
def lambda_handler(event, context):
print("Hello from Lambda!")
return {
'statusCode': 200,
'body': json.dumps('Hello from Lambda!')
}
This is my put-bucket-notification-configuration file:
{
"TopicConfigurations": [
{
"TopicArn": "arn:aws:sns:us-east-1:000000000000:my-topic",
"Events": [
"s3:ObjectCreated:*"
]
}
]
}
My problem is this: I'm not sure how to test success or failure of the "pipeline" I've tried to build from S3 Bucket to Lambda invocation.
(Please correct misunderstanding:) if I'm manually invoking a Lambda function, the "context of execution" is the shell from which I issue the AWS CLI request. But when the "context of execution" is the "pipeline" that I built from S3 Bucket to SNS notification to Lambda, I don't understand where the Lambda's print statement or return-status will be directed to. I.e. when I invoke the Lambda manually, my shell gets the return-status, and the print statement seems to get directed to the outfile I specify; e.g.:
$ aws lambda invoke --function-name first_lambda outfile.txt --endpoint-url=http://localhost:4574
{
"StatusCode": 200
}
...but how can I test end-to-end functionality, or failure, between bucket object-creation to SNS notification to Lambda invocation since there's no outfile for prints to be directed to, and no "context" to which to return a status to?