I am planning to invoke AWS Lambda function by modifying objects on AWS S3 bucket. I also need to send a large amount of data to the AWS Lambda function. How can I send the data to it in an efficient way?
3
-
What do you mean by "send" large data?helloV– helloV2016-01-04 07:39:07 +00:00Commented Jan 4, 2016 at 7:39
-
Let's say, I will retrieve the data from the database and want to send them to lambda function.anik_s– anik_s2016-01-04 07:40:55 +00:00Commented Jan 4, 2016 at 7:40
-
Why not have the Lambda function retrieve the additional data from the database?garnaat– garnaat2016-01-04 10:45:46 +00:00Commented Jan 4, 2016 at 10:45
Add a comment
|
3 Answers
I would use another S3 bucket to first send the data and then use it from the Lambda function
Comments
Your Lambda function should just read from the database your large data resides in.
Assuming your modified object on S3 contains - inside the object or as the object name - some type of foreign key to the data you need out of your database:
A) If your Lambda has access to the database directly: then you can just make your lambda function query your database directly to pull the data.
B) If your Lambda does not have direct access to the database: Then consider cloning the data as needed from the database to a secure S3 bucket for access by your lambda's when they are triggered/need it. Clone the data to S3 as JSON or some other easy to read format as logical objects for your business case (orders, customers, whatever). This method will be the fastest/most efficient for the Lambda if its possible for your use case.
Comments
I recently did this by gzipping the data before invoking the lambda function. This is super easy to do with most programming languages. Depending on your database content this will be a better or worse solution. The content of my database had a lot of data repetition and zipped very nicely.
