1

I am setting up an S3 bucket. In this S3 bucket, data is going to be written by an external process.

I am setting up an AWS Lambda that would be triggered when an object in S3 gets created/updated and would process and store the data in RDS.

Here my question is as follows:

If objects get written too fast on s3, there is a possibility for multiple Lambda functions to get triggered simulatenously.

So, in this case, is there any chance for the objects to be processed not in the order they are written to the S3 bucket?

If the answer to the above question is yes, then with Lambda, I have to push the payload to FIFO SQS and set up a listener to process the payload to store the data into RDS finally.

Improve this question

1 Answer 1

Reset to default
4

Sadly, they are not guaranteed to be in order. From docs:

Event notifications are not guaranteed to arrive in the order that the events occurred. However, notifications from events that create objects (PUTs) and delete objects contain a sequencer, which can be used to determine the order of events for a given object key.

Improve this answer
Sign up to request clarification or add additional context in comments.

2 Comments

In that case, for my use case, I think, I cannot do the DB operations inside Lambda. I have to take the help of the SQS FIFO queue, in order to enforce the ordering on the processing of events. I have to set up a listener on the queue where I have to implement the DB operations. Please correct me, if I am wrong.
@Joy Yes. FIFO guarantees the order, but only within the same message message group. So your msg producers must ensure that it pushes records to only group.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.