AWS Lambda - Deleting Recurring Failed Events From Lambda Queue

Question

So I have an few AWS lambda functions that are triggered by DynamoDB streams. I've used the 'configure test' functionality a few times, and I've triggered the function myself by just adding dummy data manually to the DynamoDB table.

My problem is that now I've got working code, but because I inserted test data that was wrong in the first place (doesn't match the look of the actual event data as it streams in), every time I update a table, or the lambda function itself, it won't process other events because it gets hung up on my old, bad, test data. As a lesser problem, it's cluttering up CloudWatch.

In theory, I could change my Lambda functions so that they DON'T work with actual data, and only work with my bad test data, and then once my real data starts showing up I could then switch my Lambda function back to normal, but I feel like there has to be a better way to do this that I'm just unaware of.

So I'm wondering if there is any way to:

1 - look at the queue of events that have failed in Lambda and are waiting to be re-processed?

2 - delete specific events that are in that queue so that they are NOT reprocessed by Lambda?

I guess this could possibly be phrased as "How do I delete things from DynamoDB streams" as well... — mrtannerjones
– mrtannerjones, Commented Jan 31, 2016 at 10:46

Luc Hendriks · Accepted Answer · 2016-02-01 13:29:10Z

4

The answer to both is basically no. The Lambda function is stuck because it cannot parse the stream and AWS retries failed functions (no way to turn that off).

You can delete the test data from dynamodb. Or, before you do anything with the Lambda function, do some validation checks on the actual event. Good practice to always do that, so as a bonus you get a more future proof Lambda function :)

If the validation check fails (on test data), just return context.succeed(); (the Lambda function can now move on instead of retry). If the validation passes, it does its thing.

The check could be something like:

exports.handler = function(event, context) {
    if (typeof event.somethingNotAvailableInTestAndAvailableOnLive === 'undefined') {
        context.succeed('Parsed test event...');
    }

    // Actual code...
}

You should check this for dynamodb specific syntax.

answered Feb 1, 2016 at 13:29

Luc Hendriks

2,54318 silver badges19 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Davos Over a year ago

If you delete the test data in DynamoDB, it creates a new Delete event in DynamoDB Streams :D If you try and update the record in DynamoDB it creates a new update event :D If you try to do anything at all it creates new events. Good advice about validating the event.

Collectives™ on Stack Overflow

AWS Lambda - Deleting Recurring Failed Events From Lambda Queue

1 - look at the queue of events that have failed in Lambda and are waiting to be re-processed?

2 - delete specific events that are in that queue so that they are NOT reprocessed by Lambda?

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 - look at the queue of events that have failed in Lambda and are waiting to be re-processed?

2 - delete specific events that are in that queue so that they are NOT reprocessed by Lambda?

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related