Need to write to a file in AWS Lambda AND access it in the next execution

Question

So, the function I have written involves saving a string to a text file at the end of each execution, and then calling that string in the next. It works fine locally, but not on Lambda. From what I can find, it looks like Lambda only supports writing to files in a /tmp folder, but it would only last for one execution, which doesn't work for me. How do I go about this?

Here is the relevant code, it's part of a tweepy bot.

def reply():
    with open('lastid.txt', 'r+') as mf:
        last = mf.readline()
        response = client.get_users_mentions(user_id, since_id=last)
        for tweet in response.data:
            text = tweet.text
            id = tweet.id
            if any(x in text.lower() for x in strings):
                client.create_tweet(in_reply_to_tweet_id=id, text=greet)
            else:
                client.create_tweet(in_reply_to_tweet_id=id, text=reply)
            mf.write(str(tweet.id))

Thank you very much for any help! Maybe the answer is obvious but honestly my brain is a little fried at the moment.

It's entirelly possible before the next execution your lambda instance terminates and a new one is spawned (or the next execution just runs on a new one without the old one terminating). You have very little control over this. — apokryfos
– apokryfos, Commented Sep 10, 2022 at 5:05
Also, for what it's worth, that code will only ever store the first line from lastid.txt in the last variable. — Anon Coward
– Anon Coward, Commented Sep 10, 2022 at 21:08

Vasu Kandimalla · Accepted Answer · 2022-09-10 05:00:28Z

1

You can write to an s3 file and use it in the next execution. Before that, you need to give access to your lambda function to access the s3 location.

answered Sep 10, 2022 at 5:00

Vasu Kandimalla

1008 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Mark B · Accepted Answer · 2022-09-10 12:46:58Z

0

Files written to /tmp aren't necessarily removed between executions. They can be retained across multiple Lambda executions if container reuse happens.

However, there is no guarantee that this will happen. And if you want to retain the file across new Lambda function version deployments then it isn't possible at all using /tmp.

Your options are:

Have your function write the file to S3 at the end of each execution, and read it from S3 at the beginning of each execution.
Add an EFS volume to your Lambda function for use as a persistent data volume.

Given your use case I would suggest the EFS solution.

answered Sep 10, 2022 at 12:46

Mark B

203k27 gold badges339 silver badges332 bronze badges

Collectives™ on Stack Overflow

Need to write to a file in AWS Lambda AND access it in the next execution

2 Answers 2

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related