Using Python tempfiles permanently

Question

I'm currently abusing tempfile a little bit by using it to generate unique names for permanent files. I'm using the following function to get a unique id:

def gen_id():
    tf = tempfile.mktemp()
    tfname = os.path.split(tf)[1]
    return tfname.strip(tempfile.gettempprefix())

Then I'm storing a file in a custom directory with a filename from that function. I use this function to give me more flexibility than the built-ins; with this function I can choose my own directory and remove the tmp prefix.

Since tempfiles are supposed to be "temporary files," are there any dangers to using their uniqueness for permanent files like this? Any reasons why my function would not be safe for generating unique ids?

EDIT: I got the idea to use tempfile for unique names from this SO answer.

The directory will only be populated with tempfiles, though. Do you mean tempfile will look at the directory before generating a name? — Luke Taylor
– Luke Taylor, Commented May 7, 2016 at 20:42
Using the deprecated tempfiles.mktemp() function does not guarantee the uniqueness it sounds like you require. — martineau
– martineau, Commented May 7, 2016 at 20:52

TessellatingHeckler · Accepted Answer · 2016-05-07 20:49:57Z

3

help(tempfile.mktemp) -> "This function is unsafe and should not be used. The file name refers to a file that did not exist at some point, but by the time you get around to creating it, someone else may have beaten you to the punch."

i.e. you could get a filename from this that has the same name as an existing file.

The replacement is tempfile.mkstemp() and it actually does create a file which you normally have to remove after use ... but you can give it a parameter for where your custom directory is and tell it to use no prefix, and let it create the files for you full stop. And it will check for existing files of the same name and make new names until it finds an unused name.

tempfile.mkstemp(suffix="", prefix=template, dir=None, text=False)

(The tempfile module is written in Python, you can see the code for it in \Lib\tempfile.py )

answered May 7, 2016 at 20:49

TessellatingHeckler

29.3k4 gold badges55 silver badges96 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

TessellatingHeckler Over a year ago

I don't think it is deprecated. The comments in the mktemp() function explain what's wrong with it, and why it shouldn't be used, and say it's only still there for backwards compatibility with older code that was written to use it. The comments / help for mkstemp and mkdtemp say nothing like that, so I assume they're as current and valid as anything else - but I can't prove it.

Luke Taylor Over a year ago

OK. I see that now, deleted my other comment because I could just look in the docs to see that it wasn't. My one remaining question is: what does "There are no race conditions in the file’s creation" mean?

TessellatingHeckler Over a year ago

A race condition is a type of bug where two bits of code do the same thing at the same time, and it's luck which one works and which one crashes - a "race to the finish line". Here, two bits of code call mktemp and it could give the same random filename to both; they both try to open the file by that name, and one will luck out and work, the other will crash. Instead, to avoid the race, mkstemp doesn't return a filename until after it has successfully opened the file. Then there is no chance for both bits of code to get the same filename, no race condition, neither crashes.

Community · Accepted Answer · 2017-05-23 12:18:31Z

0

I highly suggest just using this comment from the same answer as the way of generating unique names.

No need to abuse mktemp for that (and it's deprecated anyway).

Keep in mind using mktemp guarantees the file name won't exist during the call but if you deleted all of your temp files and cache, or even immediately after the call, the same file (or in case of mktemp the same path) can be created twice.

Using a random choice has less chance in that case to cause collisions and has no downsides. You should however have a check for the small chance a collision occurs, in which case you should generate a new name.

edited May 23, 2017 at 12:18

CommunityBot

11 silver badge

answered May 7, 2016 at 20:50

Bharel

27.5k8 gold badges52 silver badges100 bronze badges

Collectives™ on Stack Overflow

Using Python tempfiles permanently

2 Answers 2

3 Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

3 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related