6

It is typical to use the with statement to open a file so that the file handle cannot be leaked:

with open("myfile") as f:
    …

But what if the exception occurs somewhere within the open call? The open function is very likely not an atomic instruction in the Python interpreter, so it's entirely possible that an asynchronous exception such as KeyboardInterrupt would be thrown* at some moment before the open call has finished, but after the system call has already completed.

The conventional way of handle this (in, for example, POSIX signals) to use the masking mechanism: while masked, the delivery of exceptions is suspended until they are later unmasked. This allows operations such as open to be implemented in an atomic way. Does such a primitive exist in Python?


[*] One might say it's doesn't matter for KeyboardInterrupt since the program is about to die anyway, but that is not true of all programs. It's conceivable that a program might choose to catch KeyboardInterrupt on the top level and continue execution, in which case the leaked file handle can add up over time.

4
  • 1
    Is this about exceptions or signals ? Commented Jun 19, 2015 at 3:35
  • The onus lies with open to succeed, or fail and tidy up. You've got nothing to worry about. Commented Jun 19, 2015 at 4:36
  • @DavidHeffernan: that may well be fine for open, but that doesn't really answer the question of how one implements similar operations in an exception-safe way. Commented Jun 19, 2015 at 4:55
  • 1
    You are imagining a problem that does not exist. Commented Jun 19, 2015 at 5:01

2 Answers 2

2

I do not think its possible to mask exceptions , you can mask signals but not exceptions . In your case KeyboardInterrupt is the exception that is raised when the signal.SIGINT is raised (which is the Ctrl + C) .

It is not possible to mask Exceptions because well it does not make sense, right? Let's say you are doing open('file','r') , but file does not exist, this causes the open function to throw IOError Exception, we should not be able to mask these kinds of exceptions. It does not make sense to mask it, because open would never be able to complete in the above case.

exceptions – anomalous or exceptional conditions requiring special processing

For KeyboardInterrupt exception , its different because like I said, its actually a signal that causes the KeyboardInterrupt exception to be raised.

You can only mask signals in Unix starting from Python 3.3 using the function signal.pthread_sigmask [Reference]

For that you will have to move the the context expression to a different block so that we can so something like mask the signal, run the context expression to get the context manager and then unmask the signal , a sample code would look like (please note I have not personally tested this code) -

import signal
signal.pthread_sigmask(signal.SIG_BLOCK,[signal.SIGINT])
with <context expression> as variable:  # in your case ,open('filename','r')

    signal.pthread_sigmask(signal.SIG_UNBLOCK,[signal.SIGINT])
...
Sign up to request clarification or add additional context in comments.

7 Comments

While masking synchronous exceptions is not very useful, masking asynchronous ones is perfectly sensible since they can occur anywhere in the code. Masking provides a mechanism for suspending them in critical parts where atomicity is required. (Masking signals is of course always an option, but that's not portable.) Also, your example is incorrect: the unmasking needs to take place after with otherwise the signal will be delivered before the context manager reaches with.
Do you mean unmasking needs to happen after complete with block? I thought the requirement was to unmask the signal during context manager creation portion, for that my example is correct, but if the masking if for complete with block, then the unmasking needs to happen after with block.
Also, can you please give one example of asynchronous exception?
The unmasking needs to happen just as the block begins. If it happens before the block begins then the signal gets delivered and therefore the context_manager's __exit__ will never be executed.
That being said, it seems KeyboardInterrupt is the oddball here. Python by default does not seem to use exceptions asynchronously except for this one case (unless the user or some library invents their own), so perhaps there is a workaround to this …
|
1

Some clarification: it seems that asynchronous exceptions are not commonly used in Python. The standard library only documents KeyboardInterrupt AFAIK. Other libraries can implement their own via signal handlers, but I don't think (or hope?) this is a common practice, as asynchronous exceptions are notoriously tricky to work with.

Here is a naive solution that won't work:

try:
    handle = acquire_resource(…)
except BaseException as e:
    handle.release()
    raise e
else:
    with handle:
        …
  • The exception-handling part is still vulnerable to exceptions: a KeyboardInterrupt could occur a second time after the exception is caught but before release is complete.

  • There is also a "gap" between the end of the try statement and the beginning of the with statement where it is vulnerable to exceptions.

I don't think there's anyway to make it work this way.

Thinking from a different perspective, it seems that the only way in which asynchronous exceptions can arise is from signals. If this is true, one could mask them as @AnandSKumar suggested. However, masking is not portable as it requires pthreads.

Nonetheless, we can fake masking with a little trickery:

def masking_handler(queue, prev_handler):
    def handler(*args):
        queue.append(lambda: prev_handler[0](*args))
    return handler

mask_stack = []

def mask():
    queue = []
    prev_handler = []
    new_handler = masking_handler(queue, prev_handler)
    # swap out the signal handler with our own
    prev_handler[0] = signal.signal(signal.SIGINT, new_handler)
    mask_stack.append((prev_handler[0], queue))

def unmask():
    prev_handler, queue = mask_stack.pop()
    # restore the original signal handler
    signal.signal(signal.SIGINT, prev_handler)
    # the remaining part may never occur if a signal happens right now
    # but that's fine
    for event in queue:
        event()

mask()
with acquire_resource(…) as handle:
    unmask()
    …

This will work if SIGINT is the only source that we care about. Unfortunately it breaks down for multiple signals, not just because we don't know which ones are being handled, but also because we can't swap out multiple signals atomically!

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.