python asyncronous thread exception handling

Question

I'm trying to implement a timeout functionality in Python.

It works by wrapping functions with a function decorator that calls the function as a thread but also calls a 'watchdog' thread that will raise an exception in the function thread after a specified period has elapsed.

It currently works for threads that don't sleep. During the do_rand call, I suspect the 'asynchronous' exception is actually being called after the time.sleep call and after the execution has moved beyond the try/except block, as this would explain the Unhandled exception in thread started by error. Additionally, the error from the do_rand call is generated 7 seconds after the call (the duration of time.sleep).

How would I go about 'waking' a thread up (using ctypes?) to get it to respond to an asynchronous exception ?

Or possibly a different approach altogether ?

Code:

# Import System libraries
import ctypes
import random
import sys
import threading
import time

class TimeoutException(Exception):
    pass

def terminate_thread(thread, exc_type = SystemExit):
    """Terminates a python thread from another thread.

    :param thread: a threading.Thread instance
    """
    if not thread.isAlive():
        return

    exc = ctypes.py_object(exc_type)
    res = ctypes.pythonapi.PyThreadState_SetAsyncExc(ctypes.c_long(thread.ident), exc)

    if res == 0:
        raise ValueError("nonexistent thread id")
    elif res > 1:
        # """if it returns a number greater than one, you're in trouble,
        # and you should call it again with exc=NULL to revert the effect"""
        ctypes.pythonapi.PyThreadState_SetAsyncExc(thread.ident, None)
        raise SystemError("PyThreadState_SetAsyncExc failed")

class timeout_thread(threading.Thread):
    def __init__(self, interval, target_thread):
        super(timeout_thread, self).__init__()
        self.interval     = interval
        self.target_thread = target_thread
        self.done_event = threading.Event()
        self.done_event.clear()

    def run(self):
        timeout = not self.done_event.wait(self.interval)
        if timeout:
            terminate_thread(self.target_thread, TimeoutException)

class timeout_wrapper(object):
    def __init__(self, interval = 300):
        self.interval = interval

    def __call__(self, f):
        def wrap_func(*args, **kwargs):
            thread = threading.Thread(target = f, args = args, kwargs = kwargs)
            thread.setDaemon(True)
            timeout_ticker = timeout_thread(self.interval, thread)
            timeout_ticker.setDaemon(True)
            timeout_ticker.start()
            thread.start()
            thread.join()
            timeout_ticker.done_event.set()
        return wrap_func

@timeout_wrapper(2)
def print_guvnah():
    try:
        while True:
            print "guvnah"

    except TimeoutException:
        print "blimey"

def print_hello():
    try:
        while True:
            print "hello"

    except TimeoutException:
        print "Whoops, looks like I timed out"

def do_rand(*args):
    try:
        rand_num   = 7 #random.randint(0, 10)
        rand_pause = 7 #random.randint(0,  5)
        print "Got rand: %d" % rand_num
        print "Waiting for %d seconds" % rand_pause
        time.sleep(rand_pause)
    except TimeoutException:
        print "Waited too long"

print_guvnah()
timeout_wrapper(3)(print_hello)()
timeout_wrapper(2)(do_rand)()

Community · Accepted Answer · 2017-05-23 12:15:16Z

4

The problem is that time.sleep blocks. And it blocks really hard, so the only thing that can actually interrupt it is process signals. But the code with it gets really messy and in some cases even signals don't work ( when for example you are doing blocking socket.recv(), see this: recv() is not interrupted by a signal in multithreaded environment ).

So generally interrupting a thread (without killing entire process) cannot be done (not to mention that someone can simply override your signal handling from a thread).

But in this particular case instead of using time.sleep you can use Event class from threading module:

Thread 1

from threading import Event

ev = Event()
ev.clear()

state = ev.wait(rand_pause) # this blocks until timeout or .set() call

Thread 2 (make sure it has access to the same ev instance)

ev.set() # this will unlock .wait above

Note that state will be the internal state of the event. Thus state == True will mean that it was unlocked with .set() while state == False will mean that timeout occured.

7 Comments

dilbert Over a year ago

Thanks for the response, however, the focus of this code is to be a library (hopefully). As a result, I really don't want to constrain the user code to exclude the use of time.sleep or any other blocking function. do_rand is just a test. I was hoping that maybe I could modify the PyThreadState using ctypes (or similar): gist.github.com/gdementen/….

freakish Over a year ago

@dilbert As I said: system calls such as recv() might (and will in case of blocking recv()) lock your thread indefinitely. And you won't be able to interrupt it without (force) killing entire process. Thus I think that you are going to far with that, the problem can't be solved. But the question you should ask is: is that a problem in the first place? Perhaps allowing blocking entire thread is not bad at all?

Eryk Sun Over a year ago

On Windows, only the main thread can be interrupted from time.sleep. It uses WaitForSingleObject on an event, while all other threads use an uninterruptibile Sleep. The select call used by time.sleep on POSIX systems can be interrupted using pthread_kill to target the thread.

dilbert Over a year ago

@freakish, that's a fair point. The idea of this code is terminate long-running tasks that violate developer expectation of execution time.

dilbert Over a year ago

@eryksun, on the one hand, thanks for the information but, on the other hand, that's very distressing (re the Windows non-main threads).

|

Benno · Accepted Answer · 2013-11-27 05:47:25Z

2

You'd need to use something other than sleep, or you'd need to send a signal to the other thread in order to make it wake up.

One option I've used is to set up a pair of file descriptors and use select or poll instead of sleep, this lets you write something to the file descriptor to wake up the other thread. Alternatively you just wear waiting until the sleep finishes if all you need is for the operation to error out because it took too long and nothing else is depending on it.

answered Nov 27, 2013 at 5:47

Benno

5,6402 gold badges28 silver badges31 bronze badges

6 Comments

dilbert Over a year ago

How would I go about sending the thread a signal ? And which signal?

Benno Over a year ago

The signal module will do it. The file descriptor method is probably more reliable though.

dilbert Over a year ago

I very much wish to avoid use of files. So there isn't anything particular about the signal, as in, any will do?

Benno Over a year ago

Well I'd avoid SIGTERM and SIGKILL. =) You may need to add a signal handler to avoid having the signal kill your process off. You're relying on the signal delivery interrupting the sleep on the thread you're expecting, too. This may not hold across all OSes.

Benno Over a year ago

Your problem is that the sleep stops the thread from running so it doesn't see the request to raise the exception until it comes out of the sleep. If you want an interruptible sleep you need to use the select/poll mechanism I mentioned or you could look into using a condition variable or suchlike. See the threading module.

|

Collectives™ on Stack Overflow

python asyncronous thread exception handling

2 Answers 2

7 Comments

6 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

7 Comments

6 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related