1

I have a Python program that uses fairly long-lived synchronous calls so things are wrapped in threads. Here's a rough MVP:


class MyClass:
    processor = None

    def __init__(self):
        self.processor_active = threading.Event()
        self.processor_thread = threading.Thread(target=self.start_processing_thread, daemon=True)
        self.processor_thread.start()
        self.processor_active.wait()

    def start_processing_thread(self):
        self.processor = MyProcessor() # this line takes a while to run
        self.processor_active.set() # allows the constructor to release
        while self.processor_active.is_set():
            data = processor.process() # this line can take arbitrarily long to execute
            print(f"Received data: {data}")

    def shutdown(self):
        self.processor.shutdown() # to clean up resources
        self.processor_active.clear()
        self.processor_thread.join()

My problem: calling shutdown on a MyClass instance hangs forever without shutting down, or it prints "Received data: " repeatedly. It seems my thread cannot get the current state of the threading Event. How do I fix it?

Improve this question
3
  • Are you certain that self.processor.shutdown() completes? It that call doesn't return (perhaps it hangs), then of course self.processor_active.clear() won't be reached. This is why complete, executable code is wanted in questions. Else it's too often a game of blind guessng. Commented Apr 20, 2024 at 4:47
  • It continues to hang even after I remove self.processor.shutdown() Commented Apr 20, 2024 at 4:55
  • See the complete code I posted. Runs fine for me. If it doesn't for you, then there's something crucial you haven't told us. Commented Apr 20, 2024 at 5:02

1 Answer 1

Reset to default
1

Pursuant to my comment, here's complete, executable code:

import threading
import time

class MyClass:
    processor = None

    def __init__(self):
        self.processor_active = threading.Event()
        self.processor_thread = threading.Thread(target=self.start_processing_thread, daemon=True)
        self.processor_thread.start()
        self.processor_active.wait()

    def start_processing_thread(self):
        time.sleep(5)
        self.processor_active.set() # allows the constructor to release
        while self.processor_active.is_set():
            time.sleep(5)
            print(f"Received data")

    def shutdown(self):
        #self.processor.shutdown() # to clean up resources
        self.processor_active.clear()
        self.processor_thread.join()

o = MyClass()

It runs fine for me. Does it for you? I ran it in a shell with python -i. After letting it print "Received data" a few times, I entered o.shutdown() by hand, and it shut down cleanly.

So the problem is in something you didn't show us. My best blind guess was given in the comment: seems most likely that the hang is actually in the self.processor.shutdown() line. But we have no way to know anything about that.

EDIT

Or if you don't want to muck with an interactive shell, replace the tail end of the code with:

o = MyClass()
time.sleep(32)
print("shutting down")
o.shutdown()

Then you just sit back and let it run. Same outcome for me: some seconds after it prints "shutting down", the program finishes.

Improve this answer
Sign up to request clarification or add additional context in comments.

3 Comments

I think it's because (A) processor.shutdown() does hang like you said, and (b) processor.process() takes arbitrarily long to execute and only does so when there's something to execute, so the while condition doesn't get checked often enough
The issue still remains that the processor object leaks resources if it doesn't get deleted, so I'll try to get it garbage collected?
Unless you can post focused code other people can run, you're unlikely to get useful help on something as massively underspecified as "leaks resources" ;-) We don't know anything relevant about what your code is doing. so impossible to guess whether gc is relevant.. You can certainly try doing gc.collect() at times, but I doubt it will matter.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.