Example of "reference loop" which can prevent "del()" from being called?

Question

(I'm using Python 3.6.5)

I'm debugging a production problem which seems to be caused by __del__ not being called. I added debug messages in both __init_ and __del__ and it turned out sometimes (when the product problem was hit) __del__ is not called.

I Googled a lot and based on my understanding there are 2 cases where __del__ may not be called:

os._exit()
Cyclic reference (or reference loop)

I'm quite sure we did not call os._exit() so I suspect the problem was most likely caused by some reference loop but I failed to figure out what a reference loop would look like. So could someone help give an example of reference loop which can prevent __del__ from being called?

Daemon threads can also prevent __del__ from executing, as can force-killing a program with kill or similar means. — user2357112
– user2357112, Commented Apr 29, 2020 at 2:53
Reference cycles became irrelevant to this staring in Python 3.4, so that's not it: python.org/dev/peps/pep-0442 — Tim Peters
– Tim Peters, Commented Apr 29, 2020 at 3:09
So, are you sure you aren't just keeping some reference around? Why do you believe __del__ should be called? — juanpa.arrivillaga
– juanpa.arrivillaga, Commented Apr 29, 2020 at 3:14
@pynexj yes, but __del__ is not guaranteed to be called if the object is still alive when the interpreter exists. This is in the docs — juanpa.arrivillaga
– juanpa.arrivillaga, Commented Apr 29, 2020 at 3:23

pynexj · Accepted Answer · 2020-06-17 08:41:59Z

1

Seems like the PEP 0442 (Python 3.4) mentioned in the comment did not work for all scenarios. Just found an example from here (tested with Python 3.5.3):

[STEP 101] # cat ref_loop.py
class Foo():
    def __init__(self):
        print("now in __init__")
        self.foo = self

    def __del__(self):
        print("now in __del__")

foo = Foo()
exit()
[STEP 102] # python3 ref_loop.py
now in __init__
[STEP 103] # python3 --version
Python 3.5.3
[STEP 104] #

My another Python 3.6.5 can also reproduce it.

Note that if exit() is removed then __del__() will be called though I don't understand why.

edited Jun 17, 2020 at 8:41

answered Apr 29, 2020 at 7:24

pynexj

21.3k6 gold badges45 silver badges63 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Tim Peters Over a year ago

__del__ is run under 3.7.5 and 3.8.1 - at least on my box ;-) I don't have older versions in the 3 series available. As @juanpa.arrivillaga said, there's no guarantee __del__ will run for objects present at exit - interpreter shutdown is extremely complicated. It does tend to become more thorough over time, though.

Tim Peters Over a year ago

BTW, for things that absolutely must be run at (a clean) shutdown, the atexit module's facilities have always been used. But what if you want to run a cleanup action in case of garbage collection, or shutdown, whichever comes first? There was no easy answer to that before 3.4 added weakref.finalize(). See the weakref docs for that, and don't miss the "Comparing finalizers with __del__() methods" section at the end.

Collectives™ on Stack Overflow

Example of "reference loop" which can prevent "del()" from being called?

1 Answer 1

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related