how to create uncollectable garbage in python?

Question

I have a large long-running server, and, over weeks the memory usage steadily climbs.

Generally, as pointed out below, its unlikely that leaks are my problem; however, I have not got a lot to go on so I want to see if there are any leaks.

Getting at console output is tricky so I'm not running with gc.set_debug(). This is not a big problem though, as I have easily added an API to get it to run gc.collect() and then iterate through gc.garbage and send the results back out to me over HTTP.

My problem is that running it locally for a short time my gc.garbage is always empty. I can't test my bit of code that lists the leaks before I deploy it.

Is there a trivial recipe for creating an uncollectable bit of garbage so I can test my code that lists the garbage?

Not an answer, but Giovanni Bajo gave an excellent talk about debugging memory leaks at EuroPython last month. The talk video is here: ep2011.europython.eu/conference/talks/… — grifaton
– grifaton, Commented Jul 1, 2011 at 10:39

Gareth Rees · Accepted Answer · 2011-07-01 10:55:27Z

13

Any cycle of finalizable objects (that is, objects with a __del__ method) is uncollectable (because the garbage collector does not know which order to run the finalizers in):

>>> class Finalizable:
...     def __del__(self): pass
...
>>> a = Finalizable()
>>> b = Finalizable()
>>> a.x = b
>>> b.x = a
>>> del a
>>> del b
>>> import gc
>>> gc.collect()
4
>>> gc.garbage
[<__main__.Finalizable instance at 0x1004e0b48>,
 <__main__.Finalizable instance at 0x1004e73f8>]

But as a general point, it seems unlikely to me that your problem is due to uncollectable garbage, unless you are in the habit of using finalizers. It's more likely due to the accumulation of live objects, or to fragmentation of memory (since Python uses a non-moving collector).

edited Jul 1, 2011 at 10:55

answered Jul 1, 2011 at 10:43

Gareth Rees

66.1k10 gold badges139 silver badges166 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

BugShotGG Over a year ago

Excuse me but i am trying to create garbage just to see how it works in python. In your example above what is the garbage? the instances of a and b before they have their reference deleted? pythontutor.com/…

Gareth Rees Over a year ago

See for yourself: print id(a) and id(b) before you delete them, and see if these ids match id(gc.garbage[0]) and id(gc.garbage[1]) after the collection.

Collectives™ on Stack Overflow

how to create uncollectable garbage in python?

1 Answer 1

2 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related