0

I am trying to remove an object from memory in python and I am coming across an object that it is not being removed. From my understanding if there is no references to the object the garbage collector will de-allocate the memory when it is run. However after I have removed all of the references if I run 

bar = Foo()
print gc.get_referrers(bar)
del bar
baz = gc.collect()
print baz

I get a reply of

[< frame object at 0x7f1eba291e50>]

0

So how come does it not delete the object?

I get the same reply for all of the instances of objects if i do 

bar = [foo() for i in range(0, 10)]
for x in range(0,len(bar))    
    baz = bar[x]
    del bar[x]
    print gc.get_referrers(baz)

How do I completely remove all referrers from an object/any idea what the frame object that is on all is?

I thought it would be the object frame(?) that contains a list of all objects in the program but I have not been able to confirm that/find a way to rid objects from being referenced by said mystical(to me) object fram.

Any help would be greatly appreciated

Edit: Okay I rewrote the code to the simple form pulling out everything except the basics

import random, gc
class Object():
  def __init__(self):
    self.n=None
    self.p=None
    self.isAlive=True
  def setNext(self,object):
    self.n=object
  def setPrev(self, object):
    self.p=object
  def getNext(self):
    return self.n
  def getPrev(self):
    return self.p
  def simulate(self):
      if random.random() > .90:
        self.isAlive=False
  def remove(self):
    if self.p is not None and self.n is not None:
      self.n.setPrev(self.p)
      self.p.setNext(self.n)
    elif self.p is not None:
      self.p.setNext(None)
    elif self.n is not None:
      self.n.setPrev(None)
    del self

class Grid():
  def __init__(self):
    self.cells=[[Cell() for i in range(0,500)] for j in range(0,500)]
    for x in range(0,100):
      for y in range(0,100):
        for z in range(0,100):
            self.cells[x][y].addObject(Object())
  def simulate(self):
      for x in range(0,500):
          for y in range(0,500):
              self.cells[x][y].simulate()
      num=gc.collect()
      print "  " + str(num) +" deleted today."
class Cell():
  def __init__(self):
    self.objects = None
    self.objectsLast = None
  def addObject(self, object):
    if self.objects is None:
        self.objects = object
    else:
        self.objectsLast.setNext(object)
        object.setPrev(self.objectsLast)
    self.objectsLast = object
  def simulate(self):
    current = self.objects
    while current is not None:
      if current.isAlive:
        current.simulate()
        current = current.getNext()
      else:
        delete = current
        current = current.getNext()
        if delete.getPrev() is None:
          self.objects = current
        elif delete.getNext() is None:
          self.objectsLast = delete.getPrev()
        delete.remove()
def main():
    print "Building Map..."
    x = Grid()
    for y in range (1,101):
        print "Simulating day " + str(y) +"..."
        x.simulate()
if __name__ == "__main__":
    main()
Improve this question
13
  • Why do you think the object isn't being freed? Commented Oct 3, 2016 at 19:14
  • is the object really deleted if another object references it? Commented Oct 3, 2016 at 19:16
  • I'm doing this on a large scale 90% of my memory is being taken up (20 million objects at a time) and if i run gc.collect() it does not drop my memory load even though I de-reference the object(appx 100,000 objects de-referenced before I run the garbage collector). After the gc i still have 90% of memory taken up Commented Oct 3, 2016 at 19:19
  • And no it is not deleted if another object references it(according to the gc for python). I'm trying to understand how there is still something referencing it if is deleted. so at a basic level this should delete an object: bar = foo(); del bar; gc.collect() - after that there should be no more bar in memory but it is not working for me Commented Oct 3, 2016 at 19:22
  • You most likely have some bug you're not showing us. Strip your code down to the bare minimum that 1) runs and 2) demonstrates the problem when you run it, then post it here. Commented Oct 3, 2016 at 19:23

2 Answers 2

Reset to default
2

gc.get_referrers takes one argument: the object whose referers it should find.

I cannot think of any circumstance in which gc.get_referrers would return no results (there are actually such situations, as Neil commented, because GC does not track everything, but that is not relevant here), because in order to send an object to gc.get_referrers, there has to be a reference to the object.

In other words, if there was no reference to the object, it would not be possible to send it to gc.get_referrers.

At the very least, there will be a reference from the globals() or from the current execution frame (which contains the local variables):

A code block is executed in an execution frame. An execution frame contains some administrative information (used for debugging), determines where and how execution continues after the code block's execution has completed, and (perhaps most importantly) defines two namespaces, the local and the global namespace, that affect execution of the code block.

See an extended version of the example from the question:

class Foo(object):
    pass

def f():
    bar = [Foo() for i in range(0, 10)]
    
    for x in range(0, len(bar)):
        # at this point there is one reference to bar[x]: it is bar
        print len(gc.get_referrers(bar[x]))  # prints 1
    
        baz = bar[x]
    
        # at this point there are two references to baz:
        #  - bar refernces it, because it is in the list
        #  - this "execution frame" references it, because it is in variable "baz"
        print len(gc.get_referrers(bar[x]))  # prints 2
    
        del bar[x]
    
        # at this point, only the execution frame (variable baz) references the object
        print len(gc.get_referrers(baz))  # prints 1
    
        print gc.get_referrers(baz) # prints a frame object 

        del baz

        # now there are no more references to it, but there is no way to call get_referrers

f()

How to test it properly?

There is a better trick to detect whether there are referers or not: weakref.

weakref module provides a way to create weak references to an object which do not count. What it means is that even if there is a weak reference to an object, it will still be deleted when there are no other references to it. It also does not count in the gc.get_referrers.

So:

>>> x = Foo()
>>> weak_x = weakref.ref(x)
>>>
>>> gc.get_referrers(x) == [globals()]  # only one reference from global variables
True

>>> x
<__main__.Foo object at 0x000000000272D2E8>

>>> weak_x
<weakref at 0000000002726D18; to 'Foo' at 000000000272D2E8>

>>> del x
>>> weak_x
<weakref at 0000000002726D18; dead>

The weak reference says that the object is dead, so it was indeed deleted.

Improve this answer
Sign up to request clarification or add additional context in comments.

3 Comments

Thanks this is helpful in understanding why I was having the problem originally!
It's definitely possible to get gc.referrers to return no objects, since it only returns objects that are tracked by the garbage collector. For example: import gc a = {"a": object()} assert not gc.is_tracked(a) print(gc.get_referrers(a["a"]))
True. When I wrote "I cannot think of any circumstance in which gc.get_referrers would return no results" 8 years ago, I really could not think of an edge case where that could happen. There are, in fact, many such situations, but I would still call them all edge cases. The list of Foo objects from the question is not one of them, though.
0

Okay thanks to cjhanks and user2357112 I came up with this answer

The problem being that if you run the program the gc does not collect anything after each day even though there were things deleted

To test if it is deleted I instead run

print len(gc.get_objects())

each time I go through a "day" doing this shows how many objects python is tracking. Now with that information and thanks to a comment I tired changing Grid to

class Grid():
  def __init__(self):
    self.cells=[[Cell() for i in range(0,500)] for j in range(0,500)]
    self.add(100)
  def add(self, num):
      for x in range(0, 100):
          for y in range(0, 100):
              for z in range(0, num):
                  self.cells[x][y].addObject(Object())
  def simulate(self):
      for x in range(0,500):
          for y in range(0,500):
              self.cells[x][y].simulate()
      num=gc.collect()
      print "  " + str(num) +" deleted today."
      print len(gc.get_objects())

and then calling Grid.add(50) halfway through the process. My memory allocation for the program did not increase (watching top in Bash) So my learning points:

  • GC was running without my knowledge
  • Memory is allocated and never returned to system until the the program is done
  • Python will reuse the memory
Improve this answer

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.