85

I have been trying to understand how python weak reference lists/dictionaries work. I've read the documentation for it, however I cannot figure out how they work, and what they can be used for. Could anyone give me a basic example of what they do and an explanation of how they work?

(EDIT) Using Thomas's code, when i substitute obj for [1,2,3] it throws:

Traceback (most recent call last):
File "C:/Users/nonya/Desktop/test.py", line 9, in <module>
r = weakref.ref(obj)
TypeError: cannot create weak reference to 'list' object
Improve this question

4 Answers 4

Reset to default
103

Theory

The reference count usually works as such: each time you create a reference to an object, it is increased by one, and whenever you delete a reference, it is decreased by one.

Weak references allow you to create references to an object that will not increase the reference count.

The reference count is used by python's Garbage Collector when it runs: any object whose reference count is 0 will be garbage collected.

You would use weak references for expensive objects, or to avoid circle references (although the garbage collector usually does it on its own).

Usage

Here's a working example demonstrating their usage:

import weakref
import gc

class MyObject(object):
    def my_method(self):
        print 'my_method was called!'

obj = MyObject()
r = weakref.ref(obj)

gc.collect()
assert r() is obj #r() allows you to access the object referenced: it's there.

obj = 1 #Let's change what obj references to
gc.collect()
assert r() is None #There is no object left: it was gc'ed.
Improve this answer
Sign up to request clarification or add additional context in comments.

9 Comments

How does it work on list's/dictionary's, this was a great example on classes/functions though. :)
Could you please look at my edit. when substituting a list or dictionary it throws that error.
My bad, I totally overlooked this, you can't indeed create weakref to a list or another composite type - what is your exact purpose for which you'd need it?
I am working on an encryption method and i am trying to prevent memory leaks because i am passing values from list to list, and often times these lists are massive but i dont need them for very long. Any thoughts?
Python's weakref documentation suggests that you can create a weak reference to a list through subclassling : class WeakRefableList(list):pass. I'm however not sure that this will help you in what you want to do, proper scoping should be enough for the garbage collector to handle your problem. You can always use the gc mode to track down memory leaks if you want to.
|
61

Just want to point out that weakref.ref does not work for built-in list because there is no __weakref__ in the __slots__ of list. For example, the following code defines a list container that supports weakref.

import weakref

class weaklist(list):
    __slots__ = ('__weakref__',)

l = weaklist()
r = weakref.ref(l)
Improve this answer

3 Comments

Wow.. I use __slots__ a lot of times and this saved me some serious trouble :)
Is the setting of __slots__ really necessary? A simpler implementation of the class, which seems to work, is class weaklist(list):pass
@DonHatch, rather, if you set __slots__, it's mandatory that it have a __weakref__ element. Not having __slots__ at all is fine.
18

The point is that they allow references to be retained to objects without preventing them from being garbage collected.

The two main reasons why you would want this are where you do your own periodic resource management, e.g. closing files, but because the time between such passes may be long, the garbage collector may do it for you; or where you create an object, and it may be relatively expensive to track down where it is in the programme, but you still want to deal with instances that actually exist.

The second case is probably the more common - it is appropriate when you are holding e.g. a list of objects to notify, and you don't want the notification system to prevent garbage collection.

Improve this answer

3 Comments

Perhaps 3 reasons? third the breaking out of the cyclic references.
@dashesy has it right. The most common (arguably, the only safe) use case for weak references is to explicitly break reference cycles. Although the Python garbage collector can and eventually will break such cycles anyway, it's substantially less space- and time-intensive to prevent such cycles from forming in the first place. All other uses of weak references are fragile at best and should absolutely be treated with a modicum of fear and respect.
@CecilCurry You say "(arguably, the only safe)" and "All other uses of weak references are fragile at best"... that's quite a bold universal statement. Are you sure you mean that? If so, what's the argument you have in mind? A couple of counterexamples come to my mind; that is, use cases (not the two listed in this answer) that seem useful and non-fragile to me, but I fear it would be pointless to list them if you're attached to your position, since you can always dismiss each use case as unimportant.
6

Here is the example comparing dict and WeakValueDictionary:

class C: pass
ci=C()
print(ci)

wvd = weakref.WeakValueDictionary({'key' : ci})
print(dict(wvd), len(wvd)) #1
del ci
print(dict(wvd), len(wvd)) #0

ci2=C()
d=dict()
d['key']=ci2
print(d, len(d))
del ci2
print(d, len(d))

And here is the output:

<__main__.C object at 0x00000213775A1E10>
{'key': <__main__.C object at 0x00000213775A1E10>} 1
{} 0
{'key': <__main__.C object at 0x0000021306B0E588>} 1
{'key': <__main__.C object at 0x0000021306B0E588>} 1

Note how in the first case once we del ci the actual object will be also removed from the dictionary wvd.

In the case or regular Python dictionary dict class, we may try to remove the object but it will still be there as shown.

Note: if we use del, we do not to call gc.collect() after that, since just del effectively removes the object.

Improve this answer

1 Comment

"Note: if we use del, we do not to call gc.collect() after that, since just del effectively removes the object." To be clear, are you saying the del ci is guaranteed to gc the object (i.e. destroy it and reclaim its memory) immediately? This would be a surprise to me-- my understanding is that del ci only removes the reference from the variable name ci to the object; the object may be destroyed immediately in this case, since there are no other strong refs to it, or it may not, depending on the python implementation and other contextual details in the program.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.