\section{\module{weakref} ---         Weak references} \declaremodule{extension}{weakref}\modulesynopsis{Support for weak references and weak dictionaries.}\moduleauthor{Fred L. Drake, Jr.}{fdrake@acm.org}\moduleauthor{Neil Schemenauer}{nas@arctrix.com}\moduleauthor{Martin von L\"owis}{martin@loewis.home.cs.tu-berlin.de}\sectionauthor{Fred L. Drake, Jr.}{fdrake@acm.org} \versionadded{2.1} % When making changes to the examples in this file, be sure to update% Lib/test/test_weakref.py::libreftest too! The \module{weakref} module allows the Python programmer to create\dfn{weak references} to objects. In the following, the term \dfn{referent} means theobject which is referred to by a weak reference. A weak reference to an object is not enough to keep the object alive:when the only remaining references to a referent are weak references,garbage collection is free to destroy the referent and reuse its memoryfor something else.  A primary use for weak references is to implementcaches or mappings holding large objects, where it's desired that alarge object not be kept alive solely because it appears in a cache ormapping.  For example, if you have a number of large binary image objects,you may wish to associate a name with each.  If you used a Pythondictionary to map names to images, or images to names, the image objectswould remain alive just because they appeared as values or keys in thedictionaries.  The \class{WeakKeyDictionary} and\class{WeakValueDictionary} classes supplied by the \module{weakref}module are an alternative, using weak references to construct mappingsthat don't keep objects alive solely because they appear in the mappingobjects.  If, for example, an image object is a value in a\class{WeakValueDictionary}, then when the last remainingreferences to that image object are the weak references held by weakmappings, garbage collection can reclaim the object, and its correspondingentries in weak mappings are simply deleted. \class{WeakKeyDictionary} and \class{WeakValueDictionary} use weakreferences in their implementation, setting up callback functions onthe weak references that notify the weak dictionaries when a key or valuehas been reclaimed by garbage collection.  Most programs should find thatusing one of these weak dictionary types is all they need -- it'snot usually necessary to create your own weak references directly.  Thelow-level machinery used by the weak dictionary implementations is exposedby the \module{weakref} module for the benefit of advanced uses. Not all objects can be weakly referenced; those objects which caninclude class instances, functions written in Python (but not in C),methods (both bound and unbound), sets, frozensets, file objects,generators, type objects, DBcursor objects from the \module{bsddb} module,sockets, arrays, deques, and regular expression pattern objects.\versionchanged[Added support for files, sockets, arrays, and patterns]{2.4} Several builtin types such as \class{list} and \class{dict} do notdirectly support weak references but can add support through subclassing: \begin{verbatim}class Dict(dict):    pass obj = Dict(red=1, green=2, blue=3)   # this object is weak referencable\end{verbatim} Extension types can easily be made to support weak references; see``\ulink{Weak Reference Support}{../ext/weakref-support.html}'' in\citetitle[../ext/ext.html]{Extending and Embedding the PythonInterpreter}.% The referenced section used to appear in this document with the% \label weakref-extension.  It would be good to be able to generate a% redirect for the corresponding HTML page (weakref-extension.html)% for on-line versions of this document. \begin{classdesc}{ref}{object\optional{, callback}}  Return a weak reference to \var{object}.  The original object can be  retrieved by calling the reference object if the referent is still  alive; if the referent is no longer alive, calling the reference  object will cause \constant{None} to be returned.  If \var{callback} is  provided and not \constant{None}, and the returned weakref object is  still alive, the callback will be called when the object is about to be  finalized; the weak reference object will be passed as the only  parameter to the callback; the referent will no longer be available.   It is allowable for many weak references to be constructed for the  same object.  Callbacks registered for each weak reference will be  called from the most recently registered callback to the oldest  registered callback.   Exceptions raised by the callback will be noted on the standard  error output, but cannot be propagated; they are handled in exactly  the same way as exceptions raised from an object's  \method{__del__()} method.   Weak references are hashable if the \var{object} is hashable.  They  will maintain their hash value even after the \var{object} was  deleted.  If \function{hash()} is called the first time only after  the \var{object} was deleted, the call will raise  \exception{TypeError}.   Weak references support tests for equality, but not ordering.  If  the referents are still alive, two references have the same  equality relationship as their referents (regardless of the  \var{callback}).  If either referent has been deleted, the  references are equal only if the reference objects are the same  object.   \versionchanged[This is now a subclassable type rather than a                  factory function; it derives from \class{object}]                  {2.4}\end{classdesc} \begin{funcdesc}{proxy}{object\optional{, callback}}  Return a proxy to \var{object} which uses a weak reference.  This  supports use of the proxy in most contexts instead of requiring the  explicit dereferencing used with weak reference objects.  The  returned object will have a type of either \code{ProxyType} or  \code{CallableProxyType}, depending on whether \var{object} is  callable.  Proxy objects are not hashable regardless of the  referent; this avoids a number of problems related to their  fundamentally mutable nature, and prevent their use as dictionary  keys.  \var{callback} is the same as the parameter of the same name  to the \function{ref()} function.\end{funcdesc} \begin{funcdesc}{getweakrefcount}{object}  Return the number of weak references and proxies which refer to  \var{object}.\end{funcdesc} \begin{funcdesc}{getweakrefs}{object}  Return a list of all weak reference and proxy objects which refer to  \var{object}.\end{funcdesc} \begin{classdesc}{WeakKeyDictionary}{\optional{dict}}  Mapping class that references keys weakly.  Entries in the  dictionary will be discarded when there is no longer a strong  reference to the key.  This can be used to associate additional data  with an object owned by other parts of an application without adding  attributes to those objects.  This can be especially useful with  objects that override attribute accesses.   \note{Caution:  Because a \class{WeakKeyDictionary} is built on top        of a Python dictionary, it must not change size when iterating        over it.  This can be difficult to ensure for a        \class{WeakKeyDictionary} because actions performed by the        program during iteration may cause items in the dictionary        to vanish "by magic" (as a side effect of garbage collection).}\end{classdesc} \class{WeakKeyDictionary} objects have the following additionalmethods.  These expose the internal references directly.  Thereferences are not guaranteed to be ``live'' at the time they areused, so the result of calling the references needs to be checkedbefore being used.  This can be used to avoid creating references thatwill cause the garbage collector to keep the keys around longer thanneeded. \begin{methoddesc}{iterkeyrefs}{}  Return an iterator that yields the weak references to the keys.  \versionadded{2.5}\end{methoddesc} \begin{methoddesc}{keyrefs}{}  Return a list of weak references to the keys.  \versionadded{2.5}\end{methoddesc} \begin{classdesc}{WeakValueDictionary}{\optional{dict}}  Mapping class that references values weakly.  Entries in the  dictionary will be discarded when no strong reference to the value  exists any more.   \note{Caution:  Because a \class{WeakValueDictionary} is built on top        of a Python dictionary, it must not change size when iterating        over it.  This can be difficult to ensure for a        \class{WeakValueDictionary} because actions performed by the        program during iteration may cause items in the dictionary        to vanish "by magic" (as a side effect of garbage collection).}\end{classdesc} \class{WeakValueDictionary} objects have the following additionalmethods.  These method have the same issues as the\method{iterkeyrefs()} and \method{keyrefs()} methods of\class{WeakKeyDictionary} objects. \begin{methoddesc}{itervaluerefs}{}  Return an iterator that yields the weak references to the values.  \versionadded{2.5}\end{methoddesc} \begin{methoddesc}{valuerefs}{}  Return a list of weak references to the values.  \versionadded{2.5}\end{methoddesc} \begin{datadesc}{ReferenceType}  The type object for weak references objects.\end{datadesc} \begin{datadesc}{ProxyType}  The type object for proxies of objects which are not callable.\end{datadesc} \begin{datadesc}{CallableProxyType}  The type object for proxies of callable objects.\end{datadesc} \begin{datadesc}{ProxyTypes}  Sequence containing all the type objects for proxies.  This can make  it simpler to test if an object is a proxy without being dependent  on naming both proxy types.\end{datadesc} \begin{excdesc}{ReferenceError}  Exception raised when a proxy object is used but the underlying  object has been collected.  This is the same as the standard  \exception{ReferenceError} exception.\end{excdesc}  \begin{seealso}  \seepep{0205}{Weak References}{The proposal and rationale for this                feature, including links to earlier implementations                and information about similar features in other                languages.}\end{seealso}  \subsection{Weak Reference Objects            \label{weakref-objects}} Weak reference objects have no attributes or methods, but do allow thereferent to be obtained, if it still exists, by calling it: \begin{verbatim}>>> import weakref>>> class Object:...     pass...>>> o = Object()>>> r = weakref.ref(o)>>> o2 = r()>>> o is o2True\end{verbatim} If the referent no longer exists, calling the reference object returns\constant{None}: \begin{verbatim}>>> del o, o2>>> print r()None\end{verbatim} Testing that a weak reference object is still live should be doneusing the expression \code{\var{ref}() is not None}.  Normally,application code that needs to use a reference object should followthis pattern: \begin{verbatim}# r is a weak reference objecto = r()if o is None:    # referent has been garbage collected    print "Object has been deallocated; can't frobnicate."else:    print "Object is still live!"    o.do_something_useful()\end{verbatim} Using a separate test for ``liveness'' creates race conditions inthreaded applications; another thread can cause a weak reference tobecome invalidated before the weak reference is called; theidiom shown above is safe in threaded applications as well assingle-threaded applications. Specialized versions of \class{ref} objects can be created throughsubclassing.  This is used in the implementation of the\class{WeakValueDictionary} to reduce the memory overhead for eachentry in the mapping.  This may be most useful to associate additionalinformation with a reference, but could also be used to insertadditional processing on calls to retrieve the referent. This example shows how a subclass of \class{ref} can be used to storeadditional information about an object and affect the value that'sreturned when the referent is accessed: \begin{verbatim}import weakref class ExtendedRef(weakref.ref):    def __init__(self, ob, callback=None, **annotations):        super(ExtendedRef, self).__init__(ob, callback)        self.__counter = 0        for k, v in annotations.iteritems():            setattr(self, k, v)     def __call__(self):        """Return a pair containing the referent and the number of        times the reference has been called.        """        ob = super(ExtendedRef, self).__call__()        if ob is not None:            self.__counter += 1            ob = (ob, self.__counter)        return ob\end{verbatim}  \subsection{Example \label{weakref-example}} This simple example shows how an application can use objects IDs toretrieve objects that it has seen before.  The IDs of the objects canthen be used in other data structures without forcing the objects toremain alive, but the objects can still be retrieved by ID if theydo. % Example contributed by Tim Peters.\begin{verbatim}import weakref _id2obj_dict = weakref.WeakValueDictionary() def remember(obj):    oid = id(obj)    _id2obj_dict[oid] = obj    return oid def id2obj(oid):    return _id2obj_dict[oid]\end{verbatim}