261 lines · 11.9 KB · Text
RawDownload
This file describes some special Python build types enabled viacompile-time preprocessor defines. It is best to define these options in the EXTRA_CFLAGS make variable;``make EXTRA_CFLAGS="-DPy_REF_DEBUG"``. ---------------------------------------------------------------------------Py_REF_DEBUG                                              introduced in 1.4                                                 named REF_DEBUG before 1.4 Turn on aggregate reference counting.  This arranges that extern_Py_RefTotal hold a count of all references, the sum of ob_refcnt acrossall objects.  In a debug-mode build, this is where the "8288" comes fromin     >>> 23    23    [8288 refs]    >>> Note that if this count increases when you're not storing away new objects,there's probably a leak.  Remember, though, that in interactive mode thespecial name "_" holds a reference to the last result displayed! Py_REF_DEBUG also checks after every decref to verify that the refcounthasn't gone negative, and causes an immediate fatal error if it has. Special gimmicks: sys.gettotalrefcount()    Return current total of all refcounts.    Available under Py_REF_DEBUG in Python 2.3.    Before 2.3, Py_TRACE_REFS was required to enable this function.---------------------------------------------------------------------------Py_TRACE_REFS                                             introduced in 1.4                                                named TRACE_REFS before 1.4 Turn on heavy reference debugging.  This is major surgery.  Every PyObjectgrows two more pointers, to maintain a doubly-linked list of all liveheap-allocated objects.  Most builtin type objects are not in this list,as they're statically allocated.  Starting in Python 2.3, if COUNT_ALLOCS(see below) is also defined, a static type object T does appear in thislist if at least one object of type T has been created. Note that because the fundamental PyObject layout changes, Python modulescompiled with Py_TRACE_REFS are incompatible with modules compiled withoutit. Py_TRACE_REFS implies Py_REF_DEBUG. Special gimmicks: sys.getobjects(max[, type])    Return list of the (no more than) max most-recently allocated objects,    most recently allocated first in the list, least-recently allocated    last in the list.  max=0 means no limit on list length.    If an optional type object is passed, the list is also restricted to    objects of that type.    The return list itself, and some temp objects created just to call    sys.getobjects(), are excluded from the return list.  Note that the    list returned is just another object, though, so may appear in the    return list the next time you call getobjects(); note that every    object in the list is kept alive too, simply by virtue of being in    the list. envar PYTHONDUMPREFS    If this envar exists, Py_Finalize() arranges to print a list of    all still-live heap objects.  This is printed twice, in different    formats, before and after Py_Finalize has cleaned up everything it    can clean up.  The first output block produces the repr() of each    object so is more informative; however, a lot of stuff destined to    die is still alive then.  The second output block is much harder    to work with (repr() can't be invoked anymore -- the interpreter    has been torn down too far), but doesn't list any objects that will    die.  The tool script combinerefs.py can be run over this to combine    the info from both output blocks.  The second output block, and    combinerefs.py, were new in Python 2.3b1.---------------------------------------------------------------------------PYMALLOC_DEBUG                                            introduced in 2.3 When pymalloc is enabled (WITH_PYMALLOC is defined), calls to the PyObject_memory routines are handled by Python's own small-object allocator, whilecalls to the PyMem_ memory routines are directed to the system malloc/realloc/free.  If PYMALLOC_DEBUG is also defined, calls to both PyObject_and PyMem_ memory routines are directed to a special debugging mode ofPython's small-object allocator. This mode fills dynamically allocated memory blocks with special,recognizable bit patterns, and adds debugging info on each end ofdynamically allocated memory blocks.  The special bit patterns are: #define CLEANBYTE     0xCB   /* clean (newly allocated) memory */#define DEADBYTE      0xDB   /* dead (newly freed) memory */#define FORBIDDENBYTE 0xFB   /* fordidden -- untouchable bytes */ Strings of these bytes are unlikely to be valid addresses, floats, or 7-bitASCII strings. Let S = sizeof(size_t). 2*S bytes are added at each end of each block of Nbytes requested.  The memory layout is like so, where p represents theaddress returned by a malloc-like or realloc-like function (p[i:j] meansthe slice of bytes from *(p+i) inclusive up to *(p+j) exclusive; note thatthe treatment of negative indices differs from a Python slice): p[-2*S:-S]    Number of bytes originally asked for.  This is a size_t, big-endian    (easier to read in a memory dump).p[-S:0]    Copies of FORBIDDENBYTE.  Used to catch under- writes and reads.p[0:N]    The requested memory, filled with copies of CLEANBYTE, used to catch    reference to uninitialized memory.    When a realloc-like function is called requesting a larger memory    block, the new excess bytes are also filled with CLEANBYTE.    When a free-like function is called, these are overwritten with    DEADBYTE, to catch reference to freed memory.  When a realloc-    like function is called requesting a smaller memory block, the excess    old bytes are also filled with DEADBYTE.p[N:N+S]    Copies of FORBIDDENBYTE.  Used to catch over- writes and reads.p[N+S:N+2*S]    A serial number, incremented by 1 on each call to a malloc-like or    realloc-like function.    Big-endian size_t.    If "bad memory" is detected later, the serial number gives an    excellent way to set a breakpoint on the next run, to capture the    instant at which this block was passed out.  The static function    bumpserialno() in obmalloc.c is the only place the serial number    is incremented, and exists so you can set such a breakpoint easily. A realloc-like or free-like function first checks that the FORBIDDENBYTEsat each end are intact.  If they've been altered, diagnostic output iswritten to stderr, and the program is aborted via Py_FatalError().  Theother main failure mode is provoking a memory error when a programreads up one of the special bit patterns and tries to use it as an address.If you get in a debugger then and look at the object, you're likelyto see that it's entirely filled with 0xDB (meaning freed memory isgetting used) or 0xCB (meaning uninitialized memory is getting used). Note that PYMALLOC_DEBUG requires WITH_PYMALLOC. Special gimmicks: envar PYTHONMALLOCSTATS    If this envar exists, a report of pymalloc summary statistics is    printed to stderr whenever a new arena is allocated, and also    by Py_Finalize(). Changed in 2.5:  The number of extra bytes allocated is 4*sizeof(size_t).Before it was 16 on all boxes, reflecting that Python couldn't make use ofallocations >= 2**32 bytes even on 64-bit boxes before 2.5.---------------------------------------------------------------------------Py_DEBUG                                                  introduced in 1.5                                                     named DEBUG before 1.5 This is what is generally meant by "a debug build" of Python. Py_DEBUG implies LLTRACE, Py_REF_DEBUG, Py_TRACE_REFS, andPYMALLOC_DEBUG (if WITH_PYMALLOC is enabled).  In addition, Cassert()s are enabled (via the C way: by not defining NDEBUG), andsome routines do additional sanity checks inside "#ifdef Py_DEBUG"blocks.---------------------------------------------------------------------------COUNT_ALLOCS                                            introduced in 0.9.9                                             partly broken in 2.2 and 2.2.1 Each type object grows three new members:     /* Number of times an object of this type was allocated. */    int tp_allocs;     /* Number of times an object of this type was deallocated. */    int tp_frees;     /* Highwater mark:  the maximum value of tp_allocs - tp_frees so     * far; or, IOW, the largest number of objects of this type alive at     * the same time.     */    int tp_maxalloc; Allocation and deallocation code keeps these counts up to date.Py_Finalize() displays a summary of the info returned by sys.getcounts()(see below), along with assorted other special allocation counts (likethe number of tuple allocations satisfied by a tuple free-list, the numberof 1-character strings allocated, etc). Before Python 2.2, type objects were immortal, and the COUNT_ALLOCSimplementation relies on that.  As of Python 2.2, heap-allocated type/class objects can go away.  COUNT_ALLOCS can blow up in 2.2 and 2.2.1because of this; this was fixed in 2.2.2.  Use of COUNT_ALLOCS makesall heap-allocated type objects immortal, except for those for which noobject of that type is ever allocated. Starting with Python 2.3, If Py_TRACE_REFS is also defined, COUNT_ALLOCSarranges to ensure that the type object for each allocated objectappears in the doubly-linked list of all objects maintained byPy_TRACE_REFS. Special gimmicks: sys.getcounts()    Return a list of 4-tuples, one entry for each type object for which    at least one object of that type was allocated.  Each tuple is of    the form:         (tp_name, tp_allocs, tp_frees, tp_maxalloc)     Each distinct type object gets a distinct entry in this list, even    if two or more type objects have the same tp_name (in which case    there's no way to distinguish them by looking at this list).  The    list is ordered by time of first object allocation:  the type object    for which the first allocation of an object of that type occurred    most recently is at the front of the list.---------------------------------------------------------------------------LLTRACE                                          introduced well before 1.0 Compile in support for Low Level TRACE-ing of the main interpreter loop. When this preprocessor symbol is defined, before PyEval_EvalFrame(eval_frame in 2.3 and 2.2, eval_code2 before that) executes a frame's codeit checks the frame's global namespace for a variable "__lltrace__".  Ifsuch a variable is found, mounds of information about what the interpreteris doing are sprayed to stdout, such as every opcode and opcode argumentand values pushed onto and popped off the value stack. Not useful very often, but very useful when needed. ---------------------------------------------------------------------------CALL_PROFILE                                      introduced for Python 2.3 Count the number of function calls executed. When this symbol is defined, the ceval mainloop and helper functionscount the number of function calls made.  It keeps detailed statisticsabout what kind of object was called and whether the call hit any ofthe special fast paths in the code. ---------------------------------------------------------------------------WITH_TSC                                          introduced for Python 2.4 Super-lowlevel profiling of the interpreter.  When enabled, the sysmodule grows a new function: settscdump(bool)    If true, tell the Python interpreter to dump VM measurements to    stderr.  If false, turn off dump.  The measurements are based on the    processor's time-stamp counter. This build option requires a small amount of platform specific code.Currently this code is present for linux/x86 and any PowerPC platformthat uses GCC (i.e. OS X and linux/ppc). On the PowerPC the rate at which the time base register is incrementedis not defined by the architecture specification, so you'll need tofind the manual for your specific processor.  For the 750CX, 750CXeand 750FX (all sold as the G3) we find:     The time base counter is clocked at a frequency that is    one-fourth that of the bus clock. This build is enabled by the --with-tsc flag to configure.