\documentclass{manual}\usepackage{distutils} % $Id: dist.tex 53487 2007-01-19 05:52:46Z neal.norwitz $ % TODO%   Document extension.read_setup_file%   Document build_clib command% \title{Distributing Python Modules} \input{boilerplate} \author{Greg Ward\\        Anthony Baxter}\authoraddress{	\strong{Python Software Foundation}\\	Email: \email{distutils-sig@python.org}} \makeindex\makemodindex \begin{document} \maketitle \input{copyright} \begin{abstract}  \noindent  This document describes the Python Distribution Utilities  (``Distutils'') from the module developer's point of view, describing  how to use the Distutils to make Python modules and extensions easily  available to a wider audience with very little overhead for  build/release/install mechanics.\end{abstract} % The ugly "%begin{latexonly}" pseudo-environment suppresses the table% of contents for HTML generation.%%begin{latexonly}\tableofcontents%end{latexonly}  \chapter{An Introduction to Distutils}\label{intro} This document covers using the Distutils to distribute your Pythonmodules, concentrating on the role of developer/distributor: ifyou're looking for information on installing Python modules, youshould refer to the \citetitle[../inst/inst.html]{Installing PythonModules} manual.  \section{Concepts \& Terminology}\label{concepts} Using the Distutils is quite simple, both for module developers and forusers/administrators installing third-party modules.  As a developer,your responsibilities (apart from writing solid, well-documented andwell-tested code, of course!) are:\begin{itemize}\item write a setup script (\file{setup.py} by convention)\item (optional) write a setup configuration file\item create a source distribution\item (optional) create one or more built (binary) distributions\end{itemize}Each of these tasks is covered in this document. Not all module developers have access to a multitude of platforms, soit's not always feasible to expect them to create a multitude of builtdistributions.  It is hoped that a class of intermediaries, called\emph{packagers}, will arise to address this need.  Packagers will takesource distributions released by module developers, build them on one ormore platforms, and release the resulting built distributions.  Thus,users on the most popular platforms will be able to install most popularPython module distributions in the most natural way for their platform,without having to run a single setup script or compile a line of code.  \section{A Simple Example}\label{simple-example} The setup script is usually quite simple, although since it's writtenin Python, there are no arbitrary limits to what you can do with it,though you should be careful about putting arbitrarily expensiveoperations in your setup script. Unlike, say, Autoconf-style configurescripts, the setup script may be run multiple times in the course ofbuilding and installing your module distribution.   If all you want to do is distribute a module called \module{foo},contained in a file \file{foo.py}, then your setup script can be assimple as this: \begin{verbatim}from distutils.core import setupsetup(name='foo',      version='1.0',      py_modules=['foo'],      )\end{verbatim} Some observations:\begin{itemize}\item most information that you supply to the Distutils is supplied as  keyword arguments to the \function{setup()} function\item those keyword arguments fall into two categories: package  metadata (name, version number) and information about what's in the  package (a list of pure Python modules, in this case)\item modules are specified by module name, not filename (the same will  hold true for packages and extensions)\item it's recommended that you supply a little more metadata, in  particular your name, email address and a URL for the project  (see section~\ref{setup-script} for an example)\end{itemize} To create a source distribution for this module, you would create asetup script, \file{setup.py}, containing the above code, and run: \begin{verbatim}python setup.py sdist\end{verbatim} which will create an archive file (e.g., tarball on \UNIX, ZIP file onWindows) containing your setup script \file{setup.py}, and your module\file{foo.py}.  The archive file will be named \file{foo-1.0.tar.gz} (or\file{.zip}), and will unpack into a directory \file{foo-1.0}. If an end-user wishes to install your \module{foo} module, all she hasto do is download \file{foo-1.0.tar.gz} (or \file{.zip}), unpack it,and---from the \file{foo-1.0} directory---run \begin{verbatim}python setup.py install\end{verbatim} which will ultimately copy \file{foo.py} to the appropriate directoryfor third-party modules in their Python installation. This simple example demonstrates some fundamental concepts of theDistutils. First, both developers and installers have the same basicuser interface, i.e. the setup script.  The difference is whichDistutils \emph{commands} they use: the \command{sdist} command isalmost exclusively for module developers, while \command{install} ismore often for installers (although most developers will want to installtheir own code occasionally). If you want to make things really easy for your users, you can createone or more built distributions for them.  For instance, if you arerunning on a Windows machine, and want to make things easy for otherWindows users, you can create an executable installer (the mostappropriate type of built distribution for this platform) with the\command{bdist\_wininst} command.  For example: \begin{verbatim}python setup.py bdist_wininst\end{verbatim} will create an executable installer, \file{foo-1.0.win32.exe}, in thecurrent directory. Other useful built distribution formats are RPM, implemented by the\command{bdist\_rpm} command, Solaris \program{pkgtool}(\command{bdist\_pkgtool}), and HP-UX \program{swinstall}(\command{bdist_sdux}).  For example, the following command willcreate an RPM file called \file{foo-1.0.noarch.rpm}: \begin{verbatim}python setup.py bdist_rpm\end{verbatim} (The \command{bdist\_rpm} command uses the \command{rpm} executable,therefore this has to be run on an RPM-based system such as Red HatLinux, SuSE Linux, or Mandrake Linux.) You can find out what distribution formats are available at any time byrunning \begin{verbatim}python setup.py bdist --help-formats\end{verbatim}  \section{General Python terminology}\label{python-terms} If you're reading this document, you probably have a good idea of whatmodules, extensions, and so forth are.  Nevertheless, just to be surethat everyone is operating from a common starting point, we offer thefollowing glossary of common Python terms:\begin{description}\item[module] the basic unit of code reusability in Python: a block of  code imported by some other code.  Three types of modules concern us  here: pure Python modules, extension modules, and packages. \item[pure Python module] a module written in Python and contained in a  single \file{.py} file (and possibly associated \file{.pyc} and/or  \file{.pyo} files).  Sometimes referred to as a ``pure module.'' \item[extension module] a module written in the low-level language of  the Python implementation: C/\Cpp{} for Python, Java for Jython.  Typically contained in a single dynamically loadable pre-compiled  file, e.g. a shared object (\file{.so}) file for Python extensions on  \UNIX, a DLL (given the \file{.pyd} extension) for Python extensions  on Windows, or a Java class file for Jython extensions.  (Note that  currently, the Distutils only handles C/\Cpp{} extensions for Python.) \item[package] a module that contains other modules; typically contained  in a directory in the filesystem and distinguished from other  directories by the presence of a file \file{\_\_init\_\_.py}. \item[root package] the root of the hierarchy of packages.  (This isn't  really a package, since it doesn't have an \file{\_\_init\_\_.py}  file.  But we have to call it something.)  The vast majority of the  standard library is in the root package, as are many small, standalone  third-party modules that don't belong to a larger module collection.  Unlike regular packages, modules in the root package can be found in  many directories: in fact, every directory listed in \code{sys.path}  contributes modules to the root package.\end{description}  \section{Distutils-specific terminology}\label{distutils-term} The following terms apply more specifically to the domain ofdistributing Python modules using the Distutils:\begin{description}\item[module distribution] a collection of Python modules distributed  together as a single downloadable resource and meant to be installed  \emph{en masse}.  Examples of some well-known module distributions are  Numeric Python, PyXML, PIL (the Python Imaging Library), or  mxBase.  (This would be called a \emph{package}, except that term  is already taken in the Python context: a single module distribution  may contain zero, one, or many Python packages.) \item[pure module distribution] a module distribution that contains only  pure Python modules and packages.  Sometimes referred to as a ``pure  distribution.'' \item[non-pure module distribution] a module distribution that contains  at least one extension module.  Sometimes referred to as a ``non-pure  distribution.'' \item[distribution root] the top-level directory of your source tree (or   source distribution); the directory where \file{setup.py} exists.  Generally   \file{setup.py} will be run from this directory.\end{description}  \chapter{Writing the Setup Script}\label{setup-script} The setup script is the centre of all activity in building,distributing, and installing modules using the Distutils.  The mainpurpose of the setup script is to describe your module distribution tothe Distutils, so that the various commands that operate on your modulesdo the right thing.  As we saw in section~\ref{simple-example} above,the setup script consists mainly of a call to \function{setup()}, andmost information supplied to the Distutils by the module developer issupplied as keyword arguments to \function{setup()}. Here's a slightly more involved example, which we'll follow for the nextcouple of sections: the Distutils' own setup script.  (Keep in mind thatalthough the Distutils are included with Python 1.6 and later, they alsohave an independent existence so that Python 1.5.2 users can use them toinstall other module distributions.  The Distutils' own setup script,shown here, is used to install the package into Python 1.5.2.) \begin{verbatim}#!/usr/bin/env python from distutils.core import setup setup(name='Distutils',      version='1.0',      description='Python Distribution Utilities',      author='Greg Ward',      author_email='gward@python.net',      url='http://www.python.org/sigs/distutils-sig/',      packages=['distutils', 'distutils.command'],     )\end{verbatim} There are only two differences between this and the trivial one-filedistribution presented in section~\ref{simple-example}: moremetadata, and the specification of pure Python modules by package,rather than by module.  This is important since the Distutils consist ofa couple of dozen modules split into (so far) two packages; an explicitlist of every module would be tedious to generate and difficult tomaintain.  For more information on the additional meta-data, seesection~\ref{meta-data}. Note that any pathnames (files or directories) supplied in the setupscript should be written using the \UNIX{} convention, i.e.slash-separated.  The Distutils will take care of converting thisplatform-neutral representation into whatever is appropriate on yourcurrent platform before actually using the pathname.  This makes yoursetup script portable across operating systems, which of course is oneof the major goals of the Distutils.  In this spirit, all pathnames inthis document are slash-separated.  (Mac OS 9 programmers should keep inmind that the \emph{absence} of a leading slash indicates a relativepath, the opposite of the Mac OS convention with colons.) This, of course, only applies to pathnames given to Distutilsfunctions.  If you, for example, use standard Python functions such as\function{glob.glob()} or \function{os.listdir()} to specify files, youshould be careful to write portable code instead of hardcoding pathseparators: \begin{verbatim}    glob.glob(os.path.join('mydir', 'subdir', '*.html'))    os.listdir(os.path.join('mydir', 'subdir'))\end{verbatim}  \section{Listing whole packages}\label{listing-packages} The \option{packages} option tells the Distutils to process (build,distribute, install, etc.) all pure Python modules found in each packagementioned in the \option{packages} list.  In order to do this, ofcourse, there has to be a correspondence between package names anddirectories in the filesystem.  The default correspondence is the mostobvious one, i.e. package \module{distutils} is found in the directory\file{distutils} relative to the distribution root.  Thus, when you say\code{packages = ['foo']} in your setup script, you are promising thatthe Distutils will find a file \file{foo/\_\_init\_\_.py} (which mightbe spelled differently on your system, but you get the idea) relative tothe directory where your setup script lives.  If you break thispromise, the Distutils will issue a warning but still process the brokenpackage anyways. If you use a different convention to lay out your source directory,that's no problem: you just have to supply the \option{package\_dir}option to tell the Distutils about your convention.  For example, sayyou keep all Python source under \file{lib}, so that modules in the``root package'' (i.e., not in any package at all) are in\file{lib}, modules in the \module{foo} package are in \file{lib/foo},and so forth.  Then you would put \begin{verbatim}package_dir = {'': 'lib'}\end{verbatim} in your setup script.  The keys to this dictionary are package names,and an empty package name stands for the root package.  The values aredirectory names relative to your distribution root.  In this case, whenyou say \code{packages = ['foo']}, you are promising that the file\file{lib/foo/\_\_init\_\_.py} exists. Another possible convention is to put the \module{foo} package right in \file{lib}, the \module{foo.bar} package in \file{lib/bar}, etc.  Thiswould be written in the setup script as \begin{verbatim}package_dir = {'foo': 'lib'}\end{verbatim} A \code{\var{package}: \var{dir}} entry in the \option{package\_dir}dictionary implicitly applies to all packages below \var{package}, sothe \module{foo.bar} case is automatically handled here.  In thisexample, having \code{packages = ['foo', 'foo.bar']} tells the Distutilsto look for \file{lib/\_\_init\_\_.py} and\file{lib/bar/\_\_init\_\_.py}.  (Keep in mind that although\option{package\_dir} applies recursively, you must explicitly list allpackages in \option{packages}: the Distutils will \emph{not} recursivelyscan your source tree looking for any directory with an\file{\_\_init\_\_.py} file.)  \section{Listing individual modules}\label{listing-modules} For a small module distribution, you might prefer to list all modulesrather than listing packages---especially the case of a single modulethat goes in the ``root package'' (i.e., no package at all).  Thissimplest case was shown in section~\ref{simple-example}; here is aslightly more involved example: \begin{verbatim}py_modules = ['mod1', 'pkg.mod2']\end{verbatim} This describes two modules, one of them in the ``root'' package, theother in the \module{pkg} package.  Again, the default package/directorylayout implies that these two modules can be found in \file{mod1.py} and\file{pkg/mod2.py}, and that \file{pkg/\_\_init\_\_.py} exists as well.And again, you can override the package/directory correspondence usingthe \option{package\_dir} option.  \section{Describing extension modules}\label{describing-extensions} % XXX read over this sectionJust as writing Python extension modules is a bit more complicated thanwriting pure Python modules, describing them to the Distutils is a bitmore complicated.  Unlike pure modules, it's not enough just to listmodules or packages and expect the Distutils to go out and find theright files; you have to specify the extension name, source file(s), andany compile/link requirements (include directories, libraries to linkwith, etc.). All of this is done through another keyword argument to\function{setup()}, the \option{ext_modules} option.  \option{ext_modules}is just a list of \class{Extension} instances, each of which describes asingle extension module.  Suppose your distribution includes a singleextension, called \module{foo} and implemented by \file{foo.c}.  If noadditional instructions to the compiler/linker are needed, describingthis extension is quite simple: \begin{verbatim}Extension('foo', ['foo.c'])\end{verbatim} The \class{Extension} class can be imported from\module{distutils.core} along with \function{setup()}.  Thus, the setupscript for a module distribution that contains only this one extensionand nothing else might be: \begin{verbatim}from distutils.core import setup, Extensionsetup(name='foo',      version='1.0',      ext_modules=[Extension('foo', ['foo.c'])],      )\end{verbatim} The \class{Extension} class (actually, the underlying extension-buildingmachinery implemented by the \command{build\_ext} command) supports agreat deal of flexibility in describing Python extensions, which isexplained in the following sections.    \subsection{Extension names and packages} The first argument to the \class{Extension} constructor is always thename of the extension, including any package names.  For example, \begin{verbatim}Extension('foo', ['src/foo1.c', 'src/foo2.c'])\end{verbatim} describes an extension that lives in the root package, while \begin{verbatim}Extension('pkg.foo', ['src/foo1.c', 'src/foo2.c'])\end{verbatim} describes the same extension in the \module{pkg} package.  The sourcefiles and resulting object code are identical in both cases; the onlydifference is where in the filesystem (and therefore where in Python'snamespace hierarchy) the resulting extension lives. If you have a number of extensions all in the same package (or all underthe same base package), use the \option{ext\_package} keyword argumentto \function{setup()}.  For example, \begin{verbatim}setup(...      ext_package='pkg',      ext_modules=[Extension('foo', ['foo.c']),                   Extension('subpkg.bar', ['bar.c'])],     )\end{verbatim} will compile \file{foo.c} to the extension \module{pkg.foo}, and\file{bar.c} to \module{pkg.subpkg.bar}.  \subsection{Extension source files} The second argument to the \class{Extension} constructor is a list ofsource files.  Since the Distutils currently only support C, \Cpp, andObjective-C extensions, these are normally C/\Cpp/Objective-C sourcefiles.  (Be sure to use appropriate extensions to distinguish \Cpp\source files: \file{.cc} and \file{.cpp} seem to be recognized by both\UNIX{} and Windows compilers.) However, you can also include SWIG interface (\file{.i}) files in thelist; the \command{build\_ext} command knows how to deal with SWIGextensions: it will run SWIG on the interface file and compile theresulting C/\Cpp{} file into your extension. \XXX{SWIG support is rough around the edges and largely untested;  especially SWIG support for \Cpp{} extensions!  Explain in more detail  here when the interface firms up.} On some platforms, you can include non-source files that are processedby the compiler and included in your extension.  Currently, this justmeans Windows message text (\file{.mc}) files and resource definition(\file{.rc}) files for Visual \Cpp. These will be compiled to binary resource(\file{.res}) files and linked into the executable.  \subsection{Preprocessor options}

Lines 1–500 of 3795. Load more