diff options
Diffstat (limited to 'Doc/packaging')
| -rw-r--r-- | Doc/packaging/builtdist.rst | 304 | ||||
| -rw-r--r-- | Doc/packaging/commandhooks.rst | 47 | ||||
| -rw-r--r-- | Doc/packaging/commandref.rst | 350 | ||||
| -rw-r--r-- | Doc/packaging/configfile.rst | 125 | ||||
| -rw-r--r-- | Doc/packaging/examples.rst | 334 | ||||
| -rw-r--r-- | Doc/packaging/extending.rst | 95 | ||||
| -rw-r--r-- | Doc/packaging/index.rst | 45 | ||||
| -rw-r--r-- | Doc/packaging/introduction.rst | 193 | ||||
| -rw-r--r-- | Doc/packaging/packageindex.rst | 104 | ||||
| -rw-r--r-- | Doc/packaging/setupcfg.rst | 865 | ||||
| -rw-r--r-- | Doc/packaging/setupscript.rst | 689 | ||||
| -rw-r--r-- | Doc/packaging/sourcedist.rst | 266 | ||||
| -rw-r--r-- | Doc/packaging/tutorial.rst | 112 | ||||
| -rw-r--r-- | Doc/packaging/uploading.rst | 80 |
14 files changed, 3609 insertions, 0 deletions
diff --git a/Doc/packaging/builtdist.rst b/Doc/packaging/builtdist.rst new file mode 100644 index 0000000..b1e5e93 --- /dev/null +++ b/Doc/packaging/builtdist.rst @@ -0,0 +1,304 @@ +.. _packaging-built-dist: + +**************************** +Creating Built Distributions +**************************** + +A "built distribution" is what you're probably used to thinking of either as a +"binary package" or an "installer" (depending on your background). It's not +necessarily binary, though, because it might contain only Python source code +and/or byte-code; and we don't call it a package, because that word is already +spoken for in Python. (And "installer" is a term specific to the world of +mainstream desktop systems.) + +A built distribution is how you make life as easy as possible for installers of +your module distribution: for users of RPM-based Linux systems, it's a binary +RPM; for Windows users, it's an executable installer; for Debian-based Linux +users, it's a Debian package; and so forth. Obviously, no one person will be +able to create built distributions for every platform under the sun, so the +Distutils are designed to enable module developers to concentrate on their +specialty---writing code and creating source distributions---while an +intermediary species called *packagers* springs up to turn source distributions +into built distributions for as many platforms as there are packagers. + +Of course, the module developer could be his own packager; or the packager could +be a volunteer "out there" somewhere who has access to a platform which the +original developer does not; or it could be software periodically grabbing new +source distributions and turning them into built distributions for as many +platforms as the software has access to. Regardless of who they are, a packager +uses the setup script and the :command:`bdist` command family to generate built +distributions. + +As a simple example, if I run the following command in the Distutils source +tree:: + + python setup.py bdist + +then the Distutils builds my module distribution (the Distutils itself in this +case), does a "fake" installation (also in the :file:`build` directory), and +creates the default type of built distribution for my platform. The default +format for built distributions is a "dumb" tar file on Unix, and a simple +executable installer on Windows. (That tar file is considered "dumb" because it +has to be unpacked in a specific location to work.) + +Thus, the above command on a Unix system creates +:file:`Distutils-1.0.{plat}.tar.gz`; unpacking this tarball from the right place +installs the Distutils just as though you had downloaded the source distribution +and run ``python setup.py install``. (The "right place" is either the root of +the filesystem or Python's :file:`{prefix}` directory, depending on the options +given to the :command:`bdist_dumb` command; the default is to make dumb +distributions relative to :file:`{prefix}`.) + +Obviously, for pure Python distributions, this isn't any simpler than just +running ``python setup.py install``\ ---but for non-pure distributions, which +include extensions that would need to be compiled, it can mean the difference +between someone being able to use your extensions or not. And creating "smart" +built distributions, such as an executable installer for +Windows, is far more convenient for users even if your distribution doesn't +include any extensions. + +The :command:`bdist` command has a :option:`--formats` option, similar to the +:command:`sdist` command, which you can use to select the types of built +distribution to generate: for example, :: + + python setup.py bdist --format=zip + +would, when run on a Unix system, create :file:`Distutils-1.0.{plat}.zip`\ +---again, this archive would be unpacked from the root directory to install the +Distutils. + +The available formats for built distributions are: + ++-------------+------------------------------+---------+ +| Format | Description | Notes | ++=============+==============================+=========+ +| ``gztar`` | gzipped tar file | (1),(3) | +| | (:file:`.tar.gz`) | | ++-------------+------------------------------+---------+ +| ``tar`` | tar file (:file:`.tar`) | \(3) | ++-------------+------------------------------+---------+ +| ``zip`` | zip file (:file:`.zip`) | (2),(4) | ++-------------+------------------------------+---------+ +| ``wininst`` | self-extracting ZIP file for | \(4) | +| | Windows | | ++-------------+------------------------------+---------+ +| ``msi`` | Microsoft Installer. | | ++-------------+------------------------------+---------+ + + +Notes: + +(1) + default on Unix + +(2) + default on Windows + +(3) + requires external utilities: :program:`tar` and possibly one of :program:`gzip` + or :program:`bzip2` + +(4) + requires either external :program:`zip` utility or :mod:`zipfile` module (part + of the standard Python library since Python 1.6) + +You don't have to use the :command:`bdist` command with the :option:`--formats` +option; you can also use the command that directly implements the format you're +interested in. Some of these :command:`bdist` "sub-commands" actually generate +several similar formats; for instance, the :command:`bdist_dumb` command +generates all the "dumb" archive formats (``tar``, ``gztar``, and +``zip``). The :command:`bdist` sub-commands, and the formats generated by +each, are: + ++--------------------------+-----------------------+ +| Command | Formats | ++==========================+=======================+ +| :command:`bdist_dumb` | tar, gztar, zip | ++--------------------------+-----------------------+ +| :command:`bdist_wininst` | wininst | ++--------------------------+-----------------------+ +| :command:`bdist_msi` | msi | ++--------------------------+-----------------------+ + +The following sections give details on the individual :command:`bdist_\*` +commands. + + +.. _packaging-creating-dumb: + +Creating dumb built distributions +================================= + +.. XXX Need to document absolute vs. prefix-relative packages here, but first + I have to implement it! + + +.. _packaging-creating-wininst: + +Creating Windows Installers +=========================== + +Executable installers are the natural format for binary distributions on +Windows. They display a nice graphical user interface, display some information +about the module distribution to be installed taken from the metadata in the +setup script, let the user select a few options, and start or cancel the +installation. + +Since the metadata is taken from the setup script, creating Windows installers +is usually as easy as running:: + + python setup.py bdist_wininst + +or the :command:`bdist` command with the :option:`--formats` option:: + + python setup.py bdist --formats=wininst + +If you have a pure module distribution (only containing pure Python modules and +packages), the resulting installer will be version independent and have a name +like :file:`foo-1.0.win32.exe`. These installers can even be created on Unix +platforms or Mac OS X. + +If you have a non-pure distribution, the extensions can only be created on a +Windows platform, and will be Python version dependent. The installer filename +will reflect this and now has the form :file:`foo-1.0.win32-py2.0.exe`. You +have to create a separate installer for every Python version you want to +support. + +.. TODO Add :term: markup to bytecode when merging into the stdlib + +The installer will try to compile pure modules into bytecode after installation +on the target system in normal and optimizing mode. If you don't want this to +happen for some reason, you can run the :command:`bdist_wininst` command with +the :option:`--no-target-compile` and/or the :option:`--no-target-optimize` +option. + +By default the installer will display the cool "Python Powered" logo when it is +run, but you can also supply your own 152x261 bitmap which must be a Windows +:file:`.bmp` file with the :option:`--bitmap` option. + +The installer will also display a large title on the desktop background window +when it is run, which is constructed from the name of your distribution and the +version number. This can be changed to another text by using the +:option:`--title` option. + +The installer file will be written to the "distribution directory" --- normally +:file:`dist/`, but customizable with the :option:`--dist-dir` option. + +.. _packaging-cross-compile-windows: + +Cross-compiling on Windows +========================== + +Starting with Python 2.6, packaging is capable of cross-compiling between +Windows platforms. In practice, this means that with the correct tools +installed, you can use a 32bit version of Windows to create 64bit extensions +and vice-versa. + +To build for an alternate platform, specify the :option:`--plat-name` option +to the build command. Valid values are currently 'win32', 'win-amd64' and +'win-ia64'. For example, on a 32bit version of Windows, you could execute:: + + python setup.py build --plat-name=win-amd64 + +to build a 64bit version of your extension. The Windows Installers also +support this option, so the command:: + + python setup.py build --plat-name=win-amd64 bdist_wininst + +would create a 64bit installation executable on your 32bit version of Windows. + +To cross-compile, you must download the Python source code and cross-compile +Python itself for the platform you are targetting - it is not possible from a +binary installtion of Python (as the .lib etc file for other platforms are +not included.) In practice, this means the user of a 32 bit operating +system will need to use Visual Studio 2008 to open the +:file:`PCBuild/PCbuild.sln` solution in the Python source tree and build the +"x64" configuration of the 'pythoncore' project before cross-compiling +extensions is possible. + +Note that by default, Visual Studio 2008 does not install 64bit compilers or +tools. You may need to reexecute the Visual Studio setup process and select +these tools (using Control Panel->[Add/Remove] Programs is a convenient way to +check or modify your existing install.) + +.. _packaging-postinstallation-script: + +The Postinstallation script +--------------------------- + +Starting with Python 2.3, a postinstallation script can be specified with the +:option:`--install-script` option. The basename of the script must be +specified, and the script filename must also be listed in the scripts argument +to the setup function. + +This script will be run at installation time on the target system after all the +files have been copied, with ``argv[1]`` set to :option:`-install`, and again at +uninstallation time before the files are removed with ``argv[1]`` set to +:option:`-remove`. + +The installation script runs embedded in the windows installer, every output +(``sys.stdout``, ``sys.stderr``) is redirected into a buffer and will be +displayed in the GUI after the script has finished. + +Some functions especially useful in this context are available as additional +built-in functions in the installation script. + +.. currentmodule:: bdist_wininst-postinst-script + +.. function:: directory_created(path) + file_created(path) + + These functions should be called when a directory or file is created by the + postinstall script at installation time. It will register *path* with the + uninstaller, so that it will be removed when the distribution is uninstalled. + To be safe, directories are only removed if they are empty. + + +.. function:: get_special_folder_path(csidl_string) + + This function can be used to retrieve special folder locations on Windows like + the Start Menu or the Desktop. It returns the full path to the folder. + *csidl_string* must be one of the following strings:: + + "CSIDL_APPDATA" + + "CSIDL_COMMON_STARTMENU" + "CSIDL_STARTMENU" + + "CSIDL_COMMON_DESKTOPDIRECTORY" + "CSIDL_DESKTOPDIRECTORY" + + "CSIDL_COMMON_STARTUP" + "CSIDL_STARTUP" + + "CSIDL_COMMON_PROGRAMS" + "CSIDL_PROGRAMS" + + "CSIDL_FONTS" + + If the folder cannot be retrieved, :exc:`OSError` is raised. + + Which folders are available depends on the exact Windows version, and probably + also the configuration. For details refer to Microsoft's documentation of the + :c:func:`SHGetSpecialFolderPath` function. + + +.. function:: create_shortcut(target, description, filename[, arguments[, workdir[, iconpath[, iconindex]]]]) + + This function creates a shortcut. *target* is the path to the program to be + started by the shortcut. *description* is the description of the shortcut. + *filename* is the title of the shortcut that the user will see. *arguments* + specifies the command-line arguments, if any. *workdir* is the working directory + for the program. *iconpath* is the file containing the icon for the shortcut, + and *iconindex* is the index of the icon in the file *iconpath*. Again, for + details consult the Microsoft documentation for the :class:`IShellLink` + interface. + + +Vista User Access Control (UAC) +=============================== + +Starting with Python 2.6, bdist_wininst supports a :option:`--user-access-control` +option. The default is 'none' (meaning no UAC handling is done), and other +valid values are 'auto' (meaning prompt for UAC elevation if Python was +installed for all users) and 'force' (meaning always prompt for elevation). diff --git a/Doc/packaging/commandhooks.rst b/Doc/packaging/commandhooks.rst new file mode 100644 index 0000000..b261d00 --- /dev/null +++ b/Doc/packaging/commandhooks.rst @@ -0,0 +1,47 @@ +.. TODO integrate this in commandref and configfile + +.. _packaging-command-hooks: + +============= +Command hooks +============= + +Packaging provides a way of extending its commands by the use of pre- and +post-command hooks. Hooks are Python functions (or any callable object) that +take a command object as argument. They're specified in :ref:`config files +<packaging-config-filenames>` using their fully qualified names. After a +command is finalized (its options are processed), the pre-command hooks are +executed, then the command itself is run, and finally the post-command hooks are +executed. + +See also global setup hooks in :ref:`setupcfg-spec`. + + +.. _packaging-finding-hooks: + +Finding hooks +============= + +As a hook is configured with a Python dotted name, it must either be defined in +a module installed on the system, or in a module present in the project +directory, where the :file:`setup.cfg` file lives:: + + # file: _setuphooks.py + + def hook(install_cmd): + metadata = install_cmd.dist.metadata + print('Hooked while installing %r %s!' % (metadata['Name'], + metadata['Version'])) + +Then you need to configure it in :file:`setup.cfg`:: + + [install_dist] + pre-hook.a = _setuphooks.hook + +Packaging will add the project directory to :data:`sys.path` and find the +``_setuphooks`` module. + +Hooks defined in different config files (system-wide, user-wide and +project-wide) do not override each other as long as they are specified with +different aliases (additional names after the dot). The alias in the example +above is ``a``. diff --git a/Doc/packaging/commandref.rst b/Doc/packaging/commandref.rst new file mode 100644 index 0000000..0ec7505 --- /dev/null +++ b/Doc/packaging/commandref.rst @@ -0,0 +1,350 @@ +.. _packaging-command-reference: + +***************** +Command Reference +***************** + +This reference briefly documents all standard Packaging commands and some of +their options. + +.. FIXME does not work: Use pysetup run --help-commands to list all + standard and extra commands availavble on your system, with their + description. Use pysetup run <command> --help to get help about the options + of one command. + +.. XXX sections from this document should be merged with other docs (e.g. check + and upload with uploading.rst, install_* with install/install.rst, etc.); + there is no value in partially duplicating information. this file could + however serve as an index, i.e. just a list of all commands with links to + every section that describes options or usage + + +Preparing distributions +======================= + +:command:`check` +---------------- + +Perform some tests on the metadata of a distribution. + +For example, it verifies that all required metadata fields are provided in the +:file:`setup.cfg` file. + +.. TODO document reST checks + + +:command:`test` +--------------- + +Run a test suite. + +When doing test-driven development, or running automated builds that need +testing before they are installed for downloading or use, it's often useful to +be able to run a project's unit tests without actually installing the project +anywhere. The :command:`test` command runs project's unit tests without +actually installing it, by temporarily putting the project's source on +:data:`sys.path`, after first running :command:`build_ext -i` to ensure that any +C extensions are built. + +You can use this command in one of two ways: either by specifying a +unittest-compatible test suite for your project (or any callable that returns +it) or by passing a test runner function that will run your tests and display +results in the console. Both options take a Python dotted name in the form +``package.module.callable`` to specify the object to use. + +If none of these options are specified, Packaging will try to perform test +discovery using either unittest (for Python 3.2 and higher) or unittest2 (for +older versions, if installed). + +.. this is a pseudo-command name used to disambiguate the options in indexes and + links +.. program:: packaging test + +.. cmdoption:: --suite=NAME, -s NAME + + Specify the test suite (or module, class, or method) to be run. The default + for this option can be set by in the project's :file:`setup.cfg` file: + + .. code-block:: cfg + + [test] + suite = mypackage.tests.get_all_tests + +.. cmdoption:: --runner=NAME, -r NAME + + Specify the test runner to be called. + + +:command:`config` +----------------- + +Perform distribution configuration. + + +The build step +============== + +This step is mainly useful to compile C/C++ libraries or extension modules. The +build commands can be run manually to check for syntax errors or packaging +issues (for example if the addition of a new source file was forgotten in the +:file:`setup.cfg` file), and is also run automatically by commands which need +it. Packaging checks the mtime of source and built files to avoid re-building +if it's not necessary. + + +:command:`build` +---------------- + +Build all files of a distribution, delegating to the other :command:`build_*` +commands to do the work. + + +:command:`build_clib` +--------------------- + +Build C libraries. + + +:command:`build_ext` +-------------------- + +Build C/C++ extension modules. + + +:command:`build_py` +------------------- + +Build the Python modules (just copy them to the build directory) and +byte-compile them to .pyc files. + + +:command:`build_scripts` +------------------------ +Build the scripts (just copy them to the build directory and adjust their +shebang if they're Python scripts). + + +:command:`clean` +---------------- + +Clean the build tree of the release. + +.. program:: packaging clean + +.. cmdoption:: --all, -a + + Remove build directories for modules, scripts, etc., not only temporary build + by-products. + + +Creating source and built distributions +======================================= + +:command:`sdist` +---------------- + +Build a source distribution for a release. + +It is recommended that you always build and upload a source distribution. Users +of OSes with easy access to compilers and users of advanced packaging tools will +prefer to compile from source rather than using pre-built distributions. For +Windows users, providing a binary installer is also recommended practice. + + +:command:`bdist` +---------------- + +Build a binary distribution for a release. + +This command will call other :command:`bdist_*` commands to create one or more +distributions depending on the options given. The default is to create a +.tar.gz archive on Unix and a zip archive on Windows or OS/2. + +.. program:: packaging bdist + +.. cmdoption:: --formats + + Binary formats to build (comma-separated list). + +.. cmdoption:: --show-formats + + Dump list of available formats. + + +:command:`bdist_dumb` +--------------------- + +Build a "dumb" installer, a simple archive of files that could be unpacked under +``$prefix`` or ``$exec_prefix``. + + +:command:`bdist_wininst` +------------------------ + +Build a Windows installer. + + +:command:`bdist_msi` +-------------------- + +Build a `Microsoft Installer`_ (.msi) file. + +.. _Microsoft Installer: http://msdn.microsoft.com/en-us/library/cc185688(VS.85).aspx + +In most cases, the :command:`bdist_msi` installer is a better choice than the +:command:`bdist_wininst` installer, because it provides better support for Win64 +platforms, allows administrators to perform non-interactive installations, and +allows installation through group policies. + + +Publishing distributions +======================== + +:command:`register` +------------------- + +This command registers the current release with the Python Package Index. This +is described in more detail in :PEP:`301`. + +.. TODO explain user and project registration with the web UI + + +:command:`upload` +----------------- + +Upload source and/or binary distributions to PyPI. + +The distributions have to be built on the same command line as the +:command:`upload` command; see :ref:`packaging-package-upload` for more info. + +.. program:: packaging upload + +.. cmdoption:: --sign, -s + + Sign each uploaded file using GPG (GNU Privacy Guard). The ``gpg`` program + must be available for execution on the system ``PATH``. + +.. cmdoption:: --identity=NAME, -i NAME + + Specify the identity or key name for GPG to use when signing. The value of + this option will be passed through the ``--local-user`` option of the + ``gpg`` program. + +.. cmdoption:: --show-response + + Display the full response text from server; this is useful for debugging + PyPI problems. + +.. cmdoption:: --repository=URL, -r URL + + The URL of the repository to upload to. Defaults to + http://pypi.python.org/pypi (i.e., the main PyPI installation). + +.. cmdoption:: --upload-docs + + Also run :command:`upload_docs`. Mainly useful as a default value in + :file:`setup.cfg` (on the command line, it's shorter to just type both + commands). + + +:command:`upload_docs` +---------------------- + +Upload HTML documentation to PyPI. + +PyPI now supports publishing project documentation at a URI of the form +``http://packages.python.org/<project>``. :command:`upload_docs` will create +the necessary zip file out of a documentation directory and will post to the +repository. + +Note that to upload the documentation of a project, the corresponding version +must already be registered with PyPI, using the :command:`register` command --- +just like with :command:`upload`. + +Assuming there is an ``Example`` project with documentation in the subdirectory +:file:`docs`, for example:: + + Example/ + example.py + setup.cfg + docs/ + build/ + html/ + index.html + tips_tricks.html + conf.py + index.txt + tips_tricks.txt + +You can simply specify the directory with the HTML files in your +:file:`setup.cfg` file: + +.. code-block:: cfg + + [upload_docs] + upload-dir = docs/build/html + + +.. program:: packaging upload_docs + +.. cmdoption:: --upload-dir + + The directory to be uploaded to the repository. By default documentation + is searched for in ``docs`` (or ``doc``) directory in project root. + +.. cmdoption:: --show-response + + Display the full response text from server; this is useful for debugging + PyPI problems. + +.. cmdoption:: --repository=URL, -r URL + + The URL of the repository to upload to. Defaults to + http://pypi.python.org/pypi (i.e., the main PyPI installation). + + +The install step +================ + +These commands are used by end-users of a project using :program:`pysetup` or +another compatible installer. Each command will run the corresponding +:command:`build_*` command and then move the built files to their destination on +the target system. + + +:command:`install_dist` +----------------------- + +Install a distribution, delegating to the other :command:`install_*` commands to +do the work. See :ref:`packaging-how-install-works` for complete usage +instructions. + + +:command:`install_data` +----------------------- + +Install data files. + + +:command:`install_distinfo` +--------------------------- + +Install files recording details of the installation as specified in :PEP:`376`. + + +:command:`install_headers` +-------------------------- + +Install C/C++ header files. + + +:command:`install_lib` +---------------------- + +Install C library files. + + +:command:`install_scripts` +-------------------------- + +Install scripts. diff --git a/Doc/packaging/configfile.rst b/Doc/packaging/configfile.rst new file mode 100644 index 0000000..825b5cb --- /dev/null +++ b/Doc/packaging/configfile.rst @@ -0,0 +1,125 @@ +.. _packaging-setup-config: + +************************************ +Writing the Setup Configuration File +************************************ + +Often, it's not possible to write down everything needed to build a distribution +*a priori*: you may need to get some information from the user, or from the +user's system, in order to proceed. As long as that information is fairly +simple---a list of directories to search for C header files or libraries, for +example---then providing a configuration file, :file:`setup.cfg`, for users to +edit is a cheap and easy way to solicit it. Configuration files also let you +provide default values for any command option, which the installer can then +override either on the command line or by editing the config file. + +The setup configuration file is a useful middle-ground between the setup script +---which, ideally, would be opaque to installers [#]_---and the command line to +the setup script, which is outside of your control and entirely up to the +installer. In fact, :file:`setup.cfg` (and any other Distutils configuration +files present on the target system) are processed after the contents of the +setup script, but before the command line. This has several useful +consequences: + +.. If you have more advanced needs, such as determining which extensions to + build based on what capabilities are present on the target system, then you + need the Distutils auto-configuration facility. This started to appear in + Distutils 0.9 but, as of this writing, isn't mature or stable enough yet + for real-world use. + +* installers can override some of what you put in :file:`setup.py` by editing + :file:`setup.cfg` + +* you can provide non-standard defaults for options that are not easily set in + :file:`setup.py` + +* installers can override anything in :file:`setup.cfg` using the command-line + options to :file:`setup.py` + +The basic syntax of the configuration file is simple:: + + [command] + option = value + ... + +where *command* is one of the Distutils commands (e.g. :command:`build_py`, +:command:`install_dist`), and *option* is one of the options that command supports. +Any number of options can be supplied for each command, and any number of +command sections can be included in the file. Blank lines are ignored, as are +comments, which run from a ``'#'`` character until the end of the line. Long +option values can be split across multiple lines simply by indenting the +continuation lines. + +You can find out the list of options supported by a particular command with the +universal :option:`--help` option, e.g. :: + + > python setup.py --help build_ext + [...] + Options for 'build_ext' command: + --build-lib (-b) directory for compiled extension modules + --build-temp (-t) directory for temporary files (build by-products) + --inplace (-i) ignore build-lib and put compiled extensions into the + source directory alongside your pure Python modules + --include-dirs (-I) list of directories to search for header files + --define (-D) C preprocessor macros to define + --undef (-U) C preprocessor macros to undefine + --swig-opts list of SWIG command-line options + [...] + +.. XXX do we want to support ``setup.py --help metadata``? + +Note that an option spelled :option:`--foo-bar` on the command line is spelled +:option:`foo_bar` in configuration files. + +For example, say you want your extensions to be built "in-place"---that is, you +have an extension :mod:`pkg.ext`, and you want the compiled extension file +(:file:`ext.so` on Unix, say) to be put in the same source directory as your +pure Python modules :mod:`pkg.mod1` and :mod:`pkg.mod2`. You can always use the +:option:`--inplace` option on the command line to ensure this:: + + python setup.py build_ext --inplace + +But this requires that you always specify the :command:`build_ext` command +explicitly, and remember to provide :option:`--inplace`. An easier way is to +"set and forget" this option, by encoding it in :file:`setup.cfg`, the +configuration file for this distribution:: + + [build_ext] + inplace = 1 + +This will affect all builds of this module distribution, whether or not you +explicitly specify :command:`build_ext`. If you include :file:`setup.cfg` in +your source distribution, it will also affect end-user builds---which is +probably a bad idea for this option, since always building extensions in-place +would break installation of the module distribution. In certain peculiar cases, +though, modules are built right in their installation directory, so this is +conceivably a useful ability. (Distributing extensions that expect to be built +in their installation directory is almost always a bad idea, though.) + +Another example: certain commands take options that vary from project to +project but not depending on the installation system, for example, +:command:`test` needs to know where your test suite is located and what test +runner to use; likewise, :command:`upload_docs` can find HTML documentation in +a :file:`doc` or :file:`docs` directory, but needs an option to find files in +:file:`docs/build/html`. Instead of having to type out these options each +time you want to run the command, you can put them in the project's +:file:`setup.cfg`:: + + [test] + suite = packaging.tests + + [upload_docs] + upload-dir = docs/build/html + + +.. seealso:: + + :ref:`packaging-config-syntax` in "Installing Python Projects" + More information on the configuration files is available in the manual for + system administrators. + + +.. rubric:: Footnotes + +.. [#] This ideal probably won't be achieved until auto-configuration is fully + supported by the Distutils. diff --git a/Doc/packaging/examples.rst b/Doc/packaging/examples.rst new file mode 100644 index 0000000..594ade0 --- /dev/null +++ b/Doc/packaging/examples.rst @@ -0,0 +1,334 @@ +.. _packaging-examples: + +******** +Examples +******** + +This chapter provides a number of basic examples to help get started with +Packaging. + + +.. _packaging-pure-mod: + +Pure Python distribution (by module) +==================================== + +If you're just distributing a couple of modules, especially if they don't live +in a particular package, you can specify them individually using the +:option:`py_modules` option in the setup script. + +In the simplest case, you'll have two files to worry about: a setup script and +the single module you're distributing, :file:`foo.py` in this example:: + + <root>/ + setup.py + foo.py + +(In all diagrams in this section, *<root>* will refer to the distribution root +directory.) A minimal setup script to describe this situation would be:: + + from packaging.core import setup + setup(name='foo', + version='1.0', + py_modules=['foo']) + +Note that the name of the distribution is specified independently with the +:option:`name` option, and there's no rule that says it has to be the same as +the name of the sole module in the distribution (although that's probably a good +convention to follow). However, the distribution name is used to generate +filenames, so you should stick to letters, digits, underscores, and hyphens. + +Since :option:`py_modules` is a list, you can of course specify multiple +modules, e.g. if you're distributing modules :mod:`foo` and :mod:`bar`, your +setup might look like this:: + + <root>/ + setup.py + foo.py + bar.py + +and the setup script might be :: + + from packaging.core import setup + setup(name='foobar', + version='1.0', + py_modules=['foo', 'bar']) + +You can put module source files into another directory, but if you have enough +modules to do that, it's probably easier to specify modules by package rather +than listing them individually. + + +.. _packaging-pure-pkg: + +Pure Python distribution (by package) +===================================== + +If you have more than a couple of modules to distribute, especially if they are +in multiple packages, it's probably easier to specify whole packages rather than +individual modules. This works even if your modules are not in a package; you +can just tell the Distutils to process modules from the root package, and that +works the same as any other package (except that you don't have to have an +:file:`__init__.py` file). + +The setup script from the last example could also be written as :: + + from packaging.core import setup + setup(name='foobar', + version='1.0', + packages=['']) + +(The empty string stands for the root package.) + +If those two files are moved into a subdirectory, but remain in the root +package, e.g.:: + + <root>/ + setup.py + src/ + foo.py + bar.py + +then you would still specify the root package, but you have to tell the +Distutils where source files in the root package live:: + + from packaging.core import setup + setup(name='foobar', + version='1.0', + package_dir={'': 'src'}, + packages=['']) + +More typically, though, you will want to distribute multiple modules in the same +package (or in sub-packages). For example, if the :mod:`foo` and :mod:`bar` +modules belong in package :mod:`foobar`, one way to lay out your source tree is + +:: + + <root>/ + setup.py + foobar/ + __init__.py + foo.py + bar.py + +This is in fact the default layout expected by the Distutils, and the one that +requires the least work to describe in your setup script:: + + from packaging.core import setup + setup(name='foobar', + version='1.0', + packages=['foobar']) + +If you want to put modules in directories not named for their package, then you +need to use the :option:`package_dir` option again. For example, if the +:file:`src` directory holds modules in the :mod:`foobar` package:: + + <root>/ + setup.py + src/ + __init__.py + foo.py + bar.py + +an appropriate setup script would be :: + + from packaging.core import setup + setup(name='foobar', + version='1.0', + package_dir={'foobar': 'src'}, + packages=['foobar']) + +Or, you might put modules from your main package right in the distribution +root:: + + <root>/ + setup.py + __init__.py + foo.py + bar.py + +in which case your setup script would be :: + + from packaging.core import setup + setup(name='foobar', + version='1.0', + package_dir={'foobar': ''}, + packages=['foobar']) + +(The empty string also stands for the current directory.) + +If you have sub-packages, they must be explicitly listed in :option:`packages`, +but any entries in :option:`package_dir` automatically extend to sub-packages. +(In other words, the Distutils does *not* scan your source tree, trying to +figure out which directories correspond to Python packages by looking for +:file:`__init__.py` files.) Thus, if the default layout grows a sub-package:: + + <root>/ + setup.py + foobar/ + __init__.py + foo.py + bar.py + subfoo/ + __init__.py + blah.py + +then the corresponding setup script would be :: + + from packaging.core import setup + setup(name='foobar', + version='1.0', + packages=['foobar', 'foobar.subfoo']) + +(Again, the empty string in :option:`package_dir` stands for the current +directory.) + + +.. _packaging-single-ext: + +Single extension module +======================= + +Extension modules are specified using the :option:`ext_modules` option. +:option:`package_dir` has no effect on where extension source files are found; +it only affects the source for pure Python modules. The simplest case, a +single extension module in a single C source file, is:: + + <root>/ + setup.py + foo.c + +If the :mod:`foo` extension belongs in the root package, the setup script for +this could be :: + + from packaging.core import setup, Extension + setup(name='foobar', + version='1.0', + ext_modules=[Extension('foo', ['foo.c'])]) + +If the extension actually belongs in a package, say :mod:`foopkg`, then + +With exactly the same source tree layout, this extension can be put in the +:mod:`foopkg` package simply by changing the name of the extension:: + + from packaging.core import setup, Extension + setup(name='foobar', + version='1.0', + packages=['foopkg'], + ext_modules=[Extension('foopkg.foo', ['foo.c'])]) + + +Checking metadata +================= + +The ``check`` command allows you to verify if your project's metadata +meets the minimum requirements to build a distribution. + +To run it, just call it using your :file:`setup.py` script. If something is +missing, ``check`` will display a warning. + +Let's take an example with a simple script:: + + from packaging.core import setup + + setup(name='foobar') + +.. TODO configure logging StreamHandler to match this output + +Running the ``check`` command will display some warnings:: + + $ python setup.py check + running check + warning: check: missing required metadata: version, home_page + warning: check: missing metadata: either (author and author_email) or + (maintainer and maintainer_email) must be supplied + + +If you use the reStructuredText syntax in the ``long_description`` field and +`Docutils <http://docutils.sourceforge.net/>`_ is installed you can check if +the syntax is fine with the ``check`` command, using the ``restructuredtext`` +option. + +For example, if the :file:`setup.py` script is changed like this:: + + from packaging.core import setup + + desc = """\ + Welcome to foobar! + =============== + + This is the description of the ``foobar`` project. + """ + + setup(name='foobar', + version='1.0', + author=u'Tarek Ziadé', + author_email='tarek@ziade.org', + summary='Foobar utilities' + description=desc, + home_page='http://example.com') + +Where the long description is broken, ``check`` will be able to detect it +by using the :mod:`docutils` parser:: + + $ python setup.py check --restructuredtext + running check + warning: check: Title underline too short. (line 2) + warning: check: Could not finish the parsing. + + +.. _packaging-reading-metadata: + +Reading the metadata +==================== + +The :func:`packaging.core.setup` function provides a command-line interface +that allows you to query the metadata fields of a project through the +:file:`setup.py` script of a given project:: + + $ python setup.py --name + foobar + +This call reads the ``name`` metadata by running the +:func:`packaging.core.setup` function. When a source or binary +distribution is created with Distutils, the metadata fields are written +in a static file called :file:`PKG-INFO`. When a Distutils-based project is +installed in Python, the :file:`PKG-INFO` file is copied alongside the modules +and packages of the distribution under :file:`NAME-VERSION-pyX.X.egg-info`, +where ``NAME`` is the name of the project, ``VERSION`` its version as defined +in the Metadata, and ``pyX.X`` the major and minor version of Python like +``2.7`` or ``3.2``. + +You can read back this static file, by using the +:class:`packaging.dist.Metadata` class and its +:func:`read_pkg_file` method:: + + >>> from packaging.metadata import Metadata + >>> metadata = Metadata() + >>> metadata.read_pkg_file(open('distribute-0.6.8-py2.7.egg-info')) + >>> metadata.name + 'distribute' + >>> metadata.version + '0.6.8' + >>> metadata.description + 'Easily download, build, install, upgrade, and uninstall Python packages' + +Notice that the class can also be instantiated with a metadata file path to +loads its values:: + + >>> pkg_info_path = 'distribute-0.6.8-py2.7.egg-info' + >>> Metadata(pkg_info_path).name + 'distribute' + + +.. XXX These comments have been here for at least ten years. Write the + sections or delete the comments (we can maybe ask Greg Ward about + the planned contents). (Unindent to make them section titles) + + .. multiple-ext:: + + Multiple extension modules + ========================== + + Putting it all together + ======================= diff --git a/Doc/packaging/extending.rst b/Doc/packaging/extending.rst new file mode 100644 index 0000000..f2d3863 --- /dev/null +++ b/Doc/packaging/extending.rst @@ -0,0 +1,95 @@ +.. _extending-packaging: + +******************* +Extending Distutils +******************* + +Distutils can be extended in various ways. Most extensions take the form of new +commands or replacements for existing commands. New commands may be written to +support new types of platform-specific packaging, for example, while +replacements for existing commands may be made to modify details of how the +command operates on a package. + +Most extensions of the packaging are made within :file:`setup.py` scripts that +want to modify existing commands; many simply add a few file extensions that +should be copied into packages in addition to :file:`.py` files as a +convenience. + +Most packaging command implementations are subclasses of the +:class:`packaging.cmd.Command` class. New commands may directly inherit from +:class:`Command`, while replacements often derive from :class:`Command` +indirectly, directly subclassing the command they are replacing. Commands are +required to derive from :class:`Command`. + +.. .. _extend-existing: + Extending existing commands + =========================== + + +.. .. _new-commands: + Writing new commands + ==================== + + +Integrating new commands +======================== + +There are different ways to integrate new command implementations into +packaging. The most difficult is to lobby for the inclusion of the new features +in packaging itself, and wait for (and require) a version of Python that +provides that support. This is really hard for many reasons. + +The most common, and possibly the most reasonable for most needs, is to include +the new implementations with your :file:`setup.py` script, and cause the +:func:`packaging.core.setup` function use them:: + + from packaging.core import setup + from packaging.command.build_py import build_py as _build_py + + class build_py(_build_py): + """Specialized Python source builder.""" + + # implement whatever needs to be different... + + setup(..., cmdclass={'build_py': build_py}) + +This approach is most valuable if the new implementations must be used to use a +particular package, as everyone interested in the package will need to have the +new command implementation. + +Beginning with Python 2.4, a third option is available, intended to allow new +commands to be added which can support existing :file:`setup.py` scripts without +requiring modifications to the Python installation. This is expected to allow +third-party extensions to provide support for additional packaging systems, but +the commands can be used for anything packaging commands can be used for. A new +configuration option, :option:`command_packages` (command-line option +:option:`--command-packages`), can be used to specify additional packages to be +searched for modules implementing commands. Like all packaging options, this +can be specified on the command line or in a configuration file. This option +can only be set in the ``[global]`` section of a configuration file, or before +any commands on the command line. If set in a configuration file, it can be +overridden from the command line; setting it to an empty string on the command +line causes the default to be used. This should never be set in a configuration +file provided with a package. + +This new option can be used to add any number of packages to the list of +packages searched for command implementations; multiple package names should be +separated by commas. When not specified, the search is only performed in the +:mod:`packaging.command` package. When :file:`setup.py` is run with the option +:option:`--command-packages` :option:`distcmds,buildcmds`, however, the packages +:mod:`packaging.command`, :mod:`distcmds`, and :mod:`buildcmds` will be searched +in that order. New commands are expected to be implemented in modules of the +same name as the command by classes sharing the same name. Given the example +command-line option above, the command :command:`bdist_openpkg` could be +implemented by the class :class:`distcmds.bdist_openpkg.bdist_openpkg` or +:class:`buildcmds.bdist_openpkg.bdist_openpkg`. + + +Adding new distribution types +============================= + +Commands that create distributions (files in the :file:`dist/` directory) need +to add ``(command, filename)`` pairs to ``self.distribution.dist_files`` so that +:command:`upload` can upload it to PyPI. The *filename* in the pair contains no +path information, only the name of the file itself. In dry-run mode, pairs +should still be added to represent what would have been created. diff --git a/Doc/packaging/index.rst b/Doc/packaging/index.rst new file mode 100644 index 0000000..d3d0dec --- /dev/null +++ b/Doc/packaging/index.rst @@ -0,0 +1,45 @@ +.. _packaging-index: + +############################## + Distributing Python Projects +############################## + +:Authors: The Fellowship of the Packaging +:Email: distutils-sig@python.org +:Release: |version| +:Date: |today| + +This document describes Packaging for Python authors, describing how to use the +module to make Python applications, packages or modules easily available to a +wider audience with very little overhead for build/release/install mechanics. + +.. toctree:: + :maxdepth: 2 + :numbered: + + tutorial + setupcfg + introduction + setupscript + configfile + sourcedist + builtdist + packageindex + uploading + examples + extending + commandhooks + commandref + + +.. seealso:: + + :ref:`packaging-install-index` + A user-centered manual which includes information on adding projects + into an existing Python installation. You do not need to be a Python + programmer to read this manual. + + :mod:`packaging` + A library reference for developers of packaging tools wanting to use + standalone building blocks like :mod:`~packaging.version` or + :mod:`~packaging.metadata`, or extend Packaging itself. diff --git a/Doc/packaging/introduction.rst b/Doc/packaging/introduction.rst new file mode 100644 index 0000000..a757ffc --- /dev/null +++ b/Doc/packaging/introduction.rst @@ -0,0 +1,193 @@ +.. _packaging-intro: + +***************************** +An Introduction to Packaging +***************************** + +This document covers using Packaging to distribute your Python modules, +concentrating on the role of developer/distributor. If you're looking for +information on installing Python modules you should refer to the +:ref:`packaging-install-index` chapter. + +Throughout this documentation, the terms "Distutils", "the Distutils" and +"Packaging" will be used interchangeably. + +.. _packaging-concepts: + +Concepts & Terminology +====================== + +Using Distutils is quite simple both for module developers and for +users/administrators installing third-party modules. As a developer, your +responsibilities (apart from writing solid, well-documented and well-tested +code, of course!) are: + +* writing a setup script (:file:`setup.py` by convention) + +* (optional) writing a setup configuration file + +* creating a source distribution + +* (optional) creating one or more "built" (binary) distributions of your + project + +All of these tasks are covered in this document. + +Not all module developers have access to multiple platforms, so one cannot +expect them to create buildt distributions for every platform. To remedy +this, it is hoped that intermediaries called *packagers* will arise to address +this need. Packagers take source distributions released by module developers, +build them on one or more platforms and release the resulting built +distributions. Thus, users on a greater range of platforms will be able to +install the most popular Python modules in the most natural way for their +platform without having to run a setup script or compile a single line of code. + + +.. _packaging-simple-example: + +A Simple Example +================ + +A setup script is usually quite simple, although since it's written in Python +there are no arbitrary limits to what you can do with it, though you should be +careful about putting expensive operations in your setup script. +Unlike, say, Autoconf-style configure scripts the setup script may be run +multiple times in the course of building and installing a module +distribution. + +If all you want to do is distribute a module called :mod:`foo`, contained in a +file :file:`foo.py`, then your setup script can be as simple as:: + + from packaging.core import setup + setup(name='foo', + version='1.0', + py_modules=['foo']) + +Some observations: + +* most information that you supply to the Distutils is supplied as keyword + arguments to the :func:`setup` function + +* those keyword arguments fall into two categories: package metadata (name, + version number, etc.) and information about what's in the package (a list + of pure Python modules in this case) + +* modules are specified by module name, not filename (the same will hold true + for packages and extensions) + +* it's recommended that you supply a little more metadata than we have in the + example. In particular your name, email address and a URL for the + project if appropriate (see section :ref:`packaging-setup-script` for an example) + +To create a source distribution for this module you would create a setup +script, :file:`setup.py`, containing the above code and run:: + + python setup.py sdist + +which will create an archive file (e.g., tarball on Unix, ZIP file on Windows) +containing your setup script :file:`setup.py`, and your module :file:`foo.py`. +The archive file will be named :file:`foo-1.0.tar.gz` (or :file:`.zip`), and +will unpack into a directory :file:`foo-1.0`. + +If an end-user wishes to install your :mod:`foo` module all he has to do is +download :file:`foo-1.0.tar.gz` (or :file:`.zip`), unpack it, and from the +:file:`foo-1.0` directory run :: + + python setup.py install + +which will copy :file:`foo.py` to the appropriate directory for +third-party modules in their Python installation. + +This simple example demonstrates some fundamental concepts of Distutils. +First, both developers and installers have the same basic user interface, i.e. +the setup script. The difference is which Distutils *commands* they use: the +:command:`sdist` command is almost exclusively for module developers, while +:command:`install` is more often used by installers (although some developers +will want to install their own code occasionally). + +If you want to make things really easy for your users, you can create more +than one built distributions for them. For instance, if you are running on a +Windows machine and want to make things easy for other Windows users, you can +create an executable installer (the most appropriate type of built distribution +for this platform) with the :command:`bdist_wininst` command. For example:: + + python setup.py bdist_wininst + +will create an executable installer, :file:`foo-1.0.win32.exe`, in the current +directory. You can find out what distribution formats are available at any time +by running :: + + python setup.py bdist --help-formats + + +.. _packaging-python-terms: + +General Python terminology +========================== + +If you're reading this document, you probably have a good idea of what Python +modules, extensions and so forth are. Nevertheless, just to be sure that +everyone is on the same page, here's a quick overview of Python terms: + +module + The basic unit of code reusability in Python: a block of code imported by + some other code. Three types of modules are important to us here: pure + Python modules, extension modules and packages. + +pure Python module + A module written in Python and contained in a single :file:`.py` file (and + possibly associated :file:`.pyc` and/or :file:`.pyo` files). Sometimes + referred to as a "pure module." + +extension module + A module written in the low-level language of the Python implementation: C/C++ + for Python, Java for Jython. Typically contained in a single dynamically + loaded pre-compiled file, e.g. a shared object (:file:`.so`) file for Python + extensions on Unix, a DLL (given the :file:`.pyd` extension) for Python + extensions on Windows, or a Java class file for Jython extensions. Note that + currently Distutils only handles C/C++ extensions for Python. + +package + A module that contains other modules, typically contained in a directory of + the filesystem and distinguished from other directories by the presence of a + file :file:`__init__.py`. + +root package + The root of the hierarchy of packages. (This isn't really a package, + since it doesn't have an :file:`__init__.py` file. But... we have to + call it something, right?) The vast majority of the standard library is + in the root package, as are many small standalone third-party modules that + don't belong to a larger module collection. Unlike regular packages, + modules in the root package can be found in many directories: in fact, + every directory listed in ``sys.path`` contributes modules to the root + package. + + +.. _packaging-term: + +Distutils-specific terminology +============================== + +The following terms apply more specifically to the domain of distributing Python +modules using Distutils: + +module distribution + A collection of Python modules distributed together as a single downloadable + resource and meant to be installed all as one. Examples of some well-known + module distributions are NumPy, SciPy, PIL (the Python Imaging + Library) or mxBase. (Module distributions would be called a *package*, + except that term is already taken in the Python context: a single module + distribution may contain zero, one, or many Python packages.) + +pure module distribution + A module distribution that contains only pure Python modules and packages. + Sometimes referred to as a "pure distribution." + +non-pure module distribution + A module distribution that contains at least one extension module. Sometimes + referred to as a "non-pure distribution." + +distribution root + The top-level directory of your source tree (or source distribution). The + directory where :file:`setup.py` exists. Generally :file:`setup.py` will + be run from this directory. diff --git a/Doc/packaging/packageindex.rst b/Doc/packaging/packageindex.rst new file mode 100644 index 0000000..cd1d598 --- /dev/null +++ b/Doc/packaging/packageindex.rst @@ -0,0 +1,104 @@ +.. _packaging-package-index: + +********************************** +Registering with the Package Index +********************************** + +The Python Package Index (PyPI) holds metadata describing distributions +packaged with packaging. The packaging command :command:`register` is used to +submit your distribution's metadata to the index. It is invoked as follows:: + + python setup.py register + +Distutils will respond with the following prompt:: + + running register + We need to know who you are, so please choose either: + 1. use your existing login, + 2. register as a new user, + 3. have the server generate a new password for you (and email it to you), or + 4. quit + Your selection [default 1]: + +Note: if your username and password are saved locally, you will not see this +menu. + +If you have not registered with PyPI, then you will need to do so now. You +should choose option 2, and enter your details as required. Soon after +submitting your details, you will receive an email which will be used to confirm +your registration. + +Once you are registered, you may choose option 1 from the menu. You will be +prompted for your PyPI username and password, and :command:`register` will then +submit your metadata to the index. + +You may submit any number of versions of your distribution to the index. If you +alter the metadata for a particular version, you may submit it again and the +index will be updated. + +PyPI holds a record for each (name, version) combination submitted. The first +user to submit information for a given name is designated the Owner of that +name. They may submit changes through the :command:`register` command or through +the web interface. They may also designate other users as Owners or Maintainers. +Maintainers may edit the package information, but not designate other Owners or +Maintainers. + +By default PyPI will list all versions of a given package. To hide certain +versions, the Hidden property should be set to yes. This must be edited through +the web interface. + + +.. _packaging-pypirc: + +The .pypirc file +================ + +The format of the :file:`.pypirc` file is as follows:: + + [packaging] + index-servers = + pypi + + [pypi] + repository: <repository-url> + username: <username> + password: <password> + +The *packaging* section defines a *index-servers* variable that lists the +name of all sections describing a repository. + +Each section describing a repository defines three variables: + +- *repository*, that defines the url of the PyPI server. Defaults to + ``http://www.python.org/pypi``. +- *username*, which is the registered username on the PyPI server. +- *password*, that will be used to authenticate. If omitted the user + will be prompt to type it when needed. + +If you want to define another server a new section can be created and +listed in the *index-servers* variable:: + + [packaging] + index-servers = + pypi + other + + [pypi] + repository: <repository-url> + username: <username> + password: <password> + + [other] + repository: http://example.com/pypi + username: <username> + password: <password> + +:command:`register` can then be called with the -r option to point the +repository to work with:: + + python setup.py register -r http://example.com/pypi + +For convenience, the name of the section that describes the repository +may also be used:: + + python setup.py register -r other diff --git a/Doc/packaging/setupcfg.rst b/Doc/packaging/setupcfg.rst new file mode 100644 index 0000000..1a394b5 --- /dev/null +++ b/Doc/packaging/setupcfg.rst @@ -0,0 +1,865 @@ +.. highlightlang:: cfg + +.. _setupcfg-spec: + +******************************************* +Specification of the :file:`setup.cfg` file +******************************************* + +:version: 0.9 + +This document describes the :file:`setup.cfg`, an ini-style configuration file +used by Packaging to replace the :file:`setup.py` file used by Distutils. +This specification is language-agnostic, and will therefore repeat some +information that's already documented for Python in the +:class:`configparser.RawConfigParser` documentation. + +.. contents:: + :depth: 3 + :local: + + +Syntax +====== + +The ini-style format used in the configuration file is a simple collection of +sections that group sets of key-value fields separated by ``=`` or ``:`` and +optional whitespace. Lines starting with ``#`` or ``;`` are comments and will +be ignored. Empty lines are also ignored. Example:: + + [section1] + # comment + name = value + name2 = "other value" + + [section2] + foo = bar + + +Parsing values +--------------- + +Here are a set of rules to parse values: + +- If a value is quoted with ``"`` chars, it's a string. If a quote character is + present in the quoted value, it can be escaped as ``\"`` or left as-is. + +- If the value is ``true``, ``t``, ``yes``, ``y`` (case-insensitive) or ``1``, + it's converted to the language equivalent of a ``True`` value; if it's + ``false``, ``f``, ``no``, ``n`` (case-insensitive) or ``0``, it's converted to + the equivalent of ``False``. + +- A value can contain multiple lines. When read, lines are converted into a + sequence of values. Each line after the first must start with a least one + space or tab character; this leading indentation will be stripped. + +- All other values are considered strings. + +Examples:: + + [section] + foo = one + two + three + + bar = false + baz = 1.3 + boo = "ok" + beee = "wqdqw pojpj w\"ddq" + + +Extending files +--------------- + +A configuration file can be extended (i.e. included) by other files. For this, +a ``DEFAULT`` section must contain an ``extends`` key whose value points to one +or more files which will be merged into the current files by adding new sections +and fields. If a file loaded by ``extends`` contains sections or keys that +already exist in the original file, they will not override the previous values. + +Contents of :file:`one.cfg`:: + + [section1] + name = value + + [section2] + foo = foo from one.cfg + +Contents of :file:`two.cfg`:: + + [DEFAULT] + extends = one.cfg + + [section2] + foo = foo from two.cfg + baz = baz from two.cfg + +The result of parsing :file:`two.cfg` is equivalent to this file:: + + [section1] + name = value + + [section2] + foo = foo from one.cfg + baz = baz from two.cfg + +Example use of multi-line notation to include more than one file:: + + [DEFAULT] + extends = one.cfg + two.cfg + +When several files are provided, they are processed sequentially, following the +precedence rules explained above. This means that the list of files should go +from most specialized to most common. + +**Tools will need to provide a way to produce a merged version of the +file**. This will be useful to let users publish a single file. + + +Description of sections and fields +================================== + +Each section contains a description of its options. + +- Options that are marked *multi* can have multiple values, one value per + line. +- Options that are marked *optional* can be omitted. +- Options that are marked *environ* can use environment markers, as described + in :PEP:`345`. + + +The sections are: + +global + Global options not related to one command. + +metadata + Name, version and other information defined by :PEP:`345`. + +files + Modules, scripts, data, documentation and other files to include in the + distribution. + +extension sections + Options used to build extension modules. + +command sections + Options given for specific commands, identical to those that can be given + on the command line. + + +Global options +-------------- + +Contains global options for Packaging. This section is shared with Distutils. + + +commands + Defined Packaging command. A command is defined by its fully + qualified name. *optional*, *multi* + + Examples:: + + [global] + commands = + package.setup.CustomSdistCommand + package.setup.BdistDeb + +compilers + Defined Packaging compiler. A compiler is defined by its fully + qualified name. *optional*, *multi* + + Example:: + + [global] + compilers = + hotcompiler.SmartCCompiler + +setup_hooks + Defines a list of callables to be called right after the :file:`setup.cfg` + file is read, before any other processing. Each value is a Python dotted + name to an object, which has to be defined in a module present in the project + directory alonside :file:`setup.cfg` or on Python's :data:`sys.path` (see + :ref:`packaging-finding-hooks`). The callables are executed in the + order they're found in the file; if one of them cannot be found, tools should + not stop, but for example produce a warning and continue with the next line. + Each callable receives the configuration as a dictionary (keys are + :file:`setup.cfg` sections, values are dictionaries of fields) and can make + any change to it. *optional*, *multi* + + Example:: + + [global] + setup_hooks = _setuphooks.customize_config + + +Metadata +-------- + +The metadata section contains the metadata for the project as described in +:PEP:`345`. Field names are case-insensitive. + +Fields: + +name + Name of the project. + +version + Version of the project. Must comply with :PEP:`386`. + +platform + Platform specification describing an operating system + supported by the distribution which is not listed in the "Operating System" + Trove classifiers (:PEP:`301`). *optional*, *multi* + +supported-platform + Binary distributions containing a PKG-INFO file will + use the Supported-Platform field in their metadata to specify the OS and + CPU for which the binary distribution was compiled. The semantics of + the Supported-Platform field are free form. *optional*, *multi* + +summary + A one-line summary of what the distribution does. + (Used to be called *description* in Distutils1.) + +description + A longer description. (Used to be called *long_description* + in Distutils1.) A file can be provided in the *description-file* field. + *optional* + +keywords + A list of additional keywords to be used to assist searching + for the distribution in a larger catalog. Comma or space-separated. + *optional* + +home-page + The URL for the distribution's home page. + +download-url + The URL from which this version of the distribution + can be downloaded. *optional* + +author + Author's name. *optional* + +author-email + Author's e-mail. *optional* + +maintainer + Maintainer's name. *optional* + +maintainer-email + Maintainer's e-mail. *optional* + +license + A text indicating the term of uses, when a trove classifier does + not match. *optional*. + +classifiers + Classification for the distribution, as described in PEP 301. + *optional*, *multi*, *environ* + +requires-dist + name of another packaging project required as a dependency. + The format is *name (version)* where version is an optional + version declaration, as described in PEP 345. *optional*, *multi*, *environ* + +provides-dist + name of another packaging project contained within this + distribution. Same format than *requires-dist*. *optional*, *multi*, + *environ* + +obsoletes-dist + name of another packaging project this version obsoletes. + Same format than *requires-dist*. *optional*, *multi*, *environ* + +requires-python + Specifies the Python version the distribution requires. The value is a + comma-separated list of version predicates, as described in PEP 345. + *optional*, *environ* + +requires-externals + a dependency in the system. This field is free-form, + and just a hint for downstream maintainers. *optional*, *multi*, + *environ* + +project-url + A label, followed by a browsable URL for the project. + "label, url". The label is limited to 32 signs. *optional*, *multi* + +One extra field not present in PEP 345 is supported: + +description-file + Path to a text file that will be used to fill the ``description`` field. + Multiple values are accepted; they must be separated by whitespace. + ``description-file`` and ``description`` are mutually exclusive. *optional* + + + +Example:: + + [metadata] + name = pypi2rpm + version = 0.1 + author = Tarek Ziadé + author-email = tarek@ziade.org + summary = Script that transforms an sdist archive into a RPM package + description-file = README + home-page = http://bitbucket.org/tarek/pypi2rpm/wiki/Home + project-url: + Repository, http://bitbucket.org/tarek/pypi2rpm/ + RSS feed, https://bitbucket.org/tarek/pypi2rpm/rss + classifier = + Development Status :: 3 - Alpha + License :: OSI Approved :: Mozilla Public License 1.1 (MPL 1.1) + +You should not give any explicit value for metadata-version: it will be guessed +from the fields present in the file. + + +Files +----- + +This section describes the files included in the project. + +packages_root + the root directory containing all packages and modules + (default: current directory). *optional* + +packages + a list of packages the project includes *optional*, *multi* + +modules + a list of packages the project includes *optional*, *multi* + +scripts + a list of scripts the project includes *optional*, *multi* + +extra_files + a list of patterns to include extra files *optional*, + *multi* + +Example:: + + [files] + packages_root = src + packages = + pypi2rpm + pypi2rpm.command + + scripts = + pypi2rpm/pypi2rpm.py + + extra_files = + setup.py + README + + +.. Note:: + The :file:`setup.cfg` configuration file is included by default. Contrary to + Distutils, :file:`README` (or :file:`README.txt`) and :file:`setup.py` are + not included by default. + + +Resources +^^^^^^^^^ + +This section describes the files used by the project which must not be installed +in the same place that python modules or libraries, they are called +**resources**. They are for example documentation files, script files, +databases, etc... + +For declaring resources, you must use this notation:: + + source = destination + +Data-files are declared in the **resources** field in the **file** section, for +example:: + + [files] + resources = + source1 = destination1 + source2 = destination2 + +The **source** part of the declaration are relative paths of resources files +(using unix path separator **/**). For example, if you've this source tree:: + + foo/ + doc/ + doc.man + scripts/ + foo.sh + +Your setup.cfg will look like:: + + [files] + resources = + doc/doc.man = destination_doc + scripts/foo.sh = destination_scripts + +The final paths where files will be placed are composed by : **source** + +**destination**. In the previous example, **doc/doc.man** will be placed in +**destination_doc/doc/doc.man** and **scripts/foo.sh** will be placed in +**destination_scripts/scripts/foo.sh**. (If you want more control on the final +path, take a look at :ref:`setupcfg-resources-base-prefix`). + +The **destination** part of resources declaration are paths with categories. +Indeed, it's generally a bad idea to give absolute path as it will be cross +incompatible. So, you must use resources categories in your **destination** +declaration. Categories will be replaced by their real path at the installation +time. Using categories is all benefit, your declaration will be simpler, cross +platform and it will allow packager to place resources files where they want +without breaking your code. + +Categories can be specified by using this syntax:: + + {category} + +Default categories are: + +* config +* appdata +* appdata.arch +* appdata.persistent +* appdata.disposable +* help +* icon +* scripts +* doc +* info +* man + +A special category also exists **{distribution.name}** that will be replaced by +the name of the distribution, but as most of the defaults categories use them, +so it's not necessary to add **{distribution.name}** into your destination. + +If you use categories in your declarations, and you are encouraged to do, final +path will be:: + + source + destination_expanded + +.. _example_final_path: + +For example, if you have this setup.cfg:: + + [metadata] + name = foo + + [files] + resources = + doc/doc.man = {doc} + +And if **{doc}** is replaced by **{datadir}/doc/{distribution.name}**, final +path will be:: + + {datadir}/doc/foo/doc/doc.man + +Where {datafir} category will be platform-dependent. + + +More control on source part +""""""""""""""""""""""""""" + +Glob syntax +''''''''''' + +When you declare source file, you can use a glob-like syntax to match multiples file, for example:: + + scripts/* = {script} + +Will match all the files in the scripts directory and placed them in the script category. + +Glob tokens are: + + * ``*``: match all files. + * ``?``: match any character. + * ``**``: match any level of tree recursion (even 0). + * ``{}``: will match any part separated by comma (example: ``{sh,bat}``). + +.. TODO Add examples + +Order of declaration +'''''''''''''''''''' + +The order of declaration is important if one file match multiple rules. The last +rules matched by file is used, this is useful if you have this source tree:: + + foo/ + doc/ + index.rst + setup.rst + documentation.txt + doc.tex + README + +And you want all the files in the doc directory to be placed in {doc} category, +but README must be placed in {help} category, instead of listing all the files +one by one, you can declare them in this way:: + + [files] + resources = + doc/* = {doc} + doc/README = {help} + +Exclude +''''''' + +You can exclude some files of resources declaration by giving no destination, it +can be useful if you have a non-resources file in the same directory of +resources files:: + + foo/ + doc/ + RELEASES + doc.tex + documentation.txt + docu.rst + +Your **files** section will be:: + + [files] + resources = + doc/* = {doc} + doc/RELEASES = + +More control on destination part +"""""""""""""""""""""""""""""""" + +.. _setupcfg-resources-base-prefix: + +Defining a base prefix +'''''''''''''''''''''' + +When you define your resources, you can have more control of how the final path +is computed. + +By default, the final path is:: + + destination + source + +This can generate long paths, for example (example_final_path_):: + + {datadir}/doc/foo/doc/doc.man + +When you declare your source, you can use whitespace to split the source in +**prefix** **suffix**. So, for example, if you have this source:: + + docs/ doc.man + +The **prefix** is "docs/" and the **suffix** is "doc.html". + +.. note:: + + Separator can be placed after a path separator or replace it. So these two + sources are equivalent:: + + docs/ doc.man + docs doc.man + +.. note:: + + Glob syntax is working the same way with standard source and split source. + So these rules:: + + docs/* + docs/ * + docs * + + Will match all the files in the docs directory. + +When you use split source, the final path is computed this way:: + + destination + prefix + +So for example, if you have this setup.cfg:: + + [metadata] + name = foo + + [files] + resources = + doc/ doc.man = {doc} + +And if **{doc}** is replaced by **{datadir}/doc/{distribution.name}**, final +path will be:: + + {datadir}/doc/foo/doc.man + + +Overwriting paths for categories +"""""""""""""""""""""""""""""""" + +This part is intended for system administrators or downstream OS packagers. + +The real paths of categories are registered in the *sysconfig.cfg* file +installed in your python installation. This file uses an ini format too. +The content of the file is organized into several sections: + +* globals: Standard categories's paths. +* posix_prefix: Standard paths for categories and installation paths for posix + system. +* other ones XXX + +Standard categories paths are platform independent, they generally refers to +other categories, which are platform dependent. :mod:`sysconfig` will choose +these category from sections matching os.name. For example:: + + doc = {datadir}/doc/{distribution.name} + +It refers to datadir category, which can be different between platforms. In +posix system, it may be:: + + datadir = /usr/share + +So the final path will be:: + + doc = /usr/share/doc/{distribution.name} + +The platform-dependent categories are: + +* confdir +* datadir +* libdir +* base + + +Defining extra categories +""""""""""""""""""""""""" + +.. TODO + + +Examples +"""""""" + +These examples are incremental but work unitarily. + +Resources in root dir +''''''''''''''''''''' + +Source tree:: + + babar-1.0/ + README + babar.sh + launch.sh + babar.py + +:file:`setup.cfg`:: + + [files] + resources = + README = {doc} + *.sh = {scripts} + +So babar.sh and launch.sh will be placed in {scripts} directory. + +Now let's move all the scripts into a scripts directory. + +Resources in sub-directory +'''''''''''''''''''''''''' + +Source tree:: + + babar-1.1/ + README + scripts/ + babar.sh + launch.sh + LAUNCH + babar.py + +:file:`setup.cfg`:: + + [files] + resources = + README = {doc} + scripts/ LAUNCH = {doc} + scripts/ *.sh = {scripts} + +It's important to use the separator after scripts/ to install all the shell +scripts into {scripts} instead of {scripts}/scripts. + +Now let's add some docs. + +Resources in multiple sub-directories +''''''''''''''''''''''''''''''''''''' + +Source tree:: + + babar-1.2/ + README + scripts/ + babar.sh + launch.sh + LAUNCH + docs/ + api + man + babar.py + +:file:`setup.cfg`:: + + [files] + resources = + README = {doc} + scripts/ LAUNCH = {doc} + scripts/ *.sh = {scripts} + doc/ * = {doc} + doc/ man = {man} + +You want to place all the file in the docs script into {doc} category, instead +of man, which must be placed into {man} category, we will use the order of +declaration of globs to choose the destination, the last glob that match the +file is used. + +Now let's add some scripts for windows users. + +Complete example +'''''''''''''''' + +Source tree:: + + babar-1.3/ + README + doc/ + api + man + scripts/ + babar.sh + launch.sh + babar.bat + launch.bat + LAUNCH + +:file:`setup.cfg`:: + + [files] + resources = + README = {doc} + scripts/ LAUNCH = {doc} + scripts/ *.{sh,bat} = {scripts} + doc/ * = {doc} + doc/ man = {man} + +We use brace expansion syntax to place all the shell and batch scripts into +{scripts} category. + + +Extension sections +------------------ + +If a project includes extension modules written in C or C++, each one of them +needs to have its options defined in a dedicated section. Here's an example:: + + [files] + packages = coconut + + [extension: coconut._fastcoconut] + language = cxx + sources = cxx_src/cononut_utils.cxx + cxx_src/python_module.cxx + include_dirs = /usr/include/gecode + /usr/include/blitz + extra_compile_args = + -fPIC -O2 + -DGECODE_VERSION=$(./gecode_version) -- sys.platform != 'win32' + /DGECODE_VERSION='win32' -- sys.platform == 'win32' + +The section name must start with ``extension:``; the right-hand part is used as +the full name (including a parent package, if any) of the extension. Whitespace +around the extension name is allowed. If the extension module is not standalone +(e.g. ``_bisect``) but part of a package (e.g. ``thing._speedups``), the parent +package must be listed in the ``packages`` field. +Valid fields and their values are listed in the documentation of the +:class:`packaging.compiler.extension.Extension` class; values documented as +Python lists translate to multi-line values in the configuration file. In +addition, multi-line values accept environment markers on each line, after a +``--``. + + +Command sections +---------------- + +To pass options to commands without having to type them on the command line +for each invocation, you can write them in the :file:`setup.cfg` file, in a +section named after the command. Example:: + + [sdist] + # special function to add custom files + manifest-builders = package.setup.list_extra_files + + [build] + use-2to3 = True + + [build_ext] + inplace = on + + [check] + strict = on + all = on + +Option values given in the configuration file can be overriden on the command +line. See :ref:`packaging-setup-config` for more information. + + +Extensibility +============= + +Every section can have fields that are not part of this specification. They are +called **extensions**. + +An extension field starts with ``X-``. Example:: + + [metadata] + name = Distribute + X-Debian-Name = python-distribute + + +Changes in the specification +============================ + +The versioning scheme for this specification is **MAJOR.MINOR**. Changes in the +specification will cause the version number to be updated. + +Changes to the minor number reflect backwards-compatible changes: + +- New fields and sections (optional or mandatory) can be added. +- Optional fields can be removed. + +The major number will be incremented for backwards-incompatible changes: + +- Mandatory fields or sections are removed. +- Fields change their meaning. + +As a consequence, a tool written to consume 1.5 has these properties: + +- Can read 1.1, 1.2 and all versions < 1.5, since the tool knows what + optional fields weren't there. + + .. XXX clarify + +- Can also read 1.6 and other 1.x versions: The tool will just ignore fields it + doesn't know about, even if they are mandatory in the new version. If + optional fields were removed, the tool will just consider them absent. + +- Cannot read 2.x and should refuse to interpret such files. + +A tool written to produce 1.x should have these properties: + +- Writes all mandatory fields. +- May write optional fields. + + +Acknowledgments +=============== + +This specification includes work and feedback from these people: + +- Tarek Ziadé +- Julien Jehannet +- Boris Feld +- Éric Araujo + +(If your name is missing, please :ref:`let us know <reporting-bugs>`.) diff --git a/Doc/packaging/setupscript.rst b/Doc/packaging/setupscript.rst new file mode 100644 index 0000000..dbac3dd --- /dev/null +++ b/Doc/packaging/setupscript.rst @@ -0,0 +1,689 @@ +.. _packaging-setup-script: + +************************ +Writing the Setup Script +************************ + +The setup script is the center of all activity in building, distributing, and +installing modules using Distutils. The main purpose of the setup script is +to describe your module distribution to Distutils, so that the various +commands that operate on your modules do the right thing. As we saw in section +:ref:`packaging-simple-example`, the setup script consists mainly of a +call to :func:`setup` where the most information is supplied as +keyword arguments to :func:`setup`. + +Here's a slightly more involved example, which we'll follow for the next couple +of sections: a setup script that could be used for Packaging itself:: + + #!/usr/bin/env python + + from packaging.core import setup, find_packages + + setup(name='Packaging', + version='1.0', + summary='Python Distribution Utilities', + keywords=['packaging', 'packaging'], + author=u'Tarek Ziadé', + author_email='tarek@ziade.org', + home_page='http://bitbucket.org/tarek/packaging/wiki/Home', + license='PSF', + packages=find_packages()) + + +There are only two differences between this and the trivial one-file +distribution presented in section :ref:`packaging-simple-example`: more +metadata and the specification of pure Python modules by package rather than +by module. This is important since Ristutils consist of a couple of dozen +modules split into (so far) two packages; an explicit list of every module +would be tedious to generate and difficult to maintain. For more information +on the additional metadata, see section :ref:`packaging-metadata`. + +Note that any pathnames (files or directories) supplied in the setup script +should be written using the Unix convention, i.e. slash-separated. The +Distutils will take care of converting this platform-neutral representation into +whatever is appropriate on your current platform before actually using the +pathname. This makes your setup script portable across operating systems, which +of course is one of the major goals of the Distutils. In this spirit, all +pathnames in this document are slash-separated. + +This, of course, only applies to pathnames given to Distutils functions. If +you, for example, use standard Python functions such as :func:`glob.glob` or +:func:`os.listdir` to specify files, you should be careful to write portable +code instead of hardcoding path separators:: + + glob.glob(os.path.join('mydir', 'subdir', '*.html')) + os.listdir(os.path.join('mydir', 'subdir')) + + +.. _packaging-listing-packages: + +Listing whole packages +====================== + +The :option:`packages` option tells the Distutils to process (build, distribute, +install, etc.) all pure Python modules found in each package mentioned in the +:option:`packages` list. In order to do this, of course, there has to be a +correspondence between package names and directories in the filesystem. The +default correspondence is the most obvious one, i.e. package :mod:`packaging` is +found in the directory :file:`packaging` relative to the distribution root. +Thus, when you say ``packages = ['foo']`` in your setup script, you are +promising that the Distutils will find a file :file:`foo/__init__.py` (which +might be spelled differently on your system, but you get the idea) relative to +the directory where your setup script lives. If you break this promise, the +Distutils will issue a warning but still process the broken package anyway. + +If you use a different convention to lay out your source directory, that's no +problem: you just have to supply the :option:`package_dir` option to tell the +Distutils about your convention. For example, say you keep all Python source +under :file:`lib`, so that modules in the "root package" (i.e., not in any +package at all) are in :file:`lib`, modules in the :mod:`foo` package are in +:file:`lib/foo`, and so forth. Then you would put :: + + package_dir = {'': 'lib'} + +in your setup script. The keys to this dictionary are package names, and an +empty package name stands for the root package. The values are directory names +relative to your distribution root. In this case, when you say ``packages = +['foo']``, you are promising that the file :file:`lib/foo/__init__.py` exists. + +Another possible convention is to put the :mod:`foo` package right in +:file:`lib`, the :mod:`foo.bar` package in :file:`lib/bar`, etc. This would be +written in the setup script as :: + + package_dir = {'foo': 'lib'} + +A ``package: dir`` entry in the :option:`package_dir` dictionary implicitly +applies to all packages below *package*, so the :mod:`foo.bar` case is +automatically handled here. In this example, having ``packages = ['foo', +'foo.bar']`` tells the Distutils to look for :file:`lib/__init__.py` and +:file:`lib/bar/__init__.py`. (Keep in mind that although :option:`package_dir` +applies recursively, you must explicitly list all packages in +:option:`packages`: the Distutils will *not* recursively scan your source tree +looking for any directory with an :file:`__init__.py` file.) + + +.. _packaging-listing-modules: + +Listing individual modules +========================== + +For a small module distribution, you might prefer to list all modules rather +than listing packages---especially the case of a single module that goes in the +"root package" (i.e., no package at all). This simplest case was shown in +section :ref:`packaging-simple-example`; here is a slightly more involved +example:: + + py_modules = ['mod1', 'pkg.mod2'] + +This describes two modules, one of them in the "root" package, the other in the +:mod:`pkg` package. Again, the default package/directory layout implies that +these two modules can be found in :file:`mod1.py` and :file:`pkg/mod2.py`, and +that :file:`pkg/__init__.py` exists as well. And again, you can override the +package/directory correspondence using the :option:`package_dir` option. + + +.. _packaging-describing-extensions: + +Describing extension modules +============================ + +Just as writing Python extension modules is a bit more complicated than writing +pure Python modules, describing them to the Distutils is a bit more complicated. +Unlike pure modules, it's not enough just to list modules or packages and expect +the Distutils to go out and find the right files; you have to specify the +extension name, source file(s), and any compile/link requirements (include +directories, libraries to link with, etc.). + +.. XXX read over this section + +All of this is done through another keyword argument to :func:`setup`, the +:option:`ext_modules` option. :option:`ext_modules` is just a list of +:class:`Extension` instances, each of which describes a single extension module. +Suppose your distribution includes a single extension, called :mod:`foo` and +implemented by :file:`foo.c`. If no additional instructions to the +compiler/linker are needed, describing this extension is quite simple:: + + Extension('foo', ['foo.c']) + +The :class:`Extension` class can be imported from :mod:`packaging.core` along +with :func:`setup`. Thus, the setup script for a module distribution that +contains only this one extension and nothing else might be:: + + from packaging.core import setup, Extension + setup(name='foo', + version='1.0', + ext_modules=[Extension('foo', ['foo.c'])]) + +The :class:`Extension` class (actually, the underlying extension-building +machinery implemented by the :command:`build_ext` command) supports a great deal +of flexibility in describing Python extensions, which is explained in the +following sections. + + +Extension names and packages +---------------------------- + +The first argument to the :class:`Extension` constructor is always the name of +the extension, including any package names. For example, :: + + Extension('foo', ['src/foo1.c', 'src/foo2.c']) + +describes an extension that lives in the root package, while :: + + Extension('pkg.foo', ['src/foo1.c', 'src/foo2.c']) + +describes the same extension in the :mod:`pkg` package. The source files and +resulting object code are identical in both cases; the only difference is where +in the filesystem (and therefore where in Python's namespace hierarchy) the +resulting extension lives. + +If you have a number of extensions all in the same package (or all under the +same base package), use the :option:`ext_package` keyword argument to +:func:`setup`. For example, :: + + setup(..., + ext_package='pkg', + ext_modules=[Extension('foo', ['foo.c']), + Extension('subpkg.bar', ['bar.c'])]) + +will compile :file:`foo.c` to the extension :mod:`pkg.foo`, and :file:`bar.c` to +:mod:`pkg.subpkg.bar`. + + +Extension source files +---------------------- + +The second argument to the :class:`Extension` constructor is a list of source +files. Since the Distutils currently only support C, C++, and Objective-C +extensions, these are normally C/C++/Objective-C source files. (Be sure to use +appropriate extensions to distinguish C++\ source files: :file:`.cc` and +:file:`.cpp` seem to be recognized by both Unix and Windows compilers.) + +However, you can also include SWIG interface (:file:`.i`) files in the list; the +:command:`build_ext` command knows how to deal with SWIG extensions: it will run +SWIG on the interface file and compile the resulting C/C++ file into your +extension. + +.. XXX SWIG support is rough around the edges and largely untested! + +This warning notwithstanding, options to SWIG can be currently passed like +this:: + + setup(..., + ext_modules=[Extension('_foo', ['foo.i'], + swig_opts=['-modern', '-I../include'])], + py_modules=['foo']) + +Or on the command line like this:: + + > python setup.py build_ext --swig-opts="-modern -I../include" + +On some platforms, you can include non-source files that are processed by the +compiler and included in your extension. Currently, this just means Windows +message text (:file:`.mc`) files and resource definition (:file:`.rc`) files for +Visual C++. These will be compiled to binary resource (:file:`.res`) files and +linked into the executable. + + +Preprocessor options +-------------------- + +Three optional arguments to :class:`Extension` will help if you need to specify +include directories to search or preprocessor macros to define/undefine: +``include_dirs``, ``define_macros``, and ``undef_macros``. + +For example, if your extension requires header files in the :file:`include` +directory under your distribution root, use the ``include_dirs`` option:: + + Extension('foo', ['foo.c'], include_dirs=['include']) + +You can specify absolute directories there; if you know that your extension will +only be built on Unix systems with X11R6 installed to :file:`/usr`, you can get +away with :: + + Extension('foo', ['foo.c'], include_dirs=['/usr/include/X11']) + +You should avoid this sort of non-portable usage if you plan to distribute your +code: it's probably better to write C code like :: + + #include <X11/Xlib.h> + +If you need to include header files from some other Python extension, you can +take advantage of the fact that header files are installed in a consistent way +by the Distutils :command:`install_header` command. For example, the Numerical +Python header files are installed (on a standard Unix installation) to +:file:`/usr/local/include/python1.5/Numerical`. (The exact location will differ +according to your platform and Python installation.) Since the Python include +directory---\ :file:`/usr/local/include/python1.5` in this case---is always +included in the search path when building Python extensions, the best approach +is to write C code like :: + + #include <Numerical/arrayobject.h> + +.. TODO check if it's d2.sysconfig or the new sysconfig module now + +If you must put the :file:`Numerical` include directory right into your header +search path, though, you can find that directory using the Distutils +:mod:`packaging.sysconfig` module:: + + from packaging.sysconfig import get_python_inc + incdir = os.path.join(get_python_inc(plat_specific=1), 'Numerical') + setup(..., + Extension(..., include_dirs=[incdir])) + +Even though this is quite portable---it will work on any Python installation, +regardless of platform---it's probably easier to just write your C code in the +sensible way. + +You can define and undefine preprocessor macros with the ``define_macros`` and +``undef_macros`` options. ``define_macros`` takes a list of ``(name, value)`` +tuples, where ``name`` is the name of the macro to define (a string) and +``value`` is its value: either a string or ``None``. (Defining a macro ``FOO`` +to ``None`` is the equivalent of a bare ``#define FOO`` in your C source: with +most compilers, this sets ``FOO`` to the string ``1``.) ``undef_macros`` is +just a list of macros to undefine. + +For example:: + + Extension(..., + define_macros=[('NDEBUG', '1'), + ('HAVE_STRFTIME', None)], + undef_macros=['HAVE_FOO', 'HAVE_BAR']) + +is the equivalent of having this at the top of every C source file:: + + #define NDEBUG 1 + #define HAVE_STRFTIME + #undef HAVE_FOO + #undef HAVE_BAR + + +Library options +--------------- + +You can also specify the libraries to link against when building your extension, +and the directories to search for those libraries. The ``libraries`` option is +a list of libraries to link against, ``library_dirs`` is a list of directories +to search for libraries at link-time, and ``runtime_library_dirs`` is a list of +directories to search for shared (dynamically loaded) libraries at run-time. + +For example, if you need to link against libraries known to be in the standard +library search path on target systems :: + + Extension(..., + libraries=['gdbm', 'readline']) + +If you need to link with libraries in a non-standard location, you'll have to +include the location in ``library_dirs``:: + + Extension(..., + library_dirs=['/usr/X11R6/lib'], + libraries=['X11', 'Xt']) + +(Again, this sort of non-portable construct should be avoided if you intend to +distribute your code.) + +.. XXX Should mention clib libraries here or somewhere else! + + +Other options +------------- + +There are still some other options which can be used to handle special cases. + +The :option:`optional` option is a boolean; if it is true, +a build failure in the extension will not abort the build process, but +instead simply not install the failing extension. + +The :option:`extra_objects` option is a list of object files to be passed to the +linker. These files must not have extensions, as the default extension for the +compiler is used. + +:option:`extra_compile_args` and :option:`extra_link_args` can be used to +specify additional command-line options for the respective compiler and linker +command lines. + +:option:`export_symbols` is only useful on Windows. It can contain a list of +symbols (functions or variables) to be exported. This option is not needed when +building compiled extensions: Distutils will automatically add ``initmodule`` +to the list of exported symbols. + +The :option:`depends` option is a list of files that the extension depends on +(for example header files). The build command will call the compiler on the +sources to rebuild extension if any on this files has been modified since the +previous build. + +Relationships between Distributions and Packages +================================================ + +.. FIXME rewrite to update to PEP 345 (but without dist/release confusion) + +A distribution may relate to packages in three specific ways: + +#. It can require packages or modules. + +#. It can provide packages or modules. + +#. It can obsolete packages or modules. + +These relationships can be specified using keyword arguments to the +:func:`packaging.core.setup` function. + +Dependencies on other Python modules and packages can be specified by supplying +the *requires* keyword argument to :func:`setup`. The value must be a list of +strings. Each string specifies a package that is required, and optionally what +versions are sufficient. + +To specify that any version of a module or package is required, the string +should consist entirely of the module or package name. Examples include +``'mymodule'`` and ``'xml.parsers.expat'``. + +If specific versions are required, a sequence of qualifiers can be supplied in +parentheses. Each qualifier may consist of a comparison operator and a version +number. The accepted comparison operators are:: + + < > == + <= >= != + +These can be combined by using multiple qualifiers separated by commas (and +optional whitespace). In this case, all of the qualifiers must be matched; a +logical AND is used to combine the evaluations. + +Let's look at a bunch of examples: + ++-------------------------+----------------------------------------------+ +| Requires Expression | Explanation | ++=========================+==============================================+ +| ``==1.0`` | Only version ``1.0`` is compatible | ++-------------------------+----------------------------------------------+ +| ``>1.0, !=1.5.1, <2.0`` | Any version after ``1.0`` and before ``2.0`` | +| | is compatible, except ``1.5.1`` | ++-------------------------+----------------------------------------------+ + +Now that we can specify dependencies, we also need to be able to specify what we +provide that other distributions can require. This is done using the *provides* +keyword argument to :func:`setup`. The value for this keyword is a list of +strings, each of which names a Python module or package, and optionally +identifies the version. If the version is not specified, it is assumed to match +that of the distribution. + +Some examples: + ++---------------------+----------------------------------------------+ +| Provides Expression | Explanation | ++=====================+==============================================+ +| ``mypkg`` | Provide ``mypkg``, using the distribution | +| | version | ++---------------------+----------------------------------------------+ +| ``mypkg (1.1)`` | Provide ``mypkg`` version 1.1, regardless of | +| | the distribution version | ++---------------------+----------------------------------------------+ + +A package can declare that it obsoletes other packages using the *obsoletes* +keyword argument. The value for this is similar to that of the *requires* +keyword: a list of strings giving module or package specifiers. Each specifier +consists of a module or package name optionally followed by one or more version +qualifiers. Version qualifiers are given in parentheses after the module or +package name. + +The versions identified by the qualifiers are those that are obsoleted by the +distribution being described. If no qualifiers are given, all versions of the +named module or package are understood to be obsoleted. + +.. _packaging-installing-scripts: + +Installing Scripts +================== + +So far we have been dealing with pure and non-pure Python modules, which are +usually not run by themselves but imported by scripts. + +Scripts are files containing Python source code, intended to be started from the +command line. Scripts don't require Distutils to do anything very complicated. +The only clever feature is that if the first line of the script starts with +``#!`` and contains the word "python", the Distutils will adjust the first line +to refer to the current interpreter location. By default, it is replaced with +the current interpreter location. The :option:`--executable` (or :option:`-e`) +option will allow the interpreter path to be explicitly overridden. + +The :option:`scripts` option simply is a list of files to be handled in this +way. From the PyXML setup script:: + + setup(..., + scripts=['scripts/xmlproc_parse', 'scripts/xmlproc_val']) + +All the scripts will also be added to the ``MANIFEST`` file if no template is +provided. See :ref:`packaging-manifest`. + +.. _packaging-installing-package-data: + +Installing Package Data +======================= + +Often, additional files need to be installed into a package. These files are +often data that's closely related to the package's implementation, or text files +containing documentation that might be of interest to programmers using the +package. These files are called :dfn:`package data`. + +Package data can be added to packages using the ``package_data`` keyword +argument to the :func:`setup` function. The value must be a mapping from +package name to a list of relative path names that should be copied into the +package. The paths are interpreted as relative to the directory containing the +package (information from the ``package_dir`` mapping is used if appropriate); +that is, the files are expected to be part of the package in the source +directories. They may contain glob patterns as well. + +The path names may contain directory portions; any necessary directories will be +created in the installation. + +For example, if a package should contain a subdirectory with several data files, +the files can be arranged like this in the source tree:: + + setup.py + src/ + mypkg/ + __init__.py + module.py + data/ + tables.dat + spoons.dat + forks.dat + +The corresponding call to :func:`setup` might be:: + + setup(..., + packages=['mypkg'], + package_dir={'mypkg': 'src/mypkg'}, + package_data={'mypkg': ['data/*.dat']}) + + +All the files that match ``package_data`` will be added to the ``MANIFEST`` +file if no template is provided. See :ref:`packaging-manifest`. + + +.. _packaging-additional-files: + +Installing Additional Files +=========================== + +The :option:`data_files` option can be used to specify additional files needed +by the module distribution: configuration files, message catalogs, data files, +anything which doesn't fit in the previous categories. + +:option:`data_files` specifies a sequence of (*directory*, *files*) pairs in the +following way:: + + setup(..., + data_files=[('bitmaps', ['bm/b1.gif', 'bm/b2.gif']), + ('config', ['cfg/data.cfg']), + ('/etc/init.d', ['init-script'])]) + +Note that you can specify the directory names where the data files will be +installed, but you cannot rename the data files themselves. + +Each (*directory*, *files*) pair in the sequence specifies the installation +directory and the files to install there. If *directory* is a relative path, it +is interpreted relative to the installation prefix (Python's ``sys.prefix`` for +pure-Python packages, ``sys.exec_prefix`` for packages that contain extension +modules). Each file name in *files* is interpreted relative to the +:file:`setup.py` script at the top of the package source distribution. No +directory information from *files* is used to determine the final location of +the installed file; only the name of the file is used. + +You can specify the :option:`data_files` options as a simple sequence of files +without specifying a target directory, but this is not recommended, and the +:command:`install_dist` command will print a warning in this case. To install data +files directly in the target directory, an empty string should be given as the +directory. + +All the files that match ``data_files`` will be added to the ``MANIFEST`` file +if no template is provided. See :ref:`packaging-manifest`. + + + +.. _packaging-metadata: + +Metadata reference +================== + +The setup script may include additional metadata beyond the name and version. +This table describes required and additional information: + +.. TODO synchronize with setupcfg; link to it (but don't remove it, it's a + useful summary) + ++----------------------+---------------------------+-----------------+--------+ +| Meta-Data | Description | Value | Notes | ++======================+===========================+=================+========+ +| ``name`` | name of the project | short string | \(1) | ++----------------------+---------------------------+-----------------+--------+ +| ``version`` | version of this release | short string | (1)(2) | ++----------------------+---------------------------+-----------------+--------+ +| ``author`` | project author's name | short string | \(3) | ++----------------------+---------------------------+-----------------+--------+ +| ``author_email`` | email address of the | email address | \(3) | +| | project author | | | ++----------------------+---------------------------+-----------------+--------+ +| ``maintainer`` | project maintainer's name | short string | \(3) | ++----------------------+---------------------------+-----------------+--------+ +| ``maintainer_email`` | email address of the | email address | \(3) | +| | project maintainer | | | ++----------------------+---------------------------+-----------------+--------+ +| ``home_page`` | home page for the project | URL | \(1) | ++----------------------+---------------------------+-----------------+--------+ +| ``summary`` | short description of the | short string | | +| | project | | | ++----------------------+---------------------------+-----------------+--------+ +| ``description`` | longer description of the | long string | \(5) | +| | project | | | ++----------------------+---------------------------+-----------------+--------+ +| ``download_url`` | location where the | URL | | +| | project may be downloaded | | | ++----------------------+---------------------------+-----------------+--------+ +| ``classifiers`` | a list of classifiers | list of strings | \(4) | ++----------------------+---------------------------+-----------------+--------+ +| ``platforms`` | a list of platforms | list of strings | | ++----------------------+---------------------------+-----------------+--------+ +| ``license`` | license for the release | short string | \(6) | ++----------------------+---------------------------+-----------------+--------+ + +Notes: + +(1) + These fields are required. + +(2) + It is recommended that versions take the form *major.minor[.patch[.sub]]*. + +(3) + Either the author or the maintainer must be identified. + +(4) + The list of classifiers is available from the `PyPI website + <http://pypi.python.org/pypi>`_. See also :mod:`packaging.create`. + +(5) + The ``description`` field is used by PyPI when you are registering a + release, to build its PyPI page. + +(6) + The ``license`` field is a text indicating the license covering the + distribution where the license is not a selection from the "License" Trove + classifiers. See the ``Classifier`` field. Notice that + there's a ``licence`` distribution option which is deprecated but still + acts as an alias for ``license``. + +'short string' + A single line of text, not more than 200 characters. + +'long string' + Multiple lines of plain text in reStructuredText format (see + http://docutils.sf.net/). + +'list of strings' + See below. + +In Python 2.x, "string value" means a unicode object. If a byte string (str or +bytes) is given, it has to be valid ASCII. + +.. TODO move this section to the version document, keep a summary, add a link + +Encoding the version information is an art in itself. Python projects generally +adhere to the version format *major.minor[.patch][sub]*. The major number is 0 +for initial, experimental releases of software. It is incremented for releases +that represent major milestones in a project. The minor number is incremented +when important new features are added to the project. The patch number +increments when bug-fix releases are made. Additional trailing version +information is sometimes used to indicate sub-releases. These are +"a1,a2,...,aN" (for alpha releases, where functionality and API may change), +"b1,b2,...,bN" (for beta releases, which only fix bugs) and "pr1,pr2,...,prN" +(for final pre-release release testing). Some examples: + +0.1.0 + the first, experimental release of a project + +1.0.1a2 + the second alpha release of the first patch version of 1.0 + +:option:`classifiers` are specified in a Python list:: + + setup(..., + classifiers=[ + 'Development Status :: 4 - Beta', + 'Environment :: Console', + 'Environment :: Web Environment', + 'Intended Audience :: End Users/Desktop', + 'Intended Audience :: Developers', + 'Intended Audience :: System Administrators', + 'License :: OSI Approved :: Python Software Foundation License', + 'Operating System :: MacOS :: MacOS X', + 'Operating System :: Microsoft :: Windows', + 'Operating System :: POSIX', + 'Programming Language :: Python', + 'Topic :: Communications :: Email', + 'Topic :: Office/Business', + 'Topic :: Software Development :: Bug Tracking', + ]) + + +Debugging the setup script +========================== + +Sometimes things go wrong, and the setup script doesn't do what the developer +wants. + +Distutils catches any exceptions when running the setup script, and print a +simple error message before the script is terminated. The motivation for this +behaviour is to not confuse administrators who don't know much about Python and +are trying to install a project. If they get a big long traceback from deep +inside the guts of Distutils, they may think the project or the Python +installation is broken because they don't read all the way down to the bottom +and see that it's a permission problem. + +.. FIXME DISTUTILS_DEBUG is dead, document logging/warnings here + +On the other hand, this doesn't help the developer to find the cause of the +failure. For this purpose, the DISTUTILS_DEBUG environment variable can be set +to anything except an empty string, and Packaging will now print detailed +information about what it is doing, and prints the full traceback in case an +exception occurs. diff --git a/Doc/packaging/sourcedist.rst b/Doc/packaging/sourcedist.rst new file mode 100644 index 0000000..2cedc15 --- /dev/null +++ b/Doc/packaging/sourcedist.rst @@ -0,0 +1,266 @@ +.. _packaging-source-dist: + +****************************** +Creating a Source Distribution +****************************** + +As shown in section :ref:`packaging-simple-example`, you use the :command:`sdist` command +to create a source distribution. In the simplest case, :: + + python setup.py sdist + +(assuming you haven't specified any :command:`sdist` options in the setup script +or config file), :command:`sdist` creates the archive of the default format for +the current platform. The default format is a gzip'ed tar file +(:file:`.tar.gz`) on Unix, and ZIP file on Windows. + +You can specify as many formats as you like using the :option:`--formats` +option, for example:: + + python setup.py sdist --formats=gztar,zip + +to create a gzipped tarball and a zip file. The available formats are: + ++-----------+-------------------------+---------+ +| Format | Description | Notes | ++===========+=========================+=========+ +| ``zip`` | zip file (:file:`.zip`) | (1),(3) | ++-----------+-------------------------+---------+ +| ``gztar`` | gzip'ed tar file | \(2) | +| | (:file:`.tar.gz`) | | ++-----------+-------------------------+---------+ +| ``bztar`` | bzip2'ed tar file | | +| | (:file:`.tar.bz2`) | | ++-----------+-------------------------+---------+ +| ``tar`` | tar file (:file:`.tar`) | | ++-----------+-------------------------+---------+ + +Notes: + +(1) + default on Windows + +(2) + default on Unix + +(3) + requires either external :program:`zip` utility or :mod:`zipfile` module (part + of the standard Python library since Python 1.6) + +When using any ``tar`` format (``gztar``, ``bztar`` or +``tar``) under Unix, you can specify the ``owner`` and ``group`` names +that will be set for each member of the archive. + +For example, if you want all files of the archive to be owned by root:: + + python setup.py sdist --owner=root --group=root + + +.. _packaging-manifest: + +Specifying the files to distribute +================================== + +If you don't supply an explicit list of files (or instructions on how to +generate one), the :command:`sdist` command puts a minimal default set into the +source distribution: + +* all Python source files implied by the :option:`py_modules` and + :option:`packages` options + +* all C source files mentioned in the :option:`ext_modules` or + :option:`libraries` options + +* scripts identified by the :option:`scripts` option + See :ref:`packaging-installing-scripts`. + +* anything that looks like a test script: :file:`test/test\*.py` (currently, the + Packaging don't do anything with test scripts except include them in source + distributions, but in the future there will be a standard for testing Python + module distributions) + +* the configuration file :file:`setup.cfg` + +* all files that matches the ``package_data`` metadata. + See :ref:`packaging-installing-package-data`. + +* all files that matches the ``data_files`` metadata. + See :ref:`packaging-additional-files`. + +Contrary to Distutils, :file:`README` (or :file:`README.txt`) and +:file:`setup.py` are not included by default. + +Sometimes this is enough, but usually you will want to specify additional files +to distribute. The typical way to do this is to write a *manifest template*, +called :file:`MANIFEST.in` by default. The manifest template is just a list of +instructions for how to generate your manifest file, :file:`MANIFEST`, which is +the exact list of files to include in your source distribution. The +:command:`sdist` command processes this template and generates a manifest based +on its instructions and what it finds in the filesystem. + +If you prefer to roll your own manifest file, the format is simple: one filename +per line, regular files (or symlinks to them) only. If you do supply your own +:file:`MANIFEST`, you must specify everything: the default set of files +described above does not apply in this case. + +:file:`MANIFEST` files start with a comment indicating they are generated. +Files without this comment are not overwritten or removed. + +See :ref:`packaging-manifest-template` section for a syntax reference. + + +.. _packaging-manifest-options: + +Manifest-related options +======================== + +The normal course of operations for the :command:`sdist` command is as follows: + +* if the manifest file, :file:`MANIFEST` doesn't exist, read :file:`MANIFEST.in` + and create the manifest + +* if neither :file:`MANIFEST` nor :file:`MANIFEST.in` exist, create a manifest + with just the default file set + +* if either :file:`MANIFEST.in` or the setup script (:file:`setup.py`) are more + recent than :file:`MANIFEST`, recreate :file:`MANIFEST` by reading + :file:`MANIFEST.in` + +* use the list of files now in :file:`MANIFEST` (either just generated or read + in) to create the source distribution archive(s) + +There are a couple of options that modify this behaviour. First, use the +:option:`--no-defaults` and :option:`--no-prune` to disable the standard +"include" and "exclude" sets. + +Second, you might just want to (re)generate the manifest, but not create a +source distribution:: + + python setup.py sdist --manifest-only + +:option:`-o` is a shortcut for :option:`--manifest-only`. + + +.. _packaging-manifest-template: + +The MANIFEST.in template +======================== + +A :file:`MANIFEST.in` file can be added in a project to define the list of +files to include in the distribution built by the :command:`sdist` command. + +When :command:`sdist` is run, it will look for the :file:`MANIFEST.in` file +and interpret it to generate the :file:`MANIFEST` file that contains the +list of files that will be included in the package. + +This mechanism can be used when the default list of files is not enough. +(See :ref:`packaging-manifest`). + +Principle +--------- + +The manifest template has one command per line, where each command specifies a +set of files to include or exclude from the source distribution. For an +example, let's look at the Packaging' own manifest template:: + + include *.txt + recursive-include examples *.txt *.py + prune examples/sample?/build + +The meanings should be fairly clear: include all files in the distribution root +matching :file:`\*.txt`, all files anywhere under the :file:`examples` directory +matching :file:`\*.txt` or :file:`\*.py`, and exclude all directories matching +:file:`examples/sample?/build`. All of this is done *after* the standard +include set, so you can exclude files from the standard set with explicit +instructions in the manifest template. (Or, you can use the +:option:`--no-defaults` option to disable the standard set entirely.) + +The order of commands in the manifest template matters: initially, we have the +list of default files as described above, and each command in the template adds +to or removes from that list of files. Once we have fully processed the +manifest template, we remove files that should not be included in the source +distribution: + +* all files in the Packaging "build" tree (default :file:`build/`) + +* all files in directories named :file:`RCS`, :file:`CVS`, :file:`.svn`, + :file:`.hg`, :file:`.git`, :file:`.bzr` or :file:`_darcs` + +Now we have our complete list of files, which is written to the manifest for +future reference, and then used to build the source distribution archive(s). + +You can disable the default set of included files with the +:option:`--no-defaults` option, and you can disable the standard exclude set +with :option:`--no-prune`. + +Following the Packaging' own manifest template, let's trace how the +:command:`sdist` command builds the list of files to include in the Packaging +source distribution: + +#. include all Python source files in the :file:`packaging` and + :file:`packaging/command` subdirectories (because packages corresponding to + those two directories were mentioned in the :option:`packages` option in the + setup script---see section :ref:`packaging-setup-script`) + +#. include :file:`README.txt`, :file:`setup.py`, and :file:`setup.cfg` (standard + files) + +#. include :file:`test/test\*.py` (standard files) + +#. include :file:`\*.txt` in the distribution root (this will find + :file:`README.txt` a second time, but such redundancies are weeded out later) + +#. include anything matching :file:`\*.txt` or :file:`\*.py` in the sub-tree + under :file:`examples`, + +#. exclude all files in the sub-trees starting at directories matching + :file:`examples/sample?/build`\ ---this may exclude files included by the + previous two steps, so it's important that the ``prune`` command in the manifest + template comes after the ``recursive-include`` command + +#. exclude the entire :file:`build` tree, and any :file:`RCS`, :file:`CVS`, + :file:`.svn`, :file:`.hg`, :file:`.git`, :file:`.bzr` and :file:`_darcs` + directories + +Just like in the setup script, file and directory names in the manifest template +should always be slash-separated; the Packaging will take care of converting +them to the standard representation on your platform. That way, the manifest +template is portable across operating systems. + +Commands +-------- + +The manifest template commands are: + ++-------------------------------------------+-----------------------------------------------+ +| Command | Description | ++===========================================+===============================================+ +| :command:`include pat1 pat2 ...` | include all files matching any of the listed | +| | patterns | ++-------------------------------------------+-----------------------------------------------+ +| :command:`exclude pat1 pat2 ...` | exclude all files matching any of the listed | +| | patterns | ++-------------------------------------------+-----------------------------------------------+ +| :command:`recursive-include dir pat1 pat2 | include all files under *dir* matching any of | +| ...` | the listed patterns | ++-------------------------------------------+-----------------------------------------------+ +| :command:`recursive-exclude dir pat1 pat2 | exclude all files under *dir* matching any of | +| ...` | the listed patterns | ++-------------------------------------------+-----------------------------------------------+ +| :command:`global-include pat1 pat2 ...` | include all files anywhere in the source tree | +| | matching --- & any of the listed patterns | ++-------------------------------------------+-----------------------------------------------+ +| :command:`global-exclude pat1 pat2 ...` | exclude all files anywhere in the source tree | +| | matching --- & any of the listed patterns | ++-------------------------------------------+-----------------------------------------------+ +| :command:`prune dir` | exclude all files under *dir* | ++-------------------------------------------+-----------------------------------------------+ +| :command:`graft dir` | include all files under *dir* | ++-------------------------------------------+-----------------------------------------------+ + +The patterns here are Unix-style "glob" patterns: ``*`` matches any sequence of +regular filename characters, ``?`` matches any single regular filename +character, and ``[range]`` matches any of the characters in *range* (e.g., +``a-z``, ``a-zA-Z``, ``a-f0-9_.``). The definition of "regular filename +character" is platform-specific: on Unix it is anything except slash; on Windows +anything except backslash or colon. diff --git a/Doc/packaging/tutorial.rst b/Doc/packaging/tutorial.rst new file mode 100644 index 0000000..04f41e5 --- /dev/null +++ b/Doc/packaging/tutorial.rst @@ -0,0 +1,112 @@ +================== +Packaging tutorial +================== + +Welcome to the Packaging tutorial! We will learn how to use Packaging +to package your project. + +.. TODO merge with introduction.rst + + +Getting started +--------------- + +Packaging works with the *setup.cfg* file. It contains all the metadata for +your project, as defined in PEP 345, but also declare what your project +contains. + +Let's say you have a project called *CLVault* containing one package called +*clvault*, and a few scripts inside. You can use the *pysetup* script to create +a *setup.cfg* file for the project. The script will ask you a few questions:: + + $ mkdir CLVault + $ cd CLVault + $ pysetup create + Project name [CLVault]: + Current version number: 0.1 + Package description: + >Command-line utility to store and retrieve passwords + Author name: Tarek Ziade + Author e-mail address: tarek@ziade.org + Project Home Page: http://bitbucket.org/tarek/clvault + Do you want to add a package ? (y/n): y + Package name: clvault + Do you want to add a package ? (y/n): n + Do you want to set Trove classifiers? (y/n): y + Please select the project status: + + 1 - Planning + 2 - Pre-Alpha + 3 - Alpha + 4 - Beta + 5 - Production/Stable + 6 - Mature + 7 - Inactive + + Status: 3 + What license do you use: GPL + Matching licenses: + + 1) License :: OSI Approved :: GNU General Public License (GPL) + 2) License :: OSI Approved :: GNU Library or Lesser General Public License (LGPL) + + Type the number of the license you wish to use or ? to try again:: 1 + Do you want to set other trove identifiers (y/n) [n]: n + Wrote "setup.cfg". + + +A setup.cfg file is created, containing the metadata of your project and the +list of the packages it contains:: + + $ cat setup.cfg + [metadata] + name = CLVault + version = 0.1 + author = Tarek Ziade + author_email = tarek@ziade.org + description = Command-line utility to store and retrieve passwords + home_page = http://bitbucket.org/tarek/clvault + + classifier = Development Status :: 3 - Alpha + License :: OSI Approved :: GNU General Public License (GPL) + + [files] + packages = clvault + + +Our project will depend on the *keyring* project. Let's add it in the +[metadata] section:: + + [metadata] + ... + requires_dist = + keyring + + +Running commands +---------------- + +You can run useful commands on your project once the setup.cfg file is ready: + +- sdist: creates a source distribution +- register: register your project to PyPI +- upload: upload the distribution to PyPI +- install_dist: install it + +All commands are run using the run script:: + + $ pysetup run install_dist + $ pysetup run sdist + $ pysetup run upload + +If you want to push a source distribution of your project to PyPI, do:: + + $ pysetup run sdist register upload + + +Installing the project +---------------------- + +The project can be installed by manually running the packaging install command:: + + $ pysetup run install_dist diff --git a/Doc/packaging/uploading.rst b/Doc/packaging/uploading.rst new file mode 100644 index 0000000..297518b --- /dev/null +++ b/Doc/packaging/uploading.rst @@ -0,0 +1,80 @@ +.. _packaging-package-upload: + +*************************************** +Uploading Packages to the Package Index +*************************************** + +The Python Package Index (PyPI) not only stores the package info, but also the +package data if the author of the package wishes to. The packaging command +:command:`upload` pushes the distribution files to PyPI. + +The command is invoked immediately after building one or more distribution +files. For example, the command :: + + python setup.py sdist bdist_wininst upload + +will cause the source distribution and the Windows installer to be uploaded to +PyPI. Note that these will be uploaded even if they are built using an earlier +invocation of :file:`setup.py`, but that only distributions named on the command +line for the invocation including the :command:`upload` command are uploaded. + +The :command:`upload` command uses the username, password, and repository URL +from the :file:`$HOME/.pypirc` file (see section :ref:`packaging-pypirc` for more on this +file). If a :command:`register` command was previously called in the same +command, and if the password was entered in the prompt, :command:`upload` will +reuse the entered password. This is useful if you do not want to store a clear +text password in the :file:`$HOME/.pypirc` file. + +You can specify another PyPI server with the :option:`--repository=*url*` +option:: + + python setup.py sdist bdist_wininst upload -r http://example.com/pypi + +See section :ref:`packaging-pypirc` for more on defining several servers. + +You can use the :option:`--sign` option to tell :command:`upload` to sign each +uploaded file using GPG (GNU Privacy Guard). The :program:`gpg` program must +be available for execution on the system :envvar:`PATH`. You can also specify +which key to use for signing using the :option:`--identity=*name*` option. + +Other :command:`upload` options include :option:`--repository=<url>` or +:option:`--repository=<section>` where *url* is the url of the server and +*section* the name of the section in :file:`$HOME/.pypirc`, and +:option:`--show-response` (which displays the full response text from the PyPI +server for help in debugging upload problems). + +PyPI package display +==================== + +The ``description`` field plays a special role at PyPI. It is used by +the server to display a home page for the registered package. + +If you use the `reStructuredText <http://docutils.sourceforge.net/rst.html>`_ +syntax for this field, PyPI will parse it and display an HTML output for +the package home page. + +The ``description`` field can be filled from a text file located in the +project:: + + from packaging.core import setup + + fp = open('README.txt') + try: + description = fp.read() + finally: + fp.close() + + setup(name='Packaging', + description=description) + +In that case, :file:`README.txt` is a regular reStructuredText text file located +in the root of the package besides :file:`setup.py`. + +To prevent registering broken reStructuredText content, you can use the +:program:`rst2html` program that is provided by the :mod:`docutils` package +and check the ``description`` from the command line:: + + $ python setup.py --description | rst2html.py > output.html + +:mod:`docutils` will display a warning if there's something wrong with your +syntax. |