diff options
35 files changed, 144 insertions, 294 deletions
diff --git a/Doc/install/install.rst b/Doc/install/install.rst index 33f3e9c..b88d3a6 100644 --- a/Doc/install/install.rst +++ b/Doc/install/install.rst @@ -842,6 +842,8 @@ Additionally, there's a ``global`` section for options that affect every command Sections consist of one or more lines containing a single option specified as ``option = value``. +.. XXX use dry-run in the next example or use a pysetup option as example + For example, here's a complete configuration file that forces all commands to run quietly by default:: diff --git a/Doc/install/pysetup.rst b/Doc/install/pysetup.rst index f6f1f83..d472c24 100644 --- a/Doc/install/pysetup.rst +++ b/Doc/install/pysetup.rst @@ -149,7 +149,7 @@ Getting a list of all pysetup actions and global options:: list: List installed projects graph: Display a graph create: Create a project - generate-setup: Generate a backward-comptatible setup.py + generate-setup: Generate a backward-compatible setup.py To get more help on an action, use: diff --git a/Doc/library/packaging.compiler.rst b/Doc/library/packaging.compiler.rst index 10780d1..f23c551 100644 --- a/Doc/library/packaging.compiler.rst +++ b/Doc/library/packaging.compiler.rst @@ -15,7 +15,7 @@ function. Compiler types provided by Packaging are listed in Public functions ---------------- -.. function:: new_compiler(plat=None, compiler=None, verbose=0, dry_run=0, force=0) +.. function:: new_compiler(plat=None, compiler=None, dry_run=False, force=False) Factory function to generate an instance of some :class:`~.ccompiler.CCompiler` subclass for the requested platform or @@ -165,7 +165,7 @@ link steps needed to build a single project. Methods are provided to set options for the compiler --- macro definitions, include directories, link path, libraries and the like. -.. class:: CCompiler([verbose=0, dry_run=0, force=0]) +.. class:: CCompiler(dry_run=False, force=False) The abstract base class :class:`CCompiler` defines the interface that must be implemented by real compiler classes. The class also has some utility @@ -180,11 +180,11 @@ libraries and the like. per-compilation or per-link basis. The constructor for each subclass creates an instance of the Compiler object. - Flags are *verbose* (show verbose output), *dry_run* (don't actually execute + Flags are *dry_run* (don't actually execute the steps) and *force* (rebuild everything, regardless of dependencies). All - of these flags default to ``0`` (off). Note that you probably don't want to + of these flags default to ``False`` (off). Note that you probably don't want to instantiate :class:`CCompiler` or one of its subclasses directly - use the - :func:`packaging.CCompiler.new_compiler` factory function instead. + :func:`new_compiler` factory function instead. The following methods allow you to manually alter compiler options for the instance of the Compiler class. diff --git a/Doc/library/packaging.util.rst b/Doc/library/packaging.util.rst index 2b3103c..e628c32 100644 --- a/Doc/library/packaging.util.rst +++ b/Doc/library/packaging.util.rst @@ -90,7 +90,7 @@ This module contains various helpers for the other modules. Search the path for a given executable name. -.. function:: execute(func, args[, msg=None, verbose=0, dry_run=0]) +.. function:: execute(func, args, msg=None, dry_run=False) Perform some action that affects the outside world (for instance, writing to the filesystem). Such actions are special because they are disabled by the @@ -117,7 +117,8 @@ This module contains various helpers for the other modules. :exc:`ValueError` if *val* is anything else. -.. function:: byte_compile(py_files[, optimize=0, force=0, prefix=None, base_dir=None, verbose=1, dry_run=0, direct=None]) +.. function:: byte_compile(py_files, optimize=0, force=0, prefix=None, \ + base_dir=None, dry_run=0, direct=None) Byte-compile a collection of Python source files to either :file:`.pyc` or :file:`.pyo` files in a :file:`__pycache__` subdirectory (see :pep:`3147`), @@ -131,6 +132,9 @@ This module contains various helpers for the other modules. * ``1`` - normal optimization (like ``python -O``) * ``2`` - extra optimization (like ``python -OO``) + This function is independent from the running Python's :option:`-O` or + :option:`-B` options; it is fully controlled by the parameters passed in. + If *force* is true, all files are recompiled regardless of timestamps. The source filename encoded in each :term:`bytecode` file defaults to the filenames @@ -149,6 +153,3 @@ This module contains various helpers for the other modules. figure out to use direct compilation or not (see the source for details). The *direct* flag is used by the script generated in indirect mode; unless you know what you're doing, leave it set to ``None``. - - This function is independent from the running Python's :option:`-O` or - :option:`-B` options; it is fully controlled by the parameters passed in. diff --git a/Doc/packaging/setupcfg.rst b/Doc/packaging/setupcfg.rst index 9b09a4c..a381017 100644 --- a/Doc/packaging/setupcfg.rst +++ b/Doc/packaging/setupcfg.rst @@ -779,7 +779,7 @@ needs to have its options defined in a dedicated section. Here's an example:: extra_compile_args = -fPIC -O2 -DGECODE_VERSION=$(./gecode_version) -- sys.platform != 'win32' - /DGECODE_VERSION='win32' -- sys.platform == 'win32' + /DGECODE_VERSION=win32 -- sys.platform == 'win32' The section name must start with ``extension:``; the right-hand part is used as the full name (including a parent package, if any) of the extension. Whitespace diff --git a/Doc/whatsnew/3.3.rst b/Doc/whatsnew/3.3.rst index 7f4517f..7e877fc 100644 --- a/Doc/whatsnew/3.3.rst +++ b/Doc/whatsnew/3.3.rst @@ -596,7 +596,7 @@ Other issues paths. In previous versions, it did. See changeset for doc changes in various files. Contributed by Carl Meyer with editions by Éric Araujo. -.. Issue #10998: -Q command-line flags are related artifacts have been +.. Issue #10998: the -Q command-line flag and related artifacts have been removed. Code checking sys.flags.division_warning will need updating. Contributed by Éric Araujo. diff --git a/Lib/packaging/command/build_ext.py b/Lib/packaging/command/build_ext.py index 4051a2d..4f375b6 100644 --- a/Lib/packaging/command/build_ext.py +++ b/Lib/packaging/command/build_ext.py @@ -4,7 +4,6 @@ import os import re import sys import site -import logging import sysconfig from packaging.util import get_platform @@ -288,14 +287,9 @@ class build_ext(Command): self.libraries.extend(build_clib.get_library_names() or []) self.library_dirs.append(build_clib.build_clib) - # Temporary kludge until we remove the verbose arguments and use - # logging everywhere - verbose = logger.getEffectiveLevel() >= logging.DEBUG - # Setup the CCompiler object that we'll use to do all the # compiling and linking self.compiler_obj = new_compiler(compiler=self.compiler, - verbose=verbose, dry_run=self.dry_run, force=self.force) diff --git a/Lib/packaging/command/build_py.py b/Lib/packaging/command/build_py.py index 5faae39..0062140 100644 --- a/Lib/packaging/command/build_py.py +++ b/Lib/packaging/command/build_py.py @@ -18,7 +18,7 @@ class build_py(Command, Mixin2to3): description = "build pure Python modules (copy to build directory)" - # The options for controlling byte compilations are two independent sets; + # The options for controlling byte compilation are two independent sets; # more info in install_lib or the reST docs user_options = [ @@ -113,7 +113,8 @@ class build_py(Command, Mixin2to3): self.run_2to3(self._updated_files, self._doctests_2to3, self.use_2to3_fixers) - self.byte_compile(self.get_outputs(include_bytecode=False)) + self.byte_compile(self.get_outputs(include_bytecode=False), + prefix=self.build_lib) # -- Top-level worker functions ------------------------------------ @@ -335,11 +336,9 @@ class build_py(Command, Mixin2to3): outputs.append(filename) if include_bytecode: if self.compile: - outputs.append(imp.cache_from_source(filename, - debug_override=True)) - if self.optimize > 0: - outputs.append(imp.cache_from_source(filename, - debug_override=False)) + outputs.append(imp.cache_from_source(filename, True)) + if self.optimize: + outputs.append(imp.cache_from_source(filename, False)) outputs += [ os.path.join(build_dir, filename) @@ -391,19 +390,3 @@ class build_py(Command, Mixin2to3): for package_, module, module_file in modules: assert package == package_ self.build_module(module, module_file, package) - - def byte_compile(self, files): - from packaging.util import byte_compile # FIXME use compileall - prefix = self.build_lib - if prefix[-1] != os.sep: - prefix = prefix + os.sep - - # XXX this code is essentially the same as the 'byte_compile() - # method of the "install_lib" command, except for the determination - # of the 'prefix' string. Hmmm. - if self.compile: - byte_compile(files, optimize=0, - force=self.force, prefix=prefix, dry_run=self.dry_run) - if self.optimize > 0: - byte_compile(files, optimize=self.optimize, - force=self.force, prefix=prefix, dry_run=self.dry_run) diff --git a/Lib/packaging/command/cmd.py b/Lib/packaging/command/cmd.py index a88df02..25e6a72 100644 --- a/Lib/packaging/command/cmd.py +++ b/Lib/packaging/command/cmd.py @@ -10,7 +10,7 @@ from packaging.errors import PackagingOptionError class Command: """Abstract base class for defining command classes, the "worker bees" - of the Packaging. A useful analogy for command classes is to think of + of Packaging. A useful analogy for command classes is to think of them as subroutines with local variables called "options". The options are "declared" in 'initialize_options()' and "defined" (given their final values, aka "finalized") in 'finalize_options()', both of which @@ -351,7 +351,7 @@ class Command: def execute(self, func, args, msg=None, level=1): util.execute(func, args, msg, dry_run=self.dry_run) - def mkpath(self, name, mode=0o777, dry_run=None, verbose=0): + def mkpath(self, name, mode=0o777, dry_run=None): if dry_run is None: dry_run = self.dry_run name = os.path.normpath(name) @@ -367,9 +367,11 @@ class Command: def copy_file(self, infile, outfile, preserve_mode=True, preserve_times=True, link=None, level=1): - """Copy a file respecting verbose, dry-run and force flags. (The - former two default to whatever is in the Distribution object, and - the latter defaults to false for commands that don't define it.)""" + """Copy a file respecting dry-run and force flags. + + (dry-run defaults to whatever is in the Distribution object, and + force to false for commands that don't define it.) + """ if self.dry_run: # XXX add a comment return @@ -380,12 +382,13 @@ class Command: def copy_tree(self, infile, outfile, preserve_mode=True, preserve_times=True, preserve_symlinks=False, level=1): - """Copy an entire directory tree respecting verbose, dry-run, + """Copy an entire directory tree respecting dry-run and force flags. """ if self.dry_run: - return # see if we want to display something - + # XXX should not return but let copy_tree log and decide to execute + # or not based on its dry_run argument + return return util.copy_tree(infile, outfile, preserve_mode, preserve_times, preserve_symlinks, not self.force, dry_run=self.dry_run) @@ -393,7 +396,7 @@ class Command: def move_file(self, src, dst, level=1): """Move a file respecting the dry-run flag.""" if self.dry_run: - return # XXX log ? + return # XXX same thing return move(src, dst) def spawn(self, cmd, search_path=True, level=1): @@ -439,3 +442,20 @@ class Command: # Otherwise, print the "skip" message else: logger.debug(skip_msg) + + def byte_compile(self, files, prefix=None): + """Byte-compile files to pyc and/or pyo files. + + This method requires that the calling class define compile and + optimize options, like build_py and install_lib. It also + automatically respects the force and dry-run options. + + prefix, if given, is a string that will be stripped off the + filenames encoded in bytecode files. + """ + if self.compile: + util.byte_compile(files, optimize=False, prefix=prefix, + force=self.force, dry_run=self.dry_run) + if self.optimize: + util.byte_compile(files, optimize=self.optimize, prefix=prefix, + force=self.force, dry_run=self.dry_run) diff --git a/Lib/packaging/command/install_lib.py b/Lib/packaging/command/install_lib.py index f6c785f..ffc5d45 100644 --- a/Lib/packaging/command/install_lib.py +++ b/Lib/packaging/command/install_lib.py @@ -2,7 +2,6 @@ import os import imp -import logging from packaging import logger from packaging.command.cmd import Command @@ -21,7 +20,7 @@ class install_lib(Command): description = "install all modules (extensions and pure Python)" - # The options for controlling byte compilations are two independent sets: + # The options for controlling byte compilation are two independent sets: # 'compile' is strictly boolean, and only decides whether to # generate .pyc files. 'optimize' is three-way (0, 1, or 2), and # decides both whether to generate .pyo files and what level of @@ -84,9 +83,14 @@ class install_lib(Command): # having a build directory!) outfiles = self.install() - # (Optionally) compile .py to .pyc + # (Optionally) compile .py to .pyc and/or .pyo if outfiles is not None and self.distribution.has_pure_modules(): - self.byte_compile(outfiles) + # XXX comment from distutils: "This [prefix stripping] is far from + # complete, but it should at least generate usable bytecode in RPM + # distributions." -> need to find exact requirements for + # byte-compiled files and fix it + install_root = self.get_finalized_command('install_dist').root + self.byte_compile(outfiles, prefix=install_root) # -- Top-level worker functions ------------------------------------ # (called from 'run()') @@ -108,28 +112,6 @@ class install_lib(Command): return return outfiles - def byte_compile(self, files): - from packaging.util import byte_compile # FIXME use compileall - - # Get the "--root" directory supplied to the "install_dist" command, - # and use it as a prefix to strip off the purported filename - # encoded in bytecode files. This is far from complete, but it - # should at least generate usable bytecode in RPM distributions. - install_root = self.get_finalized_command('install_dist').root - - # Temporary kludge until we remove the verbose arguments and use - # logging everywhere - verbose = logger.getEffectiveLevel() >= logging.DEBUG - - if self.compile: - byte_compile(files, optimize=0, - force=self.force, prefix=install_root, - verbose=verbose, dry_run=self.dry_run) - if self.optimize > 0: - byte_compile(files, optimize=self.optimize, - force=self.force, prefix=install_root, - verbose=verbose, dry_run=self.dry_run) - # -- Utility methods ----------------------------------------------- def _mutate_outputs(self, has_any, build_cmd, cmd_option, output_dir): @@ -157,11 +139,9 @@ class install_lib(Command): if ext != PYTHON_SOURCE_EXTENSION: continue if self.compile: - bytecode_files.append(imp.cache_from_source( - py_file, debug_override=True)) - if self.optimize > 0: - bytecode_files.append(imp.cache_from_source( - py_file, debug_override=False)) + bytecode_files.append(imp.cache_from_source(py_file, True)) + if self.optimize: + bytecode_files.append(imp.cache_from_source(py_file, False)) return bytecode_files diff --git a/Lib/packaging/command/sdist.py b/Lib/packaging/command/sdist.py index 09b26db..d399981 100644 --- a/Lib/packaging/command/sdist.py +++ b/Lib/packaging/command/sdist.py @@ -337,12 +337,11 @@ class sdist(Command): """ return self.archive_files - def create_tree(self, base_dir, files, mode=0o777, verbose=1, - dry_run=False): + def create_tree(self, base_dir, files, mode=0o777, dry_run=False): need_dir = set() for file in files: need_dir.add(os.path.join(base_dir, os.path.dirname(file))) # Now create them for dir in sorted(need_dir): - self.mkpath(dir, mode, verbose=verbose, dry_run=dry_run) + self.mkpath(dir, mode, dry_run=dry_run) diff --git a/Lib/packaging/command/test.py b/Lib/packaging/command/test.py index 5b62a12..4d5348f 100644 --- a/Lib/packaging/command/test.py +++ b/Lib/packaging/command/test.py @@ -60,8 +60,7 @@ class test(Command): self.run_command('build') sys.path.insert(0, build.build_lib) - # Temporary kludge until we remove the verbose arguments and use - # logging everywhere + # XXX maybe we could pass the verbose argument of pysetup here logger = logging.getLogger('packaging') verbose = logger.getEffectiveLevel() >= logging.DEBUG verbosity = verbose + 1 diff --git a/Lib/packaging/compiler/__init__.py b/Lib/packaging/compiler/__init__.py index e267e9f..d8e02ce 100644 --- a/Lib/packaging/compiler/__init__.py +++ b/Lib/packaging/compiler/__init__.py @@ -153,8 +153,7 @@ def show_compilers(): pretty_printer.print_help("List of available compilers:") -def new_compiler(plat=None, compiler=None, verbose=0, dry_run=False, - force=False): +def new_compiler(plat=None, compiler=None, dry_run=False, force=False): """Generate an instance of some CCompiler subclass for the supplied platform/compiler combination. 'plat' defaults to 'os.name' (eg. 'posix', 'nt'), and 'compiler' defaults to the default compiler @@ -183,11 +182,7 @@ def new_compiler(plat=None, compiler=None, verbose=0, dry_run=False, cls = resolve_name(cls) _COMPILERS[compiler] = cls - - # XXX The None is necessary to preserve backwards compatibility - # with classes that expect verbose to be the first positional - # argument. - return cls(None, dry_run, force) + return cls(dry_run, force) def gen_preprocess_options(macros, include_dirs): diff --git a/Lib/packaging/compiler/bcppcompiler.py b/Lib/packaging/compiler/bcppcompiler.py index a4aa2fa..06c758c 100644 --- a/Lib/packaging/compiler/bcppcompiler.py +++ b/Lib/packaging/compiler/bcppcompiler.py @@ -47,8 +47,8 @@ class BCPPCompiler(CCompiler) : exe_extension = '.exe' - def __init__(self, verbose=0, dry_run=False, force=False): - super(BCPPCompiler, self).__init__(verbose, dry_run, force) + def __init__(self, dry_run=False, force=False): + super(BCPPCompiler, self).__init__(dry_run, force) # These executables are assumed to all be in the path. # Borland doesn't seem to use any special registry settings to diff --git a/Lib/packaging/compiler/ccompiler.py b/Lib/packaging/compiler/ccompiler.py index 02e22d2..98c4b68 100644 --- a/Lib/packaging/compiler/ccompiler.py +++ b/Lib/packaging/compiler/ccompiler.py @@ -79,10 +79,9 @@ class CCompiler: } language_order = ["c++", "objc", "c"] - def __init__(self, verbose=0, dry_run=False, force=False): + def __init__(self, dry_run=False, force=False): self.dry_run = dry_run self.force = force - self.verbose = verbose # 'output_dir': a common output directory for object, library, # shared object, and shared library files diff --git a/Lib/packaging/compiler/cygwinccompiler.py b/Lib/packaging/compiler/cygwinccompiler.py index 916a1cc..3eec067 100644 --- a/Lib/packaging/compiler/cygwinccompiler.py +++ b/Lib/packaging/compiler/cygwinccompiler.py @@ -92,8 +92,8 @@ class CygwinCCompiler(UnixCCompiler): shared_lib_format = "%s%s" exe_extension = ".exe" - def __init__(self, verbose=0, dry_run=False, force=False): - super(CygwinCCompiler, self).__init__(verbose, dry_run, force) + def __init__(self, dry_run=False, force=False): + super(CygwinCCompiler, self).__init__(dry_run, force) status, details = check_config_h() logger.debug("Python's GCC status: %s (details: %s)", status, details) @@ -233,12 +233,11 @@ class CygwinCCompiler(UnixCCompiler): if not debug: extra_preargs.append("-s") - UnixCCompiler.link(self, target_desc, objects, output_filename, - output_dir, libraries, library_dirs, - runtime_library_dirs, - None, # export_symbols, we do this in our def-file - debug, extra_preargs, extra_postargs, build_temp, - target_lang) + super(CygwinCCompiler, self).link( + target_desc, objects, output_filename, output_dir, libraries, + library_dirs, runtime_library_dirs, + None, # export_symbols, we do this in our def-file + debug, extra_preargs, extra_postargs, build_temp, target_lang) # -- Miscellaneous methods ----------------------------------------- @@ -271,8 +270,8 @@ class Mingw32CCompiler(CygwinCCompiler): name = 'mingw32' description = 'MinGW32 compiler' - def __init__(self, verbose=0, dry_run=False, force=False): - super(Mingw32CCompiler, self).__init__(verbose, dry_run, force) + def __init__(self, dry_run=False, force=False): + super(Mingw32CCompiler, self).__init__(dry_run, force) # ld_version >= "2.13" support -shared so use it instead of # -mdll -static diff --git a/Lib/packaging/compiler/msvc9compiler.py b/Lib/packaging/compiler/msvc9compiler.py index 539df73..029aa77 100644 --- a/Lib/packaging/compiler/msvc9compiler.py +++ b/Lib/packaging/compiler/msvc9compiler.py @@ -309,8 +309,8 @@ class MSVCCompiler(CCompiler) : static_lib_format = shared_lib_format = '%s%s' exe_extension = '.exe' - def __init__(self, verbose=0, dry_run=False, force=False): - super(MSVCCompiler, self).__init__(verbose, dry_run, force) + def __init__(self, dry_run=False, force=False): + super(MSVCCompiler, self).__init__(dry_run, force) self.__version = VERSION self.__root = r"Software\Microsoft\VisualStudio" # self.__macros = MACROS diff --git a/Lib/packaging/compiler/msvccompiler.py b/Lib/packaging/compiler/msvccompiler.py index 740bc30..39a10b2 100644 --- a/Lib/packaging/compiler/msvccompiler.py +++ b/Lib/packaging/compiler/msvccompiler.py @@ -236,8 +236,8 @@ class MSVCCompiler(CCompiler): static_lib_format = shared_lib_format = '%s%s' exe_extension = '.exe' - def __init__(self, verbose=0, dry_run=False, force=False): - super(MSVCCompiler, self).__init__(verbose, dry_run, force) + def __init__(self, dry_run=False, force=False): + super(MSVCCompiler, self).__init__(dry_run, force) self.__version = get_build_version() self.__arch = get_build_architecture() if self.__arch == "Intel": diff --git a/Lib/packaging/metadata.py b/Lib/packaging/metadata.py index 2d0ffa4..2993ebb 100644 --- a/Lib/packaging/metadata.py +++ b/Lib/packaging/metadata.py @@ -229,8 +229,10 @@ class Metadata: def __delitem__(self, name): field_name = self._convert_name(name) - # we let a KeyError propagate - del self._fields[field_name] + try: + del self._fields[field_name] + except KeyError: + raise KeyError(name) self._set_best_version() def __contains__(self, name): diff --git a/Lib/packaging/run.py b/Lib/packaging/run.py index 671a4bf..4756f7c 100644 --- a/Lib/packaging/run.py +++ b/Lib/packaging/run.py @@ -368,7 +368,7 @@ actions = [ ('list', 'List installed projects', _list), ('graph', 'Display a graph', _graph), ('create', 'Create a project', _create), - ('generate-setup', 'Generate a backward-comptatible setup.py', _generate), + ('generate-setup', 'Generate a backward-compatible setup.py', _generate), ] diff --git a/Lib/packaging/tests/__init__.py b/Lib/packaging/tests/__init__.py index 56e487a..cb82004 100644 --- a/Lib/packaging/tests/__init__.py +++ b/Lib/packaging/tests/__init__.py @@ -14,16 +14,11 @@ standard library for packaging tests and unittest2 for distutils2 tests. import os import sys import unittest -from io import StringIO -# XXX move helpers to support, add tests for them, remove things that -# duplicate test.support (or keep them for the backport; needs thinking) - -here = os.path.dirname(__file__) or os.curdir -verbose = 1 def test_suite(): suite = unittest.TestSuite() + here = os.path.dirname(__file__) or os.curdir for fn in os.listdir(here): if fn.startswith("test") and fn.endswith(".py"): modname = "packaging.tests." + fn[:-3] @@ -31,100 +26,3 @@ def test_suite(): module = sys.modules[modname] suite.addTest(module.test_suite()) return suite - - -class Error(Exception): - """Base class for regression test exceptions.""" - - -class TestFailed(Error): - """Test failed.""" - - -class BasicTestRunner: - def run(self, test): - result = unittest.TestResult() - test(result) - return result - - -def _run_suite(suite, verbose_=1): - """Run tests from a unittest.TestSuite-derived class.""" - global verbose - verbose = verbose_ - if verbose_: - runner = unittest.TextTestRunner(sys.stdout, verbosity=2) - else: - runner = BasicTestRunner() - - result = runner.run(suite) - if not result.wasSuccessful(): - if len(result.errors) == 1 and not result.failures: - err = result.errors[0][1] - elif len(result.failures) == 1 and not result.errors: - err = result.failures[0][1] - else: - err = "errors occurred; run in verbose mode for details" - raise TestFailed(err) - - -def run_unittest(classes, verbose_=1): - """Run tests from unittest.TestCase-derived classes. - - Originally extracted from stdlib test.test_support and modified to - support unittest2. - """ - valid_types = (unittest.TestSuite, unittest.TestCase) - suite = unittest.TestSuite() - for cls in classes: - if isinstance(cls, str): - if cls in sys.modules: - suite.addTest(unittest.findTestCases(sys.modules[cls])) - else: - raise ValueError("str arguments must be keys in sys.modules") - elif isinstance(cls, valid_types): - suite.addTest(cls) - else: - suite.addTest(unittest.makeSuite(cls)) - _run_suite(suite, verbose_) - - -def reap_children(): - """Use this function at the end of test_main() whenever sub-processes - are started. This will help ensure that no extra children (zombies) - stick around to hog resources and create problems when looking - for refleaks. - - Extracted from stdlib test.support. - """ - - # Reap all our dead child processes so we don't leave zombies around. - # These hog resources and might be causing some of the buildbots to die. - if hasattr(os, 'waitpid'): - any_process = -1 - while True: - try: - # This will raise an exception on Windows. That's ok. - pid, status = os.waitpid(any_process, os.WNOHANG) - if pid == 0: - break - except: - break - - -def captured_stdout(func, *args, **kw): - orig_stdout = getattr(sys, 'stdout') - setattr(sys, 'stdout', StringIO()) - try: - res = func(*args, **kw) - sys.stdout.seek(0) - return res, sys.stdout.read() - finally: - setattr(sys, 'stdout', orig_stdout) - - -def unload(name): - try: - del sys.modules[name] - except KeyError: - pass diff --git a/Lib/packaging/tests/pypi_server.py b/Lib/packaging/tests/pypi_server.py index 377a2b9..13c30cf 100644 --- a/Lib/packaging/tests/pypi_server.py +++ b/Lib/packaging/tests/pypi_server.py @@ -33,7 +33,6 @@ import os import queue import select import threading -import socketserver from functools import wraps from http.server import HTTPServer, SimpleHTTPRequestHandler from xmlrpc.server import SimpleXMLRPCServer @@ -270,7 +269,7 @@ class PyPIRequestHandler(SimpleHTTPRequestHandler): class PyPIXMLRPCServer(SimpleXMLRPCServer): def server_bind(self): """Override server_bind to store the server name.""" - socketserver.TCPServer.server_bind(self) + super(PyPIXMLRPCServer, self).server_bind() host, port = self.socket.getsockname()[:2] self.server_port = port @@ -371,12 +370,13 @@ class MockDist: 'requires_python': self.requires_python, 'classifiers': [], 'name': self.name, - 'licence': self.licence, + 'licence': self.licence, # XXX licence or license? 'summary': self.summary, 'home_page': self.homepage, 'stable_version': self.stable_version, - 'provides_dist': self.provides_dist or "%s (%s)" % (self.name, - self.version), + # FIXME doesn't that reproduce the bug from 6527d3106e9f? + 'provides_dist': (self.provides_dist or + "%s (%s)" % (self.name, self.version)), 'requires': self.requires, 'cheesecake_installability_id': self.cheesecake_installability_id, } diff --git a/Lib/packaging/tests/test_command_bdist.py b/Lib/packaging/tests/test_command_bdist.py index dd10188..7b2ea01 100644 --- a/Lib/packaging/tests/test_command_bdist.py +++ b/Lib/packaging/tests/test_command_bdist.py @@ -1,7 +1,8 @@ """Tests for distutils.command.bdist.""" import os +from test.support import captured_stdout from packaging.command.bdist import bdist, show_formats -from packaging.tests import unittest, support, captured_stdout +from packaging.tests import unittest, support class BuildTestCase(support.TempdirManager, @@ -42,7 +43,9 @@ class BuildTestCase(support.TempdirManager, '%s should take --skip-build from bdist' % name) def test_show_formats(self): - __, stdout = captured_stdout(show_formats) + with captured_stdout() as stdout: + show_formats() + stdout = stdout.getvalue() # the output should be a header line + one line per format num_formats = len(bdist.format_commands) diff --git a/Lib/packaging/tests/test_command_install_lib.py b/Lib/packaging/tests/test_command_install_lib.py index e58e812..79e8fa8 100644 --- a/Lib/packaging/tests/test_command_install_lib.py +++ b/Lib/packaging/tests/test_command_install_lib.py @@ -44,8 +44,8 @@ class InstallLibTestCase(support.TempdirManager, f = os.path.join(project_dir, 'foo.py') self.write_file(f, '# python file') cmd.byte_compile([f]) - pyc_file = imp.cache_from_source('foo.py', debug_override=True) - pyo_file = imp.cache_from_source('foo.py', debug_override=False) + pyc_file = imp.cache_from_source('foo.py', True) + pyo_file = imp.cache_from_source('foo.py', False) self.assertTrue(os.path.exists(pyc_file)) self.assertTrue(os.path.exists(pyo_file)) diff --git a/Lib/packaging/tests/test_command_sdist.py b/Lib/packaging/tests/test_command_sdist.py index 0b9f5be..47c455e 100644 --- a/Lib/packaging/tests/test_command_sdist.py +++ b/Lib/packaging/tests/test_command_sdist.py @@ -17,8 +17,8 @@ from packaging.util import find_executable from packaging.errors import PackagingOptionError from packaging.command.sdist import sdist, show_formats +from test.support import captured_stdout from packaging.tests import support, unittest -from packaging.tests import captured_stdout from packaging.tests.support import requires_zlib @@ -234,7 +234,9 @@ class SDistTestCase(support.TempdirManager, self.assertIn("'setup.cfg' file not found", warnings[1]) def test_show_formats(self): - __, stdout = captured_stdout(show_formats) + with captured_stdout() as stdout: + show_formats() + stdout = stdout.getvalue() # the output should be a header line + one line per format num_formats = len(get_archive_formats()) diff --git a/Lib/packaging/tests/test_config.py b/Lib/packaging/tests/test_config.py index a0e96d0..613d493 100644 --- a/Lib/packaging/tests/test_config.py +++ b/Lib/packaging/tests/test_config.py @@ -119,7 +119,7 @@ include_dirs = /usr/include/gecode /usr/include/blitz extra_compile_args = -fPIC -O2 -DGECODE_VERSION=$(./gecode_version) -- sys.platform != 'win32' - /DGECODE_VERSION='win32' -- sys.platform == 'win32' + /DGECODE_VERSION=win32 -- sys.platform == 'win32' language = cxx # corner case: if the parent package of an extension is declared but diff --git a/Lib/packaging/tests/test_database.py b/Lib/packaging/tests/test_database.py index 6f63e19..ad91b94 100644 --- a/Lib/packaging/tests/test_database.py +++ b/Lib/packaging/tests/test_database.py @@ -10,6 +10,7 @@ from textwrap import dedent from packaging.tests.test_util import GlobTestCaseBase from packaging.tests.support import requires_zlib +import packaging.database from packaging.config import get_resources_dests from packaging.errors import PackagingError from packaging.metadata import Metadata @@ -279,6 +280,12 @@ class TestDatabase(support.LoggingCatcher, sys.path.insert(0, self.fake_dists_path) self.addCleanup(sys.path.remove, self.fake_dists_path) + def test_caches(self): + # sanity check for internal caches + for name in ('_cache_name', '_cache_name_egg', + '_cache_path', '_cache_path_egg'): + self.assertEqual(getattr(packaging.database, name), {}) + def test_distinfo_dirname(self): # Given a name and a version, we expect the distinfo_dirname function # to return a standard distribution information directory name. diff --git a/Lib/packaging/tests/test_dist.py b/Lib/packaging/tests/test_dist.py index 4cac842..0623990 100644 --- a/Lib/packaging/tests/test_dist.py +++ b/Lib/packaging/tests/test_dist.py @@ -8,7 +8,6 @@ import packaging.dist from packaging.dist import Distribution from packaging.command.cmd import Command from packaging.errors import PackagingModuleError, PackagingOptionError -from packaging.tests import captured_stdout from packaging.tests import support, unittest from packaging.tests.support import create_distribution, use_command from test.support import unload diff --git a/Lib/packaging/tests/test_manifest.py b/Lib/packaging/tests/test_manifest.py index 5f89331..f768c47 100644 --- a/Lib/packaging/tests/test_manifest.py +++ b/Lib/packaging/tests/test_manifest.py @@ -27,14 +27,6 @@ class ManifestTestCase(support.TempdirManager, support.LoggingCatcher, unittest.TestCase): - def setUp(self): - super(ManifestTestCase, self).setUp() - self.cwd = os.getcwd() - - def tearDown(self): - os.chdir(self.cwd) - super(ManifestTestCase, self).tearDown() - def assertNoWarnings(self): self.assertEqual(self.get_logs(), []) diff --git a/Lib/packaging/tests/test_uninstall.py b/Lib/packaging/tests/test_uninstall.py index 9e03b54..b0d9ba7 100644 --- a/Lib/packaging/tests/test_uninstall.py +++ b/Lib/packaging/tests/test_uninstall.py @@ -31,14 +31,9 @@ class UninstallTestCase(support.TempdirManager, def setUp(self): super(UninstallTestCase, self).setUp() self.addCleanup(enable_cache) - self.root_dir = self.mkdtemp() - self.cwd = os.getcwd() + self.addCleanup(packaging.util._path_created.clear) disable_cache() - def tearDown(self): - packaging.util._path_created.clear() - super(UninstallTestCase, self).tearDown() - def get_path(self, dist, name): # the dist argument must contain an install_dist command correctly # initialized with a prefix option and finalized befored this method @@ -79,7 +74,7 @@ class UninstallTestCase(support.TempdirManager, dist.parse_config_files() dist.finalize_options() dist.run_command('install_dist', - {'prefix': ('command line', self.root_dir)}) + {'prefix': ('command line', self.mkdtemp())}) site_packages = self.get_path(dist, 'purelib') return dist, site_packages diff --git a/Lib/packaging/tests/test_util.py b/Lib/packaging/tests/test_util.py index 262fef4..737503e 100644 --- a/Lib/packaging/tests/test_util.py +++ b/Lib/packaging/tests/test_util.py @@ -602,14 +602,6 @@ class GlobTestCaseBase(support.TempdirManager, class GlobTestCase(GlobTestCaseBase): - def setUp(self): - super(GlobTestCase, self).setUp() - self.cwd = os.getcwd() - - def tearDown(self): - os.chdir(self.cwd) - super(GlobTestCase, self).tearDown() - def assertGlobMatch(self, glob, spec): tempdir = self.build_files_tree(spec) expected = self.clean_tree(spec) diff --git a/Lib/packaging/util.py b/Lib/packaging/util.py index e76e3c3..5b651b1 100644 --- a/Lib/packaging/util.py +++ b/Lib/packaging/util.py @@ -259,7 +259,7 @@ def split_multiline(value): if element] -def execute(func, args, msg=None, verbose=0, dry_run=False): +def execute(func, args, msg=None, dry_run=False): """Perform some action that affects the outside world. Some actions (e.g. writing to the filesystem) are special because @@ -296,7 +296,7 @@ def strtobool(val): def byte_compile(py_files, optimize=0, force=False, prefix=None, - base_dir=None, verbose=0, dry_run=False, direct=None): + base_dir=None, dry_run=False, direct=None): """Byte-compile a collection of Python source files to either .pyc or .pyo files in a __pycache__ subdirectory. @@ -305,6 +305,9 @@ def byte_compile(py_files, optimize=0, force=False, prefix=None, 0 - don't optimize (generate .pyc) 1 - normal optimization (like "python -O") 2 - extra optimization (like "python -OO") + This function is independent from the running Python's -O or -B options; + it is fully controlled by the parameters passed in. + If 'force' is true, all files are recompiled regardless of timestamps. @@ -325,10 +328,9 @@ def byte_compile(py_files, optimize=0, force=False, prefix=None, the source for details). The 'direct' flag is used by the script generated in indirect mode; unless you know what you're doing, leave it set to None. - - This function is independent from the running Python's -O or -B options; - it is fully controlled by the parameters passed in. """ + # FIXME use compileall + remove direct/indirect shenanigans + # First, if the caller didn't force us into direct or indirect mode, # figure out which mode we should be in. We take a conservative # approach: choose direct mode *only* if the current interpreter is @@ -381,15 +383,11 @@ files = [ script.write(""" byte_compile(files, optimize=%r, force=%r, prefix=%r, base_dir=%r, - verbose=%r, dry_run=False, + dry_run=False, direct=True) -""" % (optimize, force, prefix, base_dir, verbose)) +""" % (optimize, force, prefix, base_dir)) cmd = [sys.executable, script_name] - if optimize == 1: - cmd.insert(1, "-O") - elif optimize == 2: - cmd.insert(1, "-OO") env = os.environ.copy() env['PYTHONPATH'] = os.path.pathsep.join(sys.path) @@ -415,8 +413,10 @@ byte_compile(files, optimize=%r, force=%r, # Terminology from the py_compile module: # cfile - byte-compiled file # dfile - purported source filename (same as 'file' by default) - debug_override = not optimize - cfile = imp.cache_from_source(file, debug_override) + # The second argument to cache_from_source forces the extension to + # be .pyc (if true) or .pyo (if false); without it, the extension + # would depend on the calling Python's -O option + cfile = imp.cache_from_source(file, not optimize) dfile = file if prefix: @@ -681,7 +681,7 @@ if sys.platform == 'darwin': _cfg_target_split = None -def spawn(cmd, search_path=True, verbose=0, dry_run=False, env=None): +def spawn(cmd, search_path=True, dry_run=False, env=None): """Run another program specified as a command list 'cmd' in a new process. 'cmd' is just the argument list for the new process, ie. @@ -1331,10 +1331,9 @@ def get_install_method(path): # XXX to be replaced by shutil.copytree def copy_tree(src, dst, preserve_mode=True, preserve_times=True, - preserve_symlinks=False, update=False, verbose=True, - dry_run=False): + preserve_symlinks=False, update=False, dry_run=False): # FIXME use of this function is why we get spurious logging message on - # stdout when tests run; kill and replace by shuil! + # stdout when tests run; kill and replace by shutil! from distutils.file_util import copy_file if not dry_run and not os.path.isdir(src): @@ -1351,7 +1350,7 @@ def copy_tree(src, dst, preserve_mode=True, preserve_times=True, "error listing files in '%s': %s" % (src, errstr)) if not dry_run: - _mkpath(dst, verbose=verbose) + _mkpath(dst) outputs = [] @@ -1361,8 +1360,7 @@ def copy_tree(src, dst, preserve_mode=True, preserve_times=True, if preserve_symlinks and os.path.islink(src_name): link_dest = os.readlink(src_name) - if verbose >= 1: - logger.info("linking %s -> %s", dst_name, link_dest) + logger.info("linking %s -> %s", dst_name, link_dest) if not dry_run: os.symlink(link_dest, dst_name) outputs.append(dst_name) @@ -1371,11 +1369,10 @@ def copy_tree(src, dst, preserve_mode=True, preserve_times=True, outputs.extend( copy_tree(src_name, dst_name, preserve_mode, preserve_times, preserve_symlinks, update, - verbose=verbose, dry_run=dry_run)) + dry_run=dry_run)) else: copy_file(src_name, dst_name, preserve_mode, - preserve_times, update, verbose=verbose, - dry_run=dry_run) + preserve_times, update, dry_run=dry_run) outputs.append(dst_name) return outputs @@ -1388,7 +1385,7 @@ _path_created = set() # I don't use os.makedirs because a) it's new to Python 1.5.2, and # b) it blows up if the directory already exists (I want to silently # succeed in that case). -def _mkpath(name, mode=0o777, verbose=True, dry_run=False): +def _mkpath(name, mode=0o777, dry_run=False): # Detect a common bug -- name is None if not isinstance(name, str): raise PackagingInternalError( @@ -1423,9 +1420,7 @@ def _mkpath(name, mode=0o777, verbose=True, dry_run=False): if abs_head in _path_created: continue - if verbose >= 1: - logger.info("creating %s", head) - + logger.info("creating %s", head) if not dry_run: try: os.mkdir(head, mode) @@ -1448,8 +1443,7 @@ def encode_multipart(fields, files, boundary=None): Returns (content_type: bytes, body: bytes) ready for http.client.HTTP. """ - # Taken from - # http://code.activestate.com/recipes/146306-http-client-to-post-using-multipartform-data/ + # Taken from http://code.activestate.com/recipes/146306 if boundary is None: boundary = b'--------------GHSKFJDLGDS7543FJKLFHRE75642756743254' diff --git a/Lib/test/regrtest.py b/Lib/test/regrtest.py index 7f42eaf..f859c51 100755 --- a/Lib/test/regrtest.py +++ b/Lib/test/regrtest.py @@ -1066,7 +1066,7 @@ class saved_test_environment: keys = set(packaging.command._COMMANDS) return id_, keys def restore_packaging_command__COMMANDS(self, saved): - # if command._COMMANDS was bound to another dict obhect, we can't + # if command._COMMANDS was bound to another dict object, we can't # restore the previous object and contents, because the get_ method # above does not return the dict object (to ignore changes in values) for key in packaging.command._COMMANDS.keys() - saved[1]: @@ -378,6 +378,7 @@ Paul ten Hagen Rasmus Hahn Peter Haight Václav Haisman +Walker Hale IV Bob Halley Jesse Hallio Jun Hamano @@ -593,6 +594,7 @@ Eric Lindvall Gregor Lingl Nick Lockwood Stephanie Lockwood +Hugo Lopes Tavares Anne Lord Tom Loredo Justin Love @@ -606,6 +608,7 @@ Mark Lutz Jim Lynch Mikael Lyngvig Martin von Löwis +Guillermo López-Anglada Andrew I MacIntyre Tim MacKenzie Nick Maclaren @@ -805,6 +808,7 @@ Francesco Ricciardi Tim Rice Jan Pieter Riegel Armin Rigo +Arc Riley Nicholas Riley Jean-Claude Rimbault Vlad Riscutia @@ -817,6 +821,7 @@ Jim Robinson Mark Roddy Kevin Rodgers Giampaolo Rodola +Elson Rodriguez Luis Rojas Mike Romberg Armin Ronacher @@ -963,13 +963,9 @@ Library - Issue #12049: Add RAND_bytes() and RAND_pseudo_bytes() functions to the ssl module. -- Issue #12125: fixed the failures under Solaris due to improper test cleanup. - - Issue #6501: os.device_encoding() returns None on Windows if the application has no console. -- Issue #12132: Skip test_build_ext in case the xxmodule is not found. - - Issue #12105: Add O_CLOEXEC to the os module. - Issue #12079: Decimal('Infinity').fma(Decimal('0'), (3.91224318126786e+19+0j)) @@ -1695,12 +1691,6 @@ Tests - Issue #12057: Add tests for ISO 2022 codecs (iso2022_jp, iso2022_jp_2, iso2022_kr). -- Issue #12180: Fixed a few remaining errors in test_packaging when no - threading. - -- Issue #12120, #12119: skip a test in packaging and distutils - if sys.dont_write_bytecode is set to True. - - Issue #12096: Fix a race condition in test_threading.test_waitfor(). Patch written by Charles-François Natali. |