diff options
| -rw-r--r-- | Doc/library/test.rst | 219 | ||||
| -rw-r--r-- | Lib/test/support/__init__.py | 232 | ||||
| -rw-r--r-- | Lib/test/support/import_helper.py | 238 |
3 files changed, 359 insertions, 330 deletions
diff --git a/Doc/library/test.rst b/Doc/library/test.rst index 11d7484..a18197a 100644 --- a/Doc/library/test.rst +++ b/Doc/library/test.rst @@ -398,25 +398,6 @@ The :mod:`test.support` module defines the following constants: The :mod:`test.support` module defines the following functions: -.. function:: forget(module_name) - - Remove the module named *module_name* from ``sys.modules`` and delete any - byte-compiled files of the module. - - -.. function:: unload(name) - - Delete *name* from ``sys.modules``. - - -.. function:: make_legacy_pyc(source) - - Move a :pep:`3147`/:pep:`488` pyc file to its legacy pyc location and return the file - system path to the legacy pyc file. The *source* value is the file system - path to the source file. It does not need to exist, however the PEP - 3147/488 pyc file must exist. - - .. function:: is_resource_enabled(resource) Return ``True`` if *resource* is enabled and available. The list of @@ -889,67 +870,6 @@ The :mod:`test.support` module defines the following functions: Open *url*. If open fails, raises :exc:`TestFailed`. -.. function:: import_module(name, deprecated=False, *, required_on()) - - This function imports and returns the named module. Unlike a normal - import, this function raises :exc:`unittest.SkipTest` if the module - cannot be imported. - - Module and package deprecation messages are suppressed during this import - if *deprecated* is ``True``. If a module is required on a platform but - optional for others, set *required_on* to an iterable of platform prefixes - which will be compared against :data:`sys.platform`. - - .. versionadded:: 3.1 - - -.. function:: import_fresh_module(name, fresh=(), blocked=(), deprecated=False) - - This function imports and returns a fresh copy of the named Python module - by removing the named module from ``sys.modules`` before doing the import. - Note that unlike :func:`reload`, the original module is not affected by - this operation. - - *fresh* is an iterable of additional module names that are also removed - from the ``sys.modules`` cache before doing the import. - - *blocked* is an iterable of module names that are replaced with ``None`` - in the module cache during the import to ensure that attempts to import - them raise :exc:`ImportError`. - - The named module and any modules named in the *fresh* and *blocked* - parameters are saved before starting the import and then reinserted into - ``sys.modules`` when the fresh import is complete. - - Module and package deprecation messages are suppressed during this import - if *deprecated* is ``True``. - - This function will raise :exc:`ImportError` if the named module cannot be - imported. - - Example use:: - - # Get copies of the warnings module for testing without affecting the - # version being used by the rest of the test suite. One copy uses the - # C implementation, the other is forced to use the pure Python fallback - # implementation - py_warnings = import_fresh_module('warnings', blocked=['_warnings']) - c_warnings = import_fresh_module('warnings', fresh=['_warnings']) - - .. versionadded:: 3.1 - - -.. function:: modules_setup() - - Return a copy of :data:`sys.modules`. - - -.. function:: modules_cleanup(oldmodules) - - Remove modules except for *oldmodules* and ``encodings`` in order to - preserve internal cache. - - .. function:: reap_children() Use this at the end of ``test_main`` whenever sub-processes are started. @@ -1113,29 +1033,6 @@ The :mod:`test.support` module defines the following classes: On both platforms, the old value is restored by :meth:`__exit__`. -.. class:: CleanImport(*module_names) - - A context manager to force import to return a new module reference. This - is useful for testing module-level behaviors, such as the emission of a - DeprecationWarning on import. Example usage:: - - with CleanImport('foo'): - importlib.import_module('foo') # New reference. - - -.. class:: DirsOnSysPath(*paths) - - A context manager to temporarily add directories to sys.path. - - This makes a copy of :data:`sys.path`, appends any directories given - as positional arguments, then reverts :data:`sys.path` to the copied - settings when the context ends. - - Note that *all* :data:`sys.path` modifications in the body of the - context manager, including replacement of the object, - will be reverted at the end of the block. - - .. class:: SaveSignals() Class to save and restore signal handlers registered by the Python signal @@ -1646,3 +1543,119 @@ The :mod:`test.support.os_helper` module provides support for os tests. Call :func:`os.unlink` on *filename*. On Windows platforms, this is wrapped with a wait loop that checks for the existence fo the file. + + +:mod:`test.support.import_helper` --- Utilities for import tests +================================================================ + +.. module:: test.support.import_helper + :synopsis: Support for import tests. + +The :mod:`test.support.import_helper` module provides support for import tests. + +.. versionadded:: 3.10 + + +.. function:: forget(module_name) + + Remove the module named *module_name* from ``sys.modules`` and delete any + byte-compiled files of the module. + + +.. function:: import_fresh_module(name, fresh=(), blocked=(), deprecated=False) + + This function imports and returns a fresh copy of the named Python module + by removing the named module from ``sys.modules`` before doing the import. + Note that unlike :func:`reload`, the original module is not affected by + this operation. + + *fresh* is an iterable of additional module names that are also removed + from the ``sys.modules`` cache before doing the import. + + *blocked* is an iterable of module names that are replaced with ``None`` + in the module cache during the import to ensure that attempts to import + them raise :exc:`ImportError`. + + The named module and any modules named in the *fresh* and *blocked* + parameters are saved before starting the import and then reinserted into + ``sys.modules`` when the fresh import is complete. + + Module and package deprecation messages are suppressed during this import + if *deprecated* is ``True``. + + This function will raise :exc:`ImportError` if the named module cannot be + imported. + + Example use:: + + # Get copies of the warnings module for testing without affecting the + # version being used by the rest of the test suite. One copy uses the + # C implementation, the other is forced to use the pure Python fallback + # implementation + py_warnings = import_fresh_module('warnings', blocked=['_warnings']) + c_warnings = import_fresh_module('warnings', fresh=['_warnings']) + + .. versionadded:: 3.1 + + +.. function:: import_module(name, deprecated=False, *, required_on()) + + This function imports and returns the named module. Unlike a normal + import, this function raises :exc:`unittest.SkipTest` if the module + cannot be imported. + + Module and package deprecation messages are suppressed during this import + if *deprecated* is ``True``. If a module is required on a platform but + optional for others, set *required_on* to an iterable of platform prefixes + which will be compared against :data:`sys.platform`. + + .. versionadded:: 3.1 + + +.. function:: modules_setup() + + Return a copy of :data:`sys.modules`. + + +.. function:: modules_cleanup(oldmodules) + + Remove modules except for *oldmodules* and ``encodings`` in order to + preserve internal cache. + + +.. function:: unload(name) + + Delete *name* from ``sys.modules``. + + +.. function:: make_legacy_pyc(source) + + Move a :pep:`3147`/:pep:`488` pyc file to its legacy pyc location and return the file + system path to the legacy pyc file. The *source* value is the file system + path to the source file. It does not need to exist, however the PEP + 3147/488 pyc file must exist. + + +.. class:: CleanImport(*module_names) + + A context manager to force import to return a new module reference. This + is useful for testing module-level behaviors, such as the emission of a + DeprecationWarning on import. Example usage:: + + with CleanImport('foo'): + importlib.import_module('foo') # New reference. + + +.. class:: DirsOnSysPath(*paths) + + A context manager to temporarily add directories to sys.path. + + This makes a copy of :data:`sys.path`, appends any directories given + as positional arguments, then reverts :data:`sys.path` to the copied + settings when the context ends. + + Note that *all* :data:`sys.path` modifications in the body of the + context manager, including replacement of the object, + will be reverted at the end of the block. + + diff --git a/Lib/test/support/__init__.py b/Lib/test/support/__init__.py index 83b2173..3778eed 100644 --- a/Lib/test/support/__init__.py +++ b/Lib/test/support/__init__.py @@ -8,8 +8,6 @@ import errno import fnmatch import functools import glob -import importlib -import importlib.util import os import re import stat @@ -21,6 +19,11 @@ import types import unittest import warnings +from .import_helper import ( + CleanImport, DirsOnSysPath, _ignore_deprecated_imports, + _save_and_block_module, _save_and_remove_module, + forget, import_fresh_module, import_module, make_legacy_pyc, + modules_cleanup, modules_setup, unload) from .os_helper import ( FS_NONASCII, SAVEDCWD, TESTFN, TESTFN_NONASCII, TESTFN_UNENCODABLE, TESTFN_UNDECODABLE, @@ -39,10 +42,6 @@ __all__ = [ "PIPE_MAX_SIZE", "verbose", "max_memuse", "use_resources", "failfast", # exceptions "Error", "TestFailed", "TestDidNotRun", "ResourceDenied", - # imports - "import_module", "import_fresh_module", "CleanImport", - # modules - "unload", "forget", # io "record_original_stdout", "get_original_stdout", "captured_stdout", "captured_stdin", "captured_stderr", @@ -132,22 +131,6 @@ class ResourceDenied(unittest.SkipTest): and unexpected skips. """ -@contextlib.contextmanager -def _ignore_deprecated_imports(ignore=True): - """Context manager to suppress package and module deprecation - warnings when importing them. - - If ignore is False, this context manager has no effect. - """ - if ignore: - with warnings.catch_warnings(): - warnings.filterwarnings("ignore", ".+ (module|package)", - DeprecationWarning) - yield - else: - yield - - def ignore_warnings(*, category): """Decorator to suppress deprecation warnings. @@ -164,52 +147,6 @@ def ignore_warnings(*, category): return decorator -def import_module(name, deprecated=False, *, required_on=()): - """Import and return the module to be tested, raising SkipTest if - it is not available. - - If deprecated is True, any module or package deprecation messages - will be suppressed. If a module is required on a platform but optional for - others, set required_on to an iterable of platform prefixes which will be - compared against sys.platform. - """ - with _ignore_deprecated_imports(deprecated): - try: - return importlib.import_module(name) - except ImportError as msg: - if sys.platform.startswith(tuple(required_on)): - raise - raise unittest.SkipTest(str(msg)) - - -def _save_and_remove_module(name, orig_modules): - """Helper function to save and remove a module from sys.modules - - Raise ImportError if the module can't be imported. - """ - # try to import the module and raise an error if it can't be imported - if name not in sys.modules: - __import__(name) - del sys.modules[name] - for modname in list(sys.modules): - if modname == name or modname.startswith(name + '.'): - orig_modules[modname] = sys.modules[modname] - del sys.modules[modname] - -def _save_and_block_module(name, orig_modules): - """Helper function to save and block a module in sys.modules - - Return True if the module was in sys.modules, False otherwise. - """ - saved = True - try: - orig_modules[name] = sys.modules[name] - except KeyError: - saved = False - sys.modules[name] = None - return saved - - def anticipate_failure(condition): """Decorator to mark a test that is known to be broken in some cases @@ -240,56 +177,6 @@ def load_package_tests(pkg_dir, loader, standard_tests, pattern): return standard_tests -def import_fresh_module(name, fresh=(), blocked=(), deprecated=False): - """Import and return a module, deliberately bypassing sys.modules. - - This function imports and returns a fresh copy of the named Python module - by removing the named module from sys.modules before doing the import. - Note that unlike reload, the original module is not affected by - this operation. - - *fresh* is an iterable of additional module names that are also removed - from the sys.modules cache before doing the import. - - *blocked* is an iterable of module names that are replaced with None - in the module cache during the import to ensure that attempts to import - them raise ImportError. - - The named module and any modules named in the *fresh* and *blocked* - parameters are saved before starting the import and then reinserted into - sys.modules when the fresh import is complete. - - Module and package deprecation messages are suppressed during this import - if *deprecated* is True. - - This function will raise ImportError if the named module cannot be - imported. - """ - # NOTE: test_heapq, test_json and test_warnings include extra sanity checks - # to make sure that this utility function is working as expected - with _ignore_deprecated_imports(deprecated): - # Keep track of modules saved for later restoration as well - # as those which just need a blocking entry removed - orig_modules = {} - names_to_remove = [] - _save_and_remove_module(name, orig_modules) - try: - for fresh_name in fresh: - _save_and_remove_module(fresh_name, orig_modules) - for blocked_name in blocked: - if not _save_and_block_module(blocked_name, orig_modules): - names_to_remove.append(blocked_name) - fresh_module = importlib.import_module(name) - except ImportError: - fresh_module = None - finally: - for orig_name, module in orig_modules.items(): - sys.modules[orig_name] = module - for name_to_remove in names_to_remove: - del sys.modules[name_to_remove] - return fresh_module - - def get_attribute(obj, name): """Get an attribute, raising SkipTest if AttributeError is raised.""" try: @@ -318,12 +205,6 @@ def record_original_stdout(stdout): def get_original_stdout(): return _original_stdout or sys.stdout -def unload(name): - try: - del sys.modules[name] - except KeyError: - pass - def _force_run(path, func, *args): try: @@ -336,34 +217,6 @@ def _force_run(path, func, *args): return func(*args) -def make_legacy_pyc(source): - """Move a PEP 3147/488 pyc file to its legacy pyc location. - - :param source: The file system path to the source file. The source file - does not need to exist, however the PEP 3147/488 pyc file must exist. - :return: The file system path to the legacy pyc file. - """ - pyc_file = importlib.util.cache_from_source(source) - up_one = os.path.dirname(os.path.abspath(source)) - legacy_pyc = os.path.join(up_one, source + 'c') - os.rename(pyc_file, legacy_pyc) - return legacy_pyc - -def forget(modname): - """'Forget' a module was ever imported. - - This removes the module from sys.modules and deletes any PEP 3147/488 or - legacy .pyc files. - """ - unload(modname) - for dirname in sys.path: - source = os.path.join(dirname, modname + '.py') - # It doesn't matter if they exist or not, unlink all possible - # combinations of PEP 3147/488 and legacy pyc files. - unlink(source + 'c') - for opt in ('', 1, 2): - unlink(importlib.util.cache_from_source(source, optimization=opt)) - # Check whether a gui is actually available def _is_gui_available(): if hasattr(_is_gui_available, 'result'): @@ -870,63 +723,6 @@ def check_no_resource_warning(testcase): yield -class CleanImport(object): - """Context manager to force import to return a new module reference. - - This is useful for testing module-level behaviours, such as - the emission of a DeprecationWarning on import. - - Use like this: - - with CleanImport("foo"): - importlib.import_module("foo") # new reference - """ - - def __init__(self, *module_names): - self.original_modules = sys.modules.copy() - for module_name in module_names: - if module_name in sys.modules: - module = sys.modules[module_name] - # It is possible that module_name is just an alias for - # another module (e.g. stub for modules renamed in 3.x). - # In that case, we also need delete the real module to clear - # the import cache. - if module.__name__ != module_name: - del sys.modules[module.__name__] - del sys.modules[module_name] - - def __enter__(self): - return self - - def __exit__(self, *ignore_exc): - sys.modules.update(self.original_modules) - - -class DirsOnSysPath(object): - """Context manager to temporarily add directories to sys.path. - - This makes a copy of sys.path, appends any directories given - as positional arguments, then reverts sys.path to the copied - settings when the context ends. - - Note that *all* sys.path modifications in the body of the - context manager, including replacement of the object, - will be reverted at the end of the block. - """ - - def __init__(self, *paths): - self.original_value = sys.path[:] - self.original_object = sys.path - sys.path.extend(paths) - - def __enter__(self): - return self - - def __exit__(self, *ignore_exc): - sys.path = self.original_object - sys.path[:] = self.original_value - - class TransientResource(object): """Raise ResourceDenied if an exception is raised while the context manager @@ -1553,24 +1349,6 @@ def print_warning(msg): for line in msg.splitlines(): print(f"Warning -- {line}", file=sys.__stderr__, flush=True) -def modules_setup(): - return sys.modules.copy(), - -def modules_cleanup(oldmodules): - # Encoders/decoders are registered permanently within the internal - # codec cache. If we destroy the corresponding modules their - # globals will be set to None which will trip up the cached functions. - encodings = [(k, v) for k, v in sys.modules.items() - if k.startswith('encodings.')] - sys.modules.clear() - sys.modules.update(encodings) - # XXX: This kind of problem can affect more than just encodings. In particular - # extension modules (such as _ssl) don't cope with reloading properly. - # Really, test modules should be cleaning out the test specific modules they - # know they added (ala test_runpy) rather than relying on this function (as - # test_importhooks and test_pkg do currently). - # Implicitly imported *real* modules should be left alone (see issue 10556). - sys.modules.update(oldmodules) # Flag used by saved_test_environment of test.libregrtest.save_env, # to check if a test modified the environment. The flag should be set to False diff --git a/Lib/test/support/import_helper.py b/Lib/test/support/import_helper.py new file mode 100644 index 0000000..5d1e940 --- /dev/null +++ b/Lib/test/support/import_helper.py @@ -0,0 +1,238 @@ +import contextlib +import importlib +import importlib.util +import os +import sys +import unittest +import warnings + +from .os_helper import unlink + + +@contextlib.contextmanager +def _ignore_deprecated_imports(ignore=True): + """Context manager to suppress package and module deprecation + warnings when importing them. + + If ignore is False, this context manager has no effect. + """ + if ignore: + with warnings.catch_warnings(): + warnings.filterwarnings("ignore", ".+ (module|package)", + DeprecationWarning) + yield + else: + yield + + +def unload(name): + try: + del sys.modules[name] + except KeyError: + pass + + +def forget(modname): + """'Forget' a module was ever imported. + + This removes the module from sys.modules and deletes any PEP 3147/488 or + legacy .pyc files. + """ + unload(modname) + for dirname in sys.path: + source = os.path.join(dirname, modname + '.py') + # It doesn't matter if they exist or not, unlink all possible + # combinations of PEP 3147/488 and legacy pyc files. + unlink(source + 'c') + for opt in ('', 1, 2): + unlink(importlib.util.cache_from_source(source, optimization=opt)) + + +def make_legacy_pyc(source): + """Move a PEP 3147/488 pyc file to its legacy pyc location. + + :param source: The file system path to the source file. The source file + does not need to exist, however the PEP 3147/488 pyc file must exist. + :return: The file system path to the legacy pyc file. + """ + pyc_file = importlib.util.cache_from_source(source) + up_one = os.path.dirname(os.path.abspath(source)) + legacy_pyc = os.path.join(up_one, source + 'c') + os.rename(pyc_file, legacy_pyc) + return legacy_pyc + + +def import_module(name, deprecated=False, *, required_on=()): + """Import and return the module to be tested, raising SkipTest if + it is not available. + + If deprecated is True, any module or package deprecation messages + will be suppressed. If a module is required on a platform but optional for + others, set required_on to an iterable of platform prefixes which will be + compared against sys.platform. + """ + with _ignore_deprecated_imports(deprecated): + try: + return importlib.import_module(name) + except ImportError as msg: + if sys.platform.startswith(tuple(required_on)): + raise + raise unittest.SkipTest(str(msg)) + + +def _save_and_remove_module(name, orig_modules): + """Helper function to save and remove a module from sys.modules + + Raise ImportError if the module can't be imported. + """ + # try to import the module and raise an error if it can't be imported + if name not in sys.modules: + __import__(name) + del sys.modules[name] + for modname in list(sys.modules): + if modname == name or modname.startswith(name + '.'): + orig_modules[modname] = sys.modules[modname] + del sys.modules[modname] + + +def _save_and_block_module(name, orig_modules): + """Helper function to save and block a module in sys.modules + + Return True if the module was in sys.modules, False otherwise. + """ + saved = True + try: + orig_modules[name] = sys.modules[name] + except KeyError: + saved = False + sys.modules[name] = None + return saved + + +def import_fresh_module(name, fresh=(), blocked=(), deprecated=False): + """Import and return a module, deliberately bypassing sys.modules. + + This function imports and returns a fresh copy of the named Python module + by removing the named module from sys.modules before doing the import. + Note that unlike reload, the original module is not affected by + this operation. + + *fresh* is an iterable of additional module names that are also removed + from the sys.modules cache before doing the import. + + *blocked* is an iterable of module names that are replaced with None + in the module cache during the import to ensure that attempts to import + them raise ImportError. + + The named module and any modules named in the *fresh* and *blocked* + parameters are saved before starting the import and then reinserted into + sys.modules when the fresh import is complete. + + Module and package deprecation messages are suppressed during this import + if *deprecated* is True. + + This function will raise ImportError if the named module cannot be + imported. + """ + # NOTE: test_heapq, test_json and test_warnings include extra sanity checks + # to make sure that this utility function is working as expected + with _ignore_deprecated_imports(deprecated): + # Keep track of modules saved for later restoration as well + # as those which just need a blocking entry removed + orig_modules = {} + names_to_remove = [] + _save_and_remove_module(name, orig_modules) + try: + for fresh_name in fresh: + _save_and_remove_module(fresh_name, orig_modules) + for blocked_name in blocked: + if not _save_and_block_module(blocked_name, orig_modules): + names_to_remove.append(blocked_name) + fresh_module = importlib.import_module(name) + except ImportError: + fresh_module = None + finally: + for orig_name, module in orig_modules.items(): + sys.modules[orig_name] = module + for name_to_remove in names_to_remove: + del sys.modules[name_to_remove] + return fresh_module + + +class CleanImport(object): + """Context manager to force import to return a new module reference. + + This is useful for testing module-level behaviours, such as + the emission of a DeprecationWarning on import. + + Use like this: + + with CleanImport("foo"): + importlib.import_module("foo") # new reference + """ + + def __init__(self, *module_names): + self.original_modules = sys.modules.copy() + for module_name in module_names: + if module_name in sys.modules: + module = sys.modules[module_name] + # It is possible that module_name is just an alias for + # another module (e.g. stub for modules renamed in 3.x). + # In that case, we also need delete the real module to clear + # the import cache. + if module.__name__ != module_name: + del sys.modules[module.__name__] + del sys.modules[module_name] + + def __enter__(self): + return self + + def __exit__(self, *ignore_exc): + sys.modules.update(self.original_modules) + + +class DirsOnSysPath(object): + """Context manager to temporarily add directories to sys.path. + + This makes a copy of sys.path, appends any directories given + as positional arguments, then reverts sys.path to the copied + settings when the context ends. + + Note that *all* sys.path modifications in the body of the + context manager, including replacement of the object, + will be reverted at the end of the block. + """ + + def __init__(self, *paths): + self.original_value = sys.path[:] + self.original_object = sys.path + sys.path.extend(paths) + + def __enter__(self): + return self + + def __exit__(self, *ignore_exc): + sys.path = self.original_object + sys.path[:] = self.original_value + + +def modules_setup(): + return sys.modules.copy(), + + +def modules_cleanup(oldmodules): + # Encoders/decoders are registered permanently within the internal + # codec cache. If we destroy the corresponding modules their + # globals will be set to None which will trip up the cached functions. + encodings = [(k, v) for k, v in sys.modules.items() + if k.startswith('encodings.')] + sys.modules.clear() + sys.modules.update(encodings) + # XXX: This kind of problem can affect more than just encodings. + # In particular extension modules (such as _ssl) don't cope + # with reloading properly. Really, test modules should be cleaning + # out the test specific modules they know they added (ala test_runpy) + # rather than relying on this function (as test_importhooks and test_pkg + # do currently). Implicitly imported *real* modules should be left alone + # (see issue 10556). + sys.modules.update(oldmodules) |