diff options
Diffstat (limited to 'Doc/library/importlib.rst')
| -rw-r--r-- | Doc/library/importlib.rst | 500 |
1 files changed, 449 insertions, 51 deletions
diff --git a/Doc/library/importlib.rst b/Doc/library/importlib.rst index 1649063..083656e 100644 --- a/Doc/library/importlib.rst +++ b/Doc/library/importlib.rst @@ -63,7 +63,7 @@ Details on custom importers can be found in :pep:`302`. Functions --------- -.. function:: __import__(name, globals={}, locals={}, fromlist=list(), level=0) +.. function:: __import__(name, globals=None, locals=None, fromlist=(), level=0) An implementation of the built-in :func:`__import__` function. @@ -86,6 +86,29 @@ Functions that was imported (e.g. ``pkg.mod``), while :func:`__import__` returns the top-level package or module (e.g. ``pkg``). +.. function:: find_loader(name, path=None) + + Find the loader for a module, optionally within the specified *path*. If the + module is in :attr:`sys.modules`, then ``sys.modules[name].__loader__`` is + returned (unless the loader would be ``None``, in which case + :exc:`ValueError` is raised). Otherwise a search using :attr:`sys.meta_path` + is done. ``None`` is returned if no loader is found. + + A dotted name does not have its parent's implicitly imported as that requires + loading them and that may not be desired. To properly import a submodule you + will need to import all parent packages of the submodule and use the correct + argument to *path*. + +.. function:: invalidate_caches() + + Invalidate the internal caches of finders stored at + :data:`sys.meta_path`. If a finder implements ``invalidate_caches()`` then it + will be called to perform the invalidation. This function may be needed if + some modules are installed while your program is running and you expect the + program to notice the changes. + + .. versionadded:: 3.3 + :mod:`importlib.abc` -- Abstract base classes related to import --------------------------------------------------------------- @@ -97,19 +120,90 @@ The :mod:`importlib.abc` module contains all of the core abstract base classes used by :keyword:`import`. Some subclasses of the core abstract base classes are also provided to help in implementing the core ABCs. +ABC hierarchy:: + + object + +-- Finder (deprecated) + | +-- MetaPathFinder + | +-- PathEntryFinder + +-- Loader + +-- ResourceLoader --------+ + +-- InspectLoader | + +-- ExecutionLoader --+ + +-- FileLoader + +-- SourceLoader + +-- PyLoader (deprecated) + +-- PyPycLoader (deprecated) + .. class:: Finder - An abstract base class representing a :term:`finder`. - See :pep:`302` for the exact definition for a finder. + An abstract base class representing a :term:`finder`. + + .. deprecated:: 3.3 + Use :class:`MetaPathFinder` or :class:`PathEntryFinder` instead. + + .. method:: find_module(fullname, path=None) + + An abstact method for finding a :term:`loader` for the specified + module. Originally specified in :pep:`302`, this method was meant + for use in :data:`sys.meta_path` and in the path-based import subsystem. + + +.. class:: MetaPathFinder + + An abstract base class representing a :term:`meta path finder`. For + compatibility, this is a subclass of :class:`Finder`. + + .. versionadded:: 3.3 + + .. method:: find_module(fullname, path) + + An abstract method for finding a :term:`loader` for the specified + module. If this is a top-level import, *path* will be ``None``. + Otherwise, this is a search for a subpackage or module and *path* + will be the value of :attr:`__path__` from the parent + package. If a loader cannot be found, ``None`` is returned. + + .. method:: invalidate_caches() + + An optional method which, when called, should invalidate any internal + cache used by the finder. Used by :func:`importlib.invalidate_caches` + when invalidating the caches of all finders on :data:`sys.meta_path`. + - .. method:: find_module(fullname, path=None) +.. class:: PathEntryFinder - An abstract method for finding a :term:`loader` for the specified - module. If the :term:`finder` is found on :data:`sys.meta_path` and the - module to be searched for is a subpackage or module then *path* will - be the value of :attr:`__path__` from the parent package. If a loader - cannot be found, ``None`` is returned. + An abstract base class representing a :term:`path entry finder`. Though + it bears some similarities to :class:`MetaPathFinder`, ``PathEntryFinder`` + is meant for use only within the path-based import subsystem provided + by :class:`PathFinder`. This ABC is a subclass of :class:`Finder` for + compatibility. + + .. versionadded:: 3.3 + + .. method:: find_loader(fullname): + + An abstract method for finding a :term:`loader` for the specified + module. Returns a 2-tuple of ``(loader, portion)`` where ``portion`` + is a sequence of file system locations contributing to part of a namespace + package. The loader may be ``None`` while specifying ``portion`` to + signify the contribution of the file system locations to a namespace + package. An empty list can be used for ``portion`` to signify the loader + is not part of a package. If ``loader`` is ``None`` and ``portion`` is + the empty list then no loader or location for a namespace package were + found (i.e. failure to find anything for the module). + + .. method:: find_module(fullname): + + A concrete implementation of :meth:`Finder.find_module` which is + equivalent to ``self.find_loader(fullname)[0]``. + + .. method:: invalidate_caches() + + An optional method which, when called, should invalidate any internal + cache used by the finder. Used by :meth:`PathFinder.invalidate_caches` + when invalidating the caches of all cached finders. .. class:: Loader @@ -159,6 +253,13 @@ are also provided to help in implementing the core ABCs. (This is not set by the built-in import machinery, but it should be set whenever a :term:`loader` is used.) + .. method:: module_repr(module) + + An abstract method which when implemented calculates and returns the + given module's repr, as a string. + + .. versionadded: 3.3 + .. class:: ResourceLoader @@ -224,6 +325,38 @@ are also provided to help in implementing the core ABCs. module. +.. class:: FileLoader(fullname, path) + + An abstract base class which inherits from :class:`ResourceLoader` and + :class:`ExecutionLoader`, providing concreate implementations of + :meth:`ResourceLoader.get_data` and :meth:`ExecutionLoader.get_filename`. + + The *fullname* argument is a fully resolved name of the module the loader is + to handle. The *path* argument is the path to the file for the module. + + .. versionadded:: 3.3 + + .. attribute:: name + + The name of the module the loader can handle. + + .. attribute:: path + + Path to the file of the module. + + .. method:: load_module(fullname) + + Calls super's ``load_module()``. + + .. method:: get_filename(fullname) + + Returns :attr:`path`. + + .. method:: get_data(path) + + Returns the open, binary file for *path*. + + .. class:: SourceLoader An abstract base class for implementing source (and optionally bytecode) @@ -243,12 +376,31 @@ are also provided to help in implementing the core ABCs. optimization to speed up loading by removing the parsing step of Python's compiler, and so no bytecode-specific API is exposed. - .. method:: path_mtime(self, path) + .. method:: path_stats(path) + + Optional abstract method which returns a :class:`dict` containing + metadata about the specifed path. Supported dictionary keys are: + + - ``'mtime'`` (mandatory): an integer or floating-point number + representing the modification time of the source code; + - ``'size'`` (optional): the size in bytes of the source code. + + Any other keys in the dictionary are ignored, to allow for future + extensions. + + .. versionadded:: 3.3 + + .. method:: path_mtime(path) Optional abstract method which returns the modification time for the specified path. - .. method:: set_data(self, path, data) + .. deprecated:: 3.3 + This method is deprecated in favour of :meth:`path_stats`. You don't + have to implement it, but it is still available for compatibility + purposes. + + .. method:: set_data(path, data) Optional abstract method which writes the specified bytes to a file path. Any intermediate directories which do not exist are to be created @@ -257,23 +409,25 @@ are also provided to help in implementing the core ABCs. When writing to the path fails because the path is read-only (:attr:`errno.EACCES`), do not propagate the exception. - .. method:: get_code(self, fullname) + .. method:: get_code(fullname) Concrete implementation of :meth:`InspectLoader.get_code`. - .. method:: load_module(self, fullname) + .. method:: load_module(fullname) Concrete implementation of :meth:`Loader.load_module`. - .. method:: get_source(self, fullname) + .. method:: get_source(fullname) Concrete implementation of :meth:`InspectLoader.get_source`. - .. method:: is_package(self, fullname) + .. method:: is_package(fullname) Concrete implementation of :meth:`InspectLoader.is_package`. A module - is determined to be a package if its file path is a file named - ``__init__`` when the file extension is removed. + is determined to be a package if its file path (as provided by + :meth:`ExecutionLoader.get_filename`) is a file named + ``__init__`` when the file extension is removed **and** the module name + itself does not end in ``__init__``. .. class:: PyLoader @@ -374,6 +528,10 @@ are also provided to help in implementing the core ABCs. :class:`PyLoader`. Do note that this solution will not support sourceless/bytecode-only loading; only source *and* bytecode loading. + .. versionchanged:: 3.3 + Updated to parse (but not use) the new source size field in bytecode + files when reading and to write out the field properly when writing. + .. method:: source_mtime(fullname) An abstract method which returns the modification time for the source @@ -417,12 +575,59 @@ are also provided to help in implementing the core ABCs. This module contains the various objects that help :keyword:`import` find and load modules. +.. attribute:: SOURCE_SUFFIXES + + A list of strings representing the recognized file suffixes for source + modules. + + .. versionadded:: 3.3 + +.. attribute:: DEBUG_BYTECODE_SUFFIXES + + A list of strings representing the file suffixes for non-optimized bytecode + modules. + + .. versionadded:: 3.3 + +.. attribute:: OPTIMIZED_BYTECODE_SUFFIXES + + A list of strings representing the file suffixes for optimized bytecode + modules. + + .. versionadded:: 3.3 + +.. attribute:: BYTECODE_SUFFIXES + + A list of strings representing the recognized file suffixes for bytecode + modules. Set to either :attr:`DEBUG_BYTECODE_SUFFIXES` or + :attr:`OPTIMIZED_BYTECODE_SUFFIXES` based on whether ``__debug__`` is true. + + .. versionadded:: 3.3 + +.. attribute:: EXTENSION_SUFFIXES + + A list of strings representing the recognized file suffixes for + extension modules. + + .. versionadded:: 3.3 + +.. function:: all_suffixes() + + Returns a combined list of strings representing all file suffixes for + modules recognized by the standard import machinery. This is a + helper for code which simply needs to know if a filesystem path + potentially refers to a module without needing any details on the kind + of module (for example, :func:`inspect.getmodulename`) + + .. versionadded:: 3.3 + + .. class:: BuiltinImporter An :term:`importer` for built-in modules. All known built-in modules are listed in :data:`sys.builtin_module_names`. This class implements the - :class:`importlib.abc.Finder` and :class:`importlib.abc.InspectLoader` - ABCs. + :class:`importlib.abc.MetaPathFinder` and + :class:`importlib.abc.InspectLoader` ABCs. Only class methods are defined by this class to alleviate the need for instantiation. @@ -431,48 +636,223 @@ find and load modules. .. class:: FrozenImporter An :term:`importer` for frozen modules. This class implements the - :class:`importlib.abc.Finder` and :class:`importlib.abc.InspectLoader` - ABCs. + :class:`importlib.abc.MetaPathFinder` and + :class:`importlib.abc.InspectLoader` ABCs. Only class methods are defined by this class to alleviate the need for instantiation. +.. class:: WindowsRegistryFinder + + :term:`Finder` for modules declared in the Windows registry. This class + implements the :class:`importlib.abc.Finder` ABC. + + Only class methods are defined by this class to alleviate the need for + instantiation. + + .. versionadded:: 3.3 + + .. class:: PathFinder - :term:`Finder` for :data:`sys.path`. This class implements the - :class:`importlib.abc.Finder` ABC. + A :term:`Finder` for :data:`sys.path` and package ``__path__`` attributes. + This class implements the :class:`importlib.abc.MetaPathFinder` ABC. - This class does not perfectly mirror the semantics of :keyword:`import` in - terms of :data:`sys.path`. No implicit path hooks are assumed for - simplification of the class and its semantics. + Only class methods are defined by this class to alleviate the need for + instantiation. - Only class methods are defined by this class to alleviate the need for - instantiation. + .. classmethod:: find_module(fullname, path=None) + + Class method that attempts to find a :term:`loader` for the module + specified by *fullname* on :data:`sys.path` or, if defined, on + *path*. For each path entry that is searched, + :data:`sys.path_importer_cache` is checked. If a non-false object is + found then it is used as the :term:`finder` to look for the module + being searched for. If no entry is found in + :data:`sys.path_importer_cache`, then :data:`sys.path_hooks` is + searched for a finder for the path entry and, if found, is stored in + :data:`sys.path_importer_cache` along with being queried about the + module. If no finder is ever found then ``None`` is both stored in + the cache and returned. + + .. classmethod:: invalidate_caches() + + Calls :meth:`importlib.abc.PathEntryFinder.invalidate_caches` on all + finders stored in :attr:`sys.path_importer_cache`. + + +.. class:: FileFinder(path, \*loader_details) + + A concrete implementation of :class:`importlib.abc.PathEntryFinder` which + caches results from the file system. + + The *path* argument is the directory for which the finder is in charge of + searching. + + The *loader_details* argument is a variable number of 2-item tuples each + containing a loader and a sequence of file suffixes the loader recognizes. + + The finder will cache the directory contents as necessary, making stat calls + for each module search to verify the cache is not outdated. Because cache + staleness relies upon the granularity of the operating system's state + information of the file system, there is a potential race condition of + searching for a module, creating a new file, and then searching for the + module the new file represents. If the operations happen fast enough to fit + within the granularity of stat calls, then the module search will fail. To + prevent this from happening, when you create a module dynamically, make sure + to call :func:`importlib.invalidate_caches`. + + .. versionadded:: 3.3 + + .. attribute:: path + + The path the finder will search in. + + .. method:: find_module(fullname) + + Attempt to find the loader to handle *fullname* within :attr:`path`. + + .. method:: invalidate_caches() + + Clear out the internal cache. + + .. classmethod:: path_hook(\*loader_details) + + A class method which returns a closure for use on :attr:`sys.path_hooks`. + An instance of :class:`FileFinder` is returned by the closure using the + path argument given to the closure directly and *loader_details* + indirectly. + + If the argument to the closure is not an existing directory, + :exc:`ImportError` is raised. + + +.. class:: SourceFileLoader(fullname, path) + + A concrete implementation of :class:`importlib.abc.SourceLoader` by + subclassing :class:`importlib.abc.FileLoader` and providing some concrete + implementations of other methods. + + .. versionadded:: 3.3 + + .. attribute:: name + + The name of the module that this loader will handle. + + .. attribute:: path + + The path to the source file. + + .. method:: is_package(fullname) + + Return true if :attr:`path` appears to be for a package. + + .. method:: path_stats(path) + + Concrete implementation of :meth:`importlib.abc.SourceLoader.path_stats`. + + .. method:: set_data(path, data) + + Concrete implementation of :meth:`importlib.abc.SourceLoader.set_data`. + + +.. class:: SourcelessFileLoader(fullname, path) - .. classmethod:: find_module(fullname, path=None) + A concrete implementation of :class:`importlib.abc.FileLoader` which can + import bytecode files (i.e. no source code files exist). - Class method that attempts to find a :term:`loader` for the module - specified by *fullname* on :data:`sys.path` or, if defined, on - *path*. For each path entry that is searched, - :data:`sys.path_importer_cache` is checked. If an non-false object is - found then it is used as the :term:`finder` to look for the module - being searched for. If no entry is found in - :data:`sys.path_importer_cache`, then :data:`sys.path_hooks` is - searched for a finder for the path entry and, if found, is stored in - :data:`sys.path_importer_cache` along with being queried about the - module. If no finder is ever found then ``None`` is returned. + Please note that direct use of bytecode files (and thus not source code + files) inhibits your modules from being usable by all Python + implementations or new versions of Python which change the bytecode + format. + + .. versionadded:: 3.3 + + .. attribute:: name + + The name of the module the loader will handle. + + .. attribute:: path + + The path to the bytecode file. + + .. method:: is_package(fullname) + + Determines if the module is a package based on :attr:`path`. + + .. method:: get_code(fullname) + + Returns the code object for :attr:`name` created from :attr:`path`. + + .. method:: get_source(fullname) + + Returns ``None`` as bytecode files have no source when this loader is + used. + + +.. class:: ExtensionFileLoader(fullname, path) + + A concrete implementation of :class:`importlib.abc.InspectLoader` for + extension modules. + + The *fullname* argument specifies the name of the module the loader is to + support. The *path* argument is the path to the extension module's file. + + .. versionadded:: 3.3 + + .. attribute:: name + + Name of the module the loader supports. + + .. attribute:: path + + Path to the extension module. + + .. method:: load_module(fullname) + + Loads the extension module if and only if *fullname* is the same as + :attr:`name` or is ``None``. + + .. method:: is_package(fullname) + + Returns ``True`` if the file path points to a package's ``__init__`` + module based on :attr:`EXTENSION_SUFFIXES`. + + .. method:: get_code(fullname) + + Returns ``None`` as extension modules lack a code object. + + .. method:: get_source(fullname) + + Returns ``None`` as extension modules do not have source code. :mod:`importlib.util` -- Utility code for importers --------------------------------------------------- .. module:: importlib.util - :synopsis: Importers and path hooks + :synopsis: Utility code for importers This module contains the various objects that help in the construction of an :term:`importer`. +.. function:: resolve_name(name, package) + + Resolve a relative module name to an absolute one. + + If **name** has no leading dots, then **name** is simply returned. This + allows for usage such as + ``importlib.util.resolve_name('sys', __package__)`` without doing a + check to see if the **package** argument is needed. + + :exc:`ValueError` is raised if **name** is a relative module name but + package is a false value (e.g. ``None`` or the empty string). + :exc:`ValueError` is also raised a relative name would escape its containing + package (e.g. requesting ``..bacon`` from within the ``spam`` package). + + .. versionadded:: 3.3 + .. decorator:: module_for_loader A :term:`decorator` for a :term:`loader` method, @@ -481,22 +861,30 @@ an :term:`importer`. signature taking two positional arguments (e.g. ``load_module(self, module)``) for which the second argument will be the module **object** to be used by the loader. - Note that the decorator - will not work on static methods because of the assumption of two - arguments. + Note that the decorator will not work on static methods because of the + assumption of two arguments. The decorated method will take in the **name** of the module to be loaded as expected for a :term:`loader`. If the module is not found in :data:`sys.modules` then a new one is constructed with its - :attr:`__name__` attribute set. Otherwise the module found in - :data:`sys.modules` will be passed into the method. If an - exception is raised by the decorated method and a module was added to + :attr:`__name__` attribute set to **name**, :attr:`__loader__` set to + **self**, and :attr:`__package__` set if + :meth:`importlib.abc.InspectLoader.is_package` is defined for **self** and + does not raise :exc:`ImportError` for **name**. If a new module is not + needed then the module found in :data:`sys.modules` will be passed into the + method. + + If an exception is raised by the decorated method and a module was added to :data:`sys.modules` it will be removed to prevent a partially initialized module from being in left in :data:`sys.modules`. If the module was already in :data:`sys.modules` then it is left alone. Use of this decorator handles all the details of which module object a - loader should initialize as specified by :pep:`302`. + loader should initialize as specified by :pep:`302` as best as possible. + + .. versionchanged:: 3.3 + :attr:`__loader__` and :attr:`__package__` are automatically set + (when possible). .. decorator:: set_loader @@ -504,7 +892,13 @@ an :term:`importer`. to set the :attr:`__loader__` attribute on loaded modules. If the attribute is already set the decorator does nothing. It is assumed that the first positional argument to the - wrapped method is what :attr:`__loader__` should be set to. + wrapped method (i.e. ``self``) is what :attr:`__loader__` should be set to. + + .. note:: + + It is recommended that :func:`module_for_loader` be used over this + decorator as it subsumes this functionality. + .. decorator:: set_package @@ -515,8 +909,12 @@ an :term:`importer`. set on and not the module found in :data:`sys.modules`. Reliance on this decorator is discouraged when it is possible to set - :attr:`__package__` before the execution of the code is possible. By - setting it before the code for the module is executed it allows the - attribute to be used at the global level of the module during + :attr:`__package__` before importing. By + setting it beforehand the code for the module is executed with the + attribute set and thus can be used by global level code during initialization. + .. note:: + + It is recommended that :func:`module_for_loader` be used over this + decorator as it subsumes this functionality. |