diff options
Diffstat (limited to 'Doc/reference')
| -rw-r--r-- | Doc/reference/datamodel.rst | 29 | ||||
| -rw-r--r-- | Doc/reference/import_machinery.rst | 547 | ||||
| -rw-r--r-- | Doc/reference/index.rst | 1 | ||||
| -rw-r--r-- | Doc/reference/simple_stmts.rst | 174 |
4 files changed, 595 insertions, 156 deletions
diff --git a/Doc/reference/datamodel.rst b/Doc/reference/datamodel.rst index fa60723..b78997a 100644 --- a/Doc/reference/datamodel.rst +++ b/Doc/reference/datamodel.rst @@ -651,17 +651,19 @@ Modules statement: import object: module - Modules are imported by the :keyword:`import` statement (see section - :ref:`import`). A module object has a - namespace implemented by a dictionary object (this is the dictionary referenced - by the __globals__ attribute of functions defined in the module). Attribute + Modules are a basic organizational unit of Python code, and are created by + the :ref:`importmachinery` as invoked either by the :keyword:`import` + statement (see section :ref:`import`) or by calling the built in + :func:`__import__` function. A module object has a namespace implemented + by a dictionary object (this is the dictionary referenced by the + ``__globals__`` attribute of functions defined in the module). Attribute references are translated to lookups in this dictionary, e.g., ``m.x`` is - equivalent to ``m.__dict__["x"]``. A module object does not contain the code - object used to initialize the module (since it isn't needed once the + equivalent to ``m.__dict__["x"]``. A module object does not contain the + code object used to initialize the module (since it isn't needed once the initialization is done). - Attribute assignment updates the module's namespace dictionary, e.g., ``m.x = - 1`` is equivalent to ``m.__dict__["x"] = 1``. + Attribute assignment updates the module's namespace dictionary, e.g., + ``m.x = 1`` is equivalent to ``m.__dict__["x"] = 1``. .. index:: single: __dict__ (module attribute) @@ -683,11 +685,12 @@ Modules Predefined (writable) attributes: :attr:`__name__` is the module's name; :attr:`__doc__` is the module's documentation string, or ``None`` if - unavailable; :attr:`__file__` is the pathname of the file from which the module - was loaded, if it was loaded from a file. The :attr:`__file__` attribute is not - present for C modules that are statically linked into the interpreter; for - extension modules loaded dynamically from a shared library, it is the pathname - of the shared library file. + unavailable; :attr:`__file__` is the pathname of the file from which the + module was loaded, if it was loaded from a file. The :attr:`__file__` + attribute may be missing for certain types of modules, such as C modules + that are statically linked into the interpreter; for extension modules + loaded dynamically from a shared library, it is the pathname of the shared + library file. Custom classes Custom class types are typically created by class definitions (see section diff --git a/Doc/reference/import_machinery.rst b/Doc/reference/import_machinery.rst new file mode 100644 index 0000000..93d7ba2 --- /dev/null +++ b/Doc/reference/import_machinery.rst @@ -0,0 +1,547 @@ + +.. _importmachinery: + +**************** +Import machinery +**************** + +.. index:: single: import machinery + +Python code in one :term:`module` gains access to the code in another module +by the process of :term:`importing` it. Most commonly, the :keyword:`import` +statement is used to invoke the import machinery, but it can also be invoked +by calling the built-in :func:`__import__` function. + +The :keyword:`import` statement combines two operations; it searches for the +named module, then it binds the results of that search to a name in the local +scope. The search operation of the :keyword:`import` statement is defined as +a call to the :func:`__import__` function, with the appropriate arguments. +The return value of :func:`__import__` is used to perform the name binding +operation of the :keyword:`import` statement. See the :keyword:`import` +statement for the exact details of that name binding operation. + +A direct call to :func:`__import__` performs only the search for the module. +The function's return value is used like any other function call in Python; +there is no special side-effects (e.g. name binding) associated with +:func:`__import__`. + +When a module is first imported, Python searches for the module and if found, +it creates a module object, initializing it. If the named module cannot be +found, an :exc:`ImportError` is raised. Python implements various strategies +to search for the named module when the import machinery is invoked. These +strategies can be modified and extended by using various hooks described in +the sections below. The entire import machinery itself can be overridden by +replacing built-in :func:`__import__`. + + +Packages +======== + +.. index:: + single: package + +Python has only one type of module object, and all modules are of this type, +regardless of whether the module is implemented in Python, C, or something +else. To help organize modules and provide a naming hierarchy, Python has a +concept of :term:`packages <package>`. It's important to keep in mind that +all packages are modules, but not all modules are packages. Or put another +way, packages are just a special kind of module. Although usually +unnecessary, introspection of various module object attributes can determine +whether a module is a package or not. + +Packages can contain other packages and modules, while modules generally do +not contain other modules or packages. You can think of packages as the +directories on a file system and modules as files within directories, but +don't take this analogy too literally since packages and modules need not +originate from the file system. For the purposes of this documentation, we'll +use this convenient analogy of directories and files. + +All modules have a name. Packages also have names, and subpackages can be +nested arbitrarily deeply. Subpackage names are separated from their parent +package by dots, akin to Python's standard attribute access syntax. Thus you +might have a module called :mod:`sys` and a package called :mod:`email`, which +in turn has a subpackage called :mod:`email.mime` and a module within that +subpackage called :mod:`email.mime.text`. + + +Regular packages +---------------- + +.. index:: + pair: package; regular + +Python defines two types of packages, :term:`regular packages <regular +package>` and :term:`namespace packages <namespace package>`. Regular +packages are traditional packages as they existed in Python 3.2 and earlier. +A regular package is typically implemented as a directory containing an +``__init__.py`` file. When a regular package is imported, this +``__init__.py`` file is implicitly imported, and the objects it defines are +bound to names in the package's namespace. The ``__init__.py`` file can +contain the same Python code that any other module can contain, and Python +will add some additional attributes to the module when it is imported. + + +Namespace packages +------------------ + +.. index:: + pair:: package; namespace + pair:: package; portion + +A namespace package is a composite of various :term:`portions <portion>`, +where each portion contributes a subpackage to the parent package. Portions +may reside in different locations on the file system. Portions may also be +found in zip files, on the network, or anywhere else that Python searches +during import. Namespace packages may or may not correspond directly to +objects on the file system; they may be virtual modules that have no concrete +representation. + +For example, the following file system layout defines a top level ``parent`` +package with three subpackages:: + + parent/ + __init__.py + one/ + __init__.py + two/ + __init__.py + three/ + __init__.py + +Importing ``parent.one`` will implicitly import ``parent/__init__.py`` and +``parent/one/__init__.py``. Subsequent imports of ``parent.two`` or +``parent.three`` will import ``parent/two/__init__.py`` and +``parent/three/__init__.py`` respectively. + +With namespace packages, there is no ``parent/__init__.py`` file. In fact, +there may be multiple ``parent`` directories found during import search, where +each one is provided by a separate vendor installed container, and none of +them contain an ``__init__.py`` file. Thus ``parent/one`` may not be +physically located next to ``parent/two``. In this case, Python will create a +namespace package for the top-level ``parent`` package whenever it or one of +its subpackages is imported. + + +Searching +========= + +To begin the search, Python needs the :term:`fully qualified <qualified name>` +name of the module (or package, but for the purposes of this discussion, the +difference is immaterial) being imported. This name may come from various +arguments to the :keyword:`import` statement, or from the parameters to the +:func:`__import__` function. + +This name will be used in various phases of the import search, and it may be +the dotted path to a submodule, e.g. ``foo.bar.baz``. In this case, Python +first tries to import ``foo``, then ``foo.bar``, and finally ``foo.bar.baz``. +If any of the intermediate imports fail, an :exc:`ImportError` is raised. + + +The module cache +---------------- + +.. index:: + single: sys.modules + +The first place checked during import search is :data:`sys.modules`. This +mapping serves as a cache of all modules that have been previously imported, +including the intermediate paths. So if ``foo.bar.baz`` was previously +imported, :data:`sys.modules` will contain entries for ``foo``, ``foo.bar``, +and ``foo.bar.baz``. Each key will have as its value the corresponding module +object. + +During import, the module name is looked up in :data:`sys.modules` and if +present, the associated value is the module satisfying the import, and the +process completes. However, if the value is ``None``, then an +:exc:`ImportError` is raised. If the module name is missing, Python will +continue searching for the module. + +:data:`sys.modules` is writable. Deleting a key will generally not destroy +the associated module, but it will invalidate the cache entry for the named +module, causing Python to search anew for the named module upon its next +import. Beware though, because if you keep a reference to the module object, +invalidate its cache entry in :data:`sys.modules`, and then re-import the +named module, the two module objects will *not* be the same. The key can also +be assigned to ``None``, forcing the next import of the module to result in an +:exc:`ImportError`. + + +Finders and loaders +------------------- + +.. index:: + single: finder + single: loader + +If the named module is not found in :data:`sys.modules` then Python's import +protocol is invoked to find and load the module. As this implies, the import +protocol consists of two conceptual objects, :term:`finders <finder>` and +:term:`loaders <loader>`. A finder's job is to determine whether it can find +the named module using whatever strategy it knows about. For example, there +is a file system finder which know how to search the file system for the named +module. Other finders may know how to search a zip file, a web page, or a +database to find the named module. The import machinery is extensible, so new +finders can be added to extend the range and scope of module searching. + +Finders do not actually load modules. If they can find the named module, they +return a loader, which the import machinery later invokes to load the module +and create the corresponding module object. + +There are actually two types of finders, and two different but related APIs +for finders, depending on whether it is a :term:`meta path finder` or a +:term:`sys path finder`. Meta path processing occurs at the beginning of +import processing, while sys path processing happens later, by the :term:`path +importer`. + +The following sections describe the protocol for finders and loaders in more +detail, including how you can create and register new ones to extend the +import machinery. + + +Import hooks +------------ + +.. index:: + single: import hooks + single: meta hooks + single: path hooks + pair: hooks; import + pair: hooks; meta + pair: hooks; path + +The import machinery is designed to be extensible; the primary mechanism for +this are the *import hooks*. There are two types of import hooks: *meta +hooks* and *path hooks*. + +Meta hooks are called at the start of import processing, before any other +import processing has occurred. This allows meta hooks to override +:data:`sys.path` processing, frozen modules, or even built-in modules. Meta +hooks are registered by adding new finder objects to :data:`sys.meta_path`, as +described below. + +Path hooks are called as part of :data:`sys.path` (or ``package.__path__``) +processing, at the point where their associated path item is encountered. +Path hooks are registered by adding new callables to :data:`sys.path_hooks` as +described below. + + +The meta path +------------- + +.. index:: + single: sys.meta_path + pair: finder; find_module + pair: finder; find_loader + +When the named module is not found in :data:`sys.modules`, Python next +searches :data:`sys.meta_path`, which contains a list of meta path finder +objects. These finders are queried in order to see if they know how to handle +the named module. Meta path finders must implement a method called +:meth:`find_module()` which takes two arguments, a name and a path. The meta +path finder can use any strategy it wants to determine whether it can handle +the named module or not. + +If the meta path finder knows how to handle the named module, it returns a +loader object. If it cannot handle the named module, it returns ``None``. If +:data:`sys.meta_path` processing reaches the end of its list without returning +a loader, then an :exc:`ImportError` is raised. Any other exceptions raised +are simply propagated up, aborting the import process. + +The :meth:`find_module()` method of meta path finders is called with two +arguments. The first is the fully qualified name of the module being +imported, for example ``foo.bar.baz``. The second argument is the relative +path for the module search. For top-level modules, the second argument is +``None``, but for submodules or subpackages, the second argument is the value +of the parent package's ``__path__`` attribute, which must exist or an +:exc:`ImportError` is raised. + +Python's default :data:`sys.meta_path` has three meta path finders, one that +knows how to import built-in modules, one that knows how to import frozen +modules, and one that knows how to import modules from the file system +(i.e. the :term:`path importer`). + + +Meta path loaders +----------------- + +Once a loader is found via a meta path finder, the loader's +:meth:`load_module()` method is called, with a single argument, the fully +qualified name of the module being imported. This method has several +responsibilities, and should return the module object it has loaded [#fn1]_. +If it cannot load the module, it should raise an :exc:`ImportError`, although +any other exception raised during :meth:`load_module()` will be propagated. + +In many cases, the meta path finder and loader can be the same object, +e.g. :meth:`finder.find_module()` would just return ``self``. + +Loaders must satisfy the following requirements: + + * If there is an existing module object with the given name in + :data:`sys.modules`, the loader must use that existing module. (Otherwise, + the :func:`reload()` builtin will not work correctly.) If the named module + does not exist in :data:`sys.modules`, the loader must create a new module + object and add it to :data:`sys.modules`. + + Note that the module *must* exist in :data:`sys.modules` before the loader + executes the module code. This is crucial because the module code may + (directly or indirectly) import itself; adding it to :data:`sys.modules` + beforehand prevents unbounded recursion in the worst case and multiple + loading in the best. + + If the load fails, the loader needs to remove any modules it may have + inserted into ``sys.modules``. If the module was already in + ``sys.modules`` then the loader should leave it alone. + + * The loader may set the ``__file__`` attribute of the module. If set, this + attribute's value must be a string. The loader may opt to leave + ``__file__`` unset if it has no semantic meaning (e.g. a module loaded from + a database). + + * The loader may set the ``__name__`` attribute of the module. While not + required, setting this attribute is highly recommended so that the + :meth:`repr()` of the module is more informative. + + * If module is a package (either regular or namespace), the loader must set + the module object's ``__path__`` attribute. The value must be a list, but + may be empty if ``__path__`` has no further significance to the importer. + More details on the semantics of ``__path__`` are given below. + + * The ``__loader__`` attribute must be set to the loader object that loaded + the module. This is mostly for introspection and reloading, but can be + used for additional importer-specific functionality, for example getting + data associated with an importer. + + * The module's ``__package__`` attribute should be set. Its value must be a + string, but it can be the same value as its ``__name__``. This is the + recommendation when the module is a package. When the module is not a + package, ``__package__`` should be set to the parent package's name. + + This attribute is used instead of ``__name__`` to calculate explicit + relative imports for main modules, as defined in :pep:`366`. + + * If the module is a Python module (as opposed to a built-in module or a + dynamically loaded extension), it should execute the module's code in the + module's global name space (``module.__dict__``). + + +Module reprs +------------ + +By default, all modules have a usable repr, however depending on the +attributes set above, and hooks in the loader, you can more tightly control +the repr of module objects. + +Loaders may implement a :meth:`module_repr()` method which takes a single +argument, the module object. When ``repr(module)`` is called for a module +with a loader supporting this protocol, whatever is returned from +``loader.module_repr(module)`` is returned as the module's repr without +further processing. This return value must be a string. + +If the module has no ``__loader__`` attribute, or the loader has no +:meth:`module_repr()` method, then the module object implementation itself +will craft a default repr using whatever information is available. It will +try to use the ``module.__name__``, ``module.__file__``, and +``module.__loader__`` as input into the repr, with defaults for whatever +information is missing. + +Here are the exact rules used: + + * If the module has an ``__loader__`` and that loader has a + :meth:`module_repr()` method, call it with a single argument, which is the + module object. The value returned is used as the module's repr. + + * If an exception occurs in :meth:`module_repr()`, the exception is caught + and discarded, and the calculation of the module's repr continues as if + :meth:`module_repr()` did not exist. + + * If the module has an ``__file__`` attribute, this is used as part of the + module's repr. + + * If the module has no ``__file__`` but does have an ``__loader__``, then the + loader's repr is used as part of the module's repr. + + * Otherwise, just use the module's ``__name__`` in the repr. + +This example, from :pep:`420` shows how a loader can craft its own module +repr:: + + class NamespaceLoader: + @classmethod + def module_repr(cls, module): + return "<module '{}' (namespace)>".format(module.__name__) + + +module.__path__ +--------------- + +By definition, if a module has an ``__path__`` attribute, it is a package, +regardless of its value. + +A package's ``__path__`` attribute is used during imports of its subpackages. +Within the import machinery, it functions much the same as :data:`sys.path`, +i.e. providing a list of locations to search for modules during import. +However, ``__path__`` is typically much more constrained than +:data:`sys.path`. + +``__path__`` must be a list, but it may be empty. The same rules used for +:data:`sys.path` also apply to a package's ``__path__``, and +:data:`sys.path_hooks` (described below) are consulted when traversing a +package's ``__path__``. + +A package's ``__init__.py`` file may set or alter the package's ``__path__`` +attribute, and this was typically the way namespace packages were implemented +prior to :pep:`420`. With the adoption of :pep:`420`, namespace packages no +longer need to supply ``__init__.py`` files containing only ``__path__`` +manipulation code; the namespace loader automatically sets ``__path__`` +correctly for the namespace package. + + +The Path Importer +================= + +.. index:: + single: path importer + +As mentioned previously, Python comes with several default meta path finders. +One of these, called the :term:`path importer`, knows how to provide +traditional file system imports. It implements all the semantics for finding +modules on the file system, handling special file types such as Python source +code (``.py`` files), Python byte code (``.pyc`` and ``.pyo`` files) and +shared libraries (e.g. ``.so`` files). + +In addition to being able to find such modules, there is built-in support for +loading these modules. To accomplish these two related tasks, additional +hooks and protocols are provided so that you can extend and customize the path +importer semantics. + +A word of warning: this section and the previous both use the term *finder*, +distinguishing between them by using the terms :term:`meta path finder` and +:term:`sys path finder`. Meta path finders and sys path finders are very +similar, support similar protocols, and function in similar ways during the +import process, but it's important to keep in mind that they are subtly +different. In particular, meta path finders operate at the beginning of the +import process, as keyed off the :data:`sys.meta_path` traversal. + +On the other hand, sys path finders are in a sense an implementation detail of +the path importer, and in fact, if the path importer were to be removed from +:data:`sys.meta_path`, none of the sys path finder semantics would be invoked. + + +sys path finders +---------------- + +.. index:: + single: sys.path + single: sys.path_hooks + single: sys.path_importer_cache + single: PYTHONPATH + +The path importer is responsible for finding and loading Python modules and +packages from the file system. As a meta path finder, it implements the +:meth:`find_module()` protocol previously described, however it exposes +additional hooks that can be used to customize how modules are found and +loaded from the file system. + +Three variables are used during file system import, :data:`sys.path`, +:data:`sys.path_hooks` and :data:`sys.path_importer_cache`. These provide +additional ways that the import machinery can be customized, in this case +specifically during file system path import. + +:data:`sys.path` contains a list of strings providing search locations for +modules and packages. It is initialized from the :data:`PYTHONPATH` +environment variable and various other installation- and +implementation-specific defaults. Entries in :data:`sys.path` can name +directories on the file system, zip files, and potentially other "locations" +that should be searched for modules. + +The path importer is a meta path finder, so the import machinery begins file +system search by calling the path importer's :meth:`find_module()` method as +described previously. When the ``path`` argument to :meth:`find_module()` is +given, it will be a list of string paths to traverse. If not, +:data:`sys.path` is used. + +The path importer iterates over every entry in the search path, and for each +of these, searches for an appropriate sys path finder for the path entry. +Because this can be an expensive operation (e.g. there are `stat()` call +overheads for this search), the path importer maintains a cache mapping path +entries to sys path finders. This cache is maintained in +:data:`sys.path_importer_cache`. In this way, the expensive search for a +particular path location's sys path finder need only be done once. User code +is free to remove cache entries from :data:`sys.path_importer_cache` forcing +the path importer to perform the path search again. + +If the path entry is not present in the cache, the path importer iterates over +every callable in :data:`sys.path_hooks`. Each entry in this list is called +with a single argument, the path entry being searched. This callable may +either return a sys path finder that can handle the path entry, or it may +raise :exc:`ImportError`. An :exc:`ImportError` is used by the path importer +to signal that the hook cannot find a sys path finder for that path entry. +The exception is ignored and :data:`sys.path_hooks` iteration continues. + +If :data:`sys.path_hooks` iteration ends with no sys path finder being +returned then the path importer's :meth:`find_module()` method will return +``None`` and an :exc:`ImportError` will be raised. + +If a sys path finder *is* returned by one of the callables on +:data:`sys.path_hooks`, then the following protocol is used to ask the sys +path finder for a module loader. If a loader results from this step, it is +used to load the module as previously described (i.e. its +:meth:`load_module()` method is called). + + +sys path finder protocol +------------------------ + +sys path finders support the same, traditional :meth:`find_module()` method +that meta path finders support, however sys path finder :meth:`find_module()` +methods are never called with a ``path`` argument. + +The :meth:`find_module()` method on sys path finders is deprecated though, and +instead sys path finders should implement the :meth:`find_loader()` method. +If it exists on the sys path finder, :meth:`find_loader()` will always be +called instead of :meth:`find_module()`. + +:meth:`find_loader()` takes one argument, the fully qualified name of the +module being imported. :meth:`find_loader()` returns a 2-tuple where the +first item is the loader and the second item is a namespace :term:`portion`. +When the first item (i.e. the loader) is ``None``, this means that while the +sys path finder does not have a loader for the named module, it knows that the +path entry contributes to a namespace portion for the named module. This will +almost always be the case where Python is asked to import a namespace package +that has no physical presence on the file system. When a sys path finder +returns ``None`` for the loader, the second item of the 2-tuple return value +must be a sequence, although it can be empty. + +If :meth:`find_loader()` returns a non-``None`` loader value, the portion is +ignored and the loader is returned from the path importer, terminating the +:data:`sys.path` search. + + +Open issues +=========== + +XXX What to say about `imp.NullImporter` when it's found in +:data:`sys.path_importer_cache`? + +XXX It would be really nice to have a diagram. + +.. [#fn1] The importlib implementation appears not to use the return value + directly. Instead, it gets the module object by looking the module name up + in ``sys.modules``.) + + +References +========== + +The import machinery has evolved considerably since Python's early days. The +original `specification for packages +<http://www.python.org/doc/essays/packages.html>`_ is still available to read, +although some details have changed since the writing of that document. + +The original specification for :data:`sys.meta_path` was :pep:`302`, with +subsequent extension in :pep:`420`, which also introduced namespace packages +without ``__init__.py`` files in Python 3.3. :pep:`420` also introduced the +:meth:`find_loader` protocol as an alternative to :meth:`find_module`. + +:pep:`366` describes the addition of the ``__package__`` attribute for +explicit relative imports in main modules. diff --git a/Doc/reference/index.rst b/Doc/reference/index.rst index bd1a281..87166f8 100644 --- a/Doc/reference/index.rst +++ b/Doc/reference/index.rst @@ -24,6 +24,7 @@ interfaces available to C/C++ programmers in detail. lexical_analysis.rst datamodel.rst executionmodel.rst + import_machinery.rst expressions.rst simple_stmts.rst compound_stmts.rst diff --git a/Doc/reference/simple_stmts.rst b/Doc/reference/simple_stmts.rst index ce7ce92..f93f497 100644 --- a/Doc/reference/simple_stmts.rst +++ b/Doc/reference/simple_stmts.rst @@ -660,162 +660,50 @@ The :keyword:`import` statement relative_module: "."* `module` | "."+ name: `identifier` -Import statements are executed in two steps: (1) find a module, and initialize -it if necessary; (2) define a name or names in the local namespace (of the scope -where the :keyword:`import` statement occurs). The statement comes in two -forms differing on whether it uses the :keyword:`from` keyword. The first form -(without :keyword:`from`) repeats these steps for each identifier in the list. -The form with :keyword:`from` performs step (1) once, and then performs step -(2) repeatedly. For a reference implementation of step (1), see the -:mod:`importlib` module. - -.. index:: - single: package - -To understand how step (1) occurs, one must first understand how Python handles -hierarchical naming of modules. To help organize modules and provide a -hierarchy in naming, Python has a concept of packages. A package can contain -other packages and modules while modules cannot contain other modules or -packages. From a file system perspective, packages are directories and modules -are files. The original `specification for packages -<http://www.python.org/doc/essays/packages.html>`_ is still available to read, -although minor details have changed since the writing of that document. - -.. index:: - single: sys.modules - -Once the name of the module is known (unless otherwise specified, the term -"module" will refer to both packages and modules), searching -for the module or package can begin. The first place checked is -:data:`sys.modules`, the cache of all modules that have been imported -previously. If the module is found there then it is used in step (2) of import -unless ``None`` is found in :data:`sys.modules`, in which case -:exc:`ImportError` is raised. - -.. index:: - single: sys.meta_path - single: finder - pair: finder; find_module - single: __path__ - -If the module is not found in the cache, then :data:`sys.meta_path` is searched -(the specification for :data:`sys.meta_path` can be found in :pep:`302`). -The object is a list of :term:`finder` objects which are queried in order as to -whether they know how to load the module by calling their :meth:`find_module` -method with the name of the module. If the module happens to be contained -within a package (as denoted by the existence of a dot in the name), then a -second argument to :meth:`find_module` is given as the value of the -:attr:`__path__` attribute from the parent package (everything up to the last -dot in the name of the module being imported). If a finder can find the module -it returns a :term:`loader` (discussed later) or returns ``None``. - -.. index:: - single: sys.path_hooks - single: sys.path_importer_cache - single: sys.path - -If none of the finders on :data:`sys.meta_path` are able to find the module -then some implicitly defined finders are queried. Implementations of Python -vary in what implicit meta path finders are defined. The one they all do -define, though, is one that handles :data:`sys.path_hooks`, -:data:`sys.path_importer_cache`, and :data:`sys.path`. - -The implicit finder searches for the requested module in the "paths" specified -in one of two places ("paths" do not have to be file system paths). If the -module being imported is supposed to be contained within a package then the -second argument passed to :meth:`find_module`, :attr:`__path__` on the parent -package, is used as the source of paths. If the module is not contained in a -package then :data:`sys.path` is used as the source of paths. - -Once the source of paths is chosen it is iterated over to find a finder that -can handle that path. The dict at :data:`sys.path_importer_cache` caches -finders for paths and is checked for a finder. If the path does not have a -finder cached then :data:`sys.path_hooks` is searched by calling each object in -the list with a single argument of the path, returning a finder or raises -:exc:`ImportError`. If a finder is returned then it is cached in -:data:`sys.path_importer_cache` and then used for that path entry. If no finder -can be found but the path exists then a value of ``None`` is -stored in :data:`sys.path_importer_cache` to signify that an implicit, -file-based finder that handles modules stored as individual files should be -used for that path. If the path does not exist then a finder which always -returns ``None`` is placed in the cache for the path. - -.. index:: - single: loader - pair: loader; load_module - exception: ImportError - -If no finder can find the module then :exc:`ImportError` is raised. Otherwise -some finder returned a loader whose :meth:`load_module` method is called with -the name of the module to load (see :pep:`302` for the original definition of -loaders). A loader has several responsibilities to perform on a module it -loads. First, if the module already exists in :data:`sys.modules` (a -possibility if the loader is called outside of the import machinery) then it -is to use that module for initialization and not a new module. But if the -module does not exist in :data:`sys.modules` then it is to be added to that -dict before initialization begins. If an error occurs during loading of the -module and it was added to :data:`sys.modules` it is to be removed from the -dict. If an error occurs but the module was already in :data:`sys.modules` it -is left in the dict. - -.. index:: - single: __name__ - single: __file__ - single: __path__ - single: __package__ - single: __loader__ - -The loader must set several attributes on the module. :data:`__name__` is to be -set to the name of the module. :data:`__file__` is to be the "path" to the file -unless the module is built-in (and thus listed in -:data:`sys.builtin_module_names`) in which case the attribute is not set. -If what is being imported is a package then :data:`__path__` is to be set to a -list of paths to be searched when looking for modules and packages contained -within the package being imported. :data:`__package__` is optional but should -be set to the name of package that contains the module or package (the empty -string is used for module not contained in a package). :data:`__loader__` is -also optional but should be set to the loader object that is loading the -module. While loaders are required to return the module they loaded, import -itself always retrieves any modules it returns from :data:`sys.modules`. - -.. index:: - exception: ImportError - -If an error occurs during loading then the loader raises :exc:`ImportError` if -some other exception is not already being propagated. Otherwise the loader -returns the module that was loaded and initialized. +Import statements are executed in two steps: (1) find a module, loading and +initializing it if necessary; (2) define a name or names in the local +namespace (of the scope where the :keyword:`import` statement occurs). The +statement comes in two forms differing on whether it uses the :keyword:`from` +keyword. The first form (without :keyword:`from`) repeats these steps for each +identifier in the list. The form with :keyword:`from` performs step (1) once, +and then performs step (2) repeatedly. + +The details of step (1), finding and loading modules is described in greater +detail in the section on the :ref:`import machinery <importmachinery>`, which +also describes the various types of packages and modules that can be imported, +as well as all the hooks that can be used to customize Python's import. When step (1) finishes without raising an exception, step (2) can begin. -The first form of :keyword:`import` statement binds the module name in the local -namespace to the module object, and then goes on to import the next identifier, -if any. If the module name is followed by :keyword:`as`, the name following -:keyword:`as` is used as the local name for the module. +The first form of :keyword:`import` statement binds the module name in the +local namespace to the module object, and then goes on to import the next +identifier, if any. If the module name is followed by :keyword:`as`, the name +following :keyword:`as` is used as the local name for the module. .. index:: pair: name; binding exception: ImportError -The :keyword:`from` form does not bind the module name: it goes through the list -of identifiers, looks each one of them up in the module found in step (1), and -binds the name in the local namespace to the object thus found. As with the -first form of :keyword:`import`, an alternate local name can be supplied by -specifying ":keyword:`as` localname". If a name is not found, -:exc:`ImportError` is raised. If the list of identifiers is replaced by a star -(``'*'``), all public names defined in the module are bound in the local +The :keyword:`from` form does not bind the module name: it goes through the +list of identifiers, looks each one of them up in the module found in step +(1), and binds the name in the local namespace to the object thus found. As +with the first form of :keyword:`import`, an alternate local name can be +supplied by specifying ":keyword:`as` localname". If a name is not found, +:exc:`ImportError` is raised. If the list of identifiers is replaced by a +star (``'*'``), all public names defined in the module are bound in the local namespace of the :keyword:`import` statement. .. index:: single: __all__ (optional module attribute) The *public names* defined by a module are determined by checking the module's -namespace for a variable named ``__all__``; if defined, it must be a sequence of -strings which are names defined or imported by that module. The names given in -``__all__`` are all considered public and are required to exist. If ``__all__`` -is not defined, the set of public names includes all names found in the module's -namespace which do not begin with an underscore character (``'_'``). -``__all__`` should contain the entire public API. It is intended to avoid -accidentally exporting items that are not part of the API (such as library -modules which were imported and used within the module). +namespace for a variable named ``__all__``; if defined, it must be a sequence +of strings which are names defined or imported by that module. The names +given in ``__all__`` are all considered public and are required to exist. If +``__all__`` is not defined, the set of public names includes all names found +in the module's namespace which do not begin with an underscore character +(``'_'``). ``__all__`` should contain the entire public API. It is intended +to avoid accidentally exporting items that are not part of the API (such as +library modules which were imported and used within the module). The :keyword:`from` form with ``*`` may only occur in a module scope. The wild card form of import --- ``import *`` --- is only allowed at the module level. |