Integration of importdocs from the features/pep-420 repo.

4 files changed, 595 insertions, 156 deletions

diff --git a/Doc/reference/datamodel.rst b/Doc/reference/datamodel.rst
index fa60723..b78997a 100644
--- a/Doc/reference/datamodel.rst
+++ b/Doc/reference/datamodel.rst
@@ -651,17 +651,19 @@ Modules
       statement: import
       object: module
 
-   Modules are imported by the :keyword:`import` statement (see section
-   :ref:`import`). A module object has a
-   namespace implemented by a dictionary object (this is the dictionary referenced
-   by the __globals__ attribute of functions defined in the module).  Attribute
+   Modules are a basic organizational unit of Python code, and are created by
+   the :ref:`importmachinery` as invoked either by the :keyword:`import`
+   statement (see section :ref:`import`) or by calling the built in
+   :func:`__import__` function.  A module object has a namespace implemented
+   by a dictionary object (this is the dictionary referenced by the
+   ``__globals__`` attribute of functions defined in the module).  Attribute
    references are translated to lookups in this dictionary, e.g., ``m.x`` is
-   equivalent to ``m.__dict__["x"]``. A module object does not contain the code
-   object used to initialize the module (since it isn't needed once the
+   equivalent to ``m.__dict__["x"]``. A module object does not contain the
+   code object used to initialize the module (since it isn't needed once the
    initialization is done).
 
-   Attribute assignment updates the module's namespace dictionary, e.g., ``m.x =
-   1`` is equivalent to ``m.__dict__["x"] = 1``.
+   Attribute assignment updates the module's namespace dictionary, e.g.,
+   ``m.x = 1`` is equivalent to ``m.__dict__["x"] = 1``.
 
    .. index:: single: __dict__ (module attribute)
 
@@ -683,11 +685,12 @@ Modules
 
    Predefined (writable) attributes: :attr:`__name__` is the module's name;
    :attr:`__doc__` is the module's documentation string, or ``None`` if
-   unavailable; :attr:`__file__` is the pathname of the file from which the module
-   was loaded, if it was loaded from a file. The :attr:`__file__` attribute is not
-   present for C modules that are statically linked into the interpreter; for
-   extension modules loaded dynamically from a shared library, it is the pathname
-   of the shared library file.
+   unavailable; :attr:`__file__` is the pathname of the file from which the
+   module was loaded, if it was loaded from a file. The :attr:`__file__`
+   attribute may be missing for certain types of modules, such as C modules
+   that are statically linked into the interpreter; for extension modules
+   loaded dynamically from a shared library, it is the pathname of the shared
+   library file.
 
 Custom classes
    Custom class types are typically created by class definitions (see section
diff --git a/Doc/reference/import_machinery.rst b/Doc/reference/import_machinery.rst
new file mode 100644
index 0000000..93d7ba2
--- /dev/null
+++ b/Doc/reference/import_machinery.rst
@@ -0,0 +1,547 @@
+
+.. _importmachinery:
+
+****************
+Import machinery
+****************
+
+.. index:: single: import machinery
+
+Python code in one :term:`module` gains access to the code in another module
+by the process of :term:`importing` it.  Most commonly, the :keyword:`import`
+statement is used to invoke the import machinery, but it can also be invoked
+by calling the built-in :func:`__import__` function.
+
+The :keyword:`import` statement combines two operations; it searches for the
+named module, then it binds the results of that search to a name in the local
+scope.  The search operation of the :keyword:`import` statement is defined as
+a call to the :func:`__import__` function, with the appropriate arguments.
+The return value of :func:`__import__` is used to perform the name binding
+operation of the :keyword:`import` statement.  See the :keyword:`import`
+statement for the exact details of that name binding operation.
+
+A direct call to :func:`__import__` performs only the search for the module.
+The function's return value is used like any other function call in Python;
+there is no special side-effects (e.g. name binding) associated with
+:func:`__import__`.
+
+When a module is first imported, Python searches for the module and if found,
+it creates a module object, initializing it.  If the named module cannot be
+found, an :exc:`ImportError` is raised.  Python implements various strategies
+to search for the named module when the import machinery is invoked.  These
+strategies can be modified and extended by using various hooks described in
+the sections below.  The entire import machinery itself can be overridden by
+replacing built-in :func:`__import__`.
+
+
+Packages
+========
+
+.. index::
+    single: package
+
+Python has only one type of module object, and all modules are of this type,
+regardless of whether the module is implemented in Python, C, or something
+else.  To help organize modules and provide a naming hierarchy, Python has a
+concept of :term:`packages <package>`.  It's important to keep in mind that
+all packages are modules, but not all modules are packages.  Or put another
+way, packages are just a special kind of module.  Although usually
+unnecessary, introspection of various module object attributes can determine
+whether a module is a package or not.
+
+Packages can contain other packages and modules, while modules generally do
+not contain other modules or packages.  You can think of packages as the
+directories on a file system and modules as files within directories, but
+don't take this analogy too literally since packages and modules need not
+originate from the file system.  For the purposes of this documentation, we'll
+use this convenient analogy of directories and files.
+
+All modules have a name.  Packages also have names, and subpackages can be
+nested arbitrarily deeply.  Subpackage names are separated from their parent
+package by dots, akin to Python's standard attribute access syntax.  Thus you
+might have a module called :mod:`sys` and a package called :mod:`email`, which
+in turn has a subpackage called :mod:`email.mime` and a module within that
+subpackage called :mod:`email.mime.text`.
+
+
+Regular packages
+----------------
+
+.. index::
+    pair: package; regular
+
+Python defines two types of packages, :term:`regular packages <regular
+package>` and :term:`namespace packages <namespace package>`.  Regular
+packages are traditional packages as they existed in Python 3.2 and earlier.
+A regular package is typically implemented as a directory containing an
+``__init__.py`` file.  When a regular package is imported, this
+``__init__.py`` file is implicitly imported, and the objects it defines are
+bound to names in the package's namespace.  The ``__init__.py`` file can
+contain the same Python code that any other module can contain, and Python
+will add some additional attributes to the module when it is imported.
+
+
+Namespace packages
+------------------
+
+.. index::
+    pair:: package; namespace
+    pair:: package; portion
+
+A namespace package is a composite of various :term:`portions <portion>`,
+where each portion contributes a subpackage to the parent package.  Portions
+may reside in different locations on the file system.  Portions may also be
+found in zip files, on the network, or anywhere else that Python searches
+during import.  Namespace packages may or may not correspond directly to
+objects on the file system; they may be virtual modules that have no concrete
+representation.
+
+For example, the following file system layout defines a top level ``parent``
+package with three subpackages::
+
+    parent/
+        __init__.py
+        one/
+            __init__.py
+        two/
+            __init__.py
+        three/
+            __init__.py
+
+Importing ``parent.one`` will implicitly import ``parent/__init__.py`` and
+``parent/one/__init__.py``.  Subsequent imports of ``parent.two`` or
+``parent.three`` will import ``parent/two/__init__.py`` and
+``parent/three/__init__.py`` respectively.
+
+With namespace packages, there is no ``parent/__init__.py`` file.  In fact,
+there may be multiple ``parent`` directories found during import search, where
+each one is provided by a separate vendor installed container, and none of
+them contain an ``__init__.py`` file.  Thus ``parent/one`` may not be
+physically located next to ``parent/two``.  In this case, Python will create a
+namespace package for the top-level ``parent`` package whenever it or one of
+its subpackages is imported.
+
+
+Searching
+=========
+
+To begin the search, Python needs the :term:`fully qualified <qualified name>`
+name of the module (or package, but for the purposes of this discussion, the
+difference is immaterial) being imported.  This name may come from various
+arguments to the :keyword:`import` statement, or from the parameters to the
+:func:`__import__` function.
+
+This name will be used in various phases of the import search, and it may be
+the dotted path to a submodule, e.g. ``foo.bar.baz``.  In this case, Python
+first tries to import ``foo``, then ``foo.bar``, and finally ``foo.bar.baz``.
+If any of the intermediate imports fail, an :exc:`ImportError` is raised.
+
+
+The module cache
+----------------
+
+.. index::
+    single: sys.modules
+
+The first place checked during import search is :data:`sys.modules`.  This
+mapping serves as a cache of all modules that have been previously imported,
+including the intermediate paths.  So if ``foo.bar.baz`` was previously
+imported, :data:`sys.modules` will contain entries for ``foo``, ``foo.bar``,
+and ``foo.bar.baz``.  Each key will have as its value the corresponding module
+object.
+
+During import, the module name is looked up in :data:`sys.modules` and if
+present, the associated value is the module satisfying the import, and the
+process completes.  However, if the value is ``None``, then an
+:exc:`ImportError` is raised.  If the module name is missing, Python will
+continue searching for the module.
+
+:data:`sys.modules` is writable.  Deleting a key will generally not destroy
+the associated module, but it will invalidate the cache entry for the named
+module, causing Python to search anew for the named module upon its next
+import.  Beware though, because if you keep a reference to the module object,
+invalidate its cache entry in :data:`sys.modules`, and then re-import the
+named module, the two module objects will *not* be the same.  The key can also
+be assigned to ``None``, forcing the next import of the module to result in an
+:exc:`ImportError`.
+
+
+Finders and loaders
+-------------------
+
+.. index::
+    single: finder
+    single: loader
+
+If the named module is not found in :data:`sys.modules` then Python's import
+protocol is invoked to find and load the module.  As this implies, the import
+protocol consists of two conceptual objects, :term:`finders <finder>` and
+:term:`loaders <loader>`.  A finder's job is to determine whether it can find
+the named module using whatever strategy it knows about.  For example, there
+is a file system finder which know how to search the file system for the named
+module.  Other finders may know how to search a zip file, a web page, or a
+database to find the named module.  The import machinery is extensible, so new
+finders can be added to extend the range and scope of module searching.
+
+Finders do not actually load modules.  If they can find the named module, they
+return a loader, which the import machinery later invokes to load the module
+and create the corresponding module object.
+
+There are actually two types of finders, and two different but related APIs
+for finders, depending on whether it is a :term:`meta path finder` or a
+:term:`sys path finder`.  Meta path processing occurs at the beginning of
+import processing, while sys path processing happens later, by the :term:`path
+importer`.
+
+The following sections describe the protocol for finders and loaders in more
+detail, including how you can create and register new ones to extend the
+import machinery.
+
+
+Import hooks
+------------
+
+.. index::
+   single: import hooks
+   single: meta hooks
+   single: path hooks
+   pair: hooks; import
+   pair: hooks; meta
+   pair: hooks; path
+
+The import machinery is designed to be extensible; the primary mechanism for
+this are the *import hooks*.  There are two types of import hooks: *meta
+hooks* and *path hooks*.
+
+Meta hooks are called at the start of import processing, before any other
+import processing has occurred.  This allows meta hooks to override
+:data:`sys.path` processing, frozen modules, or even built-in modules.  Meta
+hooks are registered by adding new finder objects to :data:`sys.meta_path`, as
+described below.
+
+Path hooks are called as part of :data:`sys.path` (or ``package.__path__``)
+processing, at the point where their associated path item is encountered.
+Path hooks are registered by adding new callables to :data:`sys.path_hooks` as
+described below.
+
+
+The meta path
+-------------
+
+.. index::
+    single: sys.meta_path
+    pair: finder; find_module
+    pair: finder; find_loader
+
+When the named module is not found in :data:`sys.modules`, Python next
+searches :data:`sys.meta_path`, which contains a list of meta path finder
+objects.  These finders are queried in order to see if they know how to handle
+the named module.  Meta path finders must implement a method called
+:meth:`find_module()` which takes two arguments, a name and a path.  The meta
+path finder can use any strategy it wants to determine whether it can handle
+the named module or not.
+
+If the meta path finder knows how to handle the named module, it returns a
+loader object.  If it cannot handle the named module, it returns ``None``.  If
+:data:`sys.meta_path` processing reaches the end of its list without returning
+a loader, then an :exc:`ImportError` is raised.  Any other exceptions raised
+are simply propagated up, aborting the import process.
+
+The :meth:`find_module()` method of meta path finders is called with two
+arguments.  The first is the fully qualified name of the module being
+imported, for example ``foo.bar.baz``.  The second argument is the relative
+path for the module search.  For top-level modules, the second argument is
+``None``, but for submodules or subpackages, the second argument is the value
+of the parent package's ``__path__`` attribute, which must exist or an
+:exc:`ImportError` is raised.
+
+Python's default :data:`sys.meta_path` has three meta path finders, one that
+knows how to import built-in modules, one that knows how to import frozen
+modules, and one that knows how to import modules from the file system
+(i.e. the :term:`path importer`).
+
+
+Meta path loaders
+-----------------
+
+Once a loader is found via a meta path finder, the loader's
+:meth:`load_module()` method is called, with a single argument, the fully
+qualified name of the module being imported.  This method has several
+responsibilities, and should return the module object it has loaded [#fn1]_.
+If it cannot load the module, it should raise an :exc:`ImportError`, although
+any other exception raised during :meth:`load_module()` will be propagated.
+
+In many cases, the meta path finder and loader can be the same object,
+e.g. :meth:`finder.find_module()` would just return ``self``.
+
+Loaders must satisfy the following requirements:
+
+ * If there is an existing module object with the given name in
+   :data:`sys.modules`, the loader must use that existing module.  (Otherwise,
+   the :func:`reload()` builtin will not work correctly.)  If the named module
+   does not exist in :data:`sys.modules`, the loader must create a new module
+   object and add it to :data:`sys.modules`.
+
+   Note that the module *must* exist in :data:`sys.modules` before the loader
+   executes the module code.  This is crucial because the module code may
+   (directly or indirectly) import itself; adding it to :data:`sys.modules`
+   beforehand prevents unbounded recursion in the worst case and multiple
+   loading in the best.
+
+   If the load fails, the loader needs to remove any modules it may have
+   inserted into ``sys.modules``.  If the module was already in
+   ``sys.modules`` then the loader should leave it alone.
+
+ * The loader may set the ``__file__`` attribute of the module.  If set, this
+   attribute's value must be a string.  The loader may opt to leave
+   ``__file__`` unset if it has no semantic meaning (e.g. a module loaded from
+   a database).
+
+ * The loader may set the ``__name__`` attribute of the module.  While not
+   required, setting this attribute is highly recommended so that the
+   :meth:`repr()` of the module is more informative.
+
+ * If module is a package (either regular or namespace), the loader must set
+   the module object's ``__path__`` attribute.  The value must be a list, but
+   may be empty if ``__path__`` has no further significance to the importer.
+   More details on the semantics of ``__path__`` are given below.
+
+ * The ``__loader__`` attribute must be set to the loader object that loaded
+   the module.  This is mostly for introspection and reloading, but can be
+   used for additional importer-specific functionality, for example getting
+   data associated with an importer.
+
+ * The module's ``__package__`` attribute should be set.  Its value must be a
+   string, but it can be the same value as its ``__name__``.  This is the
+   recommendation when the module is a package.  When the module is not a
+   package, ``__package__`` should be set to the parent package's name.
+
+   This attribute is used instead of ``__name__`` to calculate explicit
+   relative imports for main modules, as defined in :pep:`366`.
+
+ * If the module is a Python module (as opposed to a built-in module or a
+   dynamically loaded extension), it should execute the module's code in the
+   module's global name space (``module.__dict__``).
+
+
+Module reprs
+------------
+
+By default, all modules have a usable repr, however depending on the
+attributes set above, and hooks in the loader, you can more tightly control
+the repr of module objects.
+
+Loaders may implement a :meth:`module_repr()` method which takes a single
+argument, the module object.  When ``repr(module)`` is called for a module
+with a loader supporting this protocol, whatever is returned from
+``loader.module_repr(module)`` is returned as the module's repr without
+further processing.  This return value must be a string.
+
+If the module has no ``__loader__`` attribute, or the loader has no
+:meth:`module_repr()` method, then the module object implementation itself
+will craft a default repr using whatever information is available.  It will
+try to use the ``module.__name__``, ``module.__file__``, and
+``module.__loader__`` as input into the repr, with defaults for whatever
+information is missing.
+
+Here are the exact rules used:
+
+ * If the module has an ``__loader__`` and that loader has a
+   :meth:`module_repr()` method, call it with a single argument, which is the
+   module object.  The value returned is used as the module's repr.
+
+ * If an exception occurs in :meth:`module_repr()`, the exception is caught
+   and discarded, and the calculation of the module's repr continues as if
+   :meth:`module_repr()` did not exist.
+
+ * If the module has an ``__file__`` attribute, this is used as part of the
+   module's repr.
+
+ * If the module has no ``__file__`` but does have an ``__loader__``, then the
+   loader's repr is used as part of the module's repr.
+
+ * Otherwise, just use the module's ``__name__`` in the repr.
+
+This example, from :pep:`420` shows how a loader can craft its own module
+repr::
+
+    class NamespaceLoader:
+        @classmethod
+        def module_repr(cls, module):
+            return "<module '{}' (namespace)>".format(module.__name__)
+
+
+module.__path__
+---------------
+
+By definition, if a module has an ``__path__`` attribute, it is a package,
+regardless of its value.
+
+A package's ``__path__`` attribute is used during imports of its subpackages.
+Within the import machinery, it functions much the same as :data:`sys.path`,
+i.e. providing a list of locations to search for modules during import.
+However, ``__path__`` is typically much more constrained than
+:data:`sys.path`.
+
+``__path__`` must be a list, but it may be empty.  The same rules used for
+:data:`sys.path` also apply to a package's ``__path__``, and
+:data:`sys.path_hooks` (described below) are consulted when traversing a
+package's ``__path__``.
+
+A package's ``__init__.py`` file may set or alter the package's ``__path__``
+attribute, and this was typically the way namespace packages were implemented
+prior to :pep:`420`.  With the adoption of :pep:`420`, namespace packages no
+longer need to supply ``__init__.py`` files containing only ``__path__``
+manipulation code; the namespace loader automatically sets ``__path__``
+correctly for the namespace package.
+
+
+The Path Importer
+=================
+
+.. index::
+    single: path importer
+
+As mentioned previously, Python comes with several default meta path finders.
+One of these, called the :term:`path importer`, knows how to provide
+traditional file system imports.  It implements all the semantics for finding
+modules on the file system, handling special file types such as Python source
+code (``.py`` files), Python byte code (``.pyc`` and ``.pyo`` files) and
+shared libraries (e.g. ``.so`` files).
+
+In addition to being able to find such modules, there is built-in support for
+loading these modules.  To accomplish these two related tasks, additional
+hooks and protocols are provided so that you can extend and customize the path
+importer semantics.
+
+A word of warning: this section and the previous both use the term *finder*,
+distinguishing between them by using the terms :term:`meta path finder` and
+:term:`sys path finder`.  Meta path finders and sys path finders are very
+similar, support similar protocols, and function in similar ways during the
+import process, but it's important to keep in mind that they are subtly
+different.  In particular, meta path finders operate at the beginning of the
+import process, as keyed off the :data:`sys.meta_path` traversal.
+
+On the other hand, sys path finders are in a sense an implementation detail of
+the path importer, and in fact, if the path importer were to be removed from
+:data:`sys.meta_path`, none of the sys path finder semantics would be invoked.
+
+
+sys path finders
+----------------
+
+.. index::
+    single: sys.path
+    single: sys.path_hooks
+    single: sys.path_importer_cache
+    single: PYTHONPATH
+
+The path importer is responsible for finding and loading Python modules and
+packages from the file system.  As a meta path finder, it implements the
+:meth:`find_module()` protocol previously described, however it exposes
+additional hooks that can be used to customize how modules are found and
+loaded from the file system.
+
+Three variables are used during file system import, :data:`sys.path`,
+:data:`sys.path_hooks` and :data:`sys.path_importer_cache`.  These provide
+additional ways that the import machinery can be customized, in this case
+specifically during file system path import.
+
+:data:`sys.path` contains a list of strings providing search locations for
+modules and packages.  It is initialized from the :data:`PYTHONPATH`
+environment variable and various other installation- and
+implementation-specific defaults.  Entries in :data:`sys.path` can name
+directories on the file system, zip files, and potentially other "locations"
+that should be searched for modules.
+
+The path importer is a meta path finder, so the import machinery begins file
+system search by calling the path importer's :meth:`find_module()` method as
+described previously.  When the ``path`` argument to :meth:`find_module()` is
+given, it will be a list of string paths to traverse.  If not,
+:data:`sys.path` is used.
+
+The path importer iterates over every entry in the search path, and for each
+of these, searches for an appropriate sys path finder for the path entry.
+Because this can be an expensive operation (e.g. there are `stat()` call
+overheads for this search), the path importer maintains a cache mapping path
+entries to sys path finders.  This cache is maintained in
+:data:`sys.path_importer_cache`.  In this way, the expensive search for a
+particular path location's sys path finder need only be done once.  User code
+is free to remove cache entries from :data:`sys.path_importer_cache` forcing
+the path importer to perform the path search again.
+
+If the path entry is not present in the cache, the path importer iterates over
+every callable in :data:`sys.path_hooks`.  Each entry in this list is called
+with a single argument, the path entry being searched.  This callable may
+either return a sys path finder that can handle the path entry, or it may
+raise :exc:`ImportError`.  An :exc:`ImportError` is used by the path importer
+to signal that the hook cannot find a sys path finder for that path entry.
+The exception is ignored and :data:`sys.path_hooks` iteration continues.
+
+If :data:`sys.path_hooks` iteration ends with no sys path finder being
+returned then the path importer's :meth:`find_module()` method will return
+``None`` and an :exc:`ImportError` will be raised.
+
+If a sys path finder *is* returned by one of the callables on
+:data:`sys.path_hooks`, then the following protocol is used to ask the sys
+path finder for a module loader.  If a loader results from this step, it is
+used to load the module as previously described (i.e. its
+:meth:`load_module()` method is called).
+
+
+sys path finder protocol
+------------------------
+
+sys path finders support the same, traditional :meth:`find_module()` method
+that meta path finders support, however sys path finder :meth:`find_module()`
+methods are never called with a ``path`` argument.
+
+The :meth:`find_module()` method on sys path finders is deprecated though, and
+instead sys path finders should implement the :meth:`find_loader()` method.
+If it exists on the sys path finder, :meth:`find_loader()` will always be
+called instead of :meth:`find_module()`.
+
+:meth:`find_loader()` takes one argument, the fully qualified name of the
+module being imported.  :meth:`find_loader()` returns a 2-tuple where the
+first item is the loader and the second item is a namespace :term:`portion`.
+When the first item (i.e. the loader) is ``None``, this means that while the
+sys path finder does not have a loader for the named module, it knows that the
+path entry contributes to a namespace portion for the named module.  This will
+almost always be the case where Python is asked to import a namespace package
+that has no physical presence on the file system.  When a sys path finder
+returns ``None`` for the loader, the second item of the 2-tuple return value
+must be a sequence, although it can be empty.
+
+If :meth:`find_loader()` returns a non-``None`` loader value, the portion is
+ignored and the loader is returned from the path importer, terminating the
+:data:`sys.path` search.
+
+
+Open issues
+===========
+
+XXX What to say about `imp.NullImporter` when it's found in
+:data:`sys.path_importer_cache`?
+
+XXX It would be really nice to have a diagram.
+
+.. [#fn1] The importlib implementation appears not to use the return value
+   directly. Instead, it gets the module object by looking the module name up
+   in ``sys.modules``.)
+
+
+References
+==========
+
+The import machinery has evolved considerably since Python's early days.  The
+original `specification for packages
+<http://www.python.org/doc/essays/packages.html>`_ is still available to read,
+although some details have changed since the writing of that document.
+
+The original specification for :data:`sys.meta_path` was :pep:`302`, with
+subsequent extension in :pep:`420`, which also introduced namespace packages
+without ``__init__.py`` files in Python 3.3.  :pep:`420` also introduced the
+:meth:`find_loader` protocol as an alternative to :meth:`find_module`.
+
+:pep:`366` describes the addition of the ``__package__`` attribute for
+explicit relative imports in main modules.
diff --git a/Doc/reference/index.rst b/Doc/reference/index.rst
index bd1a281..87166f8 100644
--- a/Doc/reference/index.rst
+++ b/Doc/reference/index.rst
@@ -24,6 +24,7 @@ interfaces available to C/C++ programmers in detail.
    lexical_analysis.rst
    datamodel.rst
    executionmodel.rst
+   import_machinery.rst
    expressions.rst
    simple_stmts.rst
    compound_stmts.rst
diff --git a/Doc/reference/simple_stmts.rst b/Doc/reference/simple_stmts.rst
index ce7ce92..f93f497 100644
--- a/Doc/reference/simple_stmts.rst
+++ b/Doc/reference/simple_stmts.rst
@@ -660,162 +660,50 @@ The :keyword:`import` statement
    relative_module: "."* `module` | "."+
    name: `identifier`
 
-Import statements are executed in two steps: (1) find a module, and initialize
-it if necessary; (2) define a name or names in the local namespace (of the scope
-where the :keyword:`import` statement occurs). The statement comes in two
-forms differing on whether it uses the :keyword:`from` keyword. The first form
-(without :keyword:`from`) repeats these steps for each identifier in the list.
-The form with :keyword:`from` performs step (1) once, and then performs step
-(2) repeatedly. For a reference implementation of step (1), see the
-:mod:`importlib` module.
-
-.. index::
-    single: package
-
-To understand how step (1) occurs, one must first understand how Python handles
-hierarchical naming of modules. To help organize modules and provide a
-hierarchy in naming, Python has a concept of packages. A package can contain
-other packages and modules while modules cannot contain other modules or
-packages. From a file system perspective, packages are directories and modules
-are files. The original `specification for packages
-<http://www.python.org/doc/essays/packages.html>`_ is still available to read,
-although minor details have changed since the writing of that document.
-
-.. index::
-    single: sys.modules
-
-Once the name of the module is known (unless otherwise specified, the term
-"module" will refer to both packages and modules), searching
-for the module or package can begin. The first place checked is
-:data:`sys.modules`, the cache of all modules that have been imported
-previously. If the module is found there then it is used in step (2) of import
-unless ``None`` is found in :data:`sys.modules`, in which case
-:exc:`ImportError` is raised.
-
-.. index::
-    single: sys.meta_path
-    single: finder
-    pair: finder; find_module
-    single: __path__
-
-If the module is not found in the cache, then :data:`sys.meta_path` is searched
-(the specification for :data:`sys.meta_path` can be found in :pep:`302`).
-The object is a list of :term:`finder` objects which are queried in order as to
-whether they know how to load the module by calling their :meth:`find_module`
-method with the name of the module. If the module happens to be contained
-within a package (as denoted by the existence of a dot in the name), then a
-second argument to :meth:`find_module` is given as the value of the
-:attr:`__path__` attribute from the parent package (everything up to the last
-dot in the name of the module being imported). If a finder can find the module
-it returns a :term:`loader` (discussed later) or returns ``None``.
-
-.. index::
-    single: sys.path_hooks
-    single: sys.path_importer_cache
-    single: sys.path
-
-If none of the finders on :data:`sys.meta_path` are able to find the module
-then some implicitly defined finders are queried. Implementations of Python
-vary in what implicit meta path finders are defined. The one they all do
-define, though, is one that handles :data:`sys.path_hooks`,
-:data:`sys.path_importer_cache`, and :data:`sys.path`.
-
-The implicit finder searches for the requested module in the "paths" specified
-in one of two places ("paths" do not have to be file system paths). If the
-module being imported is supposed to be contained within a package then the
-second argument passed to :meth:`find_module`, :attr:`__path__` on the parent
-package, is used as the source of paths. If the module is not contained in a
-package then :data:`sys.path` is used as the source of paths.
-
-Once the source of paths is chosen it is iterated over to find a finder that
-can handle that path. The dict at :data:`sys.path_importer_cache` caches
-finders for paths and is checked for a finder. If the path does not have a
-finder cached then :data:`sys.path_hooks` is searched by calling each object in
-the list with a single argument of the path, returning a finder or raises
-:exc:`ImportError`. If a finder is returned then it is cached in
-:data:`sys.path_importer_cache` and then used for that path entry. If no finder
-can be found but the path exists then a value of ``None`` is
-stored in :data:`sys.path_importer_cache` to signify that an implicit,
-file-based finder that handles modules stored as individual files should be
-used for that path. If the path does not exist then a finder which always
-returns ``None`` is placed in the cache for the path.
-
-.. index::
-    single: loader
-    pair: loader; load_module
-    exception: ImportError
-
-If no finder can find the module then :exc:`ImportError` is raised. Otherwise
-some finder returned a loader whose :meth:`load_module` method is called with
-the name of the module to load (see :pep:`302` for the original definition of
-loaders). A loader has several responsibilities to perform on a module it
-loads. First, if the module already exists in :data:`sys.modules` (a
-possibility if the loader is called outside of the import machinery) then it
-is to use that module for initialization and not a new module. But if the
-module does not exist in :data:`sys.modules` then it is to be added to that
-dict before initialization begins. If an error occurs during loading of the
-module and it was added to :data:`sys.modules` it is to be removed from the
-dict. If an error occurs but the module was already in :data:`sys.modules` it
-is left in the dict.
-
-.. index::
-    single: __name__
-    single: __file__
-    single: __path__
-    single: __package__
-    single: __loader__
-
-The loader must set several attributes on the module. :data:`__name__` is to be
-set to the name of the module. :data:`__file__` is to be the "path" to the file
-unless the module is built-in (and thus listed in
-:data:`sys.builtin_module_names`) in which case the attribute is not set.
-If what is being imported is a package then :data:`__path__` is to be set to a
-list of paths to be searched when looking for modules and packages contained
-within the package being imported. :data:`__package__` is optional but should
-be set to the name of package that contains the module or package (the empty
-string is used for module not contained in a package). :data:`__loader__` is
-also optional but should be set to the loader object that is loading the
-module. While loaders are required to return the module they loaded, import
-itself always retrieves any modules it returns from :data:`sys.modules`.
-
-.. index::
-    exception: ImportError
-
-If an error occurs during loading then the loader raises :exc:`ImportError` if
-some other exception is not already being propagated. Otherwise the loader
-returns the module that was loaded and initialized.
+Import statements are executed in two steps: (1) find a module, loading and
+initializing it if necessary; (2) define a name or names in the local
+namespace (of the scope where the :keyword:`import` statement occurs). The
+statement comes in two forms differing on whether it uses the :keyword:`from`
+keyword. The first form (without :keyword:`from`) repeats these steps for each
+identifier in the list.  The form with :keyword:`from` performs step (1) once,
+and then performs step (2) repeatedly.
+
+The details of step (1), finding and loading modules is described in greater
+detail in the section on the :ref:`import machinery <importmachinery>`, which
+also describes the various types of packages and modules that can be imported,
+as well as all the hooks that can be used to customize Python's import.
 
 When step (1) finishes without raising an exception, step (2) can begin.
 
-The first form of :keyword:`import` statement binds the module name in the local
-namespace to the module object, and then goes on to import the next identifier,
-if any.  If the module name is followed by :keyword:`as`, the name following
-:keyword:`as` is used as the local name for the module.
+The first form of :keyword:`import` statement binds the module name in the
+local namespace to the module object, and then goes on to import the next
+identifier, if any.  If the module name is followed by :keyword:`as`, the name
+following :keyword:`as` is used as the local name for the module.
 
 .. index::
    pair: name; binding
    exception: ImportError
 
-The :keyword:`from` form does not bind the module name: it goes through the list
-of identifiers, looks each one of them up in the module found in step (1), and
-binds the name in the local namespace to the object thus found.  As with the
-first form of :keyword:`import`, an alternate local name can be supplied by
-specifying ":keyword:`as` localname".  If a name is not found,
-:exc:`ImportError` is raised.  If the list of identifiers is replaced by a star
-(``'*'``), all public names defined in the module are bound in the local
+The :keyword:`from` form does not bind the module name: it goes through the
+list of identifiers, looks each one of them up in the module found in step
+(1), and binds the name in the local namespace to the object thus found.  As
+with the first form of :keyword:`import`, an alternate local name can be
+supplied by specifying ":keyword:`as` localname".  If a name is not found,
+:exc:`ImportError` is raised.  If the list of identifiers is replaced by a
+star (``'*'``), all public names defined in the module are bound in the local
 namespace of the :keyword:`import` statement.
 
 .. index:: single: __all__ (optional module attribute)
 
 The *public names* defined by a module are determined by checking the module's
-namespace for a variable named ``__all__``; if defined, it must be a sequence of
-strings which are names defined or imported by that module.  The names given in
-``__all__`` are all considered public and are required to exist.  If ``__all__``
-is not defined, the set of public names includes all names found in the module's
-namespace which do not begin with an underscore character (``'_'``).
-``__all__`` should contain the entire public API. It is intended to avoid
-accidentally exporting items that are not part of the API (such as library
-modules which were imported and used within the module).
+namespace for a variable named ``__all__``; if defined, it must be a sequence
+of strings which are names defined or imported by that module.  The names
+given in ``__all__`` are all considered public and are required to exist.  If
+``__all__`` is not defined, the set of public names includes all names found
+in the module's namespace which do not begin with an underscore character
+(``'_'``).  ``__all__`` should contain the entire public API. It is intended
+to avoid accidentally exporting items that are not part of the API (such as
+library modules which were imported and used within the module).
 
 The :keyword:`from` form with ``*`` may only occur in a module scope.  The wild
 card form of import --- ``import *`` --- is only allowed at the module level.