diff options
| -rw-r--r-- | Doc/library/importlib.rst | 3 | ||||
| -rw-r--r-- | Doc/library/modules.rst | 1 | ||||
| -rw-r--r-- | Doc/library/site.rst | 4 | ||||
| -rw-r--r-- | Doc/library/sys.rst | 7 | ||||
| -rw-r--r-- | Doc/library/sys_path_init.rst | 108 | ||||
| -rw-r--r-- | Doc/tutorial/modules.rst | 2 | ||||
| -rw-r--r-- | Doc/using/windows.rst | 25 | ||||
| -rw-r--r-- | Doc/whatsnew/3.6.rst | 4 | ||||
| -rw-r--r-- | Doc/whatsnew/3.7.rst | 2 | ||||
| -rw-r--r-- | Misc/ACKS | 1 |
10 files changed, 128 insertions, 29 deletions
diff --git a/Doc/library/importlib.rst b/Doc/library/importlib.rst index 23d9081..7836538 100644 --- a/Doc/library/importlib.rst +++ b/Doc/library/importlib.rst @@ -51,6 +51,9 @@ managing aspects of Python packages: The :func:`.__import__` function The :keyword:`import` statement is syntactic sugar for this function. + :ref:`sys-path-init` + The initialization of :data:`sys.path`. + :pep:`235` Import on Case-Insensitive Platforms diff --git a/Doc/library/modules.rst b/Doc/library/modules.rst index 131dfca..6cf6eb2 100644 --- a/Doc/library/modules.rst +++ b/Doc/library/modules.rst @@ -19,3 +19,4 @@ The full list of modules described in this chapter is: importlib.rst importlib.resources.rst importlib.metadata.rst + sys_path_init.rst diff --git a/Doc/library/site.rst b/Doc/library/site.rst index e2ad3c4..5941739 100644 --- a/Doc/library/site.rst +++ b/Doc/library/site.rst @@ -276,4 +276,6 @@ value greater than 2 if there is an error. .. seealso:: - :pep:`370` -- Per user site-packages directory + * :pep:`370` -- Per user site-packages directory + * :ref:`sys-path-init` -- The initialization of :data:`sys.path`. + diff --git a/Doc/library/sys.rst b/Doc/library/sys.rst index b83b116..3b09a4c 100644 --- a/Doc/library/sys.rst +++ b/Doc/library/sys.rst @@ -1124,15 +1124,16 @@ always available. current directory first. Notice that the script directory is inserted *before* the entries inserted as a result of :envvar:`PYTHONPATH`. + The initialization of :data:`sys.path` is documented at :ref:`sys-path-init`. + A program is free to modify this list for its own purposes. Only strings and bytes should be added to :data:`sys.path`; all other data types are ignored during import. .. seealso:: - Module :mod:`site` This describes how to use .pth files to extend - :data:`sys.path`. - + * Module :mod:`site` This describes how to use .pth files to + extend :data:`sys.path`. .. data:: path_hooks diff --git a/Doc/library/sys_path_init.rst b/Doc/library/sys_path_init.rst new file mode 100644 index 0000000..72c1387 --- /dev/null +++ b/Doc/library/sys_path_init.rst @@ -0,0 +1,108 @@ +.. _sys-path-init: + +The initialization of the :data:`sys.path` module search path +============================================================= + +A module search path is initialized when Python starts. This module search path +may be accessed at :data:`sys.path`. + +The first entry in the module search path is the directory that contains the +input script, if there is one. Otherwise, the first entry is the current +directory, which is the case when executing the interactive shell, a :option:`-c` +command, or :option:`-m` module. + +The :envvar:`PYTHONPATH` environment variable is often used to add directories +to the search path. If this environment variable is found then the contents are +added to the module search path. + +.. note:: + + :envvar:`PYTHONPATH` will affect all installed Python versions/environments. + Be wary of setting this in your shell profile or global environment variables. + The :mod:`site` module offers more nuanced techniques as mentioned below. + +The next items added are the directories containing standard Python modules as +well as any :term:`extension module`\s that these modules depend on. Extension +modules are ``.pyd`` files on Windows and ``.so`` files on other platforms. The +directory with the platform-independent Python modules is called ``prefix``. +The directory with the extension modules is called ``exec_prefix``. + +The :envvar:`PYTHONHOME` environment variable may be used to set the ``prefix`` +and ``exec_prefix`` locations. Otherwise these directories are found by using +the Python executable as a starting point and then looking for various 'landmark' +files and directories. Note that any symbolic links are followed so the real +Python executable location is used as the search starting point. The Python +executable location is called ``home``. + +Once ``home`` is determined, the ``prefix`` directory is found by first looking +for :file:`python{majorversion}{minorversion}.zip` (``python311.zip``). On Windows +the zip archive is searched for in ``home`` and on Unix the archive is expected +to be in :file:`lib`. Note that the expected zip archive location is added to the +module search path even if the archive does not exist. If no archive was found, +Python on Windows will continue the search for ``prefix`` by looking for :file:`Lib\\os.py`. +Python on Unix will look for :file:`lib/python{majorversion}.{minorversion}/os.py` +(``lib/python3.11/os.py``). On Windows ``prefix`` and ``exec_prefix`` are the same, +however on other platforms :file:`lib/python{majorversion}.{minorversion}/lib-dynload` +(``lib/python3.11/lib-dynload``) is searched for and used as an anchor for +``exec_prefix``. On some platforms :file:`lib` may be :file:`lib64` or another value, +see :data:`sys.platlibdir` and :envvar:`PYTHONPLATLIBDIR`. + +Once found, ``prefix`` and ``exec_prefix`` are available at :data:`sys.prefix` and +:data:`sys.exec_prefix` respectively. + +Finally, the :mod:`site` module is processed and :file:`site-packages` directories +are added to the module search path. A common way to customize the search path is +to create :mod:`sitecustomize` or :mod:`usercustomize` modules as described in +the :mod:`site` module documentation. + +.. note:: + + Certain command line options may further affect path calculations. + See :option:`-E`, :option:`-I`, :option:`-s` and :option:`-S` for further details. + +Virtual environments +-------------------- + +If Python is run in a virtual environment (as described at :ref:`tut-venv`) +then ``prefix`` and ``exec_prefix`` are specific to the virtual environment. + +If a ``pyvenv.cfg`` file is found alongside the main executable, or in the +directory one level above the executable, the following variations apply: + +* If ``home`` is an absolute path and :envvar:`PYTHONHOME` is not set, this + path is used instead of the path to the main executable when deducing ``prefix`` + and ``exec_prefix``. + +_pth files +---------- + +To completely override :data:`sys.path` create a ``._pth`` file with the same +name as the shared library or executable (``python._pth`` or ``python311._pth``). +The shared library path is always known on Windows, however it may not be +available on other platforms. In the ``._pth`` file specify one line for each path +to add to :data:`sys.path`. The file based on the shared library name overrides +the one based on the executable, which allows paths to be restricted for any +program loading the runtime if desired. + +When the file exists, all registry and environment variables are ignored, +isolated mode is enabled, and :mod:`site` is not imported unless one line in the +file specifies ``import site``. Blank paths and lines starting with ``#`` are +ignored. Each path may be absolute or relative to the location of the file. +Import statements other than to ``site`` are not permitted, and arbitrary code +cannot be specified. + +Note that ``.pth`` files (without leading underscore) will be processed normally +by the :mod:`site` module when ``import site`` has been specified. + +Embedded Python +--------------- + +If Python is embedded within another application :c:func:`Py_InitializeFromConfig` and +the :c:type:`PyConfig` structure can be used to initialize Python. The path specific +details are described at :ref:`init-path-config`. Alternatively the older :c:func:`Py_SetPath` +can be used to bypass the initialization of the module search path. + +.. seealso:: + + * :ref:`windows_finding_modules` for detailed Windows notes. + * :ref:`using-on-unix` for Unix details. diff --git a/Doc/tutorial/modules.rst b/Doc/tutorial/modules.rst index f1d4957..a4e1cef 100644 --- a/Doc/tutorial/modules.rst +++ b/Doc/tutorial/modules.rst @@ -194,6 +194,8 @@ named :file:`spam.py` in a list of directories given by the variable * The installation-dependent default (by convention including a ``site-packages`` directory, handled by the :mod:`site` module). +More details are at :ref:`sys-path-init`. + .. note:: On file systems which support symlinks, the directory containing the input script is calculated after the symlink is followed. In other words the diff --git a/Doc/using/windows.rst b/Doc/using/windows.rst index 26c19dd..618dfea 100644 --- a/Doc/using/windows.rst +++ b/Doc/using/windows.rst @@ -946,32 +946,13 @@ target Python. -.. _finding_modules: +.. _windows_finding_modules: Finding modules =============== -Python usually stores its library (and thereby your site-packages folder) in the -installation directory. So, if you had installed Python to -:file:`C:\\Python\\`, the default library would reside in -:file:`C:\\Python\\Lib\\` and third-party modules should be stored in -:file:`C:\\Python\\Lib\\site-packages\\`. - -To completely override :data:`sys.path`, create a ``._pth`` file with the same -name as the DLL (``python37._pth``) or the executable (``python._pth``) and -specify one line for each path to add to :data:`sys.path`. The file based on the -DLL name overrides the one based on the executable, which allows paths to be -restricted for any program loading the runtime if desired. - -When the file exists, all registry and environment variables are ignored, -isolated mode is enabled, and :mod:`site` is not imported unless one line in the -file specifies ``import site``. Blank paths and lines starting with ``#`` are -ignored. Each path may be absolute or relative to the location of the file. -Import statements other than to ``site`` are not permitted, and arbitrary code -cannot be specified. - -Note that ``.pth`` files (without leading underscore) will be processed normally -by the :mod:`site` module when ``import site`` has been specified. +These notes supplement the description at :ref:`sys-path-init` with +detailed Windows notes. When no ``._pth`` file is found, this is how :data:`sys.path` is populated on Windows: diff --git a/Doc/whatsnew/3.6.rst b/Doc/whatsnew/3.6.rst index a56d7a5..d35a0fd 100644 --- a/Doc/whatsnew/3.6.rst +++ b/Doc/whatsnew/3.6.rst @@ -162,10 +162,10 @@ Windows improvements: * A ``._pth`` file can be added to force isolated mode and fully specify all search paths to avoid registry and environment lookup. See - :ref:`the documentation <finding_modules>` for more information. + :ref:`the documentation <windows_finding_modules>` for more information. * A ``python36.zip`` file now works as a landmark to infer - :envvar:`PYTHONHOME`. See :ref:`the documentation <finding_modules>` for + :envvar:`PYTHONHOME`. See :ref:`the documentation <windows_finding_modules>` for more information. diff --git a/Doc/whatsnew/3.7.rst b/Doc/whatsnew/3.7.rst index dcbd092..5ce637a 100644 --- a/Doc/whatsnew/3.7.rst +++ b/Doc/whatsnew/3.7.rst @@ -2474,7 +2474,7 @@ Windows-only Changes The file used to override :data:`sys.path` is now called ``<python-executable>._pth`` instead of ``'sys.path'``. -See :ref:`finding_modules` for more information. +See :ref:`windows_finding_modules` for more information. (Contributed by Steve Dower in :issue:`28137`.) @@ -1897,6 +1897,7 @@ Colin Watson David Watson Aaron Watters Alex Waygood +Russel Webber Henrik Weber Leon Weber Steve Weber |