summaryrefslogtreecommitdiffstats
path: root/Python/import.c
Commit message (Collapse)AuthorAgeFilesLines
* bpo-45020: Fix strict-prototypes warning (GH-29755)Christian Heimes2021-11-241-1/+1
|
* bpo-45696: Deep-freeze selected modules (GH-29118)Guido van Rossum2021-11-111-0/+7
| | | | | | | This gains 10% or more in startup time for `python -c pass` on UNIX-ish systems. The Makefile.pre.in generating code builds on Eric's work for bpo-45020, but the .c file generator is new. Windows version TBD.
* bpo-45379: clarify FROZEN_EXCLUDED and FROZEN_INVALID documentation (GH-29189)Filipe Laíns2021-10-291-2/+7
| | | | Signed-off-by: Filipe Laíns <lains@riseup.net> Co-authored-by: Łukasz Langa <lukasz@langa.pl>
* bpo-45395: Make custom frozen modules additions instead of replacements. ↵Eric Snow2021-10-281-46/+113
| | | | | | | | | (gh-28778) Currently custom modules (the array set on PyImport_FrozenModules) replace all the frozen stdlib modules. That can be problematic and is unlikely to be what the user wants. This change treats the custom frozen modules as additions instead. They take precedence over all other frozen modules except for those needed to bootstrap the import system. If the "code" field of an entry in the custom array is NULL then that frozen module is treated as disabled, which allows a custom entry to disable a frozen stdlib module. This change allows us to get rid of is_essential_frozen_module() and simplifies the logic for which frozen modules should be ignored. https://bugs.python.org/issue45395
* bpo-45379: add custom error string for FROZEN_DISABLED (GH-29190)Filipe Laíns2021-10-281-1/+3
| | | | | Signed-off-by: Filipe Laíns <lains@riseup.net> Co-authored-by: Gareth Rees <gdr@garethrees.org>
* bpo-45482: Rename namespaceobject.h to pycore_namespace.h (GH-28975)Victor Stinner2021-10-151-10/+10
| | | | | | | | | Rename Include/namespaceobject.h to Include/internal/pycore_namespace.h. The _testmultiphase extension is now built with the Py_BUILD_CORE_MODULE macro defined to access _PyNamespace_Type. object.c: remove unused "pycore_context.h" include.
* bpo-21736: Set __file__ on frozen stdlib modules. (gh-28656)Eric Snow2021-10-141-15/+32
| | | | | | | | | | | | | | | | | | | | | | | | Currently frozen modules do not have __file__ set. In their spec, origin is set to "frozen" and they are marked as not having a location. (Similarly, for frozen packages __path__ is set to an empty list.) However, for frozen stdlib modules we are able to extrapolate __file__ as long as we can determine the stdlib directory at runtime. (We now do so since gh-28586.) Having __file__ set is helpful for a number of reasons. Likewise, having a non-empty __path__ means we can import submodules of a frozen package from the filesystem (e.g. we could partially freeze the encodings module). This change sets __file__ (and adds to __path__) for frozen stdlib modules. It uses sys._stdlibdir (from gh-28586) and the frozen module alias information (from gh-28655). All that work is done in FrozenImporter (in Lib/importlib/_bootstrap.py). Also, if a frozen module is imported before importlib is bootstrapped (during interpreter initialization) then we fix up that module and its spec during the importlib bootstrapping step (i.e. imporlib._bootstrap._setup()) to match what gets set by FrozenImporter, including setting the file info (if the stdlib dir is known). To facilitate this, modules imported using PyImport_ImportFrozenModule() have __origname__ set using the frozen module alias info. __origname__ is popped off during importlib bootstrap. (To be clear, even with this change the new code to set __file__ during fixups in imporlib._bootstrap._setup() doesn't actually get triggered yet. This is because sys._stdlibdir hasn't been set yet in interpreter initialization at the point importlib is bootstrapped. However, we do fix up such modules at that point to otherwise match the result of importing through FrozenImporter, just not the __file__ and __path__ parts. Doing so will require changes in the order in which things happen during interpreter initialization. That can be addressed separately. Once it is, the file-related fixup code from this PR will kick in.) Here are things this change does not do: * set __file__ for non-stdlib modules (no way of knowing the parent dir) * set __file__ if the stdlib dir is not known (nor assume the expense of finding it) * relatedly, set __file__ if the stdlib is in a zip file * verify that the filename set to __file__ actually exists (too expensive) * update __path__ for frozen packages that alias a non-package (since there is no package dir) Other things this change skips, but we may do later: * set __file__ on modules imported using PyImport_ImportFrozenModule() * set co_filename when we unmarshal the frozen code object while importing the module (e.g. in FrozenImporter.exec_module()) -- this would allow tracebacks to show source lines * implement FrozenImporter.get_filename() and FrozenImporter.get_source() https://bugs.python.org/issue21736
* bpo-45434: Mark the PyTokenizer C API as private (GH-28924)Victor Stinner2021-10-131-1/+0
| | | | | | | | | | | | | | Rename PyTokenize functions to mark them as private: * PyTokenizer_FindEncodingFilename() => _PyTokenizer_FindEncodingFilename() * PyTokenizer_FromString() => _PyTokenizer_FromString() * PyTokenizer_FromFile() => _PyTokenizer_FromFile() * PyTokenizer_FromUTF8() => _PyTokenizer_FromUTF8() * PyTokenizer_Free() => _PyTokenizer_Free() * PyTokenizer_Get() => _PyTokenizer_Get() Remove the unused PyTokenizer_FindEncoding() function. import.c: remove unused #include "errcode.h".
* Fix typos in the Python directory (GH-28767)Christian Clauss2021-10-061-1/+1
|
* bpo-45020: Identify which frozen modules are actually aliases. (gh-28655)Eric Snow2021-10-051-11/+70
| | | | | | | In the list of generated frozen modules at the top of Tools/scripts/freeze_modules.py, you will find that some of the modules have a different name than the module (or .py file) that is actually frozen. Let's call each case an "alias". Aliases do not come into play until we get to the (generated) list of modules in Python/frozen.c. (The tool for freezing modules, Programs/_freeze_module, is only concerned with the source file, not the module it will be used for.) Knowledge of which frozen modules are aliases (and the identity of the original module) normally isn't important. However, this information is valuable when we go to set __file__ on frozen stdlib modules. This change updates Tools/scripts/freeze_modules.py to map aliases to the original module name (or None if not a stdlib module) in Python/frozen.c. We also add a helper function in Python/import.c to look up a frozen module's alias and add the result of that function to the frozen info returned from find_frozen(). https://bugs.python.org/issue45020
* bpo-45324: Capture data in FrozenImporter.find_spec() to use in ↵Eric Snow2021-10-051-80/+188
| | | | | | | | | | | exec_module(). (gh-28633) Before this change we end up duplicating effort and throwing away data in FrozenImporter.find_spec(). Now we do the work once in find_spec() and the only thing we do in FrozenImporter.exec_module() is turn the raw frozen data into a code object and then exec it. We've added _imp.find_frozen(), add an arg to _imp.get_frozen_object(), and updated FrozenImporter. We've also moved some code around to reduce duplication, get a little more consistency in outcomes, and be more efficient. Note that this change is mostly necessary if we want to set __file__ on frozen stdlib modules. (See https://bugs.python.org/issue21736.) https://bugs.python.org/issue45324
* bpo-44050: Extension modules can share state when they don't support ↵Hai Shi2021-10-051-1/+3
| | | | | sub-interpreters. (GH-27794) Automerge-Triggered-By: GH:encukou
* bpo-45020: Add -X frozen_modules=[on|off] to explicitly control use of ↵Eric Snow2021-09-141-9/+81
| | | | | | | frozen modules. (gh-28320) Currently we freeze several modules into the runtime. For each of these modules it is essential to bootstrapping the runtime that they be frozen. Any other stdlib module that we later freeze into the runtime is not essential. We can just as well import from the .py file. This PR lets users explicitly choose which should be used, with the new "-X frozen_modules=[on|off]" CLI flag. The default is "off" for now. https://bugs.python.org/issue45020
* bpo-45019: Do some cleanup related to frozen modules. (gh-28319)Eric Snow2021-09-131-0/+41
| | | | | There are a few things I missed in gh-27980. This is a follow-up that will make subsequent PRs cleaner. It includes fixes to tests and tools that reference the frozen modules. https://bugs.python.org/issue45019
* bpo-44441: _PyImport_Fini2() resets PyImport_Inittab (GH-26874)Victor Stinner2021-06-231-0/+3
| | | | | Py_RunMain() now resets PyImport_Inittab to its initial value at exit. It must be possible to call PyImport_AppendInittab() or PyImport_ExtendInittab() at each Python initialization.
* bpo-43244: Remove Yield macro from pycore_ast.h (GH-25243)Victor Stinner2021-04-071-1/+0
| | | | | | | | * pycore_ast.h no longer defines the Yield macro. * Fix a compiler warning on Windows: "warning C4005: 'Yield': macro redefinition". * Python-ast.c now defines directly functions with their real _Py_xxx() name, rather than xxx(). * Remove "#undef Yield" in C files including pycore_ast.h.
* bpo-43244: Remove ast.h, asdl.h, Python-ast.h headers (GH-24933)Victor Stinner2021-03-231-1/+0
| | | | | | | | | | | | | | | | These functions were undocumented and excluded from the limited C API. Most names defined by these header files were not prefixed by "Py" and so could create names conflicts. For example, Python-ast.h defined a "Yield" macro which was conflict with the "Yield" name used by the Windows <winbase.h> header. Use the Python ast module instead. * Move Include/asdl.h to Include/internal/pycore_asdl.h. * Move Include/Python-ast.h to Include/internal/pycore_ast.h. * Remove ast.h header file. * pycore_symtable.h no longer includes Python-ast.h.
* bpo-43551: Fix PyImport_Import() for subinterpreters (GH-24929)junyixie2021-03-221-16/+21
| | | Avoid static variables.
* bpo-43517: Fix false positive in detection of circular imports (#24895)Antoine Pitrou2021-03-201-1/+1
|
* bpo-43268: Pass interp rather than tstate to internal functions (GH-24580)Victor Stinner2021-02-191-8/+7
| | | | | | | | | | | | | | | Pass the current interpreter (interp) rather than the current Python thread state (tstate) to internal functions which only use the interpreter. Modified functions: * _PyXXX_Fini() and _PyXXX_ClearFreeList() functions * _PyEval_SignalAsyncExc(), make_pending_calls() * _PySys_GetObject(), sys_set_object(), sys_set_object_id(), sys_set_object_str() * should_audit(), set_flags_from_config(), make_flags() * _PyAtExit_Call() * init_stdio_encoding() * etc.
* bpo-43268: _Py_IsMainInterpreter() now expects interp (GH-24577)Victor Stinner2021-02-191-1/+1
| | | | The _Py_IsMainInterpreter() function now expects interp rather than tstate.
* bpo-41994: Fix refcount issues in Python/import.c (GH-22632)Serhiy Storchaka2021-01-121-56/+50
| | | https://bugs.python.org/issue41994
* bpo-1635741: Convert _imp to multi-phase init (GH-23378)Victor Stinner2020-11-181-71/+110
| | | | | | | | | | | | Convert the _imp extension module to the multi-phase initialization API (PEP 489). * Add _PyImport_BootstrapImp() which fix a bootstrap issue: import the _imp module before importlib is initialized. * Add create_builtin() sub-function, used by _imp_create_builtin(). * Initialize PyInterpreterState.import_func earlier, in pycore_init_builtins(). * Remove references to _PyImport_Cleanup(). This function has been renamed to finalize_modules() and moved to pylifecycle.c.
* bpo-42260: Initialize time and warnings earlier at startup (GH-23249)Victor Stinner2020-11-121-37/+0
| | | | | | | | | | * Call _PyTime_Init() and _PyWarnings_InitState() earlier during the Python initialization. * Inline _PyImportHooks_Init() into _PySys_InitCore(). * The _warnings initialization function no longer call _PyWarnings_InitState() to prevent resetting filters_version to 0. * _PyWarnings_InitState() now returns an int and no longer clear the state in case of error (it's done anyway at Python exit). * Rework init_importlib(), fix refleaks on errors.
* bpo-42208: Move _PyImport_Cleanup() to pylifecycle.c (GH-23040)Victor Stinner2020-10-301-227/+0
| | | | | Move _PyImport_Cleanup() to pylifecycle.c, rename it to finalize_modules(), split it (200 lines) into many smaller sub-functions and cleanup the code.
* bpo-42208: Pass tstate to _PyGC_CollectNoFail() (GH-23038)Victor Stinner2020-10-301-2/+2
| | | | | | | | | | | | Move private _PyGC_CollectNoFail() to the internal C API. Remove the private _PyGC_CollectIfEnabled() which was just an alias to the public PyGC_Collect() function since Python 3.8. Rename functions: * collect() => gc_collect_main() * collect_with_callback() => gc_collect_with_callback() * collect_generations() => gc_collect_generations()
* bpo-42152: Use PyDict_Contains and PyDict_SetDefault if appropriate. (GH-22986)Serhiy Storchaka2020-10-261-10/+14
| | | | | | | If PyDict_GetItemWithError is only used to check whether the key is in dict, it is better to use PyDict_Contains instead. And if it is used in combination with PyDict_SetItem, PyDict_SetDefault can replace the combination.
* bpo-41993: Fix possible issues in remove_module() (GH-22631)Serhiy Storchaka2020-10-111-10/+13
| | | | | | * PyMapping_HasKey() is not safe because it silences all exceptions and can return incorrect result. * Informative exceptions from PyMapping_DelItem() are overridden with RuntimeError and the original exception raised before calling remove_module() is lost. * There is a race condition between PyMapping_HasKey() and PyMapping_DelItem().
* bpo-40232: _PyImport_ReInitLock() can now safely use its lock (GH-20597)Victor Stinner2020-06-021-5/+1
| | | | | Since _PyImport_ReInitLock() now calls _PyThread_at_fork_reinit() on the import lock, the lock is now in a known state: unlocked. It became safe to acquire it after fork.
* PyOS_AfterFork_Child() uses PyStatus (GH-20596)Victor Stinner2020-06-021-6/+7
| | | | | | | | PyOS_AfterFork_Child() helper functions now return a PyStatus: PyOS_AfterFork_Child() is now responsible to handle errors. * Move _PySignal_AfterFork() to the internal C API * Add #ifdef HAVE_FORK on _PyGILState_Reinit(), _PySignal_AfterFork() and _PyInterpreterState_DeleteExceptMain().
* bpo-40417: Fix deprecation warning in PyImport_ReloadModule (GH-19750)Robert Rouhani2020-05-011-7/+7
| | | | | I can add another commit with the new test case I wrote to verify that the warning was being printed before my change, stopped printing after my change, and that the function does not return null after my change. Automerge-Triggered-By: @brettcannon
* bpo-40412: Nullify inittab_copy during finalization (GH-19746)Gregory Szorc2020-05-011-0/+1
| | | | | | | | | Otherwise we leave a dangling pointer to free'd memory. If we then initialize a new interpreter in the same process and call PyImport_ExtendInittab, we will (likely) crash when calling PyMem_RawRealloc(inittab_copy, ...) since the pointer address is bogus. Automerge-Triggered-By: @brettcannon
* bpo-40429: PyFrame_GetCode() now returns a strong reference (GH-19773)Victor Stinner2020-04-281-0/+1
|
* bpo-40421: Add PyFrame_GetCode() function (GH-19757)Victor Stinner2020-04-281-2/+1
| | | | | | | | | PyFrame_GetCode(frame): return a borrowed reference to the frame code. Replace frame->f_code with PyFrame_GetCode(frame) in most code, except in frameobject.c, genobject.c and ceval.c. Also add PyFrame_GetLineNumber() to the limited C API.
* bpo-40268: Remove unused osdefs.h includes (GH-19532)Victor Stinner2020-04-151-1/+0
| | | When the include is needed, add required symbol in a comment.
* bpo-40268: Remove unused pycore_pymem.h includes (GH-19531)Victor Stinner2020-04-151-1/+1
|
* bpo-40268: Remove explicit pythread.h includes (#19529)Victor Stinner2020-04-151-2/+0
| | | | Remove explicit pythread.h includes: it is always included by Python.h.
* bpo-40232: Update PyOS_AfterFork_Child() to use _PyThread_at_fork_reinit() ↵Dong-hee Na2020-04-141-2/+3
| | | | (GH-19450)
* bpo-40268: Remove a few pycore_pystate.h includes (GH-19510)Victor Stinner2020-04-141-2/+2
|
* bpo-40268: Rename _PyInterpreterState_GET_UNSAFE() (GH-19509)Victor Stinner2020-04-141-3/+3
| | | | | | | Rename _PyInterpreterState_GET_UNSAFE() to _PyInterpreterState_GET() for consistency with _PyThreadState_GET() and to have a shorter name (help to fit into 80 columns). Add also "assert(tstate != NULL);" to the function.
* bpo-40268: Include explicitly pycore_interp.h (GH-19505)Victor Stinner2020-04-141-0/+1
| | | | pycore_pystate.h no longer includes pycore_interp.h: it's now included explicitly in files accessing PyInterpreterState.
* bpo-40268: Add _PyInterpreterState_GetConfig() (GH-19492)Victor Stinner2020-04-131-6/+6
| | | | | | | | Don't access PyInterpreterState.config member directly anymore, but use new functions: * _PyInterpreterState_GetConfig() * _PyInterpreterState_SetConfig() * _Py_GetConfig()
* bpo-39943: Remove unnecessary casts in import.c that remove constness (GH-19209)Andy Lester2020-03-301-2/+2
|
* bpo-38644: Use _PySys_Audit(): pass tstate explicitly (GH-19183)Victor Stinner2020-03-271-4/+5
| | | Add the dependency to tstate more explicit.
* bpo-39882: Add _Py_FatalErrorFormat() function (GH-19157)Victor Stinner2020-03-251-3/+3
|
* bpo-39947: Use _PyInterpreterState_GET_UNSAFE() (GH-18978)Victor Stinner2020-03-131-2/+2
| | | | | | | Replace _PyInterpreterState_Get() function call with _PyInterpreterState_GET_UNSAFE() macro which is more efficient but don't check if tstate or interp is NULL. _Py_GetConfigsAsDict() now uses _PyThreadState_GET().
* bpo-39882: Py_FatalError() logs the function name (GH-18819)Victor Stinner2020-03-061-3/+2
| | | | | | | | | | | | The Py_FatalError() function is replaced with a macro which logs automatically the name of the current function, unless the Py_LIMITED_API macro is defined. Changes: * Add _Py_FatalErrorFunc() function. * Remove the function name from the message of Py_FatalError() calls which included the function name. * Update tests.
* bpo-39245: Switch to public API for Vectorcall (GH-18460)Petr Viktorin2020-02-111-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | The bulk of this patch was generated automatically with: for name in \ PyObject_Vectorcall \ Py_TPFLAGS_HAVE_VECTORCALL \ PyObject_VectorcallMethod \ PyVectorcall_Function \ PyObject_CallOneArg \ PyObject_CallMethodNoArgs \ PyObject_CallMethodOneArg \ ; do echo $name git grep -lwz _$name | xargs -0 sed -i "s/\b_$name\b/$name/g" done old=_PyObject_FastCallDict new=PyObject_VectorcallDict git grep -lwz $old | xargs -0 sed -i "s/\b$old\b/$new/g" and then cleaned up: - Revert changes to in docs & news - Revert changes to backcompat defines in headers - Nudge misaligned comments
* bpo-38076 Clear the interpreter state only after clearing module globals ↵Eddie Elizondo2020-02-041-2/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | (GH-18039) Currently, during runtime destruction, `_PyImport_Cleanup` is clearing the interpreter state before clearing out the modules themselves. This leads to a segfault on modules that rely on the module state to clear themselves up. For example, let's take the small snippet added in the issue by @DinoV : ``` import _struct class C: def __init__(self): self.pack = _struct.pack def __del__(self): self.pack('I', -42) _struct.x = C() ``` The module `_struct` uses the module state to run `pack`. Therefore, the module state has to be alive until after the module has been cleared out to successfully run `C.__del__`. This happens at line 606, when `_PyImport_Cleanup` calls `_PyModule_Clear`. In fact, the loop that calls `_PyModule_Clear` has in its comments: > Now, if there are any modules left alive, clear their globals to minimize potential leaks. All C extension modules actually end up here, since they are kept alive in the interpreter state. That means that we can't clear the module state (which is used by C Extensions) before we run that loop. Moving `_PyInterpreterState_ClearModules` until after it, fixes the segfault in the code snippet. Finally, this updates a test in `io` to correctly assert the error that it now throws (since it now finds the io module state). The test that uses this is: `test_create_at_shutdown_without_encoding`. Given this test is now working is a proof that the module state now stays alive even when `__del__` is called at module destruction time. Thus, I didn't add a new tests for this. https://bugs.python.org/issue38076
* bpo-39487: Merge duplicated _Py_IDENTIFIER identifiers in C code (GH-18254)Hai Shi2020-01-301-4/+3
| | | Moving repetitive `_Py_IDENTIFIER` instances to a global location helps identify them more easily in regards to sub-interpreter support.
generated by cgit v0.12 at 2024-11-23 16:53:20 (GMT)