summaryrefslogtreecommitdiffstats
path: root/Include/cpython/unicodeobject.h
Commit message (Collapse)AuthorAgeFilesLines
* gh-112026: Restore removed private C API (#112115)Victor Stinner2023-11-151-0/+131
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Restore removed private C API functions, macros and structures which have no simple replacement for now: * _PyDict_GetItem_KnownHash() * _PyDict_NewPresized() * _PyHASH_BITS * _PyHASH_IMAG * _PyHASH_INF * _PyHASH_MODULUS * _PyHASH_MULTIPLIER * _PyLong_Copy() * _PyLong_FromDigits() * _PyLong_New() * _PyLong_Sign() * _PyObject_CallMethodId() * _PyObject_CallMethodNoArgs() * _PyObject_CallMethodOneArg() * _PyObject_CallOneArg() * _PyObject_EXTRA_INIT * _PyObject_FastCallDict() * _PyObject_GetAttrId() * _PyObject_Vectorcall() * _PyObject_VectorcallMethod() * _PyStack_AsDict() * _PyThread_CurrentFrames() * _PyUnicodeWriter structure * _PyUnicodeWriter_Dealloc() * _PyUnicodeWriter_Finish() * _PyUnicodeWriter_Init() * _PyUnicodeWriter_Prepare() * _PyUnicodeWriter_PrepareKind() * _PyUnicodeWriter_WriteASCIIString() * _PyUnicodeWriter_WriteChar() * _PyUnicodeWriter_WriteLatin1String() * _PyUnicodeWriter_WriteStr() * _PyUnicodeWriter_WriteSubstring() * _PyUnicode_AsString() * _PyUnicode_FromId() * _PyVectorcall_Function() * _Py_HashDouble() * _Py_HashPointer() * _Py_IDENTIFIER() * _Py_c_abs() * _Py_c_diff() * _Py_c_neg() * _Py_c_pow() * _Py_c_prod() * _Py_c_quot() * _Py_c_sum() * _Py_static_string() * _Py_static_string_init()
* gh-111089: Revert PyUnicode_AsUTF8() changes (#111833)Victor Stinner2023-11-071-0/+16
| | | | | | | | | | | | | | | | | | | | | * Revert "gh-111089: Use PyUnicode_AsUTF8() in Argument Clinic (#111585)" This reverts commit d9b606b3d04fc56fb0bcc479d7d6c14562edb5e2. * Revert "gh-111089: Use PyUnicode_AsUTF8() in getargs.c (#111620)" This reverts commit cde1071b2a72e8261ca66053ef61431b7f3a81fd. * Revert "gh-111089: PyUnicode_AsUTF8() now raises on embedded NUL (#111091)" This reverts commit d731579bfb9a497cfb0076cb6b221058a20088fe. * Revert "gh-111089: Add PyUnicode_AsUTF8() to the limited C API (#111121)" This reverts commit d8f32be5b6a736dc2fc9dca3f1bf176c82fc9b44. * Revert "gh-111089: Use PyUnicode_AsUTF8() in sqlite3 (#111122)" This reverts commit 37e4e20eaa8f27ada926d49e5971fecf0477ad26.
* gh-111089: Add PyUnicode_AsUTF8() to the limited C API (#111121)Victor Stinner2023-10-201-13/+0
| | | | | | | | Add PyUnicode_AsUTF8() function to the limited C API. multiprocessing posixshmem now uses PyUnicode_AsUTF8() instead of PyUnicode_AsUTF8AndSize(): the extension is built with the limited C API. The function now raises an exception if the filename contains an embedded null character instead of truncating silently the filename.
* gh-111089: PyUnicode_AsUTF8() now raises on embedded NUL (#111091)Victor Stinner2023-10-201-10/+10
| | | | | | | | | * PyUnicode_AsUTF8() now raises an exception if the string contains embedded null characters. * Update related C API tests (test_capi.test_unicode). * type_new_set_doc() uses PyUnicode_AsUTF8AndSize() to silently truncate doc containing null bytes. Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
* gh-106931: Intern Statically Allocated Strings Globally (gh-107272)Eric Snow2023-07-271-1/+3
| | | | | We tried this before with a dict and for all interned strings. That ran into problems due to interpreter isolation. However, exclusively using a per-interpreter cache caused some inconsistency that can eliminate the benefit of interning. Here we circle back to using a global cache, but only for statically allocated strings. We also use a more-basic _Py_hashtable_t for that global cache instead of a dict. Ideally we would only have the global cache, but the optional isolation of each interpreter's allocator means that a non-static string object must not outlive its interpreter. Thus we would have to store a copy of each such interned string in the global cache, tied to the main interpreter.
* gh-106320: Remove private _PyUnicode C API (#107185)Victor Stinner2023-07-241-36/+0
| | | | | | | | | | | | | | | Move private _PyUnicode functions to the internal C API (pycore_unicodeobject.h): * _PyUnicode_IsCaseIgnorable() * _PyUnicode_IsCased() * _PyUnicode_IsXidContinue() * _PyUnicode_IsXidStart() * _PyUnicode_ToFoldedFull() * _PyUnicode_ToLowerFull() * _PyUnicode_ToTitleFull() * _PyUnicode_ToUpperFull() No longer export these functions.
* gh-106320: Remove private _PyUnicode_AsString() alias (#107021)Victor Stinner2023-07-221-5/+0
| | | | | | | | Remove private _PyUnicode_AsString() alias to PyUnicode_AsUTF8(). It was kept for backward compatibility with Python 3.0 - 3.2. The PyUnicode_AsUTF8() is available since Python 3.3. The PyUnicode_AsUTF8String() function can be used to keep compatibility with Python 3.2 and older.
* gh-106320: Remove _PyUnicode_TransformDecimalAndSpaceToASCII() (#106398)Victor Stinner2023-07-041-37/+0
| | | | | | | | Remove private _PyUnicode_TransformDecimalAndSpaceToASCII() and other private _PyUnicode C API functions: move them to the internal C API (pycore_unicodeobject.h). No longer most of these functions. Replace _testcapi.unicode_transformdecimalandspacetoascii() with _testinternal._PyUnicode_TransformDecimalAndSpaceToASCII().
* gh-106320: Remove private _PyUnicode codecs C API functions (#106385)Victor Stinner2023-07-041-106/+0
| | | | | Remove private _PyUnicode codecs C API functions: move them to the internal C API (pycore_unicodeobject.h). No longer export most of these functions.
* gh-106320: Remove more private _PyUnicode C API functions (#106382)Victor Stinner2023-07-031-69/+0
| | | | | | Remove more private _PyUnicode C API functions: move them to the internal C API (pycore_unicodeobject.h). No longer export most pycore_unicodeobject.h functions.
* gh-106320: Move _PyUnicodeWriter to the internal C API (#106342)Victor Stinner2023-07-031-139/+0
| | | | | | Move also _PyUnicode_FormatAdvancedWriter(). CJK codecs and multibytecodec.c now define the Py_BUILD_CORE_MODULE macro.
* gh-105156: Cleanup usage of old Py_UNICODE type (#105158)Victor Stinner2023-06-011-5/+3
| | | | | | | | | | | | * refcounts.dat: * Remove Py_UNICODE functions. * Replace Py_UNICODE argument type with wchar_t. * _PyUnicode_ToLowercase(), _PyUnicode_ToUppercase(), _PyUnicode_ToTitlecase() are no longer deprecated in comments. It's no longer needed since they now use Py_UCS4 type, rather than the deprecated Py_UNICODE type. * gdb: Remove unused char_width() method.
* gh-105156: Deprecate the old Py_UNICODE type in C API (#105157)Victor Stinner2023-06-011-2/+2
| | | | | | | | Deprecate the old Py_UNICODE and PY_UNICODE_TYPE types in the C API: use wchar_t instead. Replace Py_UNICODE with wchar_t in multiple C files. Co-authored-by: Inada Naoki <songofacandy@gmail.com>
* gh-84436: Implement Immortal Objects (gh-19474)Eddie Elizondo2023-04-221-4/+13
| | | | | | | | | This is the implementation of PEP683 Motivation: The PR introduces the ability to immortalize instances in CPython which bypasses reference counting. Tagging objects as immortal allows up to skip certain operations when we know that the object will be around for the entire execution of the runtime. Note that this by itself will bring a performance regression to the runtime due to the extra reference count checks. However, this brings the ability of having truly immutable objects that are useful in other contexts such as immutable data sharing between sub-interpreters.
* PyUnicode_KIND() uses _Py_RVALUE() (#100060)Victor Stinner2022-12-061-1/+1
| | | | The PyUnicode_KIND() macro is modified to use _Py_RVALUE(), so it can no longer be used as a l-value.
* gh-89653: PEP 670: Convert macros to functions (#99843)Victor Stinner2022-11-281-2/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Convert macros to static inline functions to avoid macro pitfalls, like duplication of side effects: * DK_ENTRIES() * DK_UNICODE_ENTRIES() * PyCode_GetNumFree() * PyFloat_AS_DOUBLE() * PyInstanceMethod_GET_FUNCTION() * PyMemoryView_GET_BASE() * PyMemoryView_GET_BUFFER() * PyMethod_GET_FUNCTION() * PyMethod_GET_SELF() * PySet_GET_SIZE() * _PyHeapType_GET_MEMBERS() Changes: * PyCode_GetNumFree() casts PyCode_GetNumFree.co_nfreevars from int to Py_ssize_t to be future proof, and because Py_ssize_t is commonly used in the C API. * PyCode_GetNumFree() doesn't cast its argument: the replaced macro already required the exact type PyCodeObject*. * Add assertions in some functions using "CAST" macros to check the arguments type when Python is built with assertions (debug build). * Remove an outdated comment in unicodeobject.h.
* gh-99706: unicodeobject: Fix padding in `PyASCIIObject.state` (GH-99707)David Hewitt2022-11-241-1/+1
|
* gh-98783: Fix crashes when `str` subclasses are used in `_PyUnicode_Equal` ↵Nikita Sobolev2022-10-301-1/+1
| | | | (#98806)
* gh-89653: PEP 670: Macros always cast arguments in cpython/ (#93766)Victor Stinner2022-06-131-46/+14
| | | | Header files in the Include/cpython/ are only included if the Py_LIMITED_API macro is not defined.
* gh-92536: Mark PyUnicode_READY() argument as unused (#93011)Wenzel Jakob2022-05-231-2/+2
|
* gh-89653: Add assertions on PyUnicode_READ() index (#92883)Victor Stinner2022-05-171-1/+9
| | | | Add assertions on the index argument of PyUnicode_READ(), PyUnicode_READ_CHAR() and PyUnicode_WRITE() functions.
* gh-89653: PEP 670: Fix PyUnicode_READ() cast (#92872)Victor Stinner2022-05-171-1/+2
| | | | _Py_CAST() cannot be used with a constant type: use _Py_STATIC_CAST() instead.
* gh-92781: Avoid mixing declarations and code in C API (#92783)Victor Stinner2022-05-151-2/+5
| | | | | Avoid mixing declarations and code in the C API to fix the compiler warning: "ISO C90 forbids mixed declarations and code" [-Werror=declaration-after-statement].
* gh-85858: Remove PyUnicode_InternImmortal() function (#92579)Victor Stinner2022-05-131-10/+3
| | | | | | | | | | | | | | | | | Remove the PyUnicode_InternImmortal() function and the SSTATE_INTERNED_IMMORTAL macro. The PyUnicode_InternImmortal() function is still exported in the stable ABI. The function is removed from the API. PyASCIIObject.state.interned size is now a single bit, rather than 2 bits. Keep SSTATE_NOT_INTERNED and SSTATE_INTERNED_MORTAL macros for backward compatibility, but no longer use them internally since the interned member is now a single bit and so can only have two values (interned or not interned). Update stats of _PyUnicode_ClearInterned().
* gh-89653: Use int type for Unicode kind (#92704)Victor Stinner2022-05-131-2/+2
| | | | Use the same type that PyUnicode_FromKindAndData() kind parameter type (public C API): int.
* gh-89653: PEP 670: Convert PyUnicode_KIND() macro to function (#92705)Victor Stinner2022-05-131-1/+15
| | | | | | | | In the limited C API version 3.12, PyUnicode_KIND() is now implemented as a static inline function. Keep the macro for the regular C API and for the limited C API version 3.11 and older to prevent introducing new compiler warnings. Update _decimal.c and stringlib/eq.h for PyUnicode_KIND().
* gh-92536: PEP 623: Remove wstr and legacy APIs from Unicode (GH-92537)Inada Naoki2022-05-121-219/+14
|
* gh-89653: PEP 670: unicodeobject.h uses _Py_CAST() (#92696)Victor Stinner2022-05-111-6/+12
| | | | | | | | | | Use _Py_CAST() and _Py_STATIC_CAST() in macros wrapping static inline functions of unicodeobject.h. Change also the kind type from unsigned int to int: same parameter type than PyUnicode_FromKindAndData(). The limited API version 3.11 no longer casts arguments to expected types.
* gh-89653: Add assertions to unicodeobject.h functions (#92692)Victor Stinner2022-05-111-2/+8
|
* gh-89653: PEP 670: Convert unicodeobject.h macros to functions (#92648)Victor Stinner2022-05-111-48/+66
| | | | | | | | | | | | | | | | | | | | Convert the following Unicode macros to static inline functions. Surrogate functions: * Py_UNICODE_IS_SURROGATE() * Py_UNICODE_IS_HIGH_SURROGATE() * Py_UNICODE_IS_LOW_SURROGATE() * Py_UNICODE_HIGH_SURROGATE() * Py_UNICODE_LOW_SURROGATE() * Py_UNICODE_JOIN_SURROGATES() "Is" functions: * Py_UNICODE_ISALNUM() * Py_UNICODE_ISSPACE() In the implementation of these functions, the character type is now well defined to Py_UCS4.
* gh-91321: Add _Py_NULL macro (#92253)Victor Stinner2022-05-031-3/+3
| | | | | | | | Fix C++ compiler warnings: "zero as null pointer constant" (clang -Wzero-as-null-pointer-constant). * Add the _Py_NULL macro used by static inline functions to use nullptr in C++. * Replace NULL with nullptr in _testcppext.cpp.
* gh-91320: Fix more old-style cast warnings in C++ (#92247)Victor Stinner2022-05-031-15/+16
| | | | | | | Use _Py_CAST(), _Py_STATIC_CAST() and _PyASCIIObject_CAST() in static inline functions to fix C++ compiler warnings: "use of old-style cast" (clang -Wold-style-cast). test_cppext now builds the C++ test extension with -Wold-style-cast.
* gh-92135: Rename _Py_reinterpret_cast() to _Py_CAST() (#92230)Victor Stinner2022-05-031-3/+3
| | | Rename also _Py_static_cast() to _Py_STATIC_CAST().
* gh-91320: Add _Py_reinterpret_cast() macro (#91959)Victor Stinner2022-04-271-3/+6
| | | | | | | | | | | | | | Fix C++ compiler warnings about "old-style cast" (g++ -Wold-style-cast) in the Python C API. Use C++ reinterpret_cast<> and static_cast<> casts when the Python C API is used in C++. Example of fixed warning: Include/object.h:107:43: error: use of old-style cast to ‘PyObject*’ {aka ‘struct _object*’} [-Werror=old-style-cast] #define _PyObject_CAST(op) ((PyObject*)(op)) Add _Py_reinterpret_cast() and _Py_static_cast() macros.
* gh-89653: PEP 670: Functions don't cast pointers (#91697)Victor Stinner2022-04-251-17/+47
| | | | | | | | | | | | | | | | | | | | | In the limited C API version 3.11 and newer, the following functions no longer cast their object pointer argument with _PyObject_CAST() or _PyObject_CAST_CONST(): * Py_REFCNT(), Py_TYPE(), Py_SIZE() * Py_SET_REFCNT(), Py_SET_TYPE(), Py_SET_SIZE() * Py_IS_TYPE() * Py_INCREF(), Py_DECREF() * Py_XINCREF(), Py_XDECREF() * Py_NewRef(), Py_XNewRef() * PyObject_TypeCheck() * PyType_Check() * PyType_CheckExact() Split Py_DECREF() implementation in 3 versions to make the code more readable. Update the xxlimited.c extension, which uses the limited C API version 3.11, to pass PyObject* to these functions.
* gh-80527: Deprecate PEP 623 Unicode functions (#91801)Victor Stinner2022-04-221-5/+11
| | | | | | | | | | | Deprecate functions: * PyUnicode_AS_DATA() * PyUnicode_AS_UNICODE() * PyUnicode_GET_DATA_SIZE() * PyUnicode_GET_SIZE() Previously, these functions were macros and so it wasn't possible to decorate them with Py_DEPRECATED().
* gh-89653: Add assertions to unicodeobject.h functions (#91800)Victor Stinner2022-04-211-0/+7
|
* gh-89653: PEP 670: Convert unicodeobject.h macros to functions (#91799)Victor Stinner2022-04-211-18/+44
| | | | | | | | | | | | Convert unicodeobject.h macros to static inline functions: * PyUnicode_AS_DATA() * PyUnicode_AS_UNICODE() * PyUnicode_GET_DATA_SIZE() * PyUnicode_GET_SIZE() Static inline functions are wrapped by macros which casts arguments with _PyObject_CAST() to prevent introducing new compiler warnings when passing "const PyObject*".
* gh-89653: PEP 670: Group deprecated API in unicodeobject.h (#91796)Victor Stinner2022-04-211-56/+58
|
* gh-89653: PEP 670: Convert unicodeobject.h macros to functions (#91773)Victor Stinner2022-04-211-57/+74
| | | | | | | | | | | | | | | | | Convert unicodeobject.h macros to static inline functions: * PyUnicode_MAX_CHAR_VALUE() * PyUnicode_READ() * PyUnicode_READY() * PyUnicode_READ_CHAR() * PyUnicode_WRITE() Move PyUnicode_READY() after _PyUnicode_Ready(), since it uses _PyUnicode_Ready(). Static inline functions are wrapped by macros which casts arguments with _PyObject_CAST() and casts 'kind' arguments to "unsigned int" to prevent introducing new compiler warnings when passing "const PyObject*".
* gh-89653: PEP 670: Convert unicodeobject.h macros to functions (#91705)Victor Stinner2022-04-211-41/+61
| | | | | | | | | | | | | | | | | | Convert unicodeobject.h macros to static inline functions: * PyUnicode_CHECK_INTERNED() * PyUnicode_DATA(), _PyUnicode_COMPACT_DATA(), _PyUnicode_NONCOMPACT_DATA() * PyUnicode_GET_LENGTH() * PyUnicode_IS_ASCII() * PyUnicode_IS_COMPACT() * PyUnicode_IS_COMPACT_ASCII() * PyUnicode_IS_READY() Reorder functions to declare functions before their first usage. Static inline functions are wrapped by macros which casts arguments with _PyObject_CAST() to prevent introducing new compiler warnings when passing "const PyObject*".
* bpo-47164: Add _PyASCIIObject_CAST() macro (GH-32191)Victor Stinner2022-03-311-33/+34
| | | | | | | | | | | | Add macros to cast objects to PyASCIIObject*, PyCompactUnicodeObject* and PyUnicodeObject*: _PyASCIIObject_CAST(), _PyCompactUnicodeObject_CAST() and _PyUnicodeObject_CAST(). Using these new macros make the code more readable and check their argument with: assert(PyUnicode_Check(op)). Remove redundant assert(PyUnicode_Check(op)) in macros using directly or indirectly these new CAST macros. Replacing existing casts with these macros.
* bpo-45490: Rename static inline functions (GH-31217)Victor Stinner2022-02-111-2/+2
| | | | | | | | | | | | When a static inline function is wrapped by a macro which casts its arguments to the expected type, there is no need that the function has a different name than the macro. Use the same name for the macro and the function to avoid confusion. Rename _PyUnicode_get_wstr_length() to PyUnicode_WSTR_LENGTH(). Don't rename static inline _Py_NewRef() and _Py_XNewRef() functions, since the C API exports Py_NewRef() and Py_XNewRef() functions as regular functions. The name cannot be reused in this case.
* bpo-46007: Exclude PyUnicode_CHECK_INTERNED() from limited C API (GH-29987)Victor Stinner2021-12-091-0/+4
| | | | | | | Exclude the PyUnicode_CHECK_INTERNED() macro from the limited C API, because it uses the PyASCIIObject structure which is excluded from the limited C API. Automerge-Triggered-By: GH:encukou
* bpo-45885: Specialize COMPARE_OP (GH-29734)Dennis Sweeney2021-12-031-0/+3
| | | | | | | * Add COMPARE_OP_ADAPTIVE adaptive instruction. * Add COMPARE_OP_FLOAT_JUMP, COMPARE_OP_INT_JUMP and COMPARE_OP_STR_JUMP specialized instructions. * Introduce and use _PyUnicode_Equal
* bpo-45467: Fix IncrementalDecoder and StreamReader in the ↵Serhiy Storchaka2021-10-141-0/+10
| | | | | | | | | "raw-unicode-escape" codec (GH-28944) They support now splitting escape sequences between input chunks. Add the third parameter "final" in codecs.raw_unicode_escape_decode(). It is True by default to match the former behavior.
* bpo-45461: Fix IncrementalDecoder and StreamReader in the "unicode-escape" ↵Serhiy Storchaka2021-10-141-1/+9
| | | | | | | | | codec (GH-28939) They support now splitting escape sequences between input chunks. Add the third parameter "final" in codecs.unicode_escape_decode(). It is True by default to match the former behavior.
* bpo-41123: Remove Py_UNICODE_COPY() and Py_UNICODE_FILL() (GH-28887)Victor Stinner2021-10-111-13/+0
|
* [codemod] Fix non-matching bracket pairs (GH-28473)Mohamad Mansour2021-09-211-1/+1
| | | | | Co-authored-by: Terry Jan Reedy <tjreedy@udel.edu> Co-authored-by: Serhiy Storchaka <storchaka@gmail.com> Co-authored-by: Łukasz Langa <lukasz@langa.pl>
* bpo-44029: Remove Py_UNICODE APIs (GH-25881)Inada Naoki2021-05-071-140/+5
| | | | | | | | | | | | Remove deprecated `Py_UNICODE` APIs: `PyUnicode_Encode`, `PyUnicode_EncodeUTF7`, `PyUnicode_EncodeUTF8`, `PyUnicode_EncodeUTF16`, `PyUnicode_EncodeUTF32`, `PyUnicode_EncodeLatin1`, `PyUnicode_EncodeMBCS`, `PyUnicode_EncodeDecimal`, `PyUnicode_EncodeRawUnicodeEscape`, `PyUnicode_EncodeCharmap`, `PyUnicode_EncodeUnicodeEscape`, `PyUnicode_TransformDecimalToASCII`, `PyUnicode_TranslateCharmap`, `PyUnicodeEncodeError_Create`, `PyUnicodeTranslateError_Create`. See :pep:`393` and :pep:`624` for reference.
generated by cgit v0.12 at 2025-09-15 19:40:02 (GMT)