bpo-34523: Support surrogatepass in locale codecs (GH-8995)

Add support for the "surrogatepass" error handler in
PyUnicode_DecodeFSDefault() and PyUnicode_EncodeFSDefault()
for the UTF-8 encoding.

Changes:

* _Py_DecodeUTF8Ex() and _Py_EncodeUTF8Ex() now support the
  surrogatepass error handler (_Py_ERROR_SURROGATEPASS).
* _Py_DecodeLocaleEx() and _Py_EncodeLocaleEx() now use
  the _Py_error_handler enum instead of "int surrogateescape" to pass
  the error handler. These functions now return -3 if the error
  handler is unknown.
* Add unit tests on _Py_DecodeLocaleEx() and _Py_EncodeLocaleEx()
  in test_codecs.
* Rename get_error_handler() to _Py_GetErrorHandler() and expose it
  as a private function.
* _freeze_importlib doesn't need config.filesystem_errors="strict"
  workaround anymore.

1 files changed, 0 insertions, 8 deletions

diff --git a/Programs/_freeze_importlib.c b/Programs/_freeze_importlib.c
index 2621a76..8830d13 100644
--- a/Programs/_freeze_importlib.c
+++ b/Programs/_freeze_importlib.c
@@ -82,14 +82,6 @@ main(int argc, char *argv[])
     /* Don't install importlib, since it could execute outdated bytecode. */
     config._install_importlib = 0;
     config._frozen = 1;
-#ifdef MS_WINDOWS
-    /* bpo-34523: initfsencoding() is not called if _install_importlib=0,
-       so interp->fscodec_initialized value remains 0.
-       PyUnicode_EncodeFSDefault() doesn't support the "surrogatepass" error
-       handler in such case, whereas it's the default error handler on Windows.
-       Force the "strict" error handler to work around this bootstrap issue. */
-    config.filesystem_errors = "strict";
-#endif
 
     _PyInitError err = _Py_InitializeFromConfig(&config);
     /* No need to call _PyCoreConfig_Clear() since we didn't allocate any