closes bpo-31650: PEP 552 (Deterministic pycs) implementation (#4575)

Python now supports checking bytecode cache up-to-dateness with a hash of the
source contents rather than volatile source metadata. See the PEP for details.

While a fairly straightforward idea, quite a lot of code had to be modified due
to the pervasiveness of pyc implementation details in the codebase. Changes in
this commit include:

- The core changes to importlib to understand how to read, validate, and
  regenerate hash-based pycs.

- Support for generating hash-based pycs in py_compile and compileall.

- Modifications to our siphash implementation to support passing a custom
  key. We then expose it to importlib through _imp.

- Updates to all places in the interpreter, standard library, and tests that
  manually generate or parse pyc files to grok the new format.

- Support in the interpreter command line code for long options like
  --check-hash-based-pycs.

- Tests and documentation for all of the above.

3 files changed, 20 insertions, 1 deletions

diff --git a/Include/internal/hash.h b/Include/internal/hash.h
new file mode 100644
index 0000000..e14b80a
--- /dev/null
+++ b/Include/internal/hash.h
@@ -0,0 +1,6 @@
+#ifndef Py_INTERNAL_HASH_H
+#define Py_INTERNAL_HASH_H
+
+uint64_t _Py_KeyedHash(uint64_t, const char *, Py_ssize_t);
+
+#endif
diff --git a/Include/internal/import.h b/Include/internal/import.h
new file mode 100644
index 0000000..4746e75
--- /dev/null
+++ b/Include/internal/import.h
@@ -0,0 +1,6 @@
+#ifndef Py_INTERNAL_IMPORT_H
+#define Py_INTERNAL_IMPORT_H
+
+extern const char *_Py_CheckHashBasedPycsMode;
+
+#endif
diff --git a/Include/pygetopt.h b/Include/pygetopt.h
index 962720c..3e3fe36 100644
--- a/Include/pygetopt.h
+++ b/Include/pygetopt.h
@@ -12,7 +12,14 @@ PyAPI_DATA(wchar_t *) _PyOS_optarg;
 
 PyAPI_FUNC(void) _PyOS_ResetGetOpt(void);
 
-PyAPI_FUNC(int) _PyOS_GetOpt(int argc, wchar_t **argv, wchar_t *optstring);
+typedef struct {
+    const wchar_t *name;
+    int has_arg;
+    int val;
+} _PyOS_LongOption;
+
+PyAPI_FUNC(int) _PyOS_GetOpt(int argc, wchar_t **argv, wchar_t *optstring,
+                             const _PyOS_LongOption *longopts, int *longindex);
 #endif /* !Py_LIMITED_API */
 
 #ifdef __cplusplus