[WIP] [PY 3.8] fix more warnings

Several locations with simple usage of deprecated "imp" module
changed to use "importlib". These match with work in #3159,
but this is not a complete implementation of #3159.

More regex patterns are changed to be raw strings.

Some strings which did not seem appropriate to change to raw
strings (e.g. contain embedded tabs, which Python should honor)
had backslashes escaped to avoid accidental Python interpretation.
Example:
   '\t<Import Project="$(VCTargetsPath)\\Microsoft.Cpp.targets" />\n'
Python 3.8 was Warning \M was an unknown escape.

More open().write(), open().read() style usage changed to use
context managers so the file is closed.

WIP part: even with Python 3.7, the tests which call sconsign.py
fail; oddly they do not fail without the patch to compat.py.
sconsign.py does an import using imp module (which is what
generates the errors) so needs to be updated anyway.  It does not
quite fit the "simple usage" pattern - can't do a simple relative
import since sconsign is normally located elsewhere in the tree than
the main scons code body.

With this version of the patch, 700 tests now pass with 3.8, and
Warning messages reduced to 2800 (current master has 200 pass,
9000 warns)

Signed-off-by: Mats Wichmann <mats@linux.com>

1 files changed, 41 insertions, 20 deletions

diff --git a/bin/SConsDoc.py b/bin/SConsDoc.py
index ed9c607..0c78525 100644
--- a/bin/SConsDoc.py
+++ b/bin/SConsDoc.py
@@ -112,7 +112,6 @@ Tool example:
     </tool>
 """
 
-import imp
 import os.path
 import re
 import sys
@@ -801,25 +800,47 @@ class SConsDocHandler(object):
         self.parseDomtree(t.root, t.xpath_context, t.nsmap)
         
 # lifted from Ka-Ping Yee's way cool pydoc module.
-def importfile(path):
-    """Import a Python source file or compiled file given its path."""
-    magic = imp.get_magic()
-    file = open(path, 'r')
-    if file.read(len(magic)) == magic:
-        kind = imp.PY_COMPILED
-    else:
-        kind = imp.PY_SOURCE
-    file.close()
-    filename = os.path.basename(path)
-    name, ext = os.path.splitext(filename)
-    file = open(path, 'r')
-    try:
-        module = imp.load_module(name, file, path, (ext, 'r', kind))
-    except ImportError as e:
-        sys.stderr.write("Could not import %s: %s\n" % (path, e))
-        return None
-    file.close()
-    return module
+if sys.version_info[0] == 2:
+    def importfile(path):
+        """Import a Python source file or compiled file given its path."""
+        import imp
+        magic = imp.get_magic()
+        file = open(path, 'r')
+        if file.read(len(magic)) == magic:
+            kind = imp.PY_COMPILED
+        else:
+            kind = imp.PY_SOURCE
+        file.close()
+        filename = os.path.basename(path)
+        name, ext = os.path.splitext(filename)
+        file = open(path, 'r')
+        try:
+            module = imp.load_module(name, file, path, (ext, 'r', kind))
+        except ImportError as e:
+            sys.stderr.write("Could not import %s: %s\n" % (path, e))
+            return None
+        file.close()
+        return module
+
+else:  # PY3 version, from newer pydoc
+    def importfile(path):
+        """Import a Python source file or compiled file given its path."""
+        import importlib
+        magic = importlib.util.MAGIC_NUMBER
+        with open(path, 'rb') as file:
+            is_bytecode = magic == file.read(len(magic))
+        filename = os.path.basename(path)
+        name, ext = os.path.splitext(filename)
+        if is_bytecode:
+            loader = importlib._bootstrap_external.SourcelessFileLoader(name, path)
+        else:
+            loader = importlib._bootstrap_external.SourceFileLoader(name, path)
+        # XXX We probably don't need to pass in the loader here.
+        spec = importlib.util.spec_from_file_location(name, path, loader=loader)
+        try:
+            return importlib._bootstrap._load(spec)
+        except:
+            raise ErrorDuringImport(path, sys.exc_info())
 
 # Local Variables:
 # tab-width:4