[PR 3462] python two handle the json

reading the cache from the json file got us unicode when
running python2, and this broke certain tests when calling
subprocess.Popen:  TypeError, because all the environment
wasn't strings, some was unicode.

Signed-off-by: Mats Wichmann <mats@linux.com>

3 files changed, 31 insertions, 8 deletions

diff --git a/.appveyor.yml b/.appveyor.yml
index ea6ad36..09cd996 100644
--- a/.appveyor.yml
+++ b/.appveyor.yml
@@ -25,7 +25,7 @@ install:
   - cmd: set STATIC_DEPS=true & C:\\%WINPYTHON%\\python.exe -m pip install -U --progress-bar off lxml
   # install 3rd party tools to test with
   - cmd: choco install --allow-empty-checksums dmd ldc swig vswhere xsltproc winflexbison
-  - cmd: set SCONS_CACHE_MSVC_CONFIG="true"
+  - cmd: set SCONS_CACHE_MSVC_CONFIG=true
   - cmd: set
 
   ### LINUX ###
diff --git a/doc/man/scons.xml b/doc/man/scons.xml
index c406b76..e210e76 100644
--- a/doc/man/scons.xml
+++ b/doc/man/scons.xml
@@ -7191,7 +7191,7 @@ env.Program('MyApp', ['Foo.cpp', 'Bar.cpp'])
 <refsect1 id='environment'><title>ENVIRONMENT</title>
 
 <para>In general, &scons; is not controlled by environment
-variables set shell used to invoke it, leaving it
+variables set in the shell used to invoke it, leaving it
 up to the SConscript file author to import those if desired.
 However the following variables are imported by
 &scons; itself if set:
@@ -7220,13 +7220,24 @@ in addition to those passed on the command line.</para>
     <listitem>
 <para>If set, save the shell environment variables generated
 in setting up the Microsoft Visual C++ compiler (and/or Build Tools),
-to give these settings (expensive to generate) persistence
-between runs of &scons;.
-If set to a True-like value (<literal>1</literal>,
-<literal>true</literal> or
-<literal>True</literal>) will cache to a file named
+to give these settings, which are expensive to generate, persistence
+across &scons; invocations.
+If set to a True-like value (<literal>"1"</literal>,
+<literal>"true"</literal> or
+<literal>"True"</literal>) will cache to a file named
 <filename>.scons_msvc_cache</filename> in the user's home directory.
 If set to a pathname, will use that pathname for the cache.</para>
+<para>
+Note: this behavior is not enabled by default because it
+might be somewhat fragile: while each major tool version
+(e.g. Visual Studio 2018 vs 2019) will get separate
+entries, if toolset updates cause a change
+to settings within a given release series, &scons; will not
+detect this and will fetch old settings.
+In case of problems, just remove the cache file.
+Use of this option is primarily intended to aid performance
+for tightly controlled Continuous Integration setups.
+</para>
     </listitem>
   </varlistentry>
 </variablelist>
diff --git a/src/engine/SCons/Tool/MSCommon/common.py b/src/engine/SCons/Tool/MSCommon/common.py
index d4997eb..eaf57f9 100644
--- a/src/engine/SCons/Tool/MSCommon/common.py
+++ b/src/engine/SCons/Tool/MSCommon/common.py
@@ -32,6 +32,8 @@ import json
 import os
 import subprocess
 import re
+import subprocess
+import sys
 
 import SCons.Util
 
@@ -56,13 +58,23 @@ CONFIG_CACHE = os.environ.get('SCONS_CACHE_MSVC_CONFIG')
 if CONFIG_CACHE in ('1', 'true', 'True'):
     CONFIG_CACHE = os.path.join(os.path.expanduser('~'), '.scons_msvc_cache')
 
+# !@$#@$? Python2
+if sys.version_info[0] == 2:
+    def str_hook(obj):
+        return {k.encode('utf-8') if isinstance(k, unicode) else k :
+                v.encode('utf-8') if isinstance(v, unicode) else v
+                for k,v in obj}
+
 def read_script_env_cache():
     """ fetch cached msvc env vars if requested, else return empty dict """
     envcache = {}
     if CONFIG_CACHE:
         try:
             with open(CONFIG_CACHE, 'r') as f:
-                envcache = json.load(f)
+                if sys.version_info[0] == 2:
+                    envcache = json.load(f, object_pairs_hook=str_hook)
+                else:
+                    envcache = json.load(f)
         #TODO can use more specific FileNotFoundError when py2 dropped
         except IOError:
             pass