VS: Fix GenerateDebugInformation values for v140 and v141 toolsets

When VS 2015 was first released, its new v140 toolset came with a
`link.xml` file that changed the `GenerateDebugInformation` boolean
(`false` and `true`) value from earlier toolsets to an enumeration
consisting of the possible values `No`, `Debug`, and `DebugFastLink`.

We first adapted to this in commit v3.4.2~2^2 (VS: Fix VS 2015 .vcxproj
file value for GenerateDebugInformation, 2016-01-08), but that broke
older toolsets that still expected the boolean.  Then commit
v3.6.0-rc1~295^2~1 (VS: Fix VS 2015 .vcxproj debug setting for older
toolsets, 2016-02-24) added a hack to fix up the value based on the
toolset in use.  Several follow-up commits fixed this for more older
toolsets because our flag table was at the time based on the generator
in use rather than the toolset in use.

Since commit v3.8.0-rc1~396^2 (VS: Choose flag map based on the toolset
name, 2016-10-17) we use a flag table based on the toolset, so the fixup
hack should not be needed.  We had to keep it around only due to our
default value for GenerateDebugInformation (`false` or `No`) still being
based on the generator instead of the toolset.

A VS 2015 update was released that changed the v140 toolset `link.xml`
file back to using `false` and `true` for the `GenerateDebugInformation`
enumeration variants previously known as `No` and `Debug`.  In order to
know which pair to use, we need to parse the `link.xml` file for the
current toolset.

Switch back to using `false` and `true` unconditionally in our
`GenerateDebugInformation` flag table entries and default value.  With
that plus the toolset-based flag table, we now get incorrect values for
`GenerateDebugInformation` only when using a v140 toolset from an older
VS 2015 installation.  Detect this case by parsing `link.xml` and add
special logic to convert `false` and `true` to `No` and `Debug` to
satisfy the older toolset specification.

Inspired-by: Ian Hojnicki <nullref@live.com>
Fixes: #17020

1 files changed, 19 insertions, 0 deletions

diff --git a/Source/cmGlobalVisualStudio10Generator.cxx b/Source/cmGlobalVisualStudio10Generator.cxx
index 0e02b0a..e3835ff 100644
--- a/Source/cmGlobalVisualStudio10Generator.cxx
+++ b/Source/cmGlobalVisualStudio10Generator.cxx
@@ -130,6 +130,7 @@ cmGlobalVisualStudio10Generator::cmGlobalVisualStudio10Generator(
   this->DefaultNasmFlagTable = cmVS10NASMFlagTable;
   this->DefaultRcFlagTable = cmVS10RCFlagTable;
   this->Version = VS10;
+  this->PlatformToolsetNeedsDebugEnum = false;
 }
 
 bool cmGlobalVisualStudio10Generator::MatchesGeneratorName(
@@ -195,6 +196,24 @@ bool cmGlobalVisualStudio10Generator::SetGeneratorToolset(
     return false;
   }
 
+  if (cmHasLiteralPrefix(this->GetPlatformToolsetString(), "v140")) {
+    // The GenerateDebugInformation link setting for the v140 toolset
+    // in VS 2015 was originally an enum with "No" and "Debug" values,
+    // differing from the "false" and "true" values used in older toolsets.
+    // A VS 2015 update changed it back.  Parse the "link.xml" file to
+    // discover which one we need.
+    std::string const link_xml = this->VCTargetsPath + "/1033/link.xml";
+    cmsys::ifstream fin(link_xml.c_str());
+    std::string line;
+    while (fin && cmSystemTools::GetLineFromStream(fin, line)) {
+      if (line.find(" Switch=\"DEBUG\" ") != std::string::npos) {
+        this->PlatformToolsetNeedsDebugEnum =
+          line.find(" Name=\"Debug\" ") != std::string::npos;
+        break;
+      }
+    }
+  }
+
   if (this->GeneratorToolsetCuda.empty()) {
     // Find the highest available version of the CUDA tools.
     std::vector<std::string> cudaTools;