diff options
| author | William Deegan <bill@baddogconsulting.com> | 2020-02-21 02:53:29 (GMT) |
|---|---|---|
| committer | GitHub <noreply@github.com> | 2020-02-21 02:53:29 (GMT) |
| commit | 0814deb56ca870f631a415a3255ce8dd99688e01 (patch) | |
| tree | 22fe52c634a5b1a483a9ad208b4e41b751a7392a /src | |
| parent | 420f23129a3eb298e20e2250f97eb8eefdd923d4 (diff) | |
| parent | 54c36335c575a98fcf6de2ff84c16b66efa7da5e (diff) | |
| download | SCons-0814deb56ca870f631a415a3255ce8dd99688e01.zip SCons-0814deb56ca870f631a415a3255ce8dd99688e01.tar.gz SCons-0814deb56ca870f631a415a3255ce8dd99688e01.tar.bz2 | |
Merge branch 'master' into topic/grossag/compoundactionscan
Diffstat (limited to 'src')
103 files changed, 1477 insertions, 1037 deletions
diff --git a/src/CHANGES.txt b/src/CHANGES.txt index 53f4d48..025e046 100755 --- a/src/CHANGES.txt +++ b/src/CHANGES.txt @@ -5,6 +5,8 @@ Change Log NOTE: The 4.0.0 Release of SCons will drop Python 2.7 Support +NOTE: Please add your name below in alphabetical order by last name. +NOTE: Please include a reference to any Issues resolved by your changes in the bullet(s) you add RELEASE VERSION/DATE TO BE FILLED IN LATER @@ -15,12 +17,39 @@ RELEASE VERSION/DATE TO BE FILLED IN LATER - Added C:\msys64\mingw64\bin to default mingw and clang windows PATH's. This is a reasonable default and also aligns with changes in Appveyor's VS2019 image. - Drop support for Python 2.7. SCons will be Python 3.5+ going forward. + - Change SCons.Node.ValueWithMemo to consider any name passed when memoizing Value() nodes + - Fix Github Issue #3550 - When using Substfile() with a value like Z:\mongo\build\install\bin + the implementation using re.sub() would end up interpreting the string and finding regex escape + characters where it should have been simply replacing existing text. Switched to use string.replace(). + - Fix Github Issue #2904 - Provide useful error message when more than one Configure Contexts are opened. + Only one open is allowed. You must call conf.Finish() to complete the currently open one before creating another + + From Jeremy Elson: + - Updated design doc to use the correct syntax for Depends() From Adam Gross: - Added support for scanning multiple entries in an action string if IMPLICIT_COMMAND_DEPENDENCIES is set to 2. This opts into more thorough action scanning where every string in the command is scanned to determine if it is a non-source and non-target path. + - Added support for taking instances of the Value class as implicit + dependencies. + - Added new module SCons.Scanner.Python to allow scanning .py files. + - Added support for explicitly passing a name when creating Value() nodes. This may be useful + when the value can't be converted to a string or if having a name is otherwise desirable. + + From Andrew Morrow: + - Fix Issue #3469 - Fixed improper reuse of temporary and compiled files by Configure when changing + the order and/or number of tests. This is done by using the hash of the generated temporary files + content and (For the target files) the hash of the action. + So where previously files would be named: + - config_1.c, config_1.o, config_1 + The will now be named (For example) + - conftest_68b375d16e812c43e6d72d6e93401e7c_0.c, + conftest_68b375d16e812c43e6d72d6e93401e7c_0_5713f09fc605f46b2ab2f7950455f187.o + or + conftest_68b375d16e812c43e6d72d6e93401e7c_0.o + conftest_68b375d16e812c43e6d72d6e93401e7c_0_5713f09fc605f46b2ab2f7950455f187 (for executable) From Mathew Robinson: - Improve performance of Subst by preventing unnecessary frame @@ -36,6 +65,16 @@ RELEASE VERSION/DATE TO BE FILLED IN LATER - str.format syntax errors fixed - a bunch of linter/checker syntax fixups - Convert remaining uses of insecure/deprecated mktemp method. + - Clean up some duplications in manpage. Clarify portion of manpage on Dir and File nodes. + - Reduce needless list conversions. + - Fixed regex in Python scanner. + - Accommodate VS 2017 Express - it's got a more liberal license then VS + Community, so some people prefer it (from 2019, no more Express) + - vswhere call should also now work even if programs aren't on the C: drive. + - Add an alternate warning message cl.exe is not found and msvc config + cache is in use (SCONS_CACHE_MSVC_CONFIG was given) - config cache + may be out of date. + - Script/Main.py now uses importlib instead of imp module. RELEASE 3.1.2 - Mon, 17 Dec 2019 02:06:27 +0000 diff --git a/src/README.txt b/src/README.txt index 40d7217..01ef436 100644 --- a/src/README.txt +++ b/src/README.txt @@ -22,13 +22,13 @@ LATEST VERSION Before going further, you can check that this package you have is the latest version by checking the SCons download page at: - http://www.scons.org/download.html + https://scons.org/pages/download.html EXECUTION REQUIREMENTS ====================== -Running SCons requires Python version 2.7.* or 3.5.* and above. There should be +Running SCons requires Python 3.5.* and above. There should be no other dependencies or requirements to run SCons. (There is, however, an additional requirement to *install* SCons from this particular package; see the next section.) @@ -193,10 +193,6 @@ You may subscribe to the mailing list by sending email to: scons-users-join@scons.org -There is also a low-volume mailing list available for announcements -about SCons. Subscribe by sending email to: - - announce-subscribe@scons.tigris.org There are other mailing lists available for SCons developers, for notification of SCons code changes, and for notification of updated @@ -243,7 +239,8 @@ many contributors, including but not at all limited to: - Anatoly Techtonik - Christoph Wiedemann - Russel Winder +- Mats Wichmann \... and many others. -Copyright (c) 2001 - 2019 The SCons Foundation +Copyright (c) 2001 - 2020 The SCons Foundation diff --git a/src/engine/MANIFEST.in b/src/engine/MANIFEST.in index 3125824..3c2c0c2 100644 --- a/src/engine/MANIFEST.in +++ b/src/engine/MANIFEST.in @@ -40,6 +40,7 @@ SCons/Scanner/Fortran.py SCons/Scanner/IDL.py SCons/Scanner/LaTeX.py SCons/Scanner/Prog.py +SCons/Scanner/Python.py SCons/Scanner/RC.py SCons/Scanner/SWIG.py SCons/SConf.py @@ -140,6 +141,7 @@ SCons/Tool/pdf.py SCons/Tool/pdflatex.py SCons/Tool/pdftex.py SCons/Tool/PharLapCommon.py +SCons/Tool/python.py SCons/Tool/qt.py SCons/Tool/rmic.py SCons/Tool/rpcgen.py diff --git a/src/engine/SCons/Action.py b/src/engine/SCons/Action.py index 3f5e4b8..8a8cf27 100644 --- a/src/engine/SCons/Action.py +++ b/src/engine/SCons/Action.py @@ -628,17 +628,9 @@ class _ActionAction(ActionBase): """ In python 3, and in some of our tests, sys.stdout is a String io object, and it takes unicode strings only - In other cases it's a regular Python 2.x file object - which takes strings (bytes), and if you pass those a - unicode object they try to decode with 'ascii' codec - which fails if the cmd line has any hi-bit-set chars. - This code assumes s is a regular string, but should - work if it's unicode too. + This code assumes s is a regular string. """ - try: - sys.stdout.write(s + u"\n") - except UnicodeDecodeError: - sys.stdout.write(s + "\n") + sys.stdout.write(s + "\n") def __call__(self, target, source, env, exitstatfunc=_null, @@ -677,7 +669,7 @@ class _ActionAction(ActionBase): source = executor.get_all_sources() t = ' and '.join(map(str, target)) l = '\n '.join(self.presub_lines(env)) - out = u"Building %s with action:\n %s\n" % (t, l) + out = "Building %s with action:\n %s\n" % (t, l) sys.stdout.write(out) cmd = None if show and self.strfunction: diff --git a/src/engine/SCons/ActionTests.py b/src/engine/SCons/ActionTests.py index 3303750..4784abf 100644 --- a/src/engine/SCons/ActionTests.py +++ b/src/engine/SCons/ActionTests.py @@ -37,10 +37,8 @@ class GlobalActFunc(object): pass -import collections import io import os -import re import sys import types import unittest @@ -339,14 +337,6 @@ class ActionTestCase(unittest.TestCase): # a singleton list returns the contained action test_positional_args(cmd_action, ["string"]) - try: - unicode - except NameError: - pass - else: - a2 = eval("SCons.Action.Action(u'string')") - assert isinstance(a2, SCons.Action.CommandAction), a2 - def line_action(a): assert isinstance(a, SCons.Action.CommandAction), a assert a.cmd_list == ["explicit", "command", "line"], a.cmd_list diff --git a/src/engine/SCons/Builder.py b/src/engine/SCons/Builder.py index e3fb396..3f0be63 100644 --- a/src/engine/SCons/Builder.py +++ b/src/engine/SCons/Builder.py @@ -100,7 +100,7 @@ There are the following methods for internal use within this module: __revision__ = "__FILE__ __REVISION__ __DATE__ __DEVELOPER__" -import collections +from collections import UserDict, UserList import SCons.Action import SCons.Debug @@ -197,7 +197,7 @@ class DictEmitter(SCons.Util.Selector): target, source = emitter(target, source, env) return (target, source) -class ListEmitter(collections.UserList): +class ListEmitter(UserList): """A callable list of emitters that calls each in sequence, returning the result. """ @@ -215,7 +215,7 @@ misleading_keywords = { 'sources' : 'source', } -class OverrideWarner(collections.UserDict): +class OverrideWarner(UserDict): """A class for warning about keyword arguments that we use as overrides in a Builder call. @@ -224,13 +224,13 @@ class OverrideWarner(collections.UserDict): warnings once, no matter how many Builders are invoked. """ def __init__(self, dict): - collections.UserDict.__init__(self, dict) + UserDict.__init__(self, dict) if SCons.Debug.track_instances: logInstanceCreation(self, 'Builder.OverrideWarner') self.already_warned = None def warn(self): if self.already_warned: return - for k in list(self.keys()): + for k in self.keys(): if k in misleading_keywords: alt = misleading_keywords[k] msg = "Did you mean to use `%s' instead of `%s'?" % (alt, k) @@ -647,6 +647,8 @@ class BuilderBase(object): env_kw = kw else: env_kw = self.overrides + + # TODO if env_kw: then the following line. there's no purpose in calling if no overrides. env = env.Override(env_kw) return self._execute(env, target, source, OverrideWarner(kw), ekw) diff --git a/src/engine/SCons/BuilderTests.py b/src/engine/SCons/BuilderTests.py index f28e201..d509219 100644 --- a/src/engine/SCons/BuilderTests.py +++ b/src/engine/SCons/BuilderTests.py @@ -20,8 +20,6 @@ # OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION # WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. # -from __future__ import print_function - __revision__ = "__FILE__ __REVISION__ __DATE__ __DEVELOPER__" import SCons.compat @@ -33,7 +31,7 @@ import SCons.compat def Func(): pass -import collections +from collections import UserList import io import os.path import re @@ -173,7 +171,7 @@ class MyNode_without_target_from_source(object): def builder_set(self, builder): self.builder = builder def has_builder(self): - return not self.builder is None + return self.builder is not None def set_explicit(self, is_explicit): self.is_explicit = is_explicit def has_explicit_builder(self): @@ -205,7 +203,7 @@ class BuilderTestCase(unittest.TestCase): """Test simple Builder creation """ builder = SCons.Builder.Builder(action="foo") - assert not builder is None, builder + assert builder is not None, builder builder = SCons.Builder.Builder(action="foo", OVERRIDE='x') x = builder.overrides['OVERRIDE'] assert x == 'x', x @@ -256,7 +254,7 @@ class BuilderTestCase(unittest.TestCase): assert not hasattr(n2, 'env') l = [1] - ul = collections.UserList([2]) + ul = UserList([2]) try: l.extend(ul) except TypeError: @@ -308,22 +306,6 @@ class BuilderTestCase(unittest.TestCase): #be = target.get_build_env() #assert be['VAR'] == 'foo', be['VAR'] - try: unicode - except NameError: - uni = str - else: - uni = unicode - - target = builder(env, target = uni('n12 n13'), - source = [uni('n14 n15')])[0] - assert target.name == uni('n12 n13') - assert target.sources[0].name == uni('n14 n15') - - target = builder(env, target = [uni('n16 n17')], - source = uni('n18 n19'))[0] - assert target.name == uni('n16 n17') - assert target.sources[0].name == uni('n18 n19') - n20 = MyNode_without_target_from_source('n20') flag = 0 try: @@ -429,7 +411,7 @@ class BuilderTestCase(unittest.TestCase): return Foo(target) builder = SCons.Builder.Builder(target_factory = FooFactory) assert builder.target_factory is FooFactory - assert not builder.source_factory is FooFactory + assert builder.source_factory is not FooFactory def test_source_factory(self): """Test a Builder that creates source nodes of a specified class @@ -440,7 +422,7 @@ class BuilderTestCase(unittest.TestCase): global Foo return Foo(source) builder = SCons.Builder.Builder(source_factory = FooFactory) - assert not builder.target_factory is FooFactory + assert builder.target_factory is not FooFactory assert builder.source_factory is FooFactory def test_splitext(self): @@ -737,7 +719,7 @@ class BuilderTestCase(unittest.TestCase): with open(str(t), 'w') as f: f.write("function2\n") for t in tlist: - if not t in list(map(str, target)): + if t not in list(map(str, target)): with open(t, 'w') as f: f.write("function2\n") return 1 @@ -768,7 +750,7 @@ class BuilderTestCase(unittest.TestCase): with open(str(t), 'w') as f: f.write("function3\n") for t in tlist: - if not t in list(map(str, target)): + if t not in list(map(str, target)): with open(t, 'w') as f: f.write("function3\n") return 1 @@ -821,7 +803,7 @@ class BuilderTestCase(unittest.TestCase): assert s == ['aaa.bar'], s builder3 = SCons.Builder.Builder(action='bld3') - assert not builder3.src_builder is builder1.src_builder + assert builder3.src_builder is not builder1.src_builder builder4 = SCons.Builder.Builder(action='bld4', src_suffix='.i', diff --git a/src/engine/SCons/Conftest.py b/src/engine/SCons/Conftest.py index c24adf8..4491884 100644 --- a/src/engine/SCons/Conftest.py +++ b/src/engine/SCons/Conftest.py @@ -315,8 +315,8 @@ int main(void) { return ret -def CheckHeader(context, header_name, header = None, language = None, - include_quotes = None): +def CheckHeader(context, header_name, header=None, language=None, + include_quotes=None): """ Configure check for a C or C++ header file "header_name". Optional "header" can be defined to do something before including the diff --git a/src/engine/SCons/Defaults.py b/src/engine/SCons/Defaults.py index a69d8b0..2bac41e 100644 --- a/src/engine/SCons/Defaults.py +++ b/src/engine/SCons/Defaults.py @@ -31,8 +31,6 @@ from distutils.msvccompiler. # OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION # WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. # -from __future__ import division - __revision__ = "__FILE__ __REVISION__ __DATE__ __DEVELOPER__" diff --git a/src/engine/SCons/DefaultsTests.py b/src/engine/SCons/DefaultsTests.py index 2cbad70..34941bc 100644 --- a/src/engine/SCons/DefaultsTests.py +++ b/src/engine/SCons/DefaultsTests.py @@ -29,8 +29,6 @@ import os import sys import unittest -from collections import UserDict - import TestCmd import SCons.Errors diff --git a/src/engine/SCons/Environment.py b/src/engine/SCons/Environment.py index 3e23196..00379e1 100644 --- a/src/engine/SCons/Environment.py +++ b/src/engine/SCons/Environment.py @@ -584,7 +584,7 @@ class SubstitutionEnvironment(object): out,err = p.communicate() status = p.wait() if err: - sys.stderr.write(u"" + err) + sys.stderr.write("" + err) if status: raise OSError("'%s' exited %d" % (command, status)) return out @@ -1854,7 +1854,7 @@ class Base(SubstitutionEnvironment): uniq = {} for executor in [n.get_executor() for n in nodes]: uniq[executor] = 1 - for executor in list(uniq.keys()): + for executor in uniq.keys(): executor.add_pre_action(action) return nodes @@ -1864,7 +1864,7 @@ class Base(SubstitutionEnvironment): uniq = {} for executor in [n.get_executor() for n in nodes]: uniq[executor] = 1 - for executor in list(uniq.keys()): + for executor in uniq.keys(): executor.add_post_action(action) return nodes @@ -2220,10 +2220,10 @@ class Base(SubstitutionEnvironment): else: return [self.subst(arg)] - def Value(self, value, built_value=None): + def Value(self, value, built_value=None, name=None): """ """ - return SCons.Node.Python.ValueWithMemo(value, built_value) + return SCons.Node.Python.ValueWithMemo(value, built_value, name) def VariantDir(self, variant_dir, src_dir, duplicate=1): variant_dir = self.arg2nodes(variant_dir, self.fs.Dir)[0] diff --git a/src/engine/SCons/Environment.xml b/src/engine/SCons/Environment.xml index 35e4bd8..e1d5e74 100644 --- a/src/engine/SCons/Environment.xml +++ b/src/engine/SCons/Environment.xml @@ -65,15 +65,6 @@ env['BUILDERS']['NewBuilder'] = foo </summary> </cvar> -<cvar name="Dir"> -<summary> -<para> -A function that converts a string -into a Dir instance relative to the target being built. -</para> -</summary> -</cvar> - <cvar name="ENV"> <summary> <para> @@ -130,15 +121,6 @@ env = Environment(ENV = {'PATH' : os.environ['PATH']}) </summary> </cvar> -<cvar name="File"> -<summary> -<para> -A function that converts a string into a File instance relative to the -target being built. -</para> -</summary> -</cvar> - <cvar name="SCANNERS"> <summary> <para> @@ -460,6 +442,8 @@ including another alias. can be called multiple times for the same alias to add additional targets to the alias, or additional actions to the list for this alias. +Aliases are global even if set through +the construction environment method. </para> <para> @@ -1163,17 +1147,16 @@ env.Decider('content') </example_commands> <para> -In addition to the above already-available functions, -the +In addition to the above already-available functions, the <varname>function</varname> -argument may be an actual Python function -that takes the following three arguments: +argument may be a Python function you supply. +Such a function must accept the following four arguments: </para> <para> <variablelist> <varlistentry> -<term><parameter>dependency</parameter></term> +<term><parameter class="function">dependency</parameter></term> <listitem> <para> The Node (file) which @@ -1187,7 +1170,7 @@ was built. </listitem> </varlistentry> <varlistentry> -<term><parameter>target</parameter></term> +<term><parameter class="function">target</parameter></term> <listitem> <para> The Node (file) being built. @@ -1200,7 +1183,7 @@ has "changed." </listitem> </varlistentry> <varlistentry> -<term><parameter>prev_ni</parameter></term> +<term><parameter class="function">prev_ni</parameter></term> <listitem> <para> Stored information about the state of the @@ -1216,12 +1199,17 @@ size, or content signature. </listitem> </varlistentry> <varlistentry> -<term><parameter>repo_node</parameter></term> +<term><parameter class="function">repo_node</parameter></term> <listitem> <para> -Use this node instead of the one specified by +If set, use this Node instead of the one specified by <varname>dependency</varname> - to determine if the dependency has changed. +to determine if the dependency has changed. +This argument is optional so should be written +as a default argument (typically it would be +written as <literal>repo_node=None</literal>). +A caller will normally only set this if the +target only exists in a Repository. </para> </listitem> </varlistentry> @@ -1351,11 +1339,10 @@ cc_values = env.Dictionary('CC', 'CCFLAGS', 'CCCOM') </arguments> <summary> <para> -This returns a Directory Node, -an object that represents the specified directory -<varname>name</varname>. +Returns Directory Node(s). +A Directory Node is an object that represents a directory. <varname>name</varname> -can be a relative or absolute path. +can be a relative or absolute path or a list of such paths. <varname>directory</varname> is an optional directory that will be used as the parent directory. If no @@ -1366,7 +1353,10 @@ is specified, the current script's directory is used as the parent. <para> If <varname>name</varname> -is a list, SCons returns a list of Dir nodes. +is a single pathname, the corresponding node is returned. +If +<varname>name</varname> +is a list, SCons returns a list of nodes. Construction variables are expanded in <varname>name</varname>. </para> @@ -1511,20 +1501,24 @@ if Execute("mkdir sub/dir/ectory"): </arguments> <summary> <para> -This returns a -File Node, -an object that represents the specified file -<varname>name</varname>. +Returns File Node(s). +A File Node is an object that represents a file. <varname>name</varname> -can be a relative or absolute path. +can be a relative or absolute path or a list of such paths. <varname>directory</varname> is an optional directory that will be used as the parent directory. +If no +<varname>directory</varname> +is specified, the current script's directory is used as the parent. </para> <para> If <varname>name</varname> -is a list, SCons returns a list of File nodes. +is a single pathname, the corresponding node is returned. +If +<varname>name</varname> +is a list, SCons returns a list of nodes. Construction variables are expanded in <varname>name</varname>. </para> @@ -3087,7 +3081,7 @@ env.Tool('opengl', toolpath = ['build/tools']) <scons_function name="Value"> <arguments> -(value, [built_value]) +(value, [built_value], [name]) </arguments> <summary> <para> @@ -3102,6 +3096,10 @@ will be rebuilt. files are up-to-date.) When using timestamp source signatures, Value Nodes' timestamps are equal to the system time when the Node is created. +<varname>name</varname> can be provided as an alternative name +for the resulting <literal>Value</literal> node; this is advised +if the <varname>value</varname> parameter can't be converted to +a string. </para> <para> diff --git a/src/engine/SCons/EnvironmentTests.py b/src/engine/SCons/EnvironmentTests.py index f3779c7..a9ec674 100644 --- a/src/engine/SCons/EnvironmentTests.py +++ b/src/engine/SCons/EnvironmentTests.py @@ -20,9 +20,6 @@ # OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION # WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. # - -from __future__ import print_function - __revision__ = "__FILE__ __REVISION__ __DATE__ __DEVELOPER__" import SCons.compat @@ -236,7 +233,7 @@ class SubstitutionTestCase(unittest.TestCase): """ env = SubstitutionEnvironment(XXX = 'x') assert 'XXX' in env - assert not 'YYY' in env + assert 'YYY' not in env def test_items(self): """Test the SubstitutionEnvironment items() method @@ -265,11 +262,6 @@ class SubstitutionTestCase(unittest.TestCase): assert isinstance(nodes[0], X) assert nodes[0].name == "Util.py UtilTests.py", nodes[0].name - nodes = env.arg2nodes(u"Util.py UtilTests.py", Factory) - assert len(nodes) == 1, nodes - assert isinstance(nodes[0], X) - assert nodes[0].name == u"Util.py UtilTests.py", nodes[0].name - nodes = env.arg2nodes(["Util.py", "UtilTests.py"], Factory) assert len(nodes) == 2, nodes assert isinstance(nodes[0], X) @@ -1759,7 +1751,7 @@ def exists(env): env2.Dictionary('ZZZ')[5] = 6 assert env1.Dictionary('XXX') is env2.Dictionary('XXX') assert 4 in env2.Dictionary('YYY') - assert not 4 in env1.Dictionary('YYY') + assert 4 not in env1.Dictionary('YYY') assert 5 in env2.Dictionary('ZZZ') assert 5 not in env1.Dictionary('ZZZ') @@ -3289,6 +3281,10 @@ def generate(env): v3 = env.Value('c', 'build-c') assert v3.value == 'c', v3.value + v4 = env.Value(b'\x00\x0F', name='name') + assert v4.value == b'\x00\x0F', v4.value + assert v4.name == 'name', v4.name + def test_Environment_global_variable(self): """Test setting Environment variable to an Environment.Base subclass""" @@ -3547,8 +3543,8 @@ class OverrideEnvironmentTestCase(unittest.TestCase,TestEnvironmentFixture): assert 'YYY' in env assert 'YYY' in env2 assert 'YYY' in env3 - assert not 'ZZZ' in env - assert not 'ZZZ' in env2 + assert 'ZZZ' not in env + assert 'ZZZ' not in env2 assert 'ZZZ' in env3 def test_items(self): diff --git a/src/engine/SCons/Executor.py b/src/engine/SCons/Executor.py index 28bb6ad..fb2224f 100644 --- a/src/engine/SCons/Executor.py +++ b/src/engine/SCons/Executor.py @@ -26,8 +26,6 @@ Nodes. # LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION # OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION # WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. -from __future__ import print_function - __revision__ = "__FILE__ __REVISION__ __DATE__ __DEVELOPER__" import collections diff --git a/src/engine/SCons/JobTests.py b/src/engine/SCons/JobTests.py index 2e3af4f..9c9bb41 100644 --- a/src/engine/SCons/JobTests.py +++ b/src/engine/SCons/JobTests.py @@ -42,8 +42,8 @@ def get_cpu_nums(): ncpus = os.sysconf( "SC_NPROCESSORS_ONLN" ) if isinstance(ncpus, int) and ncpus > 0: return ncpus - else: # OSX: - return int( os.popen2( "sysctl -n hw.ncpu")[1].read() ) + else: # OSX: + return int(os.popen2("sysctl -n hw.ncpu")[1].read() ) # Windows: if "NUMBER_OF_PROCESSORS" in os.environ: ncpus = int(os.environ["NUMBER_OF_PROCESSORS"]) diff --git a/src/engine/SCons/Memoize.py b/src/engine/SCons/Memoize.py index 5bdcf42..0595fdf 100644 --- a/src/engine/SCons/Memoize.py +++ b/src/engine/SCons/Memoize.py @@ -20,8 +20,6 @@ # OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION # WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. # -from __future__ import print_function - __revision__ = "__FILE__ __REVISION__ __DATE__ __DEVELOPER__" __doc__ = """Memoizer diff --git a/src/engine/SCons/Node/AliasTests.py b/src/engine/SCons/Node/AliasTests.py index 5d9c799..27b75b3 100644 --- a/src/engine/SCons/Node/AliasTests.py +++ b/src/engine/SCons/Node/AliasTests.py @@ -93,7 +93,7 @@ class AliasTestCase(unittest.TestCase): a2 = SCons.Node.Alias.Alias('a') assert a2.name == 'a', a2.name - assert not a1 is a2 + assert a1 is not a2 assert a1.name == a2.name class AliasNodeInfoTestCase(unittest.TestCase): diff --git a/src/engine/SCons/Node/FS.py b/src/engine/SCons/Node/FS.py index e1d6f68..bb965db 100644 --- a/src/engine/SCons/Node/FS.py +++ b/src/engine/SCons/Node/FS.py @@ -31,8 +31,6 @@ that can be used by scripts or modules looking for the canonical default. # LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION # OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION # WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. -from __future__ import print_function - __revision__ = "__FILE__ __REVISION__ __DATE__ __DEVELOPER__" import fnmatch @@ -44,6 +42,7 @@ import sys import time import codecs from itertools import chain +import importlib.util import SCons.Action import SCons.Debug @@ -1427,22 +1426,10 @@ class FS(LocalFS): This can be useful when we want to determine a toolpath based on a python module name""" dirpath = '' - if sys.version_info[0] < 3 or (sys.version_info[0] == 3 and sys.version_info[1] in (0,1,2,3,4)): - # Python2 Code - import imp - splitname = modulename.split('.') - srchpths = sys.path - for item in splitname: - file, path, desc = imp.find_module(item, srchpths) - if file is not None: - path = os.path.dirname(path) - srchpths = [path] - dirpath = path - else: - # Python3 Code - import importlib.util - modspec = importlib.util.find_spec(modulename) - dirpath = os.path.dirname(modspec.origin) + + # Python3 Code + modspec = importlib.util.find_spec(modulename) + dirpath = os.path.dirname(modspec.origin) return self._lookup(dirpath, None, Dir, True) @@ -1549,9 +1536,7 @@ class Dir(Base): self.repositories = [] self.srcdir = None - self.entries = {} - self.entries['.'] = self - self.entries['..'] = self.dir + self.entries = {'.': self, '..': self.dir} self.cwd = self self.searched = 0 self._sconsign = None @@ -1612,7 +1597,7 @@ class Dir(Base): This clears any cached information that is invalidated by changing the repository.""" - for node in list(self.entries.values()): + for node in self.entries.values(): if node != self.dir: if node != self and isinstance(node, Dir): node.__clearRepositoryCache(duplicate) @@ -1623,7 +1608,7 @@ class Dir(Base): except AttributeError: pass if duplicate is not None: - node.duplicate=duplicate + node.duplicate = duplicate def __resetDuplicate(self, node): if node != self: @@ -2306,10 +2291,8 @@ class RootDir(Dir): self._morph() self.duplicate = 0 - self._lookupDict = {} + self._lookupDict = {'': self, '/': self} - self._lookupDict[''] = self - self._lookupDict['/'] = self self.root = self # The // entry is necessary because os.path.normpath() # preserves double slashes at the beginning of a path on Posix @@ -2329,9 +2312,7 @@ class RootDir(Dir): self.repositories = [] self.srcdir = None - self.entries = {} - self.entries['.'] = self - self.entries['..'] = self.dir + self.entries = {'.': self, '..': self.dir} self.cwd = self self.searched = 0 self._sconsign = None @@ -3746,7 +3727,7 @@ class FileFinder(object): if verbose and not callable(verbose): if not SCons.Util.is_String(verbose): verbose = "find_file" - _verbose = u' %s: ' % verbose + _verbose = ' %s: ' % verbose verbose = lambda s: sys.stdout.write(_verbose + s) filedir, filename = os.path.split(filename) diff --git a/src/engine/SCons/Node/FSTests.py b/src/engine/SCons/Node/FSTests.py index 9c19481..3f2cb0e 100644 --- a/src/engine/SCons/Node/FSTests.py +++ b/src/engine/SCons/Node/FSTests.py @@ -20,8 +20,6 @@ # OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION # WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. # -from __future__ import division, print_function - __revision__ = "__FILE__ __REVISION__ __DATE__ __DEVELOPER__" import SCons.compat @@ -465,10 +463,7 @@ class VariantDirTestCase(unittest.TestCase): def __init__(self, duplicate, link, symlink, copy): self.duplicate = duplicate - self.have = {} - self.have['hard'] = link - self.have['soft'] = symlink - self.have['copy'] = copy + self.have = {'hard': link, 'soft': symlink, 'copy': copy} self.links_to_be_called = [] for link in self.duplicate.split('-'): @@ -1347,11 +1342,10 @@ class FSTestCase(_tempdirTestCase): assert f1.get_contents() == bytearray("Foo\x1aBar", 'utf-8'), f1.get_contents() # This tests to make sure we can decode UTF-8 text files. - test_string = u"Foo\x1aBar" + test_string = "Foo\x1aBar" test.write("utf8_file", test_string.encode('utf-8')) f1 = fs.File(test.workpath("utf8_file")) - assert eval('f1.get_text_contents() == u"Foo\x1aBar"'), \ - f1.get_text_contents() + f1.get_text_contents() == "Foo\x1aBar", f1.get_text_contents() # Check for string which doesn't have BOM and isn't valid # ASCII @@ -1449,7 +1443,7 @@ class FSTestCase(_tempdirTestCase): c = e.get_text_contents() try: - eval('assert c == u"", c') + eval('assert c == "", c') except SyntaxError: assert c == "" @@ -1460,10 +1454,7 @@ class FSTestCase(_tempdirTestCase): assert e.__class__ == SCons.Node.FS.Entry, e.__class__ assert c == "", c c = e.get_text_contents() - try: - eval('assert c == u"", c') - except SyntaxError: - assert c == "", c + assert c == "", c test.write("tstamp", "tstamp\n") try: @@ -3066,12 +3057,12 @@ class RepositoryTestCase(_tempdirTestCase): assert r is d1, r r = d2.rentry() - assert not r is d2, r + assert r is not d2, r r = str(r) assert r == os.path.join(self.rep1, 'd2'), r r = d3.rentry() - assert not r is d3, r + assert r is not d3, r r = str(r) assert r == os.path.join(self.rep2, 'd3'), r @@ -3079,12 +3070,12 @@ class RepositoryTestCase(_tempdirTestCase): assert r is e1, r r = e2.rentry() - assert not r is e2, r + assert r is not e2, r r = str(r) assert r == os.path.join(self.rep1, 'e2'), r r = e3.rentry() - assert not r is e3, r + assert r is not e3, r r = str(r) assert r == os.path.join(self.rep2, 'e3'), r @@ -3092,12 +3083,12 @@ class RepositoryTestCase(_tempdirTestCase): assert r is f1, r r = f2.rentry() - assert not r is f2, r + assert r is not f2, r r = str(r) assert r == os.path.join(self.rep1, 'f2'), r r = f3.rentry() - assert not r is f3, r + assert r is not f3, r r = str(r) assert r == os.path.join(self.rep2, 'f3'), r @@ -3127,12 +3118,12 @@ class RepositoryTestCase(_tempdirTestCase): assert r is d1, r r = d2.rdir() - assert not r is d2, r + assert r is not d2, r r = str(r) assert r == os.path.join(self.rep1, 'd2'), r r = d3.rdir() - assert not r is d3, r + assert r is not d3, r r = str(r) assert r == os.path.join(self.rep3, 'd3'), r @@ -3183,12 +3174,12 @@ class RepositoryTestCase(_tempdirTestCase): assert r is f1, r r = f2.rfile() - assert not r is f2, r + assert r is not f2, r r = str(r) assert r == os.path.join(self.rep1, 'f2'), r r = f3.rfile() - assert not r is f3, r + assert r is not f3, r r = f3.rstr() assert r == os.path.join(self.rep3, 'f3'), r @@ -3316,15 +3307,7 @@ class RepositoryTestCase(_tempdirTestCase): # Use a test string that has a file terminator in it to make # sure we read the entire file, regardless of its contents. - try: - eval('test_string = u"Con\x1aTents\n"') - except SyntaxError: - import collections - class FakeUnicodeString(collections.UserString): - def encode(self, encoding): - return str(self) - - test_string = FakeUnicodeString("Con\x1aTents\n") + test_string = "Con\x1aTents\n" # Test with ASCII. test.write(["rep3", "contents"], test_string.encode('ascii')) diff --git a/src/engine/SCons/Node/NodeTests.py b/src/engine/SCons/Node/NodeTests.py index d8179ff..8d9d3a9 100644 --- a/src/engine/SCons/Node/NodeTests.py +++ b/src/engine/SCons/Node/NodeTests.py @@ -1104,7 +1104,7 @@ class NodeTestCase(unittest.TestCase): for kid in [n1, n3, n4, n6, n7, n9, n10, n12]: assert kid in kids, kid for kid in [n2, n5, n8, n11]: - assert not kid in kids, kid + assert kid not in kids, kid def test_all_children(self): """Test fetching all the "children" of a Node. diff --git a/src/engine/SCons/Node/Python.py b/src/engine/SCons/Node/Python.py index a5bcd4b..68c6ee8 100644 --- a/src/engine/SCons/Node/Python.py +++ b/src/engine/SCons/Node/Python.py @@ -88,7 +88,7 @@ class Value(SCons.Node.Node): NodeInfo = ValueNodeInfo BuildInfo = ValueBuildInfo - def __init__(self, value, built_value=None): + def __init__(self, value, built_value=None, name=None): SCons.Node.Node.__init__(self) self.value = value self.changed_since_last_build = 6 @@ -96,6 +96,13 @@ class Value(SCons.Node.Node): if built_value is not None: self.built_value = built_value + # Set a name so it can be a child of a node and not break + # its parent's implementation of Node.get_contents. + if name: + self.name = name + else: + self.name = str(value) + def str_for_display(self): return repr(self.value) @@ -177,23 +184,26 @@ class Value(SCons.Node.Node): return contents -def ValueWithMemo(value, built_value=None): +def ValueWithMemo(value, built_value=None, name=None): + """ + Memoized Value() node factory. + """ global _memo_lookup_map # No current support for memoizing a value that needs to be built. if built_value: - return Value(value, built_value) + return Value(value, built_value, name=name) try: - memo_lookup_key = hash(value) + memo_lookup_key = hash((value, name)) except TypeError: # Non-primitive types will hit this codepath. - return Value(value) + return Value(value, name=name) try: return _memo_lookup_map[memo_lookup_key] except KeyError: - v = Value(value) + v = Value(value, built_value, name) _memo_lookup_map[memo_lookup_key] = v return v diff --git a/src/engine/SCons/Node/PythonTests.py b/src/engine/SCons/Node/PythonTests.py index dbb4ec8..51a49c0 100644 --- a/src/engine/SCons/Node/PythonTests.py +++ b/src/engine/SCons/Node/PythonTests.py @@ -64,6 +64,12 @@ class ValueTestCase(unittest.TestCase): v2.build() assert v2.built_value == 'faked', v2.built_value + v3 = SCons.Node.Python.Value(b'\x00\x0F', name='name') + v3.executor = fake_executor() + v3.build() + assert v3.name == 'name', v3.name + assert v3.built_value == 'faked', v3.built_value + def test_read(self): """Test the Value.read() method """ @@ -98,6 +104,9 @@ class ValueTestCase(unittest.TestCase): assert csig == 'None', csig + + + class ValueNodeInfoTestCase(unittest.TestCase): def test___init__(self): """Test ValueNodeInfo initialization""" @@ -112,6 +121,18 @@ class ValueBuildInfoTestCase(unittest.TestCase): bi = SCons.Node.Python.ValueBuildInfo() +class ValueChildTestCase(unittest.TestCase): + def test___init__(self): + """Test support for a Value() being an implicit dependency of a Node""" + value = SCons.Node.Python.Value('v') + node = SCons.Node.Node() + node._func_get_contents = 2 # Pretend to be a Dir. + node.add_to_implicit([value]) + contents = node.get_contents() + expected_contents = '%s %s\n' % (value.get_csig(), value.name) + assert contents == expected_contents + + class ValueMemoTestCase(unittest.TestCase): def test_memo(self): """Test memoization""" @@ -144,6 +165,19 @@ class ValueMemoTestCase(unittest.TestCase): v4 = SCons.Node.Python.ValueWithMemo(a) assert v3 is not v4 + def test_value_set_name(self): + """ Confirm setting name and caching takes the name into account """ + + v1 = SCons.Node.Python.ValueWithMemo(b'\x00\x0F', name='name') + v2 = SCons.Node.Python.ValueWithMemo(b'\x00\x0F', name='name2') + v3 = SCons.Node.Python.ValueWithMemo('Jibberish') + + self.assertEqual(v1.name,'name', msg=v1.name) + self.assertEqual(v2.name,'name2', msg=v2.name) + self.assertEqual(v3.name,'Jibberish', msg=v3.name) + self.assertTrue(v1 is not v2, msg="v1 and v2 should be different as they have different names but same values") + + if __name__ == "__main__": unittest.main() diff --git a/src/engine/SCons/Node/__init__.py b/src/engine/SCons/Node/__init__.py index 0b58282..c3565bf 100644 --- a/src/engine/SCons/Node/__init__.py +++ b/src/engine/SCons/Node/__init__.py @@ -19,8 +19,6 @@ be able to depend on any other type of "thing." """ -from __future__ import print_function - # # __COPYRIGHT__ # diff --git a/src/engine/SCons/PathListTests.py b/src/engine/SCons/PathListTests.py index 104be73..09b1132 100644 --- a/src/engine/SCons/PathListTests.py +++ b/src/engine/SCons/PathListTests.py @@ -185,7 +185,7 @@ class PathListTestCase(unittest.TestCase): x3 = SCons.PathList.PathList('x') - assert not x1 is x3, (x1, x3) + assert x1 is not x3, (x1, x3) if __name__ == "__main__": diff --git a/src/engine/SCons/Platform/__init__.py b/src/engine/SCons/Platform/__init__.py index 058241d..21e63b4 100644 --- a/src/engine/SCons/Platform/__init__.py +++ b/src/engine/SCons/Platform/__init__.py @@ -41,8 +41,6 @@ their own platform definition. # OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION # WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. # -from __future__ import print_function - __revision__ = "__FILE__ __REVISION__ __DATE__ __DEVELOPER__" import SCons.compat @@ -195,7 +193,7 @@ class TempFileMunge(object): # Default to the .lnk suffix for the benefit of the Phar Lap # linkloc linker, which likes to append an .lnk suffix if # none is given. - if env.has_key('TEMPFILESUFFIX'): + if 'TEMPFILESUFFIX' in env: suffix = env.subst('$TEMPFILESUFFIX') else: suffix = '.lnk' diff --git a/src/engine/SCons/Platform/win32.py b/src/engine/SCons/Platform/win32.py index bb1b46d..afe2df9 100644 --- a/src/engine/SCons/Platform/win32.py +++ b/src/engine/SCons/Platform/win32.py @@ -62,46 +62,6 @@ except AttributeError: else: parallel_msg = None - if sys.version_info.major == 2: - import __builtin__ - - _builtin_file = __builtin__.file - _builtin_open = __builtin__.open - - def _scons_fixup_mode(mode): - """Adjust 'mode' to mark handle as non-inheritable. - - SCons is multithreaded, so allowing handles to be inherited by - children opens us up to races, where (e.g.) processes spawned by - the Taskmaster may inherit and retain references to files opened - by other threads. This may lead to sharing violations and, - ultimately, build failures. - - By including 'N' as part of fopen's 'mode' parameter, all file - handles returned from these functions are atomically marked as - non-inheritable. - """ - if not mode: - # Python's default is 'r'. - # https://docs.python.org/2/library/functions.html#open - mode = 'rN' - elif 'N' not in mode: - mode += 'N' - return mode - - class _scons_file(_builtin_file): - def __init__(self, name, mode=None, *args, **kwargs): - _builtin_file.__init__(self, name, _scons_fixup_mode(mode), - *args, **kwargs) - - def _scons_open(name, mode=None, *args, **kwargs): - return _builtin_open(name, _scons_fixup_mode(mode), - *args, **kwargs) - - __builtin__.file = _scons_file - __builtin__.open = _scons_open - - if False: # Now swap out shutil.filecopy and filecopy2 for win32 api native CopyFile @@ -307,9 +267,6 @@ def get_system_root(): except: pass - # Ensure system root is a string and not unicode - # (This only matters for py27 were unicode in env passed to POpen fails) - val = str(val) _system_root = val return val diff --git a/src/engine/SCons/SConf.py b/src/engine/SCons/SConf.py index e706dae..e0e492b 100644 --- a/src/engine/SCons/SConf.py +++ b/src/engine/SCons/SConf.py @@ -33,8 +33,6 @@ libraries are installed, if some command line options are supported etc. # OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION # WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. # -from __future__ import print_function - __revision__ = "__FILE__ __REVISION__ __DATE__ __DEVELOPER__" import SCons.compat @@ -56,6 +54,7 @@ import SCons.Warnings import SCons.Conftest from SCons.Debug import Trace +from collections import defaultdict # Turn off the Conftest error logging SCons.Conftest.LogInputFiles = 0 @@ -98,7 +97,7 @@ def SetProgressDisplay(display): SConfFS = None -_ac_build_counter = 0 # incremented, whenever TryBuild is called +_ac_build_counter = defaultdict(int) _ac_config_logs = {} # all config.log files created in this build _ac_config_hs = {} # all config.h files created in this build sconf_global = None # current sconf object @@ -132,8 +131,8 @@ def CreateConfigHBuilder(env): _stringConfigH) sconfigHBld = SCons.Builder.Builder(action=action) env.Append( BUILDERS={'SConfigHBuilder':sconfigHBld} ) - for k in list(_ac_config_hs.keys()): - env.SConfigHBuilder(k, env.Value(_ac_config_hs[k])) + for k, v in _ac_config_hs.items(): + env.SConfigHBuilder(k, env.Value(v)) class SConfWarning(SCons.Warnings.Warning): @@ -161,11 +160,14 @@ class ConfigureCacheError(SConfError): def __init__(self,target): SConfError.__init__(self, '"%s" is not yet built and cache is forced.' % str(target)) + # define actions for building text files -def _createSource( target, source, env ): +def _createSource(target, source, env): fd = open(str(target[0]), "w") fd.write(source[0].get_contents().decode()) fd.close() + + def _stringSource( target, source, env ): return (str(target[0]) + ' <-\n |' + source[0].get_contents().decode().replace( '\n', "\n |" ) ) @@ -426,7 +428,8 @@ class SConfBase(object): SConfFS = SCons.Node.FS.default_fs or \ SCons.Node.FS.FS(env.fs.pathTop) if sconf_global is not None: - raise SCons.Errors.UserError + raise SCons.Errors.UserError("""Configure() called while another Configure() exists. + Please call .Finish() before creating and second Configure() context""") if log_file is not None: log_file = SConfFS.File(env.subst(log_file)) @@ -573,8 +576,7 @@ class SConfBase(object): """ return self.pspawn(sh, escape, cmd, args, env, self.logstream, self.logstream) - - def TryBuild(self, builder, text = None, extension = ""): + def TryBuild(self, builder, text=None, extension=""): """Low level TryBuild implementation. Normally you don't need to call that - you can use TryCompile / TryLink / TryRun instead """ @@ -592,8 +594,30 @@ class SConfBase(object): raise SCons.Errors.UserError('Missing SPAWN construction variable.') nodesToBeBuilt = [] + sourcetext = self.env.Value(text) + f = "conftest" + + if text is not None: + textSig = SCons.Util.MD5signature(sourcetext) + textSigCounter = str(_ac_build_counter[textSig]) + _ac_build_counter[textSig] += 1 + + f = "_".join([f, textSig, textSigCounter]) + textFile = self.confdir.File(f + extension) + textFileNode = self.env.SConfSourceBuilder(target=textFile, + source=sourcetext) + nodesToBeBuilt.extend(textFileNode) + + source = textFile + target = textFile.File(f + "SConfActionsContentDummyTarget") + else: + source = None + target = None + + action = builder.builder.action.get_contents(target=target, source=[source], env=self.env) + actionsig = SCons.Util.MD5signature(action) + f = "_".join([f, actionsig]) - f = "conftest_" + str(_ac_build_counter) pref = self.env.subst( builder.builder.prefix ) suff = self.env.subst( builder.builder.suffix ) target = self.confdir.File(pref + f + suff) @@ -602,16 +626,6 @@ class SConfBase(object): # Slide our wrapper into the construction environment as # the SPAWN function. self.env['SPAWN'] = self.pspawn_wrapper - sourcetext = self.env.Value(text) - - if text is not None: - textFile = self.confdir.File(f + extension) - textFileNode = self.env.SConfSourceBuilder(target=textFile, - source=sourcetext) - nodesToBeBuilt.extend(textFileNode) - source = textFileNode - else: - source = None nodes = builder(target = target, source = source) if not SCons.Util.is_List(nodes): @@ -622,7 +636,6 @@ class SConfBase(object): finally: self.env['SPAWN'] = save_spawn - _ac_build_counter = _ac_build_counter + 1 if result: self.lastTarget = nodes[0] else: @@ -703,7 +716,7 @@ class SConfBase(object): """Adds all the tests given in the tests dictionary to this SConf instance """ - for name in list(tests.keys()): + for name in tests.keys(): self.AddTest(name, tests[name]) def _createDir( self, node ): diff --git a/src/engine/SCons/SConfTests.py b/src/engine/SCons/SConfTests.py index c5d1fbd..e2b9133 100644 --- a/src/engine/SCons/SConfTests.py +++ b/src/engine/SCons/SConfTests.py @@ -56,18 +56,6 @@ class SConfTestCase(unittest.TestCase): os.chdir(self.save_cwd) def _resetSConfState(self): - if sys.platform in ['cygwin', 'win32'] and sys.version_info.major == 2: - # On Windows with Python2, SCons.Platform.win32 redefines the - # built-in file() and open() functions to disable handle - # inheritance. Because we are unloading all SCons modules other - # than SCons.Compat, SCons.Platform.win32 will lose the variables - # it needs. As a result, we should reset the file() and open() - # functions to their original built-in versions. - import __builtin__ - import SCons.Platform.win32 - __builtin__.file = SCons.Platform.win32._builtin_file - __builtin__.open = SCons.Platform.win32._builtin_open - # Ok, this is tricky, and i do not know, if everything is sane. # We try to reset scons' state (including all global variables) import SCons.SConsign @@ -169,10 +157,18 @@ class SConfTestCase(unittest.TestCase): log_file=self.test.workpath('config.log')) import SCons.Builder import SCons.Node + + class MyAction(object): + def get_contents(self, target, source, env): + return 'MyBuilder-MyAction $SOURCE $TARGET' + class MyBuilder(SCons.Builder.BuilderBase): def __init__(self): self.prefix = '' self.suffix = '' + # need action because temporary file name uses hash of actions get_contents() + self.action = MyAction() + def __call__(self, env, target, source): class MyNode(object): def __init__(self, name): @@ -299,10 +295,6 @@ int main(void) { return None def actionFAIL(target, source, env): return 1 - def actionUnicode(target, source, env): - with open(str(target[0]), "wb") as f: - f.write('2\302\242\n') - return None self._resetSConfState() @@ -311,14 +303,10 @@ int main(void) { log_file=self.test.workpath('config.log')) try: (ret, output) = sconf.TryAction(action=actionOK) - assert ret and output.encode('utf-8') == bytearray("RUN OK"+os.linesep,'utf-8'), (ret, output) + assert ret and output.encode('utf-8') == bytearray("RUN OK"+os.linesep, 'utf-8'), (ret, output) (ret, output) = sconf.TryAction(action=actionFAIL) assert not ret and output == "", (ret, output) - if not TestCmd.IS_PY3: - # GH Issue #3141 - unicode text and py2.7 crashes. - (ret, output) = sconf.TryAction(action=actionUnicode) - assert ret and output == u'2\xa2\n', (ret, output) finally: sconf.Finish() diff --git a/src/engine/SCons/SConsign.py b/src/engine/SCons/SConsign.py index 1115f2a..a516e1f 100644 --- a/src/engine/SCons/SConsign.py +++ b/src/engine/SCons/SConsign.py @@ -26,9 +26,6 @@ Writing and reading information to the .sconsign file or files. # OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION # WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. # - -from __future__ import print_function - __revision__ = "__FILE__ __REVISION__ __DATE__ __DEVELOPER__" import SCons.compat diff --git a/src/engine/SCons/SConsignTests.py b/src/engine/SCons/SConsignTests.py index d40a7b6..782072d 100644 --- a/src/engine/SCons/SConsignTests.py +++ b/src/engine/SCons/SConsignTests.py @@ -337,7 +337,7 @@ class SConsignFileTestCase(SConsignTestCase): SCons.SConsign.ForDirectory(DummyNode(test.workpath('dir'))) - assert not SCons.SConsign.DataBase is None, SCons.SConsign.DataBase + assert SCons.SConsign.DataBase is not None, SCons.SConsign.DataBase assert fake_dbm.name == file, fake_dbm.name assert fake_dbm.mode == "c", fake_dbm.mode diff --git a/src/engine/SCons/Scanner/ProgTests.py b/src/engine/SCons/Scanner/ProgTests.py index 45fdcb2..8981fd4 100644 --- a/src/engine/SCons/Scanner/ProgTests.py +++ b/src/engine/SCons/Scanner/ProgTests.py @@ -43,6 +43,7 @@ libs = [ 'l1.lib', 'd1/l2.lib', 'd1/d2/l3.lib', for h in libs: test.write(h, "\n") + # define some helpers: class DummyEnvironment(object): @@ -254,22 +255,6 @@ def suite(): suite.addTest(ProgramScannerTestCase8()) suite.addTest(ProgramScannerTestCase9()) suite.addTest(ProgramScannerTestCase10()) - try: unicode - except NameError: pass - else: - code = """if 1: - class ProgramScannerTestCase4(unittest.TestCase): - def runTest(self): - env = DummyEnvironment(LIBPATH=[test.workpath("d1/d2"), - test.workpath("d1")], - LIBS=u'l2 l3'.split()) - s = SCons.Scanner.Prog.ProgramScanner() - path = s.path(env) - deps = s(DummyNode('dummy'), env, path) - assert deps_match(deps, ['d1/l2.lib', 'd1/d2/l3.lib']), map(str, deps) - suite.addTest(ProgramScannerTestCase4()) - \n""" - exec(code) return suite if __name__ == "__main__": diff --git a/src/engine/SCons/Scanner/Python.py b/src/engine/SCons/Scanner/Python.py new file mode 100644 index 0000000..deb2241 --- /dev/null +++ b/src/engine/SCons/Scanner/Python.py @@ -0,0 +1,171 @@ +"""SCons.Scanner.Python + +This module implements the dependency scanner for Python code. + +One important note about the design is that this does not take any dependencies +upon packages or binaries in the Python installation unless they are listed in +PYTHONPATH. To do otherwise would have required code to determine where the +Python installation is, which is outside of the scope of a scanner like this. +If consumers want to pick up dependencies upon these packages, they must put +those directories in PYTHONPATH. + +""" + +# +# __COPYRIGHT__ +# +# Permission is hereby granted, free of charge, to any person obtaining +# a copy of this software and associated documentation files (the +# "Software"), to deal in the Software without restriction, including +# without limitation the rights to use, copy, modify, merge, publish, +# distribute, sublicense, and/or sell copies of the Software, and to +# permit persons to whom the Software is furnished to do so, subject to +# the following conditions: +# +# The above copyright notice and this permission notice shall be included +# in all copies or substantial portions of the Software. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY +# KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE +# WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND +# NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE +# LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION +# OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION +# WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. +# + +__revision__ = "__FILE__ __REVISION__ __DATE__ __DEVELOPER__" + +import itertools +import os +import re +import SCons.Scanner + +# Capture python "from a import b" and "import a" statements. +from_cre = re.compile(r'^\s*from\s+([^\s]+)\s+import\s+(.*)', re.M) +import_cre = re.compile(r'^\s*import\s+([^\s]+)', re.M) + + +def path_function(env, dir=None, target=None, source=None, argument=None): + """Retrieves a tuple with all search paths.""" + paths = env['ENV'].get('PYTHONPATH', '').split(os.pathsep) + if source: + paths.append(source[0].dir.abspath) + return tuple(paths) + + +def find_include_names(node): + """ + Scans the node for all imports. + + Returns a list of tuples. Each tuple has two elements: + 1. The main import (e.g. module, module.file, module.module2) + 2. Additional optional imports that could be functions or files + in the case of a "from X import Y" statement. In the case of a + normal "import" statement, this is None. + """ + text = node.get_text_contents() + all_matches = [] + matches = from_cre.findall(text) + if matches: + for match in matches: + imports = [i.strip() for i in match[1].split(',')] + + # Add some custom logic to strip out "as" because the regex + # includes it. + last_import_split = imports[-1].split() + if len(last_import_split) > 1: + imports[-1] = last_import_split[0] + + all_matches.append((match[0], imports)) + + matches = import_cre.findall(text) + if matches: + for match in matches: + all_matches.append((match, None)) + + return all_matches + + +def scan(node, env, path=()): + # cache the includes list in node so we only scan it once: + if node.includes is not None: + includes = node.includes + else: + includes = find_include_names(node) + # Intern the names of the include files. Saves some memory + # if the same header is included many times. + node.includes = list(map(SCons.Util.silent_intern, includes)) + + # XXX TODO: Sort? + nodes = [] + if callable(path): + path = path() + for module, imports in includes: + is_relative = module.startswith('.') + if is_relative: + # This is a relative include, so we must ignore PYTHONPATH. + module_lstripped = module.lstrip('.') + # One dot is current directory, two is parent, three is + # grandparent, etc. + num_parents = len(module) - len(module_lstripped) - 1 + current_dir = node.get_dir() + for i in itertools.repeat(None, num_parents): + current_dir = current_dir.up() + + search_paths = [current_dir.abspath] + search_string = module_lstripped + else: + search_paths = path + search_string = module + + module_components = search_string.split('.') + for search_path in search_paths: + candidate_path = os.path.join(search_path, *module_components) + # The import stored in "module" could refer to a directory or file. + import_dirs = [] + if os.path.isdir(candidate_path): + import_dirs = module_components + + # Because this resolved to a directory, there is a chance that + # additional imports (e.g. from module import A, B) could refer + # to files to import. + if imports: + for imp in imports: + file = os.path.join(candidate_path, imp + '.py') + if os.path.isfile(file): + nodes.append(file) + elif os.path.isfile(candidate_path + '.py'): + nodes.append(candidate_path + '.py') + import_dirs = module_components[:-1] + + # We can ignore imports because this resolved to a file. Any + # additional imports (e.g. from module.file import A, B) would + # only refer to functions in this file. + + # Take a dependency on all __init__.py files from all imported + # packages unless it's a relative import. If it's a relative + # import, we don't need to take the dependency because Python + # requires that all referenced packages have already been imported, + # which means that the dependency has already been established. + if import_dirs and not is_relative: + for i in range(len(import_dirs)): + init_components = module_components[:i+1] + ['__init__.py'] + init_path = os.path.join(search_path, *(init_components)) + if os.path.isfile(init_path): + nodes.append(init_path) + break + + return sorted(nodes) + + +PythonSuffixes = ['.py'] +PythonScanner = SCons.Scanner.Base(scan, name='PythonScanner', + skeys=PythonSuffixes, + path_function=path_function, recursive=1) + +# Local Variables: +# tab-width:4 +# indent-tabs-mode:nil +# End: +# vim: set expandtab tabstop=4 shiftwidth=4: diff --git a/src/engine/SCons/Scanner/PythonTests.py b/src/engine/SCons/Scanner/PythonTests.py new file mode 100644 index 0000000..0d4e628 --- /dev/null +++ b/src/engine/SCons/Scanner/PythonTests.py @@ -0,0 +1,269 @@ +# +# __COPYRIGHT__ +# +# Permission is hereby granted, free of charge, to any person obtaining +# a copy of this software and associated documentation files (the +# "Software"), to deal in the Software without restriction, including +# without limitation the rights to use, copy, modify, merge, publish, +# distribute, sublicense, and/or sell copies of the Software, and to +# permit persons to whom the Software is furnished to do so, subject to +# the following conditions: +# +# The above copyright notice and this permission notice shall be included +# in all copies or substantial portions of the Software. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY +# KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE +# WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND +# NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE +# LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION +# OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION +# WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. +# + +__revision__ = "__FILE__ __REVISION__ __DATE__ __DEVELOPER__" + +import SCons.compat + +import collections +import os +import unittest + +import TestCmd + +import SCons.Node.FS +import SCons.Scanner.Python + +test = TestCmd.TestCmd(workdir='') +test.dir_fixture('python_scanner') + +if os.path.normcase('foo') == os.path.normcase('FOO'): + my_normpath = os.path.normcase +else: + my_normpath = os.path.normpath + + +def deps_match(self, deps, headers): + global my_normpath + scanned = list(map(my_normpath, list(map(str, deps)))) + expect = list(map(my_normpath, headers)) + self.assertTrue(scanned == expect, + "expect %s != scanned %s" % (expect, scanned)) + + +# Copied from LaTeXTests.py. +class DummyEnvironment(collections.UserDict): + def __init__(self, **kw): + collections.UserDict.__init__(self) + self.data.update(kw) + self.fs = SCons.Node.FS.FS(test.workpath('')) + self['ENV'] = {} + + def Dictionary(self, *args): + return self.data + + def subst(self, strSubst, target=None, source=None, conv=None): + if strSubst[0] == '$': + return self.data[strSubst[1:]] + return strSubst + + def subst_list(self, strSubst, target=None, source=None, conv=None): + if strSubst[0] == '$': + return [self.data[strSubst[1:]]] + return [[strSubst]] + + def subst_path(self, path, target=None, source=None, conv=None): + if not isinstance(path, list): + path = [path] + return list(map(self.subst, path)) + + def get_calculator(self): + return None + + def get_factory(self, factory): + return factory or self.fs.File + + def Dir(self, filename): + return self.fs.Dir(filename) + + def File(self, filename): + return self.fs.File(filename) + + +class PythonScannerTestPythonPath(unittest.TestCase): + def runTest(self): + env = DummyEnvironment() + s = SCons.Scanner.Python.PythonScanner + env['ENV']['PYTHONPATH'] = test.workpath('') + path = s.path(env) + deps = s(env.File('imports_simple_package.py'), env, path) + files = ['simple_package/__init__.py'] + deps_match(self, deps, files) + + +class PythonScannerTestPythonCallablePath(unittest.TestCase): + def runTest(self): + env = DummyEnvironment() + s = SCons.Scanner.Python.PythonScanner + env['ENV']['PYTHONPATH'] = test.workpath('') + deps = s(env.File('imports_simple_package.py'), env, + lambda : s.path(env)) + files = ['simple_package/__init__.py'] + deps_match(self, deps, files) + + +class PythonScannerTestImportSimplePackage(unittest.TestCase): + def runTest(self): + env = DummyEnvironment() + s = SCons.Scanner.Python.PythonScanner + node = env.File('imports_simple_package.py') + path = s.path(env, source=[node]) + deps = s(node, env, path) + files = ['simple_package/__init__.py'] + deps_match(self, deps, files) + + # Repeat the test in case there are any issues caching includes. + deps = s(node, env, path) + deps_match(self, deps, files) + + +class PythonScannerTestImportSimplePackageModule1As(unittest.TestCase): + def runTest(self): + env = DummyEnvironment() + s = SCons.Scanner.Python.PythonScanner + node = env.File('import_simple_package_module1_as.py') + path = s.path(env, source=[node]) + deps = s(node, env, path) + files = ['simple_package/__init__.py', 'simple_package/module1.py'] + deps_match(self, deps, files) + + +class PythonScannerTestImportSimplePackageModuleAs(unittest.TestCase): + def runTest(self): + env = DummyEnvironment() + s = SCons.Scanner.Python.PythonScanner + node = env.File('import_simple_package_module1.py') + path = s.path(env, source=[node]) + deps = s(node, env, path) + files = ['simple_package/__init__.py', 'simple_package/module1.py'] + deps_match(self, deps, files) + + +class PythonScannerTestFromImportSimplePackageModule1(unittest.TestCase): + def runTest(self): + env = DummyEnvironment() + s = SCons.Scanner.Python.PythonScanner + node = env.File('from_import_simple_package_module1.py') + path = s.path(env, source=[node]) + deps = s(node, env, path) + files = ['simple_package/__init__.py', 'simple_package/module1.py'] + deps_match(self, deps, files) + + +class PythonScannerTestFromImportSimplePackageModule1As(unittest.TestCase): + def runTest(self): + env = DummyEnvironment() + s = SCons.Scanner.Python.PythonScanner + node = env.File('from_import_simple_package_module1_as.py') + path = s.path(env, source=[node]) + deps = s(node, env, path) + files = ['simple_package/__init__.py', 'simple_package/module1.py'] + deps_match(self, deps, files) + + +class PythonScannerTestFromImportSimplePackageModulesNoSpace( + unittest.TestCase): + def runTest(self): + env = DummyEnvironment() + s = SCons.Scanner.Python.PythonScanner + node = env.File('from_import_simple_package_modules_no_space.py') + path = s.path(env, source=[node]) + deps = s(node, env, path) + files = ['simple_package/__init__.py', 'simple_package/module1.py', + 'simple_package/module2.py'] + deps_match(self, deps, files) + + +class PythonScannerTestFromImportSimplePackageModulesWithSpace( + unittest.TestCase): + def runTest(self): + env = DummyEnvironment() + s = SCons.Scanner.Python.PythonScanner + node = env.File('from_import_simple_package_modules_with_space.py') + path = s.path(env, source=[node]) + deps = s(node, env, path) + files = ['simple_package/__init__.py', 'simple_package/module1.py', + 'simple_package/module2.py'] + deps_match(self, deps, files) + + +class PythonScannerTestCurdirReferenceScript(unittest.TestCase): + def runTest(self): + env = DummyEnvironment() + s = SCons.Scanner.Python.PythonScanner + node = env.Dir('curdir_reference').File('script.py') + path = s.path(env, source=[node]) + deps = s(node, env, path) + files = ['curdir_reference/helper.py'] + deps_match(self, deps, files) + + +class PythonScannerTestImportsNested3(unittest.TestCase): + def runTest(self): + env = DummyEnvironment() + s = SCons.Scanner.Python.PythonScanner + node = env.File('imports_nested3.py') + path = s.path(env, source=[node]) + deps = s(node, env, path) + files = ['nested1/__init__.py', 'nested1/nested2/__init__.py', + 'nested1/nested2/nested3/__init__.py'] + deps_match(self, deps, files) + + +class PythonScannerTestImportsGrandparentModule(unittest.TestCase): + def runTest(self): + env = DummyEnvironment() + s = SCons.Scanner.Python.PythonScanner + node = env.File( + 'nested1/nested2/nested3/imports_grandparent_module.py') + path = s.path(env, source=[node]) + deps = s(node, env, path) + # Note: there is some ambiguity here in what the scanner should return. + # Relative imports require that the referenced packages have already + # been imported. + files = ['nested1/module.py'] + deps_match(self, deps, files) + + +class PythonScannerTestImportsParentModule(unittest.TestCase): + def runTest(self): + env = DummyEnvironment() + s = SCons.Scanner.Python.PythonScanner + node = env.File( + 'nested1/nested2/nested3/imports_parent_module.py') + path = s.path(env, source=[node]) + deps = s(node, env, path) + files = ['nested1/nested2/module.py'] + deps_match(self, deps, files) + + +class PythonScannerTestImportsParentThenSubmodule(unittest.TestCase): + def runTest(self): + env = DummyEnvironment() + s = SCons.Scanner.Python.PythonScanner + node = env.File( + 'nested1/nested2/nested3/imports_parent_then_submodule.py') + path = s.path(env, source=[node]) + deps = s(node, env, path) + files = ['nested1/nested2a/module.py'] + deps_match(self, deps, files) + + +if __name__ == "__main__": + unittest.main() + +# Local Variables: +# tab-width:4 +# indent-tabs-mode:nil +# End: +# vim: set expandtab tabstop=4 shiftwidth=4: diff --git a/src/engine/SCons/Scanner/RCTests.py b/src/engine/SCons/Scanner/RCTests.py index 551e613..347149c 100644 --- a/src/engine/SCons/Scanner/RCTests.py +++ b/src/engine/SCons/Scanner/RCTests.py @@ -82,7 +82,7 @@ class DummyEnvironment(collections.UserDict): def Dictionary(self, *args): return self.data - def subst(self, arg, target=None, source=None, conv=None): + def subst(self, strSubst, target=None, source=None, conv=None): if strSubst[0] == '$': return self.data[strSubst[1:]] return strSubst diff --git a/src/engine/SCons/Scanner/ScannerTests.py b/src/engine/SCons/Scanner/ScannerTests.py index abe4042..6206af9 100644 --- a/src/engine/SCons/Scanner/ScannerTests.py +++ b/src/engine/SCons/Scanner/ScannerTests.py @@ -599,7 +599,7 @@ class ClassicCPPTestCase(unittest.TestCase): assert n == 'path/bbb', n assert i == 'bbb', i - n, i = s.find_include(('<', u'ccc'), 'foo', ('path',)) + n, i = s.find_include(('<', 'ccc'), 'foo', ('path',)) assert n == 'path/ccc', n assert i == 'ccc', i diff --git a/src/engine/SCons/Script/Interactive.py b/src/engine/SCons/Script/Interactive.py index cc4f23c..a414b4e 100644 --- a/src/engine/SCons/Script/Interactive.py +++ b/src/engine/SCons/Script/Interactive.py @@ -19,8 +19,6 @@ # LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION # OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION # WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. -from __future__ import print_function - __revision__ = "__FILE__ __REVISION__ __DATE__ __DEVELOPER__" __doc__ = """ @@ -247,7 +245,7 @@ version Prints SCons version information. while n: n = walker.get_next() - for node in list(seen_nodes.keys()): + for node in seen_nodes.keys(): # Call node.clear() to clear most of the state node.clear() # node.clear() doesn't reset node.state, so call diff --git a/src/engine/SCons/Script/Main.py b/src/engine/SCons/Script/Main.py index f9c8384..ce948a0 100644 --- a/src/engine/SCons/Script/Main.py +++ b/src/engine/SCons/Script/Main.py @@ -10,9 +10,6 @@ some other module. If it's specific to the "scons" script invocation, it goes here. """ -from __future__ import print_function - - unsupported_python_version = (2, 6, 0) deprecated_python_version = (2, 7, 0) @@ -43,7 +40,9 @@ __revision__ = "__FILE__ __REVISION__ __DATE__ __DEVELOPER__" import SCons.compat +import importlib.util import os +import re import sys import time import traceback @@ -65,7 +64,6 @@ import SCons.Script import SCons.Taskmaster import SCons.Util import SCons.Warnings - import SCons.Script.Interactive # Global variables @@ -698,80 +696,87 @@ def _create_path(plist): return path def _load_site_scons_dir(topdir, site_dir_name=None): - """Load the site_scons dir under topdir. - Prepends site_scons to sys.path, imports site_scons/site_init.py, - and prepends site_scons/site_tools to default toolpath.""" + """Load the site directory under topdir. + + If a site dir name is supplied use it, else use default "site_scons" + Prepend site dir to sys.path. + If a "site_tools" subdir exists, prepend to toolpath. + Import "site_init.py" from site dir if it exists. + """ if site_dir_name: err_if_not_found = True # user specified: err if missing else: site_dir_name = "site_scons" - err_if_not_found = False - + err_if_not_found = False # scons default: okay to be missing site_dir = os.path.join(topdir, site_dir_name) + if not os.path.exists(site_dir): if err_if_not_found: - raise SCons.Errors.UserError("site dir %s not found."%site_dir) + raise SCons.Errors.UserError("site dir %s not found." % site_dir) return + sys.path.insert(0, os.path.abspath(site_dir)) site_init_filename = "site_init.py" site_init_modname = "site_init" site_tools_dirname = "site_tools" - # prepend to sys.path - sys.path = [os.path.abspath(site_dir)] + sys.path site_init_file = os.path.join(site_dir, site_init_filename) site_tools_dir = os.path.join(site_dir, site_tools_dirname) - if os.path.exists(site_init_file): - import imp, re - try: - try: - fp, pathname, description = imp.find_module(site_init_modname, - [site_dir]) - # Load the file into SCons.Script namespace. This is - # opaque and clever; m is the module object for the - # SCons.Script module, and the exec ... in call executes a - # file (or string containing code) in the context of the - # module's dictionary, so anything that code defines ends - # up adding to that module. This is really short, but all - # the error checking makes it longer. - try: - m = sys.modules['SCons.Script'] - except Exception as e: - fmt = 'cannot import site_init.py: missing SCons.Script module %s' - raise SCons.Errors.InternalError(fmt % repr(e)) - try: - sfx = description[0] - modname = os.path.basename(pathname)[:-len(sfx)] - site_m = {"__file__": pathname, "__name__": modname, "__doc__": None} - re_special = re.compile("__[^_]+__") - for k in list(m.__dict__.keys()): - if not re_special.match(k): - site_m[k] = m.__dict__[k] - - # This is the magic. - exec(compile(fp.read(), fp.name, 'exec'), site_m) - except KeyboardInterrupt: - raise - except Exception as e: - fmt = '*** Error loading site_init file %s:\n' - sys.stderr.write(fmt % repr(site_init_file)) - raise - else: - for k in site_m: - if not re_special.match(k): - m.__dict__[k] = site_m[k] - except KeyboardInterrupt: - raise - except ImportError as e: - fmt = '*** cannot import site init file %s:\n' - sys.stderr.write(fmt % repr(site_init_file)) - raise - finally: - if fp: - fp.close() + if os.path.exists(site_tools_dir): - # prepend to DefaultToolpath SCons.Tool.DefaultToolpath.insert(0, os.path.abspath(site_tools_dir)) + if not os.path.exists(site_init_file): + return + + # "import" the site_init.py file into the SCons.Script namespace. + # This is a variant on the basic Python import flow in that the globals + # dict for the compile step is prepopulated from the SCons.Script + # module object; on success the SCons.Script globals are refilled + # from the site_init globals so it all appears in SCons.Script + # instead of as a separate module. + try: + try: + m = sys.modules['SCons.Script'] + except KeyError: + fmt = 'cannot import {}: missing SCons.Script module' + raise SCons.Errors.InternalError(fmt.format(site_init_file)) + + spec = importlib.util.spec_from_file_location(site_init_modname, site_init_file) + site_m = { + "__file__": spec.origin, + "__name__": spec.name, + "__doc__": None, + } + re_dunder = re.compile(r"__[^_]+__") + # update site dict with all but magic (dunder) methods + for k, v in m.__dict__.items(): + if not re_dunder.match(k): + site_m[k] = v + + with open(spec.origin, 'r') as f: + code = f.read() + try: + codeobj = compile(code, spec.name, "exec") + exec(codeobj, site_m) + except KeyboardInterrupt: + raise + except Exception: + fmt = "*** Error loading site_init file {}:\n" + sys.stderr.write(fmt.format(site_init_file)) + raise + else: + # now refill globals with site_init's symbols + for k, v in site_m.items(): + if not re_dunder.match(k): + m.__dict__[k] = v + except KeyboardInterrupt: + raise + except Exception: + fmt = "*** cannot import site init file {}:\n" + sys.stderr.write(fmt.format(site_init_file)) + raise + + def _load_all_site_scons_dirs(topdir, verbose=None): """Load all of the predefined site_scons dir. Order is significant; we load them in order from most generic @@ -810,7 +815,7 @@ def _load_all_site_scons_dirs(topdir, verbose=None): sysdirs=['/usr/share/scons', homedir('.scons')] - dirs=sysdirs + [topdir] + dirs = sysdirs + [topdir] for d in dirs: if verbose: # this is used by unit tests. print("Loading site dir ", d) diff --git a/src/engine/SCons/Script/Main.xml b/src/engine/SCons/Script/Main.xml index 5c68dee..b54df0e 100644 --- a/src/engine/SCons/Script/Main.xml +++ b/src/engine/SCons/Script/Main.xml @@ -312,7 +312,7 @@ The options supported are: <term><literal>cache_debug</literal></term> <listitem> <para> -which corresponds to --cache-debug; +which corresponds to <option>--cache-debug</option>; </para> </listitem> </varlistentry> @@ -320,7 +320,7 @@ which corresponds to --cache-debug; <term><literal>cache_disable</literal></term> <listitem> <para> -which corresponds to --cache-disable; +which corresponds to <option>--cache-disable</option>; </para> </listitem> </varlistentry> @@ -328,7 +328,7 @@ which corresponds to --cache-disable; <term><literal>cache_force</literal></term> <listitem> <para> -which corresponds to --cache-force; +which corresponds to <option>--cache-force</option>; </para> </listitem> </varlistentry> @@ -336,7 +336,7 @@ which corresponds to --cache-force; <term><literal>cache_show</literal></term> <listitem> <para> -which corresponds to --cache-show; +which corresponds to <option>--cache-show</option>; </para> </listitem> </varlistentry> @@ -344,7 +344,8 @@ which corresponds to --cache-show; <term><literal>clean</literal></term> <listitem> <para> -which corresponds to -c, --clean and --remove; +which corresponds to <option>-c</option>, <option>--clean</option> +and <option>--remove</option>; </para> </listitem> </varlistentry> @@ -352,7 +353,7 @@ which corresponds to -c, --clean and --remove; <term><literal>config</literal></term> <listitem> <para> -which corresponds to --config; +which corresponds to <option>--config</option>; </para> </listitem> </varlistentry> @@ -360,7 +361,7 @@ which corresponds to --config; <term><literal>directory</literal></term> <listitem> <para> -which corresponds to -C and --directory; +which corresponds to <option>-C</option> and <option>--directory</option>; </para> </listitem> </varlistentry> @@ -368,7 +369,7 @@ which corresponds to -C and --directory; <term><literal>diskcheck</literal></term> <listitem> <para> -which corresponds to --diskcheck +which corresponds to <option>--diskcheck</option>; </para> </listitem> </varlistentry> @@ -376,7 +377,7 @@ which corresponds to --diskcheck <term><literal>duplicate</literal></term> <listitem> <para> -which corresponds to --duplicate; +which corresponds to <option>--duplicate</option>; </para> </listitem> </varlistentry> @@ -384,7 +385,7 @@ which corresponds to --duplicate; <term><literal>file</literal></term> <listitem> <para> -which corresponds to -f, --file, --makefile and --sconstruct; +which corresponds to <option>-f</option>, <option>--file</option>, <option>--makefile</option> and <option>--sconstruct</option>; </para> </listitem> </varlistentry> @@ -392,7 +393,7 @@ which corresponds to -f, --file, --makefile and --sconstruct; <term><literal>help</literal></term> <listitem> <para> -which corresponds to -h and --help; +which corresponds to <option>-h</option> and <option>--help</option>; </para> </listitem> </varlistentry> @@ -400,7 +401,7 @@ which corresponds to -h and --help; <term><literal>ignore_errors</literal></term> <listitem> <para> -which corresponds to --ignore-errors; +which corresponds to <option>--ignore-errors</option>; </para> </listitem> </varlistentry> @@ -408,7 +409,7 @@ which corresponds to --ignore-errors; <term><literal>implicit_cache</literal></term> <listitem> <para> -which corresponds to --implicit-cache; +which corresponds to <option>--implicit-cache</option>; </para> </listitem> </varlistentry> @@ -416,7 +417,7 @@ which corresponds to --implicit-cache; <term><literal>implicit_deps_changed</literal></term> <listitem> <para> -which corresponds to --implicit-deps-changed; +which corresponds to <option>--implicit-deps-changed</option>; </para> </listitem> </varlistentry> @@ -424,7 +425,7 @@ which corresponds to --implicit-deps-changed; <term><literal>implicit_deps_unchanged</literal></term> <listitem> <para> -which corresponds to --implicit-deps-unchanged; +which corresponds to <option>--implicit-deps-unchanged</option>; </para> </listitem> </varlistentry> @@ -432,7 +433,7 @@ which corresponds to --implicit-deps-unchanged; <term><literal>interactive</literal></term> <listitem> <para> -which corresponds to --interact and --interactive; +which corresponds to <option>--interact</option> and <option>--interactive</option>; </para> </listitem> </varlistentry> @@ -440,7 +441,7 @@ which corresponds to --interact and --interactive; <term><literal>keep_going</literal></term> <listitem> <para> -which corresponds to -k and --keep-going; +which corresponds to <option>-k</option> and <option>--keep-going</option>; </para> </listitem> </varlistentry> @@ -448,7 +449,7 @@ which corresponds to -k and --keep-going; <term><literal>max_drift</literal></term> <listitem> <para> -which corresponds to --max-drift; +which corresponds to <option>--max-drift</option>; </para> </listitem> </varlistentry> @@ -456,7 +457,9 @@ which corresponds to --max-drift; <term><literal>no_exec</literal></term> <listitem> <para> -which corresponds to -n, --no-exec, --just-print, --dry-run and --recon; +which corresponds to <option>-n</option>, +<option>--no-exec</option>, <option>--just-print</option>, +<option>--dry-run</option> and <option>--recon</option>; </para> </listitem> </varlistentry> @@ -464,7 +467,7 @@ which corresponds to -n, --no-exec, --just-print, --dry-run and --recon; <term><literal>no_site_dir</literal></term> <listitem> <para> -which corresponds to --no-site-dir; +which corresponds to <option>--no-site-dir</option>; </para> </listitem> </varlistentry> @@ -472,7 +475,7 @@ which corresponds to --no-site-dir; <term><literal>num_jobs</literal></term> <listitem> <para> -which corresponds to -j and --jobs; +which corresponds to <option>-j</option> and <option>--jobs</option>; </para> </listitem> </varlistentry> @@ -480,7 +483,7 @@ which corresponds to -j and --jobs; <term><literal>profile_file</literal></term> <listitem> <para> -which corresponds to --profile; +which corresponds to <option>--profile</option>; </para> </listitem> </varlistentry> @@ -488,7 +491,7 @@ which corresponds to --profile; <term><literal>question</literal></term> <listitem> <para> -which corresponds to -q and --question; +which corresponds to <option>-q</option> and <option>--question</option>; </para> </listitem> </varlistentry> @@ -496,7 +499,7 @@ which corresponds to -q and --question; <term><literal>random</literal></term> <listitem> <para> -which corresponds to --random; +which corresponds to <option>--random</option>; </para> </listitem> </varlistentry> @@ -504,7 +507,7 @@ which corresponds to --random; <term><literal>repository</literal></term> <listitem> <para> -which corresponds to -Y, --repository and --srcdir; +which corresponds to <option>-Y</option>, <option>--repository</option> and <option>--srcdir</option>; </para> </listitem> </varlistentry> @@ -512,7 +515,7 @@ which corresponds to -Y, --repository and --srcdir; <term><literal>silent</literal></term> <listitem> <para> -which corresponds to -s, --silent and --quiet; +which corresponds to <option>-s</option>, <option>--silent</option> and <option>--quiet</option>; </para> </listitem> </varlistentry> @@ -520,7 +523,7 @@ which corresponds to -s, --silent and --quiet; <term><literal>site_dir</literal></term> <listitem> <para> -which corresponds to --site-dir; +which corresponds to <option>--site-dir</option>; </para> </listitem> </varlistentry> @@ -528,7 +531,7 @@ which corresponds to --site-dir; <term><literal>stack_size</literal></term> <listitem> <para> -which corresponds to --stack-size; +which corresponds to <option>--stack-size</option>; </para> </listitem> </varlistentry> @@ -536,7 +539,7 @@ which corresponds to --stack-size; <term><literal>taskmastertrace_file</literal></term> <listitem> <para> -which corresponds to --taskmastertrace; and +which corresponds to <option>--taskmastertrace</option>; and </para> </listitem> </varlistentry> @@ -544,7 +547,7 @@ which corresponds to --taskmastertrace; and <term><literal>warn</literal></term> <listitem> <para> -which corresponds to --warn and --warning. +which corresponds to <option>--warn</option> and <option>--warning</option>. </para> </listitem> </varlistentry> @@ -553,7 +556,7 @@ which corresponds to --warn and --warning. <para> See the documentation for the -corresponding command line object for information about each specific +corresponding command line option for information about each specific option. </para> </summary> @@ -749,7 +752,8 @@ line options from a SConscript file. The options supported are: <term><literal>clean</literal></term> <listitem> <para> -which corresponds to -c, --clean and --remove; +which corresponds to <option>-c</option>, <option>--clean</option> +and <option>--remove</option>; </para> </listitem> </varlistentry> @@ -757,7 +761,7 @@ which corresponds to -c, --clean and --remove; <term><literal>duplicate</literal></term> <listitem> <para> -which corresponds to --duplicate; +which corresponds to <option>--duplicate</option>; </para> </listitem> </varlistentry> @@ -765,7 +769,7 @@ which corresponds to --duplicate; <term><literal>help</literal></term> <listitem> <para> -which corresponds to -h and --help; +which corresponds to <option>-h</option> and <option>--help</option>; </para> </listitem> </varlistentry> @@ -773,7 +777,7 @@ which corresponds to -h and --help; <term><literal>implicit_cache</literal></term> <listitem> <para> -which corresponds to --implicit-cache; +which corresponds to <option>--implicit-cache</option>; </para> </listitem> </varlistentry> @@ -781,7 +785,7 @@ which corresponds to --implicit-cache; <term><literal>max_drift</literal></term> <listitem> <para> -which corresponds to --max-drift; +which corresponds to <option>--max-drift</option>; </para> </listitem> </varlistentry> @@ -789,7 +793,9 @@ which corresponds to --max-drift; <term><literal>no_exec</literal></term> <listitem> <para> -which corresponds to -n, --no-exec, --just-print, --dry-run and --recon; +which corresponds to <option>-n</option>, <option>--no-exec</option>, +<option>--just-print</option>, <option>--dry-run</option> +and <option>--recon</option>; </para> </listitem> </varlistentry> @@ -797,7 +803,7 @@ which corresponds to -n, --no-exec, --just-print, --dry-run and --recon; <term><literal>num_jobs</literal></term> <listitem> <para> -which corresponds to -j and --jobs; +which corresponds to <option>-j</option> and <option>--jobs</option>; </para> </listitem> </varlistentry> @@ -805,7 +811,7 @@ which corresponds to -j and --jobs; <term><literal>random</literal></term> <listitem> <para> -which corresponds to --random; and +which corresponds to <option>--random</option>; and </para> </listitem> </varlistentry> @@ -813,7 +819,7 @@ which corresponds to --random; and <term><literal>silent</literal></term> <listitem> <para> -which corresponds to --silent. +which corresponds to <option>--silent</option>. </para> </listitem> </varlistentry> @@ -830,7 +836,7 @@ which corresponds to --stack-size. <para> See the documentation for the -corresponding command line object for information about each specific +corresponding command line option for information about each specific option. </para> diff --git a/src/engine/SCons/Subst.xml b/src/engine/SCons/Subst.xml index 980a9ad..77372ce 100644 --- a/src/engine/SCons/Subst.xml +++ b/src/engine/SCons/Subst.xml @@ -39,7 +39,7 @@ or <literal>IndexError</literal> exception will expand to a <literal>''</literal> -(a null string) and not cause scons to fail. +(an empty string) and not cause scons to fail. All exceptions not in the specified list will generate an error message and terminate processing. diff --git a/src/engine/SCons/SubstTests.py b/src/engine/SCons/SubstTests.py index c25b377..574e714 100644 --- a/src/engine/SCons/SubstTests.py +++ b/src/engine/SCons/SubstTests.py @@ -20,8 +20,6 @@ # OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION # WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. # -from __future__ import print_function - __revision__ = "__FILE__ __REVISION__ __DATE__ __DEVELOPER__" import SCons.compat diff --git a/src/engine/SCons/Taskmaster.py b/src/engine/SCons/Taskmaster.py index 1e5776c..6ca0e03 100644 --- a/src/engine/SCons/Taskmaster.py +++ b/src/engine/SCons/Taskmaster.py @@ -20,8 +20,6 @@ # OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION # WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. -from __future__ import print_function - import sys __doc__ = """ @@ -171,7 +169,7 @@ class Task(object): """ global print_prepare T = self.tm.trace - if T: T.write(self.trace_message(u'Task.prepare()', self.node)) + if T: T.write(self.trace_message('Task.prepare()', self.node)) # Now that it's the appropriate time, give the TaskMaster a # chance to raise any exceptions it encountered while preparing @@ -233,7 +231,7 @@ class Task(object): prepare(), executed() or failed(). """ T = self.tm.trace - if T: T.write(self.trace_message(u'Task.execute()', self.node)) + if T: T.write(self.trace_message('Task.execute()', self.node)) try: cached_targets = [] @@ -399,7 +397,7 @@ class Task(object): """ global print_prepare T = self.tm.trace - if T: T.write(self.trace_message(u'Task.make_ready_current()', + if T: T.write(self.trace_message('Task.make_ready_current()', self.node)) self.out_of_date = [] @@ -447,7 +445,7 @@ class Task(object): that can be put back on the candidates list. """ T = self.tm.trace - if T: T.write(self.trace_message(u'Task.postprocess()', self.node)) + if T: T.write(self.trace_message('Task.postprocess()', self.node)) # We may have built multiple targets, some of which may have # common parents waiting for this build. Count up how many @@ -464,7 +462,7 @@ class Task(object): # A node can only be in the pending_children set if it has # some waiting_parents. if t.waiting_parents: - if T: T.write(self.trace_message(u'Task.postprocess()', + if T: T.write(self.trace_message('Task.postprocess()', t, 'removing')) pending_children.discard(t) @@ -493,7 +491,7 @@ class Task(object): for p, subtract in parents.items(): p.ref_count = p.ref_count - subtract - if T: T.write(self.trace_message(u'Task.postprocess()', + if T: T.write(self.trace_message('Task.postprocess()', p, 'adjusted parent ref count')) if p.ref_count == 0: @@ -555,15 +553,12 @@ class Task(object): exc_traceback = None # raise exc_type(exc_value).with_traceback(exc_traceback) - if sys.version_info[0] == 2: - exec("raise exc_type, exc_value, exc_traceback") - else: # sys.version_info[0] == 3: - if isinstance(exc_value, Exception): #hasattr(exc_value, 'with_traceback'): - # If exc_value is an exception, then just reraise - exec("raise exc_value.with_traceback(exc_traceback)") - else: - # else we'll create an exception using the value and raise that - exec("raise exc_type(exc_value).with_traceback(exc_traceback)") + if isinstance(exc_value, Exception): #hasattr(exc_value, 'with_traceback'): + # If exc_value is an exception, then just reraise + raise exc_value.with_traceback(exc_traceback) + else: + # else we'll create an exception using the value and raise that + raise exc_type(exc_value).with_traceback(exc_traceback) # raise e.__class__, e.__class__(e), sys.exc_info()[2] @@ -797,7 +792,7 @@ class Taskmaster(object): while True: node = self.next_candidate() if node is None: - if T: T.write(self.trace_message('No candidate anymore.') + u'\n') + if T: T.write(self.trace_message('No candidate anymore.') + '\n') return None node = node.disambiguate() @@ -820,7 +815,7 @@ class Taskmaster(object): else: S = None - if T: T.write(self.trace_message(u' Considering node %s and its children:' % self.trace_node(node))) + if T: T.write(self.trace_message(' Considering node %s and its children:' % self.trace_node(node))) if state == NODE_NO_STATE: # Mark this node as being on the execution stack: @@ -828,7 +823,7 @@ class Taskmaster(object): elif state > NODE_PENDING: # Skip this node if it has already been evaluated: if S: S.already_handled = S.already_handled + 1 - if T: T.write(self.trace_message(u' already handled (executed)')) + if T: T.write(self.trace_message(' already handled (executed)')) continue executor = node.get_executor() @@ -859,7 +854,7 @@ class Taskmaster(object): for child in chain(executor.get_all_prerequisites(), children): childstate = child.get_state() - if T: T.write(self.trace_message(u' ' + self.trace_node(child))) + if T: T.write(self.trace_message(' ' + self.trace_node(child))) if childstate == NODE_NO_STATE: children_not_visited.append(child) @@ -920,7 +915,7 @@ class Taskmaster(object): # count so we can be put back on the list for # re-evaluation when they've all finished. node.ref_count = node.ref_count + child.add_to_waiting_parents(node) - if T: T.write(self.trace_message(u' adjusted ref count: %s, child %s' % + if T: T.write(self.trace_message(' adjusted ref count: %s, child %s' % (self.trace_node(node), repr(str(child))))) if T: @@ -946,7 +941,7 @@ class Taskmaster(object): # The default when we've gotten through all of the checks above: # this node is ready to be built. if S: S.build = S.build + 1 - if T: T.write(self.trace_message(u'Evaluating %s\n' % + if T: T.write(self.trace_message('Evaluating %s\n' % self.trace_node(node))) # For debugging only: diff --git a/src/engine/SCons/TaskmasterTests.py b/src/engine/SCons/TaskmasterTests.py index c0c77b0..58a31aa 100644 --- a/src/engine/SCons/TaskmasterTests.py +++ b/src/engine/SCons/TaskmasterTests.py @@ -20,8 +20,6 @@ # OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION # WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. # -from __future__ import division - __revision__ = "__FILE__ __REVISION__ __DATE__ __DEVELOPER__" import SCons.compat @@ -146,7 +144,7 @@ class Node(object): pass def has_builder(self): - return not self.builder is None + return self.builder is not None def is_derived(self): return self.has_builder or self.side_effect @@ -935,7 +933,7 @@ class TaskmasterTestCase(unittest.TestCase): except SCons.Errors.UserError: pass else: - raise TestFailed("did not catch expected UserError") + self.fail("did not catch expected UserError") def raise_BuildError(): raise SCons.Errors.BuildError @@ -948,7 +946,7 @@ class TaskmasterTestCase(unittest.TestCase): except SCons.Errors.BuildError: pass else: - raise TestFailed("did not catch expected BuildError") + self.fail("did not catch expected BuildError") # On a generic (non-BuildError) exception from a Builder, # the target should throw a BuildError exception with the @@ -968,7 +966,7 @@ class TaskmasterTestCase(unittest.TestCase): exc_traceback = sys.exc_info()[2] assert isinstance(e.exc_info[2], type(exc_traceback)), e.exc_info[2] else: - raise TestFailed("did not catch expected BuildError") + self.fail("did not catch expected BuildError") built_text = None cache_text = [] @@ -1049,7 +1047,7 @@ class TaskmasterTestCase(unittest.TestCase): assert cache_text == ["n1 retrieved"], cache_text # If no binfo exists anymore, something has gone wrong... has_binfo = hasattr(n1, 'binfo') - assert has_binfo == True, has_binfo + assert has_binfo, has_binfo def test_exception(self): """Test generic Taskmaster exception handling diff --git a/src/engine/SCons/Tool/DCommon.py b/src/engine/SCons/Tool/DCommon.py index 71c4d8d..d29db3a 100644 --- a/src/engine/SCons/Tool/DCommon.py +++ b/src/engine/SCons/Tool/DCommon.py @@ -1,5 +1,3 @@ -from __future__ import print_function - """SCons.Tool.DCommon Common code for the various D tools. diff --git a/src/engine/SCons/Tool/DCommon.xml b/src/engine/SCons/Tool/DCommon.xml index 7cc47da..6d907c6 100644 --- a/src/engine/SCons/Tool/DCommon.xml +++ b/src/engine/SCons/Tool/DCommon.xml @@ -27,6 +27,7 @@ See its __doc__ string for a discussion of the format. <summary> <para> The D compiler to use. +See also &cv-link-SHDC; for compiling to shared objects. </para> </summary> </cvar> @@ -37,6 +38,18 @@ The D compiler to use. The command line used to compile a D file to an object file. Any options specified in the &cv-link-DFLAGS; construction variable is included on this command line. +See also &cv-link-SHDCOM; for compiling to shared objects. +</para> +</summary> +</cvar> + +<cvar name="DCOMSTR"> +<summary> +<para> +If set, the string displayed when a D source file +is compiled to a (static) object file. +If not set, then &cv-link-DCOM; (the command line) is displayed. +See also &cv-link-SHDCOMSTR; for compiling to shared objects. </para> </summary> </cvar> @@ -181,6 +194,7 @@ DLIBLINKSUFFIX. <summary> <para> Name of the linker to use for linking systems including D sources. +See also &cv-link-SHDLINK; for linking shared objects. </para> </summary> </cvar> @@ -189,6 +203,7 @@ Name of the linker to use for linking systems including D sources. <summary> <para> The command line to use when linking systems including D sources. +See also &cv-link-SHDLINKCOM; for linking shared objects. </para> </summary> </cvar> @@ -197,6 +212,7 @@ The command line to use when linking systems including D sources. <summary> <para> List of linker flags. +See also &cv-link-SHDLINKFLAGS; for linking shared objects. </para> </summary> </cvar> @@ -278,6 +294,7 @@ DVERSUFFIX. <para> The name of the compiler to use when compiling D source destined to be in a shared objects. +See also &cv-link-DC; for compiling to static objects. </para> </summary> </cvar> @@ -286,6 +303,18 @@ destined to be in a shared objects. <summary> <para> The command line to use when compiling code to be part of shared objects. +See also &cv-link-DCOM; for compiling to static objects. +</para> +</summary> +</cvar> + +<cvar name="SHDCOMSTR"> +<summary> +<para> +If set, the string displayed when a D source file +is compiled to a (shared) object file. +If not set, then &cv-link-SHDCOM; (the command line) is displayed. +See also &cv-link-DCOMSTR; for compiling to static objects. </para> </summary> </cvar> @@ -311,6 +340,7 @@ SHDLIBVERSIONFLAGS. <para> The linker to use when creating shared objects for code bases include D sources. +See also &cv-link-DLINK; for linking static objects. </para> </summary> </cvar> @@ -319,6 +349,7 @@ include D sources. <summary> <para> The command line to use when generating shared objects. +See also &cv-link-DLINKCOM; for linking static objects. </para> </summary> </cvar> @@ -327,6 +358,7 @@ The command line to use when generating shared objects. <summary> <para> The list of flags to use when generating a shared object. +See also &cv-link-DLINKFLAGS; for linking static objects. </para> </summary> </cvar> diff --git a/src/engine/SCons/Tool/FortranCommon.py b/src/engine/SCons/Tool/FortranCommon.py index cbb9a03..bfa1c1c 100644 --- a/src/engine/SCons/Tool/FortranCommon.py +++ b/src/engine/SCons/Tool/FortranCommon.py @@ -26,8 +26,6 @@ Stuff for processing Fortran, common to all fortran dialects. # OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION # WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. # -from __future__ import print_function - __revision__ = "__FILE__ __REVISION__ __DATE__ __DEVELOPER__" import re diff --git a/src/engine/SCons/Tool/MSCommon/common.py b/src/engine/SCons/Tool/MSCommon/common.py index 386f445..505136e 100644 --- a/src/engine/SCons/Tool/MSCommon/common.py +++ b/src/engine/SCons/Tool/MSCommon/common.py @@ -23,8 +23,6 @@ Common helper functions for working with the Microsoft tool chain. # OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION # WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. # -from __future__ import print_function - __revision__ = "__FILE__ __REVISION__ __DATE__ __DEVELOPER__" import copy @@ -152,8 +150,8 @@ def normalize_env(env, keys, force=False): Note: the environment is copied.""" normenv = {} if env: - for k in list(env.keys()): - normenv[k] = copy.deepcopy(env[k]) + for k, v in env.items(): + normenv[k] = copy.deepcopy(v) for k in keys: if k in os.environ and (force or k not in normenv): diff --git a/src/engine/SCons/Tool/MSCommon/vc.py b/src/engine/SCons/Tool/MSCommon/vc.py index 6f9bd12..82fb6b9 100644 --- a/src/engine/SCons/Tool/MSCommon/vc.py +++ b/src/engine/SCons/Tool/MSCommon/vc.py @@ -22,6 +22,9 @@ # # TODO: +# * gather all the information from a single vswhere call instead +# of calling repeatedly (use json format?) +# * support passing/setting location for vswhere in env. # * supported arch for versions: for old versions of batch file without # argument, giving bogus argument cannot be detected, so we have to hardcode # this here @@ -42,19 +45,14 @@ import os import platform import sys from string import digits as string_digits -if sys.version_info[0] == 2: - import collections +from subprocess import PIPE import SCons.Warnings from SCons.Tool import find_program_path from . import common - -debug = common.debug - -from . import sdk - -get_installed_sdks = sdk.get_installed_sdks +from .common import CONFIG_CACHE, debug +from .sdk import get_installed_sdks class VisualCException(Exception): @@ -95,6 +93,10 @@ _ARCH_TO_CANONICAL = { "aarch64" : "arm64", } +# Starting with 14.1 (aka VS2017), the tools are organized by host directory. +# subdirs for each target. They are now in .../VC/Auxuiliary/Build. +# Note 2017 Express uses Hostx86 even if it's on 64-bit Windows, +# not reflected in this table. _HOST_TARGET_TO_CL_DIR_GREATER_THAN_14 = { ("amd64","amd64") : ("Hostx64","x64"), ("amd64","x86") : ("Hostx64","x86"), @@ -106,8 +108,9 @@ _HOST_TARGET_TO_CL_DIR_GREATER_THAN_14 = { ("x86","arm64") : ("Hostx86","arm64"), } -# get path to the cl.exe dir for older VS versions -# based off a tuple of (host, target) platforms +# before 14.1 (VS2017): the original x86 tools are in the tools dir, +# any others are in a subdir named by the host/target pair, +# or just a single word if host==target _HOST_TARGET_TO_CL_DIR = { ("amd64","amd64") : "amd64", ("amd64","x86") : "amd64_x86", @@ -117,10 +120,38 @@ _HOST_TARGET_TO_CL_DIR = { ("x86","x86") : "", ("x86","arm") : "x86_arm", ("x86","arm64") : "x86_arm64", + ("arm","arm") : "arm", } -# Given a (host, target) tuple, return the argument for the bat file. -# Both host and targets should be canonalized. +# 14.1 (VS2017) and later: +# Given a (host, target) tuple, return the batch file to look for. +# We can't rely on returning an arg to use for vcvarsall.bat, +# because that script will run even if given a pair that isn't installed. +# Targets that already look like a pair are pseudo targets that +# effectively mean to skip whatever the host was specified as. +_HOST_TARGET_TO_BAT_ARCH_GT14 = { + ("amd64", "amd64"): "vcvars64.bat", + ("amd64", "x86"): "vcvarsamd64_x86.bat", + ("amd64", "x86_amd64"): "vcvarsx86_amd64.bat", + ("amd64", "x86_x86"): "vcvars32.bat", + ("amd64", "arm"): "vcvarsamd64_arm.bat", + ("amd64", "x86_arm"): "vcvarsx86_arm.bat", + ("amd64", "arm64"): "vcvarsamd64_arm64.bat", + ("amd64", "x86_arm64"): "vcvarsx86_arm64.bat", + ("x86", "x86"): "vcvars32.bat", + ("x86", "amd64"): "vcvarsx86_amd64.bat", + ("x86", "x86_amd64"): "vcvarsx86_amd64.bat", + ("x86", "arm"): "vcvarsx86_arm.bat", + ("x86", "x86_arm"): "vcvarsx86_arm.bat", + ("x86", "arm64"): "vcvarsx86_arm64.bat", + ("x86", "x86_arm64"): "vcvarsx86_arm64.bat", +} + +# before 14.1 (VS2017): +# Given a (host, target) tuple, return the argument for the bat file; +# Both host and target should be canoncalized. +# If the target already looks like a pair, return it - these are +# pseudo targets (mainly used by Express versions) _HOST_TARGET_ARCH_TO_BAT_ARCH = { ("x86", "x86"): "x86", ("x86", "amd64"): "x86_amd64", @@ -128,12 +159,16 @@ _HOST_TARGET_ARCH_TO_BAT_ARCH = { ("amd64", "x86_amd64"): "x86_amd64", # This is present in (at least) VS2012 express ("amd64", "amd64"): "amd64", ("amd64", "x86"): "x86", + ("amd64", "x86_x86"): "x86", ("x86", "ia64"): "x86_ia64", # gone since 14.0 - ("arm", "arm"): "arm", # since 14.0, maybe gone 14.1? ("x86", "arm"): "x86_arm", # since 14.0 ("x86", "arm64"): "x86_arm64", # since 14.1 ("amd64", "arm"): "amd64_arm", # since 14.0 ("amd64", "arm64"): "amd64_arm64", # since 14.1 + ("x86", "x86_arm"): "x86_arm", # since 14.0 + ("x86", "x86_arm64"): "x86_arm64", # since 14.1 + ("amd64", "x86_arm"): "x86_arm", # since 14.0 + ("amd64", "x86_arm64"): "x86_arm64", # since 14.1 } _CL_EXE_NAME = 'cl.exe' @@ -194,7 +229,7 @@ def get_host_target(env): # If you update this, update SupportedVSList in Tool/MSCommon/vs.py, and the # MSVC_VERSION documentation in Tool/msvc.xml. -_VCVER = ["14.2", "14.1", "14.0", "14.0Exp", "12.0", "12.0Exp", "11.0", "11.0Exp", "10.0", "10.0Exp", "9.0", "9.0Exp","8.0", "8.0Exp","7.1", "7.0", "6.0"] +_VCVER = ["14.2", "14.1", "14.1Exp", "14.0", "14.0Exp", "12.0", "12.0Exp", "11.0", "11.0Exp", "10.0", "10.0Exp", "9.0", "9.0Exp","8.0", "8.0Exp","7.1", "7.0", "6.0"] # if using vswhere, a further mapping is needed _VCVER_TO_VSWHERE_VER = { @@ -204,9 +239,11 @@ _VCVER_TO_VSWHERE_VER = { _VCVER_TO_PRODUCT_DIR = { '14.2' : [ - (SCons.Util.HKEY_LOCAL_MACHINE, r'')], # VS 2019 doesn't set this key + (SCons.Util.HKEY_LOCAL_MACHINE, r'')], # not set by this version '14.1' : [ - (SCons.Util.HKEY_LOCAL_MACHINE, r'')], # VS 2017 doesn't set this key + (SCons.Util.HKEY_LOCAL_MACHINE, r'')], # not set by this version + '14.1Exp' : [ + (SCons.Util.HKEY_LOCAL_MACHINE, r'')], # not set by this version '14.0' : [ (SCons.Util.HKEY_LOCAL_MACHINE, r'Microsoft\VisualStudio\14.0\Setup\VC\ProductDir')], '14.0Exp' : [ @@ -299,44 +336,46 @@ def find_vc_pdir_vswhere(msvc_version): debug("Unknown version of MSVC: %s" % msvc_version) raise UnsupportedVersion("Unknown version %s" % msvc_version) - # For bug 3333 - support default location of vswhere for both 64 and 32 bit windows - # installs. - for pf in ['Program Files (x86)', 'Program Files']: - vswhere_path = os.path.join( - 'C:\\', - pf, - 'Microsoft Visual Studio', - 'Installer', - 'vswhere.exe' - ) + # For bug 3333: support default location of vswhere for both + # 64 and 32 bit windows installs. + # For bug 3542: also accommodate not being on C: drive. + # NB: this gets called from testsuite on non-Windows platforms. + # Whether that makes sense or not, don't break it for those. + # TODO: requested to add a user-specified path to vswhere + # and have this routine set the same var if it finds it. + pfpaths = [ + os.path.expandvars(r"%ProgramFiles(x86)%\Microsoft Visual Studio\Installer"), + os.path.expandvars(r"%ProgramFiles%\Microsoft Visual Studio\Installer"), + os.path.expandvars(r"%ChocolateyInstall%\bin"), + ] + for pf in pfpaths: + vswhere_path = os.path.join(pf, "vswhere.exe") if os.path.exists(vswhere_path): - # If we found vswhere, then use it. break else: - # No vswhere on system, no install info available + # No vswhere on system, no install info available this way return None - vswhere_cmd = [vswhere_path, - '-products', '*', - '-version', vswhere_version, - '-property', 'installationPath'] - - #TODO PY27 cannot use Popen as context manager - # try putting it back to the old way for now - sp = subprocess.Popen(vswhere_cmd, - stdout=subprocess.PIPE, - stderr=subprocess.PIPE) - vsdir, err = sp.communicate() - if vsdir: - vsdir = vsdir.decode("mbcs").splitlines() - # vswhere could easily return multiple lines - # we could define a way to pick the one we prefer, but since + vswhere_cmd = [ + vswhere_path, + "-products", "*", + "-version", vswhere_version, + "-property", "installationPath", + ] + + #cp = subprocess.run(vswhere_cmd, capture_output=True) # 3.7+ only + cp = subprocess.run(vswhere_cmd, stdout=PIPE, stderr=PIPE) + + if cp.stdout: + # vswhere could return multiple lines, e.g. if Build Tools + # and {Community,Professional,Enterprise} are both installed. + # We could define a way to pick the one we prefer, but since # this data is currently only used to make a check for existence, - # returning the first hit should be good enough for now. - vc_pdir = os.path.join(vsdir[0], 'VC') - return vc_pdir + # returning the first hit should be good enough. + lines = cp.stdout.decode("mbcs").splitlines() + return os.path.join(lines[0], 'VC') else: - # No vswhere on system, no install info available + # We found vswhere, but no install info available for this version return None @@ -403,15 +442,20 @@ def find_batch_file(env,msvc_version,host_arch,target_arch): """ Find the location of the batch script which should set up the compiler for any TARGET_ARCH whose compilers were installed by Visual Studio/VCExpress + + In newer (2017+) compilers, make use of the fact there are vcvars + scripts named with a host_target pair that calls vcvarsall.bat properly, + so use that and return an indication we don't need the argument + we would have computed to run vcvarsall.bat. """ pdir = find_vc_pdir(msvc_version) if pdir is None: raise NoVersionFound("No version of Visual Studio found") - debug('find_batch_file() in {}'.format(pdir)) # filter out e.g. "Exp" from the version name msvc_ver_numeric = get_msvc_version_numeric(msvc_version) + use_arg = True vernum = float(msvc_ver_numeric) if 7 <= vernum < 8: pdir = os.path.join(pdir, os.pardir, "Common7", "Tools") @@ -422,7 +466,10 @@ def find_batch_file(env,msvc_version,host_arch,target_arch): elif 8 <= vernum <= 14: batfilename = os.path.join(pdir, "vcvarsall.bat") else: # vernum >= 14.1 VS2017 and above - batfilename = os.path.join(pdir, "Auxiliary", "Build", "vcvarsall.bat") + batfiledir = os.path.join(pdir, "Auxiliary", "Build") + targ = _HOST_TARGET_TO_BAT_ARCH_GT14[(host_arch, target_arch)] + batfilename = os.path.join(batfiledir, targ) + use_arg = False if not os.path.exists(batfilename): debug("Not found: %s" % batfilename) @@ -437,8 +484,8 @@ def find_batch_file(env,msvc_version,host_arch,target_arch): sdk_bat_file_path = os.path.join(pdir,sdk_bat_file) if os.path.exists(sdk_bat_file_path): debug('find_batch_file() sdk_bat_file_path:%s'%sdk_bat_file_path) - return (batfilename, sdk_bat_file_path) - return (batfilename, None) + return (batfilename, use_arg, sdk_bat_file_path) + return (batfilename, use_arg, None) __INSTALLED_VCS_RUN = None @@ -446,10 +493,12 @@ _VC_TOOLS_VERSION_FILE_PATH = ['Auxiliary', 'Build', 'Microsoft.VCToolsVersion.d _VC_TOOLS_VERSION_FILE = os.sep.join(_VC_TOOLS_VERSION_FILE_PATH) def _check_cl_exists_in_vc_dir(env, vc_dir, msvc_version): - """Find the cl.exe on the filesystem in the vc_dir depending on - TARGET_ARCH, HOST_ARCH and the msvc version. TARGET_ARCH and - HOST_ARCH can be extracted from the passed env, unless its None, - which then the native platform is assumed the host and target. + """Return status of finding a cl.exe to use. + + Locates cl in the vc_dir depending on TARGET_ARCH, HOST_ARCH and the + msvc version. TARGET_ARCH and HOST_ARCH can be extracted from the + passed env, unless it is None, in which case the native platform is + assumed for both host and target. Args: env: Environment @@ -484,7 +533,8 @@ def _check_cl_exists_in_vc_dir(env, vc_dir, msvc_version): # make sure the cl.exe exists meaning the tool is installed if ver_num > 14: - # 2017 and newer allowed multiple versions of the VC toolset to be installed at the same time. + # 2017 and newer allowed multiple versions of the VC toolset to be + # installed at the same time. This changes the layout. # Just get the default tool version for now #TODO: support setting a specific minor VC version default_toolset_file = os.path.join(vc_dir, _VC_TOOLS_VERSION_FILE) @@ -509,10 +559,24 @@ def _check_cl_exists_in_vc_dir(env, vc_dir, msvc_version): debug('_check_cl_exists_in_vc_dir(): found ' + _CL_EXE_NAME + '!') return True + elif host_platform == "amd64" and host_trgt_dir[0] == "Hostx64": + # Special case: fallback to Hostx86 if Hostx64 was tried + # and failed. This is because VS 2017 Express running on amd64 + # will look to our probe like the host dir should be Hostx64, + # but Express uses Hostx86 anyway. + # We should key this off the "x86_amd64" and related pseudo + # targets, but we don't see those in this function. + host_trgt_dir = ("Hostx86", host_trgt_dir[1]) + cl_path = os.path.join(vc_dir, 'Tools','MSVC', vc_specific_version, 'bin', host_trgt_dir[0], host_trgt_dir[1], _CL_EXE_NAME) + debug('_check_cl_exists_in_vc_dir(): checking for ' + _CL_EXE_NAME + ' at ' + cl_path) + if os.path.exists(cl_path): + debug('_check_cl_exists_in_vc_dir(): found ' + _CL_EXE_NAME + '!') + return True + elif 14 >= ver_num >= 8: - # Set default value to be -1 as "" which is the value for x86/x86 yields true when tested - # if not host_trgt_dir + # Set default value to be -1 as "" which is the value for x86/x86 + # yields true when tested if not host_trgt_dir host_trgt_dir = _HOST_TARGET_TO_CL_DIR.get((host_platform, target_platform), None) if host_trgt_dir is None: debug('_check_cl_exists_in_vc_dir(): unsupported host/target platform combo') @@ -604,7 +668,7 @@ def reset_installed_vcs(): # within the same scons run. Windows builds on the CI system were split # into chunks to get around single-build time limits. # With VS2019 it got even slower and an optional persistent cache file -# was introduced. The cache now also stores only the parsed vars, +# was introduced. The cache now also stores only the parsed vars, # not the entire output of running the batch file - saves a bit # of time not parsing every time. @@ -630,23 +694,7 @@ def script_env(script, args=None): script_env_cache[cache_key] = cache_data # once we updated cache, give a chance to write out if user wanted common.write_script_env_cache(script_env_cache) - else: - #TODO: Python 2 cleanup - # If we "hit" data from the json file, we have a Py2 problem: - # keys & values will be unicode. don't detect, just convert. - if sys.version_info[0] == 2: - def convert(data): - if isinstance(data, basestring): - return str(data) - elif isinstance(data, collections.Mapping): - return dict(map(convert, data.iteritems())) - elif isinstance(data, collections.Iterable): - return type(data)(map(convert, data)) - else: - return data - cache_data = convert(cache_data) - return cache_data def get_default_version(env): @@ -696,33 +744,56 @@ def msvc_setup_env_once(env): env["MSVC_SETUP_RUN"] = True def msvc_find_valid_batch_script(env, version): - debug('msvc_find_valid_batch_script()') - # Find the host platform, target platform, and if present the requested - # target platform + """Find and execute appropriate batch script to set up build env. + + The MSVC build environment depends heavily on having the shell + environment set. SCons does not inherit that, and does not count + on that being set up correctly anyway, so it tries to find the right + MSVC batch script, or the right arguments to the generic batch script + vcvarsall.bat, and run that, so we have a valid environment to build in. + There are dragons here: the batch scripts don't fail (see comments + elsewhere), they just leave you with a bad setup, so try hard to + get it right. + """ + + # Find the host, target, and if present the requested target: platforms = get_host_target(env) debug(" msvs_find_valid_batch_script(): host_platform %s, target_platform %s req_target_platform:%s" % platforms) - host_platform, target_platform, req_target_platform = platforms - try_target_archs = [target_platform] - # VS2012 has a "cross compile" environment to build 64 bit - # with x86_amd64 as the argument to the batch setup script + # Most combinations of host + target are straightforward. + # While all MSVC / Visual Studio tools are pysically 32-bit, they + # make it look like there are 64-bit tools if the host is 64-bit, + # so you can invoke the environment batch script to set up to build, + # say, amd64 host -> x86 target. Express versions are an exception: + # they always look 32-bit, so the batch scripts with 64-bit + # host parts are absent. We try to fix that up in a couple of ways. + # One is here: we make a table of "targets" to try, with the extra + # targets being tags that tell us to try a different "host" instead + # of the deduced host. + try_target_archs = [target_platform] if req_target_platform in ('amd64', 'x86_64'): try_target_archs.append('x86_amd64') - elif not req_target_platform and target_platform in ['amd64', 'x86_64']: - # There may not be "native" amd64, but maybe "cross" x86_amd64 tools - try_target_archs.append('x86_amd64') - # If the user hasn't specifically requested a TARGET_ARCH, and - # The TARGET_ARCH is amd64 then also try 32 bits if there are no viable - # 64 bit tools installed - try_target_archs.append('x86') + elif req_target_platform in ('x86',): + try_target_archs.append('x86_x86') + elif req_target_platform in ('arm',): + try_target_archs.append('x86_arm') + elif req_target_platform in ('arm64',): + try_target_archs.append('x86_arm64') + elif not req_target_platform: + if target_platform in ('amd64', 'x86_64'): + try_target_archs.append('x86_amd64') + # If the user hasn't specifically requested a TARGET_ARCH, + # and the TARGET_ARCH is amd64 then also try 32 bits + # if there are no viable 64 bit tools installed + try_target_archs.append('x86') debug("msvs_find_valid_batch_script(): host_platform: %s try_target_archs:%s"%(host_platform, try_target_archs)) d = None for tp in try_target_archs: # Set to current arch. - env['TARGET_ARCH']=tp + env['TARGET_ARCH'] = tp debug("msvc_find_valid_batch_script() trying target_platform:%s"%tp) host_target = (host_platform, tp) @@ -732,17 +803,9 @@ def msvc_find_valid_batch_script(env, version): SCons.Warnings.warn(SCons.Warnings.VisualCMissingWarning, warn_msg) arg = _HOST_TARGET_ARCH_TO_BAT_ARCH[host_target] - # Get just version numbers - maj, min = msvc_version_to_maj_min(version) - # VS2015+ - if maj >= 14: - if env.get('MSVC_UWP_APP') == '1': - # Initialize environment variables with store/universal paths - arg += ' store' - # Try to locate a batch file for this host/target platform combo try: - (vc_script, sdk_script) = find_batch_file(env, version, host_platform, tp) + (vc_script, use_arg, sdk_script) = find_batch_file(env, version, host_platform, tp) debug('msvc_find_valid_batch_script() vc_script:%s sdk_script:%s'%(vc_script,sdk_script)) except VisualCException as e: msg = str(e) @@ -758,6 +821,16 @@ def msvc_find_valid_batch_script(env, version): debug('msvc_find_valid_batch_script() use_script 2 %s, args:%s' % (repr(vc_script), arg)) found = None if vc_script: + if not use_arg: + arg = '' # bat file will supply platform type + # Get just version numbers + maj, min = msvc_version_to_maj_min(version) + # VS2015+ + if maj >= 14: + if env.get('MSVC_UWP_APP') == '1': + # Initialize environment variables with store/UWP paths + arg = (arg + ' store').lstrip() + try: d = script_env(vc_script, args=arg) found = vc_script @@ -823,14 +896,18 @@ def msvc_setup_env(env): return None for k, v in d.items(): - debug('msvc_setup_env() env:%s -> %s'%(k,v)) env.PrependENVPath(k, v, delete_existing=True) + debug("msvc_setup_env() env['ENV']['%s'] = %s" % (k, env['ENV'][k])) # final check to issue a warning if the compiler is not present - msvc_cl = find_program_path(env, 'cl') - if not msvc_cl: - SCons.Warnings.warn(SCons.Warnings.VisualCMissingWarning, - "Could not find MSVC compiler 'cl', it may need to be installed separately with Visual Studio") + if not find_program_path(env, 'cl'): + debug("msvc_setup_env() did not find 'cl'") + if CONFIG_CACHE: + propose = "SCONS_CACHE_MSVC_CONFIG caching enabled, remove cache file {} if out of date.".format(CONFIG_CACHE) + else: + propose = "It may need to be installed separately with Visual Studio." + warn_msg = "Could not find MSVC compiler 'cl'. {}".format(propose) + SCons.Warnings.warn(SCons.Warnings.VisualCMissingWarning, warn_msg) def msvc_exists(env=None, version=None): vcs = cached_get_installed_vcs(env) diff --git a/src/engine/SCons/Tool/MSCommon/vs.py b/src/engine/SCons/Tool/MSCommon/vs.py index bac35d8..e13f52f 100644 --- a/src/engine/SCons/Tool/MSCommon/vs.py +++ b/src/engine/SCons/Tool/MSCommon/vs.py @@ -205,7 +205,8 @@ SupportedVSList = [ hkeys=[], common_tools_var='VS160COMNTOOLS', executable_path=r'Common7\IDE\devenv.com', - batch_file_path=r'VC\Auxiliary\Build\vsvars32.bat', + # should be a fallback, prefer use vswhere installationPath + batch_file_path=r'Common7\Tools\VsDevCmd.bat', supported_arch=['x86', 'amd64', "arm"], ), @@ -216,10 +217,23 @@ SupportedVSList = [ hkeys=[], common_tools_var='VS150COMNTOOLS', executable_path=r'Common7\IDE\devenv.com', - batch_file_path=r'VC\Auxiliary\Build\vsvars32.bat', + # should be a fallback, prefer use vswhere installationPath + batch_file_path=r'Common7\Tools\VsDevCmd.bat', supported_arch=['x86', 'amd64', "arm"], ), + # Visual C++ 2017 Express Edition (for Desktop) + VisualStudio('14.1Exp', + vc_version='14.1', + sdk_version='10.0A', + hkeys=[], + common_tools_var='VS150COMNTOOLS', + executable_path=r'Common7\IDE\WDExpress.exe', + # should be a fallback, prefer use vswhere installationPath + batch_file_path=r'Common7\Tools\VsDevCmd.bat', + supported_arch=['x86', 'amd64', "arm"], + ), + # Visual Studio 2015 VisualStudio('14.0', vc_version='14.0', diff --git a/src/engine/SCons/Tool/__init__.py b/src/engine/SCons/Tool/__init__.py index 14306ab..4cb77c0 100644 --- a/src/engine/SCons/Tool/__init__.py +++ b/src/engine/SCons/Tool/__init__.py @@ -38,9 +38,9 @@ tool definition. __revision__ = "__FILE__ __REVISION__ __DATE__ __DEVELOPER__" import sys -import re import os -import shutil +from collections.abc import Callable +import importlib.util import SCons.Builder import SCons.Errors @@ -51,12 +51,6 @@ import SCons.Scanner.D import SCons.Scanner.LaTeX import SCons.Scanner.Prog import SCons.Scanner.SWIG -try: - # Python 3 - from collections.abc import Callable -except ImportError: - # Python 2.7 - from collections import Callable DefaultToolpath = [] @@ -142,112 +136,81 @@ class Tool(object): sys.path = self.toolpath + sys.path # sys.stderr.write("Tool:%s\nPATH:%s\n"%(self.name,sys.path)) - if sys.version_info[0] < 3 or (sys.version_info[0] == 3 and sys.version_info[1] in (0, 1, 2, 3, 4)): - # Py 2 code - try: - try: - file = None - try: - mod, file = self._load_dotted_module_py2(self.name, self.name, self.toolpath) - return mod - finally: - if file: - file.close() - except ImportError as e: - splitname = self.name.split('.') - if str(e) != "No module named %s" % splitname[0]: - raise SCons.Errors.SConsEnvironmentError(e) - try: - import zipimport - except ImportError: - pass - else: - for aPath in self.toolpath: - try: - importer = zipimport.zipimporter(aPath) - return importer.load_module(self.name) - except ImportError as e: - pass - finally: - sys.path = oldpythonpath - elif sys.version_info[1] > 4: - # From: http://stackoverflow.com/questions/67631/how-to-import-a-module-given-the-full-path/67692#67692 - # import importlib.util - # spec = importlib.util.spec_from_file_location("module.name", "/path/to/file.py") - # foo = importlib.util.module_from_spec(spec) - # spec.loader.exec_module(foo) - # foo.MyClass() - # Py 3 code - - # import pdb; pdb.set_trace() - import importlib.util - - # sys.stderr.write("toolpath:%s\n" % self.toolpath) - # sys.stderr.write("SCONS.TOOL path:%s\n" % sys.modules['SCons.Tool'].__path__) - debug = False - spec = None - found_name = self.name - add_to_scons_tools_namespace = False - for path in self.toolpath: - sepname = self.name.replace('.', os.path.sep) - file_path = os.path.join(path, "%s.py" % sepname) - file_package = os.path.join(path, sepname) - - if debug: sys.stderr.write("Trying:%s %s\n" % (file_path, file_package)) - - if os.path.isfile(file_path): - spec = importlib.util.spec_from_file_location(self.name, file_path) - if debug: print("file_Path:%s FOUND" % file_path) - break - elif os.path.isdir(file_package): - file_package = os.path.join(file_package, '__init__.py') - spec = importlib.util.spec_from_file_location(self.name, file_package) - if debug: print("PACKAGE:%s Found" % file_package) - break - - else: - continue - - if spec is None: - if debug: sys.stderr.write("NO SPEC :%s\n" % self.name) - spec = importlib.util.find_spec("." + self.name, package='SCons.Tool') - if spec: - found_name = 'SCons.Tool.' + self.name - add_to_scons_tools_namespace = True - if debug: sys.stderr.write("Spec Found? .%s :%s\n" % (self.name, spec)) - - if spec is None: - error_string = "No module named %s" % self.name - raise SCons.Errors.SConsEnvironmentError(error_string) - - module = importlib.util.module_from_spec(spec) - if module is None: - if debug: print("MODULE IS NONE:%s" % self.name) - error_string = "No module named %s" % self.name - raise SCons.Errors.SConsEnvironmentError(error_string) - - # Don't reload a tool we already loaded. - sys_modules_value = sys.modules.get(found_name, False) - - found_module = None - if sys_modules_value and sys_modules_value.__file__ == spec.origin: - found_module = sys.modules[found_name] + # From: http://stackoverflow.com/questions/67631/how-to-import-a-module-given-the-full-path/67692#67692 + # import importlib.util + # spec = importlib.util.spec_from_file_location("module.name", "/path/to/file.py") + # foo = importlib.util.module_from_spec(spec) + # spec.loader.exec_module(foo) + # foo.MyClass() + # Py 3 code + + + # sys.stderr.write("toolpath:%s\n" % self.toolpath) + # sys.stderr.write("SCONS.TOOL path:%s\n" % sys.modules['SCons.Tool'].__path__) + debug = False + spec = None + found_name = self.name + add_to_scons_tools_namespace = False + for path in self.toolpath: + sepname = self.name.replace('.', os.path.sep) + file_path = os.path.join(path, "%s.py" % sepname) + file_package = os.path.join(path, sepname) + + if debug: sys.stderr.write("Trying:%s %s\n" % (file_path, file_package)) + + if os.path.isfile(file_path): + spec = importlib.util.spec_from_file_location(self.name, file_path) + if debug: print("file_Path:%s FOUND" % file_path) + break + elif os.path.isdir(file_package): + file_package = os.path.join(file_package, '__init__.py') + spec = importlib.util.spec_from_file_location(self.name, file_package) + if debug: print("PACKAGE:%s Found" % file_package) + break + else: - # Not sure what to do in the case that there already - # exists sys.modules[self.name] but the source file is - # different.. ? - module = spec.loader.load_module(spec.name) - - sys.modules[found_name] = module - if add_to_scons_tools_namespace: - # If we found it in SCons.Tool, then add it to the module - setattr(SCons.Tool, self.name, module) + continue + + if spec is None: + if debug: sys.stderr.write("NO SPEC :%s\n" % self.name) + spec = importlib.util.find_spec("." + self.name, package='SCons.Tool') + if spec: + found_name = 'SCons.Tool.' + self.name + add_to_scons_tools_namespace = True + if debug: sys.stderr.write("Spec Found? .%s :%s\n" % (self.name, spec)) + + if spec is None: + error_string = "No module named %s" % self.name + raise SCons.Errors.SConsEnvironmentError(error_string) + + module = importlib.util.module_from_spec(spec) + if module is None: + if debug: print("MODULE IS NONE:%s" % self.name) + error_string = "No module named %s" % self.name + raise SCons.Errors.SConsEnvironmentError(error_string) + + # Don't reload a tool we already loaded. + sys_modules_value = sys.modules.get(found_name, False) + + found_module = None + if sys_modules_value and sys_modules_value.__file__ == spec.origin: + found_module = sys.modules[found_name] + else: + # Not sure what to do in the case that there already + # exists sys.modules[self.name] but the source file is + # different.. ? + module = spec.loader.load_module(spec.name) + + sys.modules[found_name] = module + if add_to_scons_tools_namespace: + # If we found it in SCons.Tool, then add it to the module + setattr(SCons.Tool, self.name, module) - found_module = module + found_module = module - if found_module is not None: - sys.path = oldpythonpath - return found_module + if found_module is not None: + sys.path = oldpythonpath + return found_module sys.path = oldpythonpath @@ -1130,7 +1093,7 @@ class ToolInitializer(object): so we no longer copy and re-bind them when the construction environment gets cloned. """ - for method in list(self.methods.values()): + for method in self.methods.values(): env.RemoveMethod(method) def apply_tools(self, env): @@ -1312,6 +1275,8 @@ def tool_list(platform, env): 'tar', 'zip', # File builders (text) 'textfile', + # Python scanner tool + 'python', ], env) tools = ([linker, c_compiler, cxx_compiler, diff --git a/src/engine/SCons/Tool/c++.xml b/src/engine/SCons/Tool/c++.xml index b59816b..019821c 100644 --- a/src/engine/SCons/Tool/c++.xml +++ b/src/engine/SCons/Tool/c++.xml @@ -47,6 +47,7 @@ Sets construction variables for generic POSIX C++ compilers. </sets> <uses> <item>CXXCOMSTR</item> +<item>SHCXXCOMSTR</item> </uses> </tool> @@ -54,6 +55,7 @@ Sets construction variables for generic POSIX C++ compilers. <summary> <para> The C++ compiler. +See also &cv-link-SHCXX; for compiling to shared objects.. </para> </summary> </cvar> @@ -65,6 +67,7 @@ The command line used to compile a C++ source file to an object file. Any options specified in the &cv-link-CXXFLAGS; and &cv-link-CPPFLAGS; construction variables are included on this command line. +See also &cv-link-SHCXXCOM; for compiling to shared objects.. </para> </summary> </cvar> @@ -72,9 +75,10 @@ are included on this command line. <cvar name="CXXCOMSTR"> <summary> <para> -The string displayed when a C++ source file +If set, the string displayed when a C++ source file is compiled to a (static) object file. -If this is not set, then &cv-link-CXXCOM; (the command line) is displayed. +If not set, then &cv-link-CXXCOM; (the command line) is displayed. +See also &cv-link-SHCXXCOMSTR; for compiling to shared objects.. </para> <example_commands> @@ -91,6 +95,7 @@ By default, this includes the value of &cv-link-CCFLAGS;, so that setting &cv-CCFLAGS; affects both C and C++ compilation. If you want to add C++-specific flags, you must set or override the value of &cv-link-CXXFLAGS;. +See also &cv-link-SHCXXFLAGS; for compiling to shared objects.. </para> </summary> </cvar> @@ -99,6 +104,7 @@ you must set or override the value of &cv-link-CXXFLAGS;. <summary> <para> The C++ compiler used for generating shared-library objects. +See also &cv-link-CXX; for compiling to static objects. </para> </summary> </cvar> @@ -111,6 +117,7 @@ to a shared-library object file. Any options specified in the &cv-link-SHCXXFLAGS; and &cv-link-CPPFLAGS; construction variables are included on this command line. +See also &cv-link-CXXCOM; for compiling to static objects. </para> </summary> </cvar> @@ -118,9 +125,10 @@ are included on this command line. <cvar name="SHCXXCOMSTR"> <summary> <para> -The string displayed when a C++ source file +If set, the string displayed when a C++ source file is compiled to a shared object file. -If this is not set, then &cv-link-SHCXXCOM; (the command line) is displayed. +If not set, then &cv-link-SHCXXCOM; (the command line) is displayed. +See also &cv-link-CXXCOMSTR; for compiling to static objects. </para> <example_commands> @@ -134,6 +142,7 @@ env = Environment(SHCXXCOMSTR = "Compiling shared object $TARGET") <para> Options that are passed to the C++ compiler to generate shared-library objects. +See also &cv-link-CXXFLAGS; for compiling to static objects. </para> </summary> </cvar> diff --git a/src/engine/SCons/Tool/cc.xml b/src/engine/SCons/Tool/cc.xml index 06e73ff..e47cf2d 100644 --- a/src/engine/SCons/Tool/cc.xml +++ b/src/engine/SCons/Tool/cc.xml @@ -51,6 +51,8 @@ Sets construction variables for generic POSIX C compilers. </sets> <uses> <item>PLATFORM</item> +<item>CCCOMSTR</item> +<item>SHCCCOMSTR</item> </uses> </tool> @@ -67,8 +69,8 @@ The C compiler. <para> The command line used to compile a C source file to a (static) object file. Any options specified in the &cv-link-CFLAGS;, &cv-link-CCFLAGS; and -&cv-link-CPPFLAGS; construction variables are included on this command -line. +&cv-link-CPPFLAGS; construction variables are included on this command line. +See also &cv-link-SHCCCOM; for compiling to shared objects. </para> </summary> </cvar> @@ -76,9 +78,10 @@ line. <cvar name="CCCOMSTR"> <summary> <para> -The string displayed when a C source file +If set, the string displayed when a C source file is compiled to a (static) object file. -If this is not set, then &cv-link-CCCOM; (the command line) is displayed. +If not set, then &cv-link-CCCOM; (the command line) is displayed. +See also &cv-link-SHCCCOMSTR; for compiling to shared objects. </para> <example_commands> @@ -91,6 +94,7 @@ env = Environment(CCCOMSTR = "Compiling static object $TARGET") <summary> <para> General options that are passed to the C and C++ compilers. +See also &cv-link-SHCCFLAGS; for compiling to shared objects. </para> </summary> </cvar> @@ -99,6 +103,7 @@ General options that are passed to the C and C++ compilers. <summary> <para> General options that are passed to the C compiler (C only; not C++). +See also &cv-link-SHCFLAGS; for compiling to shared objects. </para> </summary> </cvar> @@ -156,6 +161,7 @@ The default list is: <summary> <para> The C compiler used for generating shared-library objects. +See also &cv-link-CC; for compiling to static objects. </para> </summary> </cvar> @@ -169,6 +175,7 @@ Any options specified in the &cv-link-SHCFLAGS;, &cv-link-SHCCFLAGS; and &cv-link-CPPFLAGS; construction variables are included on this command line. +See also &cv-link-CCCOM; for compiling to static objects. </para> </summary> </cvar> @@ -176,9 +183,10 @@ are included on this command line. <cvar name="SHCCCOMSTR"> <summary> <para> -The string displayed when a C source file +If set, the string displayed when a C source file is compiled to a shared object file. -If this is not set, then &cv-link-SHCCCOM; (the command line) is displayed. +If not set, then &cv-link-SHCCCOM; (the command line) is displayed. +See also &cv-link-CCCOMSTR; for compiling to static objects. </para> <example_commands> @@ -192,6 +200,7 @@ env = Environment(SHCCCOMSTR = "Compiling shared object $TARGET") <para> Options that are passed to the C and C++ compilers to generate shared-library objects. +See also &cv-link-CCFLAGS; for compiling to static objects. </para> </summary> </cvar> @@ -201,6 +210,7 @@ to generate shared-library objects. <para> Options that are passed to the C compiler (only; not C++) to generate shared-library objects. +See also &cv-link-CFLAGS; for compiling to static objects. </para> </summary> </cvar> diff --git a/src/engine/SCons/Tool/cyglink.py b/src/engine/SCons/Tool/cyglink.py index c3d78de..fbb6d24 100644 --- a/src/engine/SCons/Tool/cyglink.py +++ b/src/engine/SCons/Tool/cyglink.py @@ -8,8 +8,6 @@ selection method. """ -from __future__ import absolute_import, print_function - import re import os diff --git a/src/engine/SCons/Tool/dmd.py b/src/engine/SCons/Tool/dmd.py index 6a64a72..3cc4ed0 100644 --- a/src/engine/SCons/Tool/dmd.py +++ b/src/engine/SCons/Tool/dmd.py @@ -1,5 +1,3 @@ -from __future__ import print_function - """SCons.Tool.dmd Tool-specific initialization for the Digital Mars D compiler. diff --git a/src/engine/SCons/Tool/docbook/docbook-xsl-1.76.1/extensions/docbook.py b/src/engine/SCons/Tool/docbook/docbook-xsl-1.76.1/extensions/docbook.py index de1375d..3d53bf7 100644 --- a/src/engine/SCons/Tool/docbook/docbook-xsl-1.76.1/extensions/docbook.py +++ b/src/engine/SCons/Tool/docbook/docbook-xsl-1.76.1/extensions/docbook.py @@ -1,7 +1,5 @@ # docbook.py: extension module # $Id: docbook.py 8353 2009-03-17 16:57:50Z mzjn $ -from __future__ import print_function - import sys import string import libxml2 @@ -69,9 +67,9 @@ def adjustColumnWidths(ctx, nodeset): relPart = 0.0 absPart = 0.0 - starPos = string.find(width, "*") + starPos = width.find("*") if starPos >= 0: - relPart, absPart = string.split(width, "*", 2) + relPart, absPart = width.split("*", 2) relPart = float(relPart) relTotal = relTotal + float(relPart) else: @@ -113,7 +111,7 @@ def adjustColumnWidths(ctx, nodeset): widths = correctRoundingError(widths) else: pixelWidth = nominalWidth - if string.find(tableWidth, "%") < 0: + if '%' not in tableWidth: pixelWidth = convertLength(tableWidth) if pixelWidth <= absTotal: @@ -127,7 +125,7 @@ def adjustColumnWidths(ctx, nodeset): relParts[count] = rel + absParts[count] absTotal = absTotal + rel + absParts[count] - if string.find(tableWidth, "%") < 0: + if '%' not in tableWidth: for count in range(len(relParts)): if foStylesheet: pixels = relParts[count] diff --git a/src/engine/SCons/Tool/docbook/docbook-xsl-1.76.1/extensions/xslt.py b/src/engine/SCons/Tool/docbook/docbook-xsl-1.76.1/extensions/xslt.py index d5529b8..0a4ff92 100644 --- a/src/engine/SCons/Tool/docbook/docbook-xsl-1.76.1/extensions/xslt.py +++ b/src/engine/SCons/Tool/docbook/docbook-xsl-1.76.1/extensions/xslt.py @@ -1,7 +1,5 @@ #!/usr/bin/python -u # $Id: xslt.py 8353 2009-03-17 16:57:50Z mzjn $ -from __future__ import print_function - import sys import libxml2 import libxslt diff --git a/src/engine/SCons/Tool/f03.xml b/src/engine/SCons/Tool/f03.xml index c020b81..61c02ef 100644 --- a/src/engine/SCons/Tool/f03.xml +++ b/src/engine/SCons/Tool/f03.xml @@ -77,9 +77,9 @@ for all Fortran versions. <cvar name="F03COMSTR"> <summary> <para> -The string displayed when a Fortran 03 source file +If set, the string displayed when a Fortran 03 source file is compiled to an object file. -If this is not set, then &cv-link-F03COM; or &cv-link-FORTRANCOM; +If not set, then &cv-link-F03COM; or &cv-link-FORTRANCOM; (the command line) is displayed. </para> </summary> @@ -217,10 +217,10 @@ for all Fortran versions. <cvar name="F03PPCOMSTR"> <summary> <para> -The string displayed when a Fortran 03 source file +If set, the string displayed when a Fortran 03 source file is compiled to an object file after first running the file through the C preprocessor. -If this is not set, then &cv-link-F03PPCOM; or &cv-link-FORTRANPPCOM; +If not set, then &cv-link-F03PPCOM; or &cv-link-FORTRANPPCOM; (the command line) is displayed. </para> </summary> @@ -256,9 +256,9 @@ for all Fortran versions. <cvar name="SHF03COMSTR"> <summary> <para> -The string displayed when a Fortran 03 source file +If set, the string displayed when a Fortran 03 source file is compiled to a shared-library object file. -If this is not set, then &cv-link-SHF03COM; or &cv-link-SHFORTRANCOM; +If not set, then &cv-link-SHF03COM; or &cv-link-SHFORTRANCOM; (the command line) is displayed. </para> </summary> @@ -299,10 +299,10 @@ for all Fortran versions. <cvar name="SHF03PPCOMSTR"> <summary> <para> -The string displayed when a Fortran 03 source file +If set, the string displayed when a Fortran 03 source file is compiled to a shared-library object file after first running the file through the C preprocessor. -If this is not set, then &cv-link-SHF03PPCOM; or &cv-link-SHFORTRANPPCOM; +If not set, then &cv-link-SHF03PPCOM; or &cv-link-SHFORTRANPPCOM; (the command line) is displayed. </para> </summary> diff --git a/src/engine/SCons/Tool/f08.py b/src/engine/SCons/Tool/f08.py index 7fa5872..64bac8e 100644 --- a/src/engine/SCons/Tool/f08.py +++ b/src/engine/SCons/Tool/f08.py @@ -8,8 +8,6 @@ selection method. """ -from __future__ import absolute_import - # # __COPYRIGHT__ # diff --git a/src/engine/SCons/Tool/f08.xml b/src/engine/SCons/Tool/f08.xml index 802e4cc..a56d60b 100644 --- a/src/engine/SCons/Tool/f08.xml +++ b/src/engine/SCons/Tool/f08.xml @@ -77,9 +77,9 @@ for all Fortran versions. <cvar name="F08COMSTR"> <summary> <para> -The string displayed when a Fortran 08 source file +If set, the string displayed when a Fortran 08 source file is compiled to an object file. -If this is not set, then &cv-link-F08COM; or &cv-link-FORTRANCOM; +If not set, then &cv-link-F08COM; or &cv-link-FORTRANCOM; (the command line) is displayed. </para> </summary> @@ -217,10 +217,10 @@ for all Fortran versions. <cvar name="F08PPCOMSTR"> <summary> <para> -The string displayed when a Fortran 08 source file +If set, the string displayed when a Fortran 08 source file is compiled to an object file after first running the file through the C preprocessor. -If this is not set, then &cv-link-F08PPCOM; or &cv-link-FORTRANPPCOM; +If not set, then &cv-link-F08PPCOM; or &cv-link-FORTRANPPCOM; (the command line) is displayed. </para> </summary> @@ -256,9 +256,9 @@ for all Fortran versions. <cvar name="SHF08COMSTR"> <summary> <para> -The string displayed when a Fortran 08 source file +If set, the string displayed when a Fortran 08 source file is compiled to a shared-library object file. -If this is not set, then &cv-link-SHF08COM; or &cv-link-SHFORTRANCOM; +If not set, then &cv-link-SHF08COM; or &cv-link-SHFORTRANCOM; (the command line) is displayed. </para> </summary> @@ -299,10 +299,10 @@ for all Fortran versions. <cvar name="SHF08PPCOMSTR"> <summary> <para> -The string displayed when a Fortran 08 source file +If set, the string displayed when a Fortran 08 source file is compiled to a shared-library object file after first running the file through the C preprocessor. -If this is not set, then &cv-link-SHF08PPCOM; or &cv-link-SHFORTRANPPCOM; +If not set, then &cv-link-SHF08PPCOM; or &cv-link-SHFORTRANPPCOM; (the command line) is displayed. </para> </summary> diff --git a/src/engine/SCons/Tool/f77.xml b/src/engine/SCons/Tool/f77.xml index abfc4a2..70ec721 100644 --- a/src/engine/SCons/Tool/f77.xml +++ b/src/engine/SCons/Tool/f77.xml @@ -108,9 +108,9 @@ F77 dialect will be used. By default, this is empty <cvar name="F77COMSTR"> <summary> <para> -The string displayed when a Fortran 77 source file +If set, the string displayed when a Fortran 77 source file is compiled to an object file. -If this is not set, then &cv-link-F77COM; or &cv-link-FORTRANCOM; +If not set, then &cv-link-F77COM; or &cv-link-FORTRANCOM; (the command line) is displayed. </para> </summary> @@ -230,10 +230,10 @@ for all Fortran versions. <cvar name="F77PPCOMSTR"> <summary> <para> -The string displayed when a Fortran 77 source file +If set, the string displayed when a Fortran 77 source file is compiled to an object file after first running the file through the C preprocessor. -If this is not set, then &cv-link-F77PPCOM; or &cv-link-FORTRANPPCOM; +If not set, then &cv-link-F77PPCOM; or &cv-link-FORTRANPPCOM; (the command line) is displayed. </para> </summary> @@ -269,9 +269,9 @@ for all Fortran versions. <cvar name="SHF77COMSTR"> <summary> <para> -The string displayed when a Fortran 77 source file +If set, the string displayed when a Fortran 77 source file is compiled to a shared-library object file. -If this is not set, then &cv-link-SHF77COM; or &cv-link-SHFORTRANCOM; +If not set, then &cv-link-SHF77COM; or &cv-link-SHFORTRANCOM; (the command line) is displayed. </para> </summary> @@ -312,10 +312,10 @@ for all Fortran versions. <cvar name="SHF77PPCOMSTR"> <summary> <para> -The string displayed when a Fortran 77 source file +If set, the string displayed when a Fortran 77 source file is compiled to a shared-library object file after first running the file through the C preprocessor. -If this is not set, then &cv-link-SHF77PPCOM; or &cv-link-SHFORTRANPPCOM; +If not set, then &cv-link-SHF77PPCOM; or &cv-link-SHFORTRANPPCOM; (the command line) is displayed. </para> </summary> diff --git a/src/engine/SCons/Tool/f90.xml b/src/engine/SCons/Tool/f90.xml index 94249a3..64dc6e1 100644 --- a/src/engine/SCons/Tool/f90.xml +++ b/src/engine/SCons/Tool/f90.xml @@ -77,9 +77,9 @@ for all Fortran versions. <cvar name="F90COMSTR"> <summary> <para> -The string displayed when a Fortran 90 source file +If set, the string displayed when a Fortran 90 source file is compiled to an object file. -If this is not set, then &cv-link-F90COM; or &cv-link-FORTRANCOM; +If not set, then &cv-link-F90COM; or &cv-link-FORTRANCOM; (the command line) is displayed. </para> </summary> @@ -217,9 +217,9 @@ for all Fortran versions. <cvar name="F90PPCOMSTR"> <summary> <para> -The string displayed when a Fortran 90 source file +If set, the string displayed when a Fortran 90 source file is compiled after first running the file through the C preprocessor. -If this is not set, then &cv-link-F90PPCOM; or &cv-link-FORTRANPPCOM; +If not set, then &cv-link-F90PPCOM; or &cv-link-FORTRANPPCOM; (the command line) is displayed. </para> </summary> @@ -255,9 +255,9 @@ for all Fortran versions. <cvar name="SHF90COMSTR"> <summary> <para> -The string displayed when a Fortran 90 source file +If set, the string displayed when a Fortran 90 source file is compiled to a shared-library object file. -If this is not set, then &cv-link-SHF90COM; or &cv-link-SHFORTRANCOM; +If not set, then &cv-link-SHF90COM; or &cv-link-SHFORTRANCOM; (the command line) is displayed. </para> </summary> @@ -298,10 +298,10 @@ for all Fortran versions. <cvar name="SHF90PPCOMSTR"> <summary> <para> -The string displayed when a Fortran 90 source file +If set, the string displayed when a Fortran 90 source file is compiled to a shared-library object file after first running the file through the C preprocessor. -If this is not set, then &cv-link-SHF90PPCOM; or &cv-link-SHFORTRANPPCOM; +If not set, then &cv-link-SHF90PPCOM; or &cv-link-SHFORTRANPPCOM; (the command line) is displayed. </para> </summary> diff --git a/src/engine/SCons/Tool/f95.xml b/src/engine/SCons/Tool/f95.xml index 4a4db46..4bda653 100644 --- a/src/engine/SCons/Tool/f95.xml +++ b/src/engine/SCons/Tool/f95.xml @@ -77,9 +77,9 @@ for all Fortran versions. <cvar name="F95COMSTR"> <summary> <para> -The string displayed when a Fortran 95 source file +If set, the string displayed when a Fortran 95 source file is compiled to an object file. -If this is not set, then &cv-link-F95COM; or &cv-link-FORTRANCOM; +If not set, then &cv-link-F95COM; or &cv-link-FORTRANCOM; (the command line) is displayed. </para> </summary> @@ -217,10 +217,10 @@ for all Fortran versions. <cvar name="F95PPCOMSTR"> <summary> <para> -The string displayed when a Fortran 95 source file +If set, the string displayed when a Fortran 95 source file is compiled to an object file after first running the file through the C preprocessor. -If this is not set, then &cv-link-F95PPCOM; or &cv-link-FORTRANPPCOM; +If not set, then &cv-link-F95PPCOM; or &cv-link-FORTRANPPCOM; (the command line) is displayed. </para> </summary> @@ -256,9 +256,9 @@ for all Fortran versions. <cvar name="SHF95COMSTR"> <summary> <para> -The string displayed when a Fortran 95 source file +If set, the string displayed when a Fortran 95 source file is compiled to a shared-library object file. -If this is not set, then &cv-link-SHF95COM; or &cv-link-SHFORTRANCOM; +If not set, then &cv-link-SHF95COM; or &cv-link-SHFORTRANCOM; (the command line) is displayed. </para> </summary> @@ -299,10 +299,10 @@ for all Fortran versions. <cvar name="SHF95PPCOMSTR"> <summary> <para> -The string displayed when a Fortran 95 source file +If set, the string displayed when a Fortran 95 source file is compiled to a shared-library object file after first running the file through the C preprocessor. -If this is not set, then &cv-link-SHF95PPCOM; or &cv-link-SHFORTRANPPCOM; +If not set, then &cv-link-SHF95PPCOM; or &cv-link-SHFORTRANPPCOM; (the command line) is displayed. </para> </summary> diff --git a/src/engine/SCons/Tool/fortran.xml b/src/engine/SCons/Tool/fortran.xml index 53bcfb1..7b3c51f 100644 --- a/src/engine/SCons/Tool/fortran.xml +++ b/src/engine/SCons/Tool/fortran.xml @@ -73,9 +73,9 @@ are included on this command line. <cvar name="FORTRANCOMSTR"> <summary> <para> -The string displayed when a Fortran source file +If set, the string displayed when a Fortran source file is compiled to an object file. -If this is not set, then &cv-link-FORTRANCOM; +If not set, then &cv-link-FORTRANCOM; (the command line) is displayed. </para> </summary> @@ -285,10 +285,10 @@ construction variables are included on this command line. <cvar name="FORTRANPPCOMSTR"> <summary> <para> -The string displayed when a Fortran source file +If set, the string displayed when a Fortran source file is compiled to an object file after first running the file through the C preprocessor. -If this is not set, then &cv-link-FORTRANPPCOM; +If not set, then &cv-link-FORTRANPPCOM; (the command line) is displayed. </para> </summary> @@ -330,9 +330,9 @@ to a shared-library object file. <cvar name="SHFORTRANCOMSTR"> <summary> <para> -The string displayed when a Fortran source file +If set, the string displayed when a Fortran source file is compiled to a shared-library object file. -If this is not set, then &cv-link-SHFORTRANCOM; +If not set, then &cv-link-SHFORTRANCOM; (the command line) is displayed. </para> </summary> @@ -364,10 +364,10 @@ are included on this command line. <cvar name="SHFORTRANPPCOMSTR"> <summary> <para> -The string displayed when a Fortran source file +If set, the string displayed when a Fortran source file is compiled to a shared-library object file after first running the file through the C preprocessor. -If this is not set, then &cv-link-SHFORTRANPPCOM; +If not set, then &cv-link-SHFORTRANPPCOM; (the command line) is displayed. </para> </summary> diff --git a/src/engine/SCons/Tool/gdc.py b/src/engine/SCons/Tool/gdc.py index 0c6a8ab..ecf4f3a 100644 --- a/src/engine/SCons/Tool/gdc.py +++ b/src/engine/SCons/Tool/gdc.py @@ -1,5 +1,3 @@ -from __future__ import print_function - """SCons.Tool.gdc Tool-specific initialization for the GDC compiler. diff --git a/src/engine/SCons/Tool/install.py b/src/engine/SCons/Tool/install.py index dcb3581..06f7902 100644 --- a/src/engine/SCons/Tool/install.py +++ b/src/engine/SCons/Tool/install.py @@ -29,8 +29,6 @@ selection method. # OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION # WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. # -from __future__ import print_function - __revision__ = "__FILE__ __REVISION__ __DATE__ __DEVELOPER__" import os diff --git a/src/engine/SCons/Tool/install.xml b/src/engine/SCons/Tool/install.xml index 150308b..ce70d91 100644 --- a/src/engine/SCons/Tool/install.xml +++ b/src/engine/SCons/Tool/install.xml @@ -50,10 +50,24 @@ a builder. </para> <example_commands> -env.Install('/usr/local/bin', source = ['foo', 'bar']) +env.Install(target='/usr/local/bin', source=['foo', 'bar']) </example_commands> <para> +Note that if target paths chosen for the +&Install; builder (and the related &InstallAs; and +&InstallVersionedLib; builders) are outside the +project tree, such as in the example above, +they may not be selected for "building" by default, +since in the absence of other instructions +&scons; builds targets that are underneath the top directory +(the directory that contains the &SConstruct; file, +usually the current directory). +Use command line targets or the &Default; function +in this case. +</para> + +<para> If the <option>--install-sandbox</option> command line option is given, the target directory will be prefixed by the directory path specified. @@ -86,12 +100,16 @@ arguments list different numbers of files or directories. </para> <example_commands> -env.InstallAs(target = '/usr/local/bin/foo', - source = 'foo_debug') -env.InstallAs(target = ['../lib/libfoo.a', '../lib/libbar.a'], - source = ['libFOO.a', 'libBAR.a']) +env.InstallAs(target='/usr/local/bin/foo', + source='foo_debug') +env.InstallAs(target=['../lib/libfoo.a', '../lib/libbar.a'], + source=['libFOO.a', 'libBAR.a']) </example_commands> +<para> +See the note under &Install;. +</para> + </summary> </builder> @@ -103,9 +121,14 @@ architecture will be generated based on symlinks of the source library. </para> <example_commands> -env.InstallVersionedLib(target = '/usr/local/bin/foo', - source = 'libxyz.1.5.2.so') +env.InstallVersionedLib(target='/usr/local/bin/foo', + source='libxyz.1.5.2.so') </example_commands> + +<para> +See the note under &Install;. +</para> + </summary> </builder> diff --git a/src/engine/SCons/Tool/intelc.py b/src/engine/SCons/Tool/intelc.py index 5a101b4..0880976 100644 --- a/src/engine/SCons/Tool/intelc.py +++ b/src/engine/SCons/Tool/intelc.py @@ -30,8 +30,6 @@ selection method. # LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION # OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION # WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. -from __future__ import division, print_function - __revision__ = "__FILE__ __REVISION__ __DATE__ __DEVELOPER__" import math, sys, os.path, glob, string, re @@ -495,15 +493,15 @@ def generate(env, version=None, abi=None, topdir=None, verbose=0): 'LIB' : libdir, 'PATH' : bindir, 'LD_LIBRARY_PATH' : libdir} - for p in list(paths.keys()): - env.PrependENVPath(p, os.path.join(topdir, paths[p])) + for p, v in paths.items(): + env.PrependENVPath(p, os.path.join(topdir, v)) if is_mac: paths={'INCLUDE' : 'include', 'LIB' : libdir, 'PATH' : bindir, 'LD_LIBRARY_PATH' : libdir} - for p in list(paths.keys()): - env.PrependENVPath(p, os.path.join(topdir, paths[p])) + for p, v in paths.items(): + env.PrependENVPath(p, os.path.join(topdir, v)) if is_windows: # env key reg valname default subdir of top paths=(('INCLUDE', 'IncludeDir', 'Include'), diff --git a/src/engine/SCons/Tool/javac.xml b/src/engine/SCons/Tool/javac.xml index bc89342..893130b 100644 --- a/src/engine/SCons/Tool/javac.xml +++ b/src/engine/SCons/Tool/javac.xml @@ -95,9 +95,9 @@ See its __doc__ string for a discussion of the format. </para> <example_commands> - env.Java(target = 'classes', source = 'src') - env.Java(target = 'classes', source = ['src1', 'src2']) - env.Java(target = 'classes', source = ['File1.java', 'File2.java']) +env.Java(target = 'classes', source = 'src') +env.Java(target = 'classes', source = ['src1', 'src2']) +env.Java(target = 'classes', source = ['File1.java', 'File2.java']) </example_commands> <para> @@ -114,8 +114,8 @@ See its __doc__ string for a discussion of the format. </para> <example_commands> - env = Environment() - env['ENV']['LANG'] = 'en_GB.UTF-8' +env = Environment() +env['ENV']['LANG'] = 'en_GB.UTF-8' </example_commands> </summary> </builder> @@ -174,7 +174,7 @@ See its __doc__ string for a discussion of the format. </para> <example_commands> - env = Environment(JAVACCOMSTR = "Compiling class files $TARGETS from $SOURCES") +env = Environment(JAVACCOMSTR = "Compiling class files $TARGETS from $SOURCES") </example_commands> </summary> </cvar> diff --git a/src/engine/SCons/Tool/ldc.py b/src/engine/SCons/Tool/ldc.py index 3e12199..f915569 100644 --- a/src/engine/SCons/Tool/ldc.py +++ b/src/engine/SCons/Tool/ldc.py @@ -1,5 +1,3 @@ -from __future__ import print_function - """SCons.Tool.ldc Tool-specific initialization for the LDC compiler. diff --git a/src/engine/SCons/Tool/link.py b/src/engine/SCons/Tool/link.py index 5d920fb..d52c90d 100644 --- a/src/engine/SCons/Tool/link.py +++ b/src/engine/SCons/Tool/link.py @@ -30,8 +30,6 @@ selection method. # OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION # WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. # -from __future__ import print_function - __revision__ = "__FILE__ __REVISION__ __DATE__ __DEVELOPER__" import sys diff --git a/src/engine/SCons/Tool/link.xml b/src/engine/SCons/Tool/link.xml index 654dafc..281e147 100644 --- a/src/engine/SCons/Tool/link.xml +++ b/src/engine/SCons/Tool/link.xml @@ -32,9 +32,6 @@ based on the types of source files. </para> </summary> <sets> -<item>SHLINK</item> -<item>SHLINKFLAGS</item> -<item>SHLINKCOM</item> <item>LINK</item> <item>LINKFLAGS</item> <item>LINKCOM</item> @@ -42,6 +39,9 @@ based on the types of source files. <item>LIBDIRSUFFIX</item> <item>LIBLINKPREFIX</item> <item>LIBLINKSUFFIX</item> +<item>SHLINK</item> +<item>SHLINKFLAGS</item> +<item>SHLINKCOM</item> <item>SHLIBSUFFIX</item> <item>__SHLIBVERSIONFLAGS</item> <item>LDMODULE</item> @@ -55,8 +55,8 @@ based on the types of source files. <item>__LDMODULEVERSIONFLAGS</item> </sets> <uses> -<item>SHLINKCOMSTR</item> <item>LINKCOMSTR</item> +<item>SHLINKCOMSTR</item> <item>LDMODULECOMSTR</item> </uses> </tool> @@ -184,8 +184,8 @@ On other systems, this is the same as &cv-link-SHLINK;. <cvar name="LDMODULECOMSTR"> <summary> <para> -The string displayed when building loadable modules. -If this is not set, then &cv-link-LDMODULECOM; (the command line) is displayed. +If set, the string displayed when building loadable modules. +If not set, then &cv-link-LDMODULECOM; (the command line) is displayed. </para> </summary> </cvar> @@ -243,6 +243,7 @@ set. <summary> <para> The linker. +See also &cv-link-SHLINK; for linking shared objects. </para> </summary> </cvar> @@ -251,6 +252,7 @@ The linker. <summary> <para> The command line used to link object files into an executable. +See also &cv-link-SHLINKCOM; for linking shared objects. </para> </summary> </cvar> @@ -258,9 +260,10 @@ The command line used to link object files into an executable. <cvar name="LINKCOMSTR"> <summary> <para> -The string displayed when object files +If set, the string displayed when object files are linked into an executable. -If this is not set, then &cv-link-LINKCOM; (the command line) is displayed. +If not set, then &cv-link-LINKCOM; (the command line) is displayed. +See also &cv-link-SHLINKCOMSTR;. for linking shared objects. </para> <example_commands> @@ -290,6 +293,7 @@ and &cv-link-_LIBDIRFLAGS; above, for the variable that expands to library search path options. +See also &cv-link-SHLINKFLAGS;. for linking shared objects. </para> </summary> </cvar> @@ -317,6 +321,7 @@ set. <summary> <para> The linker for programs that use shared libraries. +See also &cv-link-LINK; for linking static objects. </para> </summary> </cvar> @@ -325,6 +330,7 @@ The linker for programs that use shared libraries. <summary> <para> The command line used to link programs using shared libraries. +See also &cv-link-LINKCOM; for linking static objects. </para> </summary> </cvar> @@ -334,6 +340,7 @@ The command line used to link programs using shared libraries. <para> The string displayed when programs using shared libraries are linked. If this is not set, then &cv-link-SHLINKCOM; (the command line) is displayed. +See also &cv-link-LINKCOMSTR; for linking static objects. </para> <example_commands> @@ -363,6 +370,7 @@ and &cv-link-_LIBDIRFLAGS; above, for the variable that expands to library search path options. +See also &cv-link-LINKFLAGS; for linking static objects. </para> </summary> </cvar> diff --git a/src/engine/SCons/Tool/mslink.py b/src/engine/SCons/Tool/mslink.py index fc5f009..bab78d8 100644 --- a/src/engine/SCons/Tool/mslink.py +++ b/src/engine/SCons/Tool/mslink.py @@ -30,8 +30,6 @@ selection method. # OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION # WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. # -from __future__ import print_function - __revision__ = "__FILE__ __REVISION__ __DATE__ __DEVELOPER__" import os diff --git a/src/engine/SCons/Tool/msvc.xml b/src/engine/SCons/Tool/msvc.xml index dcac6cc..100c84c 100644 --- a/src/engine/SCons/Tool/msvc.xml +++ b/src/engine/SCons/Tool/msvc.xml @@ -355,6 +355,7 @@ constructor; setting it later has no effect. Valid values for Windows are <literal>14.2</literal>, <literal>14.1</literal>, +<literal>14.1Exp</literal>, <literal>14.0</literal>, <literal>14.0Exp</literal>, <literal>12.0</literal>, diff --git a/src/engine/SCons/Tool/msvs.py b/src/engine/SCons/Tool/msvs.py index 929e558..7332b49 100644 --- a/src/engine/SCons/Tool/msvs.py +++ b/src/engine/SCons/Tool/msvs.py @@ -29,9 +29,6 @@ selection method. # LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION # OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION # WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. - -from __future__ import print_function - __revision__ = "__FILE__ __REVISION__ __DATE__ __DEVELOPER__" import SCons.compat @@ -220,7 +217,7 @@ class _UserGenerator(object): for var, src in dbg_settings.items(): # Update only expected keys trg = {} - for key in [k for k in list(self.usrdebg.keys()) if k in src]: + for key in [k for k in self.usrdebg.keys() if k in src]: trg[key] = str(src[key]) self.configs[var].debug = trg @@ -544,7 +541,7 @@ class _DSPGenerator(object): if t[1] in self.env: if SCons.Util.is_List(self.env[t[1]]): for i in self.env[t[1]]: - if not i in self.sources[t[0]]: + if i not in self.sources[t[0]]: self.sources[t[0]].append(i) else: if not self.env[t[1]] in self.sources[t[0]]: @@ -578,11 +575,10 @@ class _DSPGenerator(object): for i in range(len(variants)): AddConfig(self, variants[i], buildtarget[i], outdir[i], runfile[i], cmdargs[i], cppdefines[i], cpppaths[i]) - self.platforms = [] - for key in list(self.configs.keys()): - platform = self.configs[key].platform - if platform not in self.platforms: - self.platforms.append(platform) + seen = set() + self.platforms = [p.platform for p in self.configs.values() + if not (p.platform in seen or seen.add(p.platform))] + def Build(self): pass @@ -702,7 +698,7 @@ class _GenerateV6DSP(_DSPGenerator): 'Resource Files': 'r|rc|ico|cur|bmp|dlg|rc2|rct|bin|cnt|rtf|gif|jpg|jpeg|jpe', 'Other Files': ''} - for kind in sorted(list(categories.keys()), key=lambda a: a.lower()): + for kind in sorted(categories.keys(), key=lambda a: a.lower()): if not self.sources[kind]: continue # skip empty groups @@ -1003,7 +999,7 @@ class _GenerateV7DSP(_DSPGenerator, _GenerateV7User): self.file.write('\t<Files>\n') - cats = sorted([k for k in list(categories.keys()) if self.sources[k]], + cats = sorted([k for k in categories.keys() if self.sources[k]], key=lambda a: a.lower()) for kind in cats: if len(cats) > 1: @@ -1348,7 +1344,7 @@ class _GenerateV10DSP(_DSPGenerator, _GenerateV10User): 'Resource Files': 'r;rc;ico;cur;bmp;dlg;rc2;rct;bin;cnt;rtf;gif;jpg;jpeg;jpe', 'Other Files': ''} - cats = sorted([k for k in list(categories.keys()) if self.sources[k]], + cats = sorted([k for k in categories.keys() if self.sources[k]], key = lambda a: a.lower()) # print vcxproj.filters file first @@ -1505,11 +1501,9 @@ class _GenerateV7DSW(_DSWGenerator): for variant in env['variant']: AddConfig(self, variant) - self.platforms = [] - for key in list(self.configs.keys()): - platform = self.configs[key].platform - if platform not in self.platforms: - self.platforms.append(platform) + seen = set() + self.platforms = [p.platform for p in self.configs.values() + if not (p.platform in seen or seen.add(p.platform))] def GenerateProjectFilesInfo(self): for dspfile in self.dspfiles: diff --git a/src/engine/SCons/Tool/msvsTests.py b/src/engine/SCons/Tool/msvsTests.py index cc4f717..b3373ea 100644 --- a/src/engine/SCons/Tool/msvsTests.py +++ b/src/engine/SCons/Tool/msvsTests.py @@ -20,8 +20,6 @@ # OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION # WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. # -from __future__ import print_function - __revision__ = "__FILE__ __REVISION__ __DATE__ __DEVELOPER__" import os @@ -405,20 +403,20 @@ class DummyEnv(object): return self.dict return self.dict[key] - def __setitem__(self,key,value): + def __setitem__(self, key, value): self.dict[key] = value - def __getitem__(self,key): + def __getitem__(self, key): return self.dict[key] - def __contains__(self,key): + def __contains__(self, key): return key in self.dict - def has_key(self,name): + def has_key(self, name): return name in self.dict def get(self, name, value=None): - if self.has_key(name): + if name in self.dict: return self.dict[name] else: return value @@ -771,8 +769,8 @@ class msvsTestCase(unittest.TestCase): # Check expected result self.assertListEqual(list(genDSP.configs.keys()), list(expected_configs.keys())) - for key in list(genDSP.configs.keys()): - self.assertDictEqual(genDSP.configs[key].__dict__, expected_configs[key]) + for key, v in genDSP.configs.items(): + self.assertDictEqual(v.__dict__, expected_configs[key]) genDSP.Build() diff --git a/src/engine/SCons/Tool/packaging.xml b/src/engine/SCons/Tool/packaging.xml index 8ab4912..55fecec 100644 --- a/src/engine/SCons/Tool/packaging.xml +++ b/src/engine/SCons/Tool/packaging.xml @@ -34,7 +34,7 @@ A framework for building binary and source packages. <builder name="Package"> <summary> <para> -Builds a Binary Package of the given source files. +Builds a Binary Package of the given source files. </para> <example_commands> @@ -43,68 +43,4 @@ env.Package(source = FindInstalledFiles()) </summary> </builder> -<cvar name="JAR"> -<summary> -<para> -The Java archive tool. -</para> -</summary> -</cvar> - -<cvar name="JARCHDIR"> -<summary> -<para> -The directory to which the Java archive tool should change -(using the -<option>-C</option> -option). -</para> -</summary> -</cvar> - -<cvar name="JARCOM"> -<summary> -<para> -The command line used to call the Java archive tool. -</para> -</summary> -</cvar> - -<cvar name="JARCOMSTR"> -<summary> -<para> -The string displayed when the Java archive tool -is called -If this is not set, then &cv-JARCOM; (the command line) is displayed. -</para> - -<example_commands> -env = Environment(JARCOMSTR = "JARchiving $SOURCES into $TARGET") -</example_commands> -</summary> -</cvar> - -<cvar name="JARFLAGS"> -<summary> -<para> -General options passed to the Java archive tool. -By default this is set to -<option>cf</option> -to create the necessary -<command>jar</command> -file. -</para> -</summary> -</cvar> - -<cvar name="JARSUFFIX"> -<summary> -<para> -The suffix for Java archives: -<filename>.jar</filename> -by default. -</para> -</summary> -</cvar> - </sconsdoc> diff --git a/src/engine/SCons/Tool/packaging/__init__.xml b/src/engine/SCons/Tool/packaging/__init__.xml index 31c6eed..66a7aa0 100644 --- a/src/engine/SCons/Tool/packaging/__init__.xml +++ b/src/engine/SCons/Tool/packaging/__init__.xml @@ -62,24 +62,25 @@ option or with the &cv-PACKAGETYPE; construction variable. Currently the following packagers available: </para> -<para> - * msi - Microsoft Installer - * rpm - RPM Package Manger - * ipkg - Itsy Package Management System - * tarbz2 - bzip2 compressed tar - * targz - gzip compressed tar - * tarxz - xz compressed tar - * zip - zip file - * src_tarbz2 - bzip2 compressed tar source - * src_targz - gzip compressed tar source - * src_tarxz - xz compressed tar source - * src_zip - zip file source -</para> +<para><literal>msi</literal> - Microsoft Installer</para> +<para><literal>rpm</literal> - RPM Package Manger</para> +<para><literal>ipkg</literal> - Itsy Package Management System</para> +<para><literal>tarbz2</literal> - bzip2 compressed tar</para> +<para><literal>targz</literal> - gzip compressed tar</para> +<para><literal>tarxz</literal> - xz compressed tar</para> +<para><literal>zip</literal> - zip file</para> +<para><literal>src_tarbz2</literal> - bzip2 compressed tar source</para> +<para><literal>src_targz</literal> - gzip compressed tar source</para> +<para><literal>src_tarxz</literal> - xz compressed tar source</para> +<para><literal>src_zip</literal> - zip file source</para> <para> -An updated list is always available under the "package_type" option when -running "scons --help" on a project that has packaging activated. +An updated list is always available under the +<replaceable>package_type</replaceable> option when +running <command>scons --help</command> +on a project that has packaging activated. </para> + <example_commands> env = Environment(tools=['default', 'packaging']) env.Install('/bin/', 'my_program') @@ -198,20 +199,21 @@ placed if applicable. The default value is "$NAME-$VERSION". Selects the package type to build. Currently these are available: </para> -<para> - * msi - Microsoft Installer - * rpm - Redhat Package Manger - * ipkg - Itsy Package Management System - * tarbz2 - compressed tar - * targz - compressed tar - * zip - zip file - * src_tarbz2 - compressed tar source - * src_targz - compressed tar source - * src_zip - zip file source -</para> +<para><literal>msi</literal> - Microsoft Installer</para> +<para><literal>rpm</literal> - RPM Package Manger</para> +<para><literal>ipkg</literal> - Itsy Package Management System</para> +<para><literal>tarbz2</literal> - bzip2 compressed tar</para> +<para><literal>targz</literal> - gzip compressed tar</para> +<para><literal>tarxz</literal> - xz compressed tar</para> +<para><literal>zip</literal> - zip file</para> +<para><literal>src_tarbz2</literal> - bzip2 compressed tar source</para> +<para><literal>src_targz</literal> - gzip compressed tar source</para> +<para><literal>src_tarxz</literal> - xz compressed tar source</para> +<para><literal>src_zip</literal> - zip file source</para> <para> -This may be overridden with the "package_type" command line option. +This may be overridden with the <option>package_type</option> +command line option. </para> </summary> </cvar> diff --git a/src/engine/SCons/Tool/packaging/ipk.py b/src/engine/SCons/Tool/packaging/ipk.py index fe3f49b..ac8b992 100644 --- a/src/engine/SCons/Tool/packaging/ipk.py +++ b/src/engine/SCons/Tool/packaging/ipk.py @@ -173,7 +173,7 @@ Description: $X_IPK_DESCRIPTION # # close all opened files - for f in list(opened_files.values()): + for f in opened_files.values(): f.close() # call a user specified function diff --git a/src/engine/SCons/Tool/packaging/rpm.py b/src/engine/SCons/Tool/packaging/rpm.py index ebaa701..db8ae24 100644 --- a/src/engine/SCons/Tool/packaging/rpm.py +++ b/src/engine/SCons/Tool/packaging/rpm.py @@ -284,7 +284,7 @@ def build_specfile_filesection(spec, files): for file in files: # build the tagset tags = {} - for k in list(supported_tags.keys()): + for k in supported_tags.keys(): try: v = file.GetTag(k) if v: diff --git a/src/engine/SCons/Tool/python.py b/src/engine/SCons/Tool/python.py new file mode 100644 index 0000000..c61fc8d --- /dev/null +++ b/src/engine/SCons/Tool/python.py @@ -0,0 +1,49 @@ +"""SCons.Tool.python + +Registers the Python scanner for the supported Python source file suffixes. + +""" + +# +# __COPYRIGHT__ +# +# Permission is hereby granted, free of charge, to any person obtaining +# a copy of this software and associated documentation files (the +# "Software"), to deal in the Software without restriction, including +# without limitation the rights to use, copy, modify, merge, publish, +# distribute, sublicense, and/or sell copies of the Software, and to +# permit persons to whom the Software is furnished to do so, subject to +# the following conditions: +# +# The above copyright notice and this permission notice shall be included +# in all copies or substantial portions of the Software. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY +# KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE +# WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND +# NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE +# LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION +# OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION +# WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. +# + +__revision__ = "__FILE__ __REVISION__ __DATE__ __DEVELOPER__" + +import SCons.Tool +from SCons.Scanner.Python import PythonScanner, PythonSuffixes + + +def generate(env): + """Hook the python builder and scanner into the environment.""" + for suffix in PythonSuffixes: + SCons.Tool.SourceFileScanner.add_scanner(suffix, PythonScanner) + + +def exists(env): + return True + +# Local Variables: +# tab-width:4 +# indent-tabs-mode:nil +# End: +# vim: set expandtab tabstop=4 shiftwidth=4: diff --git a/src/engine/SCons/Tool/python.xml b/src/engine/SCons/Tool/python.xml new file mode 100644 index 0000000..e3f7da3 --- /dev/null +++ b/src/engine/SCons/Tool/python.xml @@ -0,0 +1,36 @@ +<?xml version="1.0"?> +<!-- +__COPYRIGHT__ + +This file is processed by the bin/SConsDoc.py module. +See its __doc__ string for a discussion of the format. +--> + +<!DOCTYPE sconsdoc [ +<!ENTITY % scons SYSTEM '../../../../doc/scons.mod'> +%scons; +<!ENTITY % builders-mod SYSTEM '../../../../doc/generated/builders.mod'> +%builders-mod; +<!ENTITY % functions-mod SYSTEM '../../../../doc/generated/functions.mod'> +%functions-mod; +<!ENTITY % tools-mod SYSTEM '../../../../doc/generated/tools.mod'> +%tools-mod; +<!ENTITY % variables-mod SYSTEM '../../../../doc/generated/variables.mod'> +%variables-mod; +]> + +<sconsdoc xmlns="http://www.scons.org/dbxsd/v1.0" + xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" + xsi:schemaLocation="http://www.scons.org/dbxsd/v1.0 http://www.scons.org/dbxsd/v1.0/scons.xsd"> + +<tool name="python"> +<summary> +<para> +Loads the Python scanner scanner into the invoking environment. When loaded, the scanner will +attempt to find implicit dependencies for any Python source files in the list of sources +provided to an actual that uses this environment. +</para> +</summary> +</tool> + +</sconsdoc> diff --git a/src/engine/SCons/Tool/qt.py b/src/engine/SCons/Tool/qt.py index 0c471f9..3dc87c0 100644 --- a/src/engine/SCons/Tool/qt.py +++ b/src/engine/SCons/Tool/qt.py @@ -31,8 +31,6 @@ selection method. # OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION # WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. # -from __future__ import print_function - __revision__ = "__FILE__ __REVISION__ __DATE__ __DEVELOPER__" import os.path diff --git a/src/engine/SCons/Tool/rpmutils.py b/src/engine/SCons/Tool/rpmutils.py index 566f8e1..2c4fb32 100644 --- a/src/engine/SCons/Tool/rpmutils.py +++ b/src/engine/SCons/Tool/rpmutils.py @@ -34,8 +34,6 @@ exact syntax. # LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION # OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION # WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. -from __future__ import print_function - __revision__ = "__FILE__ __REVISION__ __DATE__ __DEVELOPER__" diff --git a/src/engine/SCons/Tool/swig.py b/src/engine/SCons/Tool/swig.py index f139a09..4d98494 100644 --- a/src/engine/SCons/Tool/swig.py +++ b/src/engine/SCons/Tool/swig.py @@ -7,8 +7,6 @@ It will usually be imported through the generic SCons.Tool.Tool() selection method. """ -from __future__ import print_function - # # __COPYRIGHT__ # diff --git a/src/engine/SCons/Tool/tex.py b/src/engine/SCons/Tool/tex.py index fa18cf9..1d61e2d 100644 --- a/src/engine/SCons/Tool/tex.py +++ b/src/engine/SCons/Tool/tex.py @@ -31,8 +31,6 @@ selection method. # OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION # WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. # -from __future__ import print_function - __revision__ = "__FILE__ __REVISION__ __DATE__ __DEVELOPER__" import os.path diff --git a/src/engine/SCons/Tool/textfile.py b/src/engine/SCons/Tool/textfile.py index 48a2904..c233658 100644 --- a/src/engine/SCons/Tool/textfile.py +++ b/src/engine/SCons/Tool/textfile.py @@ -71,7 +71,7 @@ def _do_subst(node, subs): contents = node.get_text_contents() if subs: for (k, val) in subs: - contents = re.sub(k, val, contents) + contents = contents.replace(k, val) if 'b' in TEXTFILE_FILE_WRITE_MODE: try: diff --git a/src/engine/SCons/Tool/textfile.xml b/src/engine/SCons/Tool/textfile.xml index 957e18c..878eb9f 100644 --- a/src/engine/SCons/Tool/textfile.xml +++ b/src/engine/SCons/Tool/textfile.xml @@ -59,7 +59,7 @@ see the &b-Substfile; description for details. <para> The prefix and suffix specified by the &cv-TEXTFILEPREFIX; and &cv-TEXTFILESUFFIX; construction variables -(the null string and <filename>.txt</filename> by default, respectively) +(an empty string and <filename>.txt</filename> by default, respectively) are automatically added to the target if they are not already present. Examples: </para> @@ -124,7 +124,7 @@ the suffix is stripped and the remainder is used as the default target name. <para> The prefix and suffix specified by the &cv-SUBSTFILEPREFIX; and &cv-SUBSTFILESUFFIX; construction variables -(the null string by default in both cases) +(an empty string by default in both cases) are automatically added to the target if they are not already present. </para> @@ -218,7 +218,7 @@ lists of tuples are also acceptable. <summary> <para> The prefix used for &b-Substfile; file names, -the null string by default. +an empty string by default. </para> </summary> </cvar> @@ -227,7 +227,7 @@ the null string by default. <summary> <para> The suffix used for &b-Substfile; file names, -the null string by default. +an empty string by default. </para> </summary> </cvar> @@ -236,7 +236,7 @@ the null string by default. <summary> <para> The prefix used for &b-Textfile; file names, -the null string by default. +an empty string by default. </para> </summary> </cvar> diff --git a/src/engine/SCons/Util.py b/src/engine/SCons/Util.py index 130cc5e..22df6fa 100644 --- a/src/engine/SCons/Util.py +++ b/src/engine/SCons/Util.py @@ -34,18 +34,11 @@ import types import codecs import pprint import hashlib +from collections import UserDict, UserList, UserString, OrderedDict +from collections.abc import MappingView -PY3 = sys.version_info[0] == 3 PYPY = hasattr(sys, 'pypy_translation_info') - -from collections import UserDict, UserList, UserString -from collections.abc import Iterable, MappingView -from collections import OrderedDict - -# Don't "from types import ..." these because we need to get at the -# types module later to look for UnicodeType. - # Below not used? # InstanceType = types.InstanceType @@ -359,27 +352,17 @@ def print_tree(root, child_func, prune=0, showtags=0, margin=[0], visited=None): DictTypes = (dict, UserDict) ListTypes = (list, UserList) -try: - # Handle getting dictionary views. - SequenceTypes = (list, tuple, UserList, MappingView) -except NameError: - SequenceTypes = (list, tuple, UserList) - +# Handle getting dictionary views. +SequenceTypes = (list, tuple, UserList, MappingView) +# TODO: PY3 check this benchmarking is still correct. # Note that profiling data shows a speed-up when comparing -# explicitly with str and unicode instead of simply comparing +# explicitly with str instead of simply comparing # with basestring. (at least on Python 2.5.1) -try: - StringTypes = (str, unicode, UserString) -except NameError: - StringTypes = (str, UserString) +StringTypes = (str, UserString) -# Empirically, it is faster to check explicitly for str and -# unicode than for basestring. -try: - BaseStringTypes = (str, unicode) -except NameError: - BaseStringTypes = str +# Empirically, it is faster to check explicitly for str than for basestring. +BaseStringTypes = str def is_Dict(obj, isinstance=isinstance, DictTypes=DictTypes): return isinstance(obj, DictTypes) @@ -447,24 +430,25 @@ def flatten_sequence(sequence, isinstance=isinstance, StringTypes=StringTypes, do_flatten(item, result) return result -# Generic convert-to-string functions that abstract away whether or -# not the Python we're executing has Unicode support. The wrapper +# Generic convert-to-string functions. The wrapper # to_String_for_signature() will use a for_signature() method if the # specified object has one. # + + def to_String(s, isinstance=isinstance, str=str, UserString=UserString, BaseStringTypes=BaseStringTypes): - if isinstance(s,BaseStringTypes): + if isinstance(s, BaseStringTypes): # Early out when already a string! return s elif isinstance(s, UserString): - # s.data can only be either a unicode or a regular - # string. Please see the UserString initializer. + # s.data can only be a regular string. Please see the UserString initializer. return s.data else: return str(s) + def to_String_for_subst(s, isinstance=isinstance, str=str, to_String=to_String, BaseStringTypes=BaseStringTypes, SequenceTypes=SequenceTypes, @@ -476,12 +460,12 @@ def to_String_for_subst(s, elif isinstance(s, SequenceTypes): return ' '.join([to_String_for_subst(e) for e in s]) elif isinstance(s, UserString): - # s.data can only be either a unicode or a regular - # string. Please see the UserString initializer. + # s.data can only a regular string. Please see the UserString initializer. return s.data else: return str(s) + def to_String_for_signature(obj, to_String_for_subst=to_String_for_subst, AttributeError=AttributeError): try: @@ -491,6 +475,7 @@ def to_String_for_signature(obj, to_String_for_subst=to_String_for_subst, # pprint will output dictionary in key sorted order # with py3.5 the order was randomized. In general depending on dictionary order # which was undefined until py3.6 (where it's by insertion order) was not wise. + # TODO: Change code when floor is raised to PY36 return pprint.pformat(obj, width=1000000) else: return to_String_for_subst(obj) @@ -1508,7 +1493,7 @@ def MD5collect(signatures): def silent_intern(x): """ Perform sys.intern() on the passed argument and return the result. - If the input is ineligible (e.g. a unicode string) the original argument is + If the input is ineligible the original argument is returned and no exception is thrown. """ try: diff --git a/src/engine/SCons/UtilTests.py b/src/engine/SCons/UtilTests.py index ee07e61..d96b63f 100644 --- a/src/engine/SCons/UtilTests.py +++ b/src/engine/SCons/UtilTests.py @@ -313,7 +313,7 @@ class UtilTestCase(unittest.TestCase): """ Test the to_Bytes method""" self.assertEqual(to_bytes('Hello'), bytearray('Hello', 'utf-8'), - "Check that to_bytes creates byte array when presented with unicode string.") + "Check that to_bytes creates byte array when presented with non byte string.") def test_to_String(self): """Test the to_String() method.""" @@ -740,7 +740,7 @@ class UtilTestCase(unittest.TestCase): def test_LogicalLines(self): """Test the LogicalLines class""" - content = u""" + content = """ foo \\ bar \\ baz @@ -761,9 +761,6 @@ bling def test_intern(self): s1 = silent_intern("spam") - # TODO: Python 3.x does not have a unicode() global function - if sys.version[0] == '2': - s2 = silent_intern(unicode("unicode spam")) s3 = silent_intern(42) s4 = silent_intern("spam") assert id(s1) == id(s4) diff --git a/src/engine/SCons/Variables/ListVariableTests.py b/src/engine/SCons/Variables/ListVariableTests.py index ef4832c..8f79e07 100644 --- a/src/engine/SCons/Variables/ListVariableTests.py +++ b/src/engine/SCons/Variables/ListVariableTests.py @@ -42,7 +42,7 @@ class ListVariableTestCase(unittest.TestCase): assert o.help == 'test option help\n (all|none|comma-separated list of names)\n allowed names: one two three', repr(o.help) assert o.default == 'all', o.default assert o.validator is None, o.validator - assert not o.converter is None, o.converter + assert o.converter is not None, o.converter opts = SCons.Variables.Variables() opts.Add(SCons.Variables.ListVariable('test2', 'test2 help', diff --git a/src/engine/SCons/Variables/PackageVariableTests.py b/src/engine/SCons/Variables/PackageVariableTests.py index cda3a4a..a7e6b0d 100644 --- a/src/engine/SCons/Variables/PackageVariableTests.py +++ b/src/engine/SCons/Variables/PackageVariableTests.py @@ -80,10 +80,10 @@ class PackageVariableTestCase(unittest.TestCase): # False when we give it str(False). This assures consistent operation # through a cycle of Variables.Save(<file>) -> Variables(<file>). x = o.converter(str(True)) - assert x == True, "converter returned a string when given str(True)" + assert x, "converter returned a string when given str(True)" x = o.converter(str(False)) - assert x == False, "converter returned a string when given str(False)" + assert not x, "converter returned a string when given str(False)" def test_validator(self): """Test the PackageVariable validator""" diff --git a/src/engine/SCons/cpp.py b/src/engine/SCons/cpp.py index 5b35390..0c20c94 100644 --- a/src/engine/SCons/cpp.py +++ b/src/engine/SCons/cpp.py @@ -89,7 +89,7 @@ del op_list override = { 'if' : 'if(?!n?def)', } -l = [override.get(x, x) for x in list(Table.keys())] +l = [override.get(x, x) for x in Table.keys()] # Turn the list of expressions into one big honkin' regular expression @@ -268,7 +268,7 @@ class PreProcessor(object): d = { 'scons_current_file' : self.scons_current_file } - for op in list(Table.keys()): + for op in Table.keys(): d[op] = getattr(self, 'do_' + op) self.default_table = d diff --git a/src/engine/SCons/cppTests.py b/src/engine/SCons/cppTests.py index eff7715..eb9189d 100644 --- a/src/engine/SCons/cppTests.py +++ b/src/engine/SCons/cppTests.py @@ -20,9 +20,6 @@ # OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION # WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. # - -from __future__ import absolute_import - __revision__ = "__FILE__ __REVISION__ __DATE__ __DEVELOPER__" import atexit diff --git a/src/engine/SCons/dblite.py b/src/engine/SCons/dblite.py index 55254b3..765516c 100644 --- a/src/engine/SCons/dblite.py +++ b/src/engine/SCons/dblite.py @@ -1,6 +1,7 @@ -# dblite.py module contributed by Ralf W. Grosse-Kunstleve. -# Extended for Unicode by Steven Knight. -from __future__ import print_function +""" +dblite.py module contributed by Ralf W. Grosse-Kunstleve. +Extended for Unicode by Steven Knight. +""" import os import pickle @@ -17,26 +18,6 @@ def corruption_warning(filename): print("Warning: Discarding corrupt database:", filename) -try: - unicode -except NameError: - def is_string(s): - return isinstance(s, str) -else: - def is_string(s): - return type(s) in (str, unicode) - - -def is_bytes(s): - return isinstance(s, bytes) - - -try: - unicode('a') -except NameError: - def unicode(s): - return s - dblite_suffix = '.dblite' # TODO: Does commenting this out break switching from py2/3? @@ -181,10 +162,13 @@ class dblite(object): def __setitem__(self, key, value): self._check_writable() - if not is_string(key): + + if not isinstance(key, str): raise TypeError("key `%s' must be a string but is %s" % (key, type(key))) - if not is_bytes(value): + + if not isinstance(value, bytes): raise TypeError("value `%s' must be a bytes but is %s" % (value, type(value))) + self._dict[key] = value self._needs_sync = 0o001 @@ -214,25 +198,21 @@ def open(file, flag=None, mode=0o666): def _exercise(): db = open("tmp", "n") assert len(db) == 0 - db["foo"] = "bar" - assert db["foo"] == "bar" - db[unicode("ufoo")] = unicode("ubar") - assert db[unicode("ufoo")] == unicode("ubar") + db["foo"] = b"bar" + assert db["foo"] == b"bar" db.sync() + db = open("tmp", "c") - assert len(db) == 2, len(db) - assert db["foo"] == "bar" - db["bar"] = "foo" - assert db["bar"] == "foo" - db[unicode("ubar")] = unicode("ufoo") - assert db[unicode("ubar")] == unicode("ufoo") + assert len(db) == 1, len(db) + assert db["foo"] == b"bar" + db["bar"] = b"foo" + assert db["bar"] == b"foo" db.sync() + db = open("tmp", "r") - assert len(db) == 4, len(db) - assert db["foo"] == "bar" - assert db["bar"] == "foo" - assert db[unicode("ufoo")] == unicode("ubar") - assert db[unicode("ubar")] == unicode("ufoo") + assert len(db) == 2, len(db) + assert db["foo"] == b"bar" + assert db["bar"] == b"foo" try: db.sync() except IOError as e: @@ -240,26 +220,31 @@ def _exercise(): else: raise RuntimeError("IOError expected.") db = open("tmp", "w") - assert len(db) == 4 - db["ping"] = "pong" + assert len(db) == 2, len(db) + db["ping"] = b"pong" db.sync() + try: db[(1, 2)] = "tuple" except TypeError as e: - assert str(e) == "key `(1, 2)' must be a string but is <type 'tuple'>", str(e) + assert str(e) == "key `(1, 2)' must be a string but is <class 'tuple'>", str(e) else: raise RuntimeError("TypeError exception expected") + try: db["list"] = [1, 2] except TypeError as e: - assert str(e) == "value `[1, 2]' must be a string but is <type 'list'>", str(e) + assert str(e) == "value `[1, 2]' must be a bytes but is <class 'list'>", str(e) else: raise RuntimeError("TypeError exception expected") + db = open("tmp", "r") - assert len(db) == 5 + assert len(db) == 3, len(db) + db = open("tmp", "n") - assert len(db) == 0 + assert len(db) == 0, len(db) dblite._open("tmp.dblite", "w") + db = open("tmp", "r") dblite._open("tmp.dblite", "w").write("x") try: @@ -268,10 +253,11 @@ def _exercise(): pass else: raise RuntimeError("pickle exception expected.") + global ignore_corrupt_dbfiles ignore_corrupt_dbfiles = 2 db = open("tmp", "r") - assert len(db) == 0 + assert len(db) == 0, len(db) os.unlink("tmp.dblite") try: db = open("tmp", "w") @@ -280,6 +266,8 @@ def _exercise(): else: raise RuntimeError("IOError expected.") + print("Completed _exercise()") + if __name__ == "__main__": _exercise() diff --git a/src/script/scons-configure-cache.py b/src/script/scons-configure-cache.py index a687c9c..716315c 100644 --- a/src/script/scons-configure-cache.py +++ b/src/script/scons-configure-cache.py @@ -31,7 +31,6 @@ digits of the signature. The prefix length used for directory names can be changed by this script. """ -from __future__ import print_function import argparse import glob import json diff --git a/src/script/scons-time.py b/src/script/scons-time.py index 91a105b..e4dd863 100644 --- a/src/script/scons-time.py +++ b/src/script/scons-time.py @@ -29,7 +29,6 @@ # LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION # OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION # WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. -from __future__ import division, print_function __revision__ = "__FILE__ __REVISION__ __DATE__ __DEVELOPER__" diff --git a/src/script/scons.py b/src/script/scons.py index 2e76365..1e12898 100755 --- a/src/script/scons.py +++ b/src/script/scons.py @@ -23,8 +23,6 @@ # OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION # WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. -from __future__ import print_function - __revision__ = "__FILE__ __REVISION__ __DATE__ __DEVELOPER__" __version__ = "__VERSION__" diff --git a/src/script/sconsign.py b/src/script/sconsign.py index 5ea5ea5..726838c 100644 --- a/src/script/sconsign.py +++ b/src/script/sconsign.py @@ -23,8 +23,6 @@ # OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION # WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. -from __future__ import print_function - __revision__ = "__FILE__ __REVISION__ __DATE__ __DEVELOPER__" __version__ = "__VERSION__" diff --git a/src/setup.py b/src/setup.py index 0f9a672..261e2a4 100755 --- a/src/setup.py +++ b/src/setup.py @@ -20,9 +20,6 @@ # OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION # WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. -from __future__ import print_function - - import distutils.command.build_scripts import distutils.command.install_scripts import distutils.command.install_lib diff --git a/src/test_files.py b/src/test_files.py index 1eee11d..fcb2b4c 100644 --- a/src/test_files.py +++ b/src/test_files.py @@ -21,8 +21,6 @@ # OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION # WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. # -from __future__ import print_function - __revision__ = "__FILE__ __REVISION__ __DATE__ __DEVELOPER__" """ diff --git a/src/test_interrupts.py b/src/test_interrupts.py index de18a54..1cac046 100644 --- a/src/test_interrupts.py +++ b/src/test_interrupts.py @@ -21,8 +21,6 @@ # OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION # WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. -from __future__ import print_function - __revision__ = "__FILE__ __REVISION__ __DATE__ __DEVELOPER__" """ @@ -105,7 +103,7 @@ for f in files: indent_list.append( (line_num, match.group('try_or_except') ) ) try_except_lines[match.group('indent')] = indent_list uncaught_this_file = [] - for indent in list(try_except_lines.keys()): + for indent in try_except_lines.keys(): exc_keyboardint_seen = 0 exc_all_seen = 0 for (l,statement) in try_except_lines[indent] + [(-1,indent + 'try')]: diff --git a/src/test_pychecker.py b/src/test_pychecker.py index 56233c2..fdc8d20 100644 --- a/src/test_pychecker.py +++ b/src/test_pychecker.py @@ -20,8 +20,6 @@ # LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION # OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION # WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. -from __future__ import print_function - __revision__ = "__FILE__ __REVISION__ __DATE__ __DEVELOPER__" """ diff --git a/src/test_setup.py b/src/test_setup.py index d9fa5b5..8410be3 100644 --- a/src/test_setup.py +++ b/src/test_setup.py @@ -21,8 +21,6 @@ # OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION # WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. # -from __future__ import print_function - __revision__ = "__FILE__ __REVISION__ __DATE__ __DEVELOPER__" """ diff --git a/src/test_strings.py b/src/test_strings.py index a8a5000..b43340f 100644 --- a/src/test_strings.py +++ b/src/test_strings.py @@ -21,8 +21,6 @@ # OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION # WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. # -from __future__ import print_function - __revision__ = "__FILE__ __REVISION__ __DATE__ __DEVELOPER__" """ |