diff options
| author | Steven Knight <knight@baldmt.com> | 2008-04-17 06:35:51 (GMT) |
|---|---|---|
| committer | Steven Knight <knight@baldmt.com> | 2008-04-17 06:35:51 (GMT) |
| commit | 12ced93c8d443babcf3ba0c0a3503ed6a8fed2d9 (patch) | |
| tree | 340da0ff0ef5002982fcfd5ee2a2a5e34a4bf5e0 /src/engine | |
| parent | f80d3313d70eace1274f0b5ffbdc53b19acf8250 (diff) | |
| download | SCons-12ced93c8d443babcf3ba0c0a3503ed6a8fed2d9.zip SCons-12ced93c8d443babcf3ba0c0a3503ed6a8fed2d9.tar.gz SCons-12ced93c8d443babcf3ba0c0a3503ed6a8fed2d9.tar.bz2 | |
Merged revisions 2725-2865 via svnmerge from
http://scons.tigris.org/svn/scons/branches/core
........
r2731 | stevenknight | 2008-04-01 09:22:37 -0700 (Tue, 01 Apr 2008) | 4 lines
Fix test/Interactive/version.py, which still had the old hard-coded
Copyright string, by moving the test copy of the Copyright string
into QMTest/TestSCons.py, where it's available to all test scripts.
........
r2732 | stevenknight | 2008-04-02 13:28:32 -0700 (Wed, 02 Apr 2008) | 3 lines
Issues 317, 323, 329: Ignore the -d, -e and -w options, which we're
not going to implement.
........
r2734 | stevenknight | 2008-04-05 07:50:49 -0700 (Sat, 05 Apr 2008) | 2 lines
Fix bootstrap.py for pre-2.3 versions of Python, which don't have __file__.
........
r2735 | stevenknight | 2008-04-05 07:54:44 -0700 (Sat, 05 Apr 2008) | 2 lines
Fix the non-breaking-hyphen regular expression(s) for Python 1.5.
........
r2736 | garyo | 2008-04-05 19:15:52 -0700 (Sat, 05 Apr 2008) | 1 line
Added Users Guide section for AddMethod.
........
r2737 | stevenknight | 2008-04-08 09:43:31 -0700 (Tue, 08 Apr 2008) | 3 lines
Don't raise strings as exceptions in the exception-handling unit tests.
That's now deprecated Python behavior.
........
r2738 | stevenknight | 2008-04-08 10:36:00 -0700 (Tue, 08 Apr 2008) | 5 lines
Address a serious inefficiency in Java builds by moving the pre-build
check for whether source files exist from the Node class to the Executor
class, so we only have to perform the check once when building a whole
bunch of targets from a whole bunch of sources.
........
r2739 | stevenknight | 2008-04-08 13:53:44 -0700 (Tue, 08 Apr 2008) | 2 lines
Add a backwards-compatibility version of the "itertools" module.
........
r2740 | stevenknight | 2008-04-08 14:12:05 -0700 (Tue, 08 Apr 2008) | 2 lines
Issue 1961: speed up SCons.Util.to_String*() functions. (Benoit Belley)
........
r2741 | stevenknight | 2008-04-08 14:15:36 -0700 (Tue, 08 Apr 2008) | 2 lines
Use the itertools.izip() method for some key Node.FS methods. (Benoit Belley)
........
r2742 | stevenknight | 2008-04-08 14:35:35 -0700 (Tue, 08 Apr 2008) | 2 lines
Issues 1961: use izip() instead of zip() where possible (Benoit Belley)
........
r2743 | stevenknight | 2008-04-08 14:48:08 -0700 (Tue, 08 Apr 2008) | 3 lines
Issue 1961: more efficient get_contents() implementation for Python
function Actions. (Benoit Belley)
........
r2744 | stevenknight | 2008-04-08 17:55:30 -0700 (Tue, 08 Apr 2008) | 3 lines
Issue 1961: make SCons.Node.* state variables global in Taskmaster.py so
we avoid unneceesary attribute fetches. (Benoit Belley)
........
r2745 | stevenknight | 2008-04-08 20:40:31 -0700 (Tue, 08 Apr 2008) | 2 lines
Issue 1961: Optimize the code in Node.get_binfo(). (Benoit Belley)
........
r2746 | stevenknight | 2008-04-08 21:05:17 -0700 (Tue, 08 Apr 2008) | 3 lines
Issue 1961: Enhance the backwards-compatibility sets() module with the
ability to compare sets. (Benoit Belley)
........
r2747 | stevenknight | 2008-04-08 22:11:20 -0700 (Tue, 08 Apr 2008) | 2 lines
Issue 1961: Optimize Executor.scan(). (Benoit Belley)
........
r2748 | stevenknight | 2008-04-08 22:17:55 -0700 (Tue, 08 Apr 2008) | 3 lines
Python 1.5 compatibility: use for p in paths.keys() for dictionaries.
(Benoit Belley)
........
r2749 | stevenknight | 2008-04-08 22:54:02 -0700 (Tue, 08 Apr 2008) | 10 lines
Issue 1961: additional build optimizations:
-- Make taskmastertrace output more useful and readable.
-- Move dependency cycle checking to avoid re-doing it.
-- Have Nodes use sets, not dictionaries, to track various things.
-- Eliminate intermediate function calls from fetching Node children.
-- Add a Task.needs_execute() method to avoid doing that check over and
over as part of needs_execute().
-- Remove the unused Node.found_includes attribute.
(Benoit Belley)
........
r2750 | stevenknight | 2008-04-09 14:47:44 -0700 (Wed, 09 Apr 2008) | 4 lines
Make target Java .class files depend *only* on the input .java files as
their sources when determining if they require rebuilding. This eliminates
O(NxM) checking for every single edge in a big, overly-connected DAG mesh.
........
r2751 | stevenknight | 2008-04-09 16:58:41 -0700 (Wed, 09 Apr 2008) | 2 lines
Remove 0.95 and 0.96* release notes.
........
r2752 | stevenknight | 2008-04-10 02:24:50 -0700 (Thu, 10 Apr 2008) | 2 lines
Issue 1956: Fix --debug=stree printing its tree twice. (Benoit Belley)
........
r2753 | stevenknight | 2008-04-10 02:33:28 -0700 (Thu, 10 Apr 2008) | 2 lines
Issue 1896: Add support for the GDC D language compiler. (Matthew Wesley)
........
r2754 | stevenknight | 2008-04-10 02:39:24 -0700 (Thu, 10 Apr 2008) | 2 lines
Fix tabs.
........
r2755 | stevenknight | 2008-04-10 02:41:50 -0700 (Thu, 10 Apr 2008) | 3 lines
Issue 1964: Fix passing variable names in a list to Return() (as
already documented in the man page!) (Mike Wake)
........
r2756 | stevenknight | 2008-04-10 02:55:40 -0700 (Thu, 10 Apr 2008) | 2 lines
Support the ability to download 2.6 candidate releases (e.g. 2.6a2).
........
r2757 | stevenknight | 2008-04-10 02:58:35 -0700 (Thu, 10 Apr 2008) | 3 lines
Issue 1669: Fix the ability to use LoadableModule() under MinGW.
(Johan Boule)
........
r2758 | stevenknight | 2008-04-10 03:03:15 -0700 (Thu, 10 Apr 2008) | 3 lines
Update the test/Interactive/tree.py script for Benoit's fix to
remove duplicate tree printing.
........
r2759 | stevenknight | 2008-04-10 06:43:44 -0700 (Thu, 10 Apr 2008) | 3 lines
Fix Tool/dmd.py when no D compiler is installed -- we don't want to
search for a path if the result is None.
........
r2760 | GregNoel | 2008-04-10 15:30:34 -0700 (Thu, 10 Apr 2008) | 1 line
Issue 2009: separate Debug.caller() by functionality
........
r2761 | stevenknight | 2008-04-11 04:47:25 -0700 (Fri, 11 Apr 2008) | 3 lines
Issue 1882: Add the scons.bat directory to %PATH% so it can find python.exe.
(Anatoly Techtonik)
........
r2762 | stevenknight | 2008-04-11 09:15:22 -0700 (Fri, 11 Apr 2008) | 4 lines
Issues 1835,1901: fix the ability to list a source file multiple
times for a target by making sure we only store unique entries in the
.sconsign file.
........
r2763 | stevenknight | 2008-04-11 10:58:26 -0700 (Fri, 11 Apr 2008) | 4 lines
Issue 1882: Fix earlier patch to scons.bat by adding ~dp0;~dp0.. to
the front of %PATH%, and only executing endlocal on NT-based systems.
(Anatoly Techtonik)
........
r2764 | stevenknight | 2008-04-11 13:06:29 -0700 (Fri, 11 Apr 2008) | 4 lines
Add a Variables object and {Bool,Envum,List,Package,Path}Variable()
functions as a first step towards eventually deprecating the
Options object and {Bool,Envum,List,Package,Path}Option() functions.
........
r2765 | stevenknight | 2008-04-11 18:13:53 -0700 (Fri, 11 Apr 2008) | 3 lines
Issue 1962: Capture a test case for ListActions that contain a
command-line string containing unicode, and Python FunctionAction.
........
r2766 | stevenknight | 2008-04-11 22:03:14 -0700 (Fri, 11 Apr 2008) | 3 lines
Issue 1933: expect .py files generated by the SWIG -python option
to be in the same (sub)directory as the target.
........
r2767 | stevenknight | 2008-04-12 06:41:57 -0700 (Sat, 12 Apr 2008) | 2 lines
Remove the SCons.Options package in favor of the new SCons.Variables package.
........
r2768 | stevenknight | 2008-04-12 13:33:52 -0700 (Sat, 12 Apr 2008) | 5 lines
Issue 1971: Move the incorporation of $CCFLAGS and $SHCCFLAGS directly
into the C++ command lines (${SHCXX,CXX}COM) instead of through indirect
expansion of $CXXFLAGS and $SHCXXFLAGS. This requires removing -fPIC
from the default setting of $SHCXXFLAGS under the GNU toolchain.
........
r2769 | stevenknight | 2008-04-13 07:01:27 -0700 (Sun, 13 Apr 2008) | 3 lines
Fix __all__ definitions in the Variables/*Variable.py file that were
cause epydoc to blow up when generating developer documentation.
........
r2770 | stevenknight | 2008-04-13 11:47:49 -0700 (Sun, 13 Apr 2008) | 3 lines
Add variables= keyword argument to Environment() creation as a first
step towards deprecating options=.
........
r2771 | stevenknight | 2008-04-13 11:54:19 -0700 (Sun, 13 Apr 2008) | 2 lines
Correct underscore-instead-of-hyphen misspellings in option names.
........
r2772 | bdbaddog | 2008-04-13 17:39:21 -0700 (Sun, 13 Apr 2008) | 3 lines
Changes to fix 15 tests which were failing on cygwin.
........
r2773 | GregNoel | 2008-04-13 22:31:07 -0700 (Sun, 13 Apr 2008) | 1 line
Various fixes to tests on Darwin using 1.5.2
........
r2774 | stevenknight | 2008-04-14 15:00:44 -0700 (Mon, 14 Apr 2008) | 2 lines
Python 1.5.2 fix in new test/SWIG/subdir.py script (use the -classic option).
........
r2775 | belley | 2008-04-14 18:02:40 -0700 (Mon, 14 Apr 2008) | 32 lines
Dont use KeyboardInterrupt to stop a build! [Issue 1907]
SCons would often hang after pressing Ctrl-C. I started investigating and I
realized that most of the Python libraries are not really safe with respect to
asynchronous exceptions. Although,there are enough try/finally blocks to handle
exceptions thrown synchronously by the library code, the Python libraries are
not always protected against exceptions being thrown asynchronously, such as a
KeyboardInterrupt being thrown at a completely random location.
For example, the function Queue.empty() does not protect its mutex with a
try/finally block. If the KeyboardInterrupt exception gets thrown while the
mutex is held, any further attempt to access the Queue will lead to dead-lock
(explaining why SCons hangs sometimes after pressing CTRL-C). Even the
threading.Condition condition variables are not async-exception safe. It
therefore seems a lost battle to try to stop a build by raising an exception.
Instead, I have implemented a signal handler that tells the Jobs (and its
associated Taskmaster) to stop the build. I have been careful to wait after the
.sconsign file has been written back to re-install the default SIGINT signal
handler that raises a KeyboardInterrupt exception.
This patch is submitted against changeset 2773 of branches/core. The
regression test suite has been run on RHEL4 using Pyhon 2.5.1 and
1.5.2. My team has been using an SCons build with this patch for a
while now on Windows, Linux and OSX.
See:
http://scons.tigris.org/issues/show_bug.cgi?id=1907
Benoit Belley
........
r2777 | cournape | 2008-04-14 20:11:56 -0700 (Mon, 14 Apr 2008) | 3 lines
Initialized merge tracking via "svnmerge" with revisions "1-2776" from
http://scons.tigris.org/svn/scons/branches/fortran_refactor
........
r2788 | stevenknight | 2008-04-14 22:09:27 -0700 (Mon, 14 Apr 2008) | 2 lines
Fix the print the "script" line in the --version output.
........
r2789 | stevenknight | 2008-04-14 22:18:27 -0700 (Mon, 14 Apr 2008) | 2 lines
Add a __COPYRIGHT__ line.
........
r2790 | stevenknight | 2008-04-14 22:20:39 -0700 (Mon, 14 Apr 2008) | 3 lines
Issue 2008: in checkpoint releases, use a '.' to separate (e.g.) 0.98.0
from 0d20080414.
........
r2817 | belley | 2008-04-15 06:44:21 -0700 (Tue, 15 Apr 2008) | 13 lines
TestSCons.up_to_date() should use match_re_dotall
I changed TestSCons.up_to_date() to use match_re_dotall instead of
match_exact. This is necessary so that I can call up_to_date() with
the TestSCons.deprecated_python_expr error message in one of my test.
Note that TestSCons.not_up_to_date() is already using match_re_dotall.
Ran the test suite on both Python 2.5.1 and 1.5.2.
Benoit
........
r2818 | belley | 2008-04-15 12:10:52 -0700 (Tue, 15 Apr 2008) | 10 lines
Improved the multiple-parents.py test to also tests the following
cases:
d) Some children are ignored
e) Some children are pre-requesites
f) Some sources are missing
The test still passes. No extra bug were found.
........
r2850 | stevenknight | 2008-04-16 11:15:24 -0700 (Wed, 16 Apr 2008) | 4 lines
Fix problems with the __del__() method referencing other module functions
through global variables, which can get deleted out from under us at
shutdown.
........
r2851 | stevenknight | 2008-04-16 11:17:07 -0700 (Wed, 16 Apr 2008) | 4 lines
Fix use of --interactive with -u/-U/-D and VariantDir() by making the
method that cleans Node states between interactive commands aware of
the alter_targets() method that tells us about an associated VariantDir().
........
Diffstat (limited to 'src/engine')
55 files changed, 1378 insertions, 769 deletions
diff --git a/src/engine/MANIFEST.in b/src/engine/MANIFEST.in index 093fbd9..e4e8046 100644 --- a/src/engine/MANIFEST.in +++ b/src/engine/MANIFEST.in @@ -3,6 +3,7 @@ SCons/Action.py SCons/Builder.py SCons/compat/__init__.py SCons/compat/_scons_hashlib.py +SCons/compat/_scons_itertools.py SCons/compat/_scons_optparse.py SCons/compat/_scons_sets.py SCons/compat/_scons_sets15.py @@ -27,12 +28,7 @@ SCons/Node/__init__.py SCons/Node/Alias.py SCons/Node/FS.py SCons/Node/Python.py -SCons/Options/__init__.py -SCons/Options/BoolOption.py -SCons/Options/EnumOption.py -SCons/Options/ListOption.py -SCons/Options/PackageOption.py -SCons/Options/PathOption.py +SCons/Options.py SCons/PathList.py SCons/Platform/__init__.py SCons/Platform/aix.py @@ -162,4 +158,10 @@ SCons/Tool/wix.py SCons/Tool/yacc.py SCons/Tool/zip.py SCons/Util.py +SCons/Variables/__init__.py +SCons/Variables/BoolVariable.py +SCons/Variables/EnumVariable.py +SCons/Variables/ListVariable.py +SCons/Variables/PackageVariable.py +SCons/Variables/PathVariable.py SCons/Warnings.py diff --git a/src/engine/SCons/Action.py b/src/engine/SCons/Action.py index 367174c..d2211b7 100644 --- a/src/engine/SCons/Action.py +++ b/src/engine/SCons/Action.py @@ -865,8 +865,11 @@ class FunctionAction(_ActionAction): except AttributeError: contents = self.funccontents - return contents + env.subst(string.join(map(lambda v: '${'+v+'}', - self.varlist))) + result = [contents] + for v in self.varlist: + result.append(env.subst('${'+v+'}')) + + return string.join(result, '') def get_implicit_deps(self, target, source, env): return [] diff --git a/src/engine/SCons/ActionTests.py b/src/engine/SCons/ActionTests.py index 74664b0..f25f232 100644 --- a/src/engine/SCons/ActionTests.py +++ b/src/engine/SCons/ActionTests.py @@ -1006,10 +1006,13 @@ class CommandActionTestCase(unittest.TestCase): expect_nonexecutable = 1 elif sys.platform == 'cygwin': expect_nonexistent = 127 - expect_nonexecutable = 127 + # Newer cygwin seems to return 126 for following + expect_nonexecutable_file = 126 + expect_nonexecutable_dir = 127 else: expect_nonexistent = 127 - expect_nonexecutable = 126 + expect_nonexecutable_file = 126 + expect_nonexecutable_dir = 126 # Test that a nonexistent command returns 127 act = SCons.Action.CommandAction(python + "_no_such_command_") @@ -1020,12 +1023,12 @@ class CommandActionTestCase(unittest.TestCase): dir, tail = os.path.split(python) act = SCons.Action.CommandAction(dir) r = act([], [], env.Clone(out = outfile)) - assert r.status == expect_nonexecutable, r.status + assert r.status == expect_nonexecutable_file, r.status # Test that trying to execute a non-executable file returns 126 act = SCons.Action.CommandAction(outfile) r = act([], [], env.Clone(out = outfile)) - assert r.status == expect_nonexecutable, r.status + assert r.status == expect_nonexecutable_dir, r.status act = SCons.Action.CommandAction('%s %s 1' % (_python_, exit_py)) r = act([], [], env) diff --git a/src/engine/SCons/Debug.py b/src/engine/SCons/Debug.py index c54eae2..095f5a2 100644 --- a/src/engine/SCons/Debug.py +++ b/src/engine/SCons/Debug.py @@ -115,39 +115,57 @@ else: res = resource.getrusage(resource.RUSAGE_SELF) return res[4] - - -caller_dicts = {} - -def caller(*backlist): +# returns caller's stack +def caller_stack(*backlist): import traceback if not backlist: backlist = [0] result = [] for back in backlist: tb = traceback.extract_stack(limit=3+back) - key = tb[1][:3] - try: - entry = caller_dicts[key] - except KeyError: - entry = caller_dicts[key] = {} key = tb[0][:3] - entry[key] = entry.get(key, 0) + 1 result.append('%s:%d(%s)' % func_shorten(key)) return result +caller_bases = {} +caller_dicts = {} + +# trace a caller's stack +def caller_trace(back=0): + import traceback + tb = traceback.extract_stack(limit=3+back) + tb.reverse() + callee = tb[1][:3] + caller_bases[callee] = caller_bases.get(callee, 0) + 1 + for caller in tb[2:]: + caller = callee + caller[:3] + try: + entry = caller_dicts[callee] + except KeyError: + caller_dicts[callee] = entry = {} + entry[caller] = entry.get(caller, 0) + 1 + callee = caller + +# print a single caller and its callers, if any +def _dump_one_caller(key, file, level=0): + l = [] + for c,v in caller_dicts[key].items(): + l.append((-v,c)) + l.sort() + leader = ' '*level + for v,c in l: + file.write("%s %6d %s:%d(%s)\n" % ((leader,-v) + func_shorten(c[-3:]))) + if caller_dicts.has_key(c): + _dump_one_caller(c, file, level+1) + +# print each call tree def dump_caller_counts(file=sys.stdout): - keys = caller_dicts.keys() + keys = caller_bases.keys() keys.sort() for k in keys: - file.write("Callers of %s:%d(%s):\n" % func_shorten(k)) - counts = caller_dicts[k] - callers = counts.keys() - callers.sort() - for c in callers: - #file.write(" counts[%s] = %s\n" % (c, counts[c])) - t = ((counts[c],) + func_shorten(c)) - file.write(" %6d %s:%d(%s)\n" % t) + file.write("Callers of %s:%d(%s), %d calls:\n" + % (func_shorten(k) + (caller_bases[k],))) + _dump_one_caller(k, file) shorten_list = [ ( '/scons/SCons/', 1), @@ -168,10 +186,8 @@ def func_shorten(func_tuple): if i >= 0: if t[1]: i = i + len(t[0]) - f = f[i:] - break - return (f,)+func_tuple[1:] - + return (f[i:],)+func_tuple[1:] + return func_tuple TraceFP = {} diff --git a/src/engine/SCons/Environment.py b/src/engine/SCons/Environment.py index ce98034..82744e0 100644 --- a/src/engine/SCons/Environment.py +++ b/src/engine/SCons/Environment.py @@ -857,7 +857,7 @@ class Base(SubstitutionEnvironment): platform=None, tools=None, toolpath=None, - options=None, + variables=None, parse_flags = None, **kw): """ @@ -901,14 +901,19 @@ class Base(SubstitutionEnvironment): self._dict['PLATFORM'] = str(platform) platform(self) - # Apply the passed-in variables and customizable options to the + # Apply the passed-in and customizable variables to the # environment before calling the tools, because they may use # some of them during initialization. + if kw.has_key('options'): + # Backwards compatibility: they may stll be using the + # old "options" keyword. + variables = kw['options'] + del kw['options'] apply(self.Replace, (), kw) keys = kw.keys() - if options: - keys = keys + options.keys() - options.Update(self) + if variables: + keys = keys + variables.keys() + variables.Update(self) save = {} for k in keys: @@ -927,7 +932,7 @@ class Base(SubstitutionEnvironment): tools = ['default'] apply_tools(self, tools, toolpath) - # Now restore the passed-in variables and customized options + # Now restore the passed-in and customized variables # to the environment, since the values the user set explicitly # should override any values set by the tools. for key, val in save.items(): diff --git a/src/engine/SCons/EnvironmentTests.py b/src/engine/SCons/EnvironmentTests.py index 4f48da9..8ebbbbc 100644 --- a/src/engine/SCons/EnvironmentTests.py +++ b/src/engine/SCons/EnvironmentTests.py @@ -844,8 +844,8 @@ class BaseTestCase(unittest.TestCase,TestEnvironmentFixture): assert not env1.has_key('__env__') assert not env2.has_key('__env__') - def test_options(self): - """Test that options only get applied once.""" + def test_variables(self): + """Test that variables only get applied once.""" class FakeOptions: def __init__(self, key, val): self.calls = 0 @@ -858,7 +858,7 @@ class BaseTestCase(unittest.TestCase,TestEnvironmentFixture): self.calls = self.calls + 1 o = FakeOptions('AAA', 'fake_opt') - env = Environment(options=o, AAA='keyword_arg') + env = Environment(variables=o, AAA='keyword_arg') assert o.calls == 1, o.calls assert env['AAA'] == 'fake_opt', env['AAA'] diff --git a/src/engine/SCons/Executor.py b/src/engine/SCons/Executor.py index 7222042..a9417bc 100644 --- a/src/engine/SCons/Executor.py +++ b/src/engine/SCons/Executor.py @@ -160,6 +160,16 @@ class Executor: self.sources_need_sorting = False return self.sources + def prepare(self): + """ + Preparatory checks for whether this Executor can go ahead + and (try to) build its targets. + """ + for s in self.get_sources(): + if s.missing(): + msg = "Source `%s' not found, needed by target `%s'." + raise SCons.Errors.StopError, msg % (s, self.targets[0]) + def add_pre_action(self, action): self.pre_actions.append(action) @@ -221,40 +231,35 @@ class Executor: This essentially short-circuits an N*M scan of the sources for each individual target, which is a hell of a lot more efficient. """ - map(lambda N: N.disambiguate(), node_list) - env = self.get_build_env() - select_specific_scanner = lambda t: (t[0], t[1].select(t[0])) - remove_null_scanners = lambda t: not t[1] is None - add_scanner_path = lambda t, s=self: \ - (t[0], t[1], s.get_build_scanner_path(t[1])) + + deps = [] if scanner: - scanner_list = map(lambda n, s=scanner: (n, s), node_list) + for node in node_list: + node.disambiguate() + scanner = scanner.select(node) + if not scanner: + continue + path = self.get_build_scanner_path(scanner) + deps.extend(node.get_implicit_deps(env, scanner, path)) else: kw = self.get_kw() - get_initial_scanners = lambda n, e=env, kw=kw: \ - (n, n.get_env_scanner(e, kw)) - scanner_list = map(get_initial_scanners, node_list) - scanner_list = filter(remove_null_scanners, scanner_list) - - scanner_list = map(select_specific_scanner, scanner_list) - scanner_list = filter(remove_null_scanners, scanner_list) - scanner_path_list = map(add_scanner_path, scanner_list) - - deps = [] - for node, scanner, path in scanner_path_list: - deps.extend(node.get_implicit_deps(env, scanner, path)) + for node in node_list: + node.disambiguate() + scanner = node.get_env_scanner(env, kw) + if not scanner: + continue + scanner = scanner.select(node) + if not scanner: + continue + path = self.get_build_scanner_path(scanner) + deps.extend(node.get_implicit_deps(env, scanner, path)) deps.extend(self.get_implicit_deps()) for tgt in self.targets: tgt.add_to_implicit(deps) - def get_missing_sources(self): - """ - """ - return filter(lambda s: s.missing(), self.get_sources()) - def _get_unignored_sources_key(self, ignore=()): return tuple(ignore) @@ -345,3 +350,5 @@ class Null(_Executor): return None def cleanup(self): pass + def prepare(self): + pass diff --git a/src/engine/SCons/ExecutorTests.py b/src/engine/SCons/ExecutorTests.py index 368e034..12c80b4 100644 --- a/src/engine/SCons/ExecutorTests.py +++ b/src/engine/SCons/ExecutorTests.py @@ -285,6 +285,24 @@ class ExecutorTestCase(unittest.TestCase): x.get_sources() assert x.sources == ['s1', 's2', 's3', 's4'], x.sources + def test_prepare(self): + """Test the Executor's prepare() method""" + env = MyEnvironment() + t1 = MyNode('t1') + s1 = MyNode('s1') + s2 = MyNode('s2') + s3 = MyNode('s3') + x = SCons.Executor.Executor('b', env, [{}], [t1], [s1, s2, s3]) + + s2.missing_val = True + + try: + r = x.prepare() + except SCons.Errors.StopError, e: + assert str(e) == "Source `s2' not found, needed by target `t1'.", e + else: + raise AssertionError, "did not catch expected StopError: %s" % r + def test_add_pre_action(self): """Test adding pre-actions to an Executor""" x = SCons.Executor.Executor('b', 'e', 'o', 't', ['s1', 's2']) @@ -403,16 +421,6 @@ class ExecutorTestCase(unittest.TestCase): assert t1.implicit == ['scanner-s1', 'scanner-s2'], t1.implicit assert t2.implicit == ['scanner-s1', 'scanner-s2'], t2.implicit - def test_get_missing_sources(self): - """Test the ability to check if any sources are missing""" - env = MyEnvironment() - targets = [MyNode('t')] - sources = [MyNode('s1'), MyNode('s2')] - x = SCons.Executor.Executor('b', env, [{}], targets, sources) - sources[0].missing_val = 1 - missing = x.get_missing_sources() - assert missing == [sources[0]], missing - def test_get_unignored_sources(self): """Test fetching the unignored source list""" env = MyEnvironment() diff --git a/src/engine/SCons/Job.py b/src/engine/SCons/Job.py index 7b51409..d93952c 100644 --- a/src/engine/SCons/Job.py +++ b/src/engine/SCons/Job.py @@ -33,6 +33,9 @@ __revision__ = "__FILE__ __REVISION__ __DATE__ __DEVELOPER__" import SCons.compat +import os +import signal + # The default stack size (in kilobytes) of the threads used to execute # jobs in parallel. @@ -44,6 +47,7 @@ import SCons.compat default_stack_size = 256 +interrupt_msg = 'Build interrupted.' class Jobs: """An instance of this class initializes N jobs, and provides @@ -80,21 +84,73 @@ class Jobs: self.job = Serial(taskmaster) self.num_jobs = 1 - def run(self): - """run the job""" + self.job.interrupted = False + + def run(self, postfunc=lambda: None): + """Run the jobs. + + postfunc() will be invoked after the jobs has run. It will be + invoked even if the jobs are interrupted by a keyboard + interrupt (well, in fact by a signal such as either SIGINT, + SIGTERM or SIGHUP). The execution of postfunc() is protected + against keyboard interrupts and is guaranteed to run to + completion.""" + self._setup_sig_handler() try: self.job.start() - except KeyboardInterrupt: - # mask any further keyboard interrupts so that scons - # can shutdown cleanly: - # (this only masks the keyboard interrupt for Python, - # child processes can still get the keyboard interrupt) - import signal - signal.signal(signal.SIGINT, signal.SIG_IGN) - raise - - def cleanup(self): - self.job.cleanup() + finally: + postfunc() + self._reset_sig_handler() + + def were_interrupted(self): + """Returns whether the jobs were interrupted by a signal.""" + return self.job.interrupted + + def _setup_sig_handler(self): + """Setup an interrupt handler so that SCons can shutdown cleanly in + various conditions: + + a) SIGINT: Keyboard interrupt + b) SIGTERM: kill or system shutdown + c) SIGHUP: Controlling shell exiting + + We handle all of these cases by stopping the taskmaster. It + turns out that it very difficult to stop the build process + by throwing asynchronously an exception such as + KeyboardInterrupt. For example, the python Condition + variables (threading.Condition) and Queue's do not seem to + asynchronous-exception-safe. It would require adding a whole + bunch of try/finally block and except KeyboardInterrupt all + over the place. + + Note also that we have to be careful to handle the case when + SCons forks before executing another process. In that case, we + want the child to exit immediately. + """ + def handler(signum, stack, parentpid=os.getpid()): + if os.getpid() == parentpid: + self.job.taskmaster.stop() + self.job.interrupted = True + else: + os._exit(2) + + self.old_sigint = signal.signal(signal.SIGINT, handler) + self.old_sigterm = signal.signal(signal.SIGTERM, handler) + try: + self.old_sighup = signal.signal(signal.SIGHUP, handler) + except AttributeError: + pass + + def _reset_sig_handler(self): + """Restore the signal handlers to their previous state (before the + call to _setup_sig_handler().""" + + signal.signal(signal.SIGINT, self.old_sigint) + signal.signal(signal.SIGTERM, self.old_sigterm) + try: + signal.signal(signal.SIGHUP, self.old_sighup) + except AttributeError: + pass class Serial: """This class is used to execute tasks in series, and is more efficient @@ -129,11 +185,18 @@ class Serial: try: task.prepare() - task.execute() - except KeyboardInterrupt: - raise + if task.needs_execute(): + task.execute() except: - task.exception_set() + if self.interrupted: + try: + raise SCons.Errors.BuildError( + task.targets[0], errstr=interrupt_msg) + except: + task.exception_set() + else: + task.exception_set() + # Let the failed() callback function arrange for the # build to stop if that's appropriate. task.failed() @@ -141,9 +204,8 @@ class Serial: task.executed() task.postprocess() + self.taskmaster.cleanup() - def cleanup(self): - pass # Trap import failure so that everything in the Job module but the # Parallel class (and its dependent classes) will work if the interpreter @@ -178,9 +240,6 @@ else: try: task.execute() - except KeyboardInterrupt: - # be explicit here for test/interrupts.py - ok = False except: task.exception_set() ok = False @@ -226,16 +285,16 @@ else: if 'prev_size' in locals().keys(): threading.stack_size(prev_size) - def put(self, obj): + def put(self, task): """Put task into request queue.""" - self.requestQueue.put(obj) + self.requestQueue.put(task) - def get(self, block = True): + def get(self): """Remove and return a result tuple from the results queue.""" - return self.resultsQueue.get(block) + return self.resultsQueue.get() - def preparation_failed(self, obj): - self.resultsQueue.put((obj, False)) + def preparation_failed(self, task): + self.resultsQueue.put((task, False)) def cleanup(self): """ @@ -309,22 +368,21 @@ else: if task is None: break - # prepare task for execution try: + # prepare task for execution task.prepare() - except KeyboardInterrupt: - raise except: - # Let the failed() callback function arrange - # for the build to stop if that's appropriate. task.exception_set() - self.tp.preparation_failed(task) - jobs = jobs + 1 - continue - - # dispatch task - self.tp.put(task) - jobs = jobs + 1 + task.failed() + task.postprocess() + else: + if task.needs_execute(): + # dispatch task + self.tp.put(task) + jobs = jobs + 1 + else: + task.executed() + task.postprocess() if not task and not jobs: break @@ -332,11 +390,20 @@ else: # back and put the next batch of tasks on the queue. while 1: task, ok = self.tp.get() - jobs = jobs - 1 + if ok: task.executed() else: + if self.interrupted: + try: + raise SCons.Errors.BuildError( + task.targets[0], errstr=interrupt_msg) + except: + task.exception_set() + + # Let the failed() callback function arrange + # for the build to stop if that's appropriate. task.failed() task.postprocess() @@ -344,5 +411,5 @@ else: if self.tp.resultsQueue.empty(): break - def cleanup(self): self.tp.cleanup() + self.taskmaster.cleanup() diff --git a/src/engine/SCons/JobTests.py b/src/engine/SCons/JobTests.py index c432581..b2a195e 100644 --- a/src/engine/SCons/JobTests.py +++ b/src/engine/SCons/JobTests.py @@ -68,6 +68,9 @@ class Task: def _do_something(self): pass + def needs_execute(self): + return True + def execute(self): self.taskmaster.test_case.failUnless(self.was_prepared, "the task wasn't prepared") @@ -120,6 +123,9 @@ class ExceptionTask: def prepare(self): self.was_prepared = 1 + def needs_execute(self): + return True + def execute(self): raise Exception @@ -200,6 +206,9 @@ class Taskmaster: def exception_set(self): pass + def cleanup(self): + pass + SaveThreadPool = None ThreadPoolCallList = [] diff --git a/src/engine/SCons/Node/FS.py b/src/engine/SCons/Node/FS.py index f73161f..6ff3833 100644 --- a/src/engine/SCons/Node/FS.py +++ b/src/engine/SCons/Node/FS.py @@ -36,6 +36,7 @@ that can be used by scripts or modules looking for the canonical default. __revision__ = "__FILE__ __REVISION__ __DATE__ __DEVELOPER__" import fnmatch +from itertools import izip import os import os.path import re @@ -1895,7 +1896,7 @@ class Dir(Base): rep_nodes = map(dir.Entry, disk_names) #rep_nodes = [ n.disambiguate() for n in rep_nodes ] rep_nodes = map(lambda n: n.disambiguate(), rep_nodes) - for node, name in zip(rep_nodes, disk_names): + for node, name in izip(rep_nodes, disk_names): n = self.Entry(name) if n.__class__ != node.__class__: n.__class__ = node.__class__ @@ -2110,7 +2111,7 @@ class FileBuildInfo(SCons.Node.BuildInfoBase): pass else: nodes = [] - for s, ni in zip(strings, nodeinfos): + for s, ni in izip(strings, nodeinfos): if not isinstance(s, SCons.Node.Node): s = ni.str_to_node(s) nodes.append(s) @@ -2119,7 +2120,7 @@ class FileBuildInfo(SCons.Node.BuildInfoBase): result = [] bkids = self.bsources + self.bdepends + self.bimplicit bkidsigs = self.bsourcesigs + self.bdependsigs + self.bimplicitsigs - for bkid, bkidsig in zip(bkids, bkidsigs): + for bkid, bkidsig in izip(bkids, bkidsigs): result.append(str(bkid) + ': ' + string.join(bkidsig.format(names=names), ' ')) result.append('%s [%s]' % (self.bactsig, self.bact)) diff --git a/src/engine/SCons/Node/NodeTests.py b/src/engine/SCons/Node/NodeTests.py index 8e9a3f8..8bceaf6 100644 --- a/src/engine/SCons/Node/NodeTests.py +++ b/src/engine/SCons/Node/NodeTests.py @@ -714,23 +714,23 @@ class NodeTestCase(unittest.TestCase): n1 = SCons.Node.Node() n1.builder_set(Builder()) node.implicit = [] - node.implicit_dict = {} - node._add_child(node.implicit, node.implicit_dict, [n1]) + node.implicit_set = set() + node._add_child(node.implicit, node.implicit_set, [n1]) node.prepare() # should not throw an exception n2 = SCons.Node.Node() n2.linked = 1 node.implicit = [] - node.implicit_dict = {} - node._add_child(node.implicit, node.implicit_dict, [n2]) + node.implicit_set = set() + node._add_child(node.implicit, node.implicit_set, [n2]) node.prepare() # should not throw an exception n3 = SCons.Node.Node() node.implicit = [] - node.implicit_dict = {} - node._add_child(node.implicit, node.implicit_dict, [n3]) + node.implicit_set = set() + node._add_child(node.implicit, node.implicit_set, [n3]) node.prepare() # should not throw an exception @@ -739,8 +739,8 @@ class NodeTestCase(unittest.TestCase): return None n4 = MyNode() node.implicit = [] - node.implicit_dict = {} - node._add_child(node.implicit, node.implicit_dict, [n4]) + node.implicit_set = set() + node._add_child(node.implicit, node.implicit_set, [n4]) exc_caught = 0 try: node.prepare() @@ -810,7 +810,7 @@ class NodeTestCase(unittest.TestCase): pass else: raise "did not catch expected exception" - assert node.sources == [zero, one, two, three, four] + assert node.sources == [zero, one, two, three, four], node.sources def test_add_ignore(self): """Test adding files whose dependencies should be ignored. @@ -1033,9 +1033,9 @@ class NodeTestCase(unittest.TestCase): node.add_source([n1, n2, n3]) node.add_dependency([n4, n5, n6]) node.implicit = [] - node.implicit_dict = {} - node._add_child(node.implicit, node.implicit_dict, [n7, n8, n9]) - node._add_child(node.implicit, node.implicit_dict, [n10, n11, n12]) + node.implicit_set = set() + node._add_child(node.implicit, node.implicit_set, [n7, n8, n9]) + node._add_child(node.implicit, node.implicit_set, [n10, n11, n12]) node.add_ignore([n2, n5, n8, n11]) kids = node.children() @@ -1064,9 +1064,9 @@ class NodeTestCase(unittest.TestCase): node.add_source([n1, n2, n3]) node.add_dependency([n4, n5, n6]) node.implicit = [] - node.implicit_dict = {} - node._add_child(node.implicit, node.implicit_dict, [n7, n8, n9]) - node._add_child(node.implicit, node.implicit_dict, [n10, n11, n12]) + node.implicit_set = set() + node._add_child(node.implicit, node.implicit_set, [n7, n8, n9]) + node._add_child(node.implicit, node.implicit_set, [n10, n11, n12]) node.add_ignore([n2, n5, n8, n11]) kids = node.all_children() @@ -1217,7 +1217,6 @@ class NodeTestCase(unittest.TestCase): n.clear() assert n.includes is None, n.includes - assert n.found_includes == {}, n.found_includes assert x.cleaned_up def test_get_subst_proxy(self): @@ -1241,32 +1240,22 @@ class NodeTestCase(unittest.TestCase): def test_postprocess(self): """Test calling the base Node postprocess() method""" n = SCons.Node.Node() - n.waiting_parents = {'foo':1, 'bar':1} + n.waiting_parents = set( ['foo','bar'] ) n.postprocess() - assert n.waiting_parents == {}, n.waiting_parents + assert n.waiting_parents == set(), n.waiting_parents def test_add_to_waiting_parents(self): """Test the add_to_waiting_parents() method""" n1 = SCons.Node.Node() n2 = SCons.Node.Node() - assert n1.waiting_parents == {}, n1.waiting_parents + assert n1.waiting_parents == set(), n1.waiting_parents r = n1.add_to_waiting_parents(n2) assert r == 1, r - assert n1.waiting_parents == {n2:1}, n1.waiting_parents + assert n1.waiting_parents == set((n2,)), n1.waiting_parents r = n1.add_to_waiting_parents(n2) assert r == 0, r - def test_call_for_all_waiting_parents(self): - """Test the call_for_all_waiting_parents() method""" - n1 = SCons.Node.Node() - n2 = SCons.Node.Node() - n1.add_to_waiting_parents(n2) - result = [] - def func(node, result=result): - result.append(node) - n1.call_for_all_waiting_parents(func) - assert result == [n1, n2], result class NodeListTestCase(unittest.TestCase): def test___str__(self): diff --git a/src/engine/SCons/Node/__init__.py b/src/engine/SCons/Node/__init__.py index 2e136f0..c765ee9 100644 --- a/src/engine/SCons/Node/__init__.py +++ b/src/engine/SCons/Node/__init__.py @@ -47,6 +47,7 @@ __revision__ = "__FILE__ __REVISION__ __DATE__ __DEVELOPER__" import SCons.compat import copy +from itertools import chain, izip import string import UserList @@ -75,7 +76,7 @@ executed = 4 failed = 5 StateString = { - 0 : "0", + 0 : "no_state", 1 : "pending", 2 : "executing", 3 : "up_to_date", @@ -202,15 +203,16 @@ class Node: # a class. (Of course, we could always still do that in the # future if we had a good reason to...). self.sources = [] # source files used to build node - self.sources_dict = {} + self.sources_set = set() + self._specific_sources = False self.depends = [] # explicit dependencies (from Depends) - self.depends_dict = {} + self.depends_set = set() self.ignore = [] # dependencies to ignore - self.ignore_dict = {} + self.ignore_set = set() self.prerequisites = SCons.Util.UniqueList() self.implicit = None # implicit (scanned) dependencies (None means not scanned yet) - self.waiting_parents = {} - self.waiting_s_e = {} + self.waiting_parents = set() + self.waiting_s_e = set() self.ref_count = 0 self.wkids = None # Kids yet to walk, when it's an array @@ -220,7 +222,6 @@ class Node: self.noclean = 0 self.nocache = 0 self.always_build = None - self.found_includes = {} self.includes = None self.attributes = self.Attrs() # Generic place to stick information about the Node. self.side_effect = 0 # true iff this node is a side effect @@ -330,24 +331,29 @@ class Node: is out-of-date and must be rebuilt, but before actually calling the method to build the Node. - This default implemenation checks that all children either exist - or are derived, and initializes the BuildInfo structure that - will hold the information about how this node is, uh, built. + This default implementation checks that explicit or implicit + dependencies either exist or are derived, and initializes the + BuildInfo structure that will hold the information about how + this node is, uh, built. + + (The existence of source files is checked separately by the + Executor, which aggregates checks for all of the targets built + by a specific action.) Overriding this method allows for for a Node subclass to remove the underlying file from the file system. Note that subclass methods should call this base class method to get the child check and the BuildInfo structure. """ - l = self.depends + for d in self.depends: + if d.missing(): + msg = "Explicit dependency `%s' not found, needed by target `%s'." + raise SCons.Errors.StopError, msg % (d, self) if not self.implicit is None: - l = l + self.implicit - missing_sources = self.get_executor().get_missing_sources() \ - + filter(lambda c: c.missing(), l) - if missing_sources: - desc = "Source `%s' not found, needed by target `%s'." % (missing_sources[0], self) - raise SCons.Errors.StopError, desc - + for i in self.implicit: + if i.missing(): + msg = "Implicit dependency `%s' not found, needed by target `%s'." + raise SCons.Errors.StopError, msg % (i, self) self.binfo = self.get_binfo() def build(self, **kw): @@ -373,7 +379,7 @@ class Node: # Clear the implicit dependency caches of any Nodes # waiting for this Node to be built. - for parent in self.waiting_parents.keys(): + for parent in self.waiting_parents: parent.implicit = None self.clear() @@ -398,7 +404,7 @@ class Node: # def add_to_waiting_s_e(self, node): - self.waiting_s_e[node] = 1 + self.waiting_s_e.add(node) def add_to_waiting_parents(self, node): """ @@ -409,23 +415,16 @@ class Node: True and False instead...) """ wp = self.waiting_parents - if wp.has_key(node): - result = 0 - else: - result = 1 - wp[node] = 1 - return result - - def call_for_all_waiting_parents(self, func): - func(self) - for parent in self.waiting_parents.keys(): - parent.call_for_all_waiting_parents(func) + if node in wp: + return 0 + wp.add(node) + return 1 def postprocess(self): """Clean up anything we don't need to hang onto after we've been built.""" self.executor_cleanup() - self.waiting_parents = {} + self.waiting_parents = set() def clear(self): """Completely clear a Node of all its cached state (so that it @@ -444,7 +443,6 @@ class Node: except AttributeError: pass self.includes = None - self.found_includes = {} def clear_memoized_values(self): self._memo = {} @@ -592,9 +590,9 @@ class Node: def add_to_implicit(self, deps): if not hasattr(self, 'implicit') or self.implicit is None: self.implicit = [] - self.implicit_dict = {} + self.implicit_set = set() self._children_reset() - self._add_child(self.implicit, self.implicit_dict, deps) + self._add_child(self.implicit, self.implicit_set, deps) def scan(self): """Scan this node's dependents for implicit dependencies.""" @@ -604,7 +602,7 @@ class Node: if not self.implicit is None: return self.implicit = [] - self.implicit_dict = {} + self.implicit_set = set() self._children_reset() if not self.has_builder(): return @@ -633,7 +631,7 @@ class Node: # one of this node's sources has changed, # so we must recalculate the implicit deps: self.implicit = [] - self.implicit_dict = {} + self.implicit_set = set() # Have the executor scan the sources. executor.scan_sources(self.builder.source_scanner) @@ -706,33 +704,44 @@ class Node: self.binfo = binfo executor = self.get_executor() - - sources = executor.get_unignored_sources(self.ignore) - - depends = self.depends - implicit = self.implicit or [] - - if self.ignore: - depends = filter(self.do_not_ignore, depends) - implicit = filter(self.do_not_ignore, implicit) - - def get_ninfo(node): - return node.get_ninfo() - - sourcesigs = map(get_ninfo, sources) - dependsigs = map(get_ninfo, depends) - implicitsigs = map(get_ninfo, implicit) + ignore_set = self.ignore_set if self.has_builder(): binfo.bact = str(executor) binfo.bactsig = SCons.Util.MD5signature(executor.get_contents()) - binfo.bsources = sources - binfo.bdepends = depends - binfo.bimplicit = implicit + if self._specific_sources: + sources = [] + for s in self.sources: + if s not in ignore_set: + sources.append(s) + else: + sources = executor.get_unignored_sources(self.ignore) + seen = set() + bsources = [] + bsourcesigs = [] + for s in sources: + if not s in seen: + seen.add(s) + bsources.append(s) + bsourcesigs.append(s.get_ninfo()) + binfo.bsources = bsources + binfo.bsourcesigs = bsourcesigs - binfo.bsourcesigs = sourcesigs + depends = self.depends + dependsigs = [] + for d in depends: + if d not in ignore_set: + dependsigs.append(d.get_ninfo()) + binfo.bdepends = depends binfo.bdependsigs = dependsigs + + implicit = self.implicit or [] + implicitsigs = [] + for i in implicit: + if i not in ignore_set: + implicitsigs.append(i.get_ninfo()) + binfo.bimplicit = implicit binfo.bimplicitsigs = implicitsigs return binfo @@ -816,7 +825,7 @@ class Node: def add_dependency(self, depend): """Adds dependencies.""" try: - self._add_child(self.depends, self.depends_dict, depend) + self._add_child(self.depends, self.depends_set, depend) except TypeError, e: e = e.args[0] if SCons.Util.is_List(e): @@ -833,7 +842,7 @@ class Node: def add_ignore(self, depend): """Adds dependencies to ignore.""" try: - self._add_child(self.ignore, self.ignore_dict, depend) + self._add_child(self.ignore, self.ignore_set, depend) except TypeError, e: e = e.args[0] if SCons.Util.is_List(e): @@ -844,8 +853,10 @@ class Node: def add_source(self, source): """Adds sources.""" + if self._specific_sources: + return try: - self._add_child(self.sources, self.sources_dict, source) + self._add_child(self.sources, self.sources_set, source) except TypeError, e: e = e.args[0] if SCons.Util.is_List(e): @@ -854,9 +865,9 @@ class Node: s = str(e) raise SCons.Errors.UserError("attempted to add a non-Node as source of %s:\n\t%s is a %s, not a Node" % (str(self), s, type(e))) - def _add_child(self, collection, dict, child): - """Adds 'child' to 'collection', first checking 'dict' to see - if it's already present.""" + def _add_child(self, collection, set, child): + """Adds 'child' to 'collection', first checking 'set' to see if it's + already present.""" #if type(child) is not type([]): # child = [child] #for c in child: @@ -864,13 +875,17 @@ class Node: # raise TypeError, c added = None for c in child: - if not dict.has_key(c): + if c not in set: + set.add(c) collection.append(c) - dict[c] = 1 added = 1 if added: self._children_reset() + def set_specific_source(self, source): + self.add_source(source) + self._specific_sources = True + def add_wkid(self, wkid): """Add a node to the list of kids waiting to be evaluated""" if self.wkids != None: @@ -882,10 +897,14 @@ class Node: # build info that it's cached so we can re-calculate it. self.executor_cleanup() - def do_not_ignore(self, node): - return node not in self.ignore + memoizer_counters.append(SCons.Memoize.CountValue('_children_get')) + + def _children_get(self): + try: + return self._memo['children_get'] + except KeyError: + pass - def _all_children_get(self): # The return list may contain duplicate Nodes, especially in # source trees where there are a lot of repeated #includes # of a tangle of .h files. Profiling shows, however, that @@ -903,21 +922,22 @@ class Node: # using dictionary keys, lose the order, and the only ordered # dictionary patterns I found all ended up using "not in" # internally anyway...) - if self.implicit is None: - return self.sources + self.depends - else: - return self.sources + self.depends + self.implicit + if self.ignore_set: + if self.implicit is None: + iter = chain(self.sources,self.depends) + else: + iter = chain(self.sources, self.depends, self.implicit) - memoizer_counters.append(SCons.Memoize.CountValue('_children_get')) + children = [] + for i in iter: + if i not in self.ignore_set: + children.append(i) + else: + if self.implicit is None: + children = self.sources + self.depends + else: + children = self.sources + self.depends + self.implicit - def _children_get(self): - try: - return self._memo['children_get'] - except KeyError: - pass - children = self._all_children_get() - if self.ignore: - children = filter(self.do_not_ignore, children) self._memo['children_get'] = children return children @@ -925,7 +945,28 @@ class Node: """Return a list of all the node's direct children.""" if scan: self.scan() - return self._all_children_get() + + # The return list may contain duplicate Nodes, especially in + # source trees where there are a lot of repeated #includes + # of a tangle of .h files. Profiling shows, however, that + # eliminating the duplicates with a brute-force approach that + # preserves the order (that is, something like: + # + # u = [] + # for n in list: + # if n not in u: + # u.append(n)" + # + # takes more cycles than just letting the underlying methods + # hand back cached values if a Node's information is requested + # multiple times. (Other methods of removing duplicates, like + # using dictionary keys, lose the order, and the only ordered + # dictionary patterns I found all ended up using "not in" + # internally anyway...) + if self.implicit is None: + return self.sources + self.depends + else: + return self.sources + self.depends + self.implicit def children(self, scan=1): """Return a list of the node's direct children, minus those @@ -1009,7 +1050,7 @@ class Node: if t: Trace(': old %s new %s' % (len(then), len(children))) result = True - for child, prev_ni in zip(children, then): + for child, prev_ni in izip(children, then): if child.changed_since_last_build(self, prev_ni): if t: Trace(': %s changed' % child) result = True @@ -1152,8 +1193,8 @@ class Node: new_bkids = new.bsources + new.bdepends + new.bimplicit new_bkidsigs = new.bsourcesigs + new.bdependsigs + new.bimplicitsigs - osig = dict(zip(old_bkids, old_bkidsigs)) - nsig = dict(zip(new_bkids, new_bkidsigs)) + osig = dict(izip(old_bkids, old_bkidsigs)) + nsig = dict(izip(new_bkids, new_bkidsigs)) # The sources and dependencies we'll want to report are all stored # as relative paths to this target's directory, but we want to diff --git a/src/engine/SCons/Options.py b/src/engine/SCons/Options.py new file mode 100644 index 0000000..0f2fde1 --- /dev/null +++ b/src/engine/SCons/Options.py @@ -0,0 +1,56 @@ +# +# __COPYRIGHT__ +# +# Permission is hereby granted, free of charge, to any person obtaining +# a copy of this software and associated documentation files (the +# "Software"), to deal in the Software without restriction, including +# without limitation the rights to use, copy, modify, merge, publish, +# distribute, sublicense, and/or sell copies of the Software, and to +# permit persons to whom the Software is furnished to do so, subject to +# the following conditions: +# +# The above copyright notice and this permission notice shall be included +# in all copies or substantial portions of the Software. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY +# KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE +# WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND +# NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE +# LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION +# OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION +# WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. +# + +__revision__ = "__FILE__ __REVISION__ __DATE__ __DEVELOPER__" + +__doc__ = """Place-holder for the old SCons.Options module hierarchy + +This is for backwards compatibility. The new equivalent is the Variables/ +class hierarchy. These will have deprecation warnings added (some day), +and will then be removed entirely (some day). +""" + +import SCons.Variables + +class Options(SCons.Variables.Variables): + + def AddOptions(self, *args, **kw): + return apply(SCons.Variables.Variables.AddVariables, + (self,) + args, + kw) + + def UnknownOptions(self, *args, **kw): + return apply(SCons.Variables.Variables.UnknownVariables, + (self,) + args, + kw) + + def FormatOptionHelpText(self, *args, **kw): + return apply(SCons.Variables.Variables.FormatVariableHelpText, + (self,) + args, + kw) + +BoolOption = SCons.Variables.BoolVariable +EnumOption = SCons.Variables.EnumVariable +ListOption = SCons.Variables.ListVariable +PackageOption = SCons.Variables.PackageVariable +PathOption = SCons.Variables.PathVariable diff --git a/src/engine/SCons/Options/.cvsignore b/src/engine/SCons/Options/.cvsignore deleted file mode 100644 index 0d20b64..0000000 --- a/src/engine/SCons/Options/.cvsignore +++ /dev/null @@ -1 +0,0 @@ -*.pyc diff --git a/src/engine/SCons/Platform/posix.py b/src/engine/SCons/Platform/posix.py index afdabe1..2a0e478 100644 --- a/src/engine/SCons/Platform/posix.py +++ b/src/engine/SCons/Platform/posix.py @@ -32,6 +32,7 @@ selection method. __revision__ = "__FILE__ __REVISION__ __DATE__ __DEVELOPER__" +import errno import os import os.path import popen2 @@ -109,21 +110,25 @@ def fork_spawn(sh, escape, cmd, args, env): def process_cmd_output(cmd_stdout, cmd_stderr, stdout, stderr): stdout_eof = stderr_eof = 0 while not (stdout_eof and stderr_eof): - (i,o,e) = select.select([cmd_stdout, cmd_stderr], [], []) - if cmd_stdout in i: - str = cmd_stdout.read() - if len(str) == 0: - stdout_eof = 1 - elif stdout != None: - stdout.write(str) - if cmd_stderr in i: - str = cmd_stderr.read() - if len(str) == 0: - #sys.__stderr__.write( "stderr_eof=1\n" ) - stderr_eof = 1 - else: - #sys.__stderr__.write( "str(stderr) = %s\n" % str ) - stderr.write(str) + try: + (i,o,e) = select.select([cmd_stdout, cmd_stderr], [], []) + if cmd_stdout in i: + str = cmd_stdout.read() + if len(str) == 0: + stdout_eof = 1 + elif stdout != None: + stdout.write(str) + if cmd_stderr in i: + str = cmd_stderr.read() + if len(str) == 0: + #sys.__stderr__.write( "stderr_eof=1\n" ) + stderr_eof = 1 + else: + #sys.__stderr__.write( "str(stderr) = %s\n" % str ) + stderr.write(str) + except select.error, (_errno, _strerror): + if _errno != errno.EINTR: + raise def exec_popen3(l, env, stdout, stderr): proc = popen2.Popen3(string.join(l), 1) diff --git a/src/engine/SCons/SConfTests.py b/src/engine/SCons/SConfTests.py index f7d33f8..25c89d2 100644 --- a/src/engine/SCons/SConfTests.py +++ b/src/engine/SCons/SConfTests.py @@ -163,9 +163,10 @@ class SConfTestCase(unittest.TestCase): def __init__(self, name): self.name = name self.state = None - self.waiting_parents = {} + self.waiting_parents = set() self.side_effects = [] self.builder = None + self.prerequisites = [] def disambiguate(self): return self def has_builder(self): diff --git a/src/engine/SCons/Script/Interactive.py b/src/engine/SCons/Script/Interactive.py index d18eec1..b1774eb 100644 --- a/src/engine/SCons/Script/Interactive.py +++ b/src/engine/SCons/Script/Interactive.py @@ -185,6 +185,16 @@ class SConsInteractiveCmd(cmd.Cmd): if not nodes: return + # Call each of the Node's alter_targets() methods, which may + # provide additional targets that ended up as part of the build + # (the canonical example being a VariantDir() when we're building + # from a source directory) and which we therefore need their + # state cleared, too. + x = [] + for n in nodes: + x.extend(n.alter_targets()[0]) + nodes.extend(x) + # Clean up so that we can perform the next build correctly. # # We do this by walking over all the children of the targets, diff --git a/src/engine/SCons/Script/Main.py b/src/engine/SCons/Script/Main.py index 80b9032..75d0117 100644 --- a/src/engine/SCons/Script/Main.py +++ b/src/engine/SCons/Script/Main.py @@ -168,28 +168,29 @@ class BuildTask(SCons.Taskmaster.Task): self.progress(self.targets[0]) return SCons.Taskmaster.Task.prepare(self) - def execute(self): - for target in self.targets: - if target.get_state() == SCons.Node.up_to_date: - continue - if target.has_builder() and not hasattr(target.builder, 'status'): - if print_time: - start_time = time.time() - global first_command_start - if first_command_start is None: - first_command_start = start_time - SCons.Taskmaster.Task.execute(self) - if print_time: - global cumulative_command_time - global last_command_end - finish_time = time.time() - last_command_end = finish_time - cumulative_command_time = cumulative_command_time+finish_time-start_time - sys.stdout.write("Command execution time: %f seconds\n"%(finish_time-start_time)) - break + def needs_execute(self): + target = self.targets[0] + if target.get_state() == SCons.Node.executing: + return True else: if self.top and target.has_builder(): display("scons: `%s' is up to date." % str(self.node)) + return False + + def execute(self): + if print_time: + start_time = time.time() + global first_command_start + if first_command_start is None: + first_command_start = start_time + SCons.Taskmaster.Task.execute(self) + if print_time: + global cumulative_command_time + global last_command_end + finish_time = time.time() + last_command_end = finish_time + cumulative_command_time = cumulative_command_time+finish_time-start_time + sys.stdout.write("Command execution time: %f seconds\n"%(finish_time-start_time)) def do_failed(self, status=2): _BuildFailures.append(self.exception[1]) @@ -207,10 +208,15 @@ class BuildTask(SCons.Taskmaster.Task): t = self.targets[0] if self.top and not t.has_builder() and not t.side_effect: if not t.exists(): - sys.stderr.write("scons: *** Do not know how to make target `%s'." % t) + errstr="Do not know how to make target `%s'." % t + sys.stderr.write("scons: *** " + errstr) if not self.options.keep_going: sys.stderr.write(" Stop.") sys.stderr.write("\n") + try: + raise SCons.Errors.BuildError(t, errstr) + except: + self.exception_set() self.do_failed() else: print "scons: Nothing to be done for `%s'." % t @@ -977,11 +983,6 @@ def _build_targets(fs, options, targets, target_top): if options.diskcheck: SCons.Node.FS.set_diskcheck(options.diskcheck) - _set_debug_values(options) - SCons.Node.implicit_cache = options.implicit_cache - SCons.Node.implicit_deps_changed = options.implicit_deps_changed - SCons.Node.implicit_deps_unchanged = options.implicit_deps_unchanged - SCons.CacheDir.cache_enabled = not options.cache_disable SCons.CacheDir.cache_debug = options.cache_debug SCons.CacheDir.cache_force = options.cache_force @@ -1133,26 +1134,29 @@ def _build_targets(fs, options, targets, target_top): memory_stats.append('before building targets:') count_stats.append(('pre-', 'build')) - try: - progress_display("scons: " + opening_message) - try: - jobs.run() - except KeyboardInterrupt: - # If we are in interactive mode, a KeyboardInterrupt - # interrupts only this current run. Return 'nodes' normally - # so that the outer loop can clean up the nodes and continue. - if options.interactive: - print "Build interrupted." - # Continue and return normally - finally: - jobs.cleanup() + def jobs_postfunc( + jobs=jobs, + options=options, + closing_message=closing_message, + failure_message=failure_message + ): + if jobs.were_interrupted(): + progress_display("scons: Build interrupted.") + global exit_status + exit_status = 2 + if exit_status: progress_display("scons: " + failure_message) else: progress_display("scons: " + closing_message) if not options.no_exec: + if jobs.were_interrupted(): + progress_display("scons: writing .sconsign file.") SCons.SConsign.write() + progress_display("scons: " + opening_message) + jobs.run(postfunc = jobs_postfunc) + memory_stats.append('after building targets:') count_stats.append(('post-', 'build')) @@ -1210,10 +1214,9 @@ def main(): parts = ["SCons by Steven Knight et al.:\n"] try: + import __main__ parts.append(version_string("script", __main__)) - except KeyboardInterrupt: - raise - except: + except (ImportError, AttributeError): # On Windows there is no scons.py, so there is no # __main__.__version__, hence there is no script version. pass @@ -1233,7 +1236,7 @@ def main(): if s: exit_status = s except KeyboardInterrupt: - print "Build interrupted." + print("scons: Build interrupted.") sys.exit(2) except SyntaxError, e: _scons_syntax_error(e) diff --git a/src/engine/SCons/Script/SConsOptions.py b/src/engine/SCons/Script/SConsOptions.py index ee34a85..2148068 100644 --- a/src/engine/SCons/Script/SConsOptions.py +++ b/src/engine/SCons/Script/SConsOptions.py @@ -26,11 +26,19 @@ __revision__ = "__FILE__ __REVISION__ __DATE__ __DEVELOPER__" import SCons.compat import optparse +import re import string import sys import textwrap try: + no_hyphen_re = re.compile(r'(\s+|(?<=[\w\!\"\'\&\.\,\?])-{2,}(?=\w))') +except re.error: + # Pre-2.0 Python versions don't have the (?<= negative + # look-behind assertion. + no_hyphen_re = re.compile(r'(\s+|-*\w{2,}-(?=\w{2,}))') + +try: from gettext import gettext except ImportError: def gettext(message): @@ -382,8 +390,16 @@ class SConsIndentedHelpFormatter(optparse.IndentedHelpFormatter): def format_option(self, option): """ A copy of the normal optparse.IndentedHelpFormatter.format_option() - method, snarfed so we can set the subsequent_indent on the - textwrap.wrap() call below... + method. This has been snarfed so we can modify text wrapping to + out liking: + + -- add our own regular expression that doesn't break on hyphens + (so things like --no-print-directory don't get broken); + + -- wrap the list of options themselves when it's too long + (the wrapper.fill(opts) call below); + + -- set the subsequent_indent when wrapping the help_text. """ # The help for each option consists of two parts: # * the opt strings and metavars @@ -410,7 +426,11 @@ class SConsIndentedHelpFormatter(optparse.IndentedHelpFormatter): opt_width = self.help_position - self.current_indent - 2 if len(opts) > opt_width: - opts = "%*s%s\n" % (self.current_indent, "", opts) + wrapper = textwrap.TextWrapper(width=self.width, + initial_indent = ' ', + subsequent_indent = ' ') + wrapper.wordsep_re = no_hyphen_re + opts = wrapper.fill(opts) + '\n' indent_first = self.help_position else: # start help on same line as opts opts = "%*s%-*s " % (self.current_indent, "", opt_width, opts) @@ -428,8 +448,10 @@ class SConsIndentedHelpFormatter(optparse.IndentedHelpFormatter): help_text = expand_default(option) # SCons: indent every line of the help text but the first. - help_lines = textwrap.wrap(help_text, self.help_width, - subsequent_indent = ' ') + wrapper = textwrap.TextWrapper(width=self.help_width, + subsequent_indent = ' ') + wrapper.wordsep_re = no_hyphen_re + help_lines = wrapper.wrap(help_text) result.append("%*s%s\n" % (indent_first, "", help_lines[0])) for line in help_lines[1:]: result.append("%*s%s\n" % (self.help_position, "", line)) @@ -503,8 +525,13 @@ def Parser(version): # options ignored for compatibility def opt_ignore(option, opt, value, parser): sys.stderr.write("Warning: ignoring %s option\n" % opt) - op.add_option("-b", "-m", "-S", "-t", - "--no-keep-going", "--stop", "--touch", + op.add_option("-b", "-d", "-e", "-m", "-S", "-t", "-w", + "--environment-overrides", + "--no-keep-going", + "--no-print-directory", + "--print-directory", + "--stop", + "--touch", action="callback", callback=opt_ignore, help="Ignored for compatibility.") @@ -557,13 +584,6 @@ def Parser(version): help = opt_config_help, metavar="MODE") - def opt_not_yet(option, opt, value, parser): - sys.stderr.write("Warning: the %s option is not yet implemented\n" % opt) - sys.exit(0) - op.add_option('-d', - action="callback", callback=opt_not_yet, - help = "Print file dependency information.") - op.add_option('-D', dest="climb_up", default=None, action="store_const", const=2, @@ -836,11 +856,12 @@ def Parser(version): # we don't want to change. These all get a "the -X option is not # yet implemented" message and don't show up in the help output. - op.add_option('-e', '--environment-overrides', - dest="environment_overrides", - action="callback", callback=opt_not_yet, - # help="Environment variables override makefiles." - help=SUPPRESS_HELP) + def opt_not_yet(option, opt, value, parser): + msg = "Warning: the %s option is not yet implemented\n" % opt + sys.stderr.write(msg) + sys.exit(0) + + op.add_option('-l', '--load-average', '--max-load', nargs=1, type="int", dest="load_average", default=0, @@ -887,16 +908,6 @@ def Parser(version): dest="no_builtin_rules", # help="Clear default environments and variables." help=SUPPRESS_HELP) - op.add_option('-w', '--print-directory', - action="callback", callback=opt_not_yet, - dest="print_directory", - # help="Print the current directory." - help=SUPPRESS_HELP) - op.add_option('--no-print-directory', - action="callback", callback=opt_not_yet, - dest="no_print_directory", - # help="Turn off -w, even if it was turned on implicitly." - help=SUPPRESS_HELP) op.add_option('--write-filenames', nargs=1, type="string", dest="write_filenames", diff --git a/src/engine/SCons/Script/SConscript.py b/src/engine/SCons/Script/SConscript.py index 36a147d..7e38ffd 100644 --- a/src/engine/SCons/Script/SConscript.py +++ b/src/engine/SCons/Script/SConscript.py @@ -39,7 +39,6 @@ import SCons.Errors import SCons.Node import SCons.Node.Alias import SCons.Node.FS -import SCons.Options import SCons.Platform import SCons.SConf import SCons.Script.Main @@ -141,7 +140,8 @@ call_stack = [] def Return(*vars, **kw): retval = [] try: - for var in vars: + fvars = SCons.Util.flatten(vars) + for var in fvars: for v in string.split(var): retval.append(call_stack[-1].globals[v]) except KeyError, x: diff --git a/src/engine/SCons/Script/__init__.py b/src/engine/SCons/Script/__init__.py index ddeaf9e..72e31d1 100644 --- a/src/engine/SCons/Script/__init__.py +++ b/src/engine/SCons/Script/__init__.py @@ -84,6 +84,7 @@ import SCons.SConf import SCons.Subst import SCons.Tool import SCons.Util +import SCons.Variables import SCons.Defaults import Main @@ -138,22 +139,31 @@ call_stack = _SConscript.call_stack Action = SCons.Action.Action AddMethod = SCons.Util.AddMethod AllowSubstExceptions = SCons.Subst.SetAllowableExceptions -BoolOption = SCons.Options.BoolOption Builder = SCons.Builder.Builder Configure = _SConscript.Configure -EnumOption = SCons.Options.EnumOption Environment = SCons.Environment.Environment #OptParser = SCons.SConsOptions.OptParser FindPathDirs = SCons.Scanner.FindPathDirs -ListOption = SCons.Options.ListOption -PackageOption = SCons.Options.PackageOption -PathOption = SCons.Options.PathOption Platform = SCons.Platform.Platform Return = _SConscript.Return Scanner = SCons.Scanner.Base Tool = SCons.Tool.Tool WhereIs = SCons.Util.WhereIs +# +BoolVariable = SCons.Variables.BoolVariable +EnumVariable = SCons.Variables.EnumVariable +ListVariable = SCons.Variables.ListVariable +PackageVariable = SCons.Variables.PackageVariable +PathVariable = SCons.Variables.PathVariable + +# Deprecated names that will go away some day. +BoolOption = SCons.Options.BoolOption +EnumOption = SCons.Options.EnumOption +ListOption = SCons.Options.ListOption +PackageOption = SCons.Options.PackageOption +PathOption = SCons.Options.PathOption + # Action factories. Chmod = SCons.Defaults.Chmod Copy = SCons.Defaults.Copy @@ -262,6 +272,9 @@ def HelpFunction(text): sconscript_reading = 0 # +def Variables(files=[], args=ARGUMENTS): + return SCons.Variables.Variables(files, args) + def Options(files=[], args=ARGUMENTS): return SCons.Options.Options(files, args) diff --git a/src/engine/SCons/Subst.py b/src/engine/SCons/Subst.py index 7a565ba..52aaae5 100644 --- a/src/engine/SCons/Subst.py +++ b/src/engine/SCons/Subst.py @@ -565,7 +565,7 @@ def scons_subst_list(strSubst, env, mode=SUBST_RAW, target=None, source=None, gv # except KeyError: # Subst_List_Strings[strSubst] = 1 # import SCons.Debug -# SCons.Debug.caller(1) +# SCons.Debug.caller_trace(1) class ListSubber(UserList.UserList): """A class to construct the results of a scons_subst_list() call. diff --git a/src/engine/SCons/Taskmaster.py b/src/engine/SCons/Taskmaster.py index 66202dc..8a0fcf7 100644 --- a/src/engine/SCons/Taskmaster.py +++ b/src/engine/SCons/Taskmaster.py @@ -52,16 +52,22 @@ __revision__ = "__FILE__ __REVISION__ __DATE__ __DEVELOPER__" import SCons.compat +from itertools import chain import operator import string import sys import traceback -import SCons.Node import SCons.Errors +import SCons.Node StateString = SCons.Node.StateString - +NODE_NO_STATE = SCons.Node.no_state +NODE_PENDING = SCons.Node.pending +NODE_EXECUTING = SCons.Node.executing +NODE_UP_TO_DATE = SCons.Node.up_to_date +NODE_EXECUTED = SCons.Node.executed +NODE_FAILED = SCons.Node.failed # A subsystem for recording stats about how different Nodes are handled by @@ -165,6 +171,17 @@ class Task: self.display(self.tm.message) self.tm.message = None + # Let the targets take care of any necessary preparations. + # This includes verifying that all of the necessary sources + # and dependencies exist, removing the target file(s), etc. + # + # As of April 2008, the get_executor().prepare() method makes + # sure that all of the aggregate sources necessary to build this + # Task's target(s) exist in one up-front check. The individual + # target t.prepare() methods check that each target's explicit + # or implicit dependencies exists, and also initialize the + # .sconsign info. + self.targets[0].get_executor().prepare() for t in self.targets: t.prepare() for s in t.side_effects: @@ -175,6 +192,17 @@ class Task: """ return self.node + def needs_execute(self): + """ + Called to determine whether the task's execute() method should + be run. + + This method allows one to skip the somethat costly execution + of the execute() method in a seperate thread. For example, + that would be unnecessary for up-to-date targets. + """ + return True + def execute(self): """ Called to execute the task. @@ -192,8 +220,6 @@ class Task: break if not everything_was_cached: self.targets[0].build() - except KeyboardInterrupt: - raise except SystemExit: exc_value = sys.exc_info()[1] raise SCons.Errors.ExplicitExit(self.targets[0], exc_value.code) @@ -212,10 +238,10 @@ class Task: the Node's callback methods. """ for t in self.targets: - if t.get_state() == SCons.Node.executing: + if t.get_state() == NODE_EXECUTING: for side_effect in t.side_effects: - side_effect.set_state(SCons.Node.no_state) - t.set_state(SCons.Node.executed) + side_effect.set_state(NODE_NO_STATE) + t.set_state(NODE_EXECUTED) def executed_with_callbacks(self): """ @@ -231,10 +257,10 @@ class Task: or not the target was an actual built target or a source Node. """ for t in self.targets: - if t.get_state() == SCons.Node.executing: + if t.get_state() == NODE_EXECUTING: for side_effect in t.side_effects: - side_effect.set_state(SCons.Node.no_state) - t.set_state(SCons.Node.executed) + side_effect.set_state(NODE_NO_STATE) + t.set_state(NODE_EXECUTED) t.built() t.visited() @@ -250,8 +276,12 @@ class Task: """ Explicit stop-the-build failure. """ - for t in self.targets: - t.set_state(SCons.Node.failed) + + # Invoke fail_continue() to clean-up the pending children + # list. + self.fail_continue() + + # Tell the taskmaster to not start any new tasks self.tm.stop() # We're stopping because of a build failure, but give the @@ -267,11 +297,43 @@ class Task: This sets failure status on the target nodes and all of their dependent parent nodes. """ + + pending_children = self.tm.pending_children + + to_visit = set() for t in self.targets: # Set failure state on all of the parents that were dependent # on this failed build. - def set_state(node): node.set_state(SCons.Node.failed) - t.call_for_all_waiting_parents(set_state) + if t.state != NODE_FAILED: + t.state = NODE_FAILED + parents = t.waiting_parents + to_visit = to_visit | parents + pending_children = pending_children - parents + + try: + while 1: + try: + node = to_visit.pop() + except AttributeError: + # Python 1.5.2 + if len(to_visit): + node = to_visit[0] + to_visit.remove(node) + else: + break + if node.state != NODE_FAILED: + node.state = NODE_FAILED + parents = node.waiting_parents + to_visit = to_visit | parents + pending_children = pending_children - parents + except KeyError: + # The container to_visit has been emptied. + pass + + # We have the stick back the pending_children list into the + # task master because the python 1.5.2 compatibility does not + # allow us to use in-place updates + self.tm.pending_children = pending_children def make_ready_all(self): """ @@ -282,9 +344,9 @@ class Task: """ self.out_of_date = self.targets[:] for t in self.targets: - t.disambiguate().set_state(SCons.Node.executing) + t.disambiguate().set_state(NODE_EXECUTING) for s in t.side_effects: - s.set_state(SCons.Node.executing) + s.set_state(NODE_EXECUTING) def make_ready_current(self): """ @@ -294,6 +356,7 @@ class Task: This is the default behavior for building only what's necessary. """ self.out_of_date = [] + needs_executing = False for t in self.targets: try: t.disambiguate().make_ready() @@ -301,13 +364,24 @@ class Task: (not t.always_build and t.is_up_to_date()) except EnvironmentError, e: raise SCons.Errors.BuildError(node=t, errstr=e.strerror, filename=e.filename) - if is_up_to_date: - t.set_state(SCons.Node.up_to_date) - else: + + if not is_up_to_date: self.out_of_date.append(t) - t.set_state(SCons.Node.executing) + needs_executing = True + + if needs_executing: + for t in self.targets: + t.set_state(NODE_EXECUTING) for s in t.side_effects: - s.set_state(SCons.Node.executing) + s.set_state(NODE_EXECUTING) + else: + for t in self.targets: + # We must invoke visited() to ensure that the node + # information has been computed before allowing the + # parent nodes to execute. (That could occur in a + # parallel build...) + t.visited() + t.set_state(NODE_UP_TO_DATE) make_ready = make_ready_current @@ -333,24 +407,25 @@ class Task: parents = {} for t in targets: - for p in t.waiting_parents.keys(): + for p in t.waiting_parents: parents[p] = parents.get(p, 0) + 1 for t in targets: for s in t.side_effects: - if s.get_state() == SCons.Node.executing: - s.set_state(SCons.Node.no_state) - for p in s.waiting_parents.keys(): - if not parents.has_key(p): - parents[p] = 1 - for p in s.waiting_s_e.keys(): + if s.get_state() == NODE_EXECUTING: + s.set_state(NODE_NO_STATE) + for p in s.waiting_parents: + parents[p] = parents.get(p, 0) + 1 + for p in s.waiting_s_e: if p.ref_count == 0: self.tm.candidates.append(p) + self.tm.pending_children.discard(p) for p, subtract in parents.items(): p.ref_count = p.ref_count - subtract if p.ref_count == 0: self.tm.candidates.append(p) + self.tm.pending_children.discard(p) for t in targets: t.postprocess() @@ -409,12 +484,15 @@ class Task: raise exc_type, exc_value, exc_traceback -def find_cycle(stack): - if stack[0] == stack[-1]: - return stack - for n in stack[-1].waiting_parents.keys(): +def find_cycle(stack, visited): + if stack[-1] in visited: + return None + visited.add(stack[-1]) + for n in stack[-1].waiting_parents: stack.append(n) - if find_cycle(stack): + if stack[0] == stack[-1]: + return stack + if find_cycle(stack, visited): return stack stack.pop() return None @@ -437,6 +515,8 @@ class Taskmaster: self.message = None self.trace = trace self.next_candidate = self.find_next_candidate + self.pending_children = set() + def find_next_candidate(self): """ @@ -505,10 +585,12 @@ class Taskmaster: self.ready_exc = None T = self.trace + if T: T.write('\nTaskmaster: Looking for a node to evaluate\n') while 1: node = self.next_candidate() if node is None: + if T: T.write('Taskmaster: No candidate anymore.\n\n') return None node = node.disambiguate() @@ -522,29 +604,27 @@ class Taskmaster: S.considered = S.considered + 1 else: S = None - - if T: T.write('Taskmaster: %s:' % repr(str(node))) - - # Skip this node if it has already been evaluated: - if state > SCons.Node.pending: + + if T: T.write('Taskmaster: Considering node <%-10s %s> and its children:\n' % + (StateString[node.get_state()], repr(str(node)))) + + if state == NODE_NO_STATE: + # Mark this node as being on the execution stack: + node.set_state(NODE_PENDING) + elif state > NODE_PENDING: + # Skip this node if it has already been evaluated: if S: S.already_handled = S.already_handled + 1 - if T: T.write(' already handled (%s)\n' % StateString[state]) + if T: T.write('Taskmaster: already handled (executed)\n') continue - # Mark this node as being on the execution stack: - node.set_state(SCons.Node.pending) - try: - children = node.children() + node.prerequisites + children = node.children() except SystemExit: exc_value = sys.exc_info()[1] e = SCons.Errors.ExplicitExit(node, exc_value.code) self.ready_exc = (SCons.Errors.ExplicitExit, e) - if T: T.write(' SystemExit\n') + if T: T.write('Taskmaster: SystemExit\n') return node - except KeyboardInterrupt: - if T: T.write(' KeyboardInterrupt\n') - raise except: # We had a problem just trying to figure out the # children (like a child couldn't be linked in to a @@ -552,70 +632,36 @@ class Taskmaster: # raise the exception when the Task is "executed." self.ready_exc = sys.exc_info() if S: S.problem = S.problem + 1 - if T: T.write(' exception\n') + if T: T.write('Taskmaster: exception while scanning children.\n') return node - if T and children: - c = map(str, children) - c.sort() - T.write(' children:\n %s\n ' % c) - - childstate = map(lambda N: (N, N.get_state()), children) - - # Detect dependency cycles: - pending_nodes = filter(lambda I: I[1] == SCons.Node.pending, childstate) - if pending_nodes: - for p in pending_nodes: - cycle = find_cycle([p[0], node]) - if cycle: - desc = "Dependency cycle: " + string.join(map(str, cycle), " -> ") - if T: T.write(' dependency cycle\n') - raise SCons.Errors.UserError, desc - - not_built = filter(lambda I: I[1] <= SCons.Node.executing, childstate) - if not_built: - # We're waiting on one or more derived targets that have - # not yet finished building. - - not_visited = filter(lambda I: not I[1], not_built) - if not_visited: - # Some of them haven't even been visited yet. - # Add them to the list so that on some next pass - # we can take a stab at evaluating them (or - # their children). - not_visited = map(lambda I: I[0], not_visited) - not_visited.reverse() - self.candidates.extend(self.order(not_visited)) - - n_b_nodes = map(lambda I: I[0], not_built) - - # Add this node to the waiting parents lists of anything - # we're waiting on, with a reference count so we can be - # put back on the list for re-evaluation when they've - # all finished. - map(lambda n, P=node: n.add_to_waiting_parents(P), n_b_nodes) - node.ref_count = len(set(n_b_nodes)) - - if S: S.not_built = S.not_built + 1 - if T: - c = map(str, n_b_nodes) - c.sort() - T.write(' waiting on unfinished children:\n %s\n' % c) - continue + children_not_visited = [] + children_pending = set() + children_not_ready = [] + children_failed = False - # Skip this node if it has side-effects that are - # currently being built: - side_effects = filter(lambda N: - N.get_state() == SCons.Node.executing, - node.side_effects) - if side_effects: - map(lambda n, P=node: n.add_to_waiting_s_e(P), side_effects) - if S: S.side_effects = S.side_effects + 1 - if T: - c = map(str, side_effects) - c.sort() - T.write(' waiting on side effects:\n %s\n' % c) - continue + for child in chain(children,node.prerequisites): + childstate = child.get_state() + + if T: T.write('Taskmaster: <%-10s %s>\n' % + (StateString[childstate], repr(str(child)))) + + if childstate == NODE_NO_STATE: + children_not_visited.append(child) + elif childstate == NODE_PENDING: + children_pending.add(child) + elif childstate == NODE_FAILED: + children_failed = True + + if childstate <= NODE_EXECUTING: + children_not_ready.append(child) + + + # These nodes have not even been visited yet. Add + # them to the list so that on some next pass we can + # take a stab at evaluating them (or their children). + children_not_visited.reverse() + self.candidates.extend(self.order(children_not_visited)) # Skip this node if any of its children have failed. # @@ -635,21 +681,47 @@ class Taskmaster: # Note that even if one of the children fails, we still # added the other children to the list of candidate nodes # to keep on building (--keep-going). - failed_children = filter(lambda I: I[1] == SCons.Node.failed, - childstate) - if failed_children: - node.set_state(SCons.Node.failed) + if children_failed: + node.set_state(NODE_FAILED) + if S: S.child_failed = S.child_failed + 1 - if T: - c = map(lambda I: str(I[0]), failed_children) - c.sort() - T.write(' children failed:\n %s\n' % c) + if T: T.write('Taskmaster:****** <%-10s %s>\n' % + (StateString[node.get_state()], repr(str(node)))) + continue + + if children_not_ready: + for child in children_not_ready: + # We're waiting on one or more derived targets + # that have not yet finished building. + if S: S.not_built = S.not_built + 1 + + # Add this node to the waiting parents lists of + # anything we're waiting on, with a reference + # count so we can be put back on the list for + # re-evaluation when they've all finished. + node.ref_count = node.ref_count + child.add_to_waiting_parents(node) + + self.pending_children = self.pending_children | children_pending + + continue + + # Skip this node if it has side-effects that are + # currently being built: + wait_side_effects = False + for se in node.side_effects: + if se.get_state() == NODE_EXECUTING: + se.add_to_waiting_s_e(node) + wait_side_effects = True + + if wait_side_effects: + if S: S.side_effects = S.side_effects + 1 continue # The default when we've gotten through all of the checks above: # this node is ready to be built. if S: S.build = S.build + 1 - if T: T.write(' evaluating %s\n' % node) + if T: T.write('Taskmaster: Evaluating <%-10s %s>\n' % + (StateString[node.get_state()], repr(str(node)))) return node return None @@ -671,8 +743,6 @@ class Taskmaster: task = self.tasker(self, tlist, node in self.original_top, node) try: task.make_ready() - except KeyboardInterrupt: - raise except: # We had a problem just trying to get this task ready (like # a child couldn't be linked in to a VariantDir when deciding @@ -692,3 +762,18 @@ class Taskmaster: Stops the current build completely. """ self.next_candidate = self.no_next_candidate + + def cleanup(self): + """ + Check for dependency cycles. + """ + if self.pending_children: + desc = 'Found dependency cycle(s):\n' + for node in self.pending_children: + cycle = find_cycle([node], set()) + if cycle: + desc = desc + " " + string.join(map(str, cycle), " -> ") + "\n" + else: + desc = desc + " Internal Error: no cycle found for node %s (%s)\n" % \ + (node, repr(node)) + raise SCons.Errors.UserError, desc diff --git a/src/engine/SCons/TaskmasterTests.py b/src/engine/SCons/TaskmasterTests.py index 9a7969b..88d3078 100644 --- a/src/engine/SCons/TaskmasterTests.py +++ b/src/engine/SCons/TaskmasterTests.py @@ -56,8 +56,8 @@ class Node: self.state = SCons.Node.no_state self.prepared = None self.ref_count = 0 - self.waiting_parents = {} - self.waiting_s_e = {} + self.waiting_parents = set() + self.waiting_s_e = set() self.side_effect = 0 self.side_effects = [] self.alttargets = [] @@ -119,17 +119,10 @@ class Node: def add_to_waiting_parents(self, node): wp = self.waiting_parents - if wp.has_key(node): - result = 0 - else: - result = 1 - wp[node] = 1 - return result - - def call_for_all_waiting_parents(self, func): - func(self) - for parent in self.waiting_parents.keys(): - parent.call_for_all_waiting_parents(func) + if node in wp: + return 0 + wp.add(node) + return 1 def get_state(self): return self.state @@ -166,13 +159,16 @@ class Node: def postprocess(self): self.postprocessed = 1 + self.waiting_parents = set() def get_executor(self): - class Executor: - pass - e = Executor() - e.targets = self.targets - return e + if not hasattr(self, 'executor'): + class Executor: + def prepare(self): + pass + self.executor = Executor() + self.executor.targets = self.targets + return self.executor class OtherError(Exception): pass @@ -469,7 +465,7 @@ class TaskmasterTestCase(unittest.TestCase): t.postprocess() s = n1.get_state() - assert s == SCons.Node.up_to_date, s + assert s == SCons.Node.executed, s s = n2.get_state() assert s == SCons.Node.executed, s @@ -815,6 +811,22 @@ class TaskmasterTestCase(unittest.TestCase): assert n9.prepared assert n10.prepared + # Make sure we call an Executor's prepare() method. + class ExceptionExecutor: + def prepare(self): + raise Exception, "Executor.prepare() exception" + + n11 = Node("n11") + n11.executor = ExceptionExecutor() + tm = SCons.Taskmaster.Taskmaster([n11]) + t = tm.next_task() + try: + t.prepare() + except Exception, e: + assert str(e) == "Executor.prepare() exception", e + else: + raise AssertionError, "did not catch expected exception" + def test_execute(self): """Test executing a task """ @@ -937,37 +949,46 @@ class TaskmasterTestCase(unittest.TestCase): ] assert str(exc_value) in exception_values, exc_value - t.exception_set(("exception 1", None)) + class Exception1(Exception): + pass + + t.exception_set((Exception1, None)) try: t.exception_raise() except: exc_type, exc_value = sys.exc_info()[:2] - assert exc_type == "exception 1", exc_type - assert exc_value is None, exc_value + assert exc_type == Exception1, exc_type + assert str(exc_value) == '', exc_value else: assert 0, "did not catch expected exception" - t.exception_set(("exception 2", "xyzzy")) + class Exception2(Exception): + pass + + t.exception_set((Exception2, "xyzzy")) try: t.exception_raise() except: exc_type, exc_value = sys.exc_info()[:2] - assert exc_type == "exception 2", exc_type - assert exc_value == "xyzzy", exc_value + assert exc_type == Exception2, exc_type + assert str(exc_value) == "xyzzy", exc_value else: assert 0, "did not catch expected exception" + class Exception3(Exception): + pass + try: 1/0 except: tb = sys.exc_info()[2] - t.exception_set(("exception 3", "arg", tb)) + t.exception_set((Exception3, "arg", tb)) try: t.exception_raise() except: exc_type, exc_value, exc_tb = sys.exc_info() - assert exc_type == 'exception 3', exc_type - assert exc_value == "arg", exc_value + assert exc_type == Exception3, exc_type + assert str(exc_value) == "arg", exc_value import traceback x = traceback.extract_tb(tb)[-1] y = traceback.extract_tb(exc_tb)[-1] @@ -1030,16 +1051,29 @@ class TaskmasterTestCase(unittest.TestCase): value = trace.getvalue() expect = """\ -Taskmaster: 'n1': evaluating n1 -Taskmaster: 'n1': already handled (executed) -Taskmaster: 'n3': children: - ['n1', 'n2'] - waiting on unfinished children: - ['n2'] -Taskmaster: 'n2': evaluating n2 -Taskmaster: 'n3': children: - ['n1', 'n2'] - evaluating n3 + +Taskmaster: Looking for a node to evaluate +Taskmaster: Considering node <no_state 'n1'> and its children: +Taskmaster: Evaluating <pending 'n1'> + +Taskmaster: Looking for a node to evaluate +Taskmaster: Considering node <executed 'n1'> and its children: +Taskmaster: already handled (executed) +Taskmaster: Considering node <no_state 'n3'> and its children: +Taskmaster: <executed 'n1'> +Taskmaster: <no_state 'n2'> +Taskmaster: Considering node <no_state 'n2'> and its children: +Taskmaster: Evaluating <pending 'n2'> + +Taskmaster: Looking for a node to evaluate +Taskmaster: Considering node <pending 'n3'> and its children: +Taskmaster: <executed 'n1'> +Taskmaster: <executed 'n2'> +Taskmaster: Evaluating <pending 'n3'> + +Taskmaster: Looking for a node to evaluate +Taskmaster: No candidate anymore. + """ assert value == expect, value diff --git a/src/engine/SCons/Tool/ToolTests.py b/src/engine/SCons/Tool/ToolTests.py index b9230f1..779aaaa 100644 --- a/src/engine/SCons/Tool/ToolTests.py +++ b/src/engine/SCons/Tool/ToolTests.py @@ -54,7 +54,6 @@ class ToolTestCase(unittest.TestCase): t = SCons.Tool.Tool('g++') t(env) assert (env['CXX'] == 'c++' or env['CXX'] == 'g++'), env['CXX'] - assert env['CXXFLAGS'] == ['$CCFLAGS'], env['CXXFLAGS'] assert env['INCPREFIX'] == '-I', env['INCPREFIX'] assert env['TOOLS'] == ['g++'], env['TOOLS'] diff --git a/src/engine/SCons/Tool/__init__.py b/src/engine/SCons/Tool/__init__.py index 1e69f1e..e0ada5e 100644 --- a/src/engine/SCons/Tool/__init__.py +++ b/src/engine/SCons/Tool/__init__.py @@ -163,10 +163,10 @@ class Tool: kw = self.init_kw env.Append(TOOLS = [ self.name ]) if hasattr(self, 'options'): - from SCons.Options import Options + import SCons.Variables if not env.has_key('options'): from SCons.Script import ARGUMENTS - env['options']=Options(args=ARGUMENTS) + env['options']=SCons.Variables.Variables(args=ARGUMENTS) opts=env['options'] self.options(opts) diff --git a/src/engine/SCons/Tool/c++.py b/src/engine/SCons/Tool/c++.py index d9370b0..a4375c0 100644 --- a/src/engine/SCons/Tool/c++.py +++ b/src/engine/SCons/Tool/c++.py @@ -71,13 +71,13 @@ def generate(env): shared_obj.add_emitter(suffix, SCons.Defaults.SharedObjectEmitter) SCons.Tool.cc.add_common_cc_variables(env) - + env['CXX'] = 'c++' - env['CXXFLAGS'] = SCons.Util.CLVar('$CCFLAGS') - env['CXXCOM'] = '$CXX -o $TARGET -c $CXXFLAGS $_CCCOMCOM $SOURCES' + env['CXXFLAGS'] = SCons.Util.CLVar('') + env['CXXCOM'] = '$CXX -o $TARGET -c $CXXFLAGS $CCFLAGS $_CCCOMCOM $SOURCES' env['SHCXX'] = '$CXX' - env['SHCXXFLAGS'] = SCons.Util.CLVar('$CXXFLAGS') - env['SHCXXCOM'] = '$SHCXX -o $TARGET -c $SHCXXFLAGS $_CCCOMCOM $SOURCES' + env['SHCXXFLAGS'] = SCons.Util.CLVar('') + env['SHCXXCOM'] = '$SHCXX -o $TARGET -c $SHCXXFLAGS $SHCCFLAGS $_CCCOMCOM $SOURCES' env['CPPDEFPREFIX'] = '-D' env['CPPDEFSUFFIX'] = '' diff --git a/src/engine/SCons/Tool/dmd.py b/src/engine/SCons/Tool/dmd.py index 64ffb68..a173162 100644 --- a/src/engine/SCons/Tool/dmd.py +++ b/src/engine/SCons/Tool/dmd.py @@ -14,14 +14,15 @@ use absolute paths. To hack around it, add '#/blah'. This will link blah.lib from the directory where SConstruct resides. Compiler variables: - DC - The name of the D compiler to use. Defaults to dmd. + DC - The name of the D compiler to use. Defaults to dmd or gdmd, + whichever is found. DPATH - List of paths to search for import modules. DVERSIONS - List of version tags to enable when compiling. DDEBUG - List of debug tags to enable when compiling. Linker related variables: LIBS - List of library files to link in. - DLINK - Name of the linker to use. Defaults to dmd. + DLINK - Name of the linker to use. Defaults to dmd or gdmd. DLINKFLAGS - List of linker flags. Lib tool variables: @@ -93,7 +94,8 @@ def generate(env): static_obj.add_emitter('.d', SCons.Defaults.StaticObjectEmitter) shared_obj.add_emitter('.d', SCons.Defaults.SharedObjectEmitter) - env['DC'] = 'dmd' + dc = env.Detect(['dmd', 'gdmd']) + env['DC'] = dc env['DCOM'] = '$DC $_DINCFLAGS $_DVERFLAGS $_DDEBUGFLAGS $_DFLAGS -c -of$TARGET $SOURCES' env['_DINCFLAGS'] = '$( ${_concat(DINCPREFIX, DPATH, DINCSUFFIX, __env__, RDirs, TARGET, SOURCE)} $)' env['_DVERFLAGS'] = '$( ${_concat(DVERPREFIX, DVERSIONS, DVERSUFFIX, __env__)} $)' @@ -105,14 +107,15 @@ def generate(env): env['DVERSIONS'] = [] env['DDEBUG'] = [] - # Add the path to the standard library. - # This is merely for the convenience of the dependency scanner. - dmd_path = env.WhereIs('dmd') - if dmd_path: - x = string.rindex(dmd_path, 'dmd') - phobosDir = dmd_path[:x] + '/../src/phobos' - if os.path.isdir(phobosDir): - env.Append(DPATH = [phobosDir]) + if dc: + # Add the path to the standard library. + # This is merely for the convenience of the dependency scanner. + dmd_path = env.WhereIs(dc) + if dmd_path: + x = string.rindex(dmd_path, dc) + phobosDir = dmd_path[:x] + '/../src/phobos' + if os.path.isdir(phobosDir): + env.Append(DPATH = [phobosDir]) env['DINCPREFIX'] = '-I' env['DINCSUFFIX'] = '' @@ -198,7 +201,10 @@ def generate(env): except KeyError: libs = [] if 'phobos' not in libs: - env.Append(LIBS = ['phobos']) + if dc is 'dmd': + env.Append(LIBS = ['phobos']) + elif dc is 'gdmd': + env.Append(LIBS = ['gphobos']) if 'pthread' not in libs: env.Append(LIBS = ['pthread']) if 'm' not in libs: @@ -209,4 +215,4 @@ def generate(env): env['LINKCOM'] = '$SMART_LINKCOM ' def exists(env): - return env.Detect('dmd') + return env.Detect(['dmd', 'gdmd']) diff --git a/src/engine/SCons/Tool/dmd.xml b/src/engine/SCons/Tool/dmd.xml index cab3ab4..835a4eb 100644 --- a/src/engine/SCons/Tool/dmd.xml +++ b/src/engine/SCons/Tool/dmd.xml @@ -6,7 +6,8 @@ See its __doc__ string for a discussion of the format. --> <tool name="dmd"> <summary> -Sets construction variables for the Digital Mars D compiler. +Sets construction variables for D language compilers +(the Digital Mars D compiler, or GDC). </summary> <sets> <!-- diff --git a/src/engine/SCons/Tool/g++.py b/src/engine/SCons/Tool/g++.py index 0576b90..9c7e477 100644 --- a/src/engine/SCons/Tool/g++.py +++ b/src/engine/SCons/Tool/g++.py @@ -63,18 +63,16 @@ def generate(env): env['SHOBJSUFFIX'] = '$OBJSUFFIX' elif env['PLATFORM'] == 'hpux': # Original line from Christian Engel added -DPIC: - #env['SHCXXFLAGS'] = SCons.Util.CLVar('$CXXFLAGS -fPIC -DPIC') - env['SHCXXFLAGS'] = SCons.Util.CLVar('$CXXFLAGS -fPIC') + #env['SHCXXFLAGS'] = SCons.Util.CLVar('$CXXFLAGS -DPIC') env['SHOBJSUFFIX'] = '.pic.o' elif env['PLATFORM'] == 'sunos': # Original line from Christian Engel added -DPIC: - #env['SHCXXFLAGS'] = SCons.Util.CLVar('$CXXFLAGS -fPIC -DPIC') - env['SHCXXFLAGS'] = SCons.Util.CLVar('$CXXFLAGS -fPIC') + #env['SHCXXFLAGS'] = SCons.Util.CLVar('$CXXFLAGS -DPIC') env['SHOBJSUFFIX'] = '.pic.o' else: # Original line from Christian Engel added -DPIC: - #env['SHCXXFLAGS'] = SCons.Util.CLVar('$CXXFLAGS -fPIC -DPIC') - env['SHCXXFLAGS'] = SCons.Util.CLVar('$CXXFLAGS -fPIC') + #env['SHCXXFLAGS'] = SCons.Util.CLVar('$CXXFLAGS -DPIC') + pass # determine compiler version if env['CXX']: line = os.popen(env['CXX'] + ' --version').readline() diff --git a/src/engine/SCons/Tool/intelc.py b/src/engine/SCons/Tool/intelc.py index 02cc52a..db5d06e 100644 --- a/src/engine/SCons/Tool/intelc.py +++ b/src/engine/SCons/Tool/intelc.py @@ -375,14 +375,14 @@ def generate(env, version=None, abi=None, topdir=None, verbose=0): 'LIB' : 'lib', 'PATH' : 'bin', 'LD_LIBRARY_PATH' : 'lib'} - for p in paths: + for p in paths.keys(): env.PrependENVPath(p, os.path.join(topdir, paths[p])) if is_mac: paths={'INCLUDE' : 'include', 'LIB' : 'lib', 'PATH' : 'bin', 'LD_LIBRARY_PATH' : 'lib'} - for p in paths: + for p in paths.keys(): env.PrependENVPath(p, os.path.join(topdir, paths[p])) if is_windows: # env key reg valname default subdir of top @@ -394,7 +394,7 @@ def generate(env, version=None, abi=None, topdir=None, verbose=0): if version is None: version = '' # Each path has a registry entry, use that or default to subdir - for p in paths: + for p in paths.keys(): try: path=get_intel_registry_value(p[1], version, abi) # These paths may have $(ICInstallDir) diff --git a/src/engine/SCons/Tool/javac.py b/src/engine/SCons/Tool/javac.py index a020727..cb79301 100644 --- a/src/engine/SCons/Tool/javac.py +++ b/src/engine/SCons/Tool/javac.py @@ -92,8 +92,9 @@ def emit_java_classes(target, source, env): raise SCons.Errors.UserError("Java source must be File or Dir, not '%s'" % entry.__class__) version = env.get('JAVAVERSION', '1.4') - tlist = [] + full_tlist = [] for f in slist: + tlist = [] source_file_based = True pkg_dir = None if not f.is_derived(): @@ -124,7 +125,12 @@ def emit_java_classes(target, source, env): t.attributes.java_classname = classname(base) tlist.append(t) - return tlist, slist + for t in tlist: + t.set_specific_source([f]) + + full_tlist.extend(tlist) + + return full_tlist, slist JavaAction = SCons.Action.Action('$JAVACCOM', '$JAVACCOMSTR') diff --git a/src/engine/SCons/Tool/mingw.py b/src/engine/SCons/Tool/mingw.py index 359c40f..8c5e68a 100644 --- a/src/engine/SCons/Tool/mingw.py +++ b/src/engine/SCons/Tool/mingw.py @@ -122,6 +122,7 @@ def generate(env): env['SHCXXFLAGS'] = SCons.Util.CLVar('$CXXFLAGS') env['SHLINKFLAGS'] = SCons.Util.CLVar('$LINKFLAGS -shared') env['SHLINKCOM'] = shlib_action + env['LDMODULECOM'] = shlib_action env.Append(SHLIBEMITTER = [shlib_emitter]) env['AS'] = 'as' diff --git a/src/engine/SCons/Tool/mingw.xml b/src/engine/SCons/Tool/mingw.xml index 048124c..f3354c1 100644 --- a/src/engine/SCons/Tool/mingw.xml +++ b/src/engine/SCons/Tool/mingw.xml @@ -15,6 +15,7 @@ CXX SHCXXFLAGS SHLINKFLAGS SHLINKCOM +LDMODULECOM AS WINDOWSDEFPREFIX WINDOWSDEFSUFFIX diff --git a/src/engine/SCons/Tool/packaging/__init__.py b/src/engine/SCons/Tool/packaging/__init__.py index 79cd4ab..02be24f 100644 --- a/src/engine/SCons/Tool/packaging/__init__.py +++ b/src/engine/SCons/Tool/packaging/__init__.py @@ -29,7 +29,7 @@ SCons Packaging Tool. __revision__ = "__FILE__ __REVISION__ __DATE__ __DEVELOPER__" import SCons.Environment -from SCons.Options import * +from SCons.Variables import * from SCons.Errors import * from SCons.Util import is_List, make_path_relative from SCons.Warnings import warn, Warning @@ -217,11 +217,11 @@ def exists(env): # XXX def options(opts): - opts.AddOptions( - EnumOption( 'PACKAGETYPE', - 'the type of package to create.', - None, allowed_values=map( str, __all__ ), - ignorecase=2 + opts.AddVariables( + EnumVariable( 'PACKAGETYPE', + 'the type of package to create.', + None, allowed_values=map( str, __all__ ), + ignorecase=2 ) ) diff --git a/src/engine/SCons/Tool/swig.py b/src/engine/SCons/Tool/swig.py index eba49a7..16f938e 100644 --- a/src/engine/SCons/Tool/swig.py +++ b/src/engine/SCons/Tool/swig.py @@ -62,7 +62,8 @@ def _swigEmitter(target, source, env): if "-python" in flags and "-noproxy" not in flags: if mnames is None: mnames = _reModule.findall(open(src).read()) - target.extend(map(lambda m: m + ".py", mnames)) + target.extend(map(lambda m, d=target[0].dir: + d.File(m + ".py"), mnames)) if "-java" in flags: if mnames is None: mnames = _reModule.findall(open(src).read()) diff --git a/src/engine/SCons/Util.py b/src/engine/SCons/Util.py index 311c6a8..af09786 100644 --- a/src/engine/SCons/Util.py +++ b/src/engine/SCons/Util.py @@ -109,41 +109,6 @@ def updrive(path): path = string.upper(drive) + rest return path -# -# Generic convert-to-string functions that abstract away whether or -# not the Python we're executing has Unicode support. The wrapper -# to_String_for_signature() will use a for_signature() method if the -# specified object has one. -# -if hasattr(types, 'UnicodeType'): - UnicodeType = types.UnicodeType - def to_String(s): - if isinstance(s, UserString): - t = type(s.data) - else: - t = type(s) - if t is UnicodeType: - return unicode(s) - else: - return str(s) -else: - to_String = str - -def to_String_for_signature(obj): - try: - f = obj.for_signature - except AttributeError: - return to_String_for_subst(obj) - else: - return f() - -def to_String_for_subst(s): - if is_Sequence( s ): - return string.join( map(to_String_for_subst, s) ) - - return to_String( s ) - - class CallableComposite(UserList): """A simple composite callable class that, when called, will invoke all of its contained callables with the same arguments.""" @@ -213,7 +178,17 @@ class DisplayEngine: def print_it(self, text, append_newline=1): if append_newline: text = text + '\n' - sys.stdout.write(text) + try: + sys.stdout.write(text) + except IOError: + # Stdout might be connected to a pipe that has been closed + # by now. The most likely reason for the pipe being closed + # is that the user has press ctrl-c. It this is the case, + # then SCons is currently shutdown. We therefore ignore + # IOError's here so that SCons can continue and shutdown + # properly so that the .sconsign is correctly written + # before SCons exits. + pass def dont_print(self, text, append_newline=1): pass @@ -437,6 +412,41 @@ except TypeError: else: flatten_sequence(item, result) return result + + # + # Generic convert-to-string functions that abstract away whether or + # not the Python we're executing has Unicode support. The wrapper + # to_String_for_signature() will use a for_signature() method if the + # specified object has one. + # + if hasattr(types, 'UnicodeType'): + UnicodeType = types.UnicodeType + def to_String(s): + if isinstance(s, UserString): + t = type(s.data) + else: + t = type(s) + if t is UnicodeType: + return unicode(s) + else: + return str(s) + else: + to_String = str + + def to_String_for_signature(obj): + try: + f = obj.for_signature + except AttributeError: + return to_String_for_subst(obj) + else: + return f() + + def to_String_for_subst(s): + if is_Sequence( s ): + return string.join( map(to_String_for_subst, s) ) + + return to_String( s ) + else: # A modern Python version with new-style classes, so we can just use # isinstance(). @@ -459,6 +469,10 @@ else: # with basestring. (at least on Python 2.5.1) StringTypes = (str, unicode, UserString) + # Empirically, it is faster to check explicitely for str and + # unicode than for basestring. + BaseStringTypes = (str, unicode) + def is_Dict(obj, isinstance=isinstance, DictTypes=DictTypes): return isinstance(obj, DictTypes) @@ -526,6 +540,56 @@ else: return result + # + # Generic convert-to-string functions that abstract away whether or + # not the Python we're executing has Unicode support. The wrapper + # to_String_for_signature() will use a for_signature() method if the + # specified object has one. + # + def to_String(s, + isinstance=isinstance, str=str, + UserString=UserString, BaseStringTypes=BaseStringTypes): + if isinstance(s,BaseStringTypes): + # Early out when already a string! + return s + elif isinstance(s, UserString): + # s.data can only be either a unicode or a regular + # string. Please see the UserString initializer. + return s.data + else: + return str(s) + + def to_String_for_subst(s, + isinstance=isinstance, join=string.join, str=str, to_String=to_String, + BaseStringTypes=BaseStringTypes, SequenceTypes=SequenceTypes, + UserString=UserString): + + # Note that the test cases are sorted by order of probability. + if isinstance(s, BaseStringTypes): + return s + elif isinstance(s, SequenceTypes): + l = [] + for e in s: + l.append(to_String_for_subst(e)) + return join( s ) + elif isinstance(s, UserString): + # s.data can only be either a unicode or a regular + # string. Please see the UserString initializer. + return s.data + else: + return str(s) + + def to_String_for_signature(obj, to_String_for_subst=to_String_for_subst, + AttributeError=AttributeError): + try: + f = obj.for_signature + except AttributeError: + return to_String_for_subst(obj) + else: + return f() + + + # The SCons "semi-deep" copy. # # This makes separate copies of lists (including UserList objects) @@ -1267,8 +1331,18 @@ class Unbuffered: def __init__(self, file): self.file = file def write(self, arg): - self.file.write(arg) - self.file.flush() + try: + self.file.write(arg) + self.file.flush() + except IOError: + # Stdout might be connected to a pipe that has been closed + # by now. The most likely reason for the pipe being closed + # is that the user has press ctrl-c. It this is the case, + # then SCons is currently shutdown. We therefore ignore + # IOError's here so that SCons can continue and shutdown + # properly so that the .sconsign is correctly written + # before SCons exits. + pass def __getattr__(self, attr): return getattr(self.file, attr) diff --git a/src/engine/SCons/Options/BoolOption.py b/src/engine/SCons/Variables/BoolVariable.py index 80b607d..92936e6 100644 --- a/src/engine/SCons/Options/BoolOption.py +++ b/src/engine/SCons/Variables/BoolVariable.py @@ -1,11 +1,11 @@ -"""engine.SCons.Options.BoolOption +"""engine.SCons.Variables.BoolVariable This file defines the option type for SCons implementing true/false values. Usage example: - opts = Options() - opts.Add(BoolOption('embedded', 'build for an embedded system', 0)) + opts = Variables() + opts.Add(BoolVariable('embedded', 'build for an embedded system', 0)) ... if env['embedded'] == 1: ... @@ -36,7 +36,7 @@ Usage example: __revision__ = "__FILE__ __REVISION__ __DATE__ __DEVELOPER__" -__all__ = ('BoolOption') +__all__ = ['BoolVariable',] import string @@ -56,7 +56,7 @@ def _text2bool(val): See '__true_strings' and '__false_strings' for values considered 'true' or 'false respectivly. - This is usable as 'converter' for SCons' Options. + This is usable as 'converter' for SCons' Variables. """ lval = string.lower(val) if lval in __true_strings: return True @@ -68,14 +68,14 @@ def _validator(key, val, env): """ Validates the given value to be either '0' or '1'. - This is usable as 'validator' for SCons' Options. + This is usable as 'validator' for SCons' Variables. """ if not env[key] in (True, False): raise SCons.Errors.UserError( 'Invalid value for boolean option %s: %s' % (key, env[key])) -def BoolOption(key, help, default): +def BoolVariable(key, help, default): """ The input parameters describe a boolen option, thus they are returned with the correct converter and validator appended. The diff --git a/src/engine/SCons/Options/BoolOptionTests.py b/src/engine/SCons/Variables/BoolVariableTests.py index 07b5b79..feb4e4e 100644 --- a/src/engine/SCons/Options/BoolOptionTests.py +++ b/src/engine/SCons/Variables/BoolVariableTests.py @@ -29,13 +29,13 @@ import sys import unittest import SCons.Errors -import SCons.Options +import SCons.Variables -class BoolOptionTestCase(unittest.TestCase): - def test_BoolOption(self): - """Test BoolOption creation""" - opts = SCons.Options.Options() - opts.Add(SCons.Options.BoolOption('test', 'test option help', 0)) +class BoolVariableTestCase(unittest.TestCase): + def test_BoolVariable(self): + """Test BoolVariable creation""" + opts = SCons.Variables.Variables() + opts.Add(SCons.Variables.BoolVariable('test', 'test option help', 0)) o = opts.options[0] assert o.key == 'test', o.key @@ -45,9 +45,9 @@ class BoolOptionTestCase(unittest.TestCase): assert not o.converter is None, o.converter def test_converter(self): - """Test the BoolOption converter""" - opts = SCons.Options.Options() - opts.Add(SCons.Options.BoolOption('test', 'test option help', 0)) + """Test the BoolVariable converter""" + opts = SCons.Variables.Variables() + opts.Add(SCons.Variables.BoolVariable('test', 'test option help', 0)) o = opts.options[0] @@ -86,9 +86,9 @@ class BoolOptionTestCase(unittest.TestCase): assert caught, "did not catch expected ValueError" def test_validator(self): - """Test the BoolOption validator""" - opts = SCons.Options.Options() - opts.Add(SCons.Options.BoolOption('test', 'test option help', 0)) + """Test the BoolVariable validator""" + opts = SCons.Variables.Variables() + opts.Add(SCons.Variables.BoolVariable('test', 'test option help', 0)) o = opts.options[0] @@ -118,6 +118,6 @@ class BoolOptionTestCase(unittest.TestCase): if __name__ == "__main__": - suite = unittest.makeSuite(BoolOptionTestCase, 'test_') + suite = unittest.makeSuite(BoolVariableTestCase, 'test_') if not unittest.TextTestRunner().run(suite).wasSuccessful(): sys.exit(1) diff --git a/src/engine/SCons/Options/EnumOption.py b/src/engine/SCons/Variables/EnumVariable.py index d4e2ac1..e1b20d2 100644 --- a/src/engine/SCons/Options/EnumOption.py +++ b/src/engine/SCons/Variables/EnumVariable.py @@ -1,12 +1,12 @@ -"""engine.SCons.Options.EnumOption +"""engine.SCons.Variables.EnumVariable This file defines the option type for SCons allowing only specified input-values. Usage example: - opts = Options() - opts.Add(EnumOption('debug', 'debug output and symbols', 'no', + opts = Variables() + opts.Add(EnumVariable('debug', 'debug output and symbols', 'no', allowed_values=('yes', 'no', 'full'), map={}, ignorecase=2)) ... @@ -39,7 +39,7 @@ Usage example: __revision__ = "__FILE__ __REVISION__ __DATE__ __DEVELOPER__" -__all__ = ('EnumOption',) +__all__ = ['EnumVariable',] import string @@ -51,14 +51,14 @@ def _validator(key, val, env, vals): 'Invalid value for option %s: %s' % (key, val)) -def EnumOption(key, help, default, allowed_values, map={}, ignorecase=0): +def EnumVariable(key, help, default, allowed_values, map={}, ignorecase=0): """ The input parameters describe a option with only certain values allowed. They are returned with an appropriate converter and validator appended. The result is usable for input to - Options.Add(). + Variables.Add(). - 'key' and 'default' are the values to be passed on to Options.Add(). + 'key' and 'default' are the values to be passed on to Variables.Add(). 'help' will be appended by the allowed values automatically diff --git a/src/engine/SCons/Options/EnumOptionTests.py b/src/engine/SCons/Variables/EnumVariableTests.py index 20ae6c3..0f9afde 100644 --- a/src/engine/SCons/Options/EnumOptionTests.py +++ b/src/engine/SCons/Variables/EnumVariableTests.py @@ -27,13 +27,13 @@ import sys import unittest import SCons.Errors -import SCons.Options +import SCons.Variables -class EnumOptionTestCase(unittest.TestCase): - def test_EnumOption(self): - """Test EnumOption creation""" - opts = SCons.Options.Options() - opts.Add(SCons.Options.EnumOption('test', 'test option help', 0, +class EnumVariableTestCase(unittest.TestCase): + def test_EnumVariable(self): + """Test EnumVariable creation""" + opts = SCons.Variables.Variables() + opts.Add(SCons.Variables.EnumVariable('test', 'test option help', 0, ['one', 'two', 'three'], {})) @@ -45,9 +45,9 @@ class EnumOptionTestCase(unittest.TestCase): assert not o.converter is None, o.converter def test_converter(self): - """Test the EnumOption converter""" - opts = SCons.Options.Options() - opts.Add(SCons.Options.EnumOption('test', 'test option help', 0, + """Test the EnumVariable converter""" + opts = SCons.Variables.Variables() + opts.Add(SCons.Variables.EnumVariable('test', 'test option help', 0, ['one', 'two', 'three'])) o = opts.options[0] @@ -56,8 +56,8 @@ class EnumOptionTestCase(unittest.TestCase): x = o.converter(a) assert x == a, x - opts = SCons.Options.Options() - opts.Add(SCons.Options.EnumOption('test', 'test option help', 0, + opts = SCons.Variables.Variables() + opts.Add(SCons.Variables.EnumVariable('test', 'test option help', 0, ['one', 'two', 'three'], {'1' : 'one', '2' : 'two', @@ -80,20 +80,20 @@ class EnumOptionTestCase(unittest.TestCase): x = o.converter('3') assert x == 'three', x - opts = SCons.Options.Options() - opts.Add(SCons.Options.EnumOption('test0', 'test option help', 0, + opts = SCons.Variables.Variables() + opts.Add(SCons.Variables.EnumVariable('test0', 'test option help', 0, ['one', 'two', 'three'], {'a' : 'one', 'b' : 'two', 'c' : 'three'}, ignorecase=0)) - opts.Add(SCons.Options.EnumOption('test1', 'test option help', 0, + opts.Add(SCons.Variables.EnumVariable('test1', 'test option help', 0, ['one', 'two', 'three'], {'a' : 'one', 'b' : 'two', 'c' : 'three'}, ignorecase=1)) - opts.Add(SCons.Options.EnumOption('test2', 'test option help', 0, + opts.Add(SCons.Variables.EnumVariable('test2', 'test option help', 0, ['one', 'two', 'three'], {'a' : 'one', 'b' : 'two', @@ -131,21 +131,21 @@ class EnumOptionTestCase(unittest.TestCase): assert x == l[2], "o2 got %s, expected %s" % (x, l[2]) def test_validator(self): - """Test the EnumOption validator""" - opts = SCons.Options.Options() - opts.Add(SCons.Options.EnumOption('test0', 'test option help', 0, + """Test the EnumVariable validator""" + opts = SCons.Variables.Variables() + opts.Add(SCons.Variables.EnumVariable('test0', 'test option help', 0, ['one', 'two', 'three'], {'a' : 'one', 'b' : 'two', 'c' : 'three'}, ignorecase=0)) - opts.Add(SCons.Options.EnumOption('test1', 'test option help', 0, + opts.Add(SCons.Variables.EnumVariable('test1', 'test option help', 0, ['one', 'two', 'three'], {'a' : 'one', 'b' : 'two', 'c' : 'three'}, ignorecase=1)) - opts.Add(SCons.Options.EnumOption('test2', 'test option help', 0, + opts.Add(SCons.Variables.EnumVariable('test2', 'test option help', 0, ['one', 'two', 'three'], {'a' : 'one', 'b' : 'two', @@ -193,6 +193,6 @@ class EnumOptionTestCase(unittest.TestCase): if __name__ == "__main__": - suite = unittest.makeSuite(EnumOptionTestCase, 'test_') + suite = unittest.makeSuite(EnumVariableTestCase, 'test_') if not unittest.TextTestRunner().run(suite).wasSuccessful(): sys.exit(1) diff --git a/src/engine/SCons/Options/ListOption.py b/src/engine/SCons/Variables/ListVariable.py index 6954905..37e3094 100644 --- a/src/engine/SCons/Options/ListOption.py +++ b/src/engine/SCons/Variables/ListVariable.py @@ -1,4 +1,4 @@ -"""engine.SCons.Options.ListOption +"""engine.SCons.Variables.ListVariable This file defines the option type for SCons implementing 'lists'. @@ -11,8 +11,8 @@ Usage example: list_of_libs = Split('x11 gl qt ical') - opts = Options() - opts.Add(ListOption('shared', + opts = Variables() + opts.Add(ListVariable('shared', 'libraries to build as shared libraries', 'all', elems = list_of_libs)) @@ -52,7 +52,7 @@ __revision__ = "__FILE__ __REVISION__ __DATE__ __DEVELOPER__" # Know Bug: This should behave like a Set-Type, but does not really, # since elements can occur twice. -__all__ = ('ListOption',) +__all__ = ['ListVariable',] import string import UserList @@ -60,7 +60,7 @@ import UserList import SCons.Util -class _ListOption(UserList.UserList): +class _ListVariable(UserList.UserList): def __init__(self, initlist=[], allowedElems=[]): UserList.UserList.__init__(self, filter(None, initlist)) self.allowedElems = allowedElems[:] @@ -103,7 +103,7 @@ def _converter(val, allowedElems, mapdict): if notAllowed: raise ValueError("Invalid value(s) for option: %s" % string.join(notAllowed, ',')) - return _ListOption(val, allowedElems) + return _ListVariable(val, allowedElems) ## def _validator(key, val, env): @@ -113,7 +113,7 @@ def _converter(val, allowedElems, mapdict): ## return 1 -def ListOption(key, help, default, names, map={}): +def ListVariable(key, help, default, names, map={}): """ The input parameters describe a 'package list' option, thus they are returned with the correct converter and validater appended. The diff --git a/src/engine/SCons/Options/ListOptionTests.py b/src/engine/SCons/Variables/ListVariableTests.py index 22378bc..9363f6d 100644 --- a/src/engine/SCons/Options/ListOptionTests.py +++ b/src/engine/SCons/Variables/ListVariableTests.py @@ -28,13 +28,13 @@ import sys import unittest import SCons.Errors -import SCons.Options +import SCons.Variables -class ListOptionTestCase(unittest.TestCase): - def test_ListOption(self): - """Test ListOption creation""" - opts = SCons.Options.Options() - opts.Add(SCons.Options.ListOption('test', 'test option help', 'all', +class ListVariableTestCase(unittest.TestCase): + def test_ListVariable(self): + """Test ListVariable creation""" + opts = SCons.Variables.Variables() + opts.Add(SCons.Variables.ListVariable('test', 'test option help', 'all', ['one', 'two', 'three'])) o = opts.options[0] @@ -44,8 +44,8 @@ class ListOptionTestCase(unittest.TestCase): assert o.validator is None, o.validator assert not o.converter is None, o.converter - opts = SCons.Options.Options() - opts.Add(SCons.Options.ListOption('test2', 'test2 help', + opts = SCons.Variables.Variables() + opts.Add(SCons.Variables.ListVariable('test2', 'test2 help', ['one', 'three'], ['one', 'two', 'three'])) @@ -53,9 +53,9 @@ class ListOptionTestCase(unittest.TestCase): assert o.default == 'one,three' def test_converter(self): - """Test the ListOption converter""" - opts = SCons.Options.Options() - opts.Add(SCons.Options.ListOption('test', 'test option help', 'all', + """Test the ListVariable converter""" + opts = SCons.Variables.Variables() + opts.Add(SCons.Variables.ListVariable('test', 'test option help', 'all', ['one', 'two', 'three'], {'ONE':'one', 'TWO':'two'})) @@ -112,9 +112,9 @@ class ListOptionTestCase(unittest.TestCase): assert caught, "did not catch expected ValueError" def test_copy(self): - """Test copying a ListOption like an Environment would""" - opts = SCons.Options.Options() - opts.Add(SCons.Options.ListOption('test', 'test option help', 'all', + """Test copying a ListVariable like an Environment would""" + opts = SCons.Variables.Variables() + opts.Add(SCons.Variables.ListVariable('test', 'test option help', 'all', ['one', 'two', 'three'])) o = opts.options[0] @@ -123,6 +123,6 @@ class ListOptionTestCase(unittest.TestCase): n = l.__class__(copy.copy(l)) if __name__ == "__main__": - suite = unittest.makeSuite(ListOptionTestCase, 'test_') + suite = unittest.makeSuite(ListVariableTestCase, 'test_') if not unittest.TextTestRunner().run(suite).wasSuccessful(): sys.exit(1) diff --git a/src/engine/SCons/Options/PackageOption.py b/src/engine/SCons/Variables/PackageVariable.py index 3b4f0ce..6be260d 100644 --- a/src/engine/SCons/Options/PackageOption.py +++ b/src/engine/SCons/Variables/PackageVariable.py @@ -1,4 +1,4 @@ -"""engine.SCons.Options.PackageOption +"""engine.SCons.Variables.PackageVariable This file defines the option type for SCons implementing 'package activation'. @@ -15,8 +15,8 @@ Usage example: To replace autoconf's --with-xxx=yyy - opts = Options() - opts.Add(PackageOption('x11', + opts = Variables() + opts.Add(PackageVariable('x11', 'use X11 installed here (yes = search some places', 'yes')) ... @@ -52,7 +52,7 @@ Usage example: __revision__ = "__FILE__ __REVISION__ __DATE__ __DEVELOPER__" -__all__ = ('PackageOption') +__all__ = ['PackageVariable',] import string @@ -86,7 +86,7 @@ def _validator(key, val, env, searchfunc): 'Path does not exist for option %s: %s' % (key, val)) -def PackageOption(key, help, default, searchfunc=None): +def PackageVariable(key, help, default, searchfunc=None): # NB: searchfunc is currenty undocumented and unsupported """ The input parameters describe a 'package list' option, thus they diff --git a/src/engine/SCons/Options/PackageOptionTests.py b/src/engine/SCons/Variables/PackageVariableTests.py index 68f14e5..54116cf 100644 --- a/src/engine/SCons/Options/PackageOptionTests.py +++ b/src/engine/SCons/Variables/PackageVariableTests.py @@ -29,15 +29,15 @@ import sys import unittest import SCons.Errors -import SCons.Options +import SCons.Variables import TestCmd -class PackageOptionTestCase(unittest.TestCase): - def test_PackageOption(self): - """Test PackageOption creation""" - opts = SCons.Options.Options() - opts.Add(SCons.Options.PackageOption('test', 'test option help', '/default/path')) +class PackageVariableTestCase(unittest.TestCase): + def test_PackageVariable(self): + """Test PackageVariable creation""" + opts = SCons.Variables.Variables() + opts.Add(SCons.Variables.PackageVariable('test', 'test option help', '/default/path')) o = opts.options[0] assert o.key == 'test', o.key @@ -47,9 +47,9 @@ class PackageOptionTestCase(unittest.TestCase): assert not o.converter is None, o.converter def test_converter(self): - """Test the PackageOption converter""" - opts = SCons.Options.Options() - opts.Add(SCons.Options.PackageOption('test', 'test option help', '/default/path')) + """Test the PackageVariable converter""" + opts = SCons.Variables.Variables() + opts.Add(SCons.Variables.PackageVariable('test', 'test option help', '/default/path')) o = opts.options[0] @@ -80,7 +80,7 @@ class PackageOptionTestCase(unittest.TestCase): # Make sure the converter returns True if we give it str(True) and # False when we give it str(False). This assures consistent operation - # through a cycle of Options.Save(<file>) -> Options(<file>). + # through a cycle of Variables.Save(<file>) -> Variables(<file>). x = o.converter(str(True)) assert x == True, "converter returned a string when given str(True)" @@ -88,9 +88,9 @@ class PackageOptionTestCase(unittest.TestCase): assert x == False, "converter returned a string when given str(False)" def test_validator(self): - """Test the PackageOption validator""" - opts = SCons.Options.Options() - opts.Add(SCons.Options.PackageOption('test', 'test option help', '/default/path')) + """Test the PackageVariable validator""" + opts = SCons.Variables.Variables() + opts.Add(SCons.Variables.PackageVariable('test', 'test option help', '/default/path')) test = TestCmd.TestCmd(workdir='') test.write('exists', 'exists\n') @@ -115,6 +115,6 @@ class PackageOptionTestCase(unittest.TestCase): if __name__ == "__main__": - suite = unittest.makeSuite(PackageOptionTestCase, 'test_') + suite = unittest.makeSuite(PackageVariableTestCase, 'test_') if not unittest.TextTestRunner().run(suite).wasSuccessful(): sys.exit(1) diff --git a/src/engine/SCons/Options/PathOption.py b/src/engine/SCons/Variables/PathVariable.py index 995eeb9..e31de96 100644 --- a/src/engine/SCons/Options/PathOption.py +++ b/src/engine/SCons/Variables/PathVariable.py @@ -1,10 +1,10 @@ -"""SCons.Options.PathOption +"""SCons.Variables.PathVariable This file defines an option type for SCons implementing path settings. To be used whenever a a user-specified path override should be allowed. -Arguments to PathOption are: +Arguments to PathVariable are: option-name = name of this option on the command line (e.g. "prefix") option-help = help string for option option-dflt = default value for this option @@ -30,16 +30,16 @@ Usage example: Examples: prefix=/usr/local - opts = Options() + opts = Variables() - opts = Options() - opts.Add(PathOption('qtdir', + opts = Variables() + opts.Add(PathVariable('qtdir', 'where the root of Qt is installed', qtdir, PathIsDir)) - opts.Add(PathOption('qt_includes', + opts.Add(PathVariable('qt_includes', 'where the Qt includes are installed', '$qtdir/includes', PathIsDirCreate)) - opts.Add(PathOption('qt_libraries', + opts.Add(PathVariable('qt_libraries', 'where the Qt library is installed', '$qtdir/lib')) @@ -70,12 +70,14 @@ Usage example: __revision__ = "__FILE__ __REVISION__ __DATE__ __DEVELOPER__" +__all__ = ['PathVariable',] + import os import os.path import SCons.Errors -class _PathOptionClass: +class _PathVariableClass: def PathAccept(self, key, val, env): """Accepts any path, no checking done.""" @@ -136,4 +138,4 @@ class _PathOptionClass: return (key, '%s ( /path/to/%s )' % (help, key), default, validator, None) -PathOption = _PathOptionClass() +PathVariable = _PathVariableClass() diff --git a/src/engine/SCons/Options/PathOptionTests.py b/src/engine/SCons/Variables/PathVariableTests.py index 22e79ae..980cc5e 100644 --- a/src/engine/SCons/Options/PathOptionTests.py +++ b/src/engine/SCons/Variables/PathVariableTests.py @@ -28,15 +28,15 @@ import sys import unittest import SCons.Errors -import SCons.Options +import SCons.Variables import TestCmd -class PathOptionTestCase(unittest.TestCase): - def test_PathOption(self): - """Test PathOption creation""" - opts = SCons.Options.Options() - opts.Add(SCons.Options.PathOption('test', +class PathVariableTestCase(unittest.TestCase): + def test_PathVariable(self): + """Test PathVariable creation""" + opts = SCons.Variables.Variables() + opts.Add(SCons.Variables.PathVariable('test', 'test option help', '/default/path')) @@ -49,11 +49,11 @@ class PathOptionTestCase(unittest.TestCase): def test_PathExists(self): """Test the PathExists validator""" - opts = SCons.Options.Options() - opts.Add(SCons.Options.PathOption('test', + opts = SCons.Variables.Variables() + opts.Add(SCons.Variables.PathVariable('test', 'test option help', '/default/path', - SCons.Options.PathOption.PathExists)) + SCons.Variables.PathVariable.PathExists)) test = TestCmd.TestCmd(workdir='') test.write('exists', 'exists\n') @@ -72,11 +72,11 @@ class PathOptionTestCase(unittest.TestCase): def test_PathIsDir(self): """Test the PathIsDir validator""" - opts = SCons.Options.Options() - opts.Add(SCons.Options.PathOption('test', + opts = SCons.Variables.Variables() + opts.Add(SCons.Variables.PathVariable('test', 'test option help', '/default/path', - SCons.Options.PathOption.PathIsDir)) + SCons.Variables.PathVariable.PathIsDir)) test = TestCmd.TestCmd(workdir='') test.subdir('dir') @@ -104,11 +104,11 @@ class PathOptionTestCase(unittest.TestCase): def test_PathIsDirCreate(self): """Test the PathIsDirCreate validator""" - opts = SCons.Options.Options() - opts.Add(SCons.Options.PathOption('test', + opts = SCons.Variables.Variables() + opts.Add(SCons.Variables.PathVariable('test', 'test option help', '/default/path', - SCons.Options.PathOption.PathIsDirCreate)) + SCons.Variables.PathVariable.PathIsDirCreate)) test = TestCmd.TestCmd(workdir='') test.write('file', "file\n") @@ -129,11 +129,11 @@ class PathOptionTestCase(unittest.TestCase): def test_PathIsFile(self): """Test the PathIsFile validator""" - opts = SCons.Options.Options() - opts.Add(SCons.Options.PathOption('test', + opts = SCons.Variables.Variables() + opts.Add(SCons.Variables.PathVariable('test', 'test option help', '/default/path', - SCons.Options.PathOption.PathIsFile)) + SCons.Variables.PathVariable.PathIsFile)) test = TestCmd.TestCmd(workdir='') test.subdir('dir') @@ -161,11 +161,11 @@ class PathOptionTestCase(unittest.TestCase): def test_PathAccept(self): """Test the PathAccept validator""" - opts = SCons.Options.Options() - opts.Add(SCons.Options.PathOption('test', + opts = SCons.Variables.Variables() + opts.Add(SCons.Variables.PathVariable('test', 'test option help', '/default/path', - SCons.Options.PathOption.PathAccept)) + SCons.Variables.PathVariable.PathAccept)) test = TestCmd.TestCmd(workdir='') test.subdir('dir') @@ -182,9 +182,9 @@ class PathOptionTestCase(unittest.TestCase): o.validator('X', dne, {}) def test_validator(self): - """Test the PathOption validator argument""" - opts = SCons.Options.Options() - opts.Add(SCons.Options.PathOption('test', + """Test the PathVariable validator argument""" + opts = SCons.Variables.Variables() + opts.Add(SCons.Variables.PathVariable('test', 'test option help', '/default/path')) @@ -207,8 +207,8 @@ class PathOptionTestCase(unittest.TestCase): def my_validator(key, val, env): raise Exception, "my_validator() got called for %s, %s!" % (key, val) - opts = SCons.Options.Options() - opts.Add(SCons.Options.PathOption('test2', + opts = SCons.Variables.Variables() + opts.Add(SCons.Variables.PathVariable('test2', 'more help', '/default/path/again', my_validator)) @@ -226,6 +226,6 @@ class PathOptionTestCase(unittest.TestCase): if __name__ == "__main__": - suite = unittest.makeSuite(PathOptionTestCase, 'test_') + suite = unittest.makeSuite(PathVariableTestCase, 'test_') if not unittest.TextTestRunner().run(suite).wasSuccessful(): sys.exit(1) diff --git a/src/engine/SCons/Options/OptionsTests.py b/src/engine/SCons/Variables/VariablesTests.py index 5ec9d7a..cd6ab1d 100644 --- a/src/engine/SCons/Options/OptionsTests.py +++ b/src/engine/SCons/Variables/VariablesTests.py @@ -28,7 +28,7 @@ import sys import unittest import TestSCons -import SCons.Options +import SCons.Variables import SCons.Subst import SCons.Warnings @@ -57,11 +57,11 @@ def checkSave(file, expected): execfile(file, gdict, ldict) assert expected == ldict, "%s\n...not equal to...\n%s" % (expected, ldict) -class OptionsTestCase(unittest.TestCase): +class VariablesTestCase(unittest.TestCase): def test_keys(self): - """Test the Options.keys() method""" - opts = SCons.Options.Options() + """Test the Variables.keys() method""" + opts = SCons.Variables.Variables() opts.Add('VAR1') opts.Add('VAR2', @@ -73,8 +73,8 @@ class OptionsTestCase(unittest.TestCase): assert keys == ['VAR1', 'VAR2'], keys def test_Add(self): - """Test adding to an Options object""" - opts = SCons.Options.Options() + """Test adding to a Variables object""" + opts = SCons.Variables.Variables() opts.Add('VAR') opts.Add('ANSWER', @@ -107,11 +107,11 @@ class OptionsTestCase(unittest.TestCase): test_it('foo-bar') test_it('foo.bar') - def test_AddOptions(self): - """Test adding a list of options to an Options object""" - opts = SCons.Options.Options() + def test_AddVariables(self): + """Test adding a list of options to a Variables object""" + opts = SCons.Variables.Variables() - opts.AddOptions(('VAR2',), + opts.AddVariables(('VAR2',), ('ANSWER2', 'THE answer to THE question', "42", @@ -137,7 +137,7 @@ class OptionsTestCase(unittest.TestCase): # Test that a default value is validated correctly. test = TestSCons.TestSCons() file = test.workpath('custom.py') - opts = SCons.Options.Options(file) + opts = SCons.Variables.Variables(file) opts.Add('ANSWER', 'THE answer to THE question', @@ -158,7 +158,7 @@ class OptionsTestCase(unittest.TestCase): test = TestSCons.TestSCons() file = test.workpath('custom.py') test.write('custom.py', 'ANSWER=54') - opts = SCons.Options.Options(file) + opts = SCons.Variables.Variables(file) opts.Add('ANSWER', 'THE answer to THE question', @@ -186,7 +186,7 @@ class OptionsTestCase(unittest.TestCase): test = TestSCons.TestSCons() file = test.workpath('custom.py') test.write('custom.py', 'ANSWER=42') - opts = SCons.Options.Options(file) + opts = SCons.Variables.Variables(file) opts.Add('ANSWER', 'THE answer to THE question', @@ -207,7 +207,7 @@ class OptionsTestCase(unittest.TestCase): test = TestSCons.TestSCons() file = test.workpath('custom.py') test.write('custom.py', 'ANSWER=10') - opts = SCons.Options.Options(file) + opts = SCons.Variables.Variables(file) opts.Add('ANSWER', 'THE answer to THE question', @@ -228,7 +228,7 @@ class OptionsTestCase(unittest.TestCase): test = TestSCons.TestSCons() file = test.workpath('custom.py') test.write('custom.py', 'ANSWER=10') - opts = SCons.Options.Options(file) + opts = SCons.Variables.Variables(file) opts.Add('ANSWER', 'THE answer to THE question', @@ -246,7 +246,7 @@ class OptionsTestCase(unittest.TestCase): # or args. test = TestSCons.TestSCons() file = test.workpath('custom.py') - opts = SCons.Options.Options(file) + opts = SCons.Variables.Variables(file) opts.Add('ANSWER', help='THE answer to THE question', @@ -259,7 +259,7 @@ class OptionsTestCase(unittest.TestCase): # Test that a default value of None is all right. test = TestSCons.TestSCons() file = test.workpath('custom.py') - opts = SCons.Options.Options(file) + opts = SCons.Variables.Variables(file) opts.Add('ANSWER', "This is the answer", @@ -278,7 +278,7 @@ class OptionsTestCase(unittest.TestCase): test = TestSCons.TestSCons() file = test.workpath('custom.py') test.write('custom.py', 'ANSWER=42') - opts = SCons.Options.Options(file, {'ANSWER':54}) + opts = SCons.Variables.Variables(file, {'ANSWER':54}) opts.Add('ANSWER', 'THE answer to THE question', @@ -298,7 +298,7 @@ class OptionsTestCase(unittest.TestCase): test = TestSCons.TestSCons() file = test.workpath('custom.py') test.write('custom.py', 'ANSWER=54') - opts = SCons.Options.Options(file, {'ANSWER':42}) + opts = SCons.Variables.Variables(file, {'ANSWER':42}) opts.Add('ANSWER', 'THE answer to THE question', @@ -315,7 +315,7 @@ class OptionsTestCase(unittest.TestCase): test = TestSCons.TestSCons() file = test.workpath('custom.py') test.write('custom.py', 'ANSWER=54') - opts = SCons.Options.Options(file, {'ANSWER':54}) + opts = SCons.Variables.Variables(file, {'ANSWER':54}) opts.Add('ANSWER', 'THE answer to THE question', @@ -328,11 +328,11 @@ class OptionsTestCase(unittest.TestCase): assert env['ANSWER'] == 54 def test_Save(self): - """Testing saving Options""" + """Testing saving Variables""" test = TestSCons.TestSCons() cache_file = test.workpath('cached.options') - opts = SCons.Options.Options() + opts = SCons.Variables.Variables() def bool_converter(val): if val in [1, 'y']: val = 1 @@ -387,7 +387,7 @@ class OptionsTestCase(unittest.TestCase): test = TestSCons.TestSCons() cache_file = test.workpath('cached.options') - opts = SCons.Options.Options() + opts = SCons.Variables.Variables() opts.Add('THIS_USED_TO_BREAK', 'An option to test', @@ -412,7 +412,7 @@ class OptionsTestCase(unittest.TestCase): def test_GenerateHelpText(self): """Test generating the default format help text""" - opts = SCons.Options.Options() + opts = SCons.Variables.Variables() opts.Add('ANSWER', 'THE answer to THE question', @@ -468,14 +468,14 @@ B: b - alpha test text = opts.GenerateHelpText(env, sort=cmp) assert text == expectAlpha, text - def test_FormatOptionHelpText(self): + def test_FormatVariableHelpText(self): """Test generating custom format help text""" - opts = SCons.Options.Options() + opts = SCons.Variables.Variables() def my_format(env, opt, help, default, actual, aliases): return '%s %s %s %s %s\n' % (opt, default, actual, help, aliases) - opts.FormatOptionHelpText = my_format + opts.FormatVariableHelpText = my_format opts.Add('ANSWER', 'THE answer to THE question', @@ -517,11 +517,11 @@ B 42 54 b - alpha test ['B'] -class UnknownOptionsTestCase(unittest.TestCase): +class UnknownVariablesTestCase(unittest.TestCase): def test_unknown(self): - """Test the UnknownOptions() method""" - opts = SCons.Options.Options() + """Test the UnknownVariables() method""" + opts = SCons.Variables.Variables() opts.Add('ANSWER', 'THE answer to THE question', @@ -535,7 +535,7 @@ class UnknownOptionsTestCase(unittest.TestCase): env = Environment() opts.Update(env, args) - r = opts.UnknownOptions() + r = opts.UnknownVariables() assert r == {'UNKNOWN' : 'unknown'}, r assert env['ANSWER'] == 'answer', env['ANSWER'] @@ -543,8 +543,8 @@ class UnknownOptionsTestCase(unittest.TestCase): if __name__ == "__main__": suite = unittest.TestSuite() - tclasses = [ OptionsTestCase, - UnknownOptionsTestCase ] + tclasses = [ VariablesTestCase, + UnknownVariablesTestCase ] for tclass in tclasses: names = unittest.getTestCaseNames(tclass, 'test_') suite.addTests(map(tclass, names)) diff --git a/src/engine/SCons/Options/__init__.py b/src/engine/SCons/Variables/__init__.py index 9389c7b..a86cd34 100644 --- a/src/engine/SCons/Options/__init__.py +++ b/src/engine/SCons/Variables/__init__.py @@ -1,6 +1,6 @@ -"""engine.SCons.Options +"""engine.SCons.Variables -This file defines the Options class that is used to add user-friendly +This file defines the Variables class that is used to add user-friendly customizable variables to an SCons build. """ @@ -40,14 +40,14 @@ import SCons.Errors import SCons.Util import SCons.Warnings -from BoolOption import BoolOption # okay -from EnumOption import EnumOption # okay -from ListOption import ListOption # naja -from PackageOption import PackageOption # naja -from PathOption import PathOption # okay +from BoolVariable import BoolVariable # okay +from EnumVariable import EnumVariable # okay +from ListVariable import ListVariable # naja +from PackageVariable import PackageVariable # naja +from PathVariable import PathVariable # okay -class Options: +class Variables: instance=None """ @@ -72,16 +72,16 @@ class Options: # create the singleton instance if is_global: - self=Options.instance + self=Variables.instance - if not Options.instance: - Options.instance=self + if not Variables.instance: + Variables.instance=self def _do_add(self, key, help="", default=None, validator=None, converter=None): - class Option: + class Variable: pass - option = Option() + option = Variable() # if we get a list or a tuple, we take the first element as the # option key and store the remaining in aliases. @@ -123,11 +123,11 @@ class Options: if not SCons.Util.is_String(key) or \ not SCons.Environment.is_valid_construction_var(key): - raise SCons.Errors.UserError, "Illegal Options.Add() key `%s'" % str(key) + raise SCons.Errors.UserError, "Illegal Variables.Add() key `%s'" % str(key) self._do_add(key, help, default, validator, converter) - def AddOptions(self, *optlist): + def AddVariables(self, *optlist): """ Add a list of options. @@ -135,7 +135,7 @@ class Options: to the underlying method for adding options. Example: - opt.AddOptions( + opt.AddVariables( ('debug', '', 0), ('CC', 'The C compiler'), ('VALIDATE', 'An option for testing validation', 'notset', @@ -213,7 +213,7 @@ class Options: if option.validator and values.has_key(option.key): option.validator(option.key, env.subst('${%s}'%option.key), env) - def UnknownOptions(self): + def UnknownVariables(self): """ Returns any options in the specified arguments lists that were not known, declared options in this object. @@ -288,7 +288,7 @@ class Options: actual = env.subst('${%s}' % opt.key) else: actual = None - return self.FormatOptionHelpText(env, opt.key, opt.help, opt.default, actual, opt.aliases) + return self.FormatVariableHelpText(env, opt.key, opt.help, opt.default, actual, opt.aliases) lines = filter(None, map(format, options)) return string.join(lines, '') @@ -296,7 +296,7 @@ class Options: format = '\n%s: %s\n default: %s\n actual: %s\n' format_ = '\n%s: %s\n default: %s\n actual: %s\n aliases: %s\n' - def FormatOptionHelpText(self, env, key, help, default, actual, aliases=[]): + def FormatVariableHelpText(self, env, key, help, default, actual, aliases=[]): # Don't display the key name itself as an alias. aliases = filter(lambda a, k=key: a != k, aliases) if len(aliases)==0: diff --git a/src/engine/SCons/compat/__init__.py b/src/engine/SCons/compat/__init__.py index 1d36dbc..20fb8ed 100644 --- a/src/engine/SCons/compat/__init__.py +++ b/src/engine/SCons/compat/__init__.py @@ -138,7 +138,12 @@ except AttributeError: return result fnmatch.filter = filter del filter - + +try: + import itertools +except ImportError: + # Pre-2.3 Python has no itertools module. + import_as('_scons_itertools', 'itertools') # If we need the compatibility version of textwrap, it must be imported # before optparse, which uses it. diff --git a/src/engine/SCons/compat/_scons_itertools.py b/src/engine/SCons/compat/_scons_itertools.py new file mode 100644 index 0000000..9720daf --- /dev/null +++ b/src/engine/SCons/compat/_scons_itertools.py @@ -0,0 +1,118 @@ +# +# __COPYRIGHT__ +# +# Permission is hereby granted, free of charge, to any person obtaining +# a copy of this software and associated documentation files (the +# "Software"), to deal in the Software without restriction, including +# without limitation the rights to use, copy, modify, merge, publish, +# distribute, sublicense, and/or sell copies of the Software, and to +# permit persons to whom the Software is furnished to do so, subject to +# the following conditions: +# +# The above copyright notice and this permission notice shall be included +# in all copies or substantial portions of the Software. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY +# KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE +# WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND +# NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE +# LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION +# OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION +# WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. +# + +__revision__ = "__FILE__ __REVISION__ __DATE__ __DEVELOPER__" + +__doc__ = """ +Implementations of itertools functions for Python versions that don't +have iterators. + +These implement the functions by creating the entire list, not returning +it element-by-element as the real itertools functions do. This means +that early Python versions won't get the performance benefit of using +the itertools, but we can still use them so the later Python versions +do get the advantages of using iterators. + +Because we return the entire list, we intentionally do not implement the +itertools functions that "return" infinitely-long lists: the count(), +cycle() and repeat() functions. Other functions below have remained +unimplemented simply because they aren't being used (yet) and it wasn't +obvious how to do it. Or, conversely, we only implemented those functions +that *were* easy to implement (mostly because the Python documentation +contained examples of equivalent code). + +Note that these do not have independent unit tests, so it's possible +that there are bugs. +""" + +def chain(*iterables): + result = [] + for x in iterables: + result.extend(list(x)) + return result + +def count(n=0): + # returns infinite length, should not be supported + raise NotImplementedError + +def cycle(iterable): + # returns infinite length, should not be supported + raise NotImplementedError + +def dropwhile(predicate, iterable): + result = [] + for x in iterable: + if not predicate(x): + result.append(x) + break + result.extend(iterable) + return result + +def groupby(iterable, *args): + raise NotImplementedError + +def ifilter(predicate, iterable): + result = [] + if predicate is None: + predicate = bool + for x in iterable: + if predicate(x): + result.append(x) + return result + +def ifilterfalse(predicate, iterable): + result = [] + if predicate is None: + predicate = bool + for x in iterable: + if not predicate(x): + result.append(x) + return result + +def imap(function, *iterables): + return apply(map, (function,) + tuple(iterables)) + +def islice(*args, **kw): + raise NotImplementedError + +def izip(*iterables): + return apply(zip, iterables) + +def repeat(*args, **kw): + # returns infinite length, should not be supported + raise NotImplementedError + +def starmap(*args, **kw): + raise NotImplementedError + +def takewhile(predicate, iterable): + result = [] + for x in iterable: + if predicate(x): + result.append(x) + else: + break + return result + +def tee(*args, **kw): + raise NotImplementedError diff --git a/src/engine/SCons/compat/_scons_sets15.py b/src/engine/SCons/compat/_scons_sets15.py index b3d0bb3..1fe5a4f 100644 --- a/src/engine/SCons/compat/_scons_sets15.py +++ b/src/engine/SCons/compat/_scons_sets15.py @@ -26,10 +26,11 @@ class Set: if seq: for elem in seq: if elem not in self.elems: + hash(elem) self.elems.append(elem) def __str__(self): - return "{%s}" % string.join(map(str, self.elems), ", ") + return "set([%s])" % string.join(map(str, self.elems), ", ") def copy(self): @@ -56,6 +57,7 @@ class Set: def add(self, elem): """Add one element to the set.""" if elem not in self.elems: + hash(elem) self.elems.append(elem) def remove(self, elem): @@ -157,3 +159,12 @@ class Set: return 0 else: return len(self - other) == 0 + + def __cmp__(self, other): + """Returns 1 if the sets are equal.""" + if self.__lt__(other): + return -1 + elif other.__lt__(self): + return 1 + else: + return 0 diff --git a/src/engine/SCons/compat/_scons_textwrap.py b/src/engine/SCons/compat/_scons_textwrap.py index 1e0e411..72ed9b9 100644 --- a/src/engine/SCons/compat/_scons_textwrap.py +++ b/src/engine/SCons/compat/_scons_textwrap.py @@ -90,13 +90,16 @@ class TextWrapper: # splits into # Hello/ /there/ /--/ /you/ /goof-/ball,/ /use/ /the/ /-b/ /option! # (after stripping out empty strings). - wordsep_re = re.compile(r'(\s+|' # any whitespace - r'-*\w{2,}-(?=\w{2,}))') # hyphenated words - # Earlier Python's don't have the (?<= - # negative look-behind assertion. It doesn't - # matter for the simple input SCons is going to - # give it, so just comment it out. - #r'(?<=[\w\!\"\'\&\.\,\?])-{2,}(?=\w))') # em-dash + try: + wordsep_re = re.compile(r'(\s+|' # any whitespace + r'[^\s\w]*\w{2,}-(?=\w{2,})|' # hyphenated words + r'(?<=[\w\!\"\'\&\.\,\?])-{2,}(?=\w))') # em-dash + except re.error: + # Pre-2.0 Python versions don't have the (?<= negative look-behind + # assertion. It mostly doesn't matter for the simple input + # SCons is going to give it, so just leave it out. + wordsep_re = re.compile(r'(\s+|' # any whitespace + r'-*\w{2,}-(?=\w{2,}))') # hyphenated words # XXX will there be a locale-or-charset-aware version of # string.lowercase in 2.3? diff --git a/src/engine/SCons/dblite.py b/src/engine/SCons/dblite.py index 36840c3..437f05a 100644 --- a/src/engine/SCons/dblite.py +++ b/src/engine/SCons/dblite.py @@ -5,12 +5,9 @@ import cPickle import time import shutil import os -import os.path import types import __builtin__ -_open = __builtin__.open # avoid name clash - keep_all_files = 00000 ignore_corrupt_dbfiles = 0 @@ -35,6 +32,24 @@ tmp_suffix = '.tmp' class dblite: + # Squirrel away references to the functions in various modules + # that we'll use when our __del__() method calls our sync() method + # during shutdown. We might get destroyed when Python is in the midst + # of tearing down the different modules we import in an essentially + # arbitrary order, and some of the various modules's global attributes + # may already be wiped out from under us. + # + # See the discussion at: + # http://mail.python.org/pipermail/python-bugs-list/2003-March/016877.html + + _open = __builtin__.open + _cPickle_dump = cPickle.dump + _os_chmod = os.chmod + _os_rename = os.rename + _os_unlink = os.unlink + _shutil_copyfile = shutil.copyfile + _time_time = time.time + def __init__(self, file_base_name, flag, mode): assert flag in (None, "r", "w", "c", "n") if (flag is None): flag = "r" @@ -51,14 +66,14 @@ class dblite: self._dict = {} self._needs_sync = 00000 if (self._flag == "n"): - _open(self._file_name, "wb", self._mode) + self._open(self._file_name, "wb", self._mode) else: try: - f = _open(self._file_name, "rb") + f = self._open(self._file_name, "rb") except IOError, e: if (self._flag != "c"): raise e - _open(self._file_name, "wb", self._mode) + self._open(self._file_name, "wb", self._mode) else: p = f.read() if (len(p) > 0): @@ -75,8 +90,8 @@ class dblite: def sync(self): self._check_writable() - f = _open(self._tmp_name, "wb", self._mode) - cPickle.dump(self._dict, f, 1) + f = self._open(self._tmp_name, "wb", self._mode) + self._cPickle_dump(self._dict, f, 1) f.close() # Windows doesn't allow renaming if the file exists, so unlink # it first, chmod'ing it to make sure we can do so. On UNIX, we @@ -84,15 +99,15 @@ class dblite: # (e.g. from a previous run as root). We should still be able to # unlink() the file if the directory's writable, though, so ignore # any OSError exception thrown by the chmod() call. - try: os.chmod(self._file_name, 0777) + try: self._os_chmod(self._file_name, 0777) except OSError: pass - os.unlink(self._file_name) - os.rename(self._tmp_name, self._file_name) + self._os_unlink(self._file_name) + self._os_rename(self._tmp_name, self._file_name) self._needs_sync = 00000 if (keep_all_files): - shutil.copyfile( + self._shutil_copyfile( self._file_name, - self._file_name + "_" + str(int(time.time()))) + self._file_name + "_" + str(int(self._time_time()))) def _check_writable(self): if (self._flag == "r"): |