diff options
author | Steven Knight <knight@baldmt.com> | 2008-04-17 06:35:51 (GMT) |
---|---|---|
committer | Steven Knight <knight@baldmt.com> | 2008-04-17 06:35:51 (GMT) |
commit | 960cd6ed58826dd88163103ea721d5a7b4d2c3ca (patch) | |
tree | 340da0ff0ef5002982fcfd5ee2a2a5e34a4bf5e0 /src/engine/SCons/Taskmaster.py | |
parent | f08434b5d652af0b471ac4c441de7d5476dff054 (diff) | |
download | SCons-960cd6ed58826dd88163103ea721d5a7b4d2c3ca.zip SCons-960cd6ed58826dd88163103ea721d5a7b4d2c3ca.tar.gz SCons-960cd6ed58826dd88163103ea721d5a7b4d2c3ca.tar.bz2 |
Merged revisions 2725-2865 via svnmerge from
http://scons.tigris.org/svn/scons/branches/core
........
r2731 | stevenknight | 2008-04-01 09:22:37 -0700 (Tue, 01 Apr 2008) | 4 lines
Fix test/Interactive/version.py, which still had the old hard-coded
Copyright string, by moving the test copy of the Copyright string
into QMTest/TestSCons.py, where it's available to all test scripts.
........
r2732 | stevenknight | 2008-04-02 13:28:32 -0700 (Wed, 02 Apr 2008) | 3 lines
Issues 317, 323, 329: Ignore the -d, -e and -w options, which we're
not going to implement.
........
r2734 | stevenknight | 2008-04-05 07:50:49 -0700 (Sat, 05 Apr 2008) | 2 lines
Fix bootstrap.py for pre-2.3 versions of Python, which don't have __file__.
........
r2735 | stevenknight | 2008-04-05 07:54:44 -0700 (Sat, 05 Apr 2008) | 2 lines
Fix the non-breaking-hyphen regular expression(s) for Python 1.5.
........
r2736 | garyo | 2008-04-05 19:15:52 -0700 (Sat, 05 Apr 2008) | 1 line
Added Users Guide section for AddMethod.
........
r2737 | stevenknight | 2008-04-08 09:43:31 -0700 (Tue, 08 Apr 2008) | 3 lines
Don't raise strings as exceptions in the exception-handling unit tests.
That's now deprecated Python behavior.
........
r2738 | stevenknight | 2008-04-08 10:36:00 -0700 (Tue, 08 Apr 2008) | 5 lines
Address a serious inefficiency in Java builds by moving the pre-build
check for whether source files exist from the Node class to the Executor
class, so we only have to perform the check once when building a whole
bunch of targets from a whole bunch of sources.
........
r2739 | stevenknight | 2008-04-08 13:53:44 -0700 (Tue, 08 Apr 2008) | 2 lines
Add a backwards-compatibility version of the "itertools" module.
........
r2740 | stevenknight | 2008-04-08 14:12:05 -0700 (Tue, 08 Apr 2008) | 2 lines
Issue 1961: speed up SCons.Util.to_String*() functions. (Benoit Belley)
........
r2741 | stevenknight | 2008-04-08 14:15:36 -0700 (Tue, 08 Apr 2008) | 2 lines
Use the itertools.izip() method for some key Node.FS methods. (Benoit Belley)
........
r2742 | stevenknight | 2008-04-08 14:35:35 -0700 (Tue, 08 Apr 2008) | 2 lines
Issues 1961: use izip() instead of zip() where possible (Benoit Belley)
........
r2743 | stevenknight | 2008-04-08 14:48:08 -0700 (Tue, 08 Apr 2008) | 3 lines
Issue 1961: more efficient get_contents() implementation for Python
function Actions. (Benoit Belley)
........
r2744 | stevenknight | 2008-04-08 17:55:30 -0700 (Tue, 08 Apr 2008) | 3 lines
Issue 1961: make SCons.Node.* state variables global in Taskmaster.py so
we avoid unneceesary attribute fetches. (Benoit Belley)
........
r2745 | stevenknight | 2008-04-08 20:40:31 -0700 (Tue, 08 Apr 2008) | 2 lines
Issue 1961: Optimize the code in Node.get_binfo(). (Benoit Belley)
........
r2746 | stevenknight | 2008-04-08 21:05:17 -0700 (Tue, 08 Apr 2008) | 3 lines
Issue 1961: Enhance the backwards-compatibility sets() module with the
ability to compare sets. (Benoit Belley)
........
r2747 | stevenknight | 2008-04-08 22:11:20 -0700 (Tue, 08 Apr 2008) | 2 lines
Issue 1961: Optimize Executor.scan(). (Benoit Belley)
........
r2748 | stevenknight | 2008-04-08 22:17:55 -0700 (Tue, 08 Apr 2008) | 3 lines
Python 1.5 compatibility: use for p in paths.keys() for dictionaries.
(Benoit Belley)
........
r2749 | stevenknight | 2008-04-08 22:54:02 -0700 (Tue, 08 Apr 2008) | 10 lines
Issue 1961: additional build optimizations:
-- Make taskmastertrace output more useful and readable.
-- Move dependency cycle checking to avoid re-doing it.
-- Have Nodes use sets, not dictionaries, to track various things.
-- Eliminate intermediate function calls from fetching Node children.
-- Add a Task.needs_execute() method to avoid doing that check over and
over as part of needs_execute().
-- Remove the unused Node.found_includes attribute.
(Benoit Belley)
........
r2750 | stevenknight | 2008-04-09 14:47:44 -0700 (Wed, 09 Apr 2008) | 4 lines
Make target Java .class files depend *only* on the input .java files as
their sources when determining if they require rebuilding. This eliminates
O(NxM) checking for every single edge in a big, overly-connected DAG mesh.
........
r2751 | stevenknight | 2008-04-09 16:58:41 -0700 (Wed, 09 Apr 2008) | 2 lines
Remove 0.95 and 0.96* release notes.
........
r2752 | stevenknight | 2008-04-10 02:24:50 -0700 (Thu, 10 Apr 2008) | 2 lines
Issue 1956: Fix --debug=stree printing its tree twice. (Benoit Belley)
........
r2753 | stevenknight | 2008-04-10 02:33:28 -0700 (Thu, 10 Apr 2008) | 2 lines
Issue 1896: Add support for the GDC D language compiler. (Matthew Wesley)
........
r2754 | stevenknight | 2008-04-10 02:39:24 -0700 (Thu, 10 Apr 2008) | 2 lines
Fix tabs.
........
r2755 | stevenknight | 2008-04-10 02:41:50 -0700 (Thu, 10 Apr 2008) | 3 lines
Issue 1964: Fix passing variable names in a list to Return() (as
already documented in the man page!) (Mike Wake)
........
r2756 | stevenknight | 2008-04-10 02:55:40 -0700 (Thu, 10 Apr 2008) | 2 lines
Support the ability to download 2.6 candidate releases (e.g. 2.6a2).
........
r2757 | stevenknight | 2008-04-10 02:58:35 -0700 (Thu, 10 Apr 2008) | 3 lines
Issue 1669: Fix the ability to use LoadableModule() under MinGW.
(Johan Boule)
........
r2758 | stevenknight | 2008-04-10 03:03:15 -0700 (Thu, 10 Apr 2008) | 3 lines
Update the test/Interactive/tree.py script for Benoit's fix to
remove duplicate tree printing.
........
r2759 | stevenknight | 2008-04-10 06:43:44 -0700 (Thu, 10 Apr 2008) | 3 lines
Fix Tool/dmd.py when no D compiler is installed -- we don't want to
search for a path if the result is None.
........
r2760 | GregNoel | 2008-04-10 15:30:34 -0700 (Thu, 10 Apr 2008) | 1 line
Issue 2009: separate Debug.caller() by functionality
........
r2761 | stevenknight | 2008-04-11 04:47:25 -0700 (Fri, 11 Apr 2008) | 3 lines
Issue 1882: Add the scons.bat directory to %PATH% so it can find python.exe.
(Anatoly Techtonik)
........
r2762 | stevenknight | 2008-04-11 09:15:22 -0700 (Fri, 11 Apr 2008) | 4 lines
Issues 1835,1901: fix the ability to list a source file multiple
times for a target by making sure we only store unique entries in the
.sconsign file.
........
r2763 | stevenknight | 2008-04-11 10:58:26 -0700 (Fri, 11 Apr 2008) | 4 lines
Issue 1882: Fix earlier patch to scons.bat by adding ~dp0;~dp0.. to
the front of %PATH%, and only executing endlocal on NT-based systems.
(Anatoly Techtonik)
........
r2764 | stevenknight | 2008-04-11 13:06:29 -0700 (Fri, 11 Apr 2008) | 4 lines
Add a Variables object and {Bool,Envum,List,Package,Path}Variable()
functions as a first step towards eventually deprecating the
Options object and {Bool,Envum,List,Package,Path}Option() functions.
........
r2765 | stevenknight | 2008-04-11 18:13:53 -0700 (Fri, 11 Apr 2008) | 3 lines
Issue 1962: Capture a test case for ListActions that contain a
command-line string containing unicode, and Python FunctionAction.
........
r2766 | stevenknight | 2008-04-11 22:03:14 -0700 (Fri, 11 Apr 2008) | 3 lines
Issue 1933: expect .py files generated by the SWIG -python option
to be in the same (sub)directory as the target.
........
r2767 | stevenknight | 2008-04-12 06:41:57 -0700 (Sat, 12 Apr 2008) | 2 lines
Remove the SCons.Options package in favor of the new SCons.Variables package.
........
r2768 | stevenknight | 2008-04-12 13:33:52 -0700 (Sat, 12 Apr 2008) | 5 lines
Issue 1971: Move the incorporation of $CCFLAGS and $SHCCFLAGS directly
into the C++ command lines (${SHCXX,CXX}COM) instead of through indirect
expansion of $CXXFLAGS and $SHCXXFLAGS. This requires removing -fPIC
from the default setting of $SHCXXFLAGS under the GNU toolchain.
........
r2769 | stevenknight | 2008-04-13 07:01:27 -0700 (Sun, 13 Apr 2008) | 3 lines
Fix __all__ definitions in the Variables/*Variable.py file that were
cause epydoc to blow up when generating developer documentation.
........
r2770 | stevenknight | 2008-04-13 11:47:49 -0700 (Sun, 13 Apr 2008) | 3 lines
Add variables= keyword argument to Environment() creation as a first
step towards deprecating options=.
........
r2771 | stevenknight | 2008-04-13 11:54:19 -0700 (Sun, 13 Apr 2008) | 2 lines
Correct underscore-instead-of-hyphen misspellings in option names.
........
r2772 | bdbaddog | 2008-04-13 17:39:21 -0700 (Sun, 13 Apr 2008) | 3 lines
Changes to fix 15 tests which were failing on cygwin.
........
r2773 | GregNoel | 2008-04-13 22:31:07 -0700 (Sun, 13 Apr 2008) | 1 line
Various fixes to tests on Darwin using 1.5.2
........
r2774 | stevenknight | 2008-04-14 15:00:44 -0700 (Mon, 14 Apr 2008) | 2 lines
Python 1.5.2 fix in new test/SWIG/subdir.py script (use the -classic option).
........
r2775 | belley | 2008-04-14 18:02:40 -0700 (Mon, 14 Apr 2008) | 32 lines
Dont use KeyboardInterrupt to stop a build! [Issue 1907]
SCons would often hang after pressing Ctrl-C. I started investigating and I
realized that most of the Python libraries are not really safe with respect to
asynchronous exceptions. Although,there are enough try/finally blocks to handle
exceptions thrown synchronously by the library code, the Python libraries are
not always protected against exceptions being thrown asynchronously, such as a
KeyboardInterrupt being thrown at a completely random location.
For example, the function Queue.empty() does not protect its mutex with a
try/finally block. If the KeyboardInterrupt exception gets thrown while the
mutex is held, any further attempt to access the Queue will lead to dead-lock
(explaining why SCons hangs sometimes after pressing CTRL-C). Even the
threading.Condition condition variables are not async-exception safe. It
therefore seems a lost battle to try to stop a build by raising an exception.
Instead, I have implemented a signal handler that tells the Jobs (and its
associated Taskmaster) to stop the build. I have been careful to wait after the
.sconsign file has been written back to re-install the default SIGINT signal
handler that raises a KeyboardInterrupt exception.
This patch is submitted against changeset 2773 of branches/core. The
regression test suite has been run on RHEL4 using Pyhon 2.5.1 and
1.5.2. My team has been using an SCons build with this patch for a
while now on Windows, Linux and OSX.
See:
http://scons.tigris.org/issues/show_bug.cgi?id=1907
Benoit Belley
........
r2777 | cournape | 2008-04-14 20:11:56 -0700 (Mon, 14 Apr 2008) | 3 lines
Initialized merge tracking via "svnmerge" with revisions "1-2776" from
http://scons.tigris.org/svn/scons/branches/fortran_refactor
........
r2788 | stevenknight | 2008-04-14 22:09:27 -0700 (Mon, 14 Apr 2008) | 2 lines
Fix the print the "script" line in the --version output.
........
r2789 | stevenknight | 2008-04-14 22:18:27 -0700 (Mon, 14 Apr 2008) | 2 lines
Add a __COPYRIGHT__ line.
........
r2790 | stevenknight | 2008-04-14 22:20:39 -0700 (Mon, 14 Apr 2008) | 3 lines
Issue 2008: in checkpoint releases, use a '.' to separate (e.g.) 0.98.0
from 0d20080414.
........
r2817 | belley | 2008-04-15 06:44:21 -0700 (Tue, 15 Apr 2008) | 13 lines
TestSCons.up_to_date() should use match_re_dotall
I changed TestSCons.up_to_date() to use match_re_dotall instead of
match_exact. This is necessary so that I can call up_to_date() with
the TestSCons.deprecated_python_expr error message in one of my test.
Note that TestSCons.not_up_to_date() is already using match_re_dotall.
Ran the test suite on both Python 2.5.1 and 1.5.2.
Benoit
........
r2818 | belley | 2008-04-15 12:10:52 -0700 (Tue, 15 Apr 2008) | 10 lines
Improved the multiple-parents.py test to also tests the following
cases:
d) Some children are ignored
e) Some children are pre-requesites
f) Some sources are missing
The test still passes. No extra bug were found.
........
r2850 | stevenknight | 2008-04-16 11:15:24 -0700 (Wed, 16 Apr 2008) | 4 lines
Fix problems with the __del__() method referencing other module functions
through global variables, which can get deleted out from under us at
shutdown.
........
r2851 | stevenknight | 2008-04-16 11:17:07 -0700 (Wed, 16 Apr 2008) | 4 lines
Fix use of --interactive with -u/-U/-D and VariantDir() by making the
method that cleans Node states between interactive commands aware of
the alter_targets() method that tells us about an associated VariantDir().
........
Diffstat (limited to 'src/engine/SCons/Taskmaster.py')
-rw-r--r-- | src/engine/SCons/Taskmaster.py | 323 |
1 files changed, 204 insertions, 119 deletions
diff --git a/src/engine/SCons/Taskmaster.py b/src/engine/SCons/Taskmaster.py index 66202dc..8a0fcf7 100644 --- a/src/engine/SCons/Taskmaster.py +++ b/src/engine/SCons/Taskmaster.py @@ -52,16 +52,22 @@ __revision__ = "__FILE__ __REVISION__ __DATE__ __DEVELOPER__" import SCons.compat +from itertools import chain import operator import string import sys import traceback -import SCons.Node import SCons.Errors +import SCons.Node StateString = SCons.Node.StateString - +NODE_NO_STATE = SCons.Node.no_state +NODE_PENDING = SCons.Node.pending +NODE_EXECUTING = SCons.Node.executing +NODE_UP_TO_DATE = SCons.Node.up_to_date +NODE_EXECUTED = SCons.Node.executed +NODE_FAILED = SCons.Node.failed # A subsystem for recording stats about how different Nodes are handled by @@ -165,6 +171,17 @@ class Task: self.display(self.tm.message) self.tm.message = None + # Let the targets take care of any necessary preparations. + # This includes verifying that all of the necessary sources + # and dependencies exist, removing the target file(s), etc. + # + # As of April 2008, the get_executor().prepare() method makes + # sure that all of the aggregate sources necessary to build this + # Task's target(s) exist in one up-front check. The individual + # target t.prepare() methods check that each target's explicit + # or implicit dependencies exists, and also initialize the + # .sconsign info. + self.targets[0].get_executor().prepare() for t in self.targets: t.prepare() for s in t.side_effects: @@ -175,6 +192,17 @@ class Task: """ return self.node + def needs_execute(self): + """ + Called to determine whether the task's execute() method should + be run. + + This method allows one to skip the somethat costly execution + of the execute() method in a seperate thread. For example, + that would be unnecessary for up-to-date targets. + """ + return True + def execute(self): """ Called to execute the task. @@ -192,8 +220,6 @@ class Task: break if not everything_was_cached: self.targets[0].build() - except KeyboardInterrupt: - raise except SystemExit: exc_value = sys.exc_info()[1] raise SCons.Errors.ExplicitExit(self.targets[0], exc_value.code) @@ -212,10 +238,10 @@ class Task: the Node's callback methods. """ for t in self.targets: - if t.get_state() == SCons.Node.executing: + if t.get_state() == NODE_EXECUTING: for side_effect in t.side_effects: - side_effect.set_state(SCons.Node.no_state) - t.set_state(SCons.Node.executed) + side_effect.set_state(NODE_NO_STATE) + t.set_state(NODE_EXECUTED) def executed_with_callbacks(self): """ @@ -231,10 +257,10 @@ class Task: or not the target was an actual built target or a source Node. """ for t in self.targets: - if t.get_state() == SCons.Node.executing: + if t.get_state() == NODE_EXECUTING: for side_effect in t.side_effects: - side_effect.set_state(SCons.Node.no_state) - t.set_state(SCons.Node.executed) + side_effect.set_state(NODE_NO_STATE) + t.set_state(NODE_EXECUTED) t.built() t.visited() @@ -250,8 +276,12 @@ class Task: """ Explicit stop-the-build failure. """ - for t in self.targets: - t.set_state(SCons.Node.failed) + + # Invoke fail_continue() to clean-up the pending children + # list. + self.fail_continue() + + # Tell the taskmaster to not start any new tasks self.tm.stop() # We're stopping because of a build failure, but give the @@ -267,11 +297,43 @@ class Task: This sets failure status on the target nodes and all of their dependent parent nodes. """ + + pending_children = self.tm.pending_children + + to_visit = set() for t in self.targets: # Set failure state on all of the parents that were dependent # on this failed build. - def set_state(node): node.set_state(SCons.Node.failed) - t.call_for_all_waiting_parents(set_state) + if t.state != NODE_FAILED: + t.state = NODE_FAILED + parents = t.waiting_parents + to_visit = to_visit | parents + pending_children = pending_children - parents + + try: + while 1: + try: + node = to_visit.pop() + except AttributeError: + # Python 1.5.2 + if len(to_visit): + node = to_visit[0] + to_visit.remove(node) + else: + break + if node.state != NODE_FAILED: + node.state = NODE_FAILED + parents = node.waiting_parents + to_visit = to_visit | parents + pending_children = pending_children - parents + except KeyError: + # The container to_visit has been emptied. + pass + + # We have the stick back the pending_children list into the + # task master because the python 1.5.2 compatibility does not + # allow us to use in-place updates + self.tm.pending_children = pending_children def make_ready_all(self): """ @@ -282,9 +344,9 @@ class Task: """ self.out_of_date = self.targets[:] for t in self.targets: - t.disambiguate().set_state(SCons.Node.executing) + t.disambiguate().set_state(NODE_EXECUTING) for s in t.side_effects: - s.set_state(SCons.Node.executing) + s.set_state(NODE_EXECUTING) def make_ready_current(self): """ @@ -294,6 +356,7 @@ class Task: This is the default behavior for building only what's necessary. """ self.out_of_date = [] + needs_executing = False for t in self.targets: try: t.disambiguate().make_ready() @@ -301,13 +364,24 @@ class Task: (not t.always_build and t.is_up_to_date()) except EnvironmentError, e: raise SCons.Errors.BuildError(node=t, errstr=e.strerror, filename=e.filename) - if is_up_to_date: - t.set_state(SCons.Node.up_to_date) - else: + + if not is_up_to_date: self.out_of_date.append(t) - t.set_state(SCons.Node.executing) + needs_executing = True + + if needs_executing: + for t in self.targets: + t.set_state(NODE_EXECUTING) for s in t.side_effects: - s.set_state(SCons.Node.executing) + s.set_state(NODE_EXECUTING) + else: + for t in self.targets: + # We must invoke visited() to ensure that the node + # information has been computed before allowing the + # parent nodes to execute. (That could occur in a + # parallel build...) + t.visited() + t.set_state(NODE_UP_TO_DATE) make_ready = make_ready_current @@ -333,24 +407,25 @@ class Task: parents = {} for t in targets: - for p in t.waiting_parents.keys(): + for p in t.waiting_parents: parents[p] = parents.get(p, 0) + 1 for t in targets: for s in t.side_effects: - if s.get_state() == SCons.Node.executing: - s.set_state(SCons.Node.no_state) - for p in s.waiting_parents.keys(): - if not parents.has_key(p): - parents[p] = 1 - for p in s.waiting_s_e.keys(): + if s.get_state() == NODE_EXECUTING: + s.set_state(NODE_NO_STATE) + for p in s.waiting_parents: + parents[p] = parents.get(p, 0) + 1 + for p in s.waiting_s_e: if p.ref_count == 0: self.tm.candidates.append(p) + self.tm.pending_children.discard(p) for p, subtract in parents.items(): p.ref_count = p.ref_count - subtract if p.ref_count == 0: self.tm.candidates.append(p) + self.tm.pending_children.discard(p) for t in targets: t.postprocess() @@ -409,12 +484,15 @@ class Task: raise exc_type, exc_value, exc_traceback -def find_cycle(stack): - if stack[0] == stack[-1]: - return stack - for n in stack[-1].waiting_parents.keys(): +def find_cycle(stack, visited): + if stack[-1] in visited: + return None + visited.add(stack[-1]) + for n in stack[-1].waiting_parents: stack.append(n) - if find_cycle(stack): + if stack[0] == stack[-1]: + return stack + if find_cycle(stack, visited): return stack stack.pop() return None @@ -437,6 +515,8 @@ class Taskmaster: self.message = None self.trace = trace self.next_candidate = self.find_next_candidate + self.pending_children = set() + def find_next_candidate(self): """ @@ -505,10 +585,12 @@ class Taskmaster: self.ready_exc = None T = self.trace + if T: T.write('\nTaskmaster: Looking for a node to evaluate\n') while 1: node = self.next_candidate() if node is None: + if T: T.write('Taskmaster: No candidate anymore.\n\n') return None node = node.disambiguate() @@ -522,29 +604,27 @@ class Taskmaster: S.considered = S.considered + 1 else: S = None - - if T: T.write('Taskmaster: %s:' % repr(str(node))) - - # Skip this node if it has already been evaluated: - if state > SCons.Node.pending: + + if T: T.write('Taskmaster: Considering node <%-10s %s> and its children:\n' % + (StateString[node.get_state()], repr(str(node)))) + + if state == NODE_NO_STATE: + # Mark this node as being on the execution stack: + node.set_state(NODE_PENDING) + elif state > NODE_PENDING: + # Skip this node if it has already been evaluated: if S: S.already_handled = S.already_handled + 1 - if T: T.write(' already handled (%s)\n' % StateString[state]) + if T: T.write('Taskmaster: already handled (executed)\n') continue - # Mark this node as being on the execution stack: - node.set_state(SCons.Node.pending) - try: - children = node.children() + node.prerequisites + children = node.children() except SystemExit: exc_value = sys.exc_info()[1] e = SCons.Errors.ExplicitExit(node, exc_value.code) self.ready_exc = (SCons.Errors.ExplicitExit, e) - if T: T.write(' SystemExit\n') + if T: T.write('Taskmaster: SystemExit\n') return node - except KeyboardInterrupt: - if T: T.write(' KeyboardInterrupt\n') - raise except: # We had a problem just trying to figure out the # children (like a child couldn't be linked in to a @@ -552,70 +632,36 @@ class Taskmaster: # raise the exception when the Task is "executed." self.ready_exc = sys.exc_info() if S: S.problem = S.problem + 1 - if T: T.write(' exception\n') + if T: T.write('Taskmaster: exception while scanning children.\n') return node - if T and children: - c = map(str, children) - c.sort() - T.write(' children:\n %s\n ' % c) - - childstate = map(lambda N: (N, N.get_state()), children) - - # Detect dependency cycles: - pending_nodes = filter(lambda I: I[1] == SCons.Node.pending, childstate) - if pending_nodes: - for p in pending_nodes: - cycle = find_cycle([p[0], node]) - if cycle: - desc = "Dependency cycle: " + string.join(map(str, cycle), " -> ") - if T: T.write(' dependency cycle\n') - raise SCons.Errors.UserError, desc - - not_built = filter(lambda I: I[1] <= SCons.Node.executing, childstate) - if not_built: - # We're waiting on one or more derived targets that have - # not yet finished building. - - not_visited = filter(lambda I: not I[1], not_built) - if not_visited: - # Some of them haven't even been visited yet. - # Add them to the list so that on some next pass - # we can take a stab at evaluating them (or - # their children). - not_visited = map(lambda I: I[0], not_visited) - not_visited.reverse() - self.candidates.extend(self.order(not_visited)) - - n_b_nodes = map(lambda I: I[0], not_built) - - # Add this node to the waiting parents lists of anything - # we're waiting on, with a reference count so we can be - # put back on the list for re-evaluation when they've - # all finished. - map(lambda n, P=node: n.add_to_waiting_parents(P), n_b_nodes) - node.ref_count = len(set(n_b_nodes)) - - if S: S.not_built = S.not_built + 1 - if T: - c = map(str, n_b_nodes) - c.sort() - T.write(' waiting on unfinished children:\n %s\n' % c) - continue + children_not_visited = [] + children_pending = set() + children_not_ready = [] + children_failed = False - # Skip this node if it has side-effects that are - # currently being built: - side_effects = filter(lambda N: - N.get_state() == SCons.Node.executing, - node.side_effects) - if side_effects: - map(lambda n, P=node: n.add_to_waiting_s_e(P), side_effects) - if S: S.side_effects = S.side_effects + 1 - if T: - c = map(str, side_effects) - c.sort() - T.write(' waiting on side effects:\n %s\n' % c) - continue + for child in chain(children,node.prerequisites): + childstate = child.get_state() + + if T: T.write('Taskmaster: <%-10s %s>\n' % + (StateString[childstate], repr(str(child)))) + + if childstate == NODE_NO_STATE: + children_not_visited.append(child) + elif childstate == NODE_PENDING: + children_pending.add(child) + elif childstate == NODE_FAILED: + children_failed = True + + if childstate <= NODE_EXECUTING: + children_not_ready.append(child) + + + # These nodes have not even been visited yet. Add + # them to the list so that on some next pass we can + # take a stab at evaluating them (or their children). + children_not_visited.reverse() + self.candidates.extend(self.order(children_not_visited)) # Skip this node if any of its children have failed. # @@ -635,21 +681,47 @@ class Taskmaster: # Note that even if one of the children fails, we still # added the other children to the list of candidate nodes # to keep on building (--keep-going). - failed_children = filter(lambda I: I[1] == SCons.Node.failed, - childstate) - if failed_children: - node.set_state(SCons.Node.failed) + if children_failed: + node.set_state(NODE_FAILED) + if S: S.child_failed = S.child_failed + 1 - if T: - c = map(lambda I: str(I[0]), failed_children) - c.sort() - T.write(' children failed:\n %s\n' % c) + if T: T.write('Taskmaster:****** <%-10s %s>\n' % + (StateString[node.get_state()], repr(str(node)))) + continue + + if children_not_ready: + for child in children_not_ready: + # We're waiting on one or more derived targets + # that have not yet finished building. + if S: S.not_built = S.not_built + 1 + + # Add this node to the waiting parents lists of + # anything we're waiting on, with a reference + # count so we can be put back on the list for + # re-evaluation when they've all finished. + node.ref_count = node.ref_count + child.add_to_waiting_parents(node) + + self.pending_children = self.pending_children | children_pending + + continue + + # Skip this node if it has side-effects that are + # currently being built: + wait_side_effects = False + for se in node.side_effects: + if se.get_state() == NODE_EXECUTING: + se.add_to_waiting_s_e(node) + wait_side_effects = True + + if wait_side_effects: + if S: S.side_effects = S.side_effects + 1 continue # The default when we've gotten through all of the checks above: # this node is ready to be built. if S: S.build = S.build + 1 - if T: T.write(' evaluating %s\n' % node) + if T: T.write('Taskmaster: Evaluating <%-10s %s>\n' % + (StateString[node.get_state()], repr(str(node)))) return node return None @@ -671,8 +743,6 @@ class Taskmaster: task = self.tasker(self, tlist, node in self.original_top, node) try: task.make_ready() - except KeyboardInterrupt: - raise except: # We had a problem just trying to get this task ready (like # a child couldn't be linked in to a VariantDir when deciding @@ -692,3 +762,18 @@ class Taskmaster: Stops the current build completely. """ self.next_candidate = self.no_next_candidate + + def cleanup(self): + """ + Check for dependency cycles. + """ + if self.pending_children: + desc = 'Found dependency cycle(s):\n' + for node in self.pending_children: + cycle = find_cycle([node], set()) + if cycle: + desc = desc + " " + string.join(map(str, cycle), " -> ") + "\n" + else: + desc = desc + " Internal Error: no cycle found for node %s (%s)\n" % \ + (node, repr(node)) + raise SCons.Errors.UserError, desc |