Merged revisions 1675-1736 via svnmerge from

http://scons.tigris.org/svn/scons/branches/core

........
  r1689 | stevenknight | 2006-11-06 20:56:29 -0600 (Mon, 06 Nov 2006) | 1 line

  0.96.D483 - Merge changes for 0.96.93 packaging from the subsidiary branch.
........
  r1690 | stevenknight | 2006-11-06 20:59:30 -0600 (Mon, 06 Nov 2006) | 1 line

  0.96.D484 - Update HOWTO for releases. Fix name type in src/CHANGES.txt.
........
  r1691 | stevenknight | 2006-11-08 13:55:36 -0600 (Wed, 08 Nov 2006) | 1 line

  0.96.D485 - Fix MergeFlags() handling of None values. (John Pye)
........
  r1692 | stevenknight | 2006-11-08 17:15:05 -0600 (Wed, 08 Nov 2006) | 1 line

  0.96.D486 - Directly execute commands on Windows when possible. (Jay Kint)
........
  r1693 | stevenknight | 2006-11-08 18:54:49 -0600 (Wed, 08 Nov 2006) | 1 line

  0.96.D487 - Remove the semi-colon from the list of characters that determine when we use cmd
........
  r1694 | stevenknight | 2006-11-09 01:34:06 -0600 (Thu, 09 Nov 2006) | 1 line

  0.96.D488 - Pick up latex/bibtex 'Rerun to get citations correct' messages. (Dmitry Mikhin)
........
  r1695 | stevenknight | 2006-11-11 08:36:33 -0600 (Sat, 11 Nov 2006) | 1 line

  0.96.D489 - Back out the direct-execution-on-Windows change until we solve a corner case.
........
  r1696 | stevenknight | 2006-11-15 10:33:10 -0600 (Wed, 15 Nov 2006) | 1 line

  0.96.D490 - Fix the sconsign script when the .sconsign.dblite file is specified with its suf
........
  r1697 | stevenknight | 2006-11-18 10:45:50 -0600 (Sat, 18 Nov 2006) | 4 lines

  Complete move of test/sconsign/script.py to underneath test/sconsign/script/.
  (This got left out of the previous checkin due to an error in the
  script that resubmits Aegis changes to Subversion.)
........
  r1698 | stevenknight | 2006-11-18 11:05:26 -0600 (Sat, 18 Nov 2006) | 1 line

  0.96.D491 - Allow an Options converter to take the construction environment as a parameter.
........
  r1699 | stevenknight | 2006-11-30 15:34:37 -0600 (Thu, 30 Nov 2006) | 1 line

  0.96.D492 - Reverse the order in which we try the arguments Options converters, first a sing
........
  r1700 | stevenknight | 2006-11-30 16:03:09 -0600 (Thu, 30 Nov 2006) | 1 line

  0.96.D493 - Speed up rel_path() by avoiding recomputation of intermediate directory relative
........
  r1701 | stevenknight | 2006-11-30 16:14:16 -0600 (Thu, 30 Nov 2006) | 1 line

  0.96.D494 - More efficient get_suffix(): compute it once when we set the name.
........
  r1702 | stevenknight | 2006-11-30 16:22:55 -0600 (Thu, 30 Nov 2006) | 1 line

  0.96.D495 - Fix missing XML end tags.
........
  r1703 | stevenknight | 2006-11-30 17:15:25 -0600 (Thu, 30 Nov 2006) | 1 line

  0.96.D496 - Turn Memoizer into a simple counter for --debug=memoizer, not something that doe
........
  r1704 | stevenknight | 2006-11-30 20:30:50 -0600 (Thu, 30 Nov 2006) | 1 line

  0.96.D497 - Add the scons-time script, with doc and tests.
........
  r1705 | stevenknight | 2006-11-30 23:28:20 -0600 (Thu, 30 Nov 2006) | 1 line

  0.96.D498 - Update the copyright years string.
........
  r1706 | stevenknight | 2006-12-01 11:54:22 -0600 (Fri, 01 Dec 2006) | 1 line

  0.96.D499 - Fix _do_Lookup => _doLookup value-caching misspellings. (Ben Leslie)
........
  r1707 | stevenknight | 2006-12-01 12:03:46 -0600 (Fri, 01 Dec 2006) | 1 line

  0.96.D500 - Fix copyright test against debian build. (Walter Franzini)
........
  r1708 | stevenknight | 2006-12-01 14:23:29 -0600 (Fri, 01 Dec 2006) | 1 line

  0.96.D501 - Add #include lines for test portability. (Gary Oberbrunner)
........
  r1709 | stevenknight | 2006-12-01 14:51:12 -0600 (Fri, 01 Dec 2006) | 1 line

  0.96.D502 - Fix tests under Python versions with no profiler (pstats module).
........
  r1710 | stevenknight | 2006-12-01 20:04:49 -0600 (Fri, 01 Dec 2006) | 1 line

  0.96.D503 - Remove unnecessary os.path.normpath() calls. (Gary Oberbrunner)
........
  r1711 | stevenknight | 2006-12-01 20:34:31 -0600 (Fri, 01 Dec 2006) | 1 line

  0.96.D504 - Accomodate arbitray white space after a SWIG %module keyword. (Anonymous)
........
  r1712 | stevenknight | 2006-12-05 14:49:54 -0600 (Tue, 05 Dec 2006) | 1 line

  0.96.D506 - Cache substitutions of of Builder source suffixes. Use a new PathList module, and a refactor Node.FS.Rfindalldirs() method, to cache calculations of values like CPPPATH.
........
  r1713 | stevenknight | 2006-12-05 18:43:36 -0600 (Tue, 05 Dec 2006) | 1 line

  0.96.D507 - Use cached stat() values in diskchecks.
........
  r1714 | stevenknight | 2006-12-05 21:11:24 -0600 (Tue, 05 Dec 2006) | 1 line

  0.96.D508 - Fix Memoizer hit counts for methods memoizing simple values. Clean up the code for memoizing return values in a dictionary. Fix comments.
........
  r1715 | stevenknight | 2006-12-06 07:23:18 -0600 (Wed, 06 Dec 2006) | 1 line

  0.96.D369 - More efficient Node.FS.Dir.current() check. Fix some Windows test portability issues.
........
  r1716 | stevenknight | 2006-12-06 12:24:32 -0600 (Wed, 06 Dec 2006) | 2 lines

  Undo previous checkin (distributed incorrect Aegis change number).
........
  r1717 | stevenknight | 2006-12-06 12:34:53 -0600 (Wed, 06 Dec 2006) | 1 line

  0.96.D505 - Update ae-{cvs,svn}-ci for newer versions of aetar, and to not truncate descriptions.
........
  r1718 | stevenknight | 2006-12-07 23:01:41 -0600 (Thu, 07 Dec 2006) | 1 line

  0.96.D509 - Only look for mslink on Windows systems. (Sohail Somani)
........
  r1719 | stevenknight | 2006-12-07 23:18:33 -0600 (Thu, 07 Dec 2006) | 1 line

  0.96.D510 - Have the D compiler Tool use the same  logic for shared libraries, too. (Paolo Invernizzi)
........
  r1720 | stevenknight | 2006-12-07 23:29:47 -0600 (Thu, 07 Dec 2006) | 1 line

  0.96.D511 - Generalize a JobTests.py test so it doesn't assume a specific order in which the operating system executes the threads.
........
  r1721 | stevenknight | 2006-12-07 23:39:37 -0600 (Thu, 07 Dec 2006) | 1 line

  0.96.D512 - Back out the Tool/dmd.py  change; it breaks shared library linking for other lanuages beside D in the construction environment.
........
  r1722 | stevenknight | 2006-12-07 23:47:11 -0600 (Thu, 07 Dec 2006) | 1 line

  0.96.D513 - Test fixes: Windows portability, handle changes to Python 2.5 messages.
........
  r1723 | stevenknight | 2006-12-08 00:00:13 -0600 (Fri, 08 Dec 2006) | 1 line

  0.96.D514 - Change how the 'as' Tool is imported to accomodate the Python 2.6 'as' keyword.
........
  r1724 | stevenknight | 2006-12-08 11:19:27 -0600 (Fri, 08 Dec 2006) | 1 line

  0.96.D515 - Cache both Node.FS.find_file() and Node.FS.Dri.srcdir_find_file().
........
  r1725 | stevenknight | 2006-12-08 17:27:35 -0600 (Fri, 08 Dec 2006) | 1 line

  0.96.D516 - Better error when we try to fetch contents from an Entry that doesn't exist. (Tom Parker)
........
  r1726 | stevenknight | 2006-12-08 23:28:55 -0600 (Fri, 08 Dec 2006) | 1 line

  0.96.D517 - Make sure we pick up the scons-local directory regardless of where we chdir internally.
........
  r1727 | stevenknight | 2006-12-11 16:25:53 -0600 (Mon, 11 Dec 2006) | 1 line

  0.96.D518 - Cache results of Executor.get_unignored_sources() and Executor.process_sources(). Eliminate some map() and disambiguate() calls when scanning for implicit dependencies.
........
  r1728 | stevenknight | 2006-12-12 14:32:22 -0600 (Tue, 12 Dec 2006) | 1 line

  0.96.D519 - Fix SideEffect() when -j is used.
........
  r1729 | stevenknight | 2006-12-12 16:58:15 -0600 (Tue, 12 Dec 2006) | 1 line

  0.96.D520 - Add a srcdir keyword to Builder calls.
........
  r1730 | stevenknight | 2006-12-12 21:40:59 -0600 (Tue, 12 Dec 2006) | 1 line

  0.96.D521 - TeX/LaTeX updates, including handling files in subdirectories. (Joel B. Mohler, Rob Managan, Dmitry Mikhin)
........
  r1731 | stevenknight | 2006-12-14 15:01:02 -0600 (Thu, 14 Dec 2006) | 1 line

  0.96.D522 - Propogate TypeErrors during variable substitution for display to the user.
........
  r1732 | stevenknight | 2006-12-14 20:01:49 -0600 (Thu, 14 Dec 2006) | 1 line

  0.96.D523 - Fix the os.path.join() calls in EnvironmentTests.py.
........
  r1733 | stevenknight | 2006-12-15 07:48:22 -0600 (Fri, 15 Dec 2006) | 1 line

  0.96.D524 - Fix source directories as dependencies of an Alias (0.96.93 problem found by LilyPond).
........
  r1735 | stevenknight | 2006-12-15 12:43:45 -0600 (Fri, 15 Dec 2006) | 1 line

  0.96.D525 - Allow printing Debug.caller() output (or other end-of-run debugging info) when using -h.
........
  r1736 | stevenknight | 2006-12-15 16:30:08 -0600 (Fri, 15 Dec 2006) | 1 line

  0.96.D526 - Add an option to debug IndexError and NameError exceptions during variable substitution.
........

1 files changed, 61 insertions, 27 deletions

diff --git a/src/engine/SCons/Executor.py b/src/engine/SCons/Executor.py
index ffc1ba3..4b15010 100644
--- a/src/engine/SCons/Executor.py
+++ b/src/engine/SCons/Executor.py
@@ -47,6 +47,8 @@ class Executor:
     if SCons.Memoize.use_memoizer:
         __metaclass__ = SCons.Memoize.Memoized_Metaclass
 
+    memoizer_counters = []
+
     def __init__(self, action, env=None, overridelist=[{}],
                  targets=[], sources=[], builder_kw={}):
         if __debug__: logInstanceCreation(self, 'Executor.Executor')
@@ -58,6 +60,7 @@ class Executor:
         self.targets = targets
         self.sources = sources[:]
         self.builder_kw = builder_kw
+        self._memo = {}
 
     def set_action_list(self, action):
         if not SCons.Util.is_List(action):
@@ -72,7 +75,6 @@ class Executor:
     def get_build_env(self):
         """Fetch or create the appropriate build Environment
         for this Executor.
-        __cacheable__
         """
         # Create the build environment instance with appropriate
         # overrides.  These get evaluated against the current
@@ -125,8 +127,7 @@ class Executor:
         self.do_execute(target, exitstatfunc, kw)
 
     def cleanup(self):
-        "__reset_cache__"
-        pass
+        self._memo = {}
 
     def add_sources(self, sources):
         """Add source files to this Executor's list.  This is necessary
@@ -151,25 +152,30 @@ class Executor:
 
 
     def __str__(self):
-        "__cacheable__"
         return self.my_str()
 
     def nullify(self):
-        "__reset_cache__"
+        self.cleanup()
         self.do_execute = self.do_nothing
         self.my_str     = lambda S=self: ''
 
+    memoizer_counters.append(SCons.Memoize.CountValue('get_contents'))
+
     def get_contents(self):
-        """Fetch the signature contents.  This, along with
-        get_raw_contents(), is the real reason this class exists, so we
-        can compute this once and cache it regardless of how many target
-        or source Nodes there are.
-        __cacheable__
+        """Fetch the signature contents.  This is the main reason this
+        class exists, so we can compute this once and cache it regardless
+        of how many target or source Nodes there are.
         """
+        try:
+            return self._memo['get_contents']
+        except KeyError:
+            pass
         env = self.get_build_env()
         get = lambda action, t=self.targets, s=self.sources, e=env: \
                      action.get_contents(t, s, e)
-        return string.join(map(get, self.get_action_list()), "")
+        result = string.join(map(get, self.get_action_list()), "")
+        self._memo['get_contents'] = result
+        return result
 
     def get_timestamp(self):
         """Fetch a time stamp for this Executor.  We don't have one, of
@@ -219,20 +225,58 @@ class Executor:
 
     def get_missing_sources(self):
         """
-        __cacheable__
         """
         return filter(lambda s: s.missing(), self.sources)
 
-    def get_unignored_sources(self, ignore):
-        """__cacheable__"""
+    def _get_unignored_sources_key(self, ignore=()):
+        return tuple(ignore)
+
+    memoizer_counters.append(SCons.Memoize.CountDict('get_unignored_sources', _get_unignored_sources_key))
+
+    def get_unignored_sources(self, ignore=()):
+        ignore = tuple(ignore)
+        try:
+            memo_dict = self._memo['get_unignored_sources']
+        except KeyError:
+            memo_dict = {}
+            self._memo['get_unignored_sources'] = memo_dict
+        else:
+            try:
+                return memo_dict[ignore]
+            except KeyError:
+                pass
+
         sourcelist = self.sources
         if ignore:
             sourcelist = filter(lambda s, i=ignore: not s in i, sourcelist)
+
+        memo_dict[ignore] = sourcelist
+
         return sourcelist
 
-    def process_sources(self, func, ignore=[]):
-        """__cacheable__"""
-        return map(func, self.get_unignored_sources(ignore))
+    def _process_sources_key(self, func, ignore=()):
+        return (func, tuple(ignore))
+
+    memoizer_counters.append(SCons.Memoize.CountDict('process_sources', _process_sources_key))
+
+    def process_sources(self, func, ignore=()):
+        memo_key = (func, tuple(ignore))
+        try:
+            memo_dict = self._memo['process_sources']
+        except KeyError:
+            memo_dict = {}
+            self._memo['process_sources'] = memo_dict
+        else:
+            try:
+                return memo_dict[memo_key]
+            except KeyError:
+                pass
+
+        result = map(func, self.get_unignored_sources(ignore))
+
+        memo_dict[memo_key] = result
+
+        return result
 
 
 _Executor = Executor
@@ -258,13 +302,3 @@ class Null(_Executor):
         return None
     def cleanup(self):
         pass
-
-
-
-if SCons.Memoize.use_old_memoization():
-    _Base = Executor
-    class Executor(SCons.Memoize.Memoizer, _Base):
-        def __init__(self, *args, **kw):
-            SCons.Memoize.Memoizer.__init__(self)
-            apply(_Base.__init__, (self,)+args, kw)
-