Merged revisions 1675-1736 via svnmerge from

http://scons.tigris.org/svn/scons/branches/core

........
  r1689 | stevenknight | 2006-11-06 20:56:29 -0600 (Mon, 06 Nov 2006) | 1 line

  0.96.D483 - Merge changes for 0.96.93 packaging from the subsidiary branch.
........
  r1690 | stevenknight | 2006-11-06 20:59:30 -0600 (Mon, 06 Nov 2006) | 1 line

  0.96.D484 - Update HOWTO for releases. Fix name type in src/CHANGES.txt.
........
  r1691 | stevenknight | 2006-11-08 13:55:36 -0600 (Wed, 08 Nov 2006) | 1 line

  0.96.D485 - Fix MergeFlags() handling of None values. (John Pye)
........
  r1692 | stevenknight | 2006-11-08 17:15:05 -0600 (Wed, 08 Nov 2006) | 1 line

  0.96.D486 - Directly execute commands on Windows when possible. (Jay Kint)
........
  r1693 | stevenknight | 2006-11-08 18:54:49 -0600 (Wed, 08 Nov 2006) | 1 line

  0.96.D487 - Remove the semi-colon from the list of characters that determine when we use cmd
........
  r1694 | stevenknight | 2006-11-09 01:34:06 -0600 (Thu, 09 Nov 2006) | 1 line

  0.96.D488 - Pick up latex/bibtex 'Rerun to get citations correct' messages. (Dmitry Mikhin)
........
  r1695 | stevenknight | 2006-11-11 08:36:33 -0600 (Sat, 11 Nov 2006) | 1 line

  0.96.D489 - Back out the direct-execution-on-Windows change until we solve a corner case.
........
  r1696 | stevenknight | 2006-11-15 10:33:10 -0600 (Wed, 15 Nov 2006) | 1 line

  0.96.D490 - Fix the sconsign script when the .sconsign.dblite file is specified with its suf
........
  r1697 | stevenknight | 2006-11-18 10:45:50 -0600 (Sat, 18 Nov 2006) | 4 lines

  Complete move of test/sconsign/script.py to underneath test/sconsign/script/.
  (This got left out of the previous checkin due to an error in the
  script that resubmits Aegis changes to Subversion.)
........
  r1698 | stevenknight | 2006-11-18 11:05:26 -0600 (Sat, 18 Nov 2006) | 1 line

  0.96.D491 - Allow an Options converter to take the construction environment as a parameter.
........
  r1699 | stevenknight | 2006-11-30 15:34:37 -0600 (Thu, 30 Nov 2006) | 1 line

  0.96.D492 - Reverse the order in which we try the arguments Options converters, first a sing
........
  r1700 | stevenknight | 2006-11-30 16:03:09 -0600 (Thu, 30 Nov 2006) | 1 line

  0.96.D493 - Speed up rel_path() by avoiding recomputation of intermediate directory relative
........
  r1701 | stevenknight | 2006-11-30 16:14:16 -0600 (Thu, 30 Nov 2006) | 1 line

  0.96.D494 - More efficient get_suffix(): compute it once when we set the name.
........
  r1702 | stevenknight | 2006-11-30 16:22:55 -0600 (Thu, 30 Nov 2006) | 1 line

  0.96.D495 - Fix missing XML end tags.
........
  r1703 | stevenknight | 2006-11-30 17:15:25 -0600 (Thu, 30 Nov 2006) | 1 line

  0.96.D496 - Turn Memoizer into a simple counter for --debug=memoizer, not something that doe
........
  r1704 | stevenknight | 2006-11-30 20:30:50 -0600 (Thu, 30 Nov 2006) | 1 line

  0.96.D497 - Add the scons-time script, with doc and tests.
........
  r1705 | stevenknight | 2006-11-30 23:28:20 -0600 (Thu, 30 Nov 2006) | 1 line

  0.96.D498 - Update the copyright years string.
........
  r1706 | stevenknight | 2006-12-01 11:54:22 -0600 (Fri, 01 Dec 2006) | 1 line

  0.96.D499 - Fix _do_Lookup => _doLookup value-caching misspellings. (Ben Leslie)
........
  r1707 | stevenknight | 2006-12-01 12:03:46 -0600 (Fri, 01 Dec 2006) | 1 line

  0.96.D500 - Fix copyright test against debian build. (Walter Franzini)
........
  r1708 | stevenknight | 2006-12-01 14:23:29 -0600 (Fri, 01 Dec 2006) | 1 line

  0.96.D501 - Add #include lines for test portability. (Gary Oberbrunner)
........
  r1709 | stevenknight | 2006-12-01 14:51:12 -0600 (Fri, 01 Dec 2006) | 1 line

  0.96.D502 - Fix tests under Python versions with no profiler (pstats module).
........
  r1710 | stevenknight | 2006-12-01 20:04:49 -0600 (Fri, 01 Dec 2006) | 1 line

  0.96.D503 - Remove unnecessary os.path.normpath() calls. (Gary Oberbrunner)
........
  r1711 | stevenknight | 2006-12-01 20:34:31 -0600 (Fri, 01 Dec 2006) | 1 line

  0.96.D504 - Accomodate arbitray white space after a SWIG %module keyword. (Anonymous)
........
  r1712 | stevenknight | 2006-12-05 14:49:54 -0600 (Tue, 05 Dec 2006) | 1 line

  0.96.D506 - Cache substitutions of of Builder source suffixes. Use a new PathList module, and a refactor Node.FS.Rfindalldirs() method, to cache calculations of values like CPPPATH.
........
  r1713 | stevenknight | 2006-12-05 18:43:36 -0600 (Tue, 05 Dec 2006) | 1 line

  0.96.D507 - Use cached stat() values in diskchecks.
........
  r1714 | stevenknight | 2006-12-05 21:11:24 -0600 (Tue, 05 Dec 2006) | 1 line

  0.96.D508 - Fix Memoizer hit counts for methods memoizing simple values. Clean up the code for memoizing return values in a dictionary. Fix comments.
........
  r1715 | stevenknight | 2006-12-06 07:23:18 -0600 (Wed, 06 Dec 2006) | 1 line

  0.96.D369 - More efficient Node.FS.Dir.current() check. Fix some Windows test portability issues.
........
  r1716 | stevenknight | 2006-12-06 12:24:32 -0600 (Wed, 06 Dec 2006) | 2 lines

  Undo previous checkin (distributed incorrect Aegis change number).
........
  r1717 | stevenknight | 2006-12-06 12:34:53 -0600 (Wed, 06 Dec 2006) | 1 line

  0.96.D505 - Update ae-{cvs,svn}-ci for newer versions of aetar, and to not truncate descriptions.
........
  r1718 | stevenknight | 2006-12-07 23:01:41 -0600 (Thu, 07 Dec 2006) | 1 line

  0.96.D509 - Only look for mslink on Windows systems. (Sohail Somani)
........
  r1719 | stevenknight | 2006-12-07 23:18:33 -0600 (Thu, 07 Dec 2006) | 1 line

  0.96.D510 - Have the D compiler Tool use the same  logic for shared libraries, too. (Paolo Invernizzi)
........
  r1720 | stevenknight | 2006-12-07 23:29:47 -0600 (Thu, 07 Dec 2006) | 1 line

  0.96.D511 - Generalize a JobTests.py test so it doesn't assume a specific order in which the operating system executes the threads.
........
  r1721 | stevenknight | 2006-12-07 23:39:37 -0600 (Thu, 07 Dec 2006) | 1 line

  0.96.D512 - Back out the Tool/dmd.py  change; it breaks shared library linking for other lanuages beside D in the construction environment.
........
  r1722 | stevenknight | 2006-12-07 23:47:11 -0600 (Thu, 07 Dec 2006) | 1 line

  0.96.D513 - Test fixes: Windows portability, handle changes to Python 2.5 messages.
........
  r1723 | stevenknight | 2006-12-08 00:00:13 -0600 (Fri, 08 Dec 2006) | 1 line

  0.96.D514 - Change how the 'as' Tool is imported to accomodate the Python 2.6 'as' keyword.
........
  r1724 | stevenknight | 2006-12-08 11:19:27 -0600 (Fri, 08 Dec 2006) | 1 line

  0.96.D515 - Cache both Node.FS.find_file() and Node.FS.Dri.srcdir_find_file().
........
  r1725 | stevenknight | 2006-12-08 17:27:35 -0600 (Fri, 08 Dec 2006) | 1 line

  0.96.D516 - Better error when we try to fetch contents from an Entry that doesn't exist. (Tom Parker)
........
  r1726 | stevenknight | 2006-12-08 23:28:55 -0600 (Fri, 08 Dec 2006) | 1 line

  0.96.D517 - Make sure we pick up the scons-local directory regardless of where we chdir internally.
........
  r1727 | stevenknight | 2006-12-11 16:25:53 -0600 (Mon, 11 Dec 2006) | 1 line

  0.96.D518 - Cache results of Executor.get_unignored_sources() and Executor.process_sources(). Eliminate some map() and disambiguate() calls when scanning for implicit dependencies.
........
  r1728 | stevenknight | 2006-12-12 14:32:22 -0600 (Tue, 12 Dec 2006) | 1 line

  0.96.D519 - Fix SideEffect() when -j is used.
........
  r1729 | stevenknight | 2006-12-12 16:58:15 -0600 (Tue, 12 Dec 2006) | 1 line

  0.96.D520 - Add a srcdir keyword to Builder calls.
........
  r1730 | stevenknight | 2006-12-12 21:40:59 -0600 (Tue, 12 Dec 2006) | 1 line

  0.96.D521 - TeX/LaTeX updates, including handling files in subdirectories. (Joel B. Mohler, Rob Managan, Dmitry Mikhin)
........
  r1731 | stevenknight | 2006-12-14 15:01:02 -0600 (Thu, 14 Dec 2006) | 1 line

  0.96.D522 - Propogate TypeErrors during variable substitution for display to the user.
........
  r1732 | stevenknight | 2006-12-14 20:01:49 -0600 (Thu, 14 Dec 2006) | 1 line

  0.96.D523 - Fix the os.path.join() calls in EnvironmentTests.py.
........
  r1733 | stevenknight | 2006-12-15 07:48:22 -0600 (Fri, 15 Dec 2006) | 1 line

  0.96.D524 - Fix source directories as dependencies of an Alias (0.96.93 problem found by LilyPond).
........
  r1735 | stevenknight | 2006-12-15 12:43:45 -0600 (Fri, 15 Dec 2006) | 1 line

  0.96.D525 - Allow printing Debug.caller() output (or other end-of-run debugging info) when using -h.
........
  r1736 | stevenknight | 2006-12-15 16:30:08 -0600 (Fri, 15 Dec 2006) | 1 line

  0.96.D526 - Add an option to debug IndexError and NameError exceptions during variable substitution.
........

1 files changed, 96 insertions, 13 deletions

diff --git a/src/engine/SCons/Builder.py b/src/engine/SCons/Builder.py
index 16f1191..d5f566a 100644
--- a/src/engine/SCons/Builder.py
+++ b/src/engine/SCons/Builder.py
@@ -126,6 +126,7 @@ import SCons.Action
 from SCons.Debug import logInstanceCreation
 from SCons.Errors import InternalError, UserError
 import SCons.Executor
+import SCons.Memoize
 import SCons.Node
 import SCons.Node.FS
 import SCons.Util
@@ -370,6 +371,8 @@ class BuilderBase:
     if SCons.Memoize.use_memoizer:
         __metaclass__ = SCons.Memoize.Memoized_Metaclass
 
+    memoizer_counters = []
+
     def __init__(self,  action = None,
                         prefix = '',
                         suffix = '',
@@ -387,6 +390,7 @@ class BuilderBase:
                         is_explicit = 1,
                         **overrides):
         if __debug__: logInstanceCreation(self, 'Builder.BuilderBase')
+        self._memo = {}
         self.action = action
         self.multi = multi
         if SCons.Util.is_Dict(prefix):
@@ -604,6 +608,16 @@ class BuilderBase:
             ekw = self.executor_kw.copy()
             ekw['chdir'] = chdir
         if kw:
+            if kw.has_key('srcdir'):
+                def prependDirIfRelative(f, srcdir=kw['srcdir']):
+                    import os.path
+                    if SCons.Util.is_String(f) and not os.path.isabs(f):
+                        f = os.path.join(srcdir, f)
+                    return f
+                if not SCons.Util.is_List(source):
+                    source = [source]
+                source = map(prependDirIfRelative, source)
+                del kw['srcdir']
             if self.overrides:
                 env_kw = self.overrides.copy()
                 env_kw.update(kw)
@@ -636,9 +650,34 @@ class BuilderBase:
             suffix = suffix(env, sources)
         return env.subst(suffix)
 
+    def _src_suffixes_key(self, env):
+        return id(env)
+
+    memoizer_counters.append(SCons.Memoize.CountDict('src_suffixes', _src_suffixes_key))
+
     def src_suffixes(self, env):
-        "__cacheable__"
-        return map(lambda x, s=self, e=env: e.subst(x), self.src_suffix)
+        """
+        Returns the list of source suffixes for this Builder.
+
+        The suffix list may contain construction variable expansions,
+        so we have to evaluate the individual strings.  To avoid doing
+        this over and over, we memoize the results for each construction
+        environment.
+        """
+        memo_key = id(env)
+        try:
+            memo_dict = self._memo['src_suffixes']
+        except KeyError:
+            memo_dict = {}
+            self._memo['src_suffixes'] = memo_dict
+        else:
+            try:
+                return memo_dict[memo_key]
+            except KeyError:
+                pass
+        result = map(lambda x, s=self, e=env: e.subst(x), self.src_suffix)
+        memo_dict[memo_key] = result
+        return result
 
     def set_src_suffix(self, src_suffix):
         if not src_suffix:
@@ -673,13 +712,7 @@ class BuilderBase:
         """
         self.emitter[suffix] = emitter
 
-if SCons.Memoize.use_old_memoization():
-    _Base = BuilderBase
-    class BuilderBase(SCons.Memoize.Memoizer, _Base):
-        "Cache-backed version of BuilderBase"
-        def __init__(self, *args, **kw):
-            apply(_Base.__init__, (self,)+args, kw)
-            SCons.Memoize.Memoizer.__init__(self)
+
 
 class ListBuilder(SCons.Util.Proxy):
     """A Proxy to support building an array of targets (for example,
@@ -718,6 +751,9 @@ class MultiStepBuilder(BuilderBase):
     builder is NOT invoked if the suffix of a source file matches
     src_suffix.
     """
+
+    memoizer_counters = []
+
     def __init__(self,  src_builder,
                         action = None,
                         prefix = '',
@@ -738,8 +774,32 @@ class MultiStepBuilder(BuilderBase):
             src_builder = [ src_builder ]
         self.src_builder = src_builder
 
+    def _get_sdict_key(self, env):
+        return id(env)
+
+    memoizer_counters.append(SCons.Memoize.CountDict('_get_sdict', _get_sdict_key))
+
     def _get_sdict(self, env):
-        "__cacheable__"
+        """
+        Returns a dictionary mapping all of the source suffixes of all
+        src_builders of this Builder to the underlying Builder that
+        should be called first.
+
+        This dictionary is used for each target specified, so we save a
+        lot of extra computation by memoizing it for each construction
+        environment.
+        """
+        memo_key = id(env)
+        try:
+            memo_dict = self._memo['_get_sdict']
+        except KeyError:
+            memo_dict = {}
+            self._memo['_get_sdict'] = memo_dict
+        else:
+            try:
+                return memo_dict[memo_key]
+            except KeyError:
+                pass
         sdict = {}
         for bld in self.src_builder:
             if SCons.Util.is_String(bld):
@@ -749,6 +809,7 @@ class MultiStepBuilder(BuilderBase):
                     continue
             for suf in bld.src_suffixes(env):
                 sdict[suf] = bld
+        memo_dict[memo_key] = sdict
         return sdict
         
     def _execute(self, env, target, source, overwarn={}, executor_kw={}):
@@ -810,14 +871,36 @@ class MultiStepBuilder(BuilderBase):
             ret.append(bld)
         return ret
 
+    def _src_suffixes_key(self, env):
+        return id(env)
+
+    memoizer_counters.append(SCons.Memoize.CountDict('src_suffixes', _src_suffixes_key))
+
     def src_suffixes(self, env):
-        """Return a list of the src_suffix attributes for all
-        src_builders of this Builder.
-        __cacheable__
         """
+        Returns the list of source suffixes for all src_builders of this
+        Builder.
+
+        The suffix list may contain construction variable expansions,
+        so we have to evaluate the individual strings.  To avoid doing
+        this over and over, we memoize the results for each construction
+        environment.
+        """
+        memo_key = id(env)
+        try:
+            memo_dict = self._memo['src_suffixes']
+        except KeyError:
+            memo_dict = {}
+            self._memo['src_suffixes'] = memo_dict
+        else:
+            try:
+                return memo_dict[memo_key]
+            except KeyError:
+                pass
         suffixes = BuilderBase.src_suffixes(self, env)
         for builder in self.get_src_builders(env):
             suffixes.extend(builder.src_suffixes(env))
+        memo_dict[memo_key] = suffixes
         return suffixes
 
 class CompositeBuilder(SCons.Util.Proxy):