Merged revisions 1675-1736 via svnmerge from

http://scons.tigris.org/svn/scons/branches/core

........
  r1689 | stevenknight | 2006-11-06 20:56:29 -0600 (Mon, 06 Nov 2006) | 1 line

  0.96.D483 - Merge changes for 0.96.93 packaging from the subsidiary branch.
........
  r1690 | stevenknight | 2006-11-06 20:59:30 -0600 (Mon, 06 Nov 2006) | 1 line

  0.96.D484 - Update HOWTO for releases. Fix name type in src/CHANGES.txt.
........
  r1691 | stevenknight | 2006-11-08 13:55:36 -0600 (Wed, 08 Nov 2006) | 1 line

  0.96.D485 - Fix MergeFlags() handling of None values. (John Pye)
........
  r1692 | stevenknight | 2006-11-08 17:15:05 -0600 (Wed, 08 Nov 2006) | 1 line

  0.96.D486 - Directly execute commands on Windows when possible. (Jay Kint)
........
  r1693 | stevenknight | 2006-11-08 18:54:49 -0600 (Wed, 08 Nov 2006) | 1 line

  0.96.D487 - Remove the semi-colon from the list of characters that determine when we use cmd
........
  r1694 | stevenknight | 2006-11-09 01:34:06 -0600 (Thu, 09 Nov 2006) | 1 line

  0.96.D488 - Pick up latex/bibtex 'Rerun to get citations correct' messages. (Dmitry Mikhin)
........
  r1695 | stevenknight | 2006-11-11 08:36:33 -0600 (Sat, 11 Nov 2006) | 1 line

  0.96.D489 - Back out the direct-execution-on-Windows change until we solve a corner case.
........
  r1696 | stevenknight | 2006-11-15 10:33:10 -0600 (Wed, 15 Nov 2006) | 1 line

  0.96.D490 - Fix the sconsign script when the .sconsign.dblite file is specified with its suf
........
  r1697 | stevenknight | 2006-11-18 10:45:50 -0600 (Sat, 18 Nov 2006) | 4 lines

  Complete move of test/sconsign/script.py to underneath test/sconsign/script/.
  (This got left out of the previous checkin due to an error in the
  script that resubmits Aegis changes to Subversion.)
........
  r1698 | stevenknight | 2006-11-18 11:05:26 -0600 (Sat, 18 Nov 2006) | 1 line

  0.96.D491 - Allow an Options converter to take the construction environment as a parameter.
........
  r1699 | stevenknight | 2006-11-30 15:34:37 -0600 (Thu, 30 Nov 2006) | 1 line

  0.96.D492 - Reverse the order in which we try the arguments Options converters, first a sing
........
  r1700 | stevenknight | 2006-11-30 16:03:09 -0600 (Thu, 30 Nov 2006) | 1 line

  0.96.D493 - Speed up rel_path() by avoiding recomputation of intermediate directory relative
........
  r1701 | stevenknight | 2006-11-30 16:14:16 -0600 (Thu, 30 Nov 2006) | 1 line

  0.96.D494 - More efficient get_suffix(): compute it once when we set the name.
........
  r1702 | stevenknight | 2006-11-30 16:22:55 -0600 (Thu, 30 Nov 2006) | 1 line

  0.96.D495 - Fix missing XML end tags.
........
  r1703 | stevenknight | 2006-11-30 17:15:25 -0600 (Thu, 30 Nov 2006) | 1 line

  0.96.D496 - Turn Memoizer into a simple counter for --debug=memoizer, not something that doe
........
  r1704 | stevenknight | 2006-11-30 20:30:50 -0600 (Thu, 30 Nov 2006) | 1 line

  0.96.D497 - Add the scons-time script, with doc and tests.
........
  r1705 | stevenknight | 2006-11-30 23:28:20 -0600 (Thu, 30 Nov 2006) | 1 line

  0.96.D498 - Update the copyright years string.
........
  r1706 | stevenknight | 2006-12-01 11:54:22 -0600 (Fri, 01 Dec 2006) | 1 line

  0.96.D499 - Fix _do_Lookup => _doLookup value-caching misspellings. (Ben Leslie)
........
  r1707 | stevenknight | 2006-12-01 12:03:46 -0600 (Fri, 01 Dec 2006) | 1 line

  0.96.D500 - Fix copyright test against debian build. (Walter Franzini)
........
  r1708 | stevenknight | 2006-12-01 14:23:29 -0600 (Fri, 01 Dec 2006) | 1 line

  0.96.D501 - Add #include lines for test portability. (Gary Oberbrunner)
........
  r1709 | stevenknight | 2006-12-01 14:51:12 -0600 (Fri, 01 Dec 2006) | 1 line

  0.96.D502 - Fix tests under Python versions with no profiler (pstats module).
........
  r1710 | stevenknight | 2006-12-01 20:04:49 -0600 (Fri, 01 Dec 2006) | 1 line

  0.96.D503 - Remove unnecessary os.path.normpath() calls. (Gary Oberbrunner)
........
  r1711 | stevenknight | 2006-12-01 20:34:31 -0600 (Fri, 01 Dec 2006) | 1 line

  0.96.D504 - Accomodate arbitray white space after a SWIG %module keyword. (Anonymous)
........
  r1712 | stevenknight | 2006-12-05 14:49:54 -0600 (Tue, 05 Dec 2006) | 1 line

  0.96.D506 - Cache substitutions of of Builder source suffixes. Use a new PathList module, and a refactor Node.FS.Rfindalldirs() method, to cache calculations of values like CPPPATH.
........
  r1713 | stevenknight | 2006-12-05 18:43:36 -0600 (Tue, 05 Dec 2006) | 1 line

  0.96.D507 - Use cached stat() values in diskchecks.
........
  r1714 | stevenknight | 2006-12-05 21:11:24 -0600 (Tue, 05 Dec 2006) | 1 line

  0.96.D508 - Fix Memoizer hit counts for methods memoizing simple values. Clean up the code for memoizing return values in a dictionary. Fix comments.
........
  r1715 | stevenknight | 2006-12-06 07:23:18 -0600 (Wed, 06 Dec 2006) | 1 line

  0.96.D369 - More efficient Node.FS.Dir.current() check. Fix some Windows test portability issues.
........
  r1716 | stevenknight | 2006-12-06 12:24:32 -0600 (Wed, 06 Dec 2006) | 2 lines

  Undo previous checkin (distributed incorrect Aegis change number).
........
  r1717 | stevenknight | 2006-12-06 12:34:53 -0600 (Wed, 06 Dec 2006) | 1 line

  0.96.D505 - Update ae-{cvs,svn}-ci for newer versions of aetar, and to not truncate descriptions.
........
  r1718 | stevenknight | 2006-12-07 23:01:41 -0600 (Thu, 07 Dec 2006) | 1 line

  0.96.D509 - Only look for mslink on Windows systems. (Sohail Somani)
........
  r1719 | stevenknight | 2006-12-07 23:18:33 -0600 (Thu, 07 Dec 2006) | 1 line

  0.96.D510 - Have the D compiler Tool use the same  logic for shared libraries, too. (Paolo Invernizzi)
........
  r1720 | stevenknight | 2006-12-07 23:29:47 -0600 (Thu, 07 Dec 2006) | 1 line

  0.96.D511 - Generalize a JobTests.py test so it doesn't assume a specific order in which the operating system executes the threads.
........
  r1721 | stevenknight | 2006-12-07 23:39:37 -0600 (Thu, 07 Dec 2006) | 1 line

  0.96.D512 - Back out the Tool/dmd.py  change; it breaks shared library linking for other lanuages beside D in the construction environment.
........
  r1722 | stevenknight | 2006-12-07 23:47:11 -0600 (Thu, 07 Dec 2006) | 1 line

  0.96.D513 - Test fixes: Windows portability, handle changes to Python 2.5 messages.
........
  r1723 | stevenknight | 2006-12-08 00:00:13 -0600 (Fri, 08 Dec 2006) | 1 line

  0.96.D514 - Change how the 'as' Tool is imported to accomodate the Python 2.6 'as' keyword.
........
  r1724 | stevenknight | 2006-12-08 11:19:27 -0600 (Fri, 08 Dec 2006) | 1 line

  0.96.D515 - Cache both Node.FS.find_file() and Node.FS.Dri.srcdir_find_file().
........
  r1725 | stevenknight | 2006-12-08 17:27:35 -0600 (Fri, 08 Dec 2006) | 1 line

  0.96.D516 - Better error when we try to fetch contents from an Entry that doesn't exist. (Tom Parker)
........
  r1726 | stevenknight | 2006-12-08 23:28:55 -0600 (Fri, 08 Dec 2006) | 1 line

  0.96.D517 - Make sure we pick up the scons-local directory regardless of where we chdir internally.
........
  r1727 | stevenknight | 2006-12-11 16:25:53 -0600 (Mon, 11 Dec 2006) | 1 line

  0.96.D518 - Cache results of Executor.get_unignored_sources() and Executor.process_sources(). Eliminate some map() and disambiguate() calls when scanning for implicit dependencies.
........
  r1728 | stevenknight | 2006-12-12 14:32:22 -0600 (Tue, 12 Dec 2006) | 1 line

  0.96.D519 - Fix SideEffect() when -j is used.
........
  r1729 | stevenknight | 2006-12-12 16:58:15 -0600 (Tue, 12 Dec 2006) | 1 line

  0.96.D520 - Add a srcdir keyword to Builder calls.
........
  r1730 | stevenknight | 2006-12-12 21:40:59 -0600 (Tue, 12 Dec 2006) | 1 line

  0.96.D521 - TeX/LaTeX updates, including handling files in subdirectories. (Joel B. Mohler, Rob Managan, Dmitry Mikhin)
........
  r1731 | stevenknight | 2006-12-14 15:01:02 -0600 (Thu, 14 Dec 2006) | 1 line

  0.96.D522 - Propogate TypeErrors during variable substitution for display to the user.
........
  r1732 | stevenknight | 2006-12-14 20:01:49 -0600 (Thu, 14 Dec 2006) | 1 line

  0.96.D523 - Fix the os.path.join() calls in EnvironmentTests.py.
........
  r1733 | stevenknight | 2006-12-15 07:48:22 -0600 (Fri, 15 Dec 2006) | 1 line

  0.96.D524 - Fix source directories as dependencies of an Alias (0.96.93 problem found by LilyPond).
........
  r1735 | stevenknight | 2006-12-15 12:43:45 -0600 (Fri, 15 Dec 2006) | 1 line

  0.96.D525 - Allow printing Debug.caller() output (or other end-of-run debugging info) when using -h.
........
  r1736 | stevenknight | 2006-12-15 16:30:08 -0600 (Fri, 15 Dec 2006) | 1 line

  0.96.D526 - Add an option to debug IndexError and NameError exceptions during variable substitution.
........

1 files changed, 62 insertions, 52 deletions

diff --git a/src/engine/SCons/Environment.py b/src/engine/SCons/Environment.py
index a5bffc8..4761ea0 100644
--- a/src/engine/SCons/Environment.py
+++ b/src/engine/SCons/Environment.py
@@ -166,6 +166,10 @@ def _set_BUILDERS(env, key, value):
         env._dict[key] = BuilderDict(kwbd, env)
     env._dict[key].update(value)
 
+def _del_SCANNERS(env, key):
+    del env._dict[key]
+    env.scanner_map_delete()
+
 def _set_SCANNERS(env, key, value):
     env._dict[key] = value
     env.scanner_map_delete()
@@ -279,29 +283,35 @@ class SubstitutionEnvironment:
         self.lookup_list = SCons.Node.arg2nodes_lookups
         self._dict = kw.copy()
         self._init_special()
+        #self._memo = {}
 
     def _init_special(self):
-        """Initial the dispatch table for special handling of
+        """Initial the dispatch tables for special handling of
         special construction variables."""
-        self._special = {}
+        self._special_del = {}
+        self._special_del['SCANNERS'] = _del_SCANNERS
+
+        self._special_set = {}
         for key in reserved_construction_var_names:
-            self._special[key] = _set_reserved
-        self._special['BUILDERS'] = _set_BUILDERS
-        self._special['SCANNERS'] = _set_SCANNERS
+            self._special_set[key] = _set_reserved
+        self._special_set['BUILDERS'] = _set_BUILDERS
+        self._special_set['SCANNERS'] = _set_SCANNERS
 
     def __cmp__(self, other):
         return cmp(self._dict, other._dict)
 
     def __delitem__(self, key):
-        "__cache_reset__"
-        del self._dict[key]
+        special = self._special_del.get(key)
+        if special:
+            special(self, key)
+        else:
+            del self._dict[key]
 
     def __getitem__(self, key):
         return self._dict[key]
 
     def __setitem__(self, key, value):
-        "__cache_reset__"
-        special = self._special.get(key)
+        special = self._special_set.get(key)
         if special:
             special(self, key, value)
         else:
@@ -663,8 +673,10 @@ class SubstitutionEnvironment:
             except KeyError:
                 orig = value
             else:
-                if len(orig) == 0: orig = []
-                elif not SCons.Util.is_List(orig): orig = [orig]
+                if not orig:
+                    orig = []
+                elif not SCons.Util.is_List(orig): 
+                    orig = [orig]
                 orig = orig + value
             t = []
             if key[-4:] == 'PATH':
@@ -694,6 +706,8 @@ class Base(SubstitutionEnvironment):
     if SCons.Memoize.use_memoizer:
         __metaclass__ = SCons.Memoize.Memoized_Metaclass
 
+    memoizer_counters = []
+
     #######################################################################
     # This is THE class for interacting with the SCons build engine,
     # and it contains a lot of stuff, so we're going to try to keep this
@@ -725,6 +739,7 @@ class Base(SubstitutionEnvironment):
         with the much simpler base class initialization.
         """
         if __debug__: logInstanceCreation(self, 'Environment.Base')
+        self._memo = {}
         self.fs = SCons.Node.FS.default_fs or SCons.Node.FS.FS()
         self.ans = SCons.Node.Alias.default_ans
         self.lookup_list = SCons.Node.arg2nodes_lookups
@@ -786,7 +801,6 @@ class Base(SubstitutionEnvironment):
             return None
 
     def get_calculator(self):
-        "__cacheable__"
         try:
             module = self._calc_module
             c = apply(SCons.Sig.Calculator, (module,), CalculatorArgs)
@@ -800,7 +814,6 @@ class Base(SubstitutionEnvironment):
     def get_factory(self, factory, default='File'):
         """Return a factory function for creating Nodes for this
         construction environment.
-        __cacheable__
         """
         name = default
         try:
@@ -827,50 +840,54 @@ class Base(SubstitutionEnvironment):
             factory = getattr(self.fs, name)
         return factory
 
+    memoizer_counters.append(SCons.Memoize.CountValue('_gsm'))
+
     def _gsm(self):
-        "__cacheable__"
         try:
-            scanners = self._dict['SCANNERS']
+            return self._memo['_gsm']
         except KeyError:
-            return None
+            pass
 
-        sm = {}
-        # Reverse the scanner list so that, if multiple scanners
-        # claim they can scan the same suffix, earlier scanners
-        # in the list will overwrite later scanners, so that
-        # the result looks like a "first match" to the user.
-        if not SCons.Util.is_List(scanners):
-            scanners = [scanners]
+        result = {}
+
+        try:
+            scanners = self._dict['SCANNERS']
+        except KeyError:
+            pass
         else:
-            scanners = scanners[:] # copy so reverse() doesn't mod original
-        scanners.reverse()
-        for scanner in scanners:
-            for k in scanner.get_skeys(self):
-                sm[k] = scanner
-        return sm
+            # Reverse the scanner list so that, if multiple scanners
+            # claim they can scan the same suffix, earlier scanners
+            # in the list will overwrite later scanners, so that
+            # the result looks like a "first match" to the user.
+            if not SCons.Util.is_List(scanners):
+                scanners = [scanners]
+            else:
+                scanners = scanners[:] # copy so reverse() doesn't mod original
+            scanners.reverse()
+            for scanner in scanners:
+                for k in scanner.get_skeys(self):
+                    result[k] = scanner
+
+        self._memo['_gsm'] = result
+
+        return result
         
     def get_scanner(self, skey):
         """Find the appropriate scanner given a key (usually a file suffix).
         """
-        sm = self._gsm()
-        try: return sm[skey]
-        except (KeyError, TypeError): return None
+        return self._gsm().get(skey)
 
-    def _smd(self):
-        "__reset_cache__"
-        pass
-    
     def scanner_map_delete(self, kw=None):
         """Delete the cached scanner map (if we need to).
         """
-        if not kw is None and not kw.has_key('SCANNERS'):
-            return
-        self._smd()
+        try:
+            del self._memo['_gsm']
+        except KeyError:
+            pass
 
     def _update(self, dict):
         """Update an environment's values directly, bypassing the normal
         checks that occur when users try to set items.
-        __cache_reset__
         """
         self._dict.update(dict)
 
@@ -1014,7 +1031,9 @@ class Base(SubstitutionEnvironment):
             clone._dict['BUILDERS'] = BuilderDict(cbd, clone)
         except KeyError:
             pass
-        
+
+        clone._memo = {}
+
         apply_tools(clone, tools, toolpath)
 
         # Apply passed-in variables after the new tools.
@@ -1030,7 +1049,7 @@ class Base(SubstitutionEnvironment):
         return apply(self.Clone, args, kw)
 
     def Detect(self, progs):
-        """Return the first available program in progs.  __cacheable__
+        """Return the first available program in progs.
         """
         if not SCons.Util.is_List(progs):
             progs = [ progs ]
@@ -1306,7 +1325,7 @@ class Base(SubstitutionEnvironment):
         tool(self)
 
     def WhereIs(self, prog, path=None, pathext=None, reject=[]):
-        """Find prog in the path.  __cacheable__
+        """Find prog in the path.
         """
         if path is None:
             try:
@@ -1841,12 +1860,3 @@ def NoSubstitutionProxy(subject):
             self.raw_to_mode(nkw)
             return apply(SCons.Subst.scons_subst, nargs, nkw)
     return _NoSubstitutionProxy(subject)
-
-if SCons.Memoize.use_old_memoization():
-    _Base = Base
-    class Base(SCons.Memoize.Memoizer, _Base):
-        def __init__(self, *args, **kw):
-            SCons.Memoize.Memoizer.__init__(self)
-            apply(_Base.__init__, (self,)+args, kw)
-    Environment = Base
-