Merged revisions 1675-1736 via svnmerge from

http://scons.tigris.org/svn/scons/branches/core

........
  r1689 | stevenknight | 2006-11-06 20:56:29 -0600 (Mon, 06 Nov 2006) | 1 line

  0.96.D483 - Merge changes for 0.96.93 packaging from the subsidiary branch.
........
  r1690 | stevenknight | 2006-11-06 20:59:30 -0600 (Mon, 06 Nov 2006) | 1 line

  0.96.D484 - Update HOWTO for releases. Fix name type in src/CHANGES.txt.
........
  r1691 | stevenknight | 2006-11-08 13:55:36 -0600 (Wed, 08 Nov 2006) | 1 line

  0.96.D485 - Fix MergeFlags() handling of None values. (John Pye)
........
  r1692 | stevenknight | 2006-11-08 17:15:05 -0600 (Wed, 08 Nov 2006) | 1 line

  0.96.D486 - Directly execute commands on Windows when possible. (Jay Kint)
........
  r1693 | stevenknight | 2006-11-08 18:54:49 -0600 (Wed, 08 Nov 2006) | 1 line

  0.96.D487 - Remove the semi-colon from the list of characters that determine when we use cmd
........
  r1694 | stevenknight | 2006-11-09 01:34:06 -0600 (Thu, 09 Nov 2006) | 1 line

  0.96.D488 - Pick up latex/bibtex 'Rerun to get citations correct' messages. (Dmitry Mikhin)
........
  r1695 | stevenknight | 2006-11-11 08:36:33 -0600 (Sat, 11 Nov 2006) | 1 line

  0.96.D489 - Back out the direct-execution-on-Windows change until we solve a corner case.
........
  r1696 | stevenknight | 2006-11-15 10:33:10 -0600 (Wed, 15 Nov 2006) | 1 line

  0.96.D490 - Fix the sconsign script when the .sconsign.dblite file is specified with its suf
........
  r1697 | stevenknight | 2006-11-18 10:45:50 -0600 (Sat, 18 Nov 2006) | 4 lines

  Complete move of test/sconsign/script.py to underneath test/sconsign/script/.
  (This got left out of the previous checkin due to an error in the
  script that resubmits Aegis changes to Subversion.)
........
  r1698 | stevenknight | 2006-11-18 11:05:26 -0600 (Sat, 18 Nov 2006) | 1 line

  0.96.D491 - Allow an Options converter to take the construction environment as a parameter.
........
  r1699 | stevenknight | 2006-11-30 15:34:37 -0600 (Thu, 30 Nov 2006) | 1 line

  0.96.D492 - Reverse the order in which we try the arguments Options converters, first a sing
........
  r1700 | stevenknight | 2006-11-30 16:03:09 -0600 (Thu, 30 Nov 2006) | 1 line

  0.96.D493 - Speed up rel_path() by avoiding recomputation of intermediate directory relative
........
  r1701 | stevenknight | 2006-11-30 16:14:16 -0600 (Thu, 30 Nov 2006) | 1 line

  0.96.D494 - More efficient get_suffix(): compute it once when we set the name.
........
  r1702 | stevenknight | 2006-11-30 16:22:55 -0600 (Thu, 30 Nov 2006) | 1 line

  0.96.D495 - Fix missing XML end tags.
........
  r1703 | stevenknight | 2006-11-30 17:15:25 -0600 (Thu, 30 Nov 2006) | 1 line

  0.96.D496 - Turn Memoizer into a simple counter for --debug=memoizer, not something that doe
........
  r1704 | stevenknight | 2006-11-30 20:30:50 -0600 (Thu, 30 Nov 2006) | 1 line

  0.96.D497 - Add the scons-time script, with doc and tests.
........
  r1705 | stevenknight | 2006-11-30 23:28:20 -0600 (Thu, 30 Nov 2006) | 1 line

  0.96.D498 - Update the copyright years string.
........
  r1706 | stevenknight | 2006-12-01 11:54:22 -0600 (Fri, 01 Dec 2006) | 1 line

  0.96.D499 - Fix _do_Lookup => _doLookup value-caching misspellings. (Ben Leslie)
........
  r1707 | stevenknight | 2006-12-01 12:03:46 -0600 (Fri, 01 Dec 2006) | 1 line

  0.96.D500 - Fix copyright test against debian build. (Walter Franzini)
........
  r1708 | stevenknight | 2006-12-01 14:23:29 -0600 (Fri, 01 Dec 2006) | 1 line

  0.96.D501 - Add #include lines for test portability. (Gary Oberbrunner)
........
  r1709 | stevenknight | 2006-12-01 14:51:12 -0600 (Fri, 01 Dec 2006) | 1 line

  0.96.D502 - Fix tests under Python versions with no profiler (pstats module).
........
  r1710 | stevenknight | 2006-12-01 20:04:49 -0600 (Fri, 01 Dec 2006) | 1 line

  0.96.D503 - Remove unnecessary os.path.normpath() calls. (Gary Oberbrunner)
........
  r1711 | stevenknight | 2006-12-01 20:34:31 -0600 (Fri, 01 Dec 2006) | 1 line

  0.96.D504 - Accomodate arbitray white space after a SWIG %module keyword. (Anonymous)
........
  r1712 | stevenknight | 2006-12-05 14:49:54 -0600 (Tue, 05 Dec 2006) | 1 line

  0.96.D506 - Cache substitutions of of Builder source suffixes. Use a new PathList module, and a refactor Node.FS.Rfindalldirs() method, to cache calculations of values like CPPPATH.
........
  r1713 | stevenknight | 2006-12-05 18:43:36 -0600 (Tue, 05 Dec 2006) | 1 line

  0.96.D507 - Use cached stat() values in diskchecks.
........
  r1714 | stevenknight | 2006-12-05 21:11:24 -0600 (Tue, 05 Dec 2006) | 1 line

  0.96.D508 - Fix Memoizer hit counts for methods memoizing simple values. Clean up the code for memoizing return values in a dictionary. Fix comments.
........
  r1715 | stevenknight | 2006-12-06 07:23:18 -0600 (Wed, 06 Dec 2006) | 1 line

  0.96.D369 - More efficient Node.FS.Dir.current() check. Fix some Windows test portability issues.
........
  r1716 | stevenknight | 2006-12-06 12:24:32 -0600 (Wed, 06 Dec 2006) | 2 lines

  Undo previous checkin (distributed incorrect Aegis change number).
........
  r1717 | stevenknight | 2006-12-06 12:34:53 -0600 (Wed, 06 Dec 2006) | 1 line

  0.96.D505 - Update ae-{cvs,svn}-ci for newer versions of aetar, and to not truncate descriptions.
........
  r1718 | stevenknight | 2006-12-07 23:01:41 -0600 (Thu, 07 Dec 2006) | 1 line

  0.96.D509 - Only look for mslink on Windows systems. (Sohail Somani)
........
  r1719 | stevenknight | 2006-12-07 23:18:33 -0600 (Thu, 07 Dec 2006) | 1 line

  0.96.D510 - Have the D compiler Tool use the same  logic for shared libraries, too. (Paolo Invernizzi)
........
  r1720 | stevenknight | 2006-12-07 23:29:47 -0600 (Thu, 07 Dec 2006) | 1 line

  0.96.D511 - Generalize a JobTests.py test so it doesn't assume a specific order in which the operating system executes the threads.
........
  r1721 | stevenknight | 2006-12-07 23:39:37 -0600 (Thu, 07 Dec 2006) | 1 line

  0.96.D512 - Back out the Tool/dmd.py  change; it breaks shared library linking for other lanuages beside D in the construction environment.
........
  r1722 | stevenknight | 2006-12-07 23:47:11 -0600 (Thu, 07 Dec 2006) | 1 line

  0.96.D513 - Test fixes: Windows portability, handle changes to Python 2.5 messages.
........
  r1723 | stevenknight | 2006-12-08 00:00:13 -0600 (Fri, 08 Dec 2006) | 1 line

  0.96.D514 - Change how the 'as' Tool is imported to accomodate the Python 2.6 'as' keyword.
........
  r1724 | stevenknight | 2006-12-08 11:19:27 -0600 (Fri, 08 Dec 2006) | 1 line

  0.96.D515 - Cache both Node.FS.find_file() and Node.FS.Dri.srcdir_find_file().
........
  r1725 | stevenknight | 2006-12-08 17:27:35 -0600 (Fri, 08 Dec 2006) | 1 line

  0.96.D516 - Better error when we try to fetch contents from an Entry that doesn't exist. (Tom Parker)
........
  r1726 | stevenknight | 2006-12-08 23:28:55 -0600 (Fri, 08 Dec 2006) | 1 line

  0.96.D517 - Make sure we pick up the scons-local directory regardless of where we chdir internally.
........
  r1727 | stevenknight | 2006-12-11 16:25:53 -0600 (Mon, 11 Dec 2006) | 1 line

  0.96.D518 - Cache results of Executor.get_unignored_sources() and Executor.process_sources(). Eliminate some map() and disambiguate() calls when scanning for implicit dependencies.
........
  r1728 | stevenknight | 2006-12-12 14:32:22 -0600 (Tue, 12 Dec 2006) | 1 line

  0.96.D519 - Fix SideEffect() when -j is used.
........
  r1729 | stevenknight | 2006-12-12 16:58:15 -0600 (Tue, 12 Dec 2006) | 1 line

  0.96.D520 - Add a srcdir keyword to Builder calls.
........
  r1730 | stevenknight | 2006-12-12 21:40:59 -0600 (Tue, 12 Dec 2006) | 1 line

  0.96.D521 - TeX/LaTeX updates, including handling files in subdirectories. (Joel B. Mohler, Rob Managan, Dmitry Mikhin)
........
  r1731 | stevenknight | 2006-12-14 15:01:02 -0600 (Thu, 14 Dec 2006) | 1 line

  0.96.D522 - Propogate TypeErrors during variable substitution for display to the user.
........
  r1732 | stevenknight | 2006-12-14 20:01:49 -0600 (Thu, 14 Dec 2006) | 1 line

  0.96.D523 - Fix the os.path.join() calls in EnvironmentTests.py.
........
  r1733 | stevenknight | 2006-12-15 07:48:22 -0600 (Fri, 15 Dec 2006) | 1 line

  0.96.D524 - Fix source directories as dependencies of an Alias (0.96.93 problem found by LilyPond).
........
  r1735 | stevenknight | 2006-12-15 12:43:45 -0600 (Fri, 15 Dec 2006) | 1 line

  0.96.D525 - Allow printing Debug.caller() output (or other end-of-run debugging info) when using -h.
........
  r1736 | stevenknight | 2006-12-15 16:30:08 -0600 (Fri, 15 Dec 2006) | 1 line

  0.96.D526 - Add an option to debug IndexError and NameError exceptions during variable substitution.
........

1 files changed, 217 insertions, 0 deletions

diff --git a/src/engine/SCons/PathList.py b/src/engine/SCons/PathList.py
new file mode 100644
index 0000000..b757bd3
--- /dev/null
+++ b/src/engine/SCons/PathList.py
@@ -0,0 +1,217 @@
+#
+# __COPYRIGHT__
+#
+# Permission is hereby granted, free of charge, to any person obtaining
+# a copy of this software and associated documentation files (the
+# "Software"), to deal in the Software without restriction, including
+# without limitation the rights to use, copy, modify, merge, publish,
+# distribute, sublicense, and/or sell copies of the Software, and to
+# permit persons to whom the Software is furnished to do so, subject to
+# the following conditions:
+#
+# The above copyright notice and this permission notice shall be included
+# in all copies or substantial portions of the Software.
+#
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY
+# KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE
+# WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+# NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
+# LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
+# OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
+# WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
+#
+
+__revision__ = "__FILE__ __REVISION__ __DATE__ __DEVELOPER__"
+
+__doc__ = """SCons.PathList
+
+A module for handling lists of directory paths (the sort of things
+that get set as CPPPATH, LIBPATH, etc.) with as much caching of data and
+efficiency as we can while still keeping the evaluation delayed so that we
+Do the Right Thing (almost) regardless of how the variable is specified.
+
+"""
+
+import os
+import string
+
+import SCons.Util
+
+#
+# Variables to specify the different types of entries in a PathList object:
+#
+
+TYPE_STRING_NO_SUBST = 0        # string with no '$'
+TYPE_STRING_SUBST = 1           # string containing '$'
+TYPE_OBJECT = 2                 # other object
+
+def node_conv(obj):
+    """
+    This is the "string conversion" routine that we have our substitutions
+    use to return Nodes, not strings.  This relies on the fact that an
+    EntryProxy object has a get() method that returns the underlying
+    Node that it wraps, which is a bit of architectural dependence
+    that we might need to break or modify in the future in response to
+    additional requirements.
+    """
+    try:
+        get = obj.get
+    except AttributeError:
+        pass
+    else:
+        obj = get()
+    return obj
+
+class _PathList:
+    """
+    An actual PathList object.
+    """
+    def __init__(self, pathlist):
+        """
+        Initializes a PathList object, canonicalizing the input and
+        pre-processing it for quicker substitution later.
+
+        The stored representation of the PathList is a list of tuples
+        containing (type, value), where the "type" is one of the TYPE_*
+        variables defined above.  We distinguish between:
+
+            strings that contain no '$' and therefore need no
+            delayed-evaluation string substitution (we expect that there
+            will be many of these and that we therefore get a pretty
+            big win from avoiding string substitution)
+
+            strings that contain '$' and therefore need substitution
+            (the hard case is things like '${TARGET.dir}/include',
+            which require re-evaluation for every target + source)
+
+            other objects (which may be something like an EntryProxy
+            that needs a method called to return a Node)
+
+        Pre-identifying the type of each element in the PathList up-front
+        and storing the type in the list of tuples is intended to reduce
+        the amount of calculation when we actually do the substitution
+        over and over for each target.
+        """
+        if SCons.Util.is_String(pathlist):
+            pathlist = string.split(pathlist, os.pathsep)
+        elif SCons.Util.is_List(pathlist) or SCons.Util.is_Tuple(pathlist):
+            pathlist = SCons.Util.flatten(pathlist)
+        else:
+            pathlist = [pathlist]
+
+        pl = []
+        for p in pathlist:
+            try:
+                index = string.find(p, '$')
+            except (AttributeError, TypeError):
+                type = TYPE_OBJECT
+            else:
+                if index == -1:
+                    type = TYPE_STRING_NO_SUBST
+                else:
+                    type = TYPE_STRING_SUBST
+            pl.append((type, p))
+
+        self.pathlist = tuple(pl)
+
+    def __len__(self): return len(self.pathlist)
+
+    def __getitem__(self, i): return self.pathlist[i]
+
+    def subst_path(self, env, target, source):
+        """
+        Performs construction variable substitution on a pre-digested
+        PathList for a specific target and source.
+        """
+        result = []
+        for type, value in self.pathlist:
+            if type == TYPE_STRING_SUBST:
+                value = env.subst(value, target=target, source=source,
+                                  conv=node_conv)
+            elif type == TYPE_OBJECT:
+                value = node_conv(value)
+            result.append(value)
+        return tuple(result)
+
+
+class PathListCache:
+    """
+    A class to handle caching of PathList lookups.
+
+    This class gets instantiated once and then deleted from the namespace,
+    so it's used as a Singleton (although we don't enforce that in the
+    usual Pythonic ways).  We could have just made the cache a dictionary
+    in the module namespace, but putting it in this class allows us to
+    use the same Memoizer pattern that we use elsewhere to count cache
+    hits and misses, which is very valuable.
+
+    Lookup keys in the cache are computed by the _PathList_key() method.
+    Cache lookup should be quick, so we don't spend cycles canonicalizing
+    all forms of the same lookup key.  For example, 'x:y' and ['x',
+    'y'] logically represent the same list, but we don't bother to
+    split string representations and treat those two equivalently.
+    (Note, however, that we do, treat lists and tuples the same.)
+
+    The main type of duplication we're trying to catch will come from
+    looking up the same path list from two different clones of the
+    same construction environment.  That is, given
+    
+        env2 = env1.Clone()
+
+    both env1 and env2 will have the same CPPPATH value, and we can
+    cheaply avoid re-parsing both values of CPPPATH by using the
+    common value from this cache.
+    """
+    if SCons.Memoize.use_memoizer:
+        __metaclass__ = SCons.Memoize.Memoized_Metaclass
+
+    memoizer_counters = []
+
+    def __init__(self):
+        self._memo = {}
+
+    def _PathList_key(self, pathlist):
+        """
+        Returns the key for memoization of PathLists.
+
+        Note that we want this to be quick, so we don't canonicalize
+        all forms of the same list.  For example, 'x:y' and ['x', 'y']
+        logically represent the same list, but we're not going to bother
+        massaging strings into canonical lists here.
+
+        The reason
+
+        """
+        if SCons.Util.is_List(pathlist):
+            pathlist = tuple(pathlist)
+        return pathlist
+        
+    memoizer_counters.append(SCons.Memoize.CountDict('PathList', _PathList_key))
+
+    def PathList(self, pathlist):
+        """
+        Returns the cached _PathList object for the specified pathlist,
+        creating and caching a new object as necessary.
+        """
+        pathlist = self._PathList_key(pathlist)
+        try:
+            memo_dict = self._memo['PathList']
+        except KeyError:
+            memo_dict = {}
+            self._memo['PathList'] = memo_dict
+        else:
+            try:
+                return memo_dict[pathlist]
+            except KeyError:
+                pass
+
+        result = _PathList(pathlist)
+
+        memo_dict[pathlist] = result
+
+        return result
+
+PathList = PathListCache().PathList
+
+
+del PathListCache