diff options
author | Steven Knight <knight@baldmt.com> | 2006-11-02 16:21:36 (GMT) |
---|---|---|
committer | Steven Knight <knight@baldmt.com> | 2006-11-02 16:21:36 (GMT) |
commit | 6fcc7d78858a5ca068e3deea183d35295b7dd88e (patch) | |
tree | ea835f8b96e6c67766b4c0e6437f363d76164d39 /src/engine | |
parent | 66edd5ceedb4e6b83e6b4d03e244af74fe720041 (diff) | |
download | SCons-6fcc7d78858a5ca068e3deea183d35295b7dd88e.zip SCons-6fcc7d78858a5ca068e3deea183d35295b7dd88e.tar.gz SCons-6fcc7d78858a5ca068e3deea183d35295b7dd88e.tar.bz2 |
Merged revisions 1582-1665 via svnmerge from
http://scons.tigris.org/svn/scons/branches/core
........
r1585 | stevenknight | 2006-08-06 21:21:12 -0500 (Sun, 06 Aug 2006) | 1 line
0.96.D430 - Fix bug with finding Fortran modules in build directories. (Nicolas Vigier)
........
r1586 | stevenknight | 2006-08-06 22:54:39 -0500 (Sun, 06 Aug 2006) | 1 line
0.96.D431 - Fix use of BuildDir when the source file is a relative-path symlink. (Nicola Vi
........
r1587 | timot | 2006-08-10 14:45:00 -0500 (Thu, 10 Aug 2006) | 1 line
fix Platform SDK init
........
r1589 | stevenknight | 2006-08-12 13:28:51 -0500 (Sat, 12 Aug 2006) | 1 line
0.96.D432 - Change the default mingw linker from g++ to gcc. (Mariusz Olko)
........
r1590 | stevenknight | 2006-08-13 11:16:32 -0500 (Sun, 13 Aug 2006) | 1 line
0.96.D433 - More runtest.py enhancements.
........
r1594 | stevenknight | 2006-08-15 04:47:46 -0500 (Tue, 15 Aug 2006) | 1 line
0.96.D434 - Print the full path of SConscript files in stack traces. (Dobes Vandermeer)
........
r1600 | timot | 2006-08-16 11:34:44 -0500 (Wed, 16 Aug 2006) | 1 line
add M4 to mingw tools
........
r1611 | stevenknight | 2006-08-19 16:25:24 -0500 (Sat, 19 Aug 2006) | 1 line
0.96.D435 - Add an SCons XMLResultStream to capture test results for mailing in. Get Aegis
........
r1617 | timot | 2006-08-21 16:19:03 -0500 (Mon, 21 Aug 2006) | 1 line
handling of spawnve returning an error code that is not in exitvalmap
........
r1619 | stevenknight | 2006-09-01 19:07:47 -0500 (Fri, 01 Sep 2006) | 1 line
0.96.D436 - Win32 test portability and other test fixes.
........
r1620 | stevenknight | 2006-09-02 20:21:51 -0500 (Sat, 02 Sep 2006) | 2 lines
Bring msvc.py in sync with Aegis repository.
........
r1621 | stevenknight | 2006-09-02 20:23:48 -0500 (Sat, 02 Sep 2006) | 2 lines
Move to keep symlink tests together.
........
r1622 | garyo | 2006-09-06 11:51:42 -0500 (Wed, 06 Sep 2006) | 1 line
Fix for Issue #1370; allow exit values not in exitvalmap. Added some tests for this kind of thing. Also improved win32 err msg if command exits with nonzero to show actual cmd, not just "cmd.exe". Note this fix improves posix and win32 behavior.
........
r1623 | stevenknight | 2006-09-07 06:35:16 -0500 (Thu, 07 Sep 2006) | 1 line
0.96.D440 - Fix runtest.py with QMTest on Windows.
........
r1625 | stevenknight | 2006-09-09 09:22:15 -0500 (Sat, 09 Sep 2006) | 3 lines
Comment out a long-command test which fails on older Pythons (1.5.2)
on Fedora Core 3. We can restore it in the future.
........
r1626 | stevenknight | 2006-09-09 16:17:44 -0500 (Sat, 09 Sep 2006) | 1 line
0.96.D441 - Allow Python Values to be the targets of Builders. (Anonymous)
........
r1627 | stevenknight | 2006-09-09 20:25:53 -0500 (Sat, 09 Sep 2006) | 1 line
0.96.D442 - Support src_dir on SConscript() calls. (Dobes Vandermeer)
........
r1628 | stevenknight | 2006-09-10 07:28:54 -0500 (Sun, 10 Sep 2006) | 1 line
0.96.D443 - Add miscellaneous utility scripts and config changes.
........
r1629 | stevenknight | 2006-09-11 04:45:01 -0500 (Mon, 11 Sep 2006) | 1 line
0.96.D444 - Add a test case for BuildDir handling of nested SConscript files. (Adam Simpkin
........
r1630 | stevenknight | 2006-09-11 11:34:07 -0500 (Mon, 11 Sep 2006) | 1 line
0.96.D445 - Workaround bug in early versions of thePython 2.4 profiler.
........
r1631 | stevenknight | 2006-09-19 19:12:51 -0500 (Tue, 19 Sep 2006) | 1 line
0.96.D446 - Fix Visual Studio common prefix handling to only treat common prefixes on comple
........
r1632 | stevenknight | 2006-09-25 07:11:44 -0500 (Mon, 25 Sep 2006) | 1 line
0.96.D447 - Fix tests that fail due to warnings from (some versions?) of gcc. (Sohail Soman
........
r1633 | stevenknight | 2006-09-25 07:57:48 -0500 (Mon, 25 Sep 2006) | 1 line
0.96.D448 - Handle python paths with quotes in tests.
........
r1634 | stevenknight | 2006-09-25 14:38:07 -0500 (Mon, 25 Sep 2006) | 1 line
0.96.D449 - Fix SCons build when python is not in the path (e.g. on Windows). (Chad Austin)
........
r1635 | stevenknight | 2006-09-26 11:28:23 -0500 (Tue, 26 Sep 2006) | 1 line
0.96.D450 - Handle warnings from Python 2.1; make sure we still test on Python 1.5.
........
r1636 | stevenknight | 2006-09-27 05:34:23 -0500 (Wed, 27 Sep 2006) | 1 line
0.96.D451 - Avoid calling Options validators and converters twice.
........
r1637 | stevenknight | 2006-09-28 08:12:38 -0500 (Thu, 28 Sep 2006) | 1 line
0.96.D452 - Allow setting MSVS_VERSION after initialization to select the Visual Studio vers
........
r1638 | stevenknight | 2006-09-30 08:38:15 -0500 (Sat, 30 Sep 2006) | 1 line
0.96.D453 - Give the MSVC resource builder a src_builder list and .rc src_suffix. (Leanid N
........
r1639 | stevenknight | 2006-10-12 08:50:58 -0500 (Thu, 12 Oct 2006) | 1 line
0.96.D454 - Test handling of env.Append() and env.Prepend(), making sure it works on later P
........
r1640 | stevenknight | 2006-10-15 20:42:09 -0500 (Sun, 15 Oct 2006) | 1 line
0.96.D455 - Support the runtest.py -f option when using QMTest.
........
r1641 | stevenknight | 2006-10-15 21:20:02 -0500 (Sun, 15 Oct 2006) | 1 line
0.96.D456 - Fix an error in ListOption handling caused by making new copies of Options objec
........
r1642 | stevenknight | 2006-10-16 05:53:14 -0500 (Mon, 16 Oct 2006) | 1 line
0.96.D457 - Fix new Append()/Prepend() handling of dictionaries in later Python versions (2.
........
r1643 | stevenknight | 2006-10-16 07:13:16 -0500 (Mon, 16 Oct 2006) | 1 line
0.96.D458 - Allow Install() to handle directories as sources. (Matthew A. Nicholson)
........
r1644 | stevenknight | 2006-10-17 09:17:58 -0500 (Tue, 17 Oct 2006) | 1 line
0.96.D459 - Add a test to make sure SideEffect() doesn't interfere with CacheDir(). Refacto
........
r1645 | stevenknight | 2006-10-17 10:20:22 -0500 (Tue, 17 Oct 2006) | 1 line
0.96.D460 - Do not use -fPIC when using gcc on win32 (MinGW). (Jan Nijtmans)
........
r1646 | stevenknight | 2006-10-17 17:21:58 -0500 (Tue, 17 Oct 2006) | 6 lines
Move all the scons.org stuff from the scons source tree itself to a
directory next to the trunk, and delete the copies from the branches.
There's a lot of stuff there (what with all of the documentation of
the different versions) and it's ridiculous to make everyone sync it
just to work on the code.
........
r1647 | stevenknight | 2006-10-17 23:18:29 -0500 (Tue, 17 Oct 2006) | 1 line
0.96.D461 - Fix the tests of runtest.py so they skip appropriately if qmtest.py isn't instal
........
r1648 | stevenknight | 2006-10-18 08:48:47 -0500 (Wed, 18 Oct 2006) | 1 line
0.96.D462 - When using --implicit-cache, do not re-scan files if the scanner returned no imp
........
r1649 | stevenknight | 2006-10-18 19:42:13 -0500 (Wed, 18 Oct 2006) | 1 line
0.96.D463 - More test portability fixes.
........
r1650 | stevenknight | 2006-10-19 00:30:23 -0500 (Thu, 19 Oct 2006) | 1 line
0.96.D464 - Add a cpp.py module that knows how to find dependencies from #include lines like
........
r1651 | stevenknight | 2006-10-20 06:49:51 -0500 (Fri, 20 Oct 2006) | 1 line
0.96.D465 - Fix unresolved variable name in win32 portion of test.
........
r1652 | stevenknight | 2006-10-23 00:20:38 -0500 (Mon, 23 Oct 2006) | 1 line
0.96.D466 - Add an option for tracing files to and from the CacheDir.
........
r1653 | stevenknight | 2006-10-23 00:29:32 -0500 (Mon, 23 Oct 2006) | 1 line
0.96.D467 - Make {Append,Prepend}Unique() handle adding elements to empty lists like {Append
........
r1654 | stevenknight | 2006-10-23 02:38:06 -0500 (Mon, 23 Oct 2006) | 1 line
0.96.D468 - Allow Debug.caller() to take multiple arguments; add a debug utility to post-pro
........
r1655 | stevenknight | 2006-10-23 03:16:42 -0500 (Mon, 23 Oct 2006) | 1 line
0.96.D469 - Reduce unnecessary calls to Node.FS.disambiguate(), undoing (?) a performance hi
........
r1656 | stevenknight | 2006-10-25 00:06:27 -0500 (Wed, 25 Oct 2006) | 1 line
0.96.D470 - More test portability fixes.
........
r1657 | stevenknight | 2006-10-25 00:16:22 -0500 (Wed, 25 Oct 2006) | 1 line
0.96.D471 - Have runtest.py fall back to the --noqmtest option (with a warning) if qmtest.py
........
r1658 | stevenknight | 2006-10-25 12:12:02 -0500 (Wed, 25 Oct 2006) | 1 line
0.96.D472 - Document the default use of the /Z7 flag for Visual Studio and ways to use /Zi.
........
r1659 | stevenknight | 2006-10-26 23:53:51 -0500 (Thu, 26 Oct 2006) | 1 line
0.96.D473 - Have runtest.py -d accomodate different Python library locations.
........
r1660 | stevenknight | 2006-10-27 00:03:59 -0500 (Fri, 27 Oct 2006) | 1 line
0.96.D474 - Patch to support running SCons under WingIDE. (Allen Bierbaum)
........
r1661 | stevenknight | 2006-10-27 12:17:27 -0500 (Fri, 27 Oct 2006) | 1 line
0.96.D475 - Restore execution of all Environment unit tests.
........
r1662 | stevenknight | 2006-10-31 23:22:58 -0600 (Tue, 31 Oct 2006) | 1 line
0.96.D476 - Eliminate unnecessary print from a test, left over from debugging.
........
r1663 | stevenknight | 2006-10-31 23:32:00 -0600 (Tue, 31 Oct 2006) | 1 line
0.96.D477 - Support creating shared object files from assembly language. (James Y. Knight)
........
r1664 | stevenknight | 2006-10-31 23:44:08 -0600 (Tue, 31 Oct 2006) | 1 line
0.96.D478 - Fix the Memoizer to deal with a compiled .pyo or .pyc file that's in a different
........
r1665 | stevenknight | 2006-11-01 21:59:18 -0600 (Wed, 01 Nov 2006) | 1 line
0.96.D479 - Put back the scons-{LICENSE,README} files in the scons-loacal package; add tests
........
Diffstat (limited to 'src/engine')
32 files changed, 1876 insertions, 288 deletions
diff --git a/src/engine/MANIFEST.in b/src/engine/MANIFEST.in index 9d6c972..c762f1c 100644 --- a/src/engine/MANIFEST.in +++ b/src/engine/MANIFEST.in @@ -2,6 +2,7 @@ SCons/__init__.py SCons/Action.py SCons/Builder.py SCons/Conftest.py +SCons/cpp.py SCons/dblite.py SCons/Debug.py SCons/Defaults.py diff --git a/src/engine/SCons/ActionTests.py b/src/engine/SCons/ActionTests.py index 1085586..57910de 100644 --- a/src/engine/SCons/ActionTests.py +++ b/src/engine/SCons/ActionTests.py @@ -182,6 +182,8 @@ if os.name == 'java': else: python = sys.executable +_python_ = '"' + python + '"' + class ActionTestCase(unittest.TestCase): """Test the Action() factory function""" @@ -903,7 +905,7 @@ class CommandActionTestCase(unittest.TestCase): except AttributeError: env = Environment() - cmd1 = r'%s %s %s xyzzy' % (python, act_py, outfile) + cmd1 = r'%s %s %s xyzzy' % (_python_, act_py, outfile) act = SCons.Action.CommandAction(cmd1) r = act([], [], env.Copy()) @@ -911,7 +913,7 @@ class CommandActionTestCase(unittest.TestCase): c = test.read(outfile, 'r') assert c == "act.py: 'xyzzy'\n", c - cmd2 = r'%s %s %s $TARGET' % (python, act_py, outfile) + cmd2 = r'%s %s %s $TARGET' % (_python_, act_py, outfile) act = SCons.Action.CommandAction(cmd2) r = act(DummyNode('foo'), [], env.Copy()) @@ -919,7 +921,7 @@ class CommandActionTestCase(unittest.TestCase): c = test.read(outfile, 'r') assert c == "act.py: 'foo'\n", c - cmd3 = r'%s %s %s ${TARGETS}' % (python, act_py, outfile) + cmd3 = r'%s %s %s ${TARGETS}' % (_python_, act_py, outfile) act = SCons.Action.CommandAction(cmd3) r = act(map(DummyNode, ['aaa', 'bbb']), [], env.Copy()) @@ -927,7 +929,7 @@ class CommandActionTestCase(unittest.TestCase): c = test.read(outfile, 'r') assert c == "act.py: 'aaa' 'bbb'\n", c - cmd4 = r'%s %s %s $SOURCES' % (python, act_py, outfile) + cmd4 = r'%s %s %s $SOURCES' % (_python_, act_py, outfile) act = SCons.Action.CommandAction(cmd4) r = act([], [DummyNode('one'), DummyNode('two')], env.Copy()) @@ -935,7 +937,7 @@ class CommandActionTestCase(unittest.TestCase): c = test.read(outfile, 'r') assert c == "act.py: 'one' 'two'\n", c - cmd4 = r'%s %s %s ${SOURCES[:2]}' % (python, act_py, outfile) + cmd4 = r'%s %s %s ${SOURCES[:2]}' % (_python_, act_py, outfile) act = SCons.Action.CommandAction(cmd4) sources = [DummyNode('three'), DummyNode('four'), DummyNode('five')] @@ -945,7 +947,7 @@ class CommandActionTestCase(unittest.TestCase): c = test.read(outfile, 'r') assert c == "act.py: 'three' 'four'\n", c - cmd5 = r'%s %s %s $TARGET XYZZY' % (python, act_py, outfile) + cmd5 = r'%s %s %s $TARGET XYZZY' % (_python_, act_py, outfile) act = SCons.Action.CommandAction(cmd5) env5 = Environment() @@ -978,7 +980,7 @@ class CommandActionTestCase(unittest.TestCase): def get_subst_proxy(self): return self - cmd6 = r'%s %s %s ${TARGETS[1]} $TARGET ${SOURCES[:2]}' % (python, act_py, outfile) + cmd6 = r'%s %s %s ${TARGETS[1]} $TARGET ${SOURCES[:2]}' % (_python_, act_py, outfile) act = SCons.Action.CommandAction(cmd6) r = act(target = [Obj('111'), Obj('222')], @@ -1016,31 +1018,31 @@ class CommandActionTestCase(unittest.TestCase): r = act([], [], env.Copy(out = outfile)) assert r == expect_nonexecutable, "r == %d" % r - act = SCons.Action.CommandAction('%s %s 1' % (python, exit_py)) + act = SCons.Action.CommandAction('%s %s 1' % (_python_, exit_py)) r = act([], [], env) assert r == 1, r - act = SCons.Action.CommandAction('@%s %s 1' % (python, exit_py)) + act = SCons.Action.CommandAction('@%s %s 1' % (_python_, exit_py)) r = act([], [], env) assert r == 1, r - act = SCons.Action.CommandAction('@-%s %s 1' % (python, exit_py)) + act = SCons.Action.CommandAction('@-%s %s 1' % (_python_, exit_py)) r = act([], [], env) assert r == 0, r - act = SCons.Action.CommandAction('-%s %s 1' % (python, exit_py)) + act = SCons.Action.CommandAction('-%s %s 1' % (_python_, exit_py)) r = act([], [], env) assert r == 0, r - act = SCons.Action.CommandAction('@ %s %s 1' % (python, exit_py)) + act = SCons.Action.CommandAction('@ %s %s 1' % (_python_, exit_py)) r = act([], [], env) assert r == 1, r - act = SCons.Action.CommandAction('@- %s %s 1' % (python, exit_py)) + act = SCons.Action.CommandAction('@- %s %s 1' % (_python_, exit_py)) r = act([], [], env) assert r == 0, r - act = SCons.Action.CommandAction('- %s %s 1' % (python, exit_py)) + act = SCons.Action.CommandAction('- %s %s 1' % (_python_, exit_py)) r = act([], [], env) assert r == 0, r @@ -1074,7 +1076,7 @@ class CommandActionTestCase(unittest.TestCase): # test redirection operators def test_redirect(self, redir, stdout_msg, stderr_msg): - cmd = r'%s %s %s xyzzy %s' % (python, act_py, outfile, redir) + cmd = r'%s %s %s xyzzy %s' % (_python_, act_py, outfile, redir) # Write the output and error messages to files because # Windows can't handle strings that are too big in its # external environment (os.spawnve() returns EINVAL, @@ -1571,7 +1573,7 @@ class ListActionTestCase(unittest.TestCase): a([], [], Environment(s = self)) assert self.inc == 3, self.inc - cmd2 = r'%s %s %s syzygy' % (python, act_py, outfile) + cmd2 = r'%s %s %s syzygy' % (_python_, act_py, outfile) def function2(target, source, env): open(env['out'], 'a').write("function2\n") @@ -1644,7 +1646,7 @@ class LazyActionTestCase(unittest.TestCase): a = SCons.Action.Action('$BAR') a([], [], env=Environment(BAR = f, s = self)) assert self.test == 1, self.test - cmd = r'%s %s %s lazy' % (python, act_py, outfile) + cmd = r'%s %s %s lazy' % (_python_, act_py, outfile) a([], [], env=Environment(BAR = cmd, s = self)) c = test.read(outfile, 'r') assert c == "act.py: 'lazy'\n", c diff --git a/src/engine/SCons/Debug.py b/src/engine/SCons/Debug.py index 1cff9c6..47d2134 100644 --- a/src/engine/SCons/Debug.py +++ b/src/engine/SCons/Debug.py @@ -119,20 +119,22 @@ else: caller_dicts = {} -def caller(back=0): +def caller(*backlist): import traceback - tb = traceback.extract_stack(limit=3+back) - key = tb[1][:3] - try: - entry = caller_dicts[key] - except KeyError: - entry = caller_dicts[key] = {} - key = tb[0][:3] - try: - entry[key] = entry[key] + 1 - except KeyError: - entry[key] = 1 - return '%s:%d(%s)' % func_shorten(key) + if not backlist: + backlist = [0] + result = [] + for back in backlist: + tb = traceback.extract_stack(limit=3+back) + key = tb[1][:3] + try: + entry = caller_dicts[key] + except KeyError: + entry = caller_dicts[key] = {} + key = tb[0][:3] + entry[key] = entry.get(key, 0) + 1 + result.append('%s:%d(%s)' % func_shorten(key)) + return result def dump_caller_counts(file=sys.stdout): keys = caller_dicts.keys() diff --git a/src/engine/SCons/Defaults.py b/src/engine/SCons/Defaults.py index cb628b8..7513c0d 100644 --- a/src/engine/SCons/Defaults.py +++ b/src/engine/SCons/Defaults.py @@ -173,14 +173,34 @@ Touch = ActionFactory(touch_func, lambda file: 'Touch("%s")' % file) # Internal utility functions -def copyFunc(dest, source, env): - """Install a source file into a destination by copying it (and its - permission/mode bits).""" - shutil.copy2(source, dest) - st = os.stat(source) - os.chmod(dest, stat.S_IMODE(st[stat.ST_MODE]) | stat.S_IWRITE) +def installFunc(dest, source, env): + """Install a source file or directory into a destination by copying, + (including copying permission/mode bits).""" + + if os.path.isdir(source): + if os.path.exists(dest): + if not os.path.isdir(dest): + raise SCons.Errors.UserError, "cannot overwrite non-directory `%s' with a directory `%s'" % (str(dest), str(source)) + else: + parent = os.path.split(dest)[0] + if not os.path.exists(parent): + os.makedirs(parent) + shutil.copytree(source, dest) + else: + shutil.copy2(source, dest) + st = os.stat(source) + os.chmod(dest, stat.S_IMODE(st[stat.ST_MODE]) | stat.S_IWRITE) + return 0 +def installStr(dest, source, env): + source = str(source) + if os.path.isdir(source): + type = 'directory' + else: + type = 'file' + return 'Install %s: "%s" as "%s"' % (type, source, dest) + def _concat(prefix, list, suffix, env, f=lambda x: x, target=None, source=None): """Creates a new list from 'list' by first interpolating each element in the list using the 'env' dictionary and then calling f @@ -334,13 +354,14 @@ ConstructionEnvironment = { 'SCANNERS' : [], 'CONFIGUREDIR' : '#/.sconf_temp', 'CONFIGURELOG' : '#/config.log', - 'INSTALLSTR' : 'Install file: "$SOURCE" as "$TARGET"', 'CPPSUFFIXES' : SCons.Tool.CSuffixes, 'DSUFFIXES' : SCons.Tool.DSuffixes, + 'ENV' : {}, 'IDLSUFFIXES' : SCons.Tool.IDLSuffixes, + 'INSTALL' : installFunc, + 'INSTALLSTR' : installStr, + '_installStr' : installStr, 'LATEXSUFFIXES' : SCons.Tool.LaTeXSuffixes, - 'ENV' : {}, - 'INSTALL' : copyFunc, '_concat' : _concat, '_defines' : _defines, '_stripixes' : _stripixes, diff --git a/src/engine/SCons/Environment.py b/src/engine/SCons/Environment.py index d76f71d..7aa1909 100644 --- a/src/engine/SCons/Environment.py +++ b/src/engine/SCons/Environment.py @@ -84,7 +84,11 @@ def installFunc(target, source, env): return install(target[0].path, source[0].path, env) def installString(target, source, env): - return env.subst_target_source(env['INSTALLSTR'], 0, target, source) + s = env.get('INSTALLSTR', '') + if callable(s): + return s(target[0].path, source[0].path, env) + else: + return env.subst_target_source(s, 0, target, source) installAction = SCons.Action.Action(installFunc, installString) @@ -742,21 +746,31 @@ class Base(SubstitutionEnvironment): # environment before calling the tools, because they may use # some of them during initialization. apply(self.Replace, (), kw) + keys = kw.keys() if options: + keys = keys + options.keys() options.Update(self) + save = {} + for k in keys: + try: + save[k] = self._dict[k] + except KeyError: + # No value may have been set if they tried to pass in a + # reserved variable name like TARGETS. + pass + if tools is None: tools = self._dict.get('TOOLS', None) if tools is None: tools = ['default'] apply_tools(self, tools, toolpath) - # Now re-apply the passed-in variables and customizable options + # Now restore the passed-in variables and customized options # to the environment, since the values the user set explicitly # should override any values set by the tools. - apply(self.Replace, (), kw) - if options: - options.Update(self) + for key, val in save.items(): + self._dict[key] = val ####################################################################### # Utility methods that are primarily for internal use by SCons. @@ -895,19 +909,19 @@ class Base(SubstitutionEnvironment): self._dict[key] = val else: try: - # Most straightforward: just try to add them - # together. This will work in most cases, when the - # original and new values are of compatible types. - self._dict[key] = orig + val - except TypeError: + # Check if the original looks like a dictionary. + # If it is, we can't just try adding the value because + # dictionaries don't have __add__() methods, and + # things like UserList will incorrectly coerce the + # original dict to a list (which we don't want). + update_dict = orig.update + except AttributeError: try: - # Try to update a dictionary value with another. - # If orig isn't a dictionary, it won't have an - # update() method; if val isn't a dictionary, - # it won't have a keys() method. Either way, - # it's an AttributeError. - orig.update(val) - except AttributeError: + # Most straightforward: just try to add them + # together. This will work in most cases, when the + # original and new values are of compatible types. + self._dict[key] = orig + val + except (KeyError, TypeError): try: # Check if the original is a list. add_to_orig = orig.append @@ -925,6 +939,17 @@ class Base(SubstitutionEnvironment): # value to it (if there's a value to append). if val: add_to_orig(val) + else: + # The original looks like a dictionary, so update it + # based on what we think the value looks like. + if SCons.Util.is_List(val): + for v in val: + orig[v] = None + else: + try: + update_dict(val) + except (AttributeError, TypeError, ValueError): + orig[val] = None self.scanner_map_delete(kw) def AppendENVPath(self, name, newpath, envname = 'ENV', sep = os.pathsep): @@ -952,7 +977,7 @@ class Base(SubstitutionEnvironment): """ kw = copy_non_reserved_keywords(kw) for key, val in kw.items(): - if not self._dict.has_key(key) or not self._dict[key]: + if not self._dict.has_key(key) or self._dict[key] in ('', None): self._dict[key] = val elif SCons.Util.is_Dict(self._dict[key]) and \ SCons.Util.is_Dict(val): @@ -1134,19 +1159,19 @@ class Base(SubstitutionEnvironment): self._dict[key] = val else: try: - # Most straightforward: just try to add them - # together. This will work in most cases, when the - # original and new values are of compatible types. - self._dict[key] = val + orig - except TypeError: + # Check if the original looks like a dictionary. + # If it is, we can't just try adding the value because + # dictionaries don't have __add__() methods, and + # things like UserList will incorrectly coerce the + # original dict to a list (which we don't want). + update_dict = orig.update + except AttributeError: try: - # Try to update a dictionary value with another. - # If orig isn't a dictionary, it won't have an - # update() method; if val isn't a dictionary, - # it won't have a keys() method. Either way, - # it's an AttributeError. - orig.update(val) - except AttributeError: + # Most straightforward: just try to add them + # together. This will work in most cases, when the + # original and new values are of compatible types. + self._dict[key] = val + orig + except (KeyError, TypeError): try: # Check if the added value is a list. add_to_val = val.append @@ -1164,6 +1189,17 @@ class Base(SubstitutionEnvironment): if orig: add_to_val(orig) self._dict[key] = val + else: + # The original looks like a dictionary, so update it + # based on what we think the value looks like. + if SCons.Util.is_List(val): + for v in val: + orig[v] = None + else: + try: + update_dict(val) + except (AttributeError, TypeError, ValueError): + orig[val] = None self.scanner_map_delete(kw) def PrependENVPath(self, name, newpath, envname = 'ENV', sep = os.pathsep): @@ -1191,7 +1227,7 @@ class Base(SubstitutionEnvironment): """ kw = copy_non_reserved_keywords(kw) for key, val in kw.items(): - if not self._dict.has_key(key) or not self._dict[key]: + if not self._dict.has_key(key) or self._dict[key] in ('', None): self._dict[key] = val elif SCons.Util.is_Dict(self._dict[key]) and \ SCons.Util.is_Dict(val): @@ -1504,25 +1540,37 @@ class Base(SubstitutionEnvironment): try: dnodes = self.arg2nodes(dir, self.fs.Dir) except TypeError: - raise SCons.Errors.UserError, "Target `%s' of Install() is a file, but should be a directory. Perhaps you have the Install() arguments backwards?" % str(dir) + fmt = "Target `%s' of Install() is a file, but should be a directory. Perhaps you have the Install() arguments backwards?" + raise SCons.Errors.UserError, fmt % str(dir) try: - sources = self.arg2nodes(source, self.fs.File) + sources = self.arg2nodes(source, self.fs.Entry) except TypeError: if SCons.Util.is_List(source): - raise SCons.Errors.UserError, "Source `%s' of Install() contains one or more non-files. Install() source must be one or more files." % repr(map(str, source)) + s = repr(map(str, source)) else: - raise SCons.Errors.UserError, "Source `%s' of Install() is not a file. Install() source must be one or more files." % str(source) + s = str(source) + fmt = "Source `%s' of Install() is neither a file nor a directory. Install() source must be one or more files or directories" + raise SCons.Errors.UserError, fmt % s tgt = [] for dnode in dnodes: for src in sources: - target = self.fs.File(src.name, dnode) + target = self.fs.Entry(src.name, dnode) tgt.extend(InstallBuilder(self, target, src)) return tgt def InstallAs(self, target, source): """Install sources as targets.""" - sources = self.arg2nodes(source, self.fs.File) - targets = self.arg2nodes(target, self.fs.File) + sources = self.arg2nodes(source, self.fs.Entry) + targets = self.arg2nodes(target, self.fs.Entry) + if len(sources) != len(targets): + if not SCons.Util.is_List(target): + target = [target] + if not SCons.Util.is_List(source): + source = [source] + t = repr(map(str, target)) + s = repr(map(str, source)) + fmt = "Target (%s) and source (%s) lists of InstallAs() must be the same length." + raise SCons.Errors.UserError, fmt % (t, s) result = [] for src, tgt in map(lambda x, y: (x, y), sources, targets): result.extend(InstallBuilder(self, tgt, src)) @@ -1632,10 +1680,10 @@ class Base(SubstitutionEnvironment): else: raise SCons.Errors.UserError, "Unknown target signature type '%s'"%type - def Value(self, value): + def Value(self, value, built_value=None): """ """ - return SCons.Node.Python.Value(value) + return SCons.Node.Python.Value(value, built_value) class OverrideEnvironment(Base): """A proxy that overrides variables in a wrapped construction diff --git a/src/engine/SCons/EnvironmentTests.py b/src/engine/SCons/EnvironmentTests.py index c56f1f5..52aac23 100644 --- a/src/engine/SCons/EnvironmentTests.py +++ b/src/engine/SCons/EnvironmentTests.py @@ -23,6 +23,7 @@ __revision__ = "__FILE__ __REVISION__ __DATE__ __DEVELOPER__" +import copy import os import string import StringIO @@ -595,15 +596,17 @@ sys.exit(1) save_stderr = sys.stderr + python = '"' + sys.executable + '"' + try: - cmd = '%s %s' % (sys.executable, test.workpath('stdout.py')) + cmd = '%s %s' % (python, test.workpath('stdout.py')) output = env.backtick(cmd) assert output == 'this came from stdout.py\n', output sys.stderr = StringIO.StringIO() - cmd = '%s %s' % (sys.executable, test.workpath('stderr.py')) + cmd = '%s %s' % (python, test.workpath('stderr.py')) output = env.backtick(cmd) errout = sys.stderr.getvalue() @@ -612,7 +615,7 @@ sys.exit(1) sys.stderr = StringIO.StringIO() - cmd = '%s %s' % (sys.executable, test.workpath('fail.py')) + cmd = '%s %s' % (python, test.workpath('fail.py')) try: env.backtick(cmd) except OSError, e: @@ -747,6 +750,24 @@ class BaseTestCase(unittest.TestCase,TestEnvironmentFixture): assert not env1.has_key('__env__') assert not env2.has_key('__env__') + def test_options(self): + """Test that options only get applied once.""" + class FakeOptions: + def __init__(self, key, val): + self.calls = 0 + self.key = key + self.val = val + def keys(self): + return [self.key] + def Update(self, env): + env[self.key] = self.val + self.calls = self.calls + 1 + + o = FakeOptions('AAA', 'fake_opt') + env = Environment(options=o, AAA='keyword_arg') + assert o.calls == 1, o.calls + assert env['AAA'] == 'fake_opt', env['AAA'] + def test_get(self): """Test the get() method.""" env = self.TestEnvironment(aaa = 'AAA') @@ -1212,6 +1233,8 @@ def exists(env): b2 = Environment()['BUILDERS'] assert b1 == b2, diff_dict(b1, b2) + import UserDict + UD = UserDict.UserDict import UserList UL = UserList.UserList @@ -1243,6 +1266,18 @@ def exists(env): UL(['i7']), [''], UL(['i7', '']), UL(['i8']), UL(['']), UL(['i8', '']), + {'d1':1}, 'D1', {'d1':1, 'D1':None}, + {'d2':1}, ['D2'], {'d2':1, 'D2':None}, + {'d3':1}, UL(['D3']), {'d3':1, 'D3':None}, + {'d4':1}, {'D4':1}, {'d4':1, 'D4':1}, + {'d5':1}, UD({'D5':1}), UD({'d5':1, 'D5':1}), + + UD({'u1':1}), 'U1', UD({'u1':1, 'U1':None}), + UD({'u2':1}), ['U2'], UD({'u2':1, 'U2':None}), + UD({'u3':1}), UL(['U3']), UD({'u3':1, 'U3':None}), + UD({'u4':1}), {'U4':1}, UD({'u4':1, 'U4':1}), + UD({'u5':1}), UD({'U5':1}), UD({'u5':1, 'U5':1}), + '', 'M1', 'M1', '', ['M2'], ['M2'], '', UL(['M3']), UL(['M3']), @@ -1293,14 +1328,21 @@ def exists(env): failed = 0 while cases: input, append, expect = cases[:3] - env['XXX'] = input - env.Append(XXX = append) - result = env['XXX'] - if result != expect: + env['XXX'] = copy.copy(input) + try: + env.Append(XXX = append) + except Exception, e: if failed == 0: print - print " %s Append %s => %s did not match %s" % \ - (repr(input), repr(append), repr(result), repr(expect)) + print " %s Append %s exception: %s" % \ + (repr(input), repr(append), e) failed = failed + 1 + else: + result = env['XXX'] + if result != expect: + if failed == 0: print + print " %s Append %s => %s did not match %s" % \ + (repr(input), repr(append), repr(result), repr(expect)) + failed = failed + 1 del cases[:3] assert failed == 0, "%d Append() cases failed" % failed @@ -1399,6 +1441,33 @@ def exists(env): assert env['CCC1'] == 'c1', env['CCC1'] assert env['CCC2'] == ['c2'], env['CCC2'] + env['CLVar'] = CLVar([]) + env.AppendUnique(CLVar = 'bar') + result = env['CLVar'] + if sys.version[0] == '1': + # Python 1.5.2 has a quirky behavior where CLVar([]) actually + # matches '' and [] due to different __coerce__() semantics + # in the UserList implementation. It isn't worth a lot of + # effort to get this corner case to work identically (support + # for Python 1.5 support will die soon anyway), so just treat + # it separately for now. + assert result == 'bar', result + else: + assert isinstance(result, CLVar), repr(result) + assert result == ['bar'], result + + env['CLVar'] = CLVar(['abc']) + env.AppendUnique(CLVar = 'bar') + result = env['CLVar'] + assert isinstance(result, CLVar), repr(result) + assert result == ['abc', 'bar'], result + + env['CLVar'] = CLVar(['bar']) + env.AppendUnique(CLVar = 'bar') + result = env['CLVar'] + assert isinstance(result, CLVar), repr(result) + assert result == ['bar'], result + def test_Copy(self): """Test construction environment copying @@ -1784,6 +1853,8 @@ f5: \ def test_Prepend(self): """Test prepending to construction variables in an Environment """ + import UserDict + UD = UserDict.UserDict import UserList UL = UserList.UserList @@ -1815,6 +1886,18 @@ f5: \ UL(['i7']), [''], UL(['', 'i7']), UL(['i8']), UL(['']), UL(['', 'i8']), + {'d1':1}, 'D1', {'d1':1, 'D1':None}, + {'d2':1}, ['D2'], {'d2':1, 'D2':None}, + {'d3':1}, UL(['D3']), {'d3':1, 'D3':None}, + {'d4':1}, {'D4':1}, {'d4':1, 'D4':1}, + {'d5':1}, UD({'D5':1}), UD({'d5':1, 'D5':1}), + + UD({'u1':1}), 'U1', UD({'u1':1, 'U1':None}), + UD({'u2':1}), ['U2'], UD({'u2':1, 'U2':None}), + UD({'u3':1}), UL(['U3']), UD({'u3':1, 'U3':None}), + UD({'u4':1}), {'U4':1}, UD({'u4':1, 'U4':1}), + UD({'u5':1}), UD({'U5':1}), UD({'u5':1, 'U5':1}), + '', 'M1', 'M1', '', ['M2'], ['M2'], '', UL(['M3']), UL(['M3']), @@ -1865,14 +1948,21 @@ f5: \ failed = 0 while cases: input, prepend, expect = cases[:3] - env['XXX'] = input - env.Prepend(XXX = prepend) - result = env['XXX'] - if result != expect: + env['XXX'] = copy.copy(input) + try: + env.Prepend(XXX = prepend) + except Exception, e: if failed == 0: print - print " %s Prepend %s => %s did not match %s" % \ - (repr(input), repr(prepend), repr(result), repr(expect)) + print " %s Prepend %s exception: %s" % \ + (repr(input), repr(prepend), e) failed = failed + 1 + else: + result = env['XXX'] + if result != expect: + if failed == 0: print + print " %s Prepend %s => %s did not match %s" % \ + (repr(input), repr(prepend), repr(result), repr(expect)) + failed = failed + 1 del cases[:3] assert failed == 0, "%d Prepend() cases failed" % failed @@ -1965,6 +2055,33 @@ f5: \ assert env['CCC1'] == 'c1', env['CCC1'] assert env['CCC2'] == ['c2'], env['CCC2'] + env['CLVar'] = CLVar([]) + env.PrependUnique(CLVar = 'bar') + result = env['CLVar'] + if sys.version[0] == '1': + # Python 1.5.2 has a quirky behavior where CLVar([]) actually + # matches '' and [] due to different __coerce__() semantics + # in the UserList implementation. It isn't worth a lot of + # effort to get this corner case to work identically (support + # for Python 1.5 support will die soon anyway), so just treat + # it separately for now. + assert result == 'bar', result + else: + assert isinstance(result, CLVar), repr(result) + assert result == ['bar'], result + + env['CLVar'] = CLVar(['abc']) + env.PrependUnique(CLVar = 'bar') + result = env['CLVar'] + assert isinstance(result, CLVar), repr(result) + assert result == ['bar', 'abc'], result + + env['CLVar'] = CLVar(['bar']) + env.PrependUnique(CLVar = 'bar') + result = env['CLVar'] + assert isinstance(result, CLVar), repr(result) + assert result == ['bar'], result + def test_Replace(self): """Test replacing construction variables in an Environment @@ -2650,23 +2767,23 @@ def generate(env): for tnode in tgt: assert tnode.builder == InstallBuilder - exc_caught = None - try: - tgt = env.Install('export', 'export') - except SCons.Errors.UserError, e: - exc_caught = 1 - assert exc_caught, "UserError should be thrown when Install() target is not a file." - match = str(e) == "Source `export' of Install() is not a file. Install() source must be one or more files." - assert match, e + tgt = env.Install('export', 'build') + paths = map(str, tgt) + paths.sort() + expect = ['export/build'] + assert paths == expect, paths + for tnode in tgt: + assert tnode.builder == InstallBuilder - exc_caught = None - try: - tgt = env.Install('export', ['export', 'build/foo1']) - except SCons.Errors.UserError, e: - exc_caught = 1 - assert exc_caught, "UserError should be thrown when Install() target containins non-files." - match = str(e) == "Source `['export', 'build/foo1']' of Install() contains one or more non-files. Install() source must be one or more files." - assert match, e + tgt = env.Install('export', ['build', 'build/foo1']) + paths = map(str, tgt) + paths.sort() + expect = ['export/build', 'export/foo1'] + assert paths == expect, paths + for tnode in tgt: + assert tnode.builder == InstallBuilder + + env.File('export/foo1') exc_caught = None try: @@ -2674,8 +2791,8 @@ def generate(env): except SCons.Errors.UserError, e: exc_caught = 1 assert exc_caught, "UserError should be thrown reversing the order of Install() targets." - match = str(e) == "Target `export/foo1' of Install() is a file, but should be a directory. Perhaps you have the Install() arguments backwards?" - assert match, e + expect = "Target `export/foo1' of Install() is a file, but should be a directory. Perhaps you have the Install() arguments backwards?" + assert str(e) == expect, e def test_InstallAs(self): """Test the InstallAs method""" @@ -2956,6 +3073,9 @@ def generate(env): assert not v1 is v2 assert v1.value == v2.value + v3 = env.Value('c', 'build-c') + assert v3.value == 'c', v3.value + def test_Environment_global_variable(type): diff --git a/src/engine/SCons/Memoize.py b/src/engine/SCons/Memoize.py index 134206f..c2a3027 100644 --- a/src/engine/SCons/Memoize.py +++ b/src/engine/SCons/Memoize.py @@ -798,9 +798,14 @@ else: # Make sure filename has os.sep+'SCons'+os.sep so that # SCons.Script.find_deepest_user_frame doesn't stop here import inspect # It's OK, can't get here for Python < 2.1 + filename = inspect.getsourcefile(_MeMoIZeR_superinit) + if not filename: + # This file was compiled at a path name different from + # how it's invoked now, so just make up something. + filename = whoami('superinit', '???') superinitcode = compile( "lambda self, *args, **kw: MPI(self, cls, args, kw)", - inspect.getsourcefile(_MeMoIZeR_superinit) or '<unknown>', + filename, "eval") superinit = eval(superinitcode, {'cls':cls, diff --git a/src/engine/SCons/Node/FS.py b/src/engine/SCons/Node/FS.py index ce5bcc0..382bca3 100644 --- a/src/engine/SCons/Node/FS.py +++ b/src/engine/SCons/Node/FS.py @@ -95,10 +95,33 @@ def save_strings(val): # there should be *no* changes to the external file system(s)... # -def _copy_func(src, dest): +if hasattr(os, 'link'): + def _hardlink_func(fs, src, dst): + # If the source is a symlink, we can't just hard-link to it + # because a relative symlink may point somewhere completely + # different. We must disambiguate the symlink and then + # hard-link the final destination file. + while fs.islink(src): + link = fs.readlink(src) + if not os.path.isabs(link): + src = link + else: + src = os.path.join(os.path.dirname(src), link) + fs.link(src, dst) +else: + _hardlink_func = None + +if hasattr(os, 'symlink'): + def _softlink_func(fs, src, dst): + fs.symlink(src, dst) +else: + _softlink_func = None + +def _copy_func(fs, src, dest): shutil.copy2(src, dest) - st=os.stat(src) - os.chmod(dest, stat.S_IMODE(st[stat.ST_MODE]) | stat.S_IWRITE) + st = fs.stat(src) + fs.chmod(dest, stat.S_IMODE(st[stat.ST_MODE]) | stat.S_IWRITE) + Valid_Duplicates = ['hard-soft-copy', 'soft-hard-copy', 'hard-copy', 'soft-copy', 'copy'] @@ -113,16 +136,6 @@ def set_duplicate(duplicate): # underlying implementations. We do this inside this function, # not in the top-level module code, so that we can remap os.link # and os.symlink for testing purposes. - try: - _hardlink_func = os.link - except AttributeError: - _hardlink_func = None - - try: - _softlink_func = os.symlink - except AttributeError: - _softlink_func = None - link_dict = { 'hard' : _hardlink_func, 'soft' : _softlink_func, @@ -152,10 +165,11 @@ def LinkFunc(target, source, env): if not Link_Funcs: # Set a default order of link functions. set_duplicate('hard-soft-copy') + fs = source[0].fs # Now link the files with the previously specified order. for func in Link_Funcs: try: - func(src,dest) + func(fs, src, dest) break except (IOError, OSError): # An OSError indicates something happened like a permissions @@ -213,13 +227,15 @@ def CacheRetrieveFunc(target, source, env): t = target[0] fs = t.fs cachedir, cachefile = t.cachepath() - if fs.exists(cachefile): - if SCons.Action.execute_actions: - fs.copy2(cachefile, t.path) - st = fs.stat(cachefile) - fs.chmod(t.path, stat.S_IMODE(st[stat.ST_MODE]) | stat.S_IWRITE) - return 0 - return 1 + if not fs.exists(cachefile): + fs.CacheDebug('CacheRetrieve(%s): %s not in cache\n', t, cachefile) + return 1 + fs.CacheDebug('CacheRetrieve(%s): retrieving from %s\n', t, cachefile) + if SCons.Action.execute_actions: + fs.copy2(cachefile, t.path) + st = fs.stat(cachefile) + fs.chmod(t.path, stat.S_IMODE(st[stat.ST_MODE]) | stat.S_IWRITE) + return 0 def CacheRetrieveString(target, source, env): t = target[0] @@ -237,9 +253,18 @@ def CachePushFunc(target, source, env): fs = t.fs cachedir, cachefile = t.cachepath() if fs.exists(cachefile): - # Don't bother copying it if it's already there. + # Don't bother copying it if it's already there. Note that + # usually this "shouldn't happen" because if the file already + # existed in cache, we'd have retrieved the file from there, + # not built it. This can happen, though, in a race, if some + # other person running the same build pushes their copy to + # the cache after we decide we need to build it but before our + # build completes. + fs.CacheDebug('CachePush(%s): %s already exists in cache\n', t, cachefile) return + fs.CacheDebug('CachePush(%s): pushing to %s\n', t, cachefile) + if not fs.isdir(cachedir): fs.makedirs(cachedir) @@ -258,7 +283,6 @@ def CachePushFunc(target, source, env): SCons.Warnings.warn(SCons.Warnings.CacheWriteErrorWarning, "Unable to copy %s to cache. Cache file is %s" % (str(target), cachefile)) - return CachePush = SCons.Action.Action(CachePushFunc, None) @@ -835,6 +859,14 @@ class LocalFS: def islink(self, path): return 0 # no symlinks + if hasattr(os, 'readlink'): + def readlink(self, file): + return os.readlink(file) + else: + def readlink(self, file): + return '' + + if SCons.Memoize.use_old_memoization(): _FSBase = LocalFS class LocalFS(SCons.Memoize.Memoizer, _FSBase): @@ -1139,6 +1171,21 @@ class FS(LocalFS): result.extend(dir.get_all_rdirs()) return result + def CacheDebugWrite(self, fmt, target, cachefile): + self.CacheDebugFP.write(fmt % (target, os.path.split(cachefile)[1])) + + def CacheDebugQuiet(self, fmt, target, cachefile): + pass + + CacheDebug = CacheDebugQuiet + + def CacheDebugEnable(self, file): + if file == '-': + self.CacheDebugFP = sys.stdout + else: + self.CacheDebugFP = open(file, 'w') + self.CacheDebug = self.CacheDebugWrite + def CacheDir(self, path): self.CachePath = path @@ -1776,7 +1823,7 @@ class File(Base): __cacheable__""" if not scanner: return [] - return map(lambda N: N.disambiguate(), scanner(self, env, path)) + return scanner(self, env, path) def _createDir(self): # ensure that the directories for this node are @@ -1818,11 +1865,14 @@ class File(Base): if self.fs.cache_show: if CacheRetrieveSilent(self, [], None, execute=1) == 0: self.build(presub=0, execute=0) + self.set_state(SCons.Node.executed) return 1 elif CacheRetrieve(self, [], None, execute=1) == 0: + self.set_state(SCons.Node.executed) return 1 return None + def built(self): """Called just after this node is successfully built. __cache_reset__""" diff --git a/src/engine/SCons/Node/FSTests.py b/src/engine/SCons/Node/FSTests.py index 98e08a9..1b38ffe 100644 --- a/src/engine/SCons/Node/FSTests.py +++ b/src/engine/SCons/Node/FSTests.py @@ -425,57 +425,39 @@ class BuildDirTestCase(unittest.TestCase): class LinkSimulator : """A class to intercept os.[sym]link() calls and track them.""" - def __init__( self, duplicate ) : + def __init__( self, duplicate, link, symlink, copy ) : self.duplicate = duplicate - self._reset() - - def _reset( self ) : - """Reset the simulator if necessary""" - if not self._need_reset() : return # skip if not needed now - self.links_to_be_called = self.duplicate - - def _need_reset( self ) : - """ - Determines whether or not the simulator needs to be reset. - A reset is necessary if the object is first being initialized, - or if all three methods have been tried already. - """ - return ( - ( not hasattr( self , "links_to_be_called" ) ) - or - (self.links_to_be_called == "") - ) + self.have = {} + self.have['hard'] = link + self.have['soft'] = symlink + self.have['copy'] = copy + + self.links_to_be_called = [] + for link in string.split(self.duplicate, '-'): + if self.have[link]: + self.links_to_be_called.append(link) def link_fail( self , src , dest ) : - self._reset() - l = string.split(self.links_to_be_called, "-") - next_link = l[0] - assert next_link == "hard", \ + next_link = self.links_to_be_called.pop(0) + assert next_link == "hard", \ "Wrong link order: expected %s to be called "\ "instead of hard" % next_link - self.links_to_be_called = string.join(l[1:], '-') raise OSError( "Simulating hard link creation error." ) def symlink_fail( self , src , dest ) : - self._reset() - l = string.split(self.links_to_be_called, "-") - next_link = l[0] - assert next_link == "soft", \ + next_link = self.links_to_be_called.pop(0) + assert next_link == "soft", \ "Wrong link order: expected %s to be called "\ "instead of soft" % next_link - self.links_to_be_called = string.join(l[1:], '-') raise OSError( "Simulating symlink creation error." ) def copy( self , src , dest ) : - self._reset() - l = string.split(self.links_to_be_called, "-") - next_link = l[0] - assert next_link == "copy", \ + next_link = self.links_to_be_called.pop(0) + assert next_link == "copy", \ "Wrong link order: expected %s to be called "\ "instead of copy" % next_link - self.links_to_be_called = string.join(l[1:], '-') # copy succeeds, but use the real copy - self._real_copy(src, dest) + self.have['copy'](src, dest) # end class LinkSimulator try: @@ -485,32 +467,31 @@ class BuildDirTestCase(unittest.TestCase): pass for duplicate in SCons.Node.FS.Valid_Duplicates: - simulator = LinkSimulator(duplicate) - # save the real functions for later restoration - real_link = None - real_symlink = None try: real_link = os.link except AttributeError: - pass + real_link = None try: real_symlink = os.symlink except AttributeError: - pass + real_symlink = None real_copy = shutil.copy2 - simulator._real_copy = real_copy # the simulator needs the real one + + simulator = LinkSimulator(duplicate, real_link, real_symlink, real_copy) # override the real functions with our simulation os.link = simulator.link_fail os.symlink = simulator.symlink_fail shutil.copy2 = simulator.copy - SCons.Node.FS.set_duplicate(duplicate) - - src_foo = test.workpath('src', 'foo') - build_foo = test.workpath('build', 'foo') try: + + SCons.Node.FS.set_duplicate(duplicate) + + src_foo = test.workpath('src', 'foo') + build_foo = test.workpath('build', 'foo') + test.write(src_foo, 'src/foo\n') os.chmod(src_foo, stat.S_IRUSR) try: diff --git a/src/engine/SCons/Node/Python.py b/src/engine/SCons/Node/Python.py index 99dc5b0..a639aee 100644 --- a/src/engine/SCons/Node/Python.py +++ b/src/engine/SCons/Node/Python.py @@ -45,16 +45,18 @@ class Value(SCons.Node.Node): NodeInfo = ValueNodeInfo BuildInfo = ValueBuildInfo - def __init__(self, value): + def __init__(self, value, built_value=None): SCons.Node.Node.__init__(self) self.value = value + if not built_value is None: + self.built_value = built_value def __str__(self): return repr(self.value) - def build(self): - """A "builder" for Values.""" - pass + def build(self, **kw): + if not hasattr(self, 'built_value'): + apply (SCons.Node.Node.build, (self,), kw) current = SCons.Node.Node.children_are_up_to_date @@ -64,9 +66,23 @@ class Value(SCons.Node.Node): # are outside the filesystem: return 1 + def write(self, built_value): + """Set the value of the node.""" + self.built_value = built_value + + def read(self): + """Return the value. If necessary, the value is built.""" + self.build() + if not hasattr(self, 'built_value'): + self.built_value = self.value + return self.built_value + def get_contents(self): - """The contents of a Value are the concatenation - of all the contents of its sources with the node's value itself.""" + """By the assumption that the node.built_value is a + deterministic product of the sources, the contents of a Value + are the concatenation of all the contents of its sources. As + the value need not be built when get_contents() is called, we + cannot use the actual node.built_value.""" contents = str(self.value) for kid in self.children(None): contents = contents + kid.get_contents() diff --git a/src/engine/SCons/Node/PythonTests.py b/src/engine/SCons/Node/PythonTests.py index 0801fb3..62bcf8b 100644 --- a/src/engine/SCons/Node/PythonTests.py +++ b/src/engine/SCons/Node/PythonTests.py @@ -45,6 +45,44 @@ class ValueTestCase(unittest.TestCase): assert not v1 is v2 assert v1.value == v2.value + v3 = SCons.Node.Python.Value('c', 'cb') + assert v3.built_value == 'cb' + + def test_build(self): + """Test "building" a Value Node + """ + class fake_executor: + def __call__(self, node, exitstatfunc): + node.write('faked') + + v1 = SCons.Node.Python.Value('b', 'built') + v1.executor = fake_executor() + v1.build() + assert v1.built_value == 'built', v1.built_value + + v2 = SCons.Node.Python.Value('b') + v2.executor = fake_executor() + v2.build() + assert v2.built_value == 'faked', v2.built_value + + def test_read(self): + """Test the Value.read() method + """ + v1 = SCons.Node.Python.Value('a') + x = v1.read() + assert x == 'a', x + + def test_write(self): + """Test the Value.write() method + """ + v1 = SCons.Node.Python.Value('a') + assert v1.value == 'a', v1.value + assert not hasattr(v1, 'built_value') + + v1.write('new') + assert v1.value == 'a', v1.value + assert v1.built_value == 'new', v1.built_value + def test_get_csig(self): """Test calculating the content signature of a Value() object """ diff --git a/src/engine/SCons/Node/__init__.py b/src/engine/SCons/Node/__init__.py index bda3a48..42be5b1 100644 --- a/src/engine/SCons/Node/__init__.py +++ b/src/engine/SCons/Node/__init__.py @@ -474,6 +474,7 @@ class Node: d = filter(lambda x, seen=seen: not seen.has_key(x), n.get_found_includes(env, scanner, path)) if d: + d = map(lambda N: N.disambiguate(), d) deps.extend(d) for n in d: seen[n] = 1 @@ -538,7 +539,7 @@ class Node: # Here's where we implement --implicit-cache. if implicit_cache and not implicit_deps_changed: implicit = self.get_stored_implicit() - if implicit: + if implicit is not None: factory = build_env.get_factory(self.builder.source_factory) nodes = [] for i in implicit: diff --git a/src/engine/SCons/Options/ListOption.py b/src/engine/SCons/Options/ListOption.py index d14b22a..5aa508a 100644 --- a/src/engine/SCons/Options/ListOption.py +++ b/src/engine/SCons/Options/ListOption.py @@ -65,7 +65,7 @@ class _ListOption(UserList.UserList): UserList.UserList.__init__(self, filter(None, initlist)) self.allowedElems = allowedElems[:] self.allowedElems.sort() - + def __cmp__(self, other): raise NotImplementedError def __eq__(self, other): @@ -88,7 +88,7 @@ class _ListOption(UserList.UserList): return string.join(self, ',') def __repr__(self): return self.__str__() - + def _converter(val, allowedElems, mapdict): """ """ diff --git a/src/engine/SCons/Options/ListOptionTests.py b/src/engine/SCons/Options/ListOptionTests.py index a892e18..22378bc 100644 --- a/src/engine/SCons/Options/ListOptionTests.py +++ b/src/engine/SCons/Options/ListOptionTests.py @@ -122,9 +122,6 @@ class ListOptionTestCase(unittest.TestCase): l = o.converter('all') n = l.__class__(copy.copy(l)) - def test___repr__(self): - """Test copying a ListOption like an Environment would""" - if __name__ == "__main__": suite = unittest.makeSuite(ListOptionTestCase, 'test_') if not unittest.TextTestRunner().run(suite).wasSuccessful(): diff --git a/src/engine/SCons/Options/OptionsTests.py b/src/engine/SCons/Options/OptionsTests.py index aeffe2d..1d9b851 100644 --- a/src/engine/SCons/Options/OptionsTests.py +++ b/src/engine/SCons/Options/OptionsTests.py @@ -59,6 +59,19 @@ def checkSave(file, expected): class OptionsTestCase(unittest.TestCase): + def test_keys(self): + """Test the Options.keys() method""" + opts = SCons.Options.Options() + + opts.Add('VAR1') + opts.Add('VAR2', + 'THE answer to THE question', + "42", + check, + lambda x: int(x) + 12) + keys = opts.keys() + assert keys == ['VAR1', 'VAR2'], keys + def test_Add(self): """Test adding to an Options object""" opts = SCons.Options.Options() diff --git a/src/engine/SCons/Options/__init__.py b/src/engine/SCons/Options/__init__.py index 7333985..83798b3 100644 --- a/src/engine/SCons/Options/__init__.py +++ b/src/engine/SCons/Options/__init__.py @@ -76,6 +76,11 @@ class Options: self.options.append(option) + def keys(self): + """ + Returns the keywords for the options + """ + return map(lambda o: o.key, self.options) def Add(self, key, help="", default=None, validator=None, converter=None, **kw): """ @@ -100,7 +105,6 @@ class Options: self._do_add(key, help, default, validator, converter) - def AddOptions(self, *optlist): """ Add a list of options. diff --git a/src/engine/SCons/Platform/posix.py b/src/engine/SCons/Platform/posix.py index 675167e..1d4e9f7 100644 --- a/src/engine/SCons/Platform/posix.py +++ b/src/engine/SCons/Platform/posix.py @@ -70,7 +70,7 @@ def exec_spawnvpe(l, env): # returned by os.waitpid() or os.system(). return stat -def exec_fork(l, env): +def exec_fork(l, env): pid = os.fork() if not pid: # Child process. @@ -78,7 +78,7 @@ def exec_fork(l, env): try: os.execvpe(l[0], l, env) except OSError, e: - exitval = exitvalmap[e[0]] + exitval = exitvalmap.get(e[0], e[0]) sys.stderr.write("scons: %s: %s\n" % (l[0], e[1])) os._exit(exitval) else: @@ -159,7 +159,7 @@ def exec_piped_fork(l, env, stdout, stderr): try: os.execvpe(l[0], l, env) except OSError, e: - exitval = exitvalmap[e[0]] + exitval = exitvalmap.get(e[0], e[0]) stderr.write("scons: %s: %s\n" % (l[0], e[1])) os._exit(exitval) else: diff --git a/src/engine/SCons/Platform/win32.py b/src/engine/SCons/Platform/win32.py index 6188098..148d9df 100644 --- a/src/engine/SCons/Platform/win32.py +++ b/src/engine/SCons/Platform/win32.py @@ -90,7 +90,11 @@ def piped_spawn(sh, escape, cmd, args, env, stdout, stderr): ret = os.spawnve(os.P_WAIT, sh, args, env) except OSError, e: # catch any error - ret = exitvalmap[e[0]] + try: + ret = exitvalmap[e[0]] + except KeyError: + result = 127 + sys.stderr.write("scons: unknown OSError exception code %d - %s: %s\n" % (e[0], cmd, e[1])) if stderr != None: stderr.write("scons: %s: %s\n" % (cmd, e[1])) # copy child output from tempfiles to our streams @@ -114,8 +118,19 @@ def exec_spawn(l, env): try: result = os.spawnve(os.P_WAIT, l[0], l, env) except OSError, e: - result = exitvalmap[e[0]] - sys.stderr.write("scons: %s: %s\n" % (l[0], e[1])) + try: + result = exitvalmap[e[0]] + sys.stderr.write("scons: %s: %s\n" % (l[0], e[1])) + except KeyError: + result = 127 + if len(l) > 2: + if len(l[2]) < 1000: + command = string.join(l[0:3]) + else: + command = l[0] + else: + command = l[0] + sys.stderr.write("scons: unknown OSError exception code %d - '%s': %s\n" % (e[0], command, e[1])) return result def spawn(sh, escape, cmd, args, env): @@ -124,9 +139,15 @@ def spawn(sh, escape, cmd, args, env): return 127 return exec_spawn([sh, '/C', escape(string.join(args))], env) -# Windows does not allow special characters in file names anyway, so -# no need for a complex escape function, we will just quote the arg. -escape = lambda x: '"' + x + '"' +# Windows does not allow special characters in file names anyway, so no +# need for a complex escape function, we will just quote the arg, except +# that "cmd /c" requires that if an argument ends with a backslash it +# needs to be escaped so as not to interfere with closing double quote +# that we add. +def escape(x): + if x[-1] == '\\': + x = x + '\\' + return '"' + x + '"' # Get the windows system directory name def get_system_root(): diff --git a/src/engine/SCons/Scanner/CTests.py b/src/engine/SCons/Scanner/CTests.py index c57e7f7..2ca522b 100644 --- a/src/engine/SCons/Scanner/CTests.py +++ b/src/engine/SCons/Scanner/CTests.py @@ -221,6 +221,7 @@ def deps_match(self, deps, headers): class CScannerTestCase1(unittest.TestCase): def runTest(self): + """Find local files with no CPPPATH""" env = DummyEnvironment(CPPPATH=[]) s = SCons.Scanner.C.CScanner() path = s.path(env) @@ -230,6 +231,7 @@ class CScannerTestCase1(unittest.TestCase): class CScannerTestCase2(unittest.TestCase): def runTest(self): + """Find a file in a CPPPATH directory""" env = DummyEnvironment(CPPPATH=[test.workpath("d1")]) s = SCons.Scanner.C.CScanner() path = s.path(env) @@ -239,6 +241,7 @@ class CScannerTestCase2(unittest.TestCase): class CScannerTestCase3(unittest.TestCase): def runTest(self): + """Find files in explicit subdirectories, ignore missing file""" env = DummyEnvironment(CPPPATH=[test.workpath("d1")]) s = SCons.Scanner.C.CScanner() path = s.path(env) @@ -248,6 +251,7 @@ class CScannerTestCase3(unittest.TestCase): class CScannerTestCase4(unittest.TestCase): def runTest(self): + """Find files in explicit subdirectories""" env = DummyEnvironment(CPPPATH=[test.workpath("d1"), test.workpath("d1/d2")]) s = SCons.Scanner.C.CScanner() path = s.path(env) @@ -257,6 +261,7 @@ class CScannerTestCase4(unittest.TestCase): class CScannerTestCase5(unittest.TestCase): def runTest(self): + """Make sure files in repositories will get scanned""" env = DummyEnvironment(CPPPATH=[]) s = SCons.Scanner.C.CScanner() path = s.path(env) @@ -280,6 +285,7 @@ class CScannerTestCase5(unittest.TestCase): class CScannerTestCase6(unittest.TestCase): def runTest(self): + """Find a same-named file in different directories when CPPPATH changes""" env1 = DummyEnvironment(CPPPATH=[test.workpath("d1")]) env2 = DummyEnvironment(CPPPATH=[test.workpath("d1/d2")]) s = SCons.Scanner.C.CScanner() @@ -294,6 +300,7 @@ class CScannerTestCase6(unittest.TestCase): class CScannerTestCase8(unittest.TestCase): def runTest(self): + """Find files in a subdirectory relative to the current directory""" env = DummyEnvironment(CPPPATH=["include"]) s = SCons.Scanner.C.CScanner() path = s.path(env) @@ -310,6 +317,7 @@ class CScannerTestCase8(unittest.TestCase): class CScannerTestCase9(unittest.TestCase): def runTest(self): + """Generate a warning when we can't find a #included file""" SCons.Warnings.enableWarningClass(SCons.Warnings.DependencyWarning) class TestOut: def __call__(self, x): @@ -334,6 +342,7 @@ class CScannerTestCase9(unittest.TestCase): class CScannerTestCase10(unittest.TestCase): def runTest(self): + """Find files in the local directory when the scanned file is elsewhere""" fs = SCons.Node.FS.FS(test.workpath('')) fs.chdir(fs.Dir('include')) env = DummyEnvironment(CPPPATH=[]) @@ -348,6 +357,7 @@ class CScannerTestCase10(unittest.TestCase): class CScannerTestCase11(unittest.TestCase): def runTest(self): + """Handle dependencies on a derived .h file in a non-existent directory""" os.chdir(test.workpath('work')) fs = SCons.Node.FS.FS(test.workpath('work')) fs.Repository(test.workpath('repository')) @@ -367,6 +377,7 @@ class CScannerTestCase11(unittest.TestCase): class CScannerTestCase12(unittest.TestCase): def runTest(self): + """Find files in BuildDir() directories""" os.chdir(test.workpath('work')) fs = SCons.Node.FS.FS(test.workpath('work')) fs.BuildDir('build1', 'src', 1) @@ -388,6 +399,7 @@ class CScannerTestCase12(unittest.TestCase): class CScannerTestCase13(unittest.TestCase): def runTest(self): + """Find files in directories named in a substituted environment variable""" class SubstEnvironment(DummyEnvironment): def subst(self, arg, test=test): return test.workpath("d1") @@ -400,6 +412,7 @@ class CScannerTestCase13(unittest.TestCase): class CScannerTestCase14(unittest.TestCase): def runTest(self): + """Find files when there's no space between "#include" and the name""" env = DummyEnvironment(CPPPATH=[]) s = SCons.Scanner.C.CScanner() path = s.path(env) @@ -409,6 +422,7 @@ class CScannerTestCase14(unittest.TestCase): class CScannerTestCase15(unittest.TestCase): def runTest(self): + """Verify scanner initialization with the suffixes in $CPPSUFFIXES""" suffixes = [".c", ".C", ".cxx", ".cpp", ".c++", ".cc", ".h", ".H", ".hxx", ".hpp", ".hh", ".F", ".fpp", ".FPP", diff --git a/src/engine/SCons/Script/Main.py b/src/engine/SCons/Script/Main.py index 73f96e2..2c18112 100644 --- a/src/engine/SCons/Script/Main.py +++ b/src/engine/SCons/Script/Main.py @@ -588,29 +588,29 @@ def _create_path(plist): path = path + '/' + d return path +def version_string(label, module): + fmt = "\t%s: v%s.%s, %s, by %s on %s\n" + return fmt % (label, + module.__version__, + module.__build__, + module.__date__, + module.__developer__, + module.__buildsys__) class OptParser(OptionParser): def __init__(self): import __main__ - import SCons + parts = ["SCons by Steven Knight et al.:\n"] try: - parts.append("\tscript: v%s.%s, %s, by %s on %s\n" % (__main__.__version__, - __main__.__build__, - __main__.__date__, - __main__.__developer__, - __main__.__buildsys__)) + parts.append(version_string("script", __main__)) except KeyboardInterrupt: raise except: # On Windows there is no scons.py, so there is no # __main__.__version__, hence there is no script version. pass - parts.append("\tengine: v%s.%s, %s, by %s on %s\n" % (SCons.__version__, - SCons.__build__, - SCons.__date__, - SCons.__developer__, - SCons.__buildsys__)) + parts.append(version_string("engine", SCons)) parts.append("__COPYRIGHT__") OptionParser.__init__(self, version=string.join(parts, ''), usage="usage: scons [OPTION] [TARGET] ...") @@ -630,6 +630,10 @@ class OptParser(OptionParser): metavar="DIR", help="Change to DIR before doing anything.") + self.add_option('--cache-debug', action="store", + dest="cache_debug", metavar="FILE", + help="Print CacheDir debug info to FILE.") + self.add_option('--cache-disable', '--no-cache', action="store_true", dest='cache_disable', default=0, help="Do not retrieve built targets from CacheDir.") @@ -1048,6 +1052,9 @@ def _main(args, parser): display.set_mode(0) if options.silent: SCons.Action.print_actions = None + + if options.cache_debug: + fs.CacheDebugEnable(options.cache_debug) if options.cache_disable: def disable(self): pass fs.CacheDir = disable @@ -1291,8 +1298,21 @@ def _exec_main(): import pdb pdb.Pdb().runcall(_main, args, parser) elif options.profile_file: - import profile - prof = profile.Profile() + from profile import Profile + + # Some versions of Python 2.4 shipped a profiler that had the + # wrong 'c_exception' entry in its dispatch table. Make sure + # we have the right one. (This may put an unnecessary entry + # in the table in earlier versions of Python, but its presence + # shouldn't hurt anything). + try: + dispatch = Profile.dispatch + except AttributeError: + pass + else: + dispatch['c_exception'] = Profile.trace_dispatch_return + + prof = Profile() try: prof.runcall(_main, args, parser) except SystemExit: diff --git a/src/engine/SCons/Script/SConscript.py b/src/engine/SCons/Script/SConscript.py index f932d30..dc896a0 100644 --- a/src/engine/SCons/Script/SConscript.py +++ b/src/engine/SCons/Script/SConscript.py @@ -176,7 +176,7 @@ def _SConscript(fs, *files, **kw): # fs match so we can open the SConscript. fs.chdir(top, change_os_dir=1) if f.rexists(): - _file_ = open(f.rstr(), "r") + _file_ = open(f.rfile().get_abspath(), "r") elif f.has_src_builder(): # The SConscript file apparently exists in a source # code management system. Build it, but then clear @@ -185,7 +185,7 @@ def _SConscript(fs, *files, **kw): f.build() f.builder_set(None) if f.exists(): - _file_ = open(str(f), "r") + _file_ = open(f.get_abspath(), "r") if _file_: # Chdir to the SConscript directory. Use a path # name relative to the SConstruct file so that if @@ -197,7 +197,18 @@ def _SConscript(fs, *files, **kw): # where the SConstruct and SConscript files might be # in different Repositories. For now, cross that # bridge when someone comes to it. - ldir = fs.Dir(f.dir.get_path(sd)) + try: + src_dir = kw['src_dir'] + except KeyError: + ldir = fs.Dir(f.dir.get_path(sd)) + else: + ldir = fs.Dir(src_dir) + if not ldir.is_under(f.dir): + # They specified a source directory, but + # it's above the SConscript directory. + # Do the sensible thing and just use the + # SConcript directory. + ldir = fs.Dir(f.dir.get_path(sd)) try: fs.chdir(ldir, change_os_dir=sconscript_chdir) except OSError: @@ -208,6 +219,7 @@ def _SConscript(fs, *files, **kw): # interpret the stuff within the SConscript file # relative to where we are logically. fs.chdir(ldir, change_os_dir=0) + # TODO Not sure how to handle src_dir here os.chdir(f.rfile().dir.get_abspath()) # Append the SConscript directory to the beginning @@ -223,7 +235,16 @@ def _SConscript(fs, *files, **kw): # SConscript can base the printed frames at this # level and not show SCons internals as well. call_stack[-1].globals.update({stack_bottom:1}) - exec _file_ in call_stack[-1].globals + old_file = call_stack[-1].globals.get('__file__') + try: + del call_stack[-1].globals['__file__'] + except KeyError: + pass + try: + exec _file_ in call_stack[-1].globals + finally: + if old_file is not None: + call_stack[-1].globals.update({__file__:old_file}) else: SCons.Warnings.warn(SCons.Warnings.MissingSConscriptWarning, "Ignoring missing SConscript '%s'" % f.path) @@ -374,6 +395,7 @@ class SConsEnvironment(SCons.Environment.Base): src_dir = kw.get('src_dir') if not src_dir: src_dir, fname = os.path.split(str(files[0])) + files = [os.path.join(str(build_dir), fname)] else: if not isinstance(src_dir, SCons.Node.Node): src_dir = self.fs.Dir(src_dir) @@ -383,11 +405,11 @@ class SConsEnvironment(SCons.Environment.Base): if fn.is_under(src_dir): # Get path relative to the source directory. fname = fn.get_path(src_dir) + files = [os.path.join(str(build_dir), fname)] else: - # Fast way to only get the terminal path component of a Node. - fname = fn.get_path(fn.dir) + files = [fn.abspath] + kw['src_dir'] = build_dir self.fs.BuildDir(build_dir, src_dir, duplicate) - files = [os.path.join(str(build_dir), fname)] return (files, exports) @@ -490,8 +512,8 @@ class SConsEnvironment(SCons.Environment.Base): subst_kw[key] = val files, exports = self._get_SConscript_filenames(ls, subst_kw) - - return apply(_SConscript, [self.fs,] + files, {'exports' : exports}) + subst_kw['exports'] = exports + return apply(_SConscript, [self.fs,] + files, subst_kw) def SConscriptChdir(self, flag): global sconscript_chdir diff --git a/src/engine/SCons/Tool/as.py b/src/engine/SCons/Tool/as.py index 5a267a5..1b1b4b3 100644 --- a/src/engine/SCons/Tool/as.py +++ b/src/engine/SCons/Tool/as.py @@ -52,11 +52,15 @@ def generate(env): for suffix in ASSuffixes: static_obj.add_action(suffix, SCons.Defaults.ASAction) + shared_obj.add_action(suffix, SCons.Defaults.ASAction) static_obj.add_emitter(suffix, SCons.Defaults.StaticObjectEmitter) + shared_obj.add_emitter(suffix, SCons.Defaults.SharedObjectEmitter) for suffix in ASPPSuffixes: static_obj.add_action(suffix, SCons.Defaults.ASPPAction) + shared_obj.add_action(suffix, SCons.Defaults.ASPPAction) static_obj.add_emitter(suffix, SCons.Defaults.StaticObjectEmitter) + shared_obj.add_emitter(suffix, SCons.Defaults.SharedObjectEmitter) env['AS'] = env.Detect(assemblers) or 'as' env['ASFLAGS'] = SCons.Util.CLVar('') diff --git a/src/engine/SCons/Tool/fortran.py b/src/engine/SCons/Tool/fortran.py index b694a58..b83e7d3 100644 --- a/src/engine/SCons/Tool/fortran.py +++ b/src/engine/SCons/Tool/fortran.py @@ -146,7 +146,7 @@ def add_to_env(env): env['FORTRANMODDIR'] = '' # where the compiler should place .mod files env['FORTRANMODDIRPREFIX'] = '' # some prefix to $FORTRANMODDIR - similar to $INCPREFIX env['FORTRANMODDIRSUFFIX'] = '' # some suffix to $FORTRANMODDIR - similar to $INCSUFFIX - env['_FORTRANMODFLAG'] = '$( ${_concat(FORTRANMODDIRPREFIX, FORTRANMODDIR, FORTRANMODDIRSUFFIX, __env__)} $)' + env['_FORTRANMODFLAG'] = '$( ${_concat(FORTRANMODDIRPREFIX, FORTRANMODDIR, FORTRANMODDIRSUFFIX, __env__, RDirs)} $)' env.AppendUnique(FORTRANSUFFIXES = FortranSuffixes + FortranPPSuffixes) diff --git a/src/engine/SCons/Tool/gcc.py b/src/engine/SCons/Tool/gcc.py index 1be665a..ad02e0d 100644 --- a/src/engine/SCons/Tool/gcc.py +++ b/src/engine/SCons/Tool/gcc.py @@ -46,7 +46,7 @@ def generate(env): cc.generate(env) env['CC'] = env.Detect(compilers) or 'gcc' - if env['PLATFORM'] == 'cygwin': + if env['PLATFORM'] in ['cygwin', 'win32']: env['SHCCFLAGS'] = SCons.Util.CLVar('$CCFLAGS') else: env['SHCCFLAGS'] = SCons.Util.CLVar('$CCFLAGS -fPIC') diff --git a/src/engine/SCons/Tool/mingw.py b/src/engine/SCons/Tool/mingw.py index 5765e25..d679b53 100644 --- a/src/engine/SCons/Tool/mingw.py +++ b/src/engine/SCons/Tool/mingw.py @@ -118,7 +118,7 @@ def generate(env): # Most of mingw is the same as gcc and friends... - gnu_tools = ['gcc', 'g++', 'gnulink', 'ar', 'gas'] + gnu_tools = ['gcc', 'g++', 'gnulink', 'ar', 'gas', 'm4'] for tool in gnu_tools: SCons.Tool.Tool(tool)(env) @@ -130,7 +130,6 @@ def generate(env): env['SHLINKFLAGS'] = SCons.Util.CLVar('$LINKFLAGS -shared') env['SHLINKCOM'] = shlib_action env.Append(SHLIBEMITTER = [shlib_emitter]) - env['LINK'] = 'g++' env['AS'] = 'as' env['WIN32DEFPREFIX'] = '' diff --git a/src/engine/SCons/Tool/mslink.xml b/src/engine/SCons/Tool/mslink.xml index 58cfb1d..6499cfe 100644 --- a/src/engine/SCons/Tool/mslink.xml +++ b/src/engine/SCons/Tool/mslink.xml @@ -37,6 +37,20 @@ Example: <example> env['PDB'] = 'hello.pdb' </example> + +The Visual C++ compiler switch that SCons uses by default +to generate PDB information is <option>/Z7</option>. +This works correctly with parallel (<option>-j</option>) builds +because it embeds the debug information in the intermediate object files, +as opposed to sharing a single PDB file between multiple object files. +This is also the only way to get debug information +embedded into a static library. +Using the <option>/Zi</option> instead may yield improved +link-time performance, +although parallel builds will no longer work. +You can generate PDB files with the <option>/Zi</option> +switch by overriding the default &cv-CCPDBFLAGS; variable; +see the entry for that variable for specific examples. </summary> </cvar> diff --git a/src/engine/SCons/Tool/msvc.py b/src/engine/SCons/Tool/msvc.py index 86cde78..80b5926 100644 --- a/src/engine/SCons/Tool/msvc.py +++ b/src/engine/SCons/Tool/msvc.py @@ -517,9 +517,10 @@ def _get_msvc8_default_paths(env, version, suite, use_mfc_dirs): include_paths.append( env_include_path ) if SCons.Util.can_read_reg and paths.has_key('FRAMEWORKSDKDIR'): - include_paths.append( os.path.join( paths['FRAMEWORKSDKDIR'], 'include' ) ) - lib_paths.append( os.path.join( paths['FRAMEWORKSDKDIR'], 'lib' ) ) - exe_paths.append( paths['FRAMEWORKSDKDIR'], 'bin' ) + fwdir = paths['FRAMEWORKSDKDIR'] + include_paths.append( os.path.join( fwdir, 'include' ) ) + lib_paths.append( os.path.join( fwdir, 'lib' ) ) + exe_paths.append( os.path.join( fwdir, 'bin' ) ) if SCons.Util.can_read_reg and paths.has_key('FRAMEWORKDIR') and paths.has_key('FRAMEWORKVERSION'): exe_paths.append( os.path.join( paths['FRAMEWORKDIR'], paths['FRAMEWORKVERSION'] ) ) @@ -658,7 +659,10 @@ pch_builder = SCons.Builder.Builder(action=pch_action, suffix='.pch', emitter=pch_emitter, source_scanner=SCons.Tool.SourceFileScanner) res_action = SCons.Action.Action('$RCCOM', '$RCCOMSTR') -res_builder = SCons.Builder.Builder(action=res_action, suffix='.res', +res_builder = SCons.Builder.Builder(action=res_action, + src_suffix='.rc', + suffix='.res', + src_builder=[], source_scanner=SCons.Tool.SourceFileScanner) SCons.Tool.SourceFileScanner.add_scanner('.rc', SCons.Defaults.CScan) diff --git a/src/engine/SCons/Tool/msvc.xml b/src/engine/SCons/Tool/msvc.xml index 252d962..1c0f3fa 100644 --- a/src/engine/SCons/Tool/msvc.xml +++ b/src/engine/SCons/Tool/msvc.xml @@ -47,6 +47,53 @@ env.RES('resource.rc') </summary> </builder> +<cvar name="CCPCHFLAGS"> +<summary> +Options added to the compiler command line +to support building with precompiled headers. +The default value expands expands to the appropriate +Microsoft Visual C++ command-line options +when the &cv-PCH; construction variable is set. +</summary> + +<cvar name="CCPDBFLAGS"> +<summary> +Options added to the compiler command line +to support storing debugging information in a +Microsoft Visual C++ PDB file. +The default value expands expands to appropriate +Microsoft Visual C++ command-line options +when the &cv-PDB; construction variable is set. + +The Visual C++ compiler option that SCons uses by default +to generate PDB information is <option>/Z7</option>. +This works correctly with parallel (<option>-j</option>) builds +because it embeds the debug information in the intermediate object files, +as opposed to sharing a single PDB file between multiple object files. +This is also the only way to get debug information +embedded into a static library. +Using the <option>/Zi</option> instead may yield improved +link-time performance, +although parallel builds will no longer work. + +You can generate PDB files with the <option>/Zi</option> +switch by overriding the default &cv-CCPDBFLAGS; variable as follows: + +<example> +import SCons.Util +env['CCPDBFLAGS'] = SCons.Util.CLVar(['${(PDB and "/Zi /Fd%s" % File(PDB)) or ""}']) +</example> + +An alternative would be to use the <option>/Zi</option> +to put the debugging information in a separate <filename>.pdb</filename> +file for each object file by overriding +the &cv-CCPDBFLAGS; variable as follows: + +<example> +env['CCPDBFLAGS'] = '/Zi /Fd${TARGET}.pdb' +</example> +</summary> + <cvar name="PCH"> <summary> The Microsoft Visual C++ precompiled header that will be used when compiling diff --git a/src/engine/SCons/Tool/msvs.py b/src/engine/SCons/Tool/msvs.py index e8aaf83..e35c92a 100644 --- a/src/engine/SCons/Tool/msvs.py +++ b/src/engine/SCons/Tool/msvs.py @@ -297,7 +297,7 @@ class _DSPGenerator: for n in sourcenames: self.sources[n].sort(lambda a, b: cmp(a.lower(), b.lower())) - def AddConfig(variant, buildtarget, outdir, runfile, cmdargs): + def AddConfig(self, variant, buildtarget, outdir, runfile, cmdargs, dspfile=dspfile): config = Config() config.buildtarget = buildtarget config.outdir = outdir @@ -316,7 +316,7 @@ class _DSPGenerator: print "Adding '" + self.name + ' - ' + config.variant + '|' + config.platform + "' to '" + str(dspfile) + "'" for i in range(len(variants)): - AddConfig(variants[i], buildtarget[i], outdir[i], runfile[i], cmdargs) + AddConfig(self, variants[i], buildtarget[i], outdir[i], runfile[i], cmdargs) self.platforms = [] for key in self.configs.keys(): @@ -690,6 +690,29 @@ class _GenerateV7DSP(_DSPGenerator): pdata = base64.encodestring(pdata) self.file.write(pdata + '-->\n') + def printSources(self, hierarchy, commonprefix): + sorteditems = hierarchy.items() + sorteditems.sort(lambda a, b: cmp(a[0].lower(), b[0].lower())) + + # First folders, then files + for key, value in sorteditems: + if SCons.Util.is_Dict(value): + self.file.write('\t\t\t<Filter\n' + '\t\t\t\tName="%s"\n' + '\t\t\t\tFilter="">\n' % (key)) + self.printSources(value, commonprefix) + self.file.write('\t\t\t</Filter>\n') + + for key, value in sorteditems: + if SCons.Util.is_String(value): + file = value + if commonprefix: + file = os.path.join(commonprefix, value) + file = os.path.normpath(file) + self.file.write('\t\t\t<File\n' + '\t\t\t\tRelativePath="%s">\n' + '\t\t\t</File>\n' % (file)) + def PrintSourceFiles(self): categories = {'Source Files': 'cpp;c;cxx;l;y;def;odl;idl;hpj;bat', 'Header Files': 'h;hpp;hxx;hm;inl', @@ -708,43 +731,26 @@ class _GenerateV7DSP(_DSPGenerator): '\t\t\tName="%s"\n' '\t\t\tFilter="%s">\n' % (kind, categories[kind])) - - def printSources(hierarchy, commonprefix): - sorteditems = hierarchy.items() - sorteditems.sort(lambda a, b: cmp(a[0].lower(), b[0].lower())) - - # First folders, then files - for key, value in sorteditems: - if SCons.Util.is_Dict(value): - self.file.write('\t\t\t<Filter\n' - '\t\t\t\tName="%s"\n' - '\t\t\t\tFilter="">\n' % (key)) - printSources(value, commonprefix) - self.file.write('\t\t\t</Filter>\n') - - for key, value in sorteditems: - if SCons.Util.is_String(value): - file = value - if commonprefix: - file = os.path.join(commonprefix, value) - file = os.path.normpath(file) - self.file.write('\t\t\t<File\n' - '\t\t\t\tRelativePath="%s">\n' - '\t\t\t</File>\n' % (file)) - sources = self.sources[kind] # First remove any common prefix commonprefix = None if len(sources) > 1: s = map(os.path.normpath, sources) - cp = os.path.commonprefix(s) + # take the dirname because the prefix may include parts + # of the filenames (e.g. if you have 'dir\abcd' and + # 'dir\acde' then the cp will be 'dir\a' ) + cp = os.path.dirname( os.path.commonprefix(s) ) if cp and s[0][len(cp)] == os.sep: - sources = map(lambda s, l=len(cp): s[l:], sources) + # +1 because the filename starts after the separator + sources = map(lambda s, l=len(cp)+1: s[l:], sources) commonprefix = cp + elif len(sources) == 1: + commonprefix = os.path.dirname( sources[0] ) + sources[0] = os.path.basename( sources[0] ) hierarchy = makeHierarchy(sources) - printSources(hierarchy, commonprefix=commonprefix) + self.printSources(hierarchy, commonprefix=commonprefix) if len(cats)>1: self.file.write('\t\t</Filter>\n') @@ -873,7 +879,7 @@ class _GenerateV7DSW(_DSWGenerator): if self.nokeep == 0 and os.path.exists(self.dswfile): self.Parse() - def AddConfig(variant): + def AddConfig(self, variant, dswfile=dswfile): config = Config() match = re.match('(.*)\|(.*)', variant) @@ -892,10 +898,10 @@ class _GenerateV7DSW(_DSWGenerator): "You must specify a 'variant' argument (i.e. 'Debug' or " +\ "'Release') to create an MSVS Solution File." elif SCons.Util.is_String(env['variant']): - AddConfig(env['variant']) + AddConfig(self, env['variant']) elif SCons.Util.is_List(env['variant']): for variant in env['variant']: - AddConfig(variant) + AddConfig(self, variant) self.platforms = [] for key in self.configs.keys(): @@ -1127,32 +1133,24 @@ def GenerateDSW(dswfile, source, env): def get_default_visualstudio_version(env): """Returns the version set in the env, or the latest version installed, if it can find it, or '6.0' if all else fails. Also - updated the environment with what it found.""" + updates the environment with what it found.""" - version = '6.0' - versions = [version] + versions = ['6.0'] if not env.has_key('MSVS') or not SCons.Util.is_Dict(env['MSVS']): - env['MSVS'] = {} - - if env['MSVS'].has_key('VERSIONS'): - versions = env['MSVS']['VERSIONS'] - elif SCons.Util.can_read_reg: - v = get_visualstudio_versions() - if v: - versions = v - if env.has_key('MSVS_VERSION'): - version = env['MSVS_VERSION'] - else: - version = versions[0] #use highest version by default - - env['MSVS_VERSION'] = version - env['MSVS']['VERSIONS'] = versions - env['MSVS']['VERSION'] = version + v = get_visualstudio_versions() + if v: + versions = v + env['MSVS'] = {'VERSIONS' : versions} else: - version = env['MSVS']['VERSION'] + versions = env['MSVS'].get('VERSIONS', versions) + + if not env.has_key('MSVS_VERSION'): + env['MSVS_VERSION'] = versions[0] #use highest version by default + + env['MSVS']['VERSION'] = env['MSVS_VERSION'] - return version + return env['MSVS_VERSION'] def get_visualstudio_versions(): """ diff --git a/src/engine/SCons/Tool/msvsTests.py b/src/engine/SCons/Tool/msvsTests.py index 6095dff..b1125e7 100644 --- a/src/engine/SCons/Tool/msvsTests.py +++ b/src/engine/SCons/Tool/msvsTests.py @@ -471,6 +471,18 @@ class msvsTestCase(unittest.TestCase): assert env['MSVS']['VERSION'] == '7.0', env['MSVS']['VERSION'] assert v2 == '7.0', v2 + env = DummyEnv() + v3 = get_default_visualstudio_version(env) + if v3 == '7.1': + override = '7.0' + else: + override = '7.1' + env['MSVS_VERSION'] = override + v3 = get_default_visualstudio_version(env) + assert env['MSVS_VERSION'] == override, env['MSVS_VERSION'] + assert env['MSVS']['VERSION'] == override, env['MSVS']['VERSION'] + assert v3 == override, v3 + def test_get_visual_studio_versions(self): """Test retrieval of the list of visual studio versions""" v1 = get_visualstudio_versions() diff --git a/src/engine/SCons/cpp.py b/src/engine/SCons/cpp.py new file mode 100644 index 0000000..9bf5779 --- /dev/null +++ b/src/engine/SCons/cpp.py @@ -0,0 +1,561 @@ +# +# __COPYRIGHT__ +# +# Permission is hereby granted, free of charge, to any person obtaining +# a copy of this software and associated documentation files (the +# "Software"), to deal in the Software without restriction, including +# without limitation the rights to use, copy, modify, merge, publish, +# distribute, sublicense, and/or sell copies of the Software, and to +# permit persons to whom the Software is furnished to do so, subject to +# the following conditions: +# +# The above copyright notice and this permission notice shall be included +# in all copies or substantial portions of the Software. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY +# KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE +# WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND +# NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE +# LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION +# OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION +# WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. +# + +__revision__ = "__FILE__ __REVISION__ __DATE__ __DEVELOPER__" + +__doc__ = """ +SCons C Pre-Processor module +""" + +import os +import re +import string +import sys + + + +import __builtin__ +try: + __builtin__.zip +except AttributeError: + def zip(*lists): + result = [] + for i in xrange(len(lists[0])): + result.append(tuple(map(lambda l, i=i: l[i], lists))) + return result + + + +# +# First "subsystem" of regular expressions that we set up: +# +# Stuff to turn the C preprocessor directives in a file's contents into +# a list of tuples that we can process easily. +# + +# A table of regular expressions that fetch the arguments from the rest of +# a C preprocessor line. Different directives have different arguments +# that we want to fetch, using the regular expressions to which the lists +# of preprocessor directives map. +cpp_lines_dict = { + # Fetch the rest of a #if/#elif/#ifdef/#ifndef/#import/#include/ + # #include_next line as one argument. + ('if', 'elif', 'ifdef', 'ifndef', 'import', 'include', 'include_next',) + : '\s+(.+)', + + # We don't care what comes after a #else or #endif line. + ('else', 'endif',) : '', + + # Fetch three arguments from a #define line: + # 1) The #defined keyword. + # 2) The optional parentheses and arguments (if it's a function-like + # macro, '' if it's not). + # 3) The expansion value. + ('define',) : '\s+([_A-Za-z][_A-Za-z0-9_]+)(\([^)]*\))?\s*(.*)', + + # Fetch the #undefed keyword from a #undef line. + ('undef',) : '\s+([_A-Za-z][A-Za-z0-9_]+)', +} + +# Create a table that maps each individual C preprocessor directive to +# the corresponding compiled regular expression that fetches the arguments +# we care about. +Table = {} +for op_list, expr in cpp_lines_dict.items(): + e = re.compile(expr) + for op in op_list: + Table[op] = e +del e +del op +del op_list + +# Create a list of the expressions we'll use to match all of the +# preprocessor directives. These are the same as the directives +# themselves *except* that we must use a negative lookahead assertion +# when matching "if" so it doesn't match the "if" in "ifdef." +override = { + 'if' : 'if(?!def)', +} +l = map(lambda x, o=override: o.get(x, x), Table.keys()) + + +# Turn the list of expressions into one big honkin' regular expression +# that will match all the preprocessor lines at once. This will return +# a list of tuples, one for each preprocessor line. The preprocessor +# directive will be the first element in each tuple, and the rest of +# the line will be the second element. +e = '^\s*#\s*(' + string.join(l, '|') + ')(.*)$' + +# And last but not least, compile the expression. +CPP_Expression = re.compile(e, re.M) + + + + +# +# Second "subsystem" of regular expressions that we set up: +# +# Stuff to translate a C preprocessor expression (as found on a #if or +# #elif line) into an equivalent Python expression that we can eval(). +# + +# A dictionary that maps the C representation of Boolean operators +# to their Python equivalents. +CPP_to_Python_Ops_Dict = { + '!' : ' not ', + '!=' : ' != ', + '&&' : ' and ', + '||' : ' or ', + '?' : ' and ', + ':' : ' or ', + '\r' : '', +} + +CPP_to_Python_Ops_Sub = lambda m, d=CPP_to_Python_Ops_Dict: d[m.group(0)] + +# We have to sort the keys by length so that longer expressions +# come *before* shorter expressions--in particular, "!=" must +# come before "!" in the alternation. Without this, the Python +# re module, as late as version 2.2.2, empirically matches the +# "!" in "!=" first, instead of finding the longest match. +# What's up with that? +l = CPP_to_Python_Ops_Dict.keys() +l.sort(lambda a, b: cmp(len(b), len(a))) + +# Turn the list of keys into one regular expression that will allow us +# to substitute all of the operators at once. +expr = string.join(map(re.escape, l), '|') + +# ...and compile the expression. +CPP_to_Python_Ops_Expression = re.compile(expr) + +# A separate list of expressions to be evaluated and substituted +# sequentially, not all at once. +CPP_to_Python_Eval_List = [ + ['defined\s+(\w+)', '__dict__.has_key("\\1")'], + ['defined\s*\((\w+)\)', '__dict__.has_key("\\1")'], + ['/\*.*\*/', ''], + ['/\*.*', ''], + ['//.*', ''], + ['(0x[0-9A-Fa-f]*)[UL]+', '\\1L'], +] + +# Replace the string representations of the regular expressions in the +# list with compiled versions. +for l in CPP_to_Python_Eval_List: + l[0] = re.compile(l[0]) + +# Wrap up all of the above into a handy function. +def CPP_to_Python(s): + """ + Converts a C pre-processor expression into an equivalent + Python expression that can be evaluated. + """ + s = CPP_to_Python_Ops_Expression.sub(CPP_to_Python_Ops_Sub, s) + for expr, repl in CPP_to_Python_Eval_List: + s = expr.sub(repl, s) + return s + + + +del expr +del l +del override + + + +class FunctionEvaluator: + """ + Handles delayed evaluation of a #define function call. + """ + def __init__(self, name, args, expansion): + """ + Squirrels away the arguments and expansion value of a #define + macro function for later evaluation when we must actually expand + a value that uses it. + """ + self.name = name + self.args = function_arg_separator.split(args) + self.expansion = string.split(expansion, '##') + def __call__(self, *values): + """ + Evaluates the expansion of a #define macro function called + with the specified values. + """ + if len(self.args) != len(values): + raise ValueError, "Incorrect number of arguments to `%s'" % self.name + # Create a dictionary that maps the macro arguments to the + # corresponding values in this "call." We'll use this when we + # eval() the expansion so that arguments will get expanded to + # the right values. + locals = {} + for k, v in zip(self.args, values): + locals[k] = v + + parts = [] + for s in self.expansion: + if not s in self.args: + s = repr(s) + parts.append(s) + statement = string.join(parts, ' + ') + + return eval(statement, globals(), locals) + + + +# Find line continuations. +line_continuations = re.compile('\\\\\r?\n') + +# Search for a "function call" macro on an expansion. Returns the +# two-tuple of the "function" name itself, and a string containing the +# arguments within the call parentheses. +function_name = re.compile('(\S+)\(([^)]*)\)') + +# Split a string containing comma-separated function call arguments into +# the separate arguments. +function_arg_separator = re.compile(',\s*') + + + +class PreProcessor: + """ + The main workhorse class for handling C pre-processing. + """ + def __init__(self, current='.', cpppath=[], dict={}, all=0): + global Table + + self.searchpath = { + '"' : [current] + cpppath, + '<' : cpppath + [current], + } + + # Initialize our C preprocessor namespace for tracking the + # values of #defined keywords. We use this namespace to look + # for keywords on #ifdef/#ifndef lines, and to eval() the + # expressions on #if/#elif lines (after massaging them from C to + # Python). + self.cpp_namespace = dict.copy() + self.cpp_namespace['__dict__'] = self.cpp_namespace + + if all: + self.do_include = self.all_include + + # For efficiency, a dispatch table maps each C preprocessor + # directive (#if, #define, etc.) to the method that should be + # called when we see it. We accomodate state changes (#if, + # #ifdef, #ifndef) by pushing the current dispatch table on a + # stack and changing what method gets called for each relevant + # directive we might see next at this level (#else, #elif). + # #endif will simply pop the stack. + d = {} + for op in Table.keys(): + d[op] = getattr(self, 'do_' + op) + self.default_table = d + + # Controlling methods. + + def tupleize(self, contents): + """ + Turns the contents of a file into a list of easily-processed + tuples describing the CPP lines in the file. + + The first element of each tuple is the line's preprocessor + directive (#if, #include, #define, etc., minus the initial '#'). + The remaining elements are specific to the type of directive, as + pulled apart by the regular expression. + """ + global CPP_Expression, Table + contents = line_continuations.sub('', contents) + cpp_tuples = CPP_Expression.findall(contents) + return map(lambda m, t=Table: + (m[0],) + t[m[0]].match(m[1]).groups(), + cpp_tuples) + + def __call__(self, contents): + """ + Pre-processes a file contents. + + This is the main entry point, which + """ + self.stack = [] + self.dispatch_table = self.default_table.copy() + self.tuples = self.tupleize(contents) + + self.result = [] + while self.tuples: + t = self.tuples.pop(0) + # Uncomment to see the list of tuples being processed (e.g., + # to validate the CPP lines are being translated correctly). + #print t + self.dispatch_table[t[0]](t) + + return self.result + + # Dispatch table stack manipulation methods. + + def save(self): + """ + Pushes the current dispatch table on the stack and re-initializes + the current dispatch table to the default. + """ + self.stack.append(self.dispatch_table) + self.dispatch_table = self.default_table.copy() + + def restore(self): + """ + Pops the previous dispatch table off the stack and makes it the + current one. + """ + try: self.dispatch_table = self.stack.pop() + except IndexError: pass + + # Utility methods. + + def do_nothing(self, t): + """ + Null method for when we explicitly want the action for a + specific preprocessor directive to do nothing. + """ + pass + + def eval_expression(self, t): + """ + Evaluates a C preprocessor expression. + + This is done by converting it to a Python equivalent and + eval()ing it in the C preprocessor namespace we use to + track #define values. + """ + t = CPP_to_Python(string.join(t[1:])) + try: return eval(t, self.cpp_namespace) + except (NameError, TypeError): return 0 + + def find_include_file(self, t): + """ + Finds the #include file for a given preprocessor tuple. + """ + fname = t[2] + for d in self.searchpath[t[1]]: + f = os.path.join(d, fname) + if os.path.isfile(f): + return f + return None + + # Start and stop processing include lines. + + def start_handling_includes(self, t=None): + """ + Causes the PreProcessor object to start processing #import, + #include and #include_next lines. + + This method will be called when a #if, #ifdef, #ifndef or #elif + evaluates True, or when we reach the #else in a #if, #ifdef, + #ifndef or #elif block where a condition already evaluated + False. + + """ + d = self.dispatch_table + d['import'] = self.do_import + d['include'] = self.do_include + d['include_next'] = self.do_include + + def stop_handling_includes(self, t=None): + """ + Causes the PreProcessor object to stop processing #import, + #include and #include_next lines. + + This method will be called when a #if, #ifdef, #ifndef or #elif + evaluates False, or when we reach the #else in a #if, #ifdef, + #ifndef or #elif block where a condition already evaluated True. + """ + d = self.dispatch_table + d['import'] = self.do_nothing + d['include'] = self.do_nothing + d['include_next'] = self.do_nothing + + # Default methods for handling all of the preprocessor directives. + # (Note that what actually gets called for a given directive at any + # point in time is really controlled by the dispatch_table.) + + def _do_if_else_condition(self, condition): + """ + Common logic for evaluating the conditions on #if, #ifdef and + #ifndef lines. + """ + self.save() + d = self.dispatch_table + if condition: + self.start_handling_includes() + d['elif'] = self.stop_handling_includes + d['else'] = self.stop_handling_includes + else: + self.stop_handling_includes() + d['elif'] = self.do_elif + d['else'] = self.start_handling_includes + + def do_ifdef(self, t): + """ + Default handling of a #ifdef line. + """ + self._do_if_else_condition(self.cpp_namespace.has_key(t[1])) + + def do_ifndef(self, t): + """ + Default handling of a #ifndef line. + """ + self._do_if_else_condition(not self.cpp_namespace.has_key(t[1])) + + def do_if(self, t): + """ + Default handling of a #if line. + """ + self._do_if_else_condition(self.eval_expression(t)) + + def do_elif(self, t): + """ + Default handling of a #elif line. + """ + d = self.dispatch_table + if self.eval_expression(t): + self.start_handling_includes() + d['elif'] = self.stop_handling_includes + d['else'] = self.stop_handling_includes + + def do_else(self, t): + """ + Default handling of a #else line. + """ + pass + + def do_endif(self, t): + """ + Default handling of a #endif line. + """ + self.restore() + + def do_define(self, t): + """ + Default handling of a #define line. + """ + _, name, args, expansion = t + try: + expansion = int(expansion) + except (TypeError, ValueError): + pass + if args: + evaluator = FunctionEvaluator(name, args[1:-1], expansion) + self.cpp_namespace[name] = evaluator + else: + self.cpp_namespace[name] = expansion + + def do_undef(self, t): + """ + Default handling of a #undef line. + """ + try: del self.cpp_namespace[t[1]] + except KeyError: pass + + def do_import(self, t): + """ + Default handling of a #import line. + """ + # XXX finish this -- maybe borrow/share logic from do_include()...? + pass + + def do_include(self, t): + """ + Default handling of a #include line. + """ + t = self.resolve_include(t) + include_file = self.find_include_file(t) + if include_file: + #print "include_file =", include_file + self.result.append(include_file) + contents = open(include_file).read() + new_tuples = self.tupleize(contents) + self.tuples[:] = new_tuples + self.tuples + + # Date: Tue, 22 Nov 2005 20:26:09 -0500 + # From: Stefan Seefeld <seefeld@sympatico.ca> + # + # By the way, #include_next is not the same as #include. The difference + # being that #include_next starts its search in the path following the + # path that let to the including file. In other words, if your system + # include paths are ['/foo', '/bar'], and you are looking at a header + # '/foo/baz.h', it might issue an '#include_next <baz.h>' which would + # correctly resolve to '/bar/baz.h' (if that exists), but *not* see + # '/foo/baz.h' again. See http://www.delorie.com/gnu/docs/gcc/cpp_11.html + # for more reasoning. + # + # I have no idea in what context 'import' might be used. + + # XXX is #include_next really the same as #include ? + do_include_next = do_include + + # Utility methods for handling resolution of include files. + + def resolve_include(self, t): + """Resolve a tuple-ized #include line. + + This handles recursive expansion of values without "" or <> + surrounding the name until an initial " or < is found, to handle + #include FILE + where FILE is a #define somewhere else. + """ + s = t[1] + while not s[0] in '<"': + #print "s =", s + try: + s = self.cpp_namespace[s] + except KeyError: + m = function_name.search(s) + s = self.cpp_namespace[m.group(1)] + if callable(s): + args = function_arg_separator.split(m.group(2)) + s = apply(s, args) + if not s: + return None + return (t[0], s[0], s[1:-1]) + + def all_include(self, t): + """ + """ + self.result.append(self.resolve_include(t)) + +class DumbPreProcessor(PreProcessor): + """A preprocessor that ignores all #if/#elif/#else/#endif directives + and just reports back *all* of the #include files (like the classic + SCons scanner did). + + This is functionally equivalent to using a regular expression to + find all of the #include lines, only slower. It exists mainly as + an example of how the main PreProcessor class can be sub-classed + to tailor its behavior. + """ + def __init__(self, *args, **kw): + apply(PreProcessor.__init__, (self,)+args, kw) + d = self.default_table + for func in ['if', 'elif', 'else', 'endif', 'ifdef', 'ifndef']: + d[func] = d[func] = self.do_nothing + +del __revision__ diff --git a/src/engine/SCons/cppTests.py b/src/engine/SCons/cppTests.py new file mode 100644 index 0000000..0959e2c --- /dev/null +++ b/src/engine/SCons/cppTests.py @@ -0,0 +1,573 @@ +# +# __COPYRIGHT__ +# +# Permission is hereby granted, free of charge, to any person obtaining +# a copy of this software and associated documentation files (the +# "Software"), to deal in the Software without restriction, including +# without limitation the rights to use, copy, modify, merge, publish, +# distribute, sublicense, and/or sell copies of the Software, and to +# permit persons to whom the Software is furnished to do so, subject to +# the following conditions: +# +# The above copyright notice and this permission notice shall be included +# in all copies or substantial portions of the Software. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY +# KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE +# WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND +# NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE +# LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION +# OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION +# WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. +# + +__revision__ = "__FILE__ __REVISION__ __DATE__ __DEVELOPER__" + +import sys +import unittest + +print sys.path +import cpp + + + +basic_input = """ +#include "file1-yes" +#include <file2-yes> +""" + + +substitution_input = """ +#define FILE3 "file3-yes" +#define FILE4 <file4-yes> + +#include FILE3 +#include FILE4 + +#define XXX_FILE5 YYY_FILE5 +#define YYY_FILE5 ZZZ_FILE5 +#define ZZZ_FILE5 FILE5 + +#define FILE5 "file5-yes" +#define FILE6 <file6-yes> + +#define XXX_FILE6 YYY_FILE6 +#define YYY_FILE6 ZZZ_FILE6 +#define ZZZ_FILE6 FILE6 + +#include XXX_FILE5 +#include XXX_FILE6 +""" + + +ifdef_input = """ +#define DEFINED 0 + +#ifdef DEFINED +#include "file7-yes" +#else +#include "file7-no" +#endif + +#ifdef NOT_DEFINED +#include <file8-no> +#else +#include <file8-yes> +#endif +""" + + +if_boolean_input = """ +#define ZERO 0 +#define ONE 1 + +#if ZERO +#include "file9-no" +#else +#include "file9-yes" +#endif + +#if ONE +#include <file10-yes> +#else +#include <file10-no> +#endif + +#if ZERO +#include "file11-no-1" +#elif ZERO +#include "file11-no-2" +#else +#include "file11-yes" +#endif + +#if ZERO +#include <file12-no-1> +#elif ONE +#include <file12-yes> +#else +#include <file12-no-2> +#endif + +#if ONE +#include "file13-yes" +#elif ZERO +#include "file13-no-1" +#else +#include "file13-no-2" +#endif + +#if ONE +#include <file14-yes> +#elif ONE +#include <file14-no-1> +#else +#include <file14-no-2> +#endif +""" + + +if_defined_input = """ +#define DEFINED 0 + +#if defined(DEFINED) +#include "file15-yes" +#endif + +#if ! defined(DEFINED) +#include <file16-no> +#else +#include <file16-yes> +#endif + +#if defined DEFINED +#include "file17-yes" +#endif + +#if ! defined DEFINED +#include <file18-no> +#else +#include <file18-yes> +#endif +""" + + +expression_input = """ +#define ZERO 0 +#define ONE 1 + +#if ZERO && ZERO +#include "file19-no" +#else +#include "file19-yes" +#endif + +#if ZERO && ONE +#include <file20-no> +#else +#include <file20-yes> +#endif + +#if ONE && ZERO +#include "file21-no" +#else +#include "file21-yes" +#endif + +#if ONE && ONE +#include <file22-yes> +#else +#include <file22-no> +#endif + +#if ZERO || ZERO +#include "file23-no" +#else +#include "file23-yes" +#endif + +#if ZERO || ONE +#include <file24-yes> +#else +#include <file24-no> +#endif + +#if ONE || ZERO +#include "file25-yes" +#else +#include "file25-no" +#endif + +#if ONE || ONE +#include <file26-yes> +#else +#include <file26-no> +#endif + +#if ONE == ONE +#include "file27-yes" +#else +#include "file27-no" +#endif + +#if ONE != ONE +#include <file28-no> +#else +#include <file28-yes> +#endif + +#if ! (ONE == ONE) +#include "file29-no" +#else +#include "file29-yes" +#endif + +#if ! (ONE != ONE) +#include <file30-yes> +#else +#include <file30-no> +#endif +""" + + +undef_input = """ +#define UNDEFINE 0 + +#ifdef UNDEFINE +#include "file31-yes" +#else +#include "file31-no" +#endif + +#undef UNDEFINE + +#ifdef UNDEFINE +#include <file32-no> +#else +#include <file32-yes> +#endif +""" + + +macro_function_input = """ +#define ZERO 0 +#define ONE 1 + +#define FUNC33(x) "file33-yes" +#define FUNC34(x) <file34-yes> + +#include FUNC33(ZERO) +#include FUNC34(ZERO) + +#define FILE35 "file35-yes" +#define FILE36 <file36-yes> + +#define FUNC35(x, y) FILE35 +#define FUNC36(x, y) FILE36 + +#include FUNC35(ZERO, ONE) +#include FUNC36(ZERO, ONE) + +#define FILE37 "file37-yes" +#define FILE38 <file38-yes> + +#define FUNC37a(x, y) FILE37 +#define FUNC38a(x, y) FILE38 + +#define FUNC37b(x, y) FUNC37a(x, y) +#define FUNC38b(x, y) FUNC38a(x, y) + +#define FUNC37c(x, y) FUNC37b(x, y) +#define FUNC38c(x, y) FUNC38b(x, y) + +#include FUNC37c(ZERO, ONE) +#include FUNC38c(ZERO, ONE) + +#define FILE39 "file39-yes" +#define FILE40 <file40-yes> + +#define FUNC39a(x0, y0) FILE39 +#define FUNC40a(x0, y0) FILE40 + +#define FUNC39b(x1, y2) FUNC39a(x1, y1) +#define FUNC40b(x1, y2) FUNC40a(x1, y1) + +#define FUNC39c(x2, y2) FUNC39b(x2, y2) +#define FUNC40c(x2, y2) FUNC40b(x2, y2) + +#include FUNC39c(ZERO, ONE) +#include FUNC40c(ZERO, ONE) +""" + + +token_pasting_input = """ +#define PASTE_QUOTE(q, name) q##name##-yes##q +#define PASTE_ANGLE(name) <##name##-yes> + +#define FUNC41 PASTE_QUOTE(", file41) +#define FUNC42 PASTE_ANGLE(file42) + +#include FUNC41 +#include FUNC42 +""" + + + +# pp_class = PreProcessor +# #pp_class = DumbPreProcessor + +# pp = pp_class(current = ".", +# cpppath = ['/usr/include'], +# print_all = 1) +# #pp(open(sys.argv[1]).read()) +# pp(input) + + +class cppTestCase(unittest.TestCase): + def setUp(self): + self.cpp = self.cpp_class(current = ".", + cpppath = ['/usr/include']) + + def test_basic(self): + """Test basic #include scanning""" + expect = self.basic_expect + result = self.cpp(basic_input) + assert expect == result, (expect, result) + + def test_substitution(self): + """Test substitution of #include files using CPP variables""" + expect = self.substitution_expect + result = self.cpp(substitution_input) + assert expect == result, (expect, result) + + def test_ifdef(self): + """Test basic #ifdef processing""" + expect = self.ifdef_expect + result = self.cpp(ifdef_input) + assert expect == result, (expect, result) + + def test_if_boolean(self): + """Test #if with Boolean values""" + expect = self.if_boolean_expect + result = self.cpp(if_boolean_input) + assert expect == result, (expect, result) + + def test_if_defined(self): + """Test #if defined() idioms""" + expect = self.if_defined_expect + result = self.cpp(if_defined_input) + assert expect == result, (expect, result) + + def test_expression(self): + """Test #if with arithmetic expressions""" + expect = self.expression_expect + result = self.cpp(expression_input) + assert expect == result, (expect, result) + + def test_undef(self): + """Test #undef handling""" + expect = self.undef_expect + result = self.cpp(undef_input) + assert expect == result, (expect, result) + + def test_macro_function(self): + """Test using macro functions to express file names""" + expect = self.macro_function_expect + result = self.cpp(macro_function_input) + assert expect == result, (expect, result) + + def test_token_pasting(self): + """Test taken-pasting to construct file names""" + expect = self.token_pasting_expect + result = self.cpp(token_pasting_input) + assert expect == result, (expect, result) + +class cppAllTestCase(cppTestCase): + def setUp(self): + self.cpp = self.cpp_class(current = ".", + cpppath = ['/usr/include'], + all=1) + +class PreProcessorTestCase(cppAllTestCase): + cpp_class = cpp.PreProcessor + + basic_expect = [ + ('include', '"', 'file1-yes'), + ('include', '<', 'file2-yes'), + ] + + substitution_expect = [ + ('include', '"', 'file3-yes'), + ('include', '<', 'file4-yes'), + ('include', '"', 'file5-yes'), + ('include', '<', 'file6-yes'), + ] + + ifdef_expect = [ + ('include', '"', 'file7-yes'), + ('include', '<', 'file8-yes'), + ] + + if_boolean_expect = [ + ('include', '"', 'file9-yes'), + ('include', '<', 'file10-yes'), + ('include', '"', 'file11-yes'), + ('include', '<', 'file12-yes'), + ('include', '"', 'file13-yes'), + ('include', '<', 'file14-yes'), + ] + + if_defined_expect = [ + ('include', '"', 'file15-yes'), + ('include', '<', 'file16-yes'), + ('include', '"', 'file17-yes'), + ('include', '<', 'file18-yes'), + ] + + expression_expect = [ + ('include', '"', 'file19-yes'), + ('include', '<', 'file20-yes'), + ('include', '"', 'file21-yes'), + ('include', '<', 'file22-yes'), + ('include', '"', 'file23-yes'), + ('include', '<', 'file24-yes'), + ('include', '"', 'file25-yes'), + ('include', '<', 'file26-yes'), + ('include', '"', 'file27-yes'), + ('include', '<', 'file28-yes'), + ('include', '"', 'file29-yes'), + ('include', '<', 'file30-yes'), + ] + + undef_expect = [ + ('include', '"', 'file31-yes'), + ('include', '<', 'file32-yes'), + ] + + macro_function_expect = [ + ('include', '"', 'file33-yes'), + ('include', '<', 'file34-yes'), + ('include', '"', 'file35-yes'), + ('include', '<', 'file36-yes'), + ('include', '"', 'file37-yes'), + ('include', '<', 'file38-yes'), + ('include', '"', 'file39-yes'), + ('include', '<', 'file40-yes'), + ] + + token_pasting_expect = [ + ('include', '"', 'file41-yes'), + ('include', '<', 'file42-yes'), + ] + +class DumbPreProcessorTestCase(cppAllTestCase): + cpp_class = cpp.DumbPreProcessor + + basic_expect = [ + ('include', '"', 'file1-yes'), + ('include', '<', 'file2-yes'), + ] + + substitution_expect = [ + ('include', '"', 'file3-yes'), + ('include', '<', 'file4-yes'), + ('include', '"', 'file5-yes'), + ('include', '<', 'file6-yes'), + ] + + ifdef_expect = [ + ('include', '"', 'file7-yes'), + ('include', '"', 'file7-no'), + ('include', '<', 'file8-no'), + ('include', '<', 'file8-yes'), + ] + + if_boolean_expect = [ + ('include', '"', 'file9-no'), + ('include', '"', 'file9-yes'), + ('include', '<', 'file10-yes'), + ('include', '<', 'file10-no'), + ('include', '"', 'file11-no-1'), + ('include', '"', 'file11-no-2'), + ('include', '"', 'file11-yes'), + ('include', '<', 'file12-no-1'), + ('include', '<', 'file12-yes'), + ('include', '<', 'file12-no-2'), + ('include', '"', 'file13-yes'), + ('include', '"', 'file13-no-1'), + ('include', '"', 'file13-no-2'), + ('include', '<', 'file14-yes'), + ('include', '<', 'file14-no-1'), + ('include', '<', 'file14-no-2'), + ] + + if_defined_expect = [ + ('include', '"', 'file15-yes'), + ('include', '<', 'file16-no'), + ('include', '<', 'file16-yes'), + ('include', '"', 'file17-yes'), + ('include', '<', 'file18-no'), + ('include', '<', 'file18-yes'), + ] + + expression_expect = [ + ('include', '"', 'file19-no'), + ('include', '"', 'file19-yes'), + ('include', '<', 'file20-no'), + ('include', '<', 'file20-yes'), + ('include', '"', 'file21-no'), + ('include', '"', 'file21-yes'), + ('include', '<', 'file22-yes'), + ('include', '<', 'file22-no'), + ('include', '"', 'file23-no'), + ('include', '"', 'file23-yes'), + ('include', '<', 'file24-yes'), + ('include', '<', 'file24-no'), + ('include', '"', 'file25-yes'), + ('include', '"', 'file25-no'), + ('include', '<', 'file26-yes'), + ('include', '<', 'file26-no'), + ('include', '"', 'file27-yes'), + ('include', '"', 'file27-no'), + ('include', '<', 'file28-no'), + ('include', '<', 'file28-yes'), + ('include', '"', 'file29-no'), + ('include', '"', 'file29-yes'), + ('include', '<', 'file30-yes'), + ('include', '<', 'file30-no'), + ] + + undef_expect = [ + ('include', '"', 'file31-yes'), + ('include', '"', 'file31-no'), + ('include', '<', 'file32-no'), + ('include', '<', 'file32-yes'), + ] + + macro_function_expect = [ + ('include', '"', 'file33-yes'), + ('include', '<', 'file34-yes'), + ('include', '"', 'file35-yes'), + ('include', '<', 'file36-yes'), + ('include', '"', 'file37-yes'), + ('include', '<', 'file38-yes'), + ('include', '"', 'file39-yes'), + ('include', '<', 'file40-yes'), + ] + + token_pasting_expect = [ + ('include', '"', 'file41-yes'), + ('include', '<', 'file42-yes'), + ] + +if __name__ == '__main__': + suite = unittest.TestSuite() + tclasses = [ PreProcessorTestCase, + DumbPreProcessorTestCase, + ] + for tclass in tclasses: + names = unittest.getTestCaseNames(tclass, 'test_') + suite.addTests(map(tclass, names)) + if not unittest.TextTestRunner().run(suite).wasSuccessful(): + sys.exit(1) + |