diff options
author | Steven Knight <knight@baldmt.com> | 2008-04-17 06:35:51 (GMT) |
---|---|---|
committer | Steven Knight <knight@baldmt.com> | 2008-04-17 06:35:51 (GMT) |
commit | 960cd6ed58826dd88163103ea721d5a7b4d2c3ca (patch) | |
tree | 340da0ff0ef5002982fcfd5ee2a2a5e34a4bf5e0 /src | |
parent | f08434b5d652af0b471ac4c441de7d5476dff054 (diff) | |
download | SCons-960cd6ed58826dd88163103ea721d5a7b4d2c3ca.zip SCons-960cd6ed58826dd88163103ea721d5a7b4d2c3ca.tar.gz SCons-960cd6ed58826dd88163103ea721d5a7b4d2c3ca.tar.bz2 |
Merged revisions 2725-2865 via svnmerge from
http://scons.tigris.org/svn/scons/branches/core
........
r2731 | stevenknight | 2008-04-01 09:22:37 -0700 (Tue, 01 Apr 2008) | 4 lines
Fix test/Interactive/version.py, which still had the old hard-coded
Copyright string, by moving the test copy of the Copyright string
into QMTest/TestSCons.py, where it's available to all test scripts.
........
r2732 | stevenknight | 2008-04-02 13:28:32 -0700 (Wed, 02 Apr 2008) | 3 lines
Issues 317, 323, 329: Ignore the -d, -e and -w options, which we're
not going to implement.
........
r2734 | stevenknight | 2008-04-05 07:50:49 -0700 (Sat, 05 Apr 2008) | 2 lines
Fix bootstrap.py for pre-2.3 versions of Python, which don't have __file__.
........
r2735 | stevenknight | 2008-04-05 07:54:44 -0700 (Sat, 05 Apr 2008) | 2 lines
Fix the non-breaking-hyphen regular expression(s) for Python 1.5.
........
r2736 | garyo | 2008-04-05 19:15:52 -0700 (Sat, 05 Apr 2008) | 1 line
Added Users Guide section for AddMethod.
........
r2737 | stevenknight | 2008-04-08 09:43:31 -0700 (Tue, 08 Apr 2008) | 3 lines
Don't raise strings as exceptions in the exception-handling unit tests.
That's now deprecated Python behavior.
........
r2738 | stevenknight | 2008-04-08 10:36:00 -0700 (Tue, 08 Apr 2008) | 5 lines
Address a serious inefficiency in Java builds by moving the pre-build
check for whether source files exist from the Node class to the Executor
class, so we only have to perform the check once when building a whole
bunch of targets from a whole bunch of sources.
........
r2739 | stevenknight | 2008-04-08 13:53:44 -0700 (Tue, 08 Apr 2008) | 2 lines
Add a backwards-compatibility version of the "itertools" module.
........
r2740 | stevenknight | 2008-04-08 14:12:05 -0700 (Tue, 08 Apr 2008) | 2 lines
Issue 1961: speed up SCons.Util.to_String*() functions. (Benoit Belley)
........
r2741 | stevenknight | 2008-04-08 14:15:36 -0700 (Tue, 08 Apr 2008) | 2 lines
Use the itertools.izip() method for some key Node.FS methods. (Benoit Belley)
........
r2742 | stevenknight | 2008-04-08 14:35:35 -0700 (Tue, 08 Apr 2008) | 2 lines
Issues 1961: use izip() instead of zip() where possible (Benoit Belley)
........
r2743 | stevenknight | 2008-04-08 14:48:08 -0700 (Tue, 08 Apr 2008) | 3 lines
Issue 1961: more efficient get_contents() implementation for Python
function Actions. (Benoit Belley)
........
r2744 | stevenknight | 2008-04-08 17:55:30 -0700 (Tue, 08 Apr 2008) | 3 lines
Issue 1961: make SCons.Node.* state variables global in Taskmaster.py so
we avoid unneceesary attribute fetches. (Benoit Belley)
........
r2745 | stevenknight | 2008-04-08 20:40:31 -0700 (Tue, 08 Apr 2008) | 2 lines
Issue 1961: Optimize the code in Node.get_binfo(). (Benoit Belley)
........
r2746 | stevenknight | 2008-04-08 21:05:17 -0700 (Tue, 08 Apr 2008) | 3 lines
Issue 1961: Enhance the backwards-compatibility sets() module with the
ability to compare sets. (Benoit Belley)
........
r2747 | stevenknight | 2008-04-08 22:11:20 -0700 (Tue, 08 Apr 2008) | 2 lines
Issue 1961: Optimize Executor.scan(). (Benoit Belley)
........
r2748 | stevenknight | 2008-04-08 22:17:55 -0700 (Tue, 08 Apr 2008) | 3 lines
Python 1.5 compatibility: use for p in paths.keys() for dictionaries.
(Benoit Belley)
........
r2749 | stevenknight | 2008-04-08 22:54:02 -0700 (Tue, 08 Apr 2008) | 10 lines
Issue 1961: additional build optimizations:
-- Make taskmastertrace output more useful and readable.
-- Move dependency cycle checking to avoid re-doing it.
-- Have Nodes use sets, not dictionaries, to track various things.
-- Eliminate intermediate function calls from fetching Node children.
-- Add a Task.needs_execute() method to avoid doing that check over and
over as part of needs_execute().
-- Remove the unused Node.found_includes attribute.
(Benoit Belley)
........
r2750 | stevenknight | 2008-04-09 14:47:44 -0700 (Wed, 09 Apr 2008) | 4 lines
Make target Java .class files depend *only* on the input .java files as
their sources when determining if they require rebuilding. This eliminates
O(NxM) checking for every single edge in a big, overly-connected DAG mesh.
........
r2751 | stevenknight | 2008-04-09 16:58:41 -0700 (Wed, 09 Apr 2008) | 2 lines
Remove 0.95 and 0.96* release notes.
........
r2752 | stevenknight | 2008-04-10 02:24:50 -0700 (Thu, 10 Apr 2008) | 2 lines
Issue 1956: Fix --debug=stree printing its tree twice. (Benoit Belley)
........
r2753 | stevenknight | 2008-04-10 02:33:28 -0700 (Thu, 10 Apr 2008) | 2 lines
Issue 1896: Add support for the GDC D language compiler. (Matthew Wesley)
........
r2754 | stevenknight | 2008-04-10 02:39:24 -0700 (Thu, 10 Apr 2008) | 2 lines
Fix tabs.
........
r2755 | stevenknight | 2008-04-10 02:41:50 -0700 (Thu, 10 Apr 2008) | 3 lines
Issue 1964: Fix passing variable names in a list to Return() (as
already documented in the man page!) (Mike Wake)
........
r2756 | stevenknight | 2008-04-10 02:55:40 -0700 (Thu, 10 Apr 2008) | 2 lines
Support the ability to download 2.6 candidate releases (e.g. 2.6a2).
........
r2757 | stevenknight | 2008-04-10 02:58:35 -0700 (Thu, 10 Apr 2008) | 3 lines
Issue 1669: Fix the ability to use LoadableModule() under MinGW.
(Johan Boule)
........
r2758 | stevenknight | 2008-04-10 03:03:15 -0700 (Thu, 10 Apr 2008) | 3 lines
Update the test/Interactive/tree.py script for Benoit's fix to
remove duplicate tree printing.
........
r2759 | stevenknight | 2008-04-10 06:43:44 -0700 (Thu, 10 Apr 2008) | 3 lines
Fix Tool/dmd.py when no D compiler is installed -- we don't want to
search for a path if the result is None.
........
r2760 | GregNoel | 2008-04-10 15:30:34 -0700 (Thu, 10 Apr 2008) | 1 line
Issue 2009: separate Debug.caller() by functionality
........
r2761 | stevenknight | 2008-04-11 04:47:25 -0700 (Fri, 11 Apr 2008) | 3 lines
Issue 1882: Add the scons.bat directory to %PATH% so it can find python.exe.
(Anatoly Techtonik)
........
r2762 | stevenknight | 2008-04-11 09:15:22 -0700 (Fri, 11 Apr 2008) | 4 lines
Issues 1835,1901: fix the ability to list a source file multiple
times for a target by making sure we only store unique entries in the
.sconsign file.
........
r2763 | stevenknight | 2008-04-11 10:58:26 -0700 (Fri, 11 Apr 2008) | 4 lines
Issue 1882: Fix earlier patch to scons.bat by adding ~dp0;~dp0.. to
the front of %PATH%, and only executing endlocal on NT-based systems.
(Anatoly Techtonik)
........
r2764 | stevenknight | 2008-04-11 13:06:29 -0700 (Fri, 11 Apr 2008) | 4 lines
Add a Variables object and {Bool,Envum,List,Package,Path}Variable()
functions as a first step towards eventually deprecating the
Options object and {Bool,Envum,List,Package,Path}Option() functions.
........
r2765 | stevenknight | 2008-04-11 18:13:53 -0700 (Fri, 11 Apr 2008) | 3 lines
Issue 1962: Capture a test case for ListActions that contain a
command-line string containing unicode, and Python FunctionAction.
........
r2766 | stevenknight | 2008-04-11 22:03:14 -0700 (Fri, 11 Apr 2008) | 3 lines
Issue 1933: expect .py files generated by the SWIG -python option
to be in the same (sub)directory as the target.
........
r2767 | stevenknight | 2008-04-12 06:41:57 -0700 (Sat, 12 Apr 2008) | 2 lines
Remove the SCons.Options package in favor of the new SCons.Variables package.
........
r2768 | stevenknight | 2008-04-12 13:33:52 -0700 (Sat, 12 Apr 2008) | 5 lines
Issue 1971: Move the incorporation of $CCFLAGS and $SHCCFLAGS directly
into the C++ command lines (${SHCXX,CXX}COM) instead of through indirect
expansion of $CXXFLAGS and $SHCXXFLAGS. This requires removing -fPIC
from the default setting of $SHCXXFLAGS under the GNU toolchain.
........
r2769 | stevenknight | 2008-04-13 07:01:27 -0700 (Sun, 13 Apr 2008) | 3 lines
Fix __all__ definitions in the Variables/*Variable.py file that were
cause epydoc to blow up when generating developer documentation.
........
r2770 | stevenknight | 2008-04-13 11:47:49 -0700 (Sun, 13 Apr 2008) | 3 lines
Add variables= keyword argument to Environment() creation as a first
step towards deprecating options=.
........
r2771 | stevenknight | 2008-04-13 11:54:19 -0700 (Sun, 13 Apr 2008) | 2 lines
Correct underscore-instead-of-hyphen misspellings in option names.
........
r2772 | bdbaddog | 2008-04-13 17:39:21 -0700 (Sun, 13 Apr 2008) | 3 lines
Changes to fix 15 tests which were failing on cygwin.
........
r2773 | GregNoel | 2008-04-13 22:31:07 -0700 (Sun, 13 Apr 2008) | 1 line
Various fixes to tests on Darwin using 1.5.2
........
r2774 | stevenknight | 2008-04-14 15:00:44 -0700 (Mon, 14 Apr 2008) | 2 lines
Python 1.5.2 fix in new test/SWIG/subdir.py script (use the -classic option).
........
r2775 | belley | 2008-04-14 18:02:40 -0700 (Mon, 14 Apr 2008) | 32 lines
Dont use KeyboardInterrupt to stop a build! [Issue 1907]
SCons would often hang after pressing Ctrl-C. I started investigating and I
realized that most of the Python libraries are not really safe with respect to
asynchronous exceptions. Although,there are enough try/finally blocks to handle
exceptions thrown synchronously by the library code, the Python libraries are
not always protected against exceptions being thrown asynchronously, such as a
KeyboardInterrupt being thrown at a completely random location.
For example, the function Queue.empty() does not protect its mutex with a
try/finally block. If the KeyboardInterrupt exception gets thrown while the
mutex is held, any further attempt to access the Queue will lead to dead-lock
(explaining why SCons hangs sometimes after pressing CTRL-C). Even the
threading.Condition condition variables are not async-exception safe. It
therefore seems a lost battle to try to stop a build by raising an exception.
Instead, I have implemented a signal handler that tells the Jobs (and its
associated Taskmaster) to stop the build. I have been careful to wait after the
.sconsign file has been written back to re-install the default SIGINT signal
handler that raises a KeyboardInterrupt exception.
This patch is submitted against changeset 2773 of branches/core. The
regression test suite has been run on RHEL4 using Pyhon 2.5.1 and
1.5.2. My team has been using an SCons build with this patch for a
while now on Windows, Linux and OSX.
See:
http://scons.tigris.org/issues/show_bug.cgi?id=1907
Benoit Belley
........
r2777 | cournape | 2008-04-14 20:11:56 -0700 (Mon, 14 Apr 2008) | 3 lines
Initialized merge tracking via "svnmerge" with revisions "1-2776" from
http://scons.tigris.org/svn/scons/branches/fortran_refactor
........
r2788 | stevenknight | 2008-04-14 22:09:27 -0700 (Mon, 14 Apr 2008) | 2 lines
Fix the print the "script" line in the --version output.
........
r2789 | stevenknight | 2008-04-14 22:18:27 -0700 (Mon, 14 Apr 2008) | 2 lines
Add a __COPYRIGHT__ line.
........
r2790 | stevenknight | 2008-04-14 22:20:39 -0700 (Mon, 14 Apr 2008) | 3 lines
Issue 2008: in checkpoint releases, use a '.' to separate (e.g.) 0.98.0
from 0d20080414.
........
r2817 | belley | 2008-04-15 06:44:21 -0700 (Tue, 15 Apr 2008) | 13 lines
TestSCons.up_to_date() should use match_re_dotall
I changed TestSCons.up_to_date() to use match_re_dotall instead of
match_exact. This is necessary so that I can call up_to_date() with
the TestSCons.deprecated_python_expr error message in one of my test.
Note that TestSCons.not_up_to_date() is already using match_re_dotall.
Ran the test suite on both Python 2.5.1 and 1.5.2.
Benoit
........
r2818 | belley | 2008-04-15 12:10:52 -0700 (Tue, 15 Apr 2008) | 10 lines
Improved the multiple-parents.py test to also tests the following
cases:
d) Some children are ignored
e) Some children are pre-requesites
f) Some sources are missing
The test still passes. No extra bug were found.
........
r2850 | stevenknight | 2008-04-16 11:15:24 -0700 (Wed, 16 Apr 2008) | 4 lines
Fix problems with the __del__() method referencing other module functions
through global variables, which can get deleted out from under us at
shutdown.
........
r2851 | stevenknight | 2008-04-16 11:17:07 -0700 (Wed, 16 Apr 2008) | 4 lines
Fix use of --interactive with -u/-U/-D and VariantDir() by making the
method that cleans Node states between interactive commands aware of
the alter_targets() method that tells us about an associated VariantDir().
........
Diffstat (limited to 'src')
59 files changed, 1583 insertions, 1206 deletions
diff --git a/src/CHANGES.txt b/src/CHANGES.txt index dee5951..9550392 100644 --- a/src/CHANGES.txt +++ b/src/CHANGES.txt @@ -8,6 +8,72 @@ +RELEASE X.XX - XXX + + From Benoit Belley: + + - Speed up the SCons.Util.to_string*() functions. + + - Optimize various Node intialization and calculations. + + - Optimize Executor scanning code. + + - Optimize Taskmaster execution, including dependency-cycle checking. + + - Fix the --debug=stree option so it prints its tree once, not twice. + + From Johan BoulĂ: + + - Fix the ability to use LoadableModule() under MinGW. + + From Steven Knight: + + - Make the -d, -e, -w and --no-print-directory options "Ignored for + compatibility." (We're not going to implement them.) + + - Fix a serious inefficiency in how SCons checks for whether any source + files are missing when a Builder call creates many targets from many + input source files. + + - In Java projects, make the target .class files depend only on the + specific source .java files where the individual classes are defined. + + - Don't store duplicate source file entries in the .sconsign file so + we don't endlessly rebuild the target(s) for no reason. + + - Add a Variables object as the first step towards deprecating the + Options object name. Similarly, add BoolVariable(), EnumVariable(), + ListVariable(), PackageVariable() and PathVariable() functions + as first steps towards replacing BoolOption(), EnumOption(), + ListOption(), PackageOption() and PathOption(). + + - Change the options= keyword argument to the Environment() function + to variables=, to avoid confusion with SCons command-line options. + Continue supporting the options= keyword for backwards compatibility. + + - When $SWIGFLAGS contains the -python flag, expect the generated .py + file to be in the same (sub)directory as the target. + + - When compiling C++ files, allow $CCFLAGS settings to show up on the + command line even when $CXXFLAGS has been redefined. + + - Fix --interactive with -u/-U/-D when a VariantDir() is used. + + From Anatoly Techtonik: + + - Have the scons.bat file add the script execution directory to its + local %PATH% on Windows, so the Python executable can be found. + + From Mike Wake: + + - Fix passing variable names as a list to the Return() function. + + From Matthew Wesley: + + - Add support for the GDC 'D' language compiler. + + + RELEASE 0.98 - Sun, 30 Mar 2008 23:33:05 -0700 From Benoit Belley: diff --git a/src/RELEASE.txt b/src/RELEASE.txt index 0c4f905..7eaaa1c 100644 --- a/src/RELEASE.txt +++ b/src/RELEASE.txt @@ -20,10 +20,41 @@ more effectively, please sign up for the scons-users mailing list at: -RELEASE 0.98 - Sun, 30 Mar 2008 23:33:05 -0700 - - This is the ninth beta release of SCons. Please consult the - CHANGES.txt file for a list of specific changes since last release. +RELEASE 0.98.1 - XXX + + This is an update to the ninth beta release of SCons. Please consult + the CHANGES.txt file for a list of specific changes since last release. + + Please note the following important changes since release 0.98: + + -- SCONS NO LONGER SETS THE GNU TOOLCHAIN -fPIC FLAG IN $SHCXXFLAGS + + The GNU toolchain support in previous versions of SCons would + add the -fPIC flag to the $SHCXXFLAGS construction variable. + The -fPIC flag has been now been removed from the default + $SHCXXFLAGS setting. Instead, the $SHCXXCOM construction variable + (the default SCons command line for compiling shared objects + from C++ source files) has been changed to add the $SHCCFLAGS + variable, which contains the -fPIC flag. + + This change was made in order to make the behavior of the default + C++ compilation line including $SHCCFLAGS consistent with the + default C compilation line including $CCFLAGS. + + This change should have no impact on configurations that use + the default $SHCXXCOM command line. It may have an impact on + configurations that were using the default $SHCXXFLAGS value + *without* the $SHCCFLAGS variable to get the -fPIC flag into a + custom command line. You can fix these by adding the $SHCCFLAGS + to the custom command line. + + Adding $SHCCFLAGS is backwards compatible with older SCons + releases, although it might cause the -fPIC flag to be repeated + on the command line if you execute it on an older version of + SCons that sets -fPIC in both the $SHCCLAFGS and $SHCXXFLAGS + variables. Duplicating the -fPIC flag on the g++ command line + will not cause any compilation problems, but the change to the + command line may cause SCons to rebuild object files. Please note the following important changes since release 0.97.0d20071212: @@ -418,442 +449,108 @@ RELEASE 0.98 - Sun, 30 Mar 2008 23:33:05 -0700 might cause a compatibility issue if a script or other utility looks for an exact match of the previous text. - Please note the following important changes since release 0.96.93: - - -- THE --debug=memoizer OPTION NOW REQUIRES PYTHON 2.2 OR LATER - - The --debug=memoizer option now prints a warning message and - does nothing if SCons is run on a version of Python that does - not support metaclasses (earlier than Python 2.2). - - -- THE --debug=nomemoizer OPTION DOES NOTHING AND IS NOW DEPRECATED - - The --debug=nomemoizer no longer does anything and instead - now generates a warning message about being deprecated. The - --debug=nomemoizer will be removed completely in a future release. - - Please note the following important changes since release 0.96.91: - - -- /opt/bin AND /sw/bin ADDED TO DEFAULT EXECUTION PATH VARIABLES - - On all POSIX systems, the default execution PATH variable has had - the /opt/bin directory added after the /usr/local/bin directory - and before /bin and /usr/bin directories. This may cause SCons - to find and/or use different compilers, linkers, etc., if you - have any same-named utilities installed in /opt/bin that SCons - previously found in /bin or /usr/bin. - - On Mac OS X (Darwin) systems, the /sw/bin directory has been added - to the end of the default execution PATH. This may cause SCons - to find compilers, linkers and other utilities it previously did - not, although it should not otherwise change existing behavior. - - -- Configure.Checklib() ARGUMENTS HAVE CHANGED TO MATCH DOCUMENTATION - - The order of the arguments to the Configure.CheckLib() function - has changed to put the "autoadd" keyword argument last, matching - the documentation in the man page. This could cause problems - for any calls to Configure.Checklib() that were relying on the - order of the arguments. Specifying all arguments as keyword - arguments will work on both older and newer versions of SCons. - - -- env.subst() NO LONGER EXPANDS $TARGET, $SOURCES, etc. BY DEFAULT - - Calls to the env.subst() method to interpolate construction - variables in strings no longer automatically expand the special - variables $TARGET, $TARGETS, $SOURCE and $SOURCES. The keyword - variables "target" and "source" must now be set to the lists - of target and source files to be used in expansion of those - variables, when desired. - - This is most likely necessary for any env.subst() calls within - a Python function being used as an SCons action for a Builder: - - def build_it(env, target, source): - env.subst('$STRING', target=targets, source=sources) - MyBuilder = Builder(action=build_it) - - The "target" and "source" keyword arguments are backwards - compatible and can be added to SConscript files without breaking - builds on systems using older SCons releases. - - -- INTERNAL FUNCTIONS AND CLASSES HAVE MOVED FROM SCons.Util - - All internal functions and classes related to string substitution - have been moved out of the SCons.Util module into their own - SCons.Subst module. The following classes have been moved: - - Literal - SpecialAttrWrapper - NLWrapper - Targets_or_Sources - Target_or_Source - - And the following functions have moved: - - quote_spaces() - escape_list() - subst_dict() - scons_subst() - scons_subst_list() - scons_subst_once() - - If your SConscript files have been using any of these function - directly from the SCons.Util module (which they ultimately should - not be!), you will need to modify them. - - Please note the following important changes since release 0.96.90: - - -- SIGNATURES ARE NOW STORED IN AN SConsignFile() BY DEFAULT, - CAUSING LIKELY REBUILDS; SPECIAL NOTE CONCERNING INTERACTION - WITH REPOSITORIES - - The default behavior has been changed to store signature - information in a single .sconsign.dblite file in the top-level - SConstruct file. This will cause rebuilds on upgrade to 0.97, - unless you were already calling the SConsignFile() function in - your SConscript files. - - The previous default behavior was to store signature information - in a .sconsign file in each directory that contained target - files that SCons knew about. The old behavior may be preserved - by specifying: - - SConsignFile(None) - - in any SConscript file. - - If you are using the Repository feature, and are not already - using the SConsignFile() function in your build, you *must* - add "SConsignFile(None)" to your build configuration to keep - interoperating with an existing Repository that uses the old - behavior of a .sconsign file in each directory. Alternatively, - you can rebuild the Repository with the new default behavior. - - -- OTHER SIGNATURE CHANGES WILL CAUSE LIKELY REBUILDS AFTER UPGRADE - - This release adds several changes to the signature mechanism that - will cause SCons to rebuild most configurations after upgrading - (and when switching back to an earlier release from 0.97). - These changes are: - - -- NORMALIZED PATHS IN SConsignFile() DATABASES ON WINDOWS - - When using an SConsignFile() database, instead of - individual .sconsign files in each directory, the path - names are stored in normalized form with / (forward slash) - separating the elements. This may cause rebuilds when - upgrading to SCons 0.97 on Windows systems with hierarchical - build configurations. - - -- STORED DEPENDENCY PATHS ARE NOW RELATIVE TO THE TARGET - - SCons used to store the paths of all source files and - dependencies relative to the top-level SConstruct directory. - It now stores them relative to the directory of the - associated target file. This makes it possible to use - content signatures to subdivide a dependency tree without - causing unnecessary rebuilds due to an intermediate file in - one build being treated as a source file in a nother build. - - This is a step towards making it possible to write a - hierarchy of SConstruct files that allow developers - to build just one portion of a tree wherever there's an - SConstruct file. (Note that this would still require some - specific code at the top of each SConstruct file, but we - hope to make this an easier/more naturally supported thing - in the future.) - - -- PYTHON FUNCTION ACTION SIGNATURES HAVE CHANGED TO AVOID - FUTURE REBUILDS AND REBUILDS BETWEEN PYTHON VERSIONS - - SCons Actions for Python functions use the function's - byte code to generate their signature. The byte code - in older versions of Python includes indications of the - line numbers at which the function's code appeared in - its original source file, which means that changes in the - location of an otherwise unmodified Python function would - trigger rebuilds. The line number byte codes are now - removed from the signature, which will cause any targets - built by Python function Actions (including various - pre-supplied SCons Actions) to be rebuilt. - - -- REMOVED CONVERSION FROM PRE-0.96 .sconsign FORMATS - - Because this release involves so many other signature - changes that cause rebuilds, the support for automatically - converting signature information from .sconsign files - written by SCons versions prior to 0.96 has been removed. - - -- ORDER OF -o FLAGS ON CERTAIN LINK COMMAND LINES HAS CHANGED - - The -o flag that specifies an output file has been moved - on certain linker command lines to place it consistently - right after the link command itself. This will cause - recompilation of target files created by these changed - lines. - - -- F95 AND F90 COMPILERS ARE NOW PREFERRED OVER F77 - - SCons now searches for Fortran 95 and Fortran 90 compilers first - in preference to Fortran 77. This may result in a different - Fortran compiler being used by default, although as Fortran 95 and - Fortran 90 are backwards compatible with Fortran 77, this should - not cause problems for standards-compliant Fortran programs. - On systems that have multiple versions of Fortran installed, - the Fortran 77 compiler may be explicitly selected by specifying - it when creating the construction environment: - - env = Environment(tools = ['default', 'f77']) - - -- SOLARIS DEFAULT SHARED OBJECT PREFIXES AND SUFFIXES HAVE CHANGED - - On Solaris, SCons now builds shared objects from C and C++ source - files with a default prefix of "so_" and a default suffix of ".o". - The previous default suffix of ".os" caused problems when trying - to use SCons with Sun WorkShop. - - -- CACHED Configure() RESULTS ARE STORED IN A DIFFERENT FILE - - The Configure() subsystem now stores its cached results in a - different file. This may cause configuration tests to be re-run - the first time after you install 0.97. - - -- setup.py INSTALLS VERSION-NUMBERED SCRIPTS AND DIRS BY DEFAULT - - The setup.py script has been changed to always install SCons in - a version-numbered directory (e.g. /usr/local/lib/scons-0.97 - or D:\Python23\scons-0.97) and with a version-numbered script - name (scons-0.97) in addition to the usual installation of an - "scons" script name. A number of new setup.py options allow - control over what does or does not get installed, and where. - See the README.txt or README files for additional information. - - -- setup.py NOW INSTALLS MAN PAGES ON UNIX AND Linux SYSTEMS - - The SCons setup.py script now installs the "scons.1" and - "sconsign.1" man pages on UNIX and Linux systems. A - new --no-install-man - - -- BUILDERS RETURN A LIST-LIKE OBJECT, NOT A REGULAR LIST - - Builder calls now return an object that behaves like a list - (and which provides some other functionality), not an underlying - Python list. In general, this should not cause any problems, - although it introduces a subtle change in the following behavior: - - obj += env.Object('foo.c') - - If "obj" is a regular Python list, Python will no longer update - the "obj" in place, because the return value from env.Object() - is no longer the same type. Python will instead allocate a - new object and assign the local variable "obj" to it. If "obj" - is defined in an SConscript file that calls another SConscript - file containing the above code, "obj" in the first SConscript - file will not contain the object file nodes created by the - env.Object() call. - - You can guarantee that a list will be updated in place regardless - of which SConscript file defines it and which adds to it by - using the list extend() method as follows: - - obj.extend(env.Object('foo.c')) - - Please note the following important changes since release 0.96.1: - - -- DIRECTORY TREES ARE NO LONGER AUTOMATICALLY SCANNED FOR CHANGES - - Custom builders and Command() calls that accept directories as - source arguments no longer scan entire on-disk directory trees by - default. This means that their targets will not be automatically - rebuilt if a file changes on disk *unless* SCons already knows - about the file from a specific Builder or File() call. Note that - the targets will still be rebuilt correctly if a file changes - that SCons already knows about due to a Builder or other call. - - The existing behavior of scanning on-disk directory trees for - any changed file can be maintained by passing the new DirScanner - global directory scanner as the source_scanner keyword argument - to the Builder call: - - bld = Builder("build < $SOURCE > $TARGET", - source_scanner = DirScanner) - - The same keyword argument can also be supplied to any Command() - calls that need to scan directory trees on-disk for changed files: - - env.Command("archive.out", "directory", - "archiver -o $TARGET $SOURCE", - source_scanner = DirScanner) - - This change was made because scanning directories by default - could cause huge slowdowns if a configurable directory like /usr - or /usr/local was passed as the source to a Builder or Command() - call, in which case SCons would scan the entire directory tree. - - -- ParseConfig() METHOD ADDS LIBRARY FILE NAMES TO THE $LIBS VARIABLE - - The ParseConfig() method now adds library file names returned - by the specified *-config command to the $LIBS construction - variable, instead of returning them (the same way it handles - the -l option). - - -- ParseConfig() METHOD DOESN'T ADD DUPLICATES TO CONSTRUCTION VARIABLES - - By default, the ParseConfig() method now avoids adding duplicate - entries to construction variables. The old behavior may be - specified using a new "unique=0" keyword argument. - - -- WINDOWS %TEMP% and %TMP% VARIABLES ARE PROPAGATED AUTOMATICALLY - - The %TEMP% and %TMP% external environment variables are now - propagated automatically to the command execution environment on - Windows systems. - - -- OUTPUT OF Configure() SUBSYSTEM CHANGED SLIGHTLY - - The Configure() subsystem now reports tests results as "yes" and - "no" instead of "ok" and "failed." This might interfere with any - scripts that automatically parse the Configure() output from SCons. - - -- VISUAL STUDIO ATL AND MFC DIRECTORIES NOT ADDED BY DEFAULT - - When compiling with Microsoft Visual Studio, SCons no longer - adds the ATL and MFC directories to the INCLUDE and LIB - environment variables by default. If you want these directories - included in your environment variables, you should now set the - $MSVS_USE_MFC_DIRS *construction* variable when initializing - your environment: - - env = Environment(MSVS_USE_MFC_DIRS = 1) - - -- DEPRECATED GLOBAL FUNCTIONS HAVE BEEN REMOVED - - The following deprecated global functions have been removed: - ParseConfig(), SetBuildSignatureType(), SetContentSignatureType(), - SetJobs() and GetJobs(). - - -- DEPRECATED "validater" KEYWORD HAS BEEN REMOVED - - The deprecated "validater" keyword to the Options.Add() method - has been removed. - - Please note the following important changes since release 0.95: - - -- BUILDERS NOW ALWAYS RETURN A LIST OF TARGETS - - All Builder calls (both built-in like Program(), Library(), - etc. and customer Builders) now always return a list of target - Nodes. If the Builder only builds one target, the Builder - call will now return a list containing that target Node, not - the target Node itself as it used to do. - - This change should be invisibile to most normal uses of the - return values from Builder calls. It will cause an error if the - SConscript file was performing some direct manipulation of the - returned Node value. For example, an attempt to print the name - of a target returned by the Object() Builder: - - target = Object('foo.c') - # OLD WAY - print target - - Will now need to access the first element in the list returned by - the Object() call: - - target = Object('foo.c') - # NEW WAY - print target[0] - - This change was introduced to make the data type returned by Builder - calls consistent (always a list), regardless of platform or number - of returned targets. - - -- DEFAULT SConsignFile() DATABASE SCHEME HAS CHANGED - - The SConsignFile() function now uses an internally-supplied - SCons.dblite module as the default DB scheme for the .sconsign file. - If you are using the SConsignFile() function without an explicitly - specified dbm_module argument, this will cause all of your targets - to be recompiled the first time you use SCons 0.96. To preserve the - previous behavior, specify the "anydbm" module explicitly: - - import anydbm - SConsignFile('.sconsign_file_name', anydbm) - - -- INTERNAL .sconsign FILE FORMAT HAS CHANGED - - The internal format of .sconsign files has been changed. This might - cause warnings about "ignoring corrupt .sconsign files" and rebuilds - when you use SCons 0.96 for the first time in a tree that was - previously built with SCons 0.95 or earlier. - - -- INTERFACE CHANGE OF scan_check FUNCTION TO CUSTOM SCANNERS - - The scan_check function that can be supplied to a custom Scanner now - must take two arguments, the Node to be checked and a construction - environment. It previously only used the Node as an argument. - - -- DEFAULT SCANNERS NO LONGER HEED INTERNAL Scanner.add_skey() METHOD - - The internal Scanner.add_skey() method no longer works for the - default scanners, which now use construction variables to hold their - lists of suffixes. If you had a custom Tool specification that was - reaching into the internals in this way to add a suffix to one of - the following scanner, you must now add the suffix to a construction - environment through which you plan to call the scanner, as follows: - - CScan.add_skey('.x') => env.Append(CPPSUFFIXES = ['.x']) - DScan.add_skey('.x') => env.Append(DSUFFIXES = ['.x']) - FortranScan.add_skey('.x') => env.Append(FORTRANSUFFIXES = ['.x']) - - -- KEYWORD ARGUMENTS TO Builder() HAVE BEEN REMOVED - - The "node_factory" and "scanner" keyword arguments to the Builder() - function have been removed. In their place, the separate and more - flexible "target_factory," "source_factory," "target_scanner" and - "source scanner" keywords should be used instead. - - -- ALL-DIGIT FILE "EXTENSIONS" ARE NOW PART OF THE FILE BASENAME - - SCons now treats file "extensions" that contain all digits (for - example, "file.123") as part of the file basename, for easier - handling of version numbers in the names of shared libraries - and other files. Builders will now add their file extensions to - file names specified with all-digit extensions. If you need to - generate a file with an all-digit extension using a Builder that - adds a file extension, you can preserve the previous behavior by - wrapping the file name in a File() call. - - -- Append()/Prepend() METHODS CHANGED WHEN USING UserList OBJECTS + Please note the following planned, future changes: - The behavior of the env.Append() and env.Prepend() methods has - changed when appending a string value to a UserList, or vice versa. - They now behave like normal Python addition of a string to - a UserList. Given an initialization and an env.Append() call like: + -- THE Options OBJECT AND RELATED FUNCTIONS WILL BE DEPRECATED + + The Options object is being replaced by a new Variables + object, which uses a new Variables.AddVariable() method + where the previous interface used Options.AddOptions(). + + Similarly, the following utility functions are being replaced + by the following similarly-named functions: + + BoolOption() BoolVariable() + EnumOption() EnumVariable() + ListOption() ListVariable() + PackageOption() PackageVariable() + PathOption() PathVariable() + + And also related, the options= keyword argument when creating + construction environments with the Environment() functions is + being replaced with a variables= keyword argument. + + In some future release a deprecation warning will be added to + existing uses of the Options object, its methods, the above + utility functions, and the options= keyword argument of the + Environment() function. At some point after the deprecation + warning is added, the Options object, related functions and + options= keyword argument will be removed entirely. + + You can prepare for this by changing all your uses of the Options + object and related functions to the Variables object and the new + function names, and changing any uses of the options= keyword + argument to variables=. + + NOTE: CONVERTING TO USING THE NEW Variables OBJECT OR THE + RELATED *Variable() FUNCTIONS, OR USING THE NEW variable= + KEYWORD ARGUMENT, IS NOT BACKWARDS COMPATIBLE TO VERSIONS OF + SCons BEFORE 0.98. YOUR SConscript FILES WILL NOT WORK ON + EARLIER VERSIONS OF SCons AFTER MAKING THIS CHANGE. + + If you change SConscript files in software that you make available + for download or otherwise distribute, other users may try to + build your software with an earlier version of SCons that does + not have the Variables object or related *Variable() functions. + We recommend preparing for this in one of two ways: - env = Environment(VAR1=UserList(['foo']), VAR2='foo') - env.Append(VAR1='bar', VAR2=UserList(['bar']) + -- Make your SConscript files backwards-compatible by + modifying your calls with Python try:-except: blocks + as follows: - The resulting values of $VAR1 and $VAR2 will now be ['foo', 'b', - 'a', 'r'] and ['f', 'o', 'o', 'bar'], respectively. This is because - Python UserList objects treat strings as sequences of letters when - adding them to the value of the UserList. + try: + vars = Variables('custom.py', ARGUMENTS) + vars.AddVariables( + BoolVariable('WARNINGS', 'cmopile with -Wall', 1), + EnumVariable('DEBUG', 'debug version', 'no' + allowed_values=('yes', 'no', 'full'), + map={}, ignorecase=0), + ListVariable('SHAREDLIBS', + 'libraries to build shared', + 'all', + names = list_of_libs), + PackageVariable('X11', + 'use X11 from here', + '/usr/bin/X11'), + PathVariable('QTDIR', 'root of Qt', qtdir), + ) + except NameError: + vars = Options('custom.py', ARGUMENTS) + vars.AddOptions( + BoolOption('WARNINGS', 'cmopile with -Wall', 1), + EnumOption('DEBUG', 'debug version', 'no' + allowed_values=('yes', 'no', 'full'), + map={}, ignorecase=0), + ListOption('SHAREDLIBS', + 'libraries to build shared', + 'all', + names = list_of_libs), + PackageOption('X11', + 'use X11 from here', + '/usr/bin/X11'), + PathOption('QTDIR', 'root of Qt', qtdir), + ) + + Additionally, you can check for availability of the new + variables= keyword argument as follows: - The old behavior of yielding $VAR1 and $VAR2 values of ['foo', - 'bar'] when either variable is a UserList object now requires that - the string variables be enclosed in a list: + try: + env = Environment(variables=vars) + except TypeError: + env = Environment(options=vars) - env = Environment(VAR1=UserList(['foo']), VAR2=['foo']) - env.Append(VAR1='bar', VAR2=UserList(['bar'])) + (Note that we plan to maintain the existing Options object + name for some time, to ensure backwards compatibility, + so in practice it may be easier to just continue to use + the old name until you're reasonably sure you won't have + people trying to build your software with versions of + SCons earlier than 0.98.1.) - Note that the SCons behavior when appending to normal lists has - *not* changed, and the behavior of all of the default values that - SCons uses to initialize all construction variables has *not* - changed. This change *only* affects any cases where you explicitly - use UserList objects to initialize or append to a variable. + -- Use the EnsureSConsVersion() function to provide a + descriptive error message if your SConscript files + are executed by an earlier version of SCons: - Please note the following planned, future changes: + EnsureSConsVersion(0, 98, 1) -- THE BuildDir() METHOD AND FUNCTION WILL BE DEPRECATED diff --git a/src/engine/MANIFEST.in b/src/engine/MANIFEST.in index 093fbd9..e4e8046 100644 --- a/src/engine/MANIFEST.in +++ b/src/engine/MANIFEST.in @@ -3,6 +3,7 @@ SCons/Action.py SCons/Builder.py SCons/compat/__init__.py SCons/compat/_scons_hashlib.py +SCons/compat/_scons_itertools.py SCons/compat/_scons_optparse.py SCons/compat/_scons_sets.py SCons/compat/_scons_sets15.py @@ -27,12 +28,7 @@ SCons/Node/__init__.py SCons/Node/Alias.py SCons/Node/FS.py SCons/Node/Python.py -SCons/Options/__init__.py -SCons/Options/BoolOption.py -SCons/Options/EnumOption.py -SCons/Options/ListOption.py -SCons/Options/PackageOption.py -SCons/Options/PathOption.py +SCons/Options.py SCons/PathList.py SCons/Platform/__init__.py SCons/Platform/aix.py @@ -162,4 +158,10 @@ SCons/Tool/wix.py SCons/Tool/yacc.py SCons/Tool/zip.py SCons/Util.py +SCons/Variables/__init__.py +SCons/Variables/BoolVariable.py +SCons/Variables/EnumVariable.py +SCons/Variables/ListVariable.py +SCons/Variables/PackageVariable.py +SCons/Variables/PathVariable.py SCons/Warnings.py diff --git a/src/engine/SCons/Action.py b/src/engine/SCons/Action.py index 367174c..d2211b7 100644 --- a/src/engine/SCons/Action.py +++ b/src/engine/SCons/Action.py @@ -865,8 +865,11 @@ class FunctionAction(_ActionAction): except AttributeError: contents = self.funccontents - return contents + env.subst(string.join(map(lambda v: '${'+v+'}', - self.varlist))) + result = [contents] + for v in self.varlist: + result.append(env.subst('${'+v+'}')) + + return string.join(result, '') def get_implicit_deps(self, target, source, env): return [] diff --git a/src/engine/SCons/ActionTests.py b/src/engine/SCons/ActionTests.py index 74664b0..f25f232 100644 --- a/src/engine/SCons/ActionTests.py +++ b/src/engine/SCons/ActionTests.py @@ -1006,10 +1006,13 @@ class CommandActionTestCase(unittest.TestCase): expect_nonexecutable = 1 elif sys.platform == 'cygwin': expect_nonexistent = 127 - expect_nonexecutable = 127 + # Newer cygwin seems to return 126 for following + expect_nonexecutable_file = 126 + expect_nonexecutable_dir = 127 else: expect_nonexistent = 127 - expect_nonexecutable = 126 + expect_nonexecutable_file = 126 + expect_nonexecutable_dir = 126 # Test that a nonexistent command returns 127 act = SCons.Action.CommandAction(python + "_no_such_command_") @@ -1020,12 +1023,12 @@ class CommandActionTestCase(unittest.TestCase): dir, tail = os.path.split(python) act = SCons.Action.CommandAction(dir) r = act([], [], env.Clone(out = outfile)) - assert r.status == expect_nonexecutable, r.status + assert r.status == expect_nonexecutable_file, r.status # Test that trying to execute a non-executable file returns 126 act = SCons.Action.CommandAction(outfile) r = act([], [], env.Clone(out = outfile)) - assert r.status == expect_nonexecutable, r.status + assert r.status == expect_nonexecutable_dir, r.status act = SCons.Action.CommandAction('%s %s 1' % (_python_, exit_py)) r = act([], [], env) diff --git a/src/engine/SCons/Debug.py b/src/engine/SCons/Debug.py index c54eae2..095f5a2 100644 --- a/src/engine/SCons/Debug.py +++ b/src/engine/SCons/Debug.py @@ -115,39 +115,57 @@ else: res = resource.getrusage(resource.RUSAGE_SELF) return res[4] - - -caller_dicts = {} - -def caller(*backlist): +# returns caller's stack +def caller_stack(*backlist): import traceback if not backlist: backlist = [0] result = [] for back in backlist: tb = traceback.extract_stack(limit=3+back) - key = tb[1][:3] - try: - entry = caller_dicts[key] - except KeyError: - entry = caller_dicts[key] = {} key = tb[0][:3] - entry[key] = entry.get(key, 0) + 1 result.append('%s:%d(%s)' % func_shorten(key)) return result +caller_bases = {} +caller_dicts = {} + +# trace a caller's stack +def caller_trace(back=0): + import traceback + tb = traceback.extract_stack(limit=3+back) + tb.reverse() + callee = tb[1][:3] + caller_bases[callee] = caller_bases.get(callee, 0) + 1 + for caller in tb[2:]: + caller = callee + caller[:3] + try: + entry = caller_dicts[callee] + except KeyError: + caller_dicts[callee] = entry = {} + entry[caller] = entry.get(caller, 0) + 1 + callee = caller + +# print a single caller and its callers, if any +def _dump_one_caller(key, file, level=0): + l = [] + for c,v in caller_dicts[key].items(): + l.append((-v,c)) + l.sort() + leader = ' '*level + for v,c in l: + file.write("%s %6d %s:%d(%s)\n" % ((leader,-v) + func_shorten(c[-3:]))) + if caller_dicts.has_key(c): + _dump_one_caller(c, file, level+1) + +# print each call tree def dump_caller_counts(file=sys.stdout): - keys = caller_dicts.keys() + keys = caller_bases.keys() keys.sort() for k in keys: - file.write("Callers of %s:%d(%s):\n" % func_shorten(k)) - counts = caller_dicts[k] - callers = counts.keys() - callers.sort() - for c in callers: - #file.write(" counts[%s] = %s\n" % (c, counts[c])) - t = ((counts[c],) + func_shorten(c)) - file.write(" %6d %s:%d(%s)\n" % t) + file.write("Callers of %s:%d(%s), %d calls:\n" + % (func_shorten(k) + (caller_bases[k],))) + _dump_one_caller(k, file) shorten_list = [ ( '/scons/SCons/', 1), @@ -168,10 +186,8 @@ def func_shorten(func_tuple): if i >= 0: if t[1]: i = i + len(t[0]) - f = f[i:] - break - return (f,)+func_tuple[1:] - + return (f[i:],)+func_tuple[1:] + return func_tuple TraceFP = {} diff --git a/src/engine/SCons/Environment.py b/src/engine/SCons/Environment.py index ce98034..82744e0 100644 --- a/src/engine/SCons/Environment.py +++ b/src/engine/SCons/Environment.py @@ -857,7 +857,7 @@ class Base(SubstitutionEnvironment): platform=None, tools=None, toolpath=None, - options=None, + variables=None, parse_flags = None, **kw): """ @@ -901,14 +901,19 @@ class Base(SubstitutionEnvironment): self._dict['PLATFORM'] = str(platform) platform(self) - # Apply the passed-in variables and customizable options to the + # Apply the passed-in and customizable variables to the # environment before calling the tools, because they may use # some of them during initialization. + if kw.has_key('options'): + # Backwards compatibility: they may stll be using the + # old "options" keyword. + variables = kw['options'] + del kw['options'] apply(self.Replace, (), kw) keys = kw.keys() - if options: - keys = keys + options.keys() - options.Update(self) + if variables: + keys = keys + variables.keys() + variables.Update(self) save = {} for k in keys: @@ -927,7 +932,7 @@ class Base(SubstitutionEnvironment): tools = ['default'] apply_tools(self, tools, toolpath) - # Now restore the passed-in variables and customized options + # Now restore the passed-in and customized variables # to the environment, since the values the user set explicitly # should override any values set by the tools. for key, val in save.items(): diff --git a/src/engine/SCons/EnvironmentTests.py b/src/engine/SCons/EnvironmentTests.py index 4f48da9..8ebbbbc 100644 --- a/src/engine/SCons/EnvironmentTests.py +++ b/src/engine/SCons/EnvironmentTests.py @@ -844,8 +844,8 @@ class BaseTestCase(unittest.TestCase,TestEnvironmentFixture): assert not env1.has_key('__env__') assert not env2.has_key('__env__') - def test_options(self): - """Test that options only get applied once.""" + def test_variables(self): + """Test that variables only get applied once.""" class FakeOptions: def __init__(self, key, val): self.calls = 0 @@ -858,7 +858,7 @@ class BaseTestCase(unittest.TestCase,TestEnvironmentFixture): self.calls = self.calls + 1 o = FakeOptions('AAA', 'fake_opt') - env = Environment(options=o, AAA='keyword_arg') + env = Environment(variables=o, AAA='keyword_arg') assert o.calls == 1, o.calls assert env['AAA'] == 'fake_opt', env['AAA'] diff --git a/src/engine/SCons/Executor.py b/src/engine/SCons/Executor.py index 7222042..a9417bc 100644 --- a/src/engine/SCons/Executor.py +++ b/src/engine/SCons/Executor.py @@ -160,6 +160,16 @@ class Executor: self.sources_need_sorting = False return self.sources + def prepare(self): + """ + Preparatory checks for whether this Executor can go ahead + and (try to) build its targets. + """ + for s in self.get_sources(): + if s.missing(): + msg = "Source `%s' not found, needed by target `%s'." + raise SCons.Errors.StopError, msg % (s, self.targets[0]) + def add_pre_action(self, action): self.pre_actions.append(action) @@ -221,40 +231,35 @@ class Executor: This essentially short-circuits an N*M scan of the sources for each individual target, which is a hell of a lot more efficient. """ - map(lambda N: N.disambiguate(), node_list) - env = self.get_build_env() - select_specific_scanner = lambda t: (t[0], t[1].select(t[0])) - remove_null_scanners = lambda t: not t[1] is None - add_scanner_path = lambda t, s=self: \ - (t[0], t[1], s.get_build_scanner_path(t[1])) + + deps = [] if scanner: - scanner_list = map(lambda n, s=scanner: (n, s), node_list) + for node in node_list: + node.disambiguate() + scanner = scanner.select(node) + if not scanner: + continue + path = self.get_build_scanner_path(scanner) + deps.extend(node.get_implicit_deps(env, scanner, path)) else: kw = self.get_kw() - get_initial_scanners = lambda n, e=env, kw=kw: \ - (n, n.get_env_scanner(e, kw)) - scanner_list = map(get_initial_scanners, node_list) - scanner_list = filter(remove_null_scanners, scanner_list) - - scanner_list = map(select_specific_scanner, scanner_list) - scanner_list = filter(remove_null_scanners, scanner_list) - scanner_path_list = map(add_scanner_path, scanner_list) - - deps = [] - for node, scanner, path in scanner_path_list: - deps.extend(node.get_implicit_deps(env, scanner, path)) + for node in node_list: + node.disambiguate() + scanner = node.get_env_scanner(env, kw) + if not scanner: + continue + scanner = scanner.select(node) + if not scanner: + continue + path = self.get_build_scanner_path(scanner) + deps.extend(node.get_implicit_deps(env, scanner, path)) deps.extend(self.get_implicit_deps()) for tgt in self.targets: tgt.add_to_implicit(deps) - def get_missing_sources(self): - """ - """ - return filter(lambda s: s.missing(), self.get_sources()) - def _get_unignored_sources_key(self, ignore=()): return tuple(ignore) @@ -345,3 +350,5 @@ class Null(_Executor): return None def cleanup(self): pass + def prepare(self): + pass diff --git a/src/engine/SCons/ExecutorTests.py b/src/engine/SCons/ExecutorTests.py index 368e034..12c80b4 100644 --- a/src/engine/SCons/ExecutorTests.py +++ b/src/engine/SCons/ExecutorTests.py @@ -285,6 +285,24 @@ class ExecutorTestCase(unittest.TestCase): x.get_sources() assert x.sources == ['s1', 's2', 's3', 's4'], x.sources + def test_prepare(self): + """Test the Executor's prepare() method""" + env = MyEnvironment() + t1 = MyNode('t1') + s1 = MyNode('s1') + s2 = MyNode('s2') + s3 = MyNode('s3') + x = SCons.Executor.Executor('b', env, [{}], [t1], [s1, s2, s3]) + + s2.missing_val = True + + try: + r = x.prepare() + except SCons.Errors.StopError, e: + assert str(e) == "Source `s2' not found, needed by target `t1'.", e + else: + raise AssertionError, "did not catch expected StopError: %s" % r + def test_add_pre_action(self): """Test adding pre-actions to an Executor""" x = SCons.Executor.Executor('b', 'e', 'o', 't', ['s1', 's2']) @@ -403,16 +421,6 @@ class ExecutorTestCase(unittest.TestCase): assert t1.implicit == ['scanner-s1', 'scanner-s2'], t1.implicit assert t2.implicit == ['scanner-s1', 'scanner-s2'], t2.implicit - def test_get_missing_sources(self): - """Test the ability to check if any sources are missing""" - env = MyEnvironment() - targets = [MyNode('t')] - sources = [MyNode('s1'), MyNode('s2')] - x = SCons.Executor.Executor('b', env, [{}], targets, sources) - sources[0].missing_val = 1 - missing = x.get_missing_sources() - assert missing == [sources[0]], missing - def test_get_unignored_sources(self): """Test fetching the unignored source list""" env = MyEnvironment() diff --git a/src/engine/SCons/Job.py b/src/engine/SCons/Job.py index 7b51409..d93952c 100644 --- a/src/engine/SCons/Job.py +++ b/src/engine/SCons/Job.py @@ -33,6 +33,9 @@ __revision__ = "__FILE__ __REVISION__ __DATE__ __DEVELOPER__" import SCons.compat +import os +import signal + # The default stack size (in kilobytes) of the threads used to execute # jobs in parallel. @@ -44,6 +47,7 @@ import SCons.compat default_stack_size = 256 +interrupt_msg = 'Build interrupted.' class Jobs: """An instance of this class initializes N jobs, and provides @@ -80,21 +84,73 @@ class Jobs: self.job = Serial(taskmaster) self.num_jobs = 1 - def run(self): - """run the job""" + self.job.interrupted = False + + def run(self, postfunc=lambda: None): + """Run the jobs. + + postfunc() will be invoked after the jobs has run. It will be + invoked even if the jobs are interrupted by a keyboard + interrupt (well, in fact by a signal such as either SIGINT, + SIGTERM or SIGHUP). The execution of postfunc() is protected + against keyboard interrupts and is guaranteed to run to + completion.""" + self._setup_sig_handler() try: self.job.start() - except KeyboardInterrupt: - # mask any further keyboard interrupts so that scons - # can shutdown cleanly: - # (this only masks the keyboard interrupt for Python, - # child processes can still get the keyboard interrupt) - import signal - signal.signal(signal.SIGINT, signal.SIG_IGN) - raise - - def cleanup(self): - self.job.cleanup() + finally: + postfunc() + self._reset_sig_handler() + + def were_interrupted(self): + """Returns whether the jobs were interrupted by a signal.""" + return self.job.interrupted + + def _setup_sig_handler(self): + """Setup an interrupt handler so that SCons can shutdown cleanly in + various conditions: + + a) SIGINT: Keyboard interrupt + b) SIGTERM: kill or system shutdown + c) SIGHUP: Controlling shell exiting + + We handle all of these cases by stopping the taskmaster. It + turns out that it very difficult to stop the build process + by throwing asynchronously an exception such as + KeyboardInterrupt. For example, the python Condition + variables (threading.Condition) and Queue's do not seem to + asynchronous-exception-safe. It would require adding a whole + bunch of try/finally block and except KeyboardInterrupt all + over the place. + + Note also that we have to be careful to handle the case when + SCons forks before executing another process. In that case, we + want the child to exit immediately. + """ + def handler(signum, stack, parentpid=os.getpid()): + if os.getpid() == parentpid: + self.job.taskmaster.stop() + self.job.interrupted = True + else: + os._exit(2) + + self.old_sigint = signal.signal(signal.SIGINT, handler) + self.old_sigterm = signal.signal(signal.SIGTERM, handler) + try: + self.old_sighup = signal.signal(signal.SIGHUP, handler) + except AttributeError: + pass + + def _reset_sig_handler(self): + """Restore the signal handlers to their previous state (before the + call to _setup_sig_handler().""" + + signal.signal(signal.SIGINT, self.old_sigint) + signal.signal(signal.SIGTERM, self.old_sigterm) + try: + signal.signal(signal.SIGHUP, self.old_sighup) + except AttributeError: + pass class Serial: """This class is used to execute tasks in series, and is more efficient @@ -129,11 +185,18 @@ class Serial: try: task.prepare() - task.execute() - except KeyboardInterrupt: - raise + if task.needs_execute(): + task.execute() except: - task.exception_set() + if self.interrupted: + try: + raise SCons.Errors.BuildError( + task.targets[0], errstr=interrupt_msg) + except: + task.exception_set() + else: + task.exception_set() + # Let the failed() callback function arrange for the # build to stop if that's appropriate. task.failed() @@ -141,9 +204,8 @@ class Serial: task.executed() task.postprocess() + self.taskmaster.cleanup() - def cleanup(self): - pass # Trap import failure so that everything in the Job module but the # Parallel class (and its dependent classes) will work if the interpreter @@ -178,9 +240,6 @@ else: try: task.execute() - except KeyboardInterrupt: - # be explicit here for test/interrupts.py - ok = False except: task.exception_set() ok = False @@ -226,16 +285,16 @@ else: if 'prev_size' in locals().keys(): threading.stack_size(prev_size) - def put(self, obj): + def put(self, task): """Put task into request queue.""" - self.requestQueue.put(obj) + self.requestQueue.put(task) - def get(self, block = True): + def get(self): """Remove and return a result tuple from the results queue.""" - return self.resultsQueue.get(block) + return self.resultsQueue.get() - def preparation_failed(self, obj): - self.resultsQueue.put((obj, False)) + def preparation_failed(self, task): + self.resultsQueue.put((task, False)) def cleanup(self): """ @@ -309,22 +368,21 @@ else: if task is None: break - # prepare task for execution try: + # prepare task for execution task.prepare() - except KeyboardInterrupt: - raise except: - # Let the failed() callback function arrange - # for the build to stop if that's appropriate. task.exception_set() - self.tp.preparation_failed(task) - jobs = jobs + 1 - continue - - # dispatch task - self.tp.put(task) - jobs = jobs + 1 + task.failed() + task.postprocess() + else: + if task.needs_execute(): + # dispatch task + self.tp.put(task) + jobs = jobs + 1 + else: + task.executed() + task.postprocess() if not task and not jobs: break @@ -332,11 +390,20 @@ else: # back and put the next batch of tasks on the queue. while 1: task, ok = self.tp.get() - jobs = jobs - 1 + if ok: task.executed() else: + if self.interrupted: + try: + raise SCons.Errors.BuildError( + task.targets[0], errstr=interrupt_msg) + except: + task.exception_set() + + # Let the failed() callback function arrange + # for the build to stop if that's appropriate. task.failed() task.postprocess() @@ -344,5 +411,5 @@ else: if self.tp.resultsQueue.empty(): break - def cleanup(self): self.tp.cleanup() + self.taskmaster.cleanup() diff --git a/src/engine/SCons/JobTests.py b/src/engine/SCons/JobTests.py index c432581..b2a195e 100644 --- a/src/engine/SCons/JobTests.py +++ b/src/engine/SCons/JobTests.py @@ -68,6 +68,9 @@ class Task: def _do_something(self): pass + def needs_execute(self): + return True + def execute(self): self.taskmaster.test_case.failUnless(self.was_prepared, "the task wasn't prepared") @@ -120,6 +123,9 @@ class ExceptionTask: def prepare(self): self.was_prepared = 1 + def needs_execute(self): + return True + def execute(self): raise Exception @@ -200,6 +206,9 @@ class Taskmaster: def exception_set(self): pass + def cleanup(self): + pass + SaveThreadPool = None ThreadPoolCallList = [] diff --git a/src/engine/SCons/Node/FS.py b/src/engine/SCons/Node/FS.py index f73161f..6ff3833 100644 --- a/src/engine/SCons/Node/FS.py +++ b/src/engine/SCons/Node/FS.py @@ -36,6 +36,7 @@ that can be used by scripts or modules looking for the canonical default. __revision__ = "__FILE__ __REVISION__ __DATE__ __DEVELOPER__" import fnmatch +from itertools import izip import os import os.path import re @@ -1895,7 +1896,7 @@ class Dir(Base): rep_nodes = map(dir.Entry, disk_names) #rep_nodes = [ n.disambiguate() for n in rep_nodes ] rep_nodes = map(lambda n: n.disambiguate(), rep_nodes) - for node, name in zip(rep_nodes, disk_names): + for node, name in izip(rep_nodes, disk_names): n = self.Entry(name) if n.__class__ != node.__class__: n.__class__ = node.__class__ @@ -2110,7 +2111,7 @@ class FileBuildInfo(SCons.Node.BuildInfoBase): pass else: nodes = [] - for s, ni in zip(strings, nodeinfos): + for s, ni in izip(strings, nodeinfos): if not isinstance(s, SCons.Node.Node): s = ni.str_to_node(s) nodes.append(s) @@ -2119,7 +2120,7 @@ class FileBuildInfo(SCons.Node.BuildInfoBase): result = [] bkids = self.bsources + self.bdepends + self.bimplicit bkidsigs = self.bsourcesigs + self.bdependsigs + self.bimplicitsigs - for bkid, bkidsig in zip(bkids, bkidsigs): + for bkid, bkidsig in izip(bkids, bkidsigs): result.append(str(bkid) + ': ' + string.join(bkidsig.format(names=names), ' ')) result.append('%s [%s]' % (self.bactsig, self.bact)) diff --git a/src/engine/SCons/Node/NodeTests.py b/src/engine/SCons/Node/NodeTests.py index 8e9a3f8..8bceaf6 100644 --- a/src/engine/SCons/Node/NodeTests.py +++ b/src/engine/SCons/Node/NodeTests.py @@ -714,23 +714,23 @@ class NodeTestCase(unittest.TestCase): n1 = SCons.Node.Node() n1.builder_set(Builder()) node.implicit = [] - node.implicit_dict = {} - node._add_child(node.implicit, node.implicit_dict, [n1]) + node.implicit_set = set() + node._add_child(node.implicit, node.implicit_set, [n1]) node.prepare() # should not throw an exception n2 = SCons.Node.Node() n2.linked = 1 node.implicit = [] - node.implicit_dict = {} - node._add_child(node.implicit, node.implicit_dict, [n2]) + node.implicit_set = set() + node._add_child(node.implicit, node.implicit_set, [n2]) node.prepare() # should not throw an exception n3 = SCons.Node.Node() node.implicit = [] - node.implicit_dict = {} - node._add_child(node.implicit, node.implicit_dict, [n3]) + node.implicit_set = set() + node._add_child(node.implicit, node.implicit_set, [n3]) node.prepare() # should not throw an exception @@ -739,8 +739,8 @@ class NodeTestCase(unittest.TestCase): return None n4 = MyNode() node.implicit = [] - node.implicit_dict = {} - node._add_child(node.implicit, node.implicit_dict, [n4]) + node.implicit_set = set() + node._add_child(node.implicit, node.implicit_set, [n4]) exc_caught = 0 try: node.prepare() @@ -810,7 +810,7 @@ class NodeTestCase(unittest.TestCase): pass else: raise "did not catch expected exception" - assert node.sources == [zero, one, two, three, four] + assert node.sources == [zero, one, two, three, four], node.sources def test_add_ignore(self): """Test adding files whose dependencies should be ignored. @@ -1033,9 +1033,9 @@ class NodeTestCase(unittest.TestCase): node.add_source([n1, n2, n3]) node.add_dependency([n4, n5, n6]) node.implicit = [] - node.implicit_dict = {} - node._add_child(node.implicit, node.implicit_dict, [n7, n8, n9]) - node._add_child(node.implicit, node.implicit_dict, [n10, n11, n12]) + node.implicit_set = set() + node._add_child(node.implicit, node.implicit_set, [n7, n8, n9]) + node._add_child(node.implicit, node.implicit_set, [n10, n11, n12]) node.add_ignore([n2, n5, n8, n11]) kids = node.children() @@ -1064,9 +1064,9 @@ class NodeTestCase(unittest.TestCase): node.add_source([n1, n2, n3]) node.add_dependency([n4, n5, n6]) node.implicit = [] - node.implicit_dict = {} - node._add_child(node.implicit, node.implicit_dict, [n7, n8, n9]) - node._add_child(node.implicit, node.implicit_dict, [n10, n11, n12]) + node.implicit_set = set() + node._add_child(node.implicit, node.implicit_set, [n7, n8, n9]) + node._add_child(node.implicit, node.implicit_set, [n10, n11, n12]) node.add_ignore([n2, n5, n8, n11]) kids = node.all_children() @@ -1217,7 +1217,6 @@ class NodeTestCase(unittest.TestCase): n.clear() assert n.includes is None, n.includes - assert n.found_includes == {}, n.found_includes assert x.cleaned_up def test_get_subst_proxy(self): @@ -1241,32 +1240,22 @@ class NodeTestCase(unittest.TestCase): def test_postprocess(self): """Test calling the base Node postprocess() method""" n = SCons.Node.Node() - n.waiting_parents = {'foo':1, 'bar':1} + n.waiting_parents = set( ['foo','bar'] ) n.postprocess() - assert n.waiting_parents == {}, n.waiting_parents + assert n.waiting_parents == set(), n.waiting_parents def test_add_to_waiting_parents(self): """Test the add_to_waiting_parents() method""" n1 = SCons.Node.Node() n2 = SCons.Node.Node() - assert n1.waiting_parents == {}, n1.waiting_parents + assert n1.waiting_parents == set(), n1.waiting_parents r = n1.add_to_waiting_parents(n2) assert r == 1, r - assert n1.waiting_parents == {n2:1}, n1.waiting_parents + assert n1.waiting_parents == set((n2,)), n1.waiting_parents r = n1.add_to_waiting_parents(n2) assert r == 0, r - def test_call_for_all_waiting_parents(self): - """Test the call_for_all_waiting_parents() method""" - n1 = SCons.Node.Node() - n2 = SCons.Node.Node() - n1.add_to_waiting_parents(n2) - result = [] - def func(node, result=result): - result.append(node) - n1.call_for_all_waiting_parents(func) - assert result == [n1, n2], result class NodeListTestCase(unittest.TestCase): def test___str__(self): diff --git a/src/engine/SCons/Node/__init__.py b/src/engine/SCons/Node/__init__.py index 2e136f0..c765ee9 100644 --- a/src/engine/SCons/Node/__init__.py +++ b/src/engine/SCons/Node/__init__.py @@ -47,6 +47,7 @@ __revision__ = "__FILE__ __REVISION__ __DATE__ __DEVELOPER__" import SCons.compat import copy +from itertools import chain, izip import string import UserList @@ -75,7 +76,7 @@ executed = 4 failed = 5 StateString = { - 0 : "0", + 0 : "no_state", 1 : "pending", 2 : "executing", 3 : "up_to_date", @@ -202,15 +203,16 @@ class Node: # a class. (Of course, we could always still do that in the # future if we had a good reason to...). self.sources = [] # source files used to build node - self.sources_dict = {} + self.sources_set = set() + self._specific_sources = False self.depends = [] # explicit dependencies (from Depends) - self.depends_dict = {} + self.depends_set = set() self.ignore = [] # dependencies to ignore - self.ignore_dict = {} + self.ignore_set = set() self.prerequisites = SCons.Util.UniqueList() self.implicit = None # implicit (scanned) dependencies (None means not scanned yet) - self.waiting_parents = {} - self.waiting_s_e = {} + self.waiting_parents = set() + self.waiting_s_e = set() self.ref_count = 0 self.wkids = None # Kids yet to walk, when it's an array @@ -220,7 +222,6 @@ class Node: self.noclean = 0 self.nocache = 0 self.always_build = None - self.found_includes = {} self.includes = None self.attributes = self.Attrs() # Generic place to stick information about the Node. self.side_effect = 0 # true iff this node is a side effect @@ -330,24 +331,29 @@ class Node: is out-of-date and must be rebuilt, but before actually calling the method to build the Node. - This default implemenation checks that all children either exist - or are derived, and initializes the BuildInfo structure that - will hold the information about how this node is, uh, built. + This default implementation checks that explicit or implicit + dependencies either exist or are derived, and initializes the + BuildInfo structure that will hold the information about how + this node is, uh, built. + + (The existence of source files is checked separately by the + Executor, which aggregates checks for all of the targets built + by a specific action.) Overriding this method allows for for a Node subclass to remove the underlying file from the file system. Note that subclass methods should call this base class method to get the child check and the BuildInfo structure. """ - l = self.depends + for d in self.depends: + if d.missing(): + msg = "Explicit dependency `%s' not found, needed by target `%s'." + raise SCons.Errors.StopError, msg % (d, self) if not self.implicit is None: - l = l + self.implicit - missing_sources = self.get_executor().get_missing_sources() \ - + filter(lambda c: c.missing(), l) - if missing_sources: - desc = "Source `%s' not found, needed by target `%s'." % (missing_sources[0], self) - raise SCons.Errors.StopError, desc - + for i in self.implicit: + if i.missing(): + msg = "Implicit dependency `%s' not found, needed by target `%s'." + raise SCons.Errors.StopError, msg % (i, self) self.binfo = self.get_binfo() def build(self, **kw): @@ -373,7 +379,7 @@ class Node: # Clear the implicit dependency caches of any Nodes # waiting for this Node to be built. - for parent in self.waiting_parents.keys(): + for parent in self.waiting_parents: parent.implicit = None self.clear() @@ -398,7 +404,7 @@ class Node: # def add_to_waiting_s_e(self, node): - self.waiting_s_e[node] = 1 + self.waiting_s_e.add(node) def add_to_waiting_parents(self, node): """ @@ -409,23 +415,16 @@ class Node: True and False instead...) """ wp = self.waiting_parents - if wp.has_key(node): - result = 0 - else: - result = 1 - wp[node] = 1 - return result - - def call_for_all_waiting_parents(self, func): - func(self) - for parent in self.waiting_parents.keys(): - parent.call_for_all_waiting_parents(func) + if node in wp: + return 0 + wp.add(node) + return 1 def postprocess(self): """Clean up anything we don't need to hang onto after we've been built.""" self.executor_cleanup() - self.waiting_parents = {} + self.waiting_parents = set() def clear(self): """Completely clear a Node of all its cached state (so that it @@ -444,7 +443,6 @@ class Node: except AttributeError: pass self.includes = None - self.found_includes = {} def clear_memoized_values(self): self._memo = {} @@ -592,9 +590,9 @@ class Node: def add_to_implicit(self, deps): if not hasattr(self, 'implicit') or self.implicit is None: self.implicit = [] - self.implicit_dict = {} + self.implicit_set = set() self._children_reset() - self._add_child(self.implicit, self.implicit_dict, deps) + self._add_child(self.implicit, self.implicit_set, deps) def scan(self): """Scan this node's dependents for implicit dependencies.""" @@ -604,7 +602,7 @@ class Node: if not self.implicit is None: return self.implicit = [] - self.implicit_dict = {} + self.implicit_set = set() self._children_reset() if not self.has_builder(): return @@ -633,7 +631,7 @@ class Node: # one of this node's sources has changed, # so we must recalculate the implicit deps: self.implicit = [] - self.implicit_dict = {} + self.implicit_set = set() # Have the executor scan the sources. executor.scan_sources(self.builder.source_scanner) @@ -706,33 +704,44 @@ class Node: self.binfo = binfo executor = self.get_executor() - - sources = executor.get_unignored_sources(self.ignore) - - depends = self.depends - implicit = self.implicit or [] - - if self.ignore: - depends = filter(self.do_not_ignore, depends) - implicit = filter(self.do_not_ignore, implicit) - - def get_ninfo(node): - return node.get_ninfo() - - sourcesigs = map(get_ninfo, sources) - dependsigs = map(get_ninfo, depends) - implicitsigs = map(get_ninfo, implicit) + ignore_set = self.ignore_set if self.has_builder(): binfo.bact = str(executor) binfo.bactsig = SCons.Util.MD5signature(executor.get_contents()) - binfo.bsources = sources - binfo.bdepends = depends - binfo.bimplicit = implicit + if self._specific_sources: + sources = [] + for s in self.sources: + if s not in ignore_set: + sources.append(s) + else: + sources = executor.get_unignored_sources(self.ignore) + seen = set() + bsources = [] + bsourcesigs = [] + for s in sources: + if not s in seen: + seen.add(s) + bsources.append(s) + bsourcesigs.append(s.get_ninfo()) + binfo.bsources = bsources + binfo.bsourcesigs = bsourcesigs - binfo.bsourcesigs = sourcesigs + depends = self.depends + dependsigs = [] + for d in depends: + if d not in ignore_set: + dependsigs.append(d.get_ninfo()) + binfo.bdepends = depends binfo.bdependsigs = dependsigs + + implicit = self.implicit or [] + implicitsigs = [] + for i in implicit: + if i not in ignore_set: + implicitsigs.append(i.get_ninfo()) + binfo.bimplicit = implicit binfo.bimplicitsigs = implicitsigs return binfo @@ -816,7 +825,7 @@ class Node: def add_dependency(self, depend): """Adds dependencies.""" try: - self._add_child(self.depends, self.depends_dict, depend) + self._add_child(self.depends, self.depends_set, depend) except TypeError, e: e = e.args[0] if SCons.Util.is_List(e): @@ -833,7 +842,7 @@ class Node: def add_ignore(self, depend): """Adds dependencies to ignore.""" try: - self._add_child(self.ignore, self.ignore_dict, depend) + self._add_child(self.ignore, self.ignore_set, depend) except TypeError, e: e = e.args[0] if SCons.Util.is_List(e): @@ -844,8 +853,10 @@ class Node: def add_source(self, source): """Adds sources.""" + if self._specific_sources: + return try: - self._add_child(self.sources, self.sources_dict, source) + self._add_child(self.sources, self.sources_set, source) except TypeError, e: e = e.args[0] if SCons.Util.is_List(e): @@ -854,9 +865,9 @@ class Node: s = str(e) raise SCons.Errors.UserError("attempted to add a non-Node as source of %s:\n\t%s is a %s, not a Node" % (str(self), s, type(e))) - def _add_child(self, collection, dict, child): - """Adds 'child' to 'collection', first checking 'dict' to see - if it's already present.""" + def _add_child(self, collection, set, child): + """Adds 'child' to 'collection', first checking 'set' to see if it's + already present.""" #if type(child) is not type([]): # child = [child] #for c in child: @@ -864,13 +875,17 @@ class Node: # raise TypeError, c added = None for c in child: - if not dict.has_key(c): + if c not in set: + set.add(c) collection.append(c) - dict[c] = 1 added = 1 if added: self._children_reset() + def set_specific_source(self, source): + self.add_source(source) + self._specific_sources = True + def add_wkid(self, wkid): """Add a node to the list of kids waiting to be evaluated""" if self.wkids != None: @@ -882,10 +897,14 @@ class Node: # build info that it's cached so we can re-calculate it. self.executor_cleanup() - def do_not_ignore(self, node): - return node not in self.ignore + memoizer_counters.append(SCons.Memoize.CountValue('_children_get')) + + def _children_get(self): + try: + return self._memo['children_get'] + except KeyError: + pass - def _all_children_get(self): # The return list may contain duplicate Nodes, especially in # source trees where there are a lot of repeated #includes # of a tangle of .h files. Profiling shows, however, that @@ -903,21 +922,22 @@ class Node: # using dictionary keys, lose the order, and the only ordered # dictionary patterns I found all ended up using "not in" # internally anyway...) - if self.implicit is None: - return self.sources + self.depends - else: - return self.sources + self.depends + self.implicit + if self.ignore_set: + if self.implicit is None: + iter = chain(self.sources,self.depends) + else: + iter = chain(self.sources, self.depends, self.implicit) - memoizer_counters.append(SCons.Memoize.CountValue('_children_get')) + children = [] + for i in iter: + if i not in self.ignore_set: + children.append(i) + else: + if self.implicit is None: + children = self.sources + self.depends + else: + children = self.sources + self.depends + self.implicit - def _children_get(self): - try: - return self._memo['children_get'] - except KeyError: - pass - children = self._all_children_get() - if self.ignore: - children = filter(self.do_not_ignore, children) self._memo['children_get'] = children return children @@ -925,7 +945,28 @@ class Node: """Return a list of all the node's direct children.""" if scan: self.scan() - return self._all_children_get() + + # The return list may contain duplicate Nodes, especially in + # source trees where there are a lot of repeated #includes + # of a tangle of .h files. Profiling shows, however, that + # eliminating the duplicates with a brute-force approach that + # preserves the order (that is, something like: + # + # u = [] + # for n in list: + # if n not in u: + # u.append(n)" + # + # takes more cycles than just letting the underlying methods + # hand back cached values if a Node's information is requested + # multiple times. (Other methods of removing duplicates, like + # using dictionary keys, lose the order, and the only ordered + # dictionary patterns I found all ended up using "not in" + # internally anyway...) + if self.implicit is None: + return self.sources + self.depends + else: + return self.sources + self.depends + self.implicit def children(self, scan=1): """Return a list of the node's direct children, minus those @@ -1009,7 +1050,7 @@ class Node: if t: Trace(': old %s new %s' % (len(then), len(children))) result = True - for child, prev_ni in zip(children, then): + for child, prev_ni in izip(children, then): if child.changed_since_last_build(self, prev_ni): if t: Trace(': %s changed' % child) result = True @@ -1152,8 +1193,8 @@ class Node: new_bkids = new.bsources + new.bdepends + new.bimplicit new_bkidsigs = new.bsourcesigs + new.bdependsigs + new.bimplicitsigs - osig = dict(zip(old_bkids, old_bkidsigs)) - nsig = dict(zip(new_bkids, new_bkidsigs)) + osig = dict(izip(old_bkids, old_bkidsigs)) + nsig = dict(izip(new_bkids, new_bkidsigs)) # The sources and dependencies we'll want to report are all stored # as relative paths to this target's directory, but we want to diff --git a/src/engine/SCons/Options.py b/src/engine/SCons/Options.py new file mode 100644 index 0000000..0f2fde1 --- /dev/null +++ b/src/engine/SCons/Options.py @@ -0,0 +1,56 @@ +# +# __COPYRIGHT__ +# +# Permission is hereby granted, free of charge, to any person obtaining +# a copy of this software and associated documentation files (the +# "Software"), to deal in the Software without restriction, including +# without limitation the rights to use, copy, modify, merge, publish, +# distribute, sublicense, and/or sell copies of the Software, and to +# permit persons to whom the Software is furnished to do so, subject to +# the following conditions: +# +# The above copyright notice and this permission notice shall be included +# in all copies or substantial portions of the Software. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY +# KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE +# WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND +# NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE +# LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION +# OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION +# WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. +# + +__revision__ = "__FILE__ __REVISION__ __DATE__ __DEVELOPER__" + +__doc__ = """Place-holder for the old SCons.Options module hierarchy + +This is for backwards compatibility. The new equivalent is the Variables/ +class hierarchy. These will have deprecation warnings added (some day), +and will then be removed entirely (some day). +""" + +import SCons.Variables + +class Options(SCons.Variables.Variables): + + def AddOptions(self, *args, **kw): + return apply(SCons.Variables.Variables.AddVariables, + (self,) + args, + kw) + + def UnknownOptions(self, *args, **kw): + return apply(SCons.Variables.Variables.UnknownVariables, + (self,) + args, + kw) + + def FormatOptionHelpText(self, *args, **kw): + return apply(SCons.Variables.Variables.FormatVariableHelpText, + (self,) + args, + kw) + +BoolOption = SCons.Variables.BoolVariable +EnumOption = SCons.Variables.EnumVariable +ListOption = SCons.Variables.ListVariable +PackageOption = SCons.Variables.PackageVariable +PathOption = SCons.Variables.PathVariable diff --git a/src/engine/SCons/Options/.cvsignore b/src/engine/SCons/Options/.cvsignore deleted file mode 100644 index 0d20b64..0000000 --- a/src/engine/SCons/Options/.cvsignore +++ /dev/null @@ -1 +0,0 @@ -*.pyc diff --git a/src/engine/SCons/Platform/posix.py b/src/engine/SCons/Platform/posix.py index afdabe1..2a0e478 100644 --- a/src/engine/SCons/Platform/posix.py +++ b/src/engine/SCons/Platform/posix.py @@ -32,6 +32,7 @@ selection method. __revision__ = "__FILE__ __REVISION__ __DATE__ __DEVELOPER__" +import errno import os import os.path import popen2 @@ -109,21 +110,25 @@ def fork_spawn(sh, escape, cmd, args, env): def process_cmd_output(cmd_stdout, cmd_stderr, stdout, stderr): stdout_eof = stderr_eof = 0 while not (stdout_eof and stderr_eof): - (i,o,e) = select.select([cmd_stdout, cmd_stderr], [], []) - if cmd_stdout in i: - str = cmd_stdout.read() - if len(str) == 0: - stdout_eof = 1 - elif stdout != None: - stdout.write(str) - if cmd_stderr in i: - str = cmd_stderr.read() - if len(str) == 0: - #sys.__stderr__.write( "stderr_eof=1\n" ) - stderr_eof = 1 - else: - #sys.__stderr__.write( "str(stderr) = %s\n" % str ) - stderr.write(str) + try: + (i,o,e) = select.select([cmd_stdout, cmd_stderr], [], []) + if cmd_stdout in i: + str = cmd_stdout.read() + if len(str) == 0: + stdout_eof = 1 + elif stdout != None: + stdout.write(str) + if cmd_stderr in i: + str = cmd_stderr.read() + if len(str) == 0: + #sys.__stderr__.write( "stderr_eof=1\n" ) + stderr_eof = 1 + else: + #sys.__stderr__.write( "str(stderr) = %s\n" % str ) + stderr.write(str) + except select.error, (_errno, _strerror): + if _errno != errno.EINTR: + raise def exec_popen3(l, env, stdout, stderr): proc = popen2.Popen3(string.join(l), 1) diff --git a/src/engine/SCons/SConfTests.py b/src/engine/SCons/SConfTests.py index f7d33f8..25c89d2 100644 --- a/src/engine/SCons/SConfTests.py +++ b/src/engine/SCons/SConfTests.py @@ -163,9 +163,10 @@ class SConfTestCase(unittest.TestCase): def __init__(self, name): self.name = name self.state = None - self.waiting_parents = {} + self.waiting_parents = set() self.side_effects = [] self.builder = None + self.prerequisites = [] def disambiguate(self): return self def has_builder(self): diff --git a/src/engine/SCons/Script/Interactive.py b/src/engine/SCons/Script/Interactive.py index d18eec1..b1774eb 100644 --- a/src/engine/SCons/Script/Interactive.py +++ b/src/engine/SCons/Script/Interactive.py @@ -185,6 +185,16 @@ class SConsInteractiveCmd(cmd.Cmd): if not nodes: return + # Call each of the Node's alter_targets() methods, which may + # provide additional targets that ended up as part of the build + # (the canonical example being a VariantDir() when we're building + # from a source directory) and which we therefore need their + # state cleared, too. + x = [] + for n in nodes: + x.extend(n.alter_targets()[0]) + nodes.extend(x) + # Clean up so that we can perform the next build correctly. # # We do this by walking over all the children of the targets, diff --git a/src/engine/SCons/Script/Main.py b/src/engine/SCons/Script/Main.py index 80b9032..75d0117 100644 --- a/src/engine/SCons/Script/Main.py +++ b/src/engine/SCons/Script/Main.py @@ -168,28 +168,29 @@ class BuildTask(SCons.Taskmaster.Task): self.progress(self.targets[0]) return SCons.Taskmaster.Task.prepare(self) - def execute(self): - for target in self.targets: - if target.get_state() == SCons.Node.up_to_date: - continue - if target.has_builder() and not hasattr(target.builder, 'status'): - if print_time: - start_time = time.time() - global first_command_start - if first_command_start is None: - first_command_start = start_time - SCons.Taskmaster.Task.execute(self) - if print_time: - global cumulative_command_time - global last_command_end - finish_time = time.time() - last_command_end = finish_time - cumulative_command_time = cumulative_command_time+finish_time-start_time - sys.stdout.write("Command execution time: %f seconds\n"%(finish_time-start_time)) - break + def needs_execute(self): + target = self.targets[0] + if target.get_state() == SCons.Node.executing: + return True else: if self.top and target.has_builder(): display("scons: `%s' is up to date." % str(self.node)) + return False + + def execute(self): + if print_time: + start_time = time.time() + global first_command_start + if first_command_start is None: + first_command_start = start_time + SCons.Taskmaster.Task.execute(self) + if print_time: + global cumulative_command_time + global last_command_end + finish_time = time.time() + last_command_end = finish_time + cumulative_command_time = cumulative_command_time+finish_time-start_time + sys.stdout.write("Command execution time: %f seconds\n"%(finish_time-start_time)) def do_failed(self, status=2): _BuildFailures.append(self.exception[1]) @@ -207,10 +208,15 @@ class BuildTask(SCons.Taskmaster.Task): t = self.targets[0] if self.top and not t.has_builder() and not t.side_effect: if not t.exists(): - sys.stderr.write("scons: *** Do not know how to make target `%s'." % t) + errstr="Do not know how to make target `%s'." % t + sys.stderr.write("scons: *** " + errstr) if not self.options.keep_going: sys.stderr.write(" Stop.") sys.stderr.write("\n") + try: + raise SCons.Errors.BuildError(t, errstr) + except: + self.exception_set() self.do_failed() else: print "scons: Nothing to be done for `%s'." % t @@ -977,11 +983,6 @@ def _build_targets(fs, options, targets, target_top): if options.diskcheck: SCons.Node.FS.set_diskcheck(options.diskcheck) - _set_debug_values(options) - SCons.Node.implicit_cache = options.implicit_cache - SCons.Node.implicit_deps_changed = options.implicit_deps_changed - SCons.Node.implicit_deps_unchanged = options.implicit_deps_unchanged - SCons.CacheDir.cache_enabled = not options.cache_disable SCons.CacheDir.cache_debug = options.cache_debug SCons.CacheDir.cache_force = options.cache_force @@ -1133,26 +1134,29 @@ def _build_targets(fs, options, targets, target_top): memory_stats.append('before building targets:') count_stats.append(('pre-', 'build')) - try: - progress_display("scons: " + opening_message) - try: - jobs.run() - except KeyboardInterrupt: - # If we are in interactive mode, a KeyboardInterrupt - # interrupts only this current run. Return 'nodes' normally - # so that the outer loop can clean up the nodes and continue. - if options.interactive: - print "Build interrupted." - # Continue and return normally - finally: - jobs.cleanup() + def jobs_postfunc( + jobs=jobs, + options=options, + closing_message=closing_message, + failure_message=failure_message + ): + if jobs.were_interrupted(): + progress_display("scons: Build interrupted.") + global exit_status + exit_status = 2 + if exit_status: progress_display("scons: " + failure_message) else: progress_display("scons: " + closing_message) if not options.no_exec: + if jobs.were_interrupted(): + progress_display("scons: writing .sconsign file.") SCons.SConsign.write() + progress_display("scons: " + opening_message) + jobs.run(postfunc = jobs_postfunc) + memory_stats.append('after building targets:') count_stats.append(('post-', 'build')) @@ -1210,10 +1214,9 @@ def main(): parts = ["SCons by Steven Knight et al.:\n"] try: + import __main__ parts.append(version_string("script", __main__)) - except KeyboardInterrupt: - raise - except: + except (ImportError, AttributeError): # On Windows there is no scons.py, so there is no # __main__.__version__, hence there is no script version. pass @@ -1233,7 +1236,7 @@ def main(): if s: exit_status = s except KeyboardInterrupt: - print "Build interrupted." + print("scons: Build interrupted.") sys.exit(2) except SyntaxError, e: _scons_syntax_error(e) diff --git a/src/engine/SCons/Script/SConsOptions.py b/src/engine/SCons/Script/SConsOptions.py index ee34a85..2148068 100644 --- a/src/engine/SCons/Script/SConsOptions.py +++ b/src/engine/SCons/Script/SConsOptions.py @@ -26,11 +26,19 @@ __revision__ = "__FILE__ __REVISION__ __DATE__ __DEVELOPER__" import SCons.compat import optparse +import re import string import sys import textwrap try: + no_hyphen_re = re.compile(r'(\s+|(?<=[\w\!\"\'\&\.\,\?])-{2,}(?=\w))') +except re.error: + # Pre-2.0 Python versions don't have the (?<= negative + # look-behind assertion. + no_hyphen_re = re.compile(r'(\s+|-*\w{2,}-(?=\w{2,}))') + +try: from gettext import gettext except ImportError: def gettext(message): @@ -382,8 +390,16 @@ class SConsIndentedHelpFormatter(optparse.IndentedHelpFormatter): def format_option(self, option): """ A copy of the normal optparse.IndentedHelpFormatter.format_option() - method, snarfed so we can set the subsequent_indent on the - textwrap.wrap() call below... + method. This has been snarfed so we can modify text wrapping to + out liking: + + -- add our own regular expression that doesn't break on hyphens + (so things like --no-print-directory don't get broken); + + -- wrap the list of options themselves when it's too long + (the wrapper.fill(opts) call below); + + -- set the subsequent_indent when wrapping the help_text. """ # The help for each option consists of two parts: # * the opt strings and metavars @@ -410,7 +426,11 @@ class SConsIndentedHelpFormatter(optparse.IndentedHelpFormatter): opt_width = self.help_position - self.current_indent - 2 if len(opts) > opt_width: - opts = "%*s%s\n" % (self.current_indent, "", opts) + wrapper = textwrap.TextWrapper(width=self.width, + initial_indent = ' ', + subsequent_indent = ' ') + wrapper.wordsep_re = no_hyphen_re + opts = wrapper.fill(opts) + '\n' indent_first = self.help_position else: # start help on same line as opts opts = "%*s%-*s " % (self.current_indent, "", opt_width, opts) @@ -428,8 +448,10 @@ class SConsIndentedHelpFormatter(optparse.IndentedHelpFormatter): help_text = expand_default(option) # SCons: indent every line of the help text but the first. - help_lines = textwrap.wrap(help_text, self.help_width, - subsequent_indent = ' ') + wrapper = textwrap.TextWrapper(width=self.help_width, + subsequent_indent = ' ') + wrapper.wordsep_re = no_hyphen_re + help_lines = wrapper.wrap(help_text) result.append("%*s%s\n" % (indent_first, "", help_lines[0])) for line in help_lines[1:]: result.append("%*s%s\n" % (self.help_position, "", line)) @@ -503,8 +525,13 @@ def Parser(version): # options ignored for compatibility def opt_ignore(option, opt, value, parser): sys.stderr.write("Warning: ignoring %s option\n" % opt) - op.add_option("-b", "-m", "-S", "-t", - "--no-keep-going", "--stop", "--touch", + op.add_option("-b", "-d", "-e", "-m", "-S", "-t", "-w", + "--environment-overrides", + "--no-keep-going", + "--no-print-directory", + "--print-directory", + "--stop", + "--touch", action="callback", callback=opt_ignore, help="Ignored for compatibility.") @@ -557,13 +584,6 @@ def Parser(version): help = opt_config_help, metavar="MODE") - def opt_not_yet(option, opt, value, parser): - sys.stderr.write("Warning: the %s option is not yet implemented\n" % opt) - sys.exit(0) - op.add_option('-d', - action="callback", callback=opt_not_yet, - help = "Print file dependency information.") - op.add_option('-D', dest="climb_up", default=None, action="store_const", const=2, @@ -836,11 +856,12 @@ def Parser(version): # we don't want to change. These all get a "the -X option is not # yet implemented" message and don't show up in the help output. - op.add_option('-e', '--environment-overrides', - dest="environment_overrides", - action="callback", callback=opt_not_yet, - # help="Environment variables override makefiles." - help=SUPPRESS_HELP) + def opt_not_yet(option, opt, value, parser): + msg = "Warning: the %s option is not yet implemented\n" % opt + sys.stderr.write(msg) + sys.exit(0) + + op.add_option('-l', '--load-average', '--max-load', nargs=1, type="int", dest="load_average", default=0, @@ -887,16 +908,6 @@ def Parser(version): dest="no_builtin_rules", # help="Clear default environments and variables." help=SUPPRESS_HELP) - op.add_option('-w', '--print-directory', - action="callback", callback=opt_not_yet, - dest="print_directory", - # help="Print the current directory." - help=SUPPRESS_HELP) - op.add_option('--no-print-directory', - action="callback", callback=opt_not_yet, - dest="no_print_directory", - # help="Turn off -w, even if it was turned on implicitly." - help=SUPPRESS_HELP) op.add_option('--write-filenames', nargs=1, type="string", dest="write_filenames", diff --git a/src/engine/SCons/Script/SConscript.py b/src/engine/SCons/Script/SConscript.py index 36a147d..7e38ffd 100644 --- a/src/engine/SCons/Script/SConscript.py +++ b/src/engine/SCons/Script/SConscript.py @@ -39,7 +39,6 @@ import SCons.Errors import SCons.Node import SCons.Node.Alias import SCons.Node.FS -import SCons.Options import SCons.Platform import SCons.SConf import SCons.Script.Main @@ -141,7 +140,8 @@ call_stack = [] def Return(*vars, **kw): retval = [] try: - for var in vars: + fvars = SCons.Util.flatten(vars) + for var in fvars: for v in string.split(var): retval.append(call_stack[-1].globals[v]) except KeyError, x: diff --git a/src/engine/SCons/Script/__init__.py b/src/engine/SCons/Script/__init__.py index ddeaf9e..72e31d1 100644 --- a/src/engine/SCons/Script/__init__.py +++ b/src/engine/SCons/Script/__init__.py @@ -84,6 +84,7 @@ import SCons.SConf import SCons.Subst import SCons.Tool import SCons.Util +import SCons.Variables import SCons.Defaults import Main @@ -138,22 +139,31 @@ call_stack = _SConscript.call_stack Action = SCons.Action.Action AddMethod = SCons.Util.AddMethod AllowSubstExceptions = SCons.Subst.SetAllowableExceptions -BoolOption = SCons.Options.BoolOption Builder = SCons.Builder.Builder Configure = _SConscript.Configure -EnumOption = SCons.Options.EnumOption Environment = SCons.Environment.Environment #OptParser = SCons.SConsOptions.OptParser FindPathDirs = SCons.Scanner.FindPathDirs -ListOption = SCons.Options.ListOption -PackageOption = SCons.Options.PackageOption -PathOption = SCons.Options.PathOption Platform = SCons.Platform.Platform Return = _SConscript.Return Scanner = SCons.Scanner.Base Tool = SCons.Tool.Tool WhereIs = SCons.Util.WhereIs +# +BoolVariable = SCons.Variables.BoolVariable +EnumVariable = SCons.Variables.EnumVariable +ListVariable = SCons.Variables.ListVariable +PackageVariable = SCons.Variables.PackageVariable +PathVariable = SCons.Variables.PathVariable + +# Deprecated names that will go away some day. +BoolOption = SCons.Options.BoolOption +EnumOption = SCons.Options.EnumOption +ListOption = SCons.Options.ListOption +PackageOption = SCons.Options.PackageOption +PathOption = SCons.Options.PathOption + # Action factories. Chmod = SCons.Defaults.Chmod Copy = SCons.Defaults.Copy @@ -262,6 +272,9 @@ def HelpFunction(text): sconscript_reading = 0 # +def Variables(files=[], args=ARGUMENTS): + return SCons.Variables.Variables(files, args) + def Options(files=[], args=ARGUMENTS): return SCons.Options.Options(files, args) diff --git a/src/engine/SCons/Subst.py b/src/engine/SCons/Subst.py index 7a565ba..52aaae5 100644 --- a/src/engine/SCons/Subst.py +++ b/src/engine/SCons/Subst.py @@ -565,7 +565,7 @@ def scons_subst_list(strSubst, env, mode=SUBST_RAW, target=None, source=None, gv # except KeyError: # Subst_List_Strings[strSubst] = 1 # import SCons.Debug -# SCons.Debug.caller(1) +# SCons.Debug.caller_trace(1) class ListSubber(UserList.UserList): """A class to construct the results of a scons_subst_list() call. diff --git a/src/engine/SCons/Taskmaster.py b/src/engine/SCons/Taskmaster.py index 66202dc..8a0fcf7 100644 --- a/src/engine/SCons/Taskmaster.py +++ b/src/engine/SCons/Taskmaster.py @@ -52,16 +52,22 @@ __revision__ = "__FILE__ __REVISION__ __DATE__ __DEVELOPER__" import SCons.compat +from itertools import chain import operator import string import sys import traceback -import SCons.Node import SCons.Errors +import SCons.Node StateString = SCons.Node.StateString - +NODE_NO_STATE = SCons.Node.no_state +NODE_PENDING = SCons.Node.pending +NODE_EXECUTING = SCons.Node.executing +NODE_UP_TO_DATE = SCons.Node.up_to_date +NODE_EXECUTED = SCons.Node.executed +NODE_FAILED = SCons.Node.failed # A subsystem for recording stats about how different Nodes are handled by @@ -165,6 +171,17 @@ class Task: self.display(self.tm.message) self.tm.message = None + # Let the targets take care of any necessary preparations. + # This includes verifying that all of the necessary sources + # and dependencies exist, removing the target file(s), etc. + # + # As of April 2008, the get_executor().prepare() method makes + # sure that all of the aggregate sources necessary to build this + # Task's target(s) exist in one up-front check. The individual + # target t.prepare() methods check that each target's explicit + # or implicit dependencies exists, and also initialize the + # .sconsign info. + self.targets[0].get_executor().prepare() for t in self.targets: t.prepare() for s in t.side_effects: @@ -175,6 +192,17 @@ class Task: """ return self.node + def needs_execute(self): + """ + Called to determine whether the task's execute() method should + be run. + + This method allows one to skip the somethat costly execution + of the execute() method in a seperate thread. For example, + that would be unnecessary for up-to-date targets. + """ + return True + def execute(self): """ Called to execute the task. @@ -192,8 +220,6 @@ class Task: break if not everything_was_cached: self.targets[0].build() - except KeyboardInterrupt: - raise except SystemExit: exc_value = sys.exc_info()[1] raise SCons.Errors.ExplicitExit(self.targets[0], exc_value.code) @@ -212,10 +238,10 @@ class Task: the Node's callback methods. """ for t in self.targets: - if t.get_state() == SCons.Node.executing: + if t.get_state() == NODE_EXECUTING: for side_effect in t.side_effects: - side_effect.set_state(SCons.Node.no_state) - t.set_state(SCons.Node.executed) + side_effect.set_state(NODE_NO_STATE) + t.set_state(NODE_EXECUTED) def executed_with_callbacks(self): """ @@ -231,10 +257,10 @@ class Task: or not the target was an actual built target or a source Node. """ for t in self.targets: - if t.get_state() == SCons.Node.executing: + if t.get_state() == NODE_EXECUTING: for side_effect in t.side_effects: - side_effect.set_state(SCons.Node.no_state) - t.set_state(SCons.Node.executed) + side_effect.set_state(NODE_NO_STATE) + t.set_state(NODE_EXECUTED) t.built() t.visited() @@ -250,8 +276,12 @@ class Task: """ Explicit stop-the-build failure. """ - for t in self.targets: - t.set_state(SCons.Node.failed) + + # Invoke fail_continue() to clean-up the pending children + # list. + self.fail_continue() + + # Tell the taskmaster to not start any new tasks self.tm.stop() # We're stopping because of a build failure, but give the @@ -267,11 +297,43 @@ class Task: This sets failure status on the target nodes and all of their dependent parent nodes. """ + + pending_children = self.tm.pending_children + + to_visit = set() for t in self.targets: # Set failure state on all of the parents that were dependent # on this failed build. - def set_state(node): node.set_state(SCons.Node.failed) - t.call_for_all_waiting_parents(set_state) + if t.state != NODE_FAILED: + t.state = NODE_FAILED + parents = t.waiting_parents + to_visit = to_visit | parents + pending_children = pending_children - parents + + try: + while 1: + try: + node = to_visit.pop() + except AttributeError: + # Python 1.5.2 + if len(to_visit): + node = to_visit[0] + to_visit.remove(node) + else: + break + if node.state != NODE_FAILED: + node.state = NODE_FAILED + parents = node.waiting_parents + to_visit = to_visit | parents + pending_children = pending_children - parents + except KeyError: + # The container to_visit has been emptied. + pass + + # We have the stick back the pending_children list into the + # task master because the python 1.5.2 compatibility does not + # allow us to use in-place updates + self.tm.pending_children = pending_children def make_ready_all(self): """ @@ -282,9 +344,9 @@ class Task: """ self.out_of_date = self.targets[:] for t in self.targets: - t.disambiguate().set_state(SCons.Node.executing) + t.disambiguate().set_state(NODE_EXECUTING) for s in t.side_effects: - s.set_state(SCons.Node.executing) + s.set_state(NODE_EXECUTING) def make_ready_current(self): """ @@ -294,6 +356,7 @@ class Task: This is the default behavior for building only what's necessary. """ self.out_of_date = [] + needs_executing = False for t in self.targets: try: t.disambiguate().make_ready() @@ -301,13 +364,24 @@ class Task: (not t.always_build and t.is_up_to_date()) except EnvironmentError, e: raise SCons.Errors.BuildError(node=t, errstr=e.strerror, filename=e.filename) - if is_up_to_date: - t.set_state(SCons.Node.up_to_date) - else: + + if not is_up_to_date: self.out_of_date.append(t) - t.set_state(SCons.Node.executing) + needs_executing = True + + if needs_executing: + for t in self.targets: + t.set_state(NODE_EXECUTING) for s in t.side_effects: - s.set_state(SCons.Node.executing) + s.set_state(NODE_EXECUTING) + else: + for t in self.targets: + # We must invoke visited() to ensure that the node + # information has been computed before allowing the + # parent nodes to execute. (That could occur in a + # parallel build...) + t.visited() + t.set_state(NODE_UP_TO_DATE) make_ready = make_ready_current @@ -333,24 +407,25 @@ class Task: parents = {} for t in targets: - for p in t.waiting_parents.keys(): + for p in t.waiting_parents: parents[p] = parents.get(p, 0) + 1 for t in targets: for s in t.side_effects: - if s.get_state() == SCons.Node.executing: - s.set_state(SCons.Node.no_state) - for p in s.waiting_parents.keys(): - if not parents.has_key(p): - parents[p] = 1 - for p in s.waiting_s_e.keys(): + if s.get_state() == NODE_EXECUTING: + s.set_state(NODE_NO_STATE) + for p in s.waiting_parents: + parents[p] = parents.get(p, 0) + 1 + for p in s.waiting_s_e: if p.ref_count == 0: self.tm.candidates.append(p) + self.tm.pending_children.discard(p) for p, subtract in parents.items(): p.ref_count = p.ref_count - subtract if p.ref_count == 0: self.tm.candidates.append(p) + self.tm.pending_children.discard(p) for t in targets: t.postprocess() @@ -409,12 +484,15 @@ class Task: raise exc_type, exc_value, exc_traceback -def find_cycle(stack): - if stack[0] == stack[-1]: - return stack - for n in stack[-1].waiting_parents.keys(): +def find_cycle(stack, visited): + if stack[-1] in visited: + return None + visited.add(stack[-1]) + for n in stack[-1].waiting_parents: stack.append(n) - if find_cycle(stack): + if stack[0] == stack[-1]: + return stack + if find_cycle(stack, visited): return stack stack.pop() return None @@ -437,6 +515,8 @@ class Taskmaster: self.message = None self.trace = trace self.next_candidate = self.find_next_candidate + self.pending_children = set() + def find_next_candidate(self): """ @@ -505,10 +585,12 @@ class Taskmaster: self.ready_exc = None T = self.trace + if T: T.write('\nTaskmaster: Looking for a node to evaluate\n') while 1: node = self.next_candidate() if node is None: + if T: T.write('Taskmaster: No candidate anymore.\n\n') return None node = node.disambiguate() @@ -522,29 +604,27 @@ class Taskmaster: S.considered = S.considered + 1 else: S = None - - if T: T.write('Taskmaster: %s:' % repr(str(node))) - - # Skip this node if it has already been evaluated: - if state > SCons.Node.pending: + + if T: T.write('Taskmaster: Considering node <%-10s %s> and its children:\n' % + (StateString[node.get_state()], repr(str(node)))) + + if state == NODE_NO_STATE: + # Mark this node as being on the execution stack: + node.set_state(NODE_PENDING) + elif state > NODE_PENDING: + # Skip this node if it has already been evaluated: if S: S.already_handled = S.already_handled + 1 - if T: T.write(' already handled (%s)\n' % StateString[state]) + if T: T.write('Taskmaster: already handled (executed)\n') continue - # Mark this node as being on the execution stack: - node.set_state(SCons.Node.pending) - try: - children = node.children() + node.prerequisites + children = node.children() except SystemExit: exc_value = sys.exc_info()[1] e = SCons.Errors.ExplicitExit(node, exc_value.code) self.ready_exc = (SCons.Errors.ExplicitExit, e) - if T: T.write(' SystemExit\n') + if T: T.write('Taskmaster: SystemExit\n') return node - except KeyboardInterrupt: - if T: T.write(' KeyboardInterrupt\n') - raise except: # We had a problem just trying to figure out the # children (like a child couldn't be linked in to a @@ -552,70 +632,36 @@ class Taskmaster: # raise the exception when the Task is "executed." self.ready_exc = sys.exc_info() if S: S.problem = S.problem + 1 - if T: T.write(' exception\n') + if T: T.write('Taskmaster: exception while scanning children.\n') return node - if T and children: - c = map(str, children) - c.sort() - T.write(' children:\n %s\n ' % c) - - childstate = map(lambda N: (N, N.get_state()), children) - - # Detect dependency cycles: - pending_nodes = filter(lambda I: I[1] == SCons.Node.pending, childstate) - if pending_nodes: - for p in pending_nodes: - cycle = find_cycle([p[0], node]) - if cycle: - desc = "Dependency cycle: " + string.join(map(str, cycle), " -> ") - if T: T.write(' dependency cycle\n') - raise SCons.Errors.UserError, desc - - not_built = filter(lambda I: I[1] <= SCons.Node.executing, childstate) - if not_built: - # We're waiting on one or more derived targets that have - # not yet finished building. - - not_visited = filter(lambda I: not I[1], not_built) - if not_visited: - # Some of them haven't even been visited yet. - # Add them to the list so that on some next pass - # we can take a stab at evaluating them (or - # their children). - not_visited = map(lambda I: I[0], not_visited) - not_visited.reverse() - self.candidates.extend(self.order(not_visited)) - - n_b_nodes = map(lambda I: I[0], not_built) - - # Add this node to the waiting parents lists of anything - # we're waiting on, with a reference count so we can be - # put back on the list for re-evaluation when they've - # all finished. - map(lambda n, P=node: n.add_to_waiting_parents(P), n_b_nodes) - node.ref_count = len(set(n_b_nodes)) - - if S: S.not_built = S.not_built + 1 - if T: - c = map(str, n_b_nodes) - c.sort() - T.write(' waiting on unfinished children:\n %s\n' % c) - continue + children_not_visited = [] + children_pending = set() + children_not_ready = [] + children_failed = False - # Skip this node if it has side-effects that are - # currently being built: - side_effects = filter(lambda N: - N.get_state() == SCons.Node.executing, - node.side_effects) - if side_effects: - map(lambda n, P=node: n.add_to_waiting_s_e(P), side_effects) - if S: S.side_effects = S.side_effects + 1 - if T: - c = map(str, side_effects) - c.sort() - T.write(' waiting on side effects:\n %s\n' % c) - continue + for child in chain(children,node.prerequisites): + childstate = child.get_state() + + if T: T.write('Taskmaster: <%-10s %s>\n' % + (StateString[childstate], repr(str(child)))) + + if childstate == NODE_NO_STATE: + children_not_visited.append(child) + elif childstate == NODE_PENDING: + children_pending.add(child) + elif childstate == NODE_FAILED: + children_failed = True + + if childstate <= NODE_EXECUTING: + children_not_ready.append(child) + + + # These nodes have not even been visited yet. Add + # them to the list so that on some next pass we can + # take a stab at evaluating them (or their children). + children_not_visited.reverse() + self.candidates.extend(self.order(children_not_visited)) # Skip this node if any of its children have failed. # @@ -635,21 +681,47 @@ class Taskmaster: # Note that even if one of the children fails, we still # added the other children to the list of candidate nodes # to keep on building (--keep-going). - failed_children = filter(lambda I: I[1] == SCons.Node.failed, - childstate) - if failed_children: - node.set_state(SCons.Node.failed) + if children_failed: + node.set_state(NODE_FAILED) + if S: S.child_failed = S.child_failed + 1 - if T: - c = map(lambda I: str(I[0]), failed_children) - c.sort() - T.write(' children failed:\n %s\n' % c) + if T: T.write('Taskmaster:****** <%-10s %s>\n' % + (StateString[node.get_state()], repr(str(node)))) + continue + + if children_not_ready: + for child in children_not_ready: + # We're waiting on one or more derived targets + # that have not yet finished building. + if S: S.not_built = S.not_built + 1 + + # Add this node to the waiting parents lists of + # anything we're waiting on, with a reference + # count so we can be put back on the list for + # re-evaluation when they've all finished. + node.ref_count = node.ref_count + child.add_to_waiting_parents(node) + + self.pending_children = self.pending_children | children_pending + + continue + + # Skip this node if it has side-effects that are + # currently being built: + wait_side_effects = False + for se in node.side_effects: + if se.get_state() == NODE_EXECUTING: + se.add_to_waiting_s_e(node) + wait_side_effects = True + + if wait_side_effects: + if S: S.side_effects = S.side_effects + 1 continue # The default when we've gotten through all of the checks above: # this node is ready to be built. if S: S.build = S.build + 1 - if T: T.write(' evaluating %s\n' % node) + if T: T.write('Taskmaster: Evaluating <%-10s %s>\n' % + (StateString[node.get_state()], repr(str(node)))) return node return None @@ -671,8 +743,6 @@ class Taskmaster: task = self.tasker(self, tlist, node in self.original_top, node) try: task.make_ready() - except KeyboardInterrupt: - raise except: # We had a problem just trying to get this task ready (like # a child couldn't be linked in to a VariantDir when deciding @@ -692,3 +762,18 @@ class Taskmaster: Stops the current build completely. """ self.next_candidate = self.no_next_candidate + + def cleanup(self): + """ + Check for dependency cycles. + """ + if self.pending_children: + desc = 'Found dependency cycle(s):\n' + for node in self.pending_children: + cycle = find_cycle([node], set()) + if cycle: + desc = desc + " " + string.join(map(str, cycle), " -> ") + "\n" + else: + desc = desc + " Internal Error: no cycle found for node %s (%s)\n" % \ + (node, repr(node)) + raise SCons.Errors.UserError, desc diff --git a/src/engine/SCons/TaskmasterTests.py b/src/engine/SCons/TaskmasterTests.py index 9a7969b..88d3078 100644 --- a/src/engine/SCons/TaskmasterTests.py +++ b/src/engine/SCons/TaskmasterTests.py @@ -56,8 +56,8 @@ class Node: self.state = SCons.Node.no_state self.prepared = None self.ref_count = 0 - self.waiting_parents = {} - self.waiting_s_e = {} + self.waiting_parents = set() + self.waiting_s_e = set() self.side_effect = 0 self.side_effects = [] self.alttargets = [] @@ -119,17 +119,10 @@ class Node: def add_to_waiting_parents(self, node): wp = self.waiting_parents - if wp.has_key(node): - result = 0 - else: - result = 1 - wp[node] = 1 - return result - - def call_for_all_waiting_parents(self, func): - func(self) - for parent in self.waiting_parents.keys(): - parent.call_for_all_waiting_parents(func) + if node in wp: + return 0 + wp.add(node) + return 1 def get_state(self): return self.state @@ -166,13 +159,16 @@ class Node: def postprocess(self): self.postprocessed = 1 + self.waiting_parents = set() def get_executor(self): - class Executor: - pass - e = Executor() - e.targets = self.targets - return e + if not hasattr(self, 'executor'): + class Executor: + def prepare(self): + pass + self.executor = Executor() + self.executor.targets = self.targets + return self.executor class OtherError(Exception): pass @@ -469,7 +465,7 @@ class TaskmasterTestCase(unittest.TestCase): t.postprocess() s = n1.get_state() - assert s == SCons.Node.up_to_date, s + assert s == SCons.Node.executed, s s = n2.get_state() assert s == SCons.Node.executed, s @@ -815,6 +811,22 @@ class TaskmasterTestCase(unittest.TestCase): assert n9.prepared assert n10.prepared + # Make sure we call an Executor's prepare() method. + class ExceptionExecutor: + def prepare(self): + raise Exception, "Executor.prepare() exception" + + n11 = Node("n11") + n11.executor = ExceptionExecutor() + tm = SCons.Taskmaster.Taskmaster([n11]) + t = tm.next_task() + try: + t.prepare() + except Exception, e: + assert str(e) == "Executor.prepare() exception", e + else: + raise AssertionError, "did not catch expected exception" + def test_execute(self): """Test executing a task """ @@ -937,37 +949,46 @@ class TaskmasterTestCase(unittest.TestCase): ] assert str(exc_value) in exception_values, exc_value - t.exception_set(("exception 1", None)) + class Exception1(Exception): + pass + + t.exception_set((Exception1, None)) try: t.exception_raise() except: exc_type, exc_value = sys.exc_info()[:2] - assert exc_type == "exception 1", exc_type - assert exc_value is None, exc_value + assert exc_type == Exception1, exc_type + assert str(exc_value) == '', exc_value else: assert 0, "did not catch expected exception" - t.exception_set(("exception 2", "xyzzy")) + class Exception2(Exception): + pass + + t.exception_set((Exception2, "xyzzy")) try: t.exception_raise() except: exc_type, exc_value = sys.exc_info()[:2] - assert exc_type == "exception 2", exc_type - assert exc_value == "xyzzy", exc_value + assert exc_type == Exception2, exc_type + assert str(exc_value) == "xyzzy", exc_value else: assert 0, "did not catch expected exception" + class Exception3(Exception): + pass + try: 1/0 except: tb = sys.exc_info()[2] - t.exception_set(("exception 3", "arg", tb)) + t.exception_set((Exception3, "arg", tb)) try: t.exception_raise() except: exc_type, exc_value, exc_tb = sys.exc_info() - assert exc_type == 'exception 3', exc_type - assert exc_value == "arg", exc_value + assert exc_type == Exception3, exc_type + assert str(exc_value) == "arg", exc_value import traceback x = traceback.extract_tb(tb)[-1] y = traceback.extract_tb(exc_tb)[-1] @@ -1030,16 +1051,29 @@ class TaskmasterTestCase(unittest.TestCase): value = trace.getvalue() expect = """\ -Taskmaster: 'n1': evaluating n1 -Taskmaster: 'n1': already handled (executed) -Taskmaster: 'n3': children: - ['n1', 'n2'] - waiting on unfinished children: - ['n2'] -Taskmaster: 'n2': evaluating n2 -Taskmaster: 'n3': children: - ['n1', 'n2'] - evaluating n3 + +Taskmaster: Looking for a node to evaluate +Taskmaster: Considering node <no_state 'n1'> and its children: +Taskmaster: Evaluating <pending 'n1'> + +Taskmaster: Looking for a node to evaluate +Taskmaster: Considering node <executed 'n1'> and its children: +Taskmaster: already handled (executed) +Taskmaster: Considering node <no_state 'n3'> and its children: +Taskmaster: <executed 'n1'> +Taskmaster: <no_state 'n2'> +Taskmaster: Considering node <no_state 'n2'> and its children: +Taskmaster: Evaluating <pending 'n2'> + +Taskmaster: Looking for a node to evaluate +Taskmaster: Considering node <pending 'n3'> and its children: +Taskmaster: <executed 'n1'> +Taskmaster: <executed 'n2'> +Taskmaster: Evaluating <pending 'n3'> + +Taskmaster: Looking for a node to evaluate +Taskmaster: No candidate anymore. + """ assert value == expect, value diff --git a/src/engine/SCons/Tool/ToolTests.py b/src/engine/SCons/Tool/ToolTests.py index b9230f1..779aaaa 100644 --- a/src/engine/SCons/Tool/ToolTests.py +++ b/src/engine/SCons/Tool/ToolTests.py @@ -54,7 +54,6 @@ class ToolTestCase(unittest.TestCase): t = SCons.Tool.Tool('g++') t(env) assert (env['CXX'] == 'c++' or env['CXX'] == 'g++'), env['CXX'] - assert env['CXXFLAGS'] == ['$CCFLAGS'], env['CXXFLAGS'] assert env['INCPREFIX'] == '-I', env['INCPREFIX'] assert env['TOOLS'] == ['g++'], env['TOOLS'] diff --git a/src/engine/SCons/Tool/__init__.py b/src/engine/SCons/Tool/__init__.py index 1e69f1e..e0ada5e 100644 --- a/src/engine/SCons/Tool/__init__.py +++ b/src/engine/SCons/Tool/__init__.py @@ -163,10 +163,10 @@ class Tool: kw = self.init_kw env.Append(TOOLS = [ self.name ]) if hasattr(self, 'options'): - from SCons.Options import Options + import SCons.Variables if not env.has_key('options'): from SCons.Script import ARGUMENTS - env['options']=Options(args=ARGUMENTS) + env['options']=SCons.Variables.Variables(args=ARGUMENTS) opts=env['options'] self.options(opts) diff --git a/src/engine/SCons/Tool/c++.py b/src/engine/SCons/Tool/c++.py index d9370b0..a4375c0 100644 --- a/src/engine/SCons/Tool/c++.py +++ b/src/engine/SCons/Tool/c++.py @@ -71,13 +71,13 @@ def generate(env): shared_obj.add_emitter(suffix, SCons.Defaults.SharedObjectEmitter) SCons.Tool.cc.add_common_cc_variables(env) - + env['CXX'] = 'c++' - env['CXXFLAGS'] = SCons.Util.CLVar('$CCFLAGS') - env['CXXCOM'] = '$CXX -o $TARGET -c $CXXFLAGS $_CCCOMCOM $SOURCES' + env['CXXFLAGS'] = SCons.Util.CLVar('') + env['CXXCOM'] = '$CXX -o $TARGET -c $CXXFLAGS $CCFLAGS $_CCCOMCOM $SOURCES' env['SHCXX'] = '$CXX' - env['SHCXXFLAGS'] = SCons.Util.CLVar('$CXXFLAGS') - env['SHCXXCOM'] = '$SHCXX -o $TARGET -c $SHCXXFLAGS $_CCCOMCOM $SOURCES' + env['SHCXXFLAGS'] = SCons.Util.CLVar('') + env['SHCXXCOM'] = '$SHCXX -o $TARGET -c $SHCXXFLAGS $SHCCFLAGS $_CCCOMCOM $SOURCES' env['CPPDEFPREFIX'] = '-D' env['CPPDEFSUFFIX'] = '' diff --git a/src/engine/SCons/Tool/dmd.py b/src/engine/SCons/Tool/dmd.py index 64ffb68..a173162 100644 --- a/src/engine/SCons/Tool/dmd.py +++ b/src/engine/SCons/Tool/dmd.py @@ -14,14 +14,15 @@ use absolute paths. To hack around it, add '#/blah'. This will link blah.lib from the directory where SConstruct resides. Compiler variables: - DC - The name of the D compiler to use. Defaults to dmd. + DC - The name of the D compiler to use. Defaults to dmd or gdmd, + whichever is found. DPATH - List of paths to search for import modules. DVERSIONS - List of version tags to enable when compiling. DDEBUG - List of debug tags to enable when compiling. Linker related variables: LIBS - List of library files to link in. - DLINK - Name of the linker to use. Defaults to dmd. + DLINK - Name of the linker to use. Defaults to dmd or gdmd. DLINKFLAGS - List of linker flags. Lib tool variables: @@ -93,7 +94,8 @@ def generate(env): static_obj.add_emitter('.d', SCons.Defaults.StaticObjectEmitter) shared_obj.add_emitter('.d', SCons.Defaults.SharedObjectEmitter) - env['DC'] = 'dmd' + dc = env.Detect(['dmd', 'gdmd']) + env['DC'] = dc env['DCOM'] = '$DC $_DINCFLAGS $_DVERFLAGS $_DDEBUGFLAGS $_DFLAGS -c -of$TARGET $SOURCES' env['_DINCFLAGS'] = '$( ${_concat(DINCPREFIX, DPATH, DINCSUFFIX, __env__, RDirs, TARGET, SOURCE)} $)' env['_DVERFLAGS'] = '$( ${_concat(DVERPREFIX, DVERSIONS, DVERSUFFIX, __env__)} $)' @@ -105,14 +107,15 @@ def generate(env): env['DVERSIONS'] = [] env['DDEBUG'] = [] - # Add the path to the standard library. - # This is merely for the convenience of the dependency scanner. - dmd_path = env.WhereIs('dmd') - if dmd_path: - x = string.rindex(dmd_path, 'dmd') - phobosDir = dmd_path[:x] + '/../src/phobos' - if os.path.isdir(phobosDir): - env.Append(DPATH = [phobosDir]) + if dc: + # Add the path to the standard library. + # This is merely for the convenience of the dependency scanner. + dmd_path = env.WhereIs(dc) + if dmd_path: + x = string.rindex(dmd_path, dc) + phobosDir = dmd_path[:x] + '/../src/phobos' + if os.path.isdir(phobosDir): + env.Append(DPATH = [phobosDir]) env['DINCPREFIX'] = '-I' env['DINCSUFFIX'] = '' @@ -198,7 +201,10 @@ def generate(env): except KeyError: libs = [] if 'phobos' not in libs: - env.Append(LIBS = ['phobos']) + if dc is 'dmd': + env.Append(LIBS = ['phobos']) + elif dc is 'gdmd': + env.Append(LIBS = ['gphobos']) if 'pthread' not in libs: env.Append(LIBS = ['pthread']) if 'm' not in libs: @@ -209,4 +215,4 @@ def generate(env): env['LINKCOM'] = '$SMART_LINKCOM ' def exists(env): - return env.Detect('dmd') + return env.Detect(['dmd', 'gdmd']) diff --git a/src/engine/SCons/Tool/dmd.xml b/src/engine/SCons/Tool/dmd.xml index cab3ab4..835a4eb 100644 --- a/src/engine/SCons/Tool/dmd.xml +++ b/src/engine/SCons/Tool/dmd.xml @@ -6,7 +6,8 @@ See its __doc__ string for a discussion of the format. --> <tool name="dmd"> <summary> -Sets construction variables for the Digital Mars D compiler. +Sets construction variables for D language compilers +(the Digital Mars D compiler, or GDC). </summary> <sets> <!-- diff --git a/src/engine/SCons/Tool/g++.py b/src/engine/SCons/Tool/g++.py index 0576b90..9c7e477 100644 --- a/src/engine/SCons/Tool/g++.py +++ b/src/engine/SCons/Tool/g++.py @@ -63,18 +63,16 @@ def generate(env): env['SHOBJSUFFIX'] = '$OBJSUFFIX' elif env['PLATFORM'] == 'hpux': # Original line from Christian Engel added -DPIC: - #env['SHCXXFLAGS'] = SCons.Util.CLVar('$CXXFLAGS -fPIC -DPIC') - env['SHCXXFLAGS'] = SCons.Util.CLVar('$CXXFLAGS -fPIC') + #env['SHCXXFLAGS'] = SCons.Util.CLVar('$CXXFLAGS -DPIC') env['SHOBJSUFFIX'] = '.pic.o' elif env['PLATFORM'] == 'sunos': # Original line from Christian Engel added -DPIC: - #env['SHCXXFLAGS'] = SCons.Util.CLVar('$CXXFLAGS -fPIC -DPIC') - env['SHCXXFLAGS'] = SCons.Util.CLVar('$CXXFLAGS -fPIC') + #env['SHCXXFLAGS'] = SCons.Util.CLVar('$CXXFLAGS -DPIC') env['SHOBJSUFFIX'] = '.pic.o' else: # Original line from Christian Engel added -DPIC: - #env['SHCXXFLAGS'] = SCons.Util.CLVar('$CXXFLAGS -fPIC -DPIC') - env['SHCXXFLAGS'] = SCons.Util.CLVar('$CXXFLAGS -fPIC') + #env['SHCXXFLAGS'] = SCons.Util.CLVar('$CXXFLAGS -DPIC') + pass # determine compiler version if env['CXX']: line = os.popen(env['CXX'] + ' --version').readline() diff --git a/src/engine/SCons/Tool/intelc.py b/src/engine/SCons/Tool/intelc.py index 02cc52a..db5d06e 100644 --- a/src/engine/SCons/Tool/intelc.py +++ b/src/engine/SCons/Tool/intelc.py @@ -375,14 +375,14 @@ def generate(env, version=None, abi=None, topdir=None, verbose=0): 'LIB' : 'lib', 'PATH' : 'bin', 'LD_LIBRARY_PATH' : 'lib'} - for p in paths: + for p in paths.keys(): env.PrependENVPath(p, os.path.join(topdir, paths[p])) if is_mac: paths={'INCLUDE' : 'include', 'LIB' : 'lib', 'PATH' : 'bin', 'LD_LIBRARY_PATH' : 'lib'} - for p in paths: + for p in paths.keys(): env.PrependENVPath(p, os.path.join(topdir, paths[p])) if is_windows: # env key reg valname default subdir of top @@ -394,7 +394,7 @@ def generate(env, version=None, abi=None, topdir=None, verbose=0): if version is None: version = '' # Each path has a registry entry, use that or default to subdir - for p in paths: + for p in paths.keys(): try: path=get_intel_registry_value(p[1], version, abi) # These paths may have $(ICInstallDir) diff --git a/src/engine/SCons/Tool/javac.py b/src/engine/SCons/Tool/javac.py index a020727..cb79301 100644 --- a/src/engine/SCons/Tool/javac.py +++ b/src/engine/SCons/Tool/javac.py @@ -92,8 +92,9 @@ def emit_java_classes(target, source, env): raise SCons.Errors.UserError("Java source must be File or Dir, not '%s'" % entry.__class__) version = env.get('JAVAVERSION', '1.4') - tlist = [] + full_tlist = [] for f in slist: + tlist = [] source_file_based = True pkg_dir = None if not f.is_derived(): @@ -124,7 +125,12 @@ def emit_java_classes(target, source, env): t.attributes.java_classname = classname(base) tlist.append(t) - return tlist, slist + for t in tlist: + t.set_specific_source([f]) + + full_tlist.extend(tlist) + + return full_tlist, slist JavaAction = SCons.Action.Action('$JAVACCOM', '$JAVACCOMSTR') diff --git a/src/engine/SCons/Tool/mingw.py b/src/engine/SCons/Tool/mingw.py index 359c40f..8c5e68a 100644 --- a/src/engine/SCons/Tool/mingw.py +++ b/src/engine/SCons/Tool/mingw.py @@ -122,6 +122,7 @@ def generate(env): env['SHCXXFLAGS'] = SCons.Util.CLVar('$CXXFLAGS') env['SHLINKFLAGS'] = SCons.Util.CLVar('$LINKFLAGS -shared') env['SHLINKCOM'] = shlib_action + env['LDMODULECOM'] = shlib_action env.Append(SHLIBEMITTER = [shlib_emitter]) env['AS'] = 'as' diff --git a/src/engine/SCons/Tool/mingw.xml b/src/engine/SCons/Tool/mingw.xml index 048124c..f3354c1 100644 --- a/src/engine/SCons/Tool/mingw.xml +++ b/src/engine/SCons/Tool/mingw.xml @@ -15,6 +15,7 @@ CXX SHCXXFLAGS SHLINKFLAGS SHLINKCOM +LDMODULECOM AS WINDOWSDEFPREFIX WINDOWSDEFSUFFIX diff --git a/src/engine/SCons/Tool/packaging/__init__.py b/src/engine/SCons/Tool/packaging/__init__.py index 79cd4ab..02be24f 100644 --- a/src/engine/SCons/Tool/packaging/__init__.py +++ b/src/engine/SCons/Tool/packaging/__init__.py @@ -29,7 +29,7 @@ SCons Packaging Tool. __revision__ = "__FILE__ __REVISION__ __DATE__ __DEVELOPER__" import SCons.Environment -from SCons.Options import * +from SCons.Variables import * from SCons.Errors import * from SCons.Util import is_List, make_path_relative from SCons.Warnings import warn, Warning @@ -217,11 +217,11 @@ def exists(env): # XXX def options(opts): - opts.AddOptions( - EnumOption( 'PACKAGETYPE', - 'the type of package to create.', - None, allowed_values=map( str, __all__ ), - ignorecase=2 + opts.AddVariables( + EnumVariable( 'PACKAGETYPE', + 'the type of package to create.', + None, allowed_values=map( str, __all__ ), + ignorecase=2 ) ) diff --git a/src/engine/SCons/Tool/swig.py b/src/engine/SCons/Tool/swig.py index eba49a7..16f938e 100644 --- a/src/engine/SCons/Tool/swig.py +++ b/src/engine/SCons/Tool/swig.py @@ -62,7 +62,8 @@ def _swigEmitter(target, source, env): if "-python" in flags and "-noproxy" not in flags: if mnames is None: mnames = _reModule.findall(open(src).read()) - target.extend(map(lambda m: m + ".py", mnames)) + target.extend(map(lambda m, d=target[0].dir: + d.File(m + ".py"), mnames)) if "-java" in flags: if mnames is None: mnames = _reModule.findall(open(src).read()) diff --git a/src/engine/SCons/Util.py b/src/engine/SCons/Util.py index 311c6a8..af09786 100644 --- a/src/engine/SCons/Util.py +++ b/src/engine/SCons/Util.py @@ -109,41 +109,6 @@ def updrive(path): path = string.upper(drive) + rest return path -# -# Generic convert-to-string functions that abstract away whether or -# not the Python we're executing has Unicode support. The wrapper -# to_String_for_signature() will use a for_signature() method if the -# specified object has one. -# -if hasattr(types, 'UnicodeType'): - UnicodeType = types.UnicodeType - def to_String(s): - if isinstance(s, UserString): - t = type(s.data) - else: - t = type(s) - if t is UnicodeType: - return unicode(s) - else: - return str(s) -else: - to_String = str - -def to_String_for_signature(obj): - try: - f = obj.for_signature - except AttributeError: - return to_String_for_subst(obj) - else: - return f() - -def to_String_for_subst(s): - if is_Sequence( s ): - return string.join( map(to_String_for_subst, s) ) - - return to_String( s ) - - class CallableComposite(UserList): """A simple composite callable class that, when called, will invoke all of its contained callables with the same arguments.""" @@ -213,7 +178,17 @@ class DisplayEngine: def print_it(self, text, append_newline=1): if append_newline: text = text + '\n' - sys.stdout.write(text) + try: + sys.stdout.write(text) + except IOError: + # Stdout might be connected to a pipe that has been closed + # by now. The most likely reason for the pipe being closed + # is that the user has press ctrl-c. It this is the case, + # then SCons is currently shutdown. We therefore ignore + # IOError's here so that SCons can continue and shutdown + # properly so that the .sconsign is correctly written + # before SCons exits. + pass def dont_print(self, text, append_newline=1): pass @@ -437,6 +412,41 @@ except TypeError: else: flatten_sequence(item, result) return result + + # + # Generic convert-to-string functions that abstract away whether or + # not the Python we're executing has Unicode support. The wrapper + # to_String_for_signature() will use a for_signature() method if the + # specified object has one. + # + if hasattr(types, 'UnicodeType'): + UnicodeType = types.UnicodeType + def to_String(s): + if isinstance(s, UserString): + t = type(s.data) + else: + t = type(s) + if t is UnicodeType: + return unicode(s) + else: + return str(s) + else: + to_String = str + + def to_String_for_signature(obj): + try: + f = obj.for_signature + except AttributeError: + return to_String_for_subst(obj) + else: + return f() + + def to_String_for_subst(s): + if is_Sequence( s ): + return string.join( map(to_String_for_subst, s) ) + + return to_String( s ) + else: # A modern Python version with new-style classes, so we can just use # isinstance(). @@ -459,6 +469,10 @@ else: # with basestring. (at least on Python 2.5.1) StringTypes = (str, unicode, UserString) + # Empirically, it is faster to check explicitely for str and + # unicode than for basestring. + BaseStringTypes = (str, unicode) + def is_Dict(obj, isinstance=isinstance, DictTypes=DictTypes): return isinstance(obj, DictTypes) @@ -526,6 +540,56 @@ else: return result + # + # Generic convert-to-string functions that abstract away whether or + # not the Python we're executing has Unicode support. The wrapper + # to_String_for_signature() will use a for_signature() method if the + # specified object has one. + # + def to_String(s, + isinstance=isinstance, str=str, + UserString=UserString, BaseStringTypes=BaseStringTypes): + if isinstance(s,BaseStringTypes): + # Early out when already a string! + return s + elif isinstance(s, UserString): + # s.data can only be either a unicode or a regular + # string. Please see the UserString initializer. + return s.data + else: + return str(s) + + def to_String_for_subst(s, + isinstance=isinstance, join=string.join, str=str, to_String=to_String, + BaseStringTypes=BaseStringTypes, SequenceTypes=SequenceTypes, + UserString=UserString): + + # Note that the test cases are sorted by order of probability. + if isinstance(s, BaseStringTypes): + return s + elif isinstance(s, SequenceTypes): + l = [] + for e in s: + l.append(to_String_for_subst(e)) + return join( s ) + elif isinstance(s, UserString): + # s.data can only be either a unicode or a regular + # string. Please see the UserString initializer. + return s.data + else: + return str(s) + + def to_String_for_signature(obj, to_String_for_subst=to_String_for_subst, + AttributeError=AttributeError): + try: + f = obj.for_signature + except AttributeError: + return to_String_for_subst(obj) + else: + return f() + + + # The SCons "semi-deep" copy. # # This makes separate copies of lists (including UserList objects) @@ -1267,8 +1331,18 @@ class Unbuffered: def __init__(self, file): self.file = file def write(self, arg): - self.file.write(arg) - self.file.flush() + try: + self.file.write(arg) + self.file.flush() + except IOError: + # Stdout might be connected to a pipe that has been closed + # by now. The most likely reason for the pipe being closed + # is that the user has press ctrl-c. It this is the case, + # then SCons is currently shutdown. We therefore ignore + # IOError's here so that SCons can continue and shutdown + # properly so that the .sconsign is correctly written + # before SCons exits. + pass def __getattr__(self, attr): return getattr(self.file, attr) diff --git a/src/engine/SCons/Options/BoolOption.py b/src/engine/SCons/Variables/BoolVariable.py index 80b607d..92936e6 100644 --- a/src/engine/SCons/Options/BoolOption.py +++ b/src/engine/SCons/Variables/BoolVariable.py @@ -1,11 +1,11 @@ -"""engine.SCons.Options.BoolOption +"""engine.SCons.Variables.BoolVariable This file defines the option type for SCons implementing true/false values. Usage example: - opts = Options() - opts.Add(BoolOption('embedded', 'build for an embedded system', 0)) + opts = Variables() + opts.Add(BoolVariable('embedded', 'build for an embedded system', 0)) ... if env['embedded'] == 1: ... @@ -36,7 +36,7 @@ Usage example: __revision__ = "__FILE__ __REVISION__ __DATE__ __DEVELOPER__" -__all__ = ('BoolOption') +__all__ = ['BoolVariable',] import string @@ -56,7 +56,7 @@ def _text2bool(val): See '__true_strings' and '__false_strings' for values considered 'true' or 'false respectivly. - This is usable as 'converter' for SCons' Options. + This is usable as 'converter' for SCons' Variables. """ lval = string.lower(val) if lval in __true_strings: return True @@ -68,14 +68,14 @@ def _validator(key, val, env): """ Validates the given value to be either '0' or '1'. - This is usable as 'validator' for SCons' Options. + This is usable as 'validator' for SCons' Variables. """ if not env[key] in (True, False): raise SCons.Errors.UserError( 'Invalid value for boolean option %s: %s' % (key, env[key])) -def BoolOption(key, help, default): +def BoolVariable(key, help, default): """ The input parameters describe a boolen option, thus they are returned with the correct converter and validator appended. The diff --git a/src/engine/SCons/Options/BoolOptionTests.py b/src/engine/SCons/Variables/BoolVariableTests.py index 07b5b79..feb4e4e 100644 --- a/src/engine/SCons/Options/BoolOptionTests.py +++ b/src/engine/SCons/Variables/BoolVariableTests.py @@ -29,13 +29,13 @@ import sys import unittest import SCons.Errors -import SCons.Options +import SCons.Variables -class BoolOptionTestCase(unittest.TestCase): - def test_BoolOption(self): - """Test BoolOption creation""" - opts = SCons.Options.Options() - opts.Add(SCons.Options.BoolOption('test', 'test option help', 0)) +class BoolVariableTestCase(unittest.TestCase): + def test_BoolVariable(self): + """Test BoolVariable creation""" + opts = SCons.Variables.Variables() + opts.Add(SCons.Variables.BoolVariable('test', 'test option help', 0)) o = opts.options[0] assert o.key == 'test', o.key @@ -45,9 +45,9 @@ class BoolOptionTestCase(unittest.TestCase): assert not o.converter is None, o.converter def test_converter(self): - """Test the BoolOption converter""" - opts = SCons.Options.Options() - opts.Add(SCons.Options.BoolOption('test', 'test option help', 0)) + """Test the BoolVariable converter""" + opts = SCons.Variables.Variables() + opts.Add(SCons.Variables.BoolVariable('test', 'test option help', 0)) o = opts.options[0] @@ -86,9 +86,9 @@ class BoolOptionTestCase(unittest.TestCase): assert caught, "did not catch expected ValueError" def test_validator(self): - """Test the BoolOption validator""" - opts = SCons.Options.Options() - opts.Add(SCons.Options.BoolOption('test', 'test option help', 0)) + """Test the BoolVariable validator""" + opts = SCons.Variables.Variables() + opts.Add(SCons.Variables.BoolVariable('test', 'test option help', 0)) o = opts.options[0] @@ -118,6 +118,6 @@ class BoolOptionTestCase(unittest.TestCase): if __name__ == "__main__": - suite = unittest.makeSuite(BoolOptionTestCase, 'test_') + suite = unittest.makeSuite(BoolVariableTestCase, 'test_') if not unittest.TextTestRunner().run(suite).wasSuccessful(): sys.exit(1) diff --git a/src/engine/SCons/Options/EnumOption.py b/src/engine/SCons/Variables/EnumVariable.py index d4e2ac1..e1b20d2 100644 --- a/src/engine/SCons/Options/EnumOption.py +++ b/src/engine/SCons/Variables/EnumVariable.py @@ -1,12 +1,12 @@ -"""engine.SCons.Options.EnumOption +"""engine.SCons.Variables.EnumVariable This file defines the option type for SCons allowing only specified input-values. Usage example: - opts = Options() - opts.Add(EnumOption('debug', 'debug output and symbols', 'no', + opts = Variables() + opts.Add(EnumVariable('debug', 'debug output and symbols', 'no', allowed_values=('yes', 'no', 'full'), map={}, ignorecase=2)) ... @@ -39,7 +39,7 @@ Usage example: __revision__ = "__FILE__ __REVISION__ __DATE__ __DEVELOPER__" -__all__ = ('EnumOption',) +__all__ = ['EnumVariable',] import string @@ -51,14 +51,14 @@ def _validator(key, val, env, vals): 'Invalid value for option %s: %s' % (key, val)) -def EnumOption(key, help, default, allowed_values, map={}, ignorecase=0): +def EnumVariable(key, help, default, allowed_values, map={}, ignorecase=0): """ The input parameters describe a option with only certain values allowed. They are returned with an appropriate converter and validator appended. The result is usable for input to - Options.Add(). + Variables.Add(). - 'key' and 'default' are the values to be passed on to Options.Add(). + 'key' and 'default' are the values to be passed on to Variables.Add(). 'help' will be appended by the allowed values automatically diff --git a/src/engine/SCons/Options/EnumOptionTests.py b/src/engine/SCons/Variables/EnumVariableTests.py index 20ae6c3..0f9afde 100644 --- a/src/engine/SCons/Options/EnumOptionTests.py +++ b/src/engine/SCons/Variables/EnumVariableTests.py @@ -27,13 +27,13 @@ import sys import unittest import SCons.Errors -import SCons.Options +import SCons.Variables -class EnumOptionTestCase(unittest.TestCase): - def test_EnumOption(self): - """Test EnumOption creation""" - opts = SCons.Options.Options() - opts.Add(SCons.Options.EnumOption('test', 'test option help', 0, +class EnumVariableTestCase(unittest.TestCase): + def test_EnumVariable(self): + """Test EnumVariable creation""" + opts = SCons.Variables.Variables() + opts.Add(SCons.Variables.EnumVariable('test', 'test option help', 0, ['one', 'two', 'three'], {})) @@ -45,9 +45,9 @@ class EnumOptionTestCase(unittest.TestCase): assert not o.converter is None, o.converter def test_converter(self): - """Test the EnumOption converter""" - opts = SCons.Options.Options() - opts.Add(SCons.Options.EnumOption('test', 'test option help', 0, + """Test the EnumVariable converter""" + opts = SCons.Variables.Variables() + opts.Add(SCons.Variables.EnumVariable('test', 'test option help', 0, ['one', 'two', 'three'])) o = opts.options[0] @@ -56,8 +56,8 @@ class EnumOptionTestCase(unittest.TestCase): x = o.converter(a) assert x == a, x - opts = SCons.Options.Options() - opts.Add(SCons.Options.EnumOption('test', 'test option help', 0, + opts = SCons.Variables.Variables() + opts.Add(SCons.Variables.EnumVariable('test', 'test option help', 0, ['one', 'two', 'three'], {'1' : 'one', '2' : 'two', @@ -80,20 +80,20 @@ class EnumOptionTestCase(unittest.TestCase): x = o.converter('3') assert x == 'three', x - opts = SCons.Options.Options() - opts.Add(SCons.Options.EnumOption('test0', 'test option help', 0, + opts = SCons.Variables.Variables() + opts.Add(SCons.Variables.EnumVariable('test0', 'test option help', 0, ['one', 'two', 'three'], {'a' : 'one', 'b' : 'two', 'c' : 'three'}, ignorecase=0)) - opts.Add(SCons.Options.EnumOption('test1', 'test option help', 0, + opts.Add(SCons.Variables.EnumVariable('test1', 'test option help', 0, ['one', 'two', 'three'], {'a' : 'one', 'b' : 'two', 'c' : 'three'}, ignorecase=1)) - opts.Add(SCons.Options.EnumOption('test2', 'test option help', 0, + opts.Add(SCons.Variables.EnumVariable('test2', 'test option help', 0, ['one', 'two', 'three'], {'a' : 'one', 'b' : 'two', @@ -131,21 +131,21 @@ class EnumOptionTestCase(unittest.TestCase): assert x == l[2], "o2 got %s, expected %s" % (x, l[2]) def test_validator(self): - """Test the EnumOption validator""" - opts = SCons.Options.Options() - opts.Add(SCons.Options.EnumOption('test0', 'test option help', 0, + """Test the EnumVariable validator""" + opts = SCons.Variables.Variables() + opts.Add(SCons.Variables.EnumVariable('test0', 'test option help', 0, ['one', 'two', 'three'], {'a' : 'one', 'b' : 'two', 'c' : 'three'}, ignorecase=0)) - opts.Add(SCons.Options.EnumOption('test1', 'test option help', 0, + opts.Add(SCons.Variables.EnumVariable('test1', 'test option help', 0, ['one', 'two', 'three'], {'a' : 'one', 'b' : 'two', 'c' : 'three'}, ignorecase=1)) - opts.Add(SCons.Options.EnumOption('test2', 'test option help', 0, + opts.Add(SCons.Variables.EnumVariable('test2', 'test option help', 0, ['one', 'two', 'three'], {'a' : 'one', 'b' : 'two', @@ -193,6 +193,6 @@ class EnumOptionTestCase(unittest.TestCase): if __name__ == "__main__": - suite = unittest.makeSuite(EnumOptionTestCase, 'test_') + suite = unittest.makeSuite(EnumVariableTestCase, 'test_') if not unittest.TextTestRunner().run(suite).wasSuccessful(): sys.exit(1) diff --git a/src/engine/SCons/Options/ListOption.py b/src/engine/SCons/Variables/ListVariable.py index 6954905..37e3094 100644 --- a/src/engine/SCons/Options/ListOption.py +++ b/src/engine/SCons/Variables/ListVariable.py @@ -1,4 +1,4 @@ -"""engine.SCons.Options.ListOption +"""engine.SCons.Variables.ListVariable This file defines the option type for SCons implementing 'lists'. @@ -11,8 +11,8 @@ Usage example: list_of_libs = Split('x11 gl qt ical') - opts = Options() - opts.Add(ListOption('shared', + opts = Variables() + opts.Add(ListVariable('shared', 'libraries to build as shared libraries', 'all', elems = list_of_libs)) @@ -52,7 +52,7 @@ __revision__ = "__FILE__ __REVISION__ __DATE__ __DEVELOPER__" # Know Bug: This should behave like a Set-Type, but does not really, # since elements can occur twice. -__all__ = ('ListOption',) +__all__ = ['ListVariable',] import string import UserList @@ -60,7 +60,7 @@ import UserList import SCons.Util -class _ListOption(UserList.UserList): +class _ListVariable(UserList.UserList): def __init__(self, initlist=[], allowedElems=[]): UserList.UserList.__init__(self, filter(None, initlist)) self.allowedElems = allowedElems[:] @@ -103,7 +103,7 @@ def _converter(val, allowedElems, mapdict): if notAllowed: raise ValueError("Invalid value(s) for option: %s" % string.join(notAllowed, ',')) - return _ListOption(val, allowedElems) + return _ListVariable(val, allowedElems) ## def _validator(key, val, env): @@ -113,7 +113,7 @@ def _converter(val, allowedElems, mapdict): ## return 1 -def ListOption(key, help, default, names, map={}): +def ListVariable(key, help, default, names, map={}): """ The input parameters describe a 'package list' option, thus they are returned with the correct converter and validater appended. The diff --git a/src/engine/SCons/Options/ListOptionTests.py b/src/engine/SCons/Variables/ListVariableTests.py index 22378bc..9363f6d 100644 --- a/src/engine/SCons/Options/ListOptionTests.py +++ b/src/engine/SCons/Variables/ListVariableTests.py @@ -28,13 +28,13 @@ import sys import unittest import SCons.Errors -import SCons.Options +import SCons.Variables -class ListOptionTestCase(unittest.TestCase): - def test_ListOption(self): - """Test ListOption creation""" - opts = SCons.Options.Options() - opts.Add(SCons.Options.ListOption('test', 'test option help', 'all', +class ListVariableTestCase(unittest.TestCase): + def test_ListVariable(self): + """Test ListVariable creation""" + opts = SCons.Variables.Variables() + opts.Add(SCons.Variables.ListVariable('test', 'test option help', 'all', ['one', 'two', 'three'])) o = opts.options[0] @@ -44,8 +44,8 @@ class ListOptionTestCase(unittest.TestCase): assert o.validator is None, o.validator assert not o.converter is None, o.converter - opts = SCons.Options.Options() - opts.Add(SCons.Options.ListOption('test2', 'test2 help', + opts = SCons.Variables.Variables() + opts.Add(SCons.Variables.ListVariable('test2', 'test2 help', ['one', 'three'], ['one', 'two', 'three'])) @@ -53,9 +53,9 @@ class ListOptionTestCase(unittest.TestCase): assert o.default == 'one,three' def test_converter(self): - """Test the ListOption converter""" - opts = SCons.Options.Options() - opts.Add(SCons.Options.ListOption('test', 'test option help', 'all', + """Test the ListVariable converter""" + opts = SCons.Variables.Variables() + opts.Add(SCons.Variables.ListVariable('test', 'test option help', 'all', ['one', 'two', 'three'], {'ONE':'one', 'TWO':'two'})) @@ -112,9 +112,9 @@ class ListOptionTestCase(unittest.TestCase): assert caught, "did not catch expected ValueError" def test_copy(self): - """Test copying a ListOption like an Environment would""" - opts = SCons.Options.Options() - opts.Add(SCons.Options.ListOption('test', 'test option help', 'all', + """Test copying a ListVariable like an Environment would""" + opts = SCons.Variables.Variables() + opts.Add(SCons.Variables.ListVariable('test', 'test option help', 'all', ['one', 'two', 'three'])) o = opts.options[0] @@ -123,6 +123,6 @@ class ListOptionTestCase(unittest.TestCase): n = l.__class__(copy.copy(l)) if __name__ == "__main__": - suite = unittest.makeSuite(ListOptionTestCase, 'test_') + suite = unittest.makeSuite(ListVariableTestCase, 'test_') if not unittest.TextTestRunner().run(suite).wasSuccessful(): sys.exit(1) diff --git a/src/engine/SCons/Options/PackageOption.py b/src/engine/SCons/Variables/PackageVariable.py index 3b4f0ce..6be260d 100644 --- a/src/engine/SCons/Options/PackageOption.py +++ b/src/engine/SCons/Variables/PackageVariable.py @@ -1,4 +1,4 @@ -"""engine.SCons.Options.PackageOption +"""engine.SCons.Variables.PackageVariable This file defines the option type for SCons implementing 'package activation'. @@ -15,8 +15,8 @@ Usage example: To replace autoconf's --with-xxx=yyy - opts = Options() - opts.Add(PackageOption('x11', + opts = Variables() + opts.Add(PackageVariable('x11', 'use X11 installed here (yes = search some places', 'yes')) ... @@ -52,7 +52,7 @@ Usage example: __revision__ = "__FILE__ __REVISION__ __DATE__ __DEVELOPER__" -__all__ = ('PackageOption') +__all__ = ['PackageVariable',] import string @@ -86,7 +86,7 @@ def _validator(key, val, env, searchfunc): 'Path does not exist for option %s: %s' % (key, val)) -def PackageOption(key, help, default, searchfunc=None): +def PackageVariable(key, help, default, searchfunc=None): # NB: searchfunc is currenty undocumented and unsupported """ The input parameters describe a 'package list' option, thus they diff --git a/src/engine/SCons/Options/PackageOptionTests.py b/src/engine/SCons/Variables/PackageVariableTests.py index 68f14e5..54116cf 100644 --- a/src/engine/SCons/Options/PackageOptionTests.py +++ b/src/engine/SCons/Variables/PackageVariableTests.py @@ -29,15 +29,15 @@ import sys import unittest import SCons.Errors -import SCons.Options +import SCons.Variables import TestCmd -class PackageOptionTestCase(unittest.TestCase): - def test_PackageOption(self): - """Test PackageOption creation""" - opts = SCons.Options.Options() - opts.Add(SCons.Options.PackageOption('test', 'test option help', '/default/path')) +class PackageVariableTestCase(unittest.TestCase): + def test_PackageVariable(self): + """Test PackageVariable creation""" + opts = SCons.Variables.Variables() + opts.Add(SCons.Variables.PackageVariable('test', 'test option help', '/default/path')) o = opts.options[0] assert o.key == 'test', o.key @@ -47,9 +47,9 @@ class PackageOptionTestCase(unittest.TestCase): assert not o.converter is None, o.converter def test_converter(self): - """Test the PackageOption converter""" - opts = SCons.Options.Options() - opts.Add(SCons.Options.PackageOption('test', 'test option help', '/default/path')) + """Test the PackageVariable converter""" + opts = SCons.Variables.Variables() + opts.Add(SCons.Variables.PackageVariable('test', 'test option help', '/default/path')) o = opts.options[0] @@ -80,7 +80,7 @@ class PackageOptionTestCase(unittest.TestCase): # Make sure the converter returns True if we give it str(True) and # False when we give it str(False). This assures consistent operation - # through a cycle of Options.Save(<file>) -> Options(<file>). + # through a cycle of Variables.Save(<file>) -> Variables(<file>). x = o.converter(str(True)) assert x == True, "converter returned a string when given str(True)" @@ -88,9 +88,9 @@ class PackageOptionTestCase(unittest.TestCase): assert x == False, "converter returned a string when given str(False)" def test_validator(self): - """Test the PackageOption validator""" - opts = SCons.Options.Options() - opts.Add(SCons.Options.PackageOption('test', 'test option help', '/default/path')) + """Test the PackageVariable validator""" + opts = SCons.Variables.Variables() + opts.Add(SCons.Variables.PackageVariable('test', 'test option help', '/default/path')) test = TestCmd.TestCmd(workdir='') test.write('exists', 'exists\n') @@ -115,6 +115,6 @@ class PackageOptionTestCase(unittest.TestCase): if __name__ == "__main__": - suite = unittest.makeSuite(PackageOptionTestCase, 'test_') + suite = unittest.makeSuite(PackageVariableTestCase, 'test_') if not unittest.TextTestRunner().run(suite).wasSuccessful(): sys.exit(1) diff --git a/src/engine/SCons/Options/PathOption.py b/src/engine/SCons/Variables/PathVariable.py index 995eeb9..e31de96 100644 --- a/src/engine/SCons/Options/PathOption.py +++ b/src/engine/SCons/Variables/PathVariable.py @@ -1,10 +1,10 @@ -"""SCons.Options.PathOption +"""SCons.Variables.PathVariable This file defines an option type for SCons implementing path settings. To be used whenever a a user-specified path override should be allowed. -Arguments to PathOption are: +Arguments to PathVariable are: option-name = name of this option on the command line (e.g. "prefix") option-help = help string for option option-dflt = default value for this option @@ -30,16 +30,16 @@ Usage example: Examples: prefix=/usr/local - opts = Options() + opts = Variables() - opts = Options() - opts.Add(PathOption('qtdir', + opts = Variables() + opts.Add(PathVariable('qtdir', 'where the root of Qt is installed', qtdir, PathIsDir)) - opts.Add(PathOption('qt_includes', + opts.Add(PathVariable('qt_includes', 'where the Qt includes are installed', '$qtdir/includes', PathIsDirCreate)) - opts.Add(PathOption('qt_libraries', + opts.Add(PathVariable('qt_libraries', 'where the Qt library is installed', '$qtdir/lib')) @@ -70,12 +70,14 @@ Usage example: __revision__ = "__FILE__ __REVISION__ __DATE__ __DEVELOPER__" +__all__ = ['PathVariable',] + import os import os.path import SCons.Errors -class _PathOptionClass: +class _PathVariableClass: def PathAccept(self, key, val, env): """Accepts any path, no checking done.""" @@ -136,4 +138,4 @@ class _PathOptionClass: return (key, '%s ( /path/to/%s )' % (help, key), default, validator, None) -PathOption = _PathOptionClass() +PathVariable = _PathVariableClass() diff --git a/src/engine/SCons/Options/PathOptionTests.py b/src/engine/SCons/Variables/PathVariableTests.py index 22e79ae..980cc5e 100644 --- a/src/engine/SCons/Options/PathOptionTests.py +++ b/src/engine/SCons/Variables/PathVariableTests.py @@ -28,15 +28,15 @@ import sys import unittest import SCons.Errors -import SCons.Options +import SCons.Variables import TestCmd -class PathOptionTestCase(unittest.TestCase): - def test_PathOption(self): - """Test PathOption creation""" - opts = SCons.Options.Options() - opts.Add(SCons.Options.PathOption('test', +class PathVariableTestCase(unittest.TestCase): + def test_PathVariable(self): + """Test PathVariable creation""" + opts = SCons.Variables.Variables() + opts.Add(SCons.Variables.PathVariable('test', 'test option help', '/default/path')) @@ -49,11 +49,11 @@ class PathOptionTestCase(unittest.TestCase): def test_PathExists(self): """Test the PathExists validator""" - opts = SCons.Options.Options() - opts.Add(SCons.Options.PathOption('test', + opts = SCons.Variables.Variables() + opts.Add(SCons.Variables.PathVariable('test', 'test option help', '/default/path', - SCons.Options.PathOption.PathExists)) + SCons.Variables.PathVariable.PathExists)) test = TestCmd.TestCmd(workdir='') test.write('exists', 'exists\n') @@ -72,11 +72,11 @@ class PathOptionTestCase(unittest.TestCase): def test_PathIsDir(self): """Test the PathIsDir validator""" - opts = SCons.Options.Options() - opts.Add(SCons.Options.PathOption('test', + opts = SCons.Variables.Variables() + opts.Add(SCons.Variables.PathVariable('test', 'test option help', '/default/path', - SCons.Options.PathOption.PathIsDir)) + SCons.Variables.PathVariable.PathIsDir)) test = TestCmd.TestCmd(workdir='') test.subdir('dir') @@ -104,11 +104,11 @@ class PathOptionTestCase(unittest.TestCase): def test_PathIsDirCreate(self): """Test the PathIsDirCreate validator""" - opts = SCons.Options.Options() - opts.Add(SCons.Options.PathOption('test', + opts = SCons.Variables.Variables() + opts.Add(SCons.Variables.PathVariable('test', 'test option help', '/default/path', - SCons.Options.PathOption.PathIsDirCreate)) + SCons.Variables.PathVariable.PathIsDirCreate)) test = TestCmd.TestCmd(workdir='') test.write('file', "file\n") @@ -129,11 +129,11 @@ class PathOptionTestCase(unittest.TestCase): def test_PathIsFile(self): """Test the PathIsFile validator""" - opts = SCons.Options.Options() - opts.Add(SCons.Options.PathOption('test', + opts = SCons.Variables.Variables() + opts.Add(SCons.Variables.PathVariable('test', 'test option help', '/default/path', - SCons.Options.PathOption.PathIsFile)) + SCons.Variables.PathVariable.PathIsFile)) test = TestCmd.TestCmd(workdir='') test.subdir('dir') @@ -161,11 +161,11 @@ class PathOptionTestCase(unittest.TestCase): def test_PathAccept(self): """Test the PathAccept validator""" - opts = SCons.Options.Options() - opts.Add(SCons.Options.PathOption('test', + opts = SCons.Variables.Variables() + opts.Add(SCons.Variables.PathVariable('test', 'test option help', '/default/path', - SCons.Options.PathOption.PathAccept)) + SCons.Variables.PathVariable.PathAccept)) test = TestCmd.TestCmd(workdir='') test.subdir('dir') @@ -182,9 +182,9 @@ class PathOptionTestCase(unittest.TestCase): o.validator('X', dne, {}) def test_validator(self): - """Test the PathOption validator argument""" - opts = SCons.Options.Options() - opts.Add(SCons.Options.PathOption('test', + """Test the PathVariable validator argument""" + opts = SCons.Variables.Variables() + opts.Add(SCons.Variables.PathVariable('test', 'test option help', '/default/path')) @@ -207,8 +207,8 @@ class PathOptionTestCase(unittest.TestCase): def my_validator(key, val, env): raise Exception, "my_validator() got called for %s, %s!" % (key, val) - opts = SCons.Options.Options() - opts.Add(SCons.Options.PathOption('test2', + opts = SCons.Variables.Variables() + opts.Add(SCons.Variables.PathVariable('test2', 'more help', '/default/path/again', my_validator)) @@ -226,6 +226,6 @@ class PathOptionTestCase(unittest.TestCase): if __name__ == "__main__": - suite = unittest.makeSuite(PathOptionTestCase, 'test_') + suite = unittest.makeSuite(PathVariableTestCase, 'test_') if not unittest.TextTestRunner().run(suite).wasSuccessful(): sys.exit(1) diff --git a/src/engine/SCons/Options/OptionsTests.py b/src/engine/SCons/Variables/VariablesTests.py index 5ec9d7a..cd6ab1d 100644 --- a/src/engine/SCons/Options/OptionsTests.py +++ b/src/engine/SCons/Variables/VariablesTests.py @@ -28,7 +28,7 @@ import sys import unittest import TestSCons -import SCons.Options +import SCons.Variables import SCons.Subst import SCons.Warnings @@ -57,11 +57,11 @@ def checkSave(file, expected): execfile(file, gdict, ldict) assert expected == ldict, "%s\n...not equal to...\n%s" % (expected, ldict) -class OptionsTestCase(unittest.TestCase): +class VariablesTestCase(unittest.TestCase): def test_keys(self): - """Test the Options.keys() method""" - opts = SCons.Options.Options() + """Test the Variables.keys() method""" + opts = SCons.Variables.Variables() opts.Add('VAR1') opts.Add('VAR2', @@ -73,8 +73,8 @@ class OptionsTestCase(unittest.TestCase): assert keys == ['VAR1', 'VAR2'], keys def test_Add(self): - """Test adding to an Options object""" - opts = SCons.Options.Options() + """Test adding to a Variables object""" + opts = SCons.Variables.Variables() opts.Add('VAR') opts.Add('ANSWER', @@ -107,11 +107,11 @@ class OptionsTestCase(unittest.TestCase): test_it('foo-bar') test_it('foo.bar') - def test_AddOptions(self): - """Test adding a list of options to an Options object""" - opts = SCons.Options.Options() + def test_AddVariables(self): + """Test adding a list of options to a Variables object""" + opts = SCons.Variables.Variables() - opts.AddOptions(('VAR2',), + opts.AddVariables(('VAR2',), ('ANSWER2', 'THE answer to THE question', "42", @@ -137,7 +137,7 @@ class OptionsTestCase(unittest.TestCase): # Test that a default value is validated correctly. test = TestSCons.TestSCons() file = test.workpath('custom.py') - opts = SCons.Options.Options(file) + opts = SCons.Variables.Variables(file) opts.Add('ANSWER', 'THE answer to THE question', @@ -158,7 +158,7 @@ class OptionsTestCase(unittest.TestCase): test = TestSCons.TestSCons() file = test.workpath('custom.py') test.write('custom.py', 'ANSWER=54') - opts = SCons.Options.Options(file) + opts = SCons.Variables.Variables(file) opts.Add('ANSWER', 'THE answer to THE question', @@ -186,7 +186,7 @@ class OptionsTestCase(unittest.TestCase): test = TestSCons.TestSCons() file = test.workpath('custom.py') test.write('custom.py', 'ANSWER=42') - opts = SCons.Options.Options(file) + opts = SCons.Variables.Variables(file) opts.Add('ANSWER', 'THE answer to THE question', @@ -207,7 +207,7 @@ class OptionsTestCase(unittest.TestCase): test = TestSCons.TestSCons() file = test.workpath('custom.py') test.write('custom.py', 'ANSWER=10') - opts = SCons.Options.Options(file) + opts = SCons.Variables.Variables(file) opts.Add('ANSWER', 'THE answer to THE question', @@ -228,7 +228,7 @@ class OptionsTestCase(unittest.TestCase): test = TestSCons.TestSCons() file = test.workpath('custom.py') test.write('custom.py', 'ANSWER=10') - opts = SCons.Options.Options(file) + opts = SCons.Variables.Variables(file) opts.Add('ANSWER', 'THE answer to THE question', @@ -246,7 +246,7 @@ class OptionsTestCase(unittest.TestCase): # or args. test = TestSCons.TestSCons() file = test.workpath('custom.py') - opts = SCons.Options.Options(file) + opts = SCons.Variables.Variables(file) opts.Add('ANSWER', help='THE answer to THE question', @@ -259,7 +259,7 @@ class OptionsTestCase(unittest.TestCase): # Test that a default value of None is all right. test = TestSCons.TestSCons() file = test.workpath('custom.py') - opts = SCons.Options.Options(file) + opts = SCons.Variables.Variables(file) opts.Add('ANSWER', "This is the answer", @@ -278,7 +278,7 @@ class OptionsTestCase(unittest.TestCase): test = TestSCons.TestSCons() file = test.workpath('custom.py') test.write('custom.py', 'ANSWER=42') - opts = SCons.Options.Options(file, {'ANSWER':54}) + opts = SCons.Variables.Variables(file, {'ANSWER':54}) opts.Add('ANSWER', 'THE answer to THE question', @@ -298,7 +298,7 @@ class OptionsTestCase(unittest.TestCase): test = TestSCons.TestSCons() file = test.workpath('custom.py') test.write('custom.py', 'ANSWER=54') - opts = SCons.Options.Options(file, {'ANSWER':42}) + opts = SCons.Variables.Variables(file, {'ANSWER':42}) opts.Add('ANSWER', 'THE answer to THE question', @@ -315,7 +315,7 @@ class OptionsTestCase(unittest.TestCase): test = TestSCons.TestSCons() file = test.workpath('custom.py') test.write('custom.py', 'ANSWER=54') - opts = SCons.Options.Options(file, {'ANSWER':54}) + opts = SCons.Variables.Variables(file, {'ANSWER':54}) opts.Add('ANSWER', 'THE answer to THE question', @@ -328,11 +328,11 @@ class OptionsTestCase(unittest.TestCase): assert env['ANSWER'] == 54 def test_Save(self): - """Testing saving Options""" + """Testing saving Variables""" test = TestSCons.TestSCons() cache_file = test.workpath('cached.options') - opts = SCons.Options.Options() + opts = SCons.Variables.Variables() def bool_converter(val): if val in [1, 'y']: val = 1 @@ -387,7 +387,7 @@ class OptionsTestCase(unittest.TestCase): test = TestSCons.TestSCons() cache_file = test.workpath('cached.options') - opts = SCons.Options.Options() + opts = SCons.Variables.Variables() opts.Add('THIS_USED_TO_BREAK', 'An option to test', @@ -412,7 +412,7 @@ class OptionsTestCase(unittest.TestCase): def test_GenerateHelpText(self): """Test generating the default format help text""" - opts = SCons.Options.Options() + opts = SCons.Variables.Variables() opts.Add('ANSWER', 'THE answer to THE question', @@ -468,14 +468,14 @@ B: b - alpha test text = opts.GenerateHelpText(env, sort=cmp) assert text == expectAlpha, text - def test_FormatOptionHelpText(self): + def test_FormatVariableHelpText(self): """Test generating custom format help text""" - opts = SCons.Options.Options() + opts = SCons.Variables.Variables() def my_format(env, opt, help, default, actual, aliases): return '%s %s %s %s %s\n' % (opt, default, actual, help, aliases) - opts.FormatOptionHelpText = my_format + opts.FormatVariableHelpText = my_format opts.Add('ANSWER', 'THE answer to THE question', @@ -517,11 +517,11 @@ B 42 54 b - alpha test ['B'] -class UnknownOptionsTestCase(unittest.TestCase): +class UnknownVariablesTestCase(unittest.TestCase): def test_unknown(self): - """Test the UnknownOptions() method""" - opts = SCons.Options.Options() + """Test the UnknownVariables() method""" + opts = SCons.Variables.Variables() opts.Add('ANSWER', 'THE answer to THE question', @@ -535,7 +535,7 @@ class UnknownOptionsTestCase(unittest.TestCase): env = Environment() opts.Update(env, args) - r = opts.UnknownOptions() + r = opts.UnknownVariables() assert r == {'UNKNOWN' : 'unknown'}, r assert env['ANSWER'] == 'answer', env['ANSWER'] @@ -543,8 +543,8 @@ class UnknownOptionsTestCase(unittest.TestCase): if __name__ == "__main__": suite = unittest.TestSuite() - tclasses = [ OptionsTestCase, - UnknownOptionsTestCase ] + tclasses = [ VariablesTestCase, + UnknownVariablesTestCase ] for tclass in tclasses: names = unittest.getTestCaseNames(tclass, 'test_') suite.addTests(map(tclass, names)) diff --git a/src/engine/SCons/Options/__init__.py b/src/engine/SCons/Variables/__init__.py index 9389c7b..a86cd34 100644 --- a/src/engine/SCons/Options/__init__.py +++ b/src/engine/SCons/Variables/__init__.py @@ -1,6 +1,6 @@ -"""engine.SCons.Options +"""engine.SCons.Variables -This file defines the Options class that is used to add user-friendly +This file defines the Variables class that is used to add user-friendly customizable variables to an SCons build. """ @@ -40,14 +40,14 @@ import SCons.Errors import SCons.Util import SCons.Warnings -from BoolOption import BoolOption # okay -from EnumOption import EnumOption # okay -from ListOption import ListOption # naja -from PackageOption import PackageOption # naja -from PathOption import PathOption # okay +from BoolVariable import BoolVariable # okay +from EnumVariable import EnumVariable # okay +from ListVariable import ListVariable # naja +from PackageVariable import PackageVariable # naja +from PathVariable import PathVariable # okay -class Options: +class Variables: instance=None """ @@ -72,16 +72,16 @@ class Options: # create the singleton instance if is_global: - self=Options.instance + self=Variables.instance - if not Options.instance: - Options.instance=self + if not Variables.instance: + Variables.instance=self def _do_add(self, key, help="", default=None, validator=None, converter=None): - class Option: + class Variable: pass - option = Option() + option = Variable() # if we get a list or a tuple, we take the first element as the # option key and store the remaining in aliases. @@ -123,11 +123,11 @@ class Options: if not SCons.Util.is_String(key) or \ not SCons.Environment.is_valid_construction_var(key): - raise SCons.Errors.UserError, "Illegal Options.Add() key `%s'" % str(key) + raise SCons.Errors.UserError, "Illegal Variables.Add() key `%s'" % str(key) self._do_add(key, help, default, validator, converter) - def AddOptions(self, *optlist): + def AddVariables(self, *optlist): """ Add a list of options. @@ -135,7 +135,7 @@ class Options: to the underlying method for adding options. Example: - opt.AddOptions( + opt.AddVariables( ('debug', '', 0), ('CC', 'The C compiler'), ('VALIDATE', 'An option for testing validation', 'notset', @@ -213,7 +213,7 @@ class Options: if option.validator and values.has_key(option.key): option.validator(option.key, env.subst('${%s}'%option.key), env) - def UnknownOptions(self): + def UnknownVariables(self): """ Returns any options in the specified arguments lists that were not known, declared options in this object. @@ -288,7 +288,7 @@ class Options: actual = env.subst('${%s}' % opt.key) else: actual = None - return self.FormatOptionHelpText(env, opt.key, opt.help, opt.default, actual, opt.aliases) + return self.FormatVariableHelpText(env, opt.key, opt.help, opt.default, actual, opt.aliases) lines = filter(None, map(format, options)) return string.join(lines, '') @@ -296,7 +296,7 @@ class Options: format = '\n%s: %s\n default: %s\n actual: %s\n' format_ = '\n%s: %s\n default: %s\n actual: %s\n aliases: %s\n' - def FormatOptionHelpText(self, env, key, help, default, actual, aliases=[]): + def FormatVariableHelpText(self, env, key, help, default, actual, aliases=[]): # Don't display the key name itself as an alias. aliases = filter(lambda a, k=key: a != k, aliases) if len(aliases)==0: diff --git a/src/engine/SCons/compat/__init__.py b/src/engine/SCons/compat/__init__.py index 1d36dbc..20fb8ed 100644 --- a/src/engine/SCons/compat/__init__.py +++ b/src/engine/SCons/compat/__init__.py @@ -138,7 +138,12 @@ except AttributeError: return result fnmatch.filter = filter del filter - + +try: + import itertools +except ImportError: + # Pre-2.3 Python has no itertools module. + import_as('_scons_itertools', 'itertools') # If we need the compatibility version of textwrap, it must be imported # before optparse, which uses it. diff --git a/src/engine/SCons/compat/_scons_itertools.py b/src/engine/SCons/compat/_scons_itertools.py new file mode 100644 index 0000000..9720daf --- /dev/null +++ b/src/engine/SCons/compat/_scons_itertools.py @@ -0,0 +1,118 @@ +# +# __COPYRIGHT__ +# +# Permission is hereby granted, free of charge, to any person obtaining +# a copy of this software and associated documentation files (the +# "Software"), to deal in the Software without restriction, including +# without limitation the rights to use, copy, modify, merge, publish, +# distribute, sublicense, and/or sell copies of the Software, and to +# permit persons to whom the Software is furnished to do so, subject to +# the following conditions: +# +# The above copyright notice and this permission notice shall be included +# in all copies or substantial portions of the Software. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY +# KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE +# WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND +# NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE +# LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION +# OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION +# WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. +# + +__revision__ = "__FILE__ __REVISION__ __DATE__ __DEVELOPER__" + +__doc__ = """ +Implementations of itertools functions for Python versions that don't +have iterators. + +These implement the functions by creating the entire list, not returning +it element-by-element as the real itertools functions do. This means +that early Python versions won't get the performance benefit of using +the itertools, but we can still use them so the later Python versions +do get the advantages of using iterators. + +Because we return the entire list, we intentionally do not implement the +itertools functions that "return" infinitely-long lists: the count(), +cycle() and repeat() functions. Other functions below have remained +unimplemented simply because they aren't being used (yet) and it wasn't +obvious how to do it. Or, conversely, we only implemented those functions +that *were* easy to implement (mostly because the Python documentation +contained examples of equivalent code). + +Note that these do not have independent unit tests, so it's possible +that there are bugs. +""" + +def chain(*iterables): + result = [] + for x in iterables: + result.extend(list(x)) + return result + +def count(n=0): + # returns infinite length, should not be supported + raise NotImplementedError + +def cycle(iterable): + # returns infinite length, should not be supported + raise NotImplementedError + +def dropwhile(predicate, iterable): + result = [] + for x in iterable: + if not predicate(x): + result.append(x) + break + result.extend(iterable) + return result + +def groupby(iterable, *args): + raise NotImplementedError + +def ifilter(predicate, iterable): + result = [] + if predicate is None: + predicate = bool + for x in iterable: + if predicate(x): + result.append(x) + return result + +def ifilterfalse(predicate, iterable): + result = [] + if predicate is None: + predicate = bool + for x in iterable: + if not predicate(x): + result.append(x) + return result + +def imap(function, *iterables): + return apply(map, (function,) + tuple(iterables)) + +def islice(*args, **kw): + raise NotImplementedError + +def izip(*iterables): + return apply(zip, iterables) + +def repeat(*args, **kw): + # returns infinite length, should not be supported + raise NotImplementedError + +def starmap(*args, **kw): + raise NotImplementedError + +def takewhile(predicate, iterable): + result = [] + for x in iterable: + if predicate(x): + result.append(x) + else: + break + return result + +def tee(*args, **kw): + raise NotImplementedError diff --git a/src/engine/SCons/compat/_scons_sets15.py b/src/engine/SCons/compat/_scons_sets15.py index b3d0bb3..1fe5a4f 100644 --- a/src/engine/SCons/compat/_scons_sets15.py +++ b/src/engine/SCons/compat/_scons_sets15.py @@ -26,10 +26,11 @@ class Set: if seq: for elem in seq: if elem not in self.elems: + hash(elem) self.elems.append(elem) def __str__(self): - return "{%s}" % string.join(map(str, self.elems), ", ") + return "set([%s])" % string.join(map(str, self.elems), ", ") def copy(self): @@ -56,6 +57,7 @@ class Set: def add(self, elem): """Add one element to the set.""" if elem not in self.elems: + hash(elem) self.elems.append(elem) def remove(self, elem): @@ -157,3 +159,12 @@ class Set: return 0 else: return len(self - other) == 0 + + def __cmp__(self, other): + """Returns 1 if the sets are equal.""" + if self.__lt__(other): + return -1 + elif other.__lt__(self): + return 1 + else: + return 0 diff --git a/src/engine/SCons/compat/_scons_textwrap.py b/src/engine/SCons/compat/_scons_textwrap.py index 1e0e411..72ed9b9 100644 --- a/src/engine/SCons/compat/_scons_textwrap.py +++ b/src/engine/SCons/compat/_scons_textwrap.py @@ -90,13 +90,16 @@ class TextWrapper: # splits into # Hello/ /there/ /--/ /you/ /goof-/ball,/ /use/ /the/ /-b/ /option! # (after stripping out empty strings). - wordsep_re = re.compile(r'(\s+|' # any whitespace - r'-*\w{2,}-(?=\w{2,}))') # hyphenated words - # Earlier Python's don't have the (?<= - # negative look-behind assertion. It doesn't - # matter for the simple input SCons is going to - # give it, so just comment it out. - #r'(?<=[\w\!\"\'\&\.\,\?])-{2,}(?=\w))') # em-dash + try: + wordsep_re = re.compile(r'(\s+|' # any whitespace + r'[^\s\w]*\w{2,}-(?=\w{2,})|' # hyphenated words + r'(?<=[\w\!\"\'\&\.\,\?])-{2,}(?=\w))') # em-dash + except re.error: + # Pre-2.0 Python versions don't have the (?<= negative look-behind + # assertion. It mostly doesn't matter for the simple input + # SCons is going to give it, so just leave it out. + wordsep_re = re.compile(r'(\s+|' # any whitespace + r'-*\w{2,}-(?=\w{2,}))') # hyphenated words # XXX will there be a locale-or-charset-aware version of # string.lowercase in 2.3? diff --git a/src/engine/SCons/dblite.py b/src/engine/SCons/dblite.py index 36840c3..437f05a 100644 --- a/src/engine/SCons/dblite.py +++ b/src/engine/SCons/dblite.py @@ -5,12 +5,9 @@ import cPickle import time import shutil import os -import os.path import types import __builtin__ -_open = __builtin__.open # avoid name clash - keep_all_files = 00000 ignore_corrupt_dbfiles = 0 @@ -35,6 +32,24 @@ tmp_suffix = '.tmp' class dblite: + # Squirrel away references to the functions in various modules + # that we'll use when our __del__() method calls our sync() method + # during shutdown. We might get destroyed when Python is in the midst + # of tearing down the different modules we import in an essentially + # arbitrary order, and some of the various modules's global attributes + # may already be wiped out from under us. + # + # See the discussion at: + # http://mail.python.org/pipermail/python-bugs-list/2003-March/016877.html + + _open = __builtin__.open + _cPickle_dump = cPickle.dump + _os_chmod = os.chmod + _os_rename = os.rename + _os_unlink = os.unlink + _shutil_copyfile = shutil.copyfile + _time_time = time.time + def __init__(self, file_base_name, flag, mode): assert flag in (None, "r", "w", "c", "n") if (flag is None): flag = "r" @@ -51,14 +66,14 @@ class dblite: self._dict = {} self._needs_sync = 00000 if (self._flag == "n"): - _open(self._file_name, "wb", self._mode) + self._open(self._file_name, "wb", self._mode) else: try: - f = _open(self._file_name, "rb") + f = self._open(self._file_name, "rb") except IOError, e: if (self._flag != "c"): raise e - _open(self._file_name, "wb", self._mode) + self._open(self._file_name, "wb", self._mode) else: p = f.read() if (len(p) > 0): @@ -75,8 +90,8 @@ class dblite: def sync(self): self._check_writable() - f = _open(self._tmp_name, "wb", self._mode) - cPickle.dump(self._dict, f, 1) + f = self._open(self._tmp_name, "wb", self._mode) + self._cPickle_dump(self._dict, f, 1) f.close() # Windows doesn't allow renaming if the file exists, so unlink # it first, chmod'ing it to make sure we can do so. On UNIX, we @@ -84,15 +99,15 @@ class dblite: # (e.g. from a previous run as root). We should still be able to # unlink() the file if the directory's writable, though, so ignore # any OSError exception thrown by the chmod() call. - try: os.chmod(self._file_name, 0777) + try: self._os_chmod(self._file_name, 0777) except OSError: pass - os.unlink(self._file_name) - os.rename(self._tmp_name, self._file_name) + self._os_unlink(self._file_name) + self._os_rename(self._tmp_name, self._file_name) self._needs_sync = 00000 if (keep_all_files): - shutil.copyfile( + self._shutil_copyfile( self._file_name, - self._file_name + "_" + str(int(time.time()))) + self._file_name + "_" + str(int(self._time_time()))) def _check_writable(self): if (self._flag == "r"): diff --git a/src/script/scons.bat b/src/script/scons.bat index f927bb1..56443c8 100644 --- a/src/script/scons.bat +++ b/src/script/scons.bat @@ -7,10 +7,14 @@ python -c "from os.path import join; import sys; sys.path = [ join(sys.prefix, ' @REM no way to set exit status of this script for 9x/Me
goto endscons
:WinNT
-@REM set path=%path%;%~dp0
+setlocal
+@REM ensure the script will be executed with the Python it was installed for
+set path=%~dp0;%~dp0..;%path%
python -c "from os.path import join; import sys; sys.path = [ join(sys.prefix, 'Lib', 'site-packages', 'scons-__VERSION__'), join(sys.prefix, 'Lib', 'site-packages', 'scons'), join(sys.prefix, 'scons-__VERSION__'), join(sys.prefix, 'scons')] + sys.path; import SCons.Script; SCons.Script.main()" %*
-if NOT "%COMSPEC%" == "%SystemRoot%\system32\cmd.exe" goto endscons
+if NOT "%COMSPEC%" == "%SystemRoot%\system32\cmd.exe" goto endsconsnt
if errorlevel 9009 echo you do not have python in your PATH
@REM color 00 causes this script to exit with non-zero exit status
if errorlevel 1 color 00
+:endsconsnt
+endlocal
:endscons
diff --git a/src/setup.py b/src/setup.py index 4660b2f..281a69c 100644 --- a/src/setup.py +++ b/src/setup.py @@ -376,12 +376,13 @@ arguments = { 'packages' : ["SCons", "SCons.compat", "SCons.Node", - "SCons.Options", "SCons.Platform", "SCons.Scanner", "SCons.Script", "SCons.Tool", - "SCons.Tool.packaging"], + "SCons.Tool.packaging", + "SCons.Variables", + ], 'package_dir' : {'' : 'engine'}, 'data_files' : [('man/man1', man_pages)], 'scripts' : scripts, |