diff options
| author | Steven Knight <knight@baldmt.com> | 2007-08-17 03:17:04 (GMT) |
|---|---|---|
| committer | Steven Knight <knight@baldmt.com> | 2007-08-17 03:17:04 (GMT) |
| commit | 8cb34cccc26935bce5d07ed3f51cc29fbbd1ab03 (patch) | |
| tree | 7bc281e44fe45cae35656e0e457fef8f308fbb7b /src/engine/SCons | |
| parent | 5b4b4c6e5384712ab1835bdcf8beea09611b6e62 (diff) | |
| download | SCons-8cb34cccc26935bce5d07ed3f51cc29fbbd1ab03.zip SCons-8cb34cccc26935bce5d07ed3f51cc29fbbd1ab03.tar.gz SCons-8cb34cccc26935bce5d07ed3f51cc29fbbd1ab03.tar.bz2 | |
Merged revisions 2136-2200,2202-2290,2292-2301 via svnmerge from
http://scons.tigris.org/svn/scons/branches/core
........
r2145 | stevenknight | 2007-07-17 09:15:12 -0500 (Tue, 17 Jul 2007) | 3 lines
Don't put null strings (from variable expansion) in a path list.
(They get turned into the current directory on later expansion.)
........
r2146 | stevenknight | 2007-07-17 10:47:39 -0500 (Tue, 17 Jul 2007) | 3 lines
Add support for optional arguments on command-line long options
by specifying nargs='?'.
........
r2149 | stevenknight | 2007-07-17 15:22:24 -0500 (Tue, 17 Jul 2007) | 2 lines
Remove left-over Optik mentions.
........
r2150 | stevenknight | 2007-07-17 15:39:34 -0500 (Tue, 17 Jul 2007) | 4 lines
Add a $SWIGPATH variable for finding SWIG dependencies, with
$SWIGINC{PREFIX,SUFFIX} for adding them to the command line.
........
r2154 | stevenknight | 2007-07-18 20:05:31 -0500 (Wed, 18 Jul 2007) | 2 lines
Fix variable misspellings in the doc added for $SWIGOUTPUT.
........
r2155 | stevenknight | 2007-07-18 20:07:28 -0500 (Wed, 18 Jul 2007) | 2 lines
Add the Python eggs info file to the RPM packaging build.
........
r2156 | stevenknight | 2007-07-18 20:15:08 -0500 (Wed, 18 Jul 2007) | 2 lines
Convert documentation from DocBook SGML to XML.
........
r2158 | stevenknight | 2007-07-19 17:16:19 -0500 (Thu, 19 Jul 2007) | 3 lines
Conditionally add the .egg-info the RPM file list only if the distutils
in the version of Python that rpmbuild will execute knows about them.
........
r2161 | stevenknight | 2007-07-19 19:12:29 -0500 (Thu, 19 Jul 2007) | 5 lines
Capture a test case (contributed by Tilo Prutz) where instantiation of
a private class causes javac to generate an additional anonymous inner
class file. (No solution yet, but there's no sense throwing away the
preparatory work.)
........
r2162 | stevenknight | 2007-07-20 11:29:56 -0500 (Fri, 20 Jul 2007) | 3 lines
Support passing a list of .java files as source to the Java() builder.
(Leanid Nazdrynau)
........
r2163 | garyo | 2007-07-20 12:00:35 -0500 (Fri, 20 Jul 2007) | 1 line
Fixed cut-n-paste error in Touch factory method doc in users guide.
........
r2167 | stevenknight | 2007-07-21 22:59:40 -0500 (Sat, 21 Jul 2007) | 2 lines
Don't execute the SWIGOUTDIR test if swig isn't installed.
........
r2168 | stevenknight | 2007-07-21 23:14:17 -0500 (Sat, 21 Jul 2007) | 2 lines
Fix the test's ability to run under a path name containing spaces.
........
r2171 | stevenknight | 2007-07-24 15:54:41 -0500 (Tue, 24 Jul 2007) | 2 lines
Handle white space in key file names in the packaging build.
........
r2172 | stevenknight | 2007-07-24 21:41:15 -0500 (Tue, 24 Jul 2007) | 2 lines
More efficient copying of construction environments.
........
r2173 | stevenknight | 2007-07-25 10:56:02 -0500 (Wed, 25 Jul 2007) | 2 lines
Update the SCons build for Subversion and general clean-up.
........
r2174 | stevenknight | 2007-07-25 11:35:16 -0500 (Wed, 25 Jul 2007) | 3 lines
Suppress the [brackets] around a node in the --tree=prune output if
the node is a source.
........
r2175 | stevenknight | 2007-07-25 12:52:18 -0500 (Wed, 25 Jul 2007) | 3 lines
Commonize the skip_test() method and make its behavior configurable
via a TESTCOMMON_PASS_SKIPS environment variable.
........
r2178 | stevenknight | 2007-07-25 21:43:47 -0500 (Wed, 25 Jul 2007) | 3 lines
Add $JAVACLASSPATH and $JAVASOURCEPATH construction variables. (Leanid
Nazdrynau)
........
r2182 | stevenknight | 2007-07-30 12:10:20 -0500 (Mon, 30 Jul 2007) | 3 lines
Refactor Builder suffix-adjusting into its own method, so we can
(potentially) re-use it for Builders with attached source Builders.
........
r2183 | stevenknight | 2007-07-30 14:51:53 -0500 (Mon, 30 Jul 2007) | 2 lines
More efficient source-builder suffix matching.
........
r2184 | stevenknight | 2007-07-30 16:01:42 -0500 (Mon, 30 Jul 2007) | 4 lines
Encapsulate initialization of the default FS object by an accessor
function in SCons.Node.FS. (This also gets rid of an unnecessary
reference to SCons.Node.FS.default_fs in the LaTeX scanner.)
........
r2193 | stevenknight | 2007-07-30 18:24:07 -0500 (Mon, 30 Jul 2007) | 3 lines
Fix interpretation of source arguments that have no suffix when the
called Builder has both a src_suffix and a src_builder.
........
r2194 | stevenknight | 2007-07-31 10:25:31 -0500 (Tue, 31 Jul 2007) | 2 lines
Increase the number of tries for random output from three to ten.
........
r2195 | stevenknight | 2007-07-31 10:52:28 -0500 (Tue, 31 Jul 2007) | 3 lines
Skip the test gracefully if the zipfile module can't read the file it
just wrote (which is the case for Python 2.1 on 64-bit systems).
........
r2196 | stevenknight | 2007-07-31 13:06:21 -0500 (Tue, 31 Jul 2007) | 2 lines
Move the "import zipfile" so it doesn't fail on Python <= 2.0.
........
r2197 | stevenknight | 2007-07-31 14:51:50 -0500 (Tue, 31 Jul 2007) | 3 lines
Commonize initialization of the various Java builders so they can be
hooked up into a multi-stage Builder chain. (Leanid Nazdrynau)
........
r2198 | stevenknight | 2007-07-31 16:15:18 -0500 (Tue, 31 Jul 2007) | 3 lines
Fix use of ${TARGET.dir} and ${SOURCE.dir} expansions in $FORTRANMODDIR
$JARCHDIR, $JARFLAGS, $LEXFLAGS, $SWIGFLAGS, $SWIGOUTDIR and $YACCFLAGS.
........
r2199 | stevenknight | 2007-07-31 16:25:48 -0500 (Tue, 31 Jul 2007) | 2 lines
Remove left-over Trace() call.
........
r2202 | stevenknight | 2007-08-01 12:31:48 -0500 (Wed, 01 Aug 2007) | 3 lines
Bail out via test.skip_test() if wix ("candle") isn't found.
Put the main body of code flush left instead of under an if: block.
........
r2203 | stevenknight | 2007-08-01 15:35:55 -0500 (Wed, 01 Aug 2007) | 5 lines
Fix Tool.packaging.rpm.package() so it doesn't always overwrite
$RPMFLAGS with -ta.
Set --buildroot in RPM packaging tests so they don't overwrite
each other when run simultaneously.
........
r2204 | stevenknight | 2007-08-01 15:37:36 -0500 (Wed, 01 Aug 2007) | 2 lines
Fix a nested scope issue with the internal build_sources() function.
........
r2205 | stevenknight | 2007-08-01 15:46:08 -0500 (Wed, 01 Aug 2007) | 5 lines
Normalize (X out) the CreationDate field inside embedded, compressed
PostScript streams within the generated PDF files. Also normalize
preceding Length field, since compression length is affected by different
patterns of input, including the variable CreationDate value.
........
r2211 | stevenknight | 2007-08-02 08:52:06 -0500 (Thu, 02 Aug 2007) | 2 lines
Add the new modules from branches/packaging to the SCons packaging build.
........
r2212 | stevenknight | 2007-08-02 19:59:01 -0500 (Thu, 02 Aug 2007) | 2 lines
Fix the JAVACLASSPATH test when javah isn't on the default $PATH.
........
r2214 | stevenknight | 2007-08-03 15:05:21 -0500 (Fri, 03 Aug 2007) | 4 lines
Hook up the Java builders into a multi-step chain underneath a Java()
pseudo-builder (wrapper) that examines its arguments and calls the
appropriate underlying file-or-dir builder.
........
r2215 | stevenknight | 2007-08-03 15:49:58 -0500 (Fri, 03 Aug 2007) | 2 lines
Fix for old Python versions: use apply() instead of *args, **kw.
........
r2216 | stevenknight | 2007-08-03 16:49:31 -0500 (Fri, 03 Aug 2007) | 2 lines
Hook up the SWIG builder as a source builder for .java files.
........
r2217 | stevenknight | 2007-08-03 17:28:19 -0500 (Fri, 03 Aug 2007) | 2 lines
Don't use .endswith(), which didn't appear until later Python versions.
........
r2218 | stevenknight | 2007-08-03 17:29:38 -0500 (Fri, 03 Aug 2007) | 2 lines
Replace tabs with spaces.
........
r2219 | stevenknight | 2007-08-04 08:06:23 -0500 (Sat, 04 Aug 2007) | 3 lines
Initialize a loop-invariant lambda for matching .java suffixes outside
the loop.
........
r2220 | stevenknight | 2007-08-07 15:06:13 -0500 (Tue, 07 Aug 2007) | 2 lines
Refactor parallel class-generation loops into one.
........
r2221 | stevenknight | 2007-08-07 16:04:06 -0500 (Tue, 07 Aug 2007) | 5 lines
Have the Java multi-step builder test actually check for generated files,
and fix the generation of .java and .class file names, and interaction
with the SWIG builder, so that the files are generated in the correct
place.
........
r2222 | stevenknight | 2007-08-07 16:45:05 -0500 (Tue, 07 Aug 2007) | 3 lines
Fix dependencies on SWIG-generated .java files so they don't have to
be built in multiple passes.
........
r2226 | stevenknight | 2007-08-07 18:00:22 -0500 (Tue, 07 Aug 2007) | 2 lines
Fix SWIG when used with BuildDir().
........
r2227 | stevenknight | 2007-08-07 22:15:55 -0500 (Tue, 07 Aug 2007) | 5 lines
User's guide updates:
- Make the multiple files example match its text.
- Expand a truncated sentence about being able to use Python function actions
in the Command() Builder.
........
r2228 | stevenknight | 2007-08-07 23:25:18 -0500 (Tue, 07 Aug 2007) | 3 lines
Don't generate an error if a #include file matches a same-named
directory in $CPPPATH (or $FORTRANPATH, etc.).
........
r2229 | stevenknight | 2007-08-07 23:40:00 -0500 (Tue, 07 Aug 2007) | 2 lines
Fix a code example. (Gary Oberbrunner)
........
r2230 | stevenknight | 2007-08-08 00:05:43 -0500 (Wed, 08 Aug 2007) | 3 lines
Capture a test case to make sure AddPostAction() doesn't interfere
with normal linking. (Matt Doar, Gary Oberbrunner)
........
r2233 | stevenknight | 2007-08-08 14:15:44 -0500 (Wed, 08 Aug 2007) | 2 lines
Fix documentation typo in a construction variable cross-reference.
........
r2234 | stevenknight | 2007-08-08 17:03:25 -0500 (Wed, 08 Aug 2007) | 2 lines
Changes to SCons packaging to support checkpoint releases.
........
r2235 | stevenknight | 2007-08-09 10:10:01 -0500 (Thu, 09 Aug 2007) | 2 lines
Sidestep false negatives on heavily loaded systems.
........
r2236 | garyo | 2007-08-09 11:16:26 -0500 (Thu, 09 Aug 2007) | 1 line
Allow unpackaged files (e.g. *.pyo) to exist in the build dir without being packaged in the RPM. Without this, on some systems the rpmbuild may error out.
........
r2237 | stevenknight | 2007-08-09 11:27:56 -0500 (Thu, 09 Aug 2007) | 5 lines
Fix test/SWIG/build-dir.py so it works on old Python versions without
distutils.sysconfig.
Instead of just cutting-and-pasting initialization code from other
SWIG tests, centralize it in some new TestSCons methods.
........
r2238 | garyo | 2007-08-09 11:30:58 -0500 (Thu, 09 Aug 2007) | 1 line
Use docbook 4.3 instead of 4.4 for the XML doctype since some older(?) jade parsers can't handle new 4-byte Unicode chars in the 4.4 version of isogrk4.ent.
........
r2240 | stevenknight | 2007-08-09 16:35:06 -0500 (Thu, 09 Aug 2007) | 2 lines
User's Guide updates (post packaging changes).
........
r2243 | stevenknight | 2007-08-10 10:31:51 -0500 (Fri, 10 Aug 2007) | 3 lines
Fix the User's Guide build to use openjade, and to accomodate a change
in the name of the main generated file (book1.html => index.html).
........
r2245 | stevenknight | 2007-08-10 11:09:16 -0500 (Fri, 10 Aug 2007) | 2 lines
Update the {CHANGES,RELEASE}.txt datestamp lines.
........
r2253 | stevenknight | 2007-08-10 16:21:54 -0500 (Fri, 10 Aug 2007) | 2 lines
Fix the wix Tool module's ability to handle null entries in $PATH.
........
r2261 | stevenknight | 2007-08-11 23:08:12 -0500 (Sat, 11 Aug 2007) | 3 lines
Remove unnecessary files (.svnt/*, .{ae,cvs}ignore, www/*) from the
scons-src packages.
........
r2262 | stevenknight | 2007-08-11 23:24:49 -0500 (Sat, 11 Aug 2007) | 2 lines
Add missing __revision__ lines.
........
r2263 | stevenknight | 2007-08-11 23:33:42 -0500 (Sat, 11 Aug 2007) | 2 lines
Skip the test if the MANIFEST file hasn't been built.
........
r2264 | stevenknight | 2007-08-11 23:36:30 -0500 (Sat, 11 Aug 2007) | 2 lines
Add recent compatibility modules to the relevant exceptions lists.
........
r2265 | stevenknight | 2007-08-11 23:39:00 -0500 (Sat, 11 Aug 2007) | 3 lines
Update __VERSION__ strings in the QMTest/*.py modules, so that packaging
tests (src/test_*.py) will pass after builds of checkpoint releases.
........
r2266 | stevenknight | 2007-08-12 07:36:19 -0500 (Sun, 12 Aug 2007) | 2 lines
Add a comment about why we construct the __VERSION__ string at run time.
........
r2267 | stevenknight | 2007-08-12 07:42:30 -0500 (Sun, 12 Aug 2007) | 2 lines
Avoid reading the MANIFEST file twice. (Courtesy review by Greg Noel.)
........
r2268 | stevenknight | 2007-08-12 08:14:53 -0500 (Sun, 12 Aug 2007) | 3 lines
Shift Install() and InstallAs() from being documented as functions
to being documented as Builders.
........
r2269 | garyo | 2007-08-13 08:49:52 -0500 (Mon, 13 Aug 2007) | 1 line
Tests: Skip some more Java tests if javac is not installed on the test machine so they don't get marked as failing.
........
r2270 | garyo | 2007-08-13 11:09:39 -0500 (Mon, 13 Aug 2007) | 1 line
Fixed typo in test (shows up on non-Linux platforms).
........
r2271 | garyo | 2007-08-13 14:09:05 -0500 (Mon, 13 Aug 2007) | 4 lines
Test portability fixes for Darwin/OSX and IRIX.
This does not make all the tests pass on those OSes,
but it takes care of some of the more obvious errors that
I have time for right now. More to come.
........
r2272 | stevenknight | 2007-08-13 15:33:29 -0500 (Mon, 13 Aug 2007) | 2 lines
Tab => space fix.
........
r2273 | stevenknight | 2007-08-13 15:33:52 -0500 (Mon, 13 Aug 2007) | 2 lines
Test for swig, too, which is used to build from the .i file.
........
r2277 | garyo | 2007-08-14 10:40:00 -0500 (Tue, 14 Aug 2007) | 8 lines
Test portability on IRIX: test/Actions/pre-post creates target file
before building target, then IRIX CC does not chmod +x afterwards.
I think this change is safe on all OSes.
test/AS/ml.py: I think this is only supposed to be run on win32
(not skipped only on win32); the sense of the skip test was backwards.
........
r2278 | stevenknight | 2007-08-14 11:04:40 -0500 (Tue, 14 Aug 2007) | 2 lines
Add -tt when running tests, to catch inconsistent tab usage.
........
r2279 | stevenknight | 2007-08-14 14:00:43 -0500 (Tue, 14 Aug 2007) | 2 lines
Minor refactor of logic in File.retrieve_from_cache().
........
r2280 | stevenknight | 2007-08-15 01:11:40 -0500 (Wed, 15 Aug 2007) | 2 lines
Refactor CacheDir support into its own module.
........
r2281 | stevenknight | 2007-08-15 07:24:51 -0500 (Wed, 15 Aug 2007) | 2 lines
Move the cachepath() method from FS.File to the CacheDir class.
........
r2282 | stevenknight | 2007-08-15 08:31:34 -0500 (Wed, 15 Aug 2007) | 2 lines
Python 1.5.2 fix in the new Null class.
........
r2283 | stevenknight | 2007-08-15 10:45:53 -0500 (Wed, 15 Aug 2007) | 5 lines
Refactor CacheDir unit tests to:
- restore functionality that was dropped in the transition;
- commonize creation of test Nodes and other (mock) objects
- separate CacheDir tests from tests of CacheDir through Node.FS.File.
........
r2284 | stevenknight | 2007-08-15 11:46:38 -0500 (Wed, 15 Aug 2007) | 3 lines
Replace the Executor.Null.NullEnvironment object with a real Null object,
so it will absorb the CacheDir method calls as well.
........
r2285 | stevenknight | 2007-08-15 11:52:57 -0500 (Wed, 15 Aug 2007) | 5 lines
Add a get_CacheDir() method to a construction environment, which will
be used to fetch per-environment CacheDir specifications. (Right now
all calls to it still just return the one attached to underlying default
FS object.)
........
r2286 | stevenknight | 2007-08-15 15:15:46 -0500 (Wed, 15 Aug 2007) | 2 lines
Support per-construction-environment configuration of CacheDir().
........
r2287 | stevenknight | 2007-08-15 15:33:04 -0500 (Wed, 15 Aug 2007) | 2 lines
Move the tests of CacheDir()-related command-line options into test/CacheDir.
........
r2293 | stevenknight | 2007-08-16 11:14:49 -0500 (Thu, 16 Aug 2007) | 3 lines
Add the Package() builder description to the documentation build,
fixing the XML so that it will build.
........
r2294 | stevenknight | 2007-08-16 12:51:19 -0500 (Thu, 16 Aug 2007) | 3 lines
Reorganize packaging documentation: alphabetize the variable definitions
(and function names), document Tag() as a function, not a builder.
........
r2296 | stevenknight | 2007-08-16 12:55:01 -0500 (Thu, 16 Aug 2007) | 2 lines
Add a build command.
........
r2300 | stevenknight | 2007-08-16 16:49:13 -0500 (Thu, 16 Aug 2007) | 2 lines
First cut at documenting packaging variables.
........
r2301 | stevenknight | 2007-08-16 16:51:21 -0500 (Thu, 16 Aug 2007) | 3 lines
Construct the .src.rpm and .arch.rpm file names independnetly, not
by trying to massage one into the other.
........
Diffstat (limited to 'src/engine/SCons')
41 files changed, 1788 insertions, 767 deletions
diff --git a/src/engine/SCons/ActionTests.py b/src/engine/SCons/ActionTests.py index 87fb4b0..01d0992 100644 --- a/src/engine/SCons/ActionTests.py +++ b/src/engine/SCons/ActionTests.py @@ -156,7 +156,7 @@ class Environment: return self.d def Clone(self, **kw): res = Environment() - res.d = SCons.Environment.our_deepcopy(self.d) + res.d = SCons.Util.semi_deepcopy(self.d) for k, v in kw.items(): res.d[k] = v return res diff --git a/src/engine/SCons/Builder.py b/src/engine/SCons/Builder.py index d21de46..ff31c5e 100644 --- a/src/engine/SCons/Builder.py +++ b/src/engine/SCons/Builder.py @@ -467,28 +467,28 @@ class BuilderBase: executor.add_sources(slist) return executor + def _adjustixes(self, files, pre, suf): + if not files: + return [] + result = [] + if not SCons.Util.is_List(files): + files = [files] + + for f in files: + if SCons.Util.is_String(f): + f = SCons.Util.adjustixes(f, pre, suf) + result.append(f) + return result + def _create_nodes(self, env, target = None, source = None): """Create and return lists of target and source nodes. """ - def _adjustixes(files, pre, suf): - if not files: - return [] - result = [] - if not SCons.Util.is_List(files): - files = [files] - - for f in files: - if SCons.Util.is_String(f): - f = SCons.Util.adjustixes(f, pre, suf) - result.append(f) - return result - src_suf = self.get_src_suffix(env) target_factory = env.get_factory(self.target_factory) source_factory = env.get_factory(self.source_factory) - source = _adjustixes(source, None, src_suf) + source = self._adjustixes(source, None, src_suf) slist = env.arg2nodes(source, source_factory) pre = self.get_prefix(env, slist) @@ -505,7 +505,7 @@ class BuilderBase: splitext = lambda S,self=self,env=env: self.splitext(S,env) tlist = [ t_from_s(pre, suf, splitext) ] else: - target = _adjustixes(target, pre, suf) + target = self._adjustixes(target, pre, suf) tlist = env.arg2nodes(target, target_factory) if self.emitter: @@ -558,7 +558,7 @@ class BuilderBase: return result overwarn.warn() - + tlist, slist = self._create_nodes(env, target, source) # Check for errors with the specified target/source lists. @@ -716,20 +716,14 @@ class BuilderBase: return sdict def src_builder_sources(self, env, source, overwarn={}): - source_factory = env.get_factory(self.source_factory) - slist = env.arg2nodes(source, source_factory) - sdict = self._get_sdict(env) src_suffixes = self.src_suffixes(env) - lengths_dict = {} - for l in map(len, src_suffixes): - lengths_dict[l] = None - lengths = lengths_dict.keys() + lengths = list(set(map(len, src_suffixes))) - def match_src_suffix(node, src_suffixes=src_suffixes, lengths=lengths): - node_suffixes = map(lambda l, n=node: n.name[-l:], lengths) + def match_src_suffix(name, src_suffixes=src_suffixes, lengths=lengths): + node_suffixes = map(lambda l, n=name: n[-l:], lengths) for suf in src_suffixes: if suf in node_suffixes: return suf @@ -737,25 +731,38 @@ class BuilderBase: result = [] - for snode in slist: - match_suffix = match_src_suffix(snode) + if SCons.Util.is_List(source): + source = SCons.Util.flatten(source) + else: + source = [source] + for s in source: + if SCons.Util.is_String(s): + match_suffix = match_src_suffix(s) + if not match_suffix and not '.' in s: + src_suf = self.get_src_suffix(env) + s = self._adjustixes(s, None, src_suf)[0] + else: + match_suffix = match_src_suffix(s.name) if match_suffix: try: bld = sdict[match_suffix] except KeyError: - result.append(snode) + result.append(s) else: - tlist = bld._execute(env, None, [snode], overwarn) + tlist = bld._execute(env, None, [s], overwarn) # If the subsidiary Builder returned more than one # target, then filter out any sources that this # Builder isn't capable of building. if len(tlist) > 1: - tlist = filter(match_src_suffix, tlist) + mss = lambda t, m=match_src_suffix: m(t.name) + tlist = filter(mss, tlist) result.extend(tlist) else: - result.append(snode) + result.append(s) - return result + source_factory = env.get_factory(self.source_factory) + + return env.arg2nodes(result, source_factory) def _get_src_builders_key(self, env): return id(env) diff --git a/src/engine/SCons/BuilderTests.py b/src/engine/SCons/BuilderTests.py index 3996d5c..bc4c52d 100644 --- a/src/engine/SCons/BuilderTests.py +++ b/src/engine/SCons/BuilderTests.py @@ -43,6 +43,8 @@ import SCons.Action import SCons.Builder import SCons.Environment import SCons.Errors +import SCons.Subst +import SCons.Util sys.stdout = StringIO.StringIO() @@ -92,12 +94,11 @@ class Environment: return self.d.get(s, s) def subst_target_source(self, string, raw=0, target=None, source=None, dict=None, conv=None): - return SCons.Util.scons_subst(string, self, raw, target, - source, dict, conv) - def subst_list(self, string, raw=0, target=None, - source=None, dict=None, conv=None): - return SCons.Util.scons_subst_list(string, self, raw, target, - source, dict, conv) + return SCons.Subst.scons_subst(string, self, raw, target, + source, dict, conv) + def subst_list(self, string, raw=0, target=None, source=None, conv=None): + return SCons.Subst.scons_subst_list(string, self, raw, target, + source, {}, {}, conv) def arg2nodes(self, args, factory, **kw): global env_arg2nodes_called env_arg2nodes_called = 1 @@ -792,12 +793,12 @@ class BuilderTestCase(unittest.TestCase): tgt = builder2(env, source=[]) assert tgt == [], tgt - tgt = builder2(env, target='baz', - source=['test.bar', 'test2.foo', 'test3.txt'])[0] + sources = ['test.bar', 'test2.foo', 'test3.txt', 'test4'] + tgt = builder2(env, target='baz', source=sources)[0] s = str(tgt) assert s == 'baz', s s = map(str, tgt.sources) - assert s == ['test.foo', 'test2.foo', 'test3.txt'], s + assert s == ['test.foo', 'test2.foo', 'test3.txt', 'test4.foo'], s s = map(str, tgt.sources[0].sources) assert s == ['test.bar'], s @@ -899,7 +900,8 @@ class BuilderTestCase(unittest.TestCase): assert tgt.builder.source_scanner is None, tgt.builder.source_scanner assert tgt.get_source_scanner(bar_y) is None, tgt.get_source_scanner(bar_y) assert not src.has_builder(), src.has_builder() - assert src.get_source_scanner(bar_y) is None, src.get_source_scanner(bar_y) + s = src.get_source_scanner(bar_y) + assert isinstance(s, SCons.Util.Null), repr(s) # An Environment that has suffix-specified SCANNERS should # provide a source scanner to the target. @@ -926,7 +928,8 @@ class BuilderTestCase(unittest.TestCase): assert tgt.get_source_scanner(bar_y), tgt.get_source_scanner(bar_y) assert str(tgt.get_source_scanner(bar_y)) == 'EnvTestScanner', tgt.get_source_scanner(bar_y) assert not src.has_builder(), src.has_builder() - assert src.get_source_scanner(bar_y) is None, src.get_source_scanner(bar_y) + s = src.get_source_scanner(bar_y) + assert isinstance(s, SCons.Util.Null), repr(s) # Can't simply specify the scanner as a builder argument; it's # global to all invocations of this builder. @@ -937,7 +940,8 @@ class BuilderTestCase(unittest.TestCase): assert tgt.get_source_scanner(bar_y), tgt.get_source_scanner(bar_y) assert str(tgt.get_source_scanner(bar_y)) == 'EnvTestScanner', tgt.get_source_scanner(bar_y) assert not src.has_builder(), src.has_builder() - assert src.get_source_scanner(bar_y) is None, src.get_source_scanner(bar_y) + s = src.get_source_scanner(bar_y) + assert isinstance(s, SCons.Util.Null), s # Now use a builder that actually has scanners and ensure that # the target is set accordingly (using the specified scanner @@ -955,7 +959,8 @@ class BuilderTestCase(unittest.TestCase): assert tgt.get_source_scanner(bar_y) == scanner, tgt.get_source_scanner(bar_y) assert str(tgt.get_source_scanner(bar_y)) == 'TestScanner', tgt.get_source_scanner(bar_y) assert not src.has_builder(), src.has_builder() - assert src.get_source_scanner(bar_y) is None, src.get_source_scanner(bar_y) + s = src.get_source_scanner(bar_y) + assert isinstance(s, SCons.Util.Null), s @@ -1595,7 +1600,7 @@ class CompositeBuilderTestCase(unittest.TestCase): assert str(e) == expect, e flag = 0 - tgt = builder(env, target='t7', source=['test7'])[0] + tgt = builder(env, target='t7', source=[env.fs.File('test7')])[0] try: tgt.build() except SCons.Errors.UserError, e: diff --git a/src/engine/SCons/CacheDir.py b/src/engine/SCons/CacheDir.py new file mode 100644 index 0000000..6a1cc04 --- /dev/null +++ b/src/engine/SCons/CacheDir.py @@ -0,0 +1,207 @@ +# +# __COPYRIGHT__ +# +# Permission is hereby granted, free of charge, to any person obtaining +# a copy of this software and associated documentation files (the +# "Software"), to deal in the Software without restriction, including +# without limitation the rights to use, copy, modify, merge, publish, +# distribute, sublicense, and/or sell copies of the Software, and to +# permit persons to whom the Software is furnished to do so, subject to +# the following conditions: +# +# The above copyright notice and this permission notice shall be included +# in all copies or substantial portions of the Software. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY +# KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE +# WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND +# NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE +# LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION +# OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION +# WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. +# + +__revision__ = "__FILE__ __REVISION__ __DATE__ __DEVELOPER__" + +__doc__ = """ +CacheDir support +""" + +import os.path +import stat +import string +import sys + +import SCons.Action + +cache_debug = False +cache_force = False +cache_show = False + +def CacheRetrieveFunc(target, source, env): + t = target[0] + fs = t.fs + cd = env.get_CacheDir() + cachedir, cachefile = cd.cachepath(t) + if not fs.exists(cachefile): + cd.CacheDebug('CacheRetrieve(%s): %s not in cache\n', t, cachefile) + return 1 + cd.CacheDebug('CacheRetrieve(%s): retrieving from %s\n', t, cachefile) + if SCons.Action.execute_actions: + if fs.islink(cachefile): + fs.symlink(fs.readlink(cachefile), t.path) + else: + fs.copy2(cachefile, t.path) + st = fs.stat(cachefile) + fs.chmod(t.path, stat.S_IMODE(st[stat.ST_MODE]) | stat.S_IWRITE) + return 0 + +def CacheRetrieveString(target, source, env): + t = target[0] + fs = t.fs + cd = env.get_CacheDir() + cachedir, cachefile = cd.cachepath(t) + if t.fs.exists(cachefile): + return "Retrieved `%s' from cache" % t.path + return None + +CacheRetrieve = SCons.Action.Action(CacheRetrieveFunc, CacheRetrieveString) + +CacheRetrieveSilent = SCons.Action.Action(CacheRetrieveFunc, None) + +def CachePushFunc(target, source, env): + t = target[0] + if t.nocache: + return + fs = t.fs + cd = env.get_CacheDir() + cachedir, cachefile = cd.cachepath(t) + if fs.exists(cachefile): + # Don't bother copying it if it's already there. Note that + # usually this "shouldn't happen" because if the file already + # existed in cache, we'd have retrieved the file from there, + # not built it. This can happen, though, in a race, if some + # other person running the same build pushes their copy to + # the cache after we decide we need to build it but before our + # build completes. + cd.CacheDebug('CachePush(%s): %s already exists in cache\n', t, cachefile) + return + + cd.CacheDebug('CachePush(%s): pushing to %s\n', t, cachefile) + + if not fs.isdir(cachedir): + fs.makedirs(cachedir) + + tempfile = cachefile+'.tmp' + try: + if fs.islink(t.path): + fs.symlink(fs.readlink(t.path), tempfile) + else: + fs.copy2(t.path, tempfile) + fs.rename(tempfile, cachefile) + st = fs.stat(t.path) + fs.chmod(cachefile, stat.S_IMODE(st[stat.ST_MODE]) | stat.S_IWRITE) + except (IOError, OSError): + # It's possible someone else tried writing the file at the + # same time we did, or else that there was some problem like + # the CacheDir being on a separate file system that's full. + # In any case, inability to push a file to cache doesn't affect + # the correctness of the build, so just print a warning. + SCons.Warnings.warn(SCons.Warnings.CacheWriteErrorWarning, + "Unable to copy %s to cache. Cache file is %s" + % (str(target), cachefile)) + +CachePush = SCons.Action.Action(CachePushFunc, None) + +class CacheDir: + + def __init__(self, path): + try: + import SCons.Sig.MD5 + except ImportError: + msg = "No MD5 module available, CacheDir() not supported" + SCons.Warnings.warn(SCons.Warnings.NoMD5ModuleWarning, msg) + else: + self.path = path + + def CacheDebugWrite(self, fmt, target, cachefile): + self.debugFP.write(fmt % (target, os.path.split(cachefile)[1])) + + def CacheDebugQuiet(self, fmt, target, cachefile): + pass + + def CacheDebugInit(self, fmt, target, cachefile): + if cache_debug: + if cache_debug == '-': + self.debugFP = sys.stdout + else: + self.debugFP = open(cache_debug, 'w') + self.CacheDebug = self.CacheDebugWrite + self.CacheDebug(fmt, target, cachefile) + else: + self.CacheDebug = self.CacheDebugQuiet + + CacheDebug = CacheDebugInit + + def cachepath(self, node): + """ + """ + sig = node.get_cachedir_bsig() + subdir = string.upper(sig[0]) + dir = os.path.join(self.path, subdir) + return dir, os.path.join(dir, sig) + + def retrieve(self, node): + """ + This method is called from multiple threads in a parallel build, + so only do thread safe stuff here. Do thread unsafe stuff in + built(). + + Note that there's a special trick here with the execute flag + (one that's not normally done for other actions). Basically + if the user requested a no_exec (-n) build, then + SCons.Action.execute_actions is set to 0 and when any action + is called, it does its showing but then just returns zero + instead of actually calling the action execution operation. + The problem for caching is that if the file does NOT exist in + cache then the CacheRetrieveString won't return anything to + show for the task, but the Action.__call__ won't call + CacheRetrieveFunc; instead it just returns zero, which makes + the code below think that the file *was* successfully + retrieved from the cache, therefore it doesn't do any + subsequent building. However, the CacheRetrieveString didn't + print anything because it didn't actually exist in the cache, + and no more build actions will be performed, so the user just + sees nothing. The fix is to tell Action.__call__ to always + execute the CacheRetrieveFunc and then have the latter + explicitly check SCons.Action.execute_actions itself. + """ + retrieved = False + + if cache_show: + if CacheRetrieveSilent(node, [], node.get_build_env(), execute=1) == 0: + node.build(presub=0, execute=0) + retrieved = 1 + else: + if CacheRetrieve(node, [], node.get_build_env(), execute=1) == 0: + retrieved = 1 + if retrieved: + # Record build signature information, but don't + # push it out to cache. (We just got it from there!) + node.set_state(SCons.Node.executed) + SCons.Node.Node.built(node) + + return retrieved + + def push(self, node): + return CachePush(node, [], node.get_build_env()) + + def push_if_forced(self, node): + if cache_force: + return self.push(node) + +class Null(SCons.Util.Null): + def repr(self): + return 'CacheDir.Null()' + def retrieve(self, node): + return False diff --git a/src/engine/SCons/CacheDirTests.py b/src/engine/SCons/CacheDirTests.py new file mode 100644 index 0000000..564b762 --- /dev/null +++ b/src/engine/SCons/CacheDirTests.py @@ -0,0 +1,295 @@ +# +# __COPYRIGHT__ +# +# Permission is hereby granted, free of charge, to any person obtaining +# a copy of this software and associated documentation files (the +# "Software"), to deal in the Software without restriction, including +# without limitation the rights to use, copy, modify, merge, publish, +# distribute, sublicense, and/or sell copies of the Software, and to +# permit persons to whom the Software is furnished to do so, subject to +# the following conditions: +# +# The above copyright notice and this permission notice shall be included +# in all copies or substantial portions of the Software. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY +# KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE +# WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND +# NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE +# LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION +# OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION +# WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. +# + +__revision__ = "__FILE__ __REVISION__ __DATE__ __DEVELOPER__" + +import os.path +import shutil +import sys +import unittest + +from TestCmd import TestCmd + +import SCons.CacheDir + +built_it = None + +class Action: + def __call__(self, targets, sources, env, errfunc, **kw): + global built_it + if kw.get('execute', 1): + built_it = 1 + return 0 + +class Builder: + def __init__(self, environment, action): + self.env = environment + self.action = action + self.overrides = {} + +class Environment: + def __init__(self, cachedir): + self.cachedir = cachedir + def Override(self, overrides): + return self + def get_CacheDir(self): + return self.cachedir + +class BaseTestCase(unittest.TestCase): + """ + Base fixtures common to our other unittest classes. + """ + def setUp(self): + self.test = TestCmd(workdir='') + + import SCons.Node.FS + self.fs = SCons.Node.FS.FS() + + self._CacheDir = SCons.CacheDir.CacheDir('cache') + + def File(self, name, bsig=None, action=Action()): + node = self.fs.File(name) + node.builder_set(Builder(Environment(self._CacheDir), action)) + if bsig: + node.binfo = node.BuildInfo(node) + node.binfo.ninfo.bsig = bsig + return node + +class CacheDirTestCase(BaseTestCase): + """ + Test calling CacheDir code directly. + """ + def test_cachepath(self): + """Test the cachepath() method""" + + # Verify how the cachepath() method determines the name + # of the file in cache. + def my_collect(list): + return list[0] + save_collect = SCons.Sig.MD5.collect + SCons.Sig.MD5.collect = my_collect + + try: + f5 = self.File("cd.f5", 'a_fake_bsig') + result = self._CacheDir.cachepath(f5) + dirname = os.path.join('cache', 'A') + filename = os.path.join(dirname, 'a_fake_bsig') + assert result == (dirname, filename), result + finally: + SCons.Sig.MD5.collect = save_collect + +class FileTestCase(BaseTestCase): + """ + Test calling CacheDir code through Node.FS.File interfaces. + """ + # These tests were originally in Nodes/FSTests.py and got moved + # when the CacheDir support was refactored into its own module. + # Look in the history for Node/FSTests.py if any of this needs + # to be re-examined. + def retrieve_succeed(self, target, source, env, execute=1): + self.retrieved.append(target) + return 0 + + def retrieve_fail(self, target, source, env, execute=1): + self.retrieved.append(target) + return 1 + + def push(self, target, source, env): + self.pushed.append(target) + return 0 + + def test_CacheRetrieve(self): + """Test the CacheRetrieve() function""" + + save_CacheRetrieve = SCons.CacheDir.CacheRetrieve + self.retrieved = [] + + f1 = self.File("cd.f1") + try: + SCons.CacheDir.CacheRetrieve = self.retrieve_succeed + self.retrieved = [] + built_it = None + + r = f1.retrieve_from_cache() + assert r == 1, r + assert self.retrieved == [f1], self.retrieved + assert built_it is None, built_it + + SCons.CacheDir.CacheRetrieve = self.retrieve_fail + self.retrieved = [] + built_it = None + + r = f1.retrieve_from_cache() + assert not r, r + assert self.retrieved == [f1], self.retrieved + assert built_it is None, built_it + finally: + SCons.CacheDir.CacheRetrieve = save_CacheRetrieve + + def test_CacheRetrieveSilent(self): + """Test the CacheRetrieveSilent() function""" + + save_CacheRetrieveSilent = SCons.CacheDir.CacheRetrieveSilent + + SCons.CacheDir.cache_show = 1 + + f2 = self.File("cd.f2", 'f2_bsig') + try: + SCons.CacheDir.CacheRetrieveSilent = self.retrieve_succeed + self.retrieved = [] + built_it = None + + r = f2.retrieve_from_cache() + assert r == 1, r + assert self.retrieved == [f2], self.retrieved + assert built_it is None, built_it + + SCons.CacheDir.CacheRetrieveSilent = self.retrieve_fail + self.retrieved = [] + built_it = None + + r = f2.retrieve_from_cache() + assert r is False, r + assert self.retrieved == [f2], self.retrieved + assert built_it is None, built_it + finally: + SCons.CacheDir.CacheRetrieveSilent = save_CacheRetrieveSilent + + def test_CachePush(self): + """Test the CachePush() function""" + + save_CachePush = SCons.CacheDir.CachePush + + SCons.CacheDir.CachePush = self.push + + try: + self.pushed = [] + + cd_f3 = self.test.workpath("cd.f3") + f3 = self.File(cd_f3) + f3.built() + assert self.pushed == [], self.pushed + self.test.write(cd_f3, "cd.f3\n") + f3.built() + assert self.pushed == [f3], self.pushed + + self.pushed = [] + + cd_f4 = self.test.workpath("cd.f4") + f4 = self.File(cd_f4) + f4.visited() + assert self.pushed == [], self.pushed + self.test.write(cd_f4, "cd.f4\n") + f4.clear() + f4.visited() + assert self.pushed == [], self.pushed + SCons.CacheDir.cache_force = 1 + f4.clear() + f4.visited() + assert self.pushed == [f4], self.pushed + finally: + SCons.CacheDir.CachePush = save_CachePush + + def test_no_bsig(self): + """Test that no bsig raises an InternalError""" + + f6 = self.File("cd.f6") + f6.binfo = f6.BuildInfo(f6) + exc_caught = 0 + try: + cp = self._CacheDir.cachepath(f6) + except SCons.Errors.InternalError: + exc_caught = 1 + assert exc_caught + + def test_warning(self): + """Test raising a warning if we can't copy a file to cache.""" + + test = TestCmd(workdir='') + + save_copy2 = shutil.copy2 + def copy2(src, dst): + raise OSError + shutil.copy2 = copy2 + save_mkdir = os.mkdir + def mkdir(dir, mode=0): + pass + os.mkdir = mkdir + old_warn_exceptions = SCons.Warnings.warningAsException(1) + SCons.Warnings.enableWarningClass(SCons.Warnings.CacheWriteErrorWarning) + + try: + cd_f7 = self.test.workpath("cd.f7") + self.test.write(cd_f7, "cd.f7\n") + f7 = self.File(cd_f7, 'f7_bsig') + + warn_caught = 0 + try: + f7.built() + except SCons.Warnings.CacheWriteErrorWarning: + warn_caught = 1 + assert warn_caught + finally: + shutil.copy2 = save_copy2 + os.mkdir = save_mkdir + SCons.Warnings.warningAsException(old_warn_exceptions) + SCons.Warnings.suppressWarningClass(SCons.Warnings.CacheWriteErrorWarning) + + def test_no_strfunction(self): + """Test handling no strfunction() for an action.""" + + save_CacheRetrieveSilent = SCons.CacheDir.CacheRetrieveSilent + + f8 = self.File("cd.f8", 'f8_bsig') + try: + SCons.CacheDir.CacheRetrieveSilent = self.retrieve_succeed + self.retrieved = [] + built_it = None + + r = f8.retrieve_from_cache() + assert r == 1, r + assert self.retrieved == [f8], self.retrieved + assert built_it is None, built_it + + SCons.CacheDir.CacheRetrieveSilent = self.retrieve_fail + self.retrieved = [] + built_it = None + + r = f8.retrieve_from_cache() + assert r is False, r + assert self.retrieved == [f8], self.retrieved + assert built_it is None, built_it + finally: + SCons.CacheDir.CacheRetrieveSilent = save_CacheRetrieveSilent + +if __name__ == "__main__": + suite = unittest.TestSuite() + tclasses = [ + CacheDirTestCase, + FileTestCase, + ] + for tclass in tclasses: + names = unittest.getTestCaseNames(tclass, 'test_') + suite.addTests(map(tclass, names)) + if not unittest.TextTestRunner().run(suite).wasSuccessful(): + sys.exit(1) diff --git a/src/engine/SCons/Defaults.py b/src/engine/SCons/Defaults.py index 9d0ad82..29868ec 100644 --- a/src/engine/SCons/Defaults.py +++ b/src/engine/SCons/Defaults.py @@ -46,6 +46,7 @@ import sys import SCons.Action import SCons.Builder +import SCons.CacheDir import SCons.Environment import SCons.PathList import SCons.Sig @@ -66,6 +67,7 @@ def DefaultEnvironment(*args, **kw): _default_env = apply(SCons.Environment.Environment, args, kw) _default_env._build_signature = 1 _default_env._calc_module = SCons.Sig.default_module + _default_env._CacheDir = SCons.CacheDir.Null() return _default_env # Emitters for setting the shared attribute on object files, diff --git a/src/engine/SCons/Environment.py b/src/engine/SCons/Environment.py index 8a7721e..012c36c 100644 --- a/src/engine/SCons/Environment.py +++ b/src/engine/SCons/Environment.py @@ -67,6 +67,8 @@ _null = _Null CleanTargets = {} CalculatorArgs = {} +semi_deepcopy = SCons.Util.semi_deepcopy + # Pull UserError into the global name space for the benefit of # Environment().SourceSignatures(), which has some import statements # which seem to mess up its ability to reference SCons directly. @@ -82,23 +84,6 @@ AliasBuilder = SCons.Builder.Builder(action = alias_builder, is_explicit = None, name='AliasBuilder') -def our_deepcopy(x): - """deepcopy lists and dictionaries, and just copy the reference - for everything else.""" - if SCons.Util.is_Dict(x): - copy = {} - for key in x.keys(): - copy[key] = our_deepcopy(x[key]) - elif SCons.Util.is_List(x): - copy = map(our_deepcopy, x) - try: - copy = x.__class__(copy) - except AttributeError: - pass - else: - copy = x - return copy - def apply_tools(env, tools, toolpath): # Store the toolpath in the Environment. if toolpath is not None: @@ -122,7 +107,7 @@ reserved_construction_var_names = \ ['TARGET', 'TARGETS', 'SOURCE', 'SOURCES'] def copy_non_reserved_keywords(dict): - result = our_deepcopy(dict) + result = semi_deepcopy(dict) for k in result.keys(): if k in reserved_construction_var_names: SCons.Warnings.warn(SCons.Warnings.ReservedVariableWarning, @@ -191,6 +176,9 @@ class BuilderDict(UserDict): self.env = env UserDict.__init__(self, dict) + def __semi_deepcopy__(self): + return self.__class__(self.data, self.env) + def __setitem__(self, item, val): UserDict.__setitem__(self, item, val) try: @@ -255,7 +243,7 @@ class SubstitutionEnvironment: """Initialization of an underlying SubstitutionEnvironment class. """ if __debug__: logInstanceCreation(self, 'Environment.SubstitutionEnvironment') - self.fs = SCons.Node.FS.default_fs or SCons.Node.FS.FS() + self.fs = SCons.Node.FS.get_default_fs() self.ans = SCons.Node.Alias.default_ans self.lookup_list = SCons.Node.arg2nodes_lookups self._dict = kw.copy() @@ -754,10 +742,10 @@ class Base(SubstitutionEnvironment): """ if __debug__: logInstanceCreation(self, 'Environment.Base') self._memo = {} - self.fs = SCons.Node.FS.default_fs or SCons.Node.FS.FS() + self.fs = SCons.Node.FS.get_default_fs() self.ans = SCons.Node.Alias.default_ans self.lookup_list = SCons.Node.arg2nodes_lookups - self._dict = our_deepcopy(SCons.Defaults.ConstructionEnvironment) + self._dict = semi_deepcopy(SCons.Defaults.ConstructionEnvironment) self._init_special() self._dict['BUILDERS'] = BuilderDict(self._dict['BUILDERS'], self) @@ -825,6 +813,14 @@ class Base(SubstitutionEnvironment): c = SCons.Defaults.DefaultEnvironment().get_calculator() return c + def get_CacheDir(self): + try: + return self._CacheDir + except AttributeError: + cd = SCons.Defaults.DefaultEnvironment()._CacheDir + self._CacheDir = cd + return cd + def get_factory(self, factory, default='File'): """Return a factory function for creating Nodes for this construction environment. @@ -980,7 +976,11 @@ class Base(SubstitutionEnvironment): try: update_dict(val) except (AttributeError, TypeError, ValueError): - orig[val] = None + if SCons.Util.is_Dict(val): + for k, v in val.items(): + orig[k] = v + else: + orig[val] = None self.scanner_map_delete(kw) def AppendENVPath(self, name, newpath, envname = 'ENV', sep = os.pathsep): @@ -1039,7 +1039,7 @@ class Base(SubstitutionEnvironment): objects in the original Environment. """ clone = copy.copy(self) - clone._dict = our_deepcopy(self._dict) + clone._dict = semi_deepcopy(self._dict) try: cbd = clone._dict['BUILDERS'] clone._dict['BUILDERS'] = BuilderDict(cbd, clone) @@ -1235,7 +1235,11 @@ class Base(SubstitutionEnvironment): try: update_dict(val) except (AttributeError, TypeError, ValueError): - orig[val] = None + if SCons.Util.is_Dict(val): + for k, v in val.items(): + orig[k] = v + else: + orig[val] = None self.scanner_map_delete(kw) def PrependENVPath(self, name, newpath, envname = 'ENV', sep = os.pathsep): @@ -1290,13 +1294,13 @@ class Base(SubstitutionEnvironment): with new construction variables and/or values. """ try: - kwbd = our_deepcopy(kw['BUILDERS']) + kwbd = semi_deepcopy(kw['BUILDERS']) del kw['BUILDERS'] self.__setitem__('BUILDERS', kwbd) except KeyError: pass kw = copy_non_reserved_keywords(kw) - self._update(our_deepcopy(kw)) + self._update(semi_deepcopy(kw)) self.scanner_map_delete(kw) def ReplaceIxes(self, path, old_prefix, old_suffix, new_prefix, new_suffix): @@ -1470,7 +1474,11 @@ class Base(SubstitutionEnvironment): return apply(SCons.Builder.Builder, [], nkw) def CacheDir(self, path): - self.fs.CacheDir(self.subst(path)) + import SCons.CacheDir + if path is None: + self._CacheDir = SCons.CacheDir.Null() + else: + self._CacheDir = SCons.CacheDir.CacheDir(self.subst(path)) def Clean(self, targets, files): global CleanTargets @@ -1830,7 +1838,7 @@ class OverrideEnvironment(Base): # Overridden public construction environment methods. def Replace(self, **kw): kw = copy_non_reserved_keywords(kw) - self.__dict__['overrides'].update(our_deepcopy(kw)) + self.__dict__['overrides'].update(semi_deepcopy(kw)) # The entry point that will be used by the external world # to refer to a construction environment. This allows the wrapper diff --git a/src/engine/SCons/EnvironmentTests.py b/src/engine/SCons/EnvironmentTests.py index 20c1eac..82f220a 100644 --- a/src/engine/SCons/EnvironmentTests.py +++ b/src/engine/SCons/EnvironmentTests.py @@ -2507,18 +2507,13 @@ def generate(env): def test_CacheDir(self): """Test the CacheDir() method""" - class MyFS: - def CacheDir(self, path): - self.CD = path - env = self.TestEnvironment(CD = 'CacheDir') - env.fs = MyFS() env.CacheDir('foo') - assert env.fs.CD == 'foo', env.fs.CD + assert env._CacheDir.path == 'foo', env._CacheDir.path env.CacheDir('$CD') - assert env.fs.CD == 'CacheDir', env.fs.CD + assert env._CacheDir.path == 'CacheDir', env._CacheDir.path def test_Clean(self): """Test the Clean() method""" diff --git a/src/engine/SCons/Executor.py b/src/engine/SCons/Executor.py index 12114bc..7d8df68 100644 --- a/src/engine/SCons/Executor.py +++ b/src/engine/SCons/Executor.py @@ -296,10 +296,8 @@ class Null(_Executor): kw['action'] = [] apply(_Executor.__init__, (self,), kw) def get_build_env(self): - class NullEnvironment: - def get_scanner(self, key): - return None - return NullEnvironment() + import SCons.Util + return SCons.Util.Null() def get_build_scanner_path(self): return None def cleanup(self): diff --git a/src/engine/SCons/Node/FS.py b/src/engine/SCons/Node/FS.py index ad21a4d..74dc655 100644 --- a/src/engine/SCons/Node/FS.py +++ b/src/engine/SCons/Node/FS.py @@ -223,77 +223,6 @@ def get_MkdirBuilder(): name = "MkdirBuilder") return MkdirBuilder -def CacheRetrieveFunc(target, source, env): - t = target[0] - fs = t.fs - cachedir, cachefile = t.cachepath() - if not fs.exists(cachefile): - fs.CacheDebug('CacheRetrieve(%s): %s not in cache\n', t, cachefile) - return 1 - fs.CacheDebug('CacheRetrieve(%s): retrieving from %s\n', t, cachefile) - if SCons.Action.execute_actions: - if fs.islink(cachefile): - fs.symlink(fs.readlink(cachefile), t.path) - else: - fs.copy2(cachefile, t.path) - st = fs.stat(cachefile) - fs.chmod(t.path, stat.S_IMODE(st[stat.ST_MODE]) | stat.S_IWRITE) - return 0 - -def CacheRetrieveString(target, source, env): - t = target[0] - cachedir, cachefile = t.cachepath() - if t.fs.exists(cachefile): - return "Retrieved `%s' from cache" % t.path - return None - -CacheRetrieve = SCons.Action.Action(CacheRetrieveFunc, CacheRetrieveString) - -CacheRetrieveSilent = SCons.Action.Action(CacheRetrieveFunc, None) - -def CachePushFunc(target, source, env): - t = target[0] - if t.nocache: - return - fs = t.fs - cachedir, cachefile = t.cachepath() - if fs.exists(cachefile): - # Don't bother copying it if it's already there. Note that - # usually this "shouldn't happen" because if the file already - # existed in cache, we'd have retrieved the file from there, - # not built it. This can happen, though, in a race, if some - # other person running the same build pushes their copy to - # the cache after we decide we need to build it but before our - # build completes. - fs.CacheDebug('CachePush(%s): %s already exists in cache\n', t, cachefile) - return - - fs.CacheDebug('CachePush(%s): pushing to %s\n', t, cachefile) - - if not fs.isdir(cachedir): - fs.makedirs(cachedir) - - tempfile = cachefile+'.tmp' - try: - if fs.islink(t.path): - fs.symlink(fs.readlink(t.path), tempfile) - else: - fs.copy2(t.path, tempfile) - fs.rename(tempfile, cachefile) - st = fs.stat(t.path) - fs.chmod(cachefile, stat.S_IMODE(st[stat.ST_MODE]) | stat.S_IWRITE) - except (IOError, OSError): - # It's possible someone else tried writing the file at the - # same time we did, or else that there was some problem like - # the CacheDir being on a separate file system that's full. - # In any case, inability to push a file to cache doesn't affect - # the correctness of the build, so just print a warning. - SCons.Warnings.warn(SCons.Warnings.CacheWriteErrorWarning, - "Unable to copy %s to cache. Cache file is %s" - % (str(target), cachefile)) - -CachePush = SCons.Action.Action(CachePushFunc, None) - class _Null: pass @@ -738,6 +667,15 @@ class Base(SCons.Node.Node): return ret def target_from_source(self, prefix, suffix, splitext=SCons.Util.splitext): + """ + + Generates a target entry that corresponds to this entry (usually + a source file) with the specified prefix and suffix. + + Note that this method can be overridden dynamically for generated + files that need different behavior. See Tool/swig.py for + an example. + """ return self.dir.Entry(prefix + splitext(self.name)[0] + suffix) def _Rfindalldirs_key(self, pathlist): @@ -783,6 +721,26 @@ class Base(SCons.Node.Node): cwd = self.cwd or self.fs._cwd return cwd.Rfindalldirs(pathlist) + memoizer_counters.append(SCons.Memoize.CountValue('rentry')) + + def rentry(self): + try: + return self._memo['rentry'] + except KeyError: + pass + result = self + if not self.exists(): + norm_name = _my_normcase(self.name) + for dir in self.dir.get_all_rdirs(): + try: + node = dir.entries[norm_name] + except KeyError: + if dir.entry_exists_on_disk(self.name): + result = dir.Entry(self.name) + break + self._memo['rentry'] = result + return result + class Entry(Base): """This is the class for generic Node.FS entries--that is, things that could be a File or a Dir, but we're just not sure yet. @@ -993,9 +951,6 @@ class FS(LocalFS): self.Root = {} self.SConstruct_dir = None - self.CachePath = None - self.cache_force = None - self.cache_show = None self.max_drift = default_max_drift self.Top = None @@ -1259,30 +1214,6 @@ class FS(LocalFS): d = self.Dir(d) self.Top.addRepository(d) - def CacheDebugWrite(self, fmt, target, cachefile): - self.CacheDebugFP.write(fmt % (target, os.path.split(cachefile)[1])) - - def CacheDebugQuiet(self, fmt, target, cachefile): - pass - - CacheDebug = CacheDebugQuiet - - def CacheDebugEnable(self, file): - if file == '-': - self.CacheDebugFP = sys.stdout - else: - self.CacheDebugFP = open(file, 'w') - self.CacheDebug = self.CacheDebugWrite - - def CacheDir(self, path): - try: - import SCons.Sig.MD5 - except ImportError: - msg = "No MD5 module available, CacheDir() not supported" - SCons.Warnings.warn(SCons.Warnings.NoMD5ModuleWarning, msg) - else: - self.CachePath = path - def build_dir_target_climb(self, orig, dir, tail): """Create targets in corresponding build directories @@ -1681,9 +1612,9 @@ class Dir(Base): def srcdir_duplicate(self, name): for dir in self.srcdir_list(): if dir.entry_exists_on_disk(name): - srcnode = dir.File(name) + srcnode = dir.Entry(name).disambiguate() if self.duplicate: - node = self.File(name) + node = self.Entry(name).disambiguate() node.do_duplicate(srcnode) return node else: @@ -1750,7 +1681,37 @@ class Dir(Base): diskcheck_sccs(self, name): try: return self.File(name) except TypeError: pass - return self.srcdir_duplicate(name) + node = self.srcdir_duplicate(name) + if isinstance(node, Dir): + node = None + return node + + def walk(self, func, arg): + """ + Walk this directory tree by calling the specified function + for each directory in the tree. + + This behaves like the os.path.walk() function, but for in-memory + Node.FS.Dir objects. The function takes the same arguments as + the functions passed to os.path.walk(): + + func(arg, dirname, fnames) + + Except that "dirname" will actually be the directory *Node*, + not the string. The '.' and '..' entries are excluded from + fnames. The fnames list may be modified in-place to filter the + subdirectories visited or otherwise impose a specific order. + The "arg" argument is always passed to func() and may be used + in any way (or ignored, passing None is common). + """ + entries = self.entries + names = entries.keys() + names.remove('.') + names.remove('..') + func(arg, self, names) + select_dirs = lambda n, e=entries: isinstance(e[n], Dir) + for dirname in filter(select_dirs, names): + entries[dirname].walk(func, arg) class RootDir(Dir): """A class for the root directory of a file system. @@ -2048,53 +2009,17 @@ class File(Base): so only do thread safe stuff here. Do thread unsafe stuff in built(). - Note that there's a special trick here with the execute flag - (one that's not normally done for other actions). Basically - if the user requested a no_exec (-n) build, then - SCons.Action.execute_actions is set to 0 and when any action - is called, it does its showing but then just returns zero - instead of actually calling the action execution operation. - The problem for caching is that if the file does NOT exist in - cache then the CacheRetrieveString won't return anything to - show for the task, but the Action.__call__ won't call - CacheRetrieveFunc; instead it just returns zero, which makes - the code below think that the file *was* successfully - retrieved from the cache, therefore it doesn't do any - subsequent building. However, the CacheRetrieveString didn't - print anything because it didn't actually exist in the cache, - and no more build actions will be performed, so the user just - sees nothing. The fix is to tell Action.__call__ to always - execute the CacheRetrieveFunc and then have the latter - explicitly check SCons.Action.execute_actions itself. - Returns true iff the node was successfully retrieved. """ if self.nocache: return None - b = self.is_derived() - if not b and not self.has_src_builder(): + if not self.is_derived(): return None - - retrieved = None - if b and self.fs.CachePath: - if self.fs.cache_show: - if CacheRetrieveSilent(self, [], None, execute=1) == 0: - self.build(presub=0, execute=0) - retrieved = 1 - else: - if CacheRetrieve(self, [], None, execute=1) == 0: - retrieved = 1 - if retrieved: - # Record build signature information, but don't - # push it out to cache. (We just got it from there!) - self.set_state(SCons.Node.executed) - SCons.Node.Node.built(self) - - return retrieved - + return self.get_build_env().get_CacheDir().retrieve(self) def built(self): - """Called just after this node is successfully built. + """ + Called just after this node is successfully built. """ # Push this file out to cache before the superclass Node.built() # method has a chance to clear the build signature, which it @@ -2104,13 +2029,13 @@ class File(Base): # cache so that the memoization of the self.exists() return # value doesn't interfere. self.clear_memoized_values() - if self.fs.CachePath and self.exists(): - CachePush(self, [], None) + if self.exists(): + self.get_build_env().get_CacheDir().push(self) SCons.Node.Node.built(self) def visited(self): - if self.fs.CachePath and self.fs.cache_force and self.exists(): - CachePush(self, None, None) + if self.exists(): + self.get_build_env().get_CacheDir().push_if_forced(self) def has_src_builder(self): """Return whether this Node has a source builder or not. @@ -2333,9 +2258,8 @@ class File(Base): def rstr(self): return str(self.rfile()) - def cachepath(self): - if self.nocache or not self.fs.CachePath: - return None, None + def get_cachedir_bsig(self): + import SCons.Sig.MD5 ninfo = self.get_binfo().ninfo if not hasattr(ninfo, 'bsig'): import SCons.Errors @@ -2346,14 +2270,16 @@ class File(Base): # Add the path to the cache signature, because multiple # targets built by the same action will all have the same # build signature, and we have to differentiate them somehow. - import SCons.Sig.MD5 - cache_sig = SCons.Sig.MD5.collect([ninfo.bsig, self.path]) - subdir = string.upper(cache_sig[0]) - dir = os.path.join(self.fs.CachePath, subdir) - return dir, os.path.join(dir, cache_sig) + return SCons.Sig.MD5.collect([ninfo.bsig, self.path]) default_fs = None +def get_default_fs(): + global default_fs + if not default_fs: + default_fs = FS() + return default_fs + class FileFinder: """ """ diff --git a/src/engine/SCons/Node/FSTests.py b/src/engine/SCons/Node/FSTests.py index 0b60fb9..ed8d6ec 100644 --- a/src/engine/SCons/Node/FSTests.py +++ b/src/engine/SCons/Node/FSTests.py @@ -89,6 +89,7 @@ class Action: pass def strfunction(self, targets, sources, env): return "" + class Builder: def __init__(self, factory, action=Action()): self.factory = factory @@ -2041,6 +2042,74 @@ class RepositoryTestCase(_tempdirTestCase): r = map(lambda x, np=os.path.normpath: np(str(x)), rep) assert r == expect, r + def test_rentry(self): + """Test the Base.entry() method""" + return_true = lambda: 1 + return_false = lambda: 0 + + d1 = self.fs.Dir('d1') + d2 = self.fs.Dir('d2') + d3 = self.fs.Dir('d3') + + e1 = self.fs.Entry('e1') + e2 = self.fs.Entry('e2') + e3 = self.fs.Entry('e3') + + f1 = self.fs.File('f1') + f2 = self.fs.File('f2') + f3 = self.fs.File('f3') + + self.test.write([self.rep1, 'd2'], "") + self.test.subdir([self.rep2, 'd3']) + self.test.write([self.rep3, 'd3'], "") + + self.test.write([self.rep1, 'e2'], "") + self.test.subdir([self.rep2, 'e3']) + self.test.write([self.rep3, 'e3'], "") + + self.test.write([self.rep1, 'f2'], "") + self.test.subdir([self.rep2, 'f3']) + self.test.write([self.rep3, 'f3'], "") + + r = d1.rentry() + assert r is d1, r + + r = d2.rentry() + assert not r is d2, r + r = str(r) + assert r == os.path.join(self.rep1, 'd2'), r + + r = d3.rentry() + assert not r is d3, r + r = str(r) + assert r == os.path.join(self.rep2, 'd3'), r + + r = e1.rentry() + assert r is e1, r + + r = e2.rentry() + assert not r is e2, r + r = str(r) + assert r == os.path.join(self.rep1, 'e2'), r + + r = e3.rentry() + assert not r is e3, r + r = str(r) + assert r == os.path.join(self.rep2, 'e3'), r + + r = f1.rentry() + assert r is f1, r + + r = f2.rentry() + assert not r is f2, r + r = str(r) + assert r == os.path.join(self.rep1, 'f2'), r + + r = f3.rentry() + assert not r is f3, r + r = str(r) + assert r == os.path.join(self.rep2, 'f3'), r + def test_rdir(self): """Test the Dir.rdir() method""" return_true = lambda: 1 @@ -2468,200 +2537,6 @@ class SConstruct_dirTestCase(unittest.TestCase): fs.set_SConstruct_dir(fs.Dir('xxx')) assert fs.SConstruct_dir.path == 'xxx' -class CacheDirTestCase(unittest.TestCase): - def runTest(self): - """Test CacheDir functionality""" - test = TestCmd(workdir='') - - global built_it - - fs = SCons.Node.FS.FS() - assert fs.CachePath is None, fs.CachePath - assert fs.cache_force is None, fs.cache_force - assert fs.cache_show is None, fs.cache_show - - fs.CacheDir('cache') - assert fs.CachePath == 'cache', fs.CachePath - - save_CacheRetrieve = SCons.Node.FS.CacheRetrieve - self.retrieved = [] - def retrieve_succeed(target, source, env, self=self, execute=1): - self.retrieved.append(target) - return 0 - def retrieve_fail(target, source, env, self=self, execute=1): - self.retrieved.append(target) - return 1 - - f1 = fs.File("cd.f1") - f1.builder_set(Builder(fs.File)) - f1.env_set(Environment()) - try: - SCons.Node.FS.CacheRetrieve = retrieve_succeed - self.retrieved = [] - built_it = None - - r = f1.retrieve_from_cache() - assert r == 1, r - assert self.retrieved == [f1], self.retrieved - assert built_it is None, built_it - - SCons.Node.FS.CacheRetrieve = retrieve_fail - self.retrieved = [] - built_it = None - - r = f1.retrieve_from_cache() - assert r is None, r - assert self.retrieved == [f1], self.retrieved - assert built_it is None, built_it - finally: - SCons.Node.FS.CacheRetrieve = save_CacheRetrieve - - save_CacheRetrieveSilent = SCons.Node.FS.CacheRetrieveSilent - - fs.cache_show = 1 - - f2 = fs.File("cd.f2") - f2.builder_set(Builder(fs.File)) - f2.env_set(Environment()) - try: - SCons.Node.FS.CacheRetrieveSilent = retrieve_succeed - self.retrieved = [] - built_it = None - - r = f2.retrieve_from_cache() - assert r == 1, r - assert self.retrieved == [f2], self.retrieved - assert built_it is None, built_it - - SCons.Node.FS.CacheRetrieveSilent = retrieve_fail - self.retrieved = [] - built_it = None - - r = f2.retrieve_from_cache() - assert r is None, r - assert self.retrieved == [f2], self.retrieved - assert built_it is None, built_it - finally: - SCons.Node.FS.CacheRetrieveSilent = save_CacheRetrieveSilent - - save_CachePush = SCons.Node.FS.CachePush - def push(target, source, env, self=self): - self.pushed.append(target) - return 0 - SCons.Node.FS.CachePush = push - - try: - self.pushed = [] - - cd_f3 = test.workpath("cd.f3") - f3 = fs.File(cd_f3) - f3.built() - assert self.pushed == [], self.pushed - test.write(cd_f3, "cd.f3\n") - f3.built() - assert self.pushed == [f3], self.pushed - - self.pushed = [] - - cd_f4 = test.workpath("cd.f4") - f4 = fs.File(cd_f4) - f4.visited() - assert self.pushed == [], self.pushed - test.write(cd_f4, "cd.f4\n") - f4.visited() - assert self.pushed == [], self.pushed - fs.cache_force = 1 - f4.visited() - assert self.pushed == [f4], self.pushed - finally: - SCons.Node.FS.CachePush = save_CachePush - - # Verify how the cachepath() method determines the name - # of the file in cache. - def my_collect(list): - return list[0] - save_collect = SCons.Sig.MD5.collect - SCons.Sig.MD5.collect = my_collect - try: - f5 = fs.File("cd.f5") - f5.binfo = f5.BuildInfo(f5) - f5.binfo.ninfo.bsig = 'a_fake_bsig' - cp = f5.cachepath() - dirname = os.path.join('cache', 'A') - filename = os.path.join(dirname, 'a_fake_bsig') - assert cp == (dirname, filename), cp - finally: - SCons.Sig.MD5.collect = save_collect - - # Verify that no bsig raises an InternalERror - f6 = fs.File("cd.f6") - f6.binfo = f6.BuildInfo(f6) - exc_caught = 0 - try: - cp = f6.cachepath() - except SCons.Errors.InternalError: - exc_caught = 1 - assert exc_caught - - # Verify that we raise a warning if we can't copy a file to cache. - save_copy2 = shutil.copy2 - def copy2(src, dst): - raise OSError - shutil.copy2 = copy2 - save_mkdir = os.mkdir - def mkdir(dir, mode=0): - pass - os.mkdir = mkdir - old_warn_exceptions = SCons.Warnings.warningAsException(1) - SCons.Warnings.enableWarningClass(SCons.Warnings.CacheWriteErrorWarning) - - try: - cd_f7 = test.workpath("cd.f7") - test.write(cd_f7, "cd.f7\n") - f7 = fs.File(cd_f7) - f7.binfo = f7.BuildInfo(f7) - f7.binfo.ninfo.bsig = 'f7_bsig' - - warn_caught = 0 - try: - f7.built() - except SCons.Warnings.CacheWriteErrorWarning: - warn_caught = 1 - assert warn_caught - finally: - shutil.copy2 = save_copy2 - os.mkdir = save_mkdir - SCons.Warnings.warningAsException(old_warn_exceptions) - SCons.Warnings.suppressWarningClass(SCons.Warnings.CacheWriteErrorWarning) - - # Verify that we don't blow up if there's no strfunction() - # for an action. - act = Action() - act.strfunction = None - f8 = fs.File("cd.f8") - f8.builder_set(Builder(fs.File, action=act)) - f8.env_set(Environment()) - try: - SCons.Node.FS.CacheRetrieveSilent = retrieve_succeed - self.retrieved = [] - built_it = None - - r = f8.retrieve_from_cache() - assert r == 1, r - assert self.retrieved == [f8], self.retrieved - assert built_it is None, built_it - - SCons.Node.FS.CacheRetrieveSilent = retrieve_fail - self.retrieved = [] - built_it = None - - r = f8.retrieve_from_cache() - assert r is None, r - assert self.retrieved == [f8], self.retrieved - assert built_it is None, built_it - finally: - SCons.Node.FS.CacheRetrieveSilent = save_CacheRetrieveSilent - class clearTestCase(unittest.TestCase): def runTest(self): """Test clearing FS nodes of cached data.""" @@ -3007,7 +2882,6 @@ if __name__ == "__main__": suite.addTest(has_src_builderTestCase()) suite.addTest(prepareTestCase()) suite.addTest(SConstruct_dirTestCase()) - suite.addTest(CacheDirTestCase()) suite.addTest(clearTestCase()) suite.addTest(disambiguateTestCase()) suite.addTest(postprocessTestCase()) diff --git a/src/engine/SCons/Node/NodeTests.py b/src/engine/SCons/Node/NodeTests.py index ad9eb66..09ab5c7 100644 --- a/src/engine/SCons/Node/NodeTests.py +++ b/src/engine/SCons/Node/NodeTests.py @@ -33,6 +33,7 @@ import UserList import SCons.Errors import SCons.Node +import SCons.Util @@ -977,7 +978,7 @@ class NodeTestCase(unittest.TestCase): target = SCons.Node.Node() source = SCons.Node.Node() s = target.get_source_scanner(source) - assert s is None, s + assert isinstance(s, SCons.Util.Null), s ts1 = Scanner() ts2 = Scanner() diff --git a/src/engine/SCons/PathList.py b/src/engine/SCons/PathList.py index 943e9ad..be645ca 100644 --- a/src/engine/SCons/PathList.py +++ b/src/engine/SCons/PathList.py @@ -138,7 +138,8 @@ class _PathList: value = string.join(map(str, value), '') elif type == TYPE_OBJECT: value = node_conv(value) - result.append(value) + if value: + result.append(value) return tuple(result) diff --git a/src/engine/SCons/PathListTests.py b/src/engine/SCons/PathListTests.py index d0ba2a0..8203ccf 100644 --- a/src/engine/SCons/PathListTests.py +++ b/src/engine/SCons/PathListTests.py @@ -47,7 +47,7 @@ class subst_pathTestCase(unittest.TestCase): s = self.kw[s] return s - self.env = FakeEnvironment(AAA = 'aaa') + self.env = FakeEnvironment(AAA = 'aaa', NULL = '') def test_node(self): """Test the subst_path() method on a Node @@ -111,10 +111,10 @@ class subst_pathTestCase(unittest.TestCase): assert result == ('x',), result def test_subst(self): - """Test the subst_path() method on a substitution string + """Test the subst_path() method on substitution strings """ - pl = SCons.PathList.PathList(('$AAA',)) + pl = SCons.PathList.PathList(('$AAA', '$NULL')) result = pl.subst_path(self.env, 'y', 'z') diff --git a/src/engine/SCons/Scanner/LaTeX.py b/src/engine/SCons/Scanner/LaTeX.py index 645a894..c0a38b5 100644 --- a/src/engine/SCons/Scanner/LaTeX.py +++ b/src/engine/SCons/Scanner/LaTeX.py @@ -29,12 +29,12 @@ This module implements the dependency scanner for LaTeX code. __revision__ = "__FILE__ __REVISION__ __DATE__ __DEVELOPER__" +import os.path +import string import SCons.Scanner -import string -import os.path -def LaTeXScanner(fs = SCons.Node.FS.default_fs): +def LaTeXScanner(): """Return a prototype Scanner instance for scanning LaTeX source files""" ds = LaTeX(name = "LaTeXScanner", suffixes = '$LATEXSUFFIXES', diff --git a/src/engine/SCons/Script/Main.py b/src/engine/SCons/Script/Main.py index a3c0a51..a76165f 100644 --- a/src/engine/SCons/Script/Main.py +++ b/src/engine/SCons/Script/Main.py @@ -54,6 +54,7 @@ import traceback # 'lib', # 'scons-%d' % SCons.__version__)] + sys.path[1:] +import SCons.CacheDir import SCons.Debug import SCons.Defaults import SCons.Environment @@ -641,10 +642,15 @@ def _load_site_scons_dir(topdir, site_dir_name=None): SCons.Tool.DefaultToolpath.append(os.path.abspath(site_tools_dir)) def version_string(label, module): - fmt = "\t%s: v%s.%s, %s, by %s on %s\n" + version = module.__version__ + build = module.__build__ + if build: + if build[0] != '.': + build = '.' + build + version = version + build + fmt = "\t%s: v%s, %s, by %s on %s\n" return fmt % (label, - module.__version__, - module.__build__, + version, module.__date__, module.__developer__, module.__buildsys__) @@ -720,7 +726,7 @@ def _main(parser): # Now that we're in the top-level SConstruct directory, go ahead # and initialize the FS object that represents the file system, # and make it the build engine default. - fs = SCons.Node.FS.default_fs = SCons.Node.FS.FS() + fs = SCons.Node.FS.get_default_fs() for rep in options.repository: fs.Repository(rep) @@ -770,15 +776,14 @@ def _main(parser): if options.silent: SCons.Action.print_actions = None - if options.cache_debug: - fs.CacheDebugEnable(options.cache_debug) if options.cache_disable: - def disable(self): pass - fs.CacheDir = disable + SCons.CacheDir.CacheDir = SCons.Util.Null() + if options.cache_debug: + SCons.CacheDir.cache_debug = options.cache_debug if options.cache_force: - fs.cache_force = 1 + SCons.CacheDir.cache_force = True if options.cache_show: - fs.cache_show = 1 + SCons.CacheDir.cache_show = True if options.site_dir: _load_site_scons_dir(d, options.site_dir) diff --git a/src/engine/SCons/Script/SConsOptions.py b/src/engine/SCons/Script/SConsOptions.py index 053fff1..46ece27 100644 --- a/src/engine/SCons/Script/SConsOptions.py +++ b/src/engine/SCons/Script/SConsOptions.py @@ -166,6 +166,65 @@ class SConsValues(optparse.Values): self.__SConscript_settings__[name] = value +class SConsOption(optparse.Option): + def convert_value(self, opt, value): + if value is not None: + if self.nargs in (1, '?'): + return self.check_value(opt, value) + else: + return tuple(map(lambda v, o=opt, s=self: s.check_value(o, v), value)) + + def process(self, opt, value, values, parser): + + # First, convert the value(s) to the right type. Howl if any + # value(s) are bogus. + value = self.convert_value(opt, value) + + # And then take whatever action is expected of us. + # This is a separate method to make life easier for + # subclasses to add new actions. + return self.take_action( + self.action, self.dest, opt, value, values, parser) + + def _check_nargs_optional(self): + if self.nargs == '?' and self._short_opts: + fmt = "option %s: nargs='?' is incompatible with short options" + raise SCons.Errors.UserError, fmt % self._short_opts[0] + + try: + _orig_CONST_ACTIONS = optparse.Option.CONST_ACTIONS + + _orig_CHECK_METHODS = optparse.Option.CHECK_METHODS + + except AttributeError: + # optparse.Option had no CONST_ACTIONS before Python 2.5. + + _orig_CONST_ACTIONS = ("store_const",) + + def _check_const(self): + if self.action not in self.CONST_ACTIONS and self.const is not None: + raise OptionError( + "'const' must not be supplied for action %r" % self.action, + self) + + # optparse.Option collects its list of unbound check functions + # up front. This sucks because it means we can't just override + # the _check_const() function like a normal method, we have to + # actually replace it in the list. This seems to be the most + # straightforward way to do that. + + _orig_CHECK_METHODS = [optparse.Option._check_action, + optparse.Option._check_type, + optparse.Option._check_choice, + optparse.Option._check_dest, + _check_const, + optparse.Option._check_nargs, + optparse.Option._check_callback] + + CHECK_METHODS = _orig_CHECK_METHODS + [_check_nargs_optional] + + CONST_ACTIONS = _orig_CONST_ACTIONS + optparse.Option.TYPED_ACTIONS + class SConsOptionGroup(optparse.OptionGroup): """ A subclass for SCons-specific option groups. @@ -232,7 +291,12 @@ class SConsOptionParser(optparse.OptionParser): option = self._long_opt[opt] if option.takes_value(): nargs = option.nargs - if len(rargs) < nargs: + if nargs == '?': + if had_explicit_value: + value = rargs.pop(0) + else: + value = option.const + elif len(rargs) < nargs: if nargs == 1: self.error(_("%s option requires an argument") % opt) else: @@ -396,7 +460,8 @@ def Parser(version): formatter = SConsIndentedHelpFormatter(max_help_position=30) - op = SConsOptionParser(add_help_option=False, + op = SConsOptionParser(option_class=SConsOption, + add_help_option=False, formatter=formatter, usage="usage: scons [OPTION] [TARGET] ...",) diff --git a/src/engine/SCons/Script/__init__.py b/src/engine/SCons/Script/__init__.py index 8204c65..4010d80 100644 --- a/src/engine/SCons/Script/__init__.py +++ b/src/engine/SCons/Script/__init__.py @@ -296,6 +296,8 @@ GlobalDefaultEnvironmentFunctions = [ 'Execute', 'File', 'FindFile', + 'FindInstalledFiles', + 'FindSourceFiles', 'Flatten', 'GetBuildPath', 'Ignore', @@ -311,11 +313,9 @@ GlobalDefaultEnvironmentFunctions = [ 'SourceCode', 'SourceSignatures', 'Split', + 'Tag', 'TargetSignatures', 'Value', - 'Tag', - 'FindInstalledFiles', - 'FindSourceFiles', ] GlobalDefaultBuilders = [ diff --git a/src/engine/SCons/Tool/JavaCommonTests.py b/src/engine/SCons/Tool/JavaCommonTests.py index 40f0a47..358f675 100644 --- a/src/engine/SCons/Tool/JavaCommonTests.py +++ b/src/engine/SCons/Tool/JavaCommonTests.py @@ -418,6 +418,39 @@ public class NestedExample expect = [ 'NestedExample$1', 'NestedExample$1$1', 'NestedExample' ] assert expect == classes, (expect, classes) + def test_private_inner_class_instantiation(self): + """Test anonymous inner class generated by private instantiation""" + + input = """\ +class test +{ + test() + { + super(); + new inner(); + } + + static class inner + { + private inner() {} + } +} +""" + + # This is what we *should* generate, apparently due to the + # private instantiation of the inner class, but don't today. + #expect = [ 'test$1', 'test$inner', 'test' ] + + # What our parser currently generates, which doesn't match + # what the Java compiler actually generates. + expect = [ 'test$inner', 'test' ] + + pkg_dir, classes = SCons.Tool.JavaCommon.parse_java(input, '1.4') + assert expect == classes, (expect, classes) + + pkg_dir, classes = SCons.Tool.JavaCommon.parse_java(input, '1.5') + assert expect == classes, (expect, classes) + if __name__ == "__main__": diff --git a/src/engine/SCons/Tool/__init__.py b/src/engine/SCons/Tool/__init__.py index bf3df56..97a4e70 100644 --- a/src/engine/SCons/Tool/__init__.py +++ b/src/engine/SCons/Tool/__init__.py @@ -43,6 +43,7 @@ import sys import SCons.Builder import SCons.Errors +import SCons.Node.FS import SCons.Scanner import SCons.Scanner.C import SCons.Scanner.D @@ -176,6 +177,9 @@ class Tool: def __str__(self): return self.name +########################################################################## +# Create common executable program / library / object builders + def createProgBuilder(env): """This is a utility function that creates the Program Builder in an Environment if it is not there already. @@ -347,6 +351,79 @@ def createCFileBuilders(env): return (c_file, cxx_file) +########################################################################## +# Create common Java builders + +def CreateJarBuilder(env): + try: + java_jar = env['BUILDERS']['Jar'] + except KeyError: + fs = SCons.Node.FS.get_default_fs() + jar_com = SCons.Action.Action('$JARCOM', '$JARCOMSTR') + java_jar = SCons.Builder.Builder(action = jar_com, + suffix = '$JARSUFFIX', + src_suffix = '$JAVACLASSSUFIX', + src_builder = 'JavaClassFile', + source_factory = fs.Entry) + env['BUILDERS']['Jar'] = java_jar + return java_jar + +def CreateJavaHBuilder(env): + try: + java_javah = env['BUILDERS']['JavaH'] + except KeyError: + fs = SCons.Node.FS.get_default_fs() + java_javah_com = SCons.Action.Action('$JAVAHCOM', '$JAVAHCOMSTR') + java_javah = SCons.Builder.Builder(action = java_javah_com, + src_suffix = '$JAVACLASSSUFFIX', + target_factory = fs.Entry, + source_factory = fs.File, + src_builder = 'JavaClassFile') + env['BUILDERS']['JavaH'] = java_javah + return java_javah + +def CreateJavaClassFileBuilder(env): + try: + java_class_file = env['BUILDERS']['JavaClassFile'] + except KeyError: + fs = SCons.Node.FS.get_default_fs() + javac_com = SCons.Action.Action('$JAVACCOM', '$JAVACCOMSTR') + java_class_file = SCons.Builder.Builder(action = javac_com, + emitter = {}, + #suffix = '$JAVACLASSSUFFIX', + src_suffix = '$JAVASUFFIX', + src_builder = ['JavaFile'], + target_factory = fs.Entry, + source_factory = fs.File) + env['BUILDERS']['JavaClassFile'] = java_class_file + return java_class_file + +def CreateJavaClassDirBuilder(env): + try: + java_class_dir = env['BUILDERS']['JavaClassDir'] + except KeyError: + fs = SCons.Node.FS.get_default_fs() + javac_com = SCons.Action.Action('$JAVACCOM', '$JAVACCOMSTR') + java_class_dir = SCons.Builder.Builder(action = javac_com, + emitter = {}, + target_factory = fs.Dir, + source_factory = fs.Dir) + env['BUILDERS']['JavaClassDir'] = java_class_dir + return java_class_dir + +def CreateJavaFileBuilder(env): + try: + java_file = env['BUILDERS']['JavaFile'] + except KeyError: + java_file = SCons.Builder.Builder(action = {}, + emitter = {}, + suffix = {None:'$JAVASUFFIX'}) + env['BUILDERS']['JavaFile'] = java_file + env['JAVASUFFIX'] = '.java' + return java_file + + + def FindTool(tools, env): for tool in tools: t = Tool(tool) diff --git a/src/engine/SCons/Tool/aixlink.py b/src/engine/SCons/Tool/aixlink.py index 2deb1f1..5c3b383 100644 --- a/src/engine/SCons/Tool/aixlink.py +++ b/src/engine/SCons/Tool/aixlink.py @@ -44,7 +44,7 @@ cplusplus = __import__('c++', globals(), locals(), []) def smart_linkflags(source, target, env, for_signature): if cplusplus.iscplusplus(source): - build_dir = env.subst('$BUILDDIR') + build_dir = env.subst('$BUILDDIR', target=target, source=source) if build_dir: return '-qtempinc=' + os.path.join(build_dir, 'tempinc') return '' diff --git a/src/engine/SCons/Tool/filesystem.py b/src/engine/SCons/Tool/filesystem.py index 5e631fd..46aefbb 100644 --- a/src/engine/SCons/Tool/filesystem.py +++ b/src/engine/SCons/Tool/filesystem.py @@ -29,6 +29,9 @@ selection method. # OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION # WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. # + +__revision__ = "__FILE__ __REVISION__ __DATE__ __DEVELOPER__" + import SCons from SCons.Tool.install import copyFunc diff --git a/src/engine/SCons/Tool/fortran.py b/src/engine/SCons/Tool/fortran.py index 8494fd6..b748303 100644 --- a/src/engine/SCons/Tool/fortran.py +++ b/src/engine/SCons/Tool/fortran.py @@ -78,8 +78,8 @@ def _fortranEmitter(target, source, env): # Remove unique items from the list modules = SCons.Util.unique(modules) # Convert module name to a .mod filename - suffix = env.subst('$FORTRANMODSUFFIX') - moddir = env.subst('$FORTRANMODDIR') + suffix = env.subst('$FORTRANMODSUFFIX', target=target, source=source) + moddir = env.subst('$FORTRANMODDIR', target=target, source=source) modules = map(lambda x, s=suffix: string.lower(x) + s, modules) for m in modules: target.append(env.fs.File(m, moddir)) diff --git a/src/engine/SCons/Tool/install.py b/src/engine/SCons/Tool/install.py index 5083c9f..828bb9e 100644 --- a/src/engine/SCons/Tool/install.py +++ b/src/engine/SCons/Tool/install.py @@ -29,8 +29,14 @@ selection method. # OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION # WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. # + +__revision__ = "__FILE__ __REVISION__ __DATE__ __DEVELOPER__" + +import os +import shutil +import stat + import SCons.Action -import shutil, os, stat from SCons.Util import make_path_relative # diff --git a/src/engine/SCons/Tool/install.xml b/src/engine/SCons/Tool/install.xml new file mode 100644 index 0000000..4b57a68 --- /dev/null +++ b/src/engine/SCons/Tool/install.xml @@ -0,0 +1,52 @@ +<!-- +__COPYRIGHT__ + +This file is processed by the bin/SConsDoc.py module. +See its __doc__ string for a discussion of the format. +--> + +<tool name="install"> +<summary> +Sets construction variables for file +and directory installation. +</summary> +<sets> +INSTALL +INSTALLSTR +</sets> +</tool> + +<builder name="Install"> +<summary> +Installs one or more source files or directories +in the specified target, +which must be a directory. +The names of the specified source files or directories +remain the same within the destination directory. + +<example> +env.Install('/usr/local/bin', source = ['foo', 'bar']) +</example> +</summary> +</builder> + +<builder name="InstallAs"> +<summary> +Installs one or more source files or directories +to specific names, +allowing changing a file or directory name +as part of the installation. +It is an error if the +target +and +source +arguments list different numbers of files or directories. + +<example> +env.InstallAs(target = '/usr/local/bin/foo', + source = 'foo_debug') +env.InstallAs(target = ['../lib/libfoo.a', '../lib/libbar.a'], + source = ['libFOO.a', 'libBAR.a']) +</example> +</summary> +</builder> diff --git a/src/engine/SCons/Tool/jar.py b/src/engine/SCons/Tool/jar.py index cb0a8eb..4f221c0 100644 --- a/src/engine/SCons/Tool/jar.py +++ b/src/engine/SCons/Tool/jar.py @@ -33,14 +33,12 @@ selection method. __revision__ = "__FILE__ __REVISION__ __DATE__ __DEVELOPER__" -import SCons.Action -import SCons.Builder import SCons.Subst import SCons.Util def jarSources(target, source, env, for_signature): """Only include sources that are not a manifest file.""" - jarchdir = env.subst('$JARCHDIR') + jarchdir = env.subst('$JARCHDIR', target=target, source=source) if jarchdir: jarchdir = env.fs.Dir(jarchdir) result = [] @@ -67,7 +65,7 @@ def jarManifest(target, source, env, for_signature): def jarFlags(target, source, env, for_signature): """If we have a manifest, make sure that the 'm' flag is specified.""" - jarflags = env.subst('$JARFLAGS') + jarflags = env.subst('$JARFLAGS', target=target, source=source) for src in source: contents = src.get_contents() if contents[:16] == "Manifest-Version": @@ -76,25 +74,17 @@ def jarFlags(target, source, env, for_signature): break return jarflags -JarAction = SCons.Action.Action('$JARCOM', '$JARCOMSTR') - -JarBuilder = SCons.Builder.Builder(action = JarAction, - source_factory = SCons.Node.FS.Entry, - suffix = '$JARSUFFIX') - def generate(env): """Add Builders and construction variables for jar to an Environment.""" - try: - env['BUILDERS']['Jar'] - except KeyError: - env['BUILDERS']['Jar'] = JarBuilder + SCons.Tool.CreateJarBuilder(env) env['JAR'] = 'jar' env['JARFLAGS'] = SCons.Util.CLVar('cf') env['_JARFLAGS'] = jarFlags env['_JARMANIFEST'] = jarManifest env['_JARSOURCES'] = jarSources - env['JARCOM'] = '$JAR $_JARFLAGS $TARGET $_JARMANIFEST $_JARSOURCES' + env['_JARCOM'] = '$JAR $_JARFLAGS $TARGET $_JARMANIFEST $_JARSOURCES' + env['JARCOM'] = "${TEMPFILE('$_JARCOM')}" env['JARSUFFIX'] = '.jar' def exists(env): diff --git a/src/engine/SCons/Tool/jar.xml b/src/engine/SCons/Tool/jar.xml index 0deaaae..a0d730e 100644 --- a/src/engine/SCons/Tool/jar.xml +++ b/src/engine/SCons/Tool/jar.xml @@ -22,7 +22,13 @@ JARCOMSTR <builder name="Jar"> <summary> Builds a Java archive (<filename>.jar</filename>) file -from a source tree of <filename>.class</filename> files. +from the specified list of sources. +Any directories in the source list +will be searched for <filename>.class</filename> files). +Any <filename>.java</filename> files in the source list +will be compiled to <filename>.class</filename> files +by calling the &b-link-Java; Builder. + If the &cv-link-JARCHDIR; value is set, the &jar; command will change to the specified directory using the @@ -39,6 +45,9 @@ option set. <example> env.Jar(target = 'foo.jar', source = 'classes') + +env.Jar(target = 'bar.jar', + source = ['bar1.java', 'bar2.java']) </example> </summary> </builder> diff --git a/src/engine/SCons/Tool/javac.py b/src/engine/SCons/Tool/javac.py index 85dfc3f..5703b2d 100644 --- a/src/engine/SCons/Tool/javac.py +++ b/src/engine/SCons/Tool/javac.py @@ -54,43 +54,73 @@ def emit_java_classes(target, source, env): java_suffix = env.get('JAVASUFFIX', '.java') class_suffix = env.get('JAVACLASSSUFFIX', '.class') + target[0].must_be_same(SCons.Node.FS.Dir) + classdir = target[0] + + s = source[0].rentry().disambiguate() + if isinstance(s, SCons.Node.FS.File): + sourcedir = s.dir.rdir() + elif isinstance(s, SCons.Node.FS.Dir): + sourcedir = s.rdir() + else: + raise SCons.Errors.UserError("Java source must be File or Dir, not '%s'" % s.__class__) + slist = [] js = _my_normcase(java_suffix) - for sdir in source: - def visit(arg, dirname, names, js=js, dirnode=sdir.rdir()): - java_files = filter(lambda n, js=js: - _my_normcase(n[-len(js):]) == js, - names) - # The on-disk entries come back in arbitrary order. Sort them - # so our target and source lists are determinate. - java_files.sort() - mydir = dirnode.Dir(dirname) - java_paths = map(lambda f, d=mydir: d.File(f), java_files) - arg.extend(java_paths) - os.path.walk(sdir.rdir().get_abspath(), visit, slist) + find_java = lambda n, js=js, ljs=len(js): _my_normcase(n[-ljs:]) == js + for entry in source: + entry = entry.rentry().disambiguate() + if isinstance(entry, SCons.Node.FS.File): + slist.append(entry) + elif isinstance(entry, SCons.Node.FS.Dir): + result = SCons.Util.OrderedDict() + def visit(arg, dirname, names, fj=find_java, dirnode=entry.rdir()): + java_files = filter(fj, names) + # The on-disk entries come back in arbitrary order. Sort + # them so our target and source lists are determinate. + java_files.sort() + mydir = dirnode.Dir(dirname) + java_paths = map(lambda f, d=mydir: d.File(f), java_files) + for jp in java_paths: + arg[jp] = True + + os.path.walk(entry.rdir().get_abspath(), visit, result) + entry.walk(visit, result) + + slist.extend(result.keys()) + else: + raise SCons.Errors.UserError("Java source must be File or Dir, not '%s'" % entry.__class__) + version = env.get('JAVAVERSION', '1.4') tlist = [] for f in slist: - version = env.get('JAVAVERSION', '1.4') - pkg_dir, classes = parse_java_file(f.get_abspath(), version) - if pkg_dir: - for c in classes: - t = target[0].Dir(pkg_dir).File(c+class_suffix) - t.attributes.java_classdir = target[0] - t.attributes.java_classname = classname(pkg_dir + os.sep + c) - tlist.append(t) - elif classes: - for c in classes: - t = target[0].File(c+class_suffix) - t.attributes.java_classdir = target[0] - t.attributes.java_classname = classname(c) - tlist.append(t) - else: - # This is an odd end case: no package and no classes. - # Just do our best based on the source file name. - base = str(f)[:-len(java_suffix)] - t = target[0].File(base + class_suffix) - t.attributes.java_classdir = target[0] + source_file_based = True + pkg_dir = None + if not f.is_derived(): + pkg_dir, classes = parse_java_file(f.rfile().get_abspath(), version) + if classes: + source_file_based = False + if pkg_dir: + d = target[0].Dir(pkg_dir) + p = pkg_dir + os.sep + else: + d = target[0] + p = '' + for c in classes: + t = d.File(c + class_suffix) + t.attributes.java_classdir = classdir + t.attributes.java_sourcedir = sourcedir + t.attributes.java_classname = classname(p + c) + tlist.append(t) + + if source_file_based: + base = f.name[:-len(java_suffix)] + if pkg_dir: + t = target[0].Dir(pkg_dir).File(base + class_suffix) + else: + t = target[0].File(base + class_suffix) + t.attributes.java_classdir = classdir + t.attributes.java_sourcedir = f.dir t.attributes.java_classname = classname(base) tlist.append(t) @@ -100,18 +130,81 @@ JavaAction = SCons.Action.Action('$JAVACCOM', '$JAVACCOMSTR') JavaBuilder = SCons.Builder.Builder(action = JavaAction, emitter = emit_java_classes, - target_factory = SCons.Node.FS.Dir, - source_factory = SCons.Node.FS.Dir) + target_factory = SCons.Node.FS.Entry, + source_factory = SCons.Node.FS.Entry) + +def getClassPath(env,target, source, for_signature): + path = "" + if env.has_key('JAVACLASSPATH') and env['JAVACLASSPATH']: + path = SCons.Util.AppendPath(path, env['JAVACLASSPATH']) + return "-classpath %s" % (path) + else: + return "" + +def getSourcePath(env,target, source, for_signature): + path = "" + if env.has_key('JAVASOURCEPATH') and env['JAVASOURCEPATH']: + path = SCons.Util.AppendPath(path, env['JAVASOURCEPATH']) + path = SCons.Util.AppendPath(path,['${TARGET.attributes.java_sourcedir}']) + return "-sourcepath %s" % (path) + +def Java(env, target, source, *args, **kw): + """ + A pseudo-Builder wrapper around the separate JavaClass{File,Dir} + Builders. + """ + if not SCons.Util.is_List(target): + target = [target] + if not SCons.Util.is_List(source): + source = [source] + + # Pad the target list with repetitions of the last element in the + # list so we have a target for every source element. + target = target + ([target[-1]] * (len(source) - len(target))) + + java_suffix = env.subst('$JAVASUFFIX') + result = [] + + for t, s in zip(target, source): + if isinstance(s, SCons.Node.FS.Base): + if isinstance(s, SCons.Node.FS.File): + b = env.JavaClassFile + else: + b = env.JavaClassDir + else: + if os.path.isfile(s): + b = env.JavaClassFile + elif os.path.isdir(s): + b = env.JavaClassDir + elif s[-len(java_suffix):] == java_suffix: + b = env.JavaClassFile + else: + b = env.JavaClassDir + result.extend(apply(b, (t, s) + args, kw)) + + return result def generate(env): """Add Builders and construction variables for javac to an Environment.""" - env['BUILDERS']['Java'] = JavaBuilder + java_file = SCons.Tool.CreateJavaFileBuilder(env) + java_class = SCons.Tool.CreateJavaClassFileBuilder(env) + java_class_dir = SCons.Tool.CreateJavaClassDirBuilder(env) + java_class.add_emitter(None, emit_java_classes) + java_class.add_emitter(env.subst('$JAVASUFFIX'), emit_java_classes) + java_class_dir.emitter = emit_java_classes + + env.AddMethod(Java) env['JAVAC'] = 'javac' env['JAVACFLAGS'] = SCons.Util.CLVar('') - env['JAVACCOM'] = '$JAVAC $JAVACFLAGS -d ${TARGET.attributes.java_classdir} -sourcepath ${SOURCE.dir.rdir()} $SOURCES' + env['JAVACLASSPATH'] = [] + env['JAVASOURCEPATH'] = [] + env['_JAVACLASSPATH'] = getClassPath + env['_JAVASOURCEPATH'] = getSourcePath + env['_JAVACCOM'] = '$JAVAC $JAVACFLAGS $_JAVACLASSPATH -d ${TARGET.attributes.java_classdir} $_JAVASOURCEPATH $SOURCES' + env['JAVACCOM'] = "${TEMPFILE('$_JAVACCOM')}" env['JAVACLASSSUFFIX'] = '.class' env['JAVASUFFIX'] = '.java' def exists(env): - return env.Detect('javac') + return 1 diff --git a/src/engine/SCons/Tool/javac.xml b/src/engine/SCons/Tool/javac.xml index 248eda2..f5af975 100644 --- a/src/engine/SCons/Tool/javac.xml +++ b/src/engine/SCons/Tool/javac.xml @@ -14,6 +14,8 @@ JAVACFLAGS JAVACCOM JAVACLASSSUFFIX JAVASUFFIX +JAVACLASSPATH +JAVASOURCEPATH </sets> <uses> JAVACCOMSTR @@ -22,16 +24,21 @@ JAVACCOMSTR <builder name="Java"> <summary> -Builds one or more Java class files -from one or more source trees of <filename>.java</filename> files. -The class files will be placed underneath -the specified target directory. +Builds one or more Java class files. +The sources may be any combination of explicit +<filename>.java</filename> files, +or directory trees which will be scanned +for <filename>.java</filename> files. + SCons will parse each source <filename>.java</filename> file to find the classes (including inner classes) defined within that file, and from that figure out the target <filename>.class</filename> files that will be created. +The class files will be placed underneath +the specified target directory. + SCons will also search each Java file for the Java package name, which it assumes can be found on a line @@ -58,6 +65,7 @@ Example: <example> env.Java(target = 'classes', source = 'src') env.Java(target = 'classes', source = ['src1', 'src2']) +env.Java(target = 'classes', source = ['File1.java', 'File2.java']) </example> </summary> </builder> @@ -107,6 +115,27 @@ builder. </summary> </cvar> +<cvar name="JAVACLASSPATH"> +<summary> +Specifies the list of directories that +will be searched for Java +<filename>.class</filename> file. +The directories in this list will be added to the +&javac; and &javah; command lines +via the <option>-classpath</option> option. +The individual directory names will be +separated by the operating system's path separate character +(<filename>:</filename> on UNIX/Linux/POSIX, +<filename>;</filename> on Windows). + +Note that this currently just adds the specified +directory via the <option>-classpath</option> option. +&SCons; does not currently search the +&cv-JAVACLASSPATH; directories for dependency +<filename>.class</filename> files. +</summary> +</cvar> + <cvar name="JAVACLASSSUFFIX"> <summary> The suffix for Java class files; @@ -115,6 +144,27 @@ by default. </summary> </cvar> +<cvar name="JAVASOURCEPATH"> +<summary> +Specifies the list of directories that +will be searched for input +<filename>.java</filename> file. +The directories in this list will be added to the +&javac; command line +via the <option>-sourcepath</option> option. +The individual directory names will be +separated by the operating system's path separate character +(<filename>:</filename> on UNIX/Linux/POSIX, +<filename>;</filename> on Windows). + +Note that this currently just adds the specified +directory via the <option>-sourcepath</option> option. +&SCons; does not currently search the +&cv-JAVASOURCEPATH; directories for dependency +<filename>.java</filename> files. +</summary> +</cvar> + <cvar name="JAVASUFFIX"> <summary> The suffix for Java files; diff --git a/src/engine/SCons/Tool/javah.py b/src/engine/SCons/Tool/javah.py index 26fcc53..7eb4969 100644 --- a/src/engine/SCons/Tool/javah.py +++ b/src/engine/SCons/Tool/javah.py @@ -60,6 +60,7 @@ def emit_java_headers(target, source, env): except AttributeError: classdir = '.' classdir = env.Dir(classdir).rdir() + if str(classdir) == '.': c_ = None else: @@ -77,10 +78,13 @@ def emit_java_headers(target, source, env): classname = classname[:-len(class_suffix)] classname = SCons.Tool.javac.classname(classname) s = src.rfile() - s.attributes.java_classdir = classdir s.attributes.java_classname = classname slist.append(s) + s = source[0].rfile() + if not hasattr(s.attributes, 'java_classdir'): + s.attributes.java_classdir = classdir + if target[0].__class__ is SCons.Node.FS.File: tlist = target else: @@ -106,22 +110,22 @@ def JavaHOutFlagGenerator(target, source, env, for_signature): except AttributeError: return '-o ' + str(t) -JavaHAction = SCons.Action.Action('$JAVAHCOM', '$JAVAHCOMSTR') - -JavaHBuilder = SCons.Builder.Builder(action = JavaHAction, - emitter = emit_java_headers, - src_suffix = '$JAVACLASSSUFFIX', - target_factory = SCons.Node.FS.Entry, - source_factory = SCons.Node.FS.File) +def getJavaHClassPath(env,target, source, for_signature): + path = "${SOURCE.attributes.java_classdir}" + if env.has_key('JAVACLASSPATH') and env['JAVACLASSPATH']: + path = SCons.Util.AppendPath(path, env['JAVACLASSPATH']) + return "-classpath %s" % (path) def generate(env): """Add Builders and construction variables for javah to an Environment.""" - env['BUILDERS']['JavaH'] = JavaHBuilder + java_javah = SCons.Tool.CreateJavaHBuilder(env) + java_javah.emitter = emit_java_headers env['_JAVAHOUTFLAG'] = JavaHOutFlagGenerator env['JAVAH'] = 'javah' env['JAVAHFLAGS'] = SCons.Util.CLVar('') - env['JAVAHCOM'] = '$JAVAH $JAVAHFLAGS $_JAVAHOUTFLAG -classpath ${SOURCE.attributes.java_classdir} ${SOURCES.attributes.java_classname}' + env['_JAVAHCLASSPATH'] = getJavaHClassPath + env['JAVAHCOM'] = '$JAVAH $JAVAHFLAGS $_JAVAHOUTFLAG $_JAVAHCLASSPATH ${SOURCES.attributes.java_classname}' env['JAVACLASSSUFFIX'] = '.class' def exists(env): diff --git a/src/engine/SCons/Tool/javah.xml b/src/engine/SCons/Tool/javah.xml index c6487e7..20fe3ee 100644 --- a/src/engine/SCons/Tool/javah.xml +++ b/src/engine/SCons/Tool/javah.xml @@ -16,6 +16,7 @@ JAVACLASSSUFFIX </sets> <uses> JAVAHCOMSTR +JAVACLASSPATH </uses> </tool> @@ -27,7 +28,10 @@ The target can be either a directory in which the header files will be written, or a header file name which will contain all of the definitions. -The source can be either the names of <filename>.class</filename> files, +The source can be the names of <filename>.class</filename> files, +the names of <filename>.java</filename> files +to be compiled into <filename>.class</filename> files +by calling the &b-link-Java; builder method, or the objects returned from the &b-Java; builder method. diff --git a/src/engine/SCons/Tool/lex.py b/src/engine/SCons/Tool/lex.py index 31f21a9..e599f8a 100644 --- a/src/engine/SCons/Tool/lex.py +++ b/src/engine/SCons/Tool/lex.py @@ -55,7 +55,8 @@ def lexEmitter(target, source, env): # Different options that are used to trigger the creation of extra files. fileGenOptions = ["--header-file=", "--tables-file="] - for option in SCons.Util.CLVar(env.subst("$LEXFLAGS")): + lexflags = env.subst("$LEXFLAGS", target=target, source=source) + for option in SCons.Util.CLVar(lexflags): for fileGenOption in fileGenOptions: l = len(fileGenOption) if option[:l] == fileGenOption: diff --git a/src/engine/SCons/Tool/packaging/__init__.py b/src/engine/SCons/Tool/packaging/__init__.py index c5f40b7..e8f6a09 100644 --- a/src/engine/SCons/Tool/packaging/__init__.py +++ b/src/engine/SCons/Tool/packaging/__init__.py @@ -175,6 +175,15 @@ def Package(env, target=None, source=None, **kw): targets.extend(env.Alias( 'package', targets )) return targets +def build_source(ss, sources): + for s in ss: + if s.__class__==SCons.Node.FS.Dir: + build_source(s.all_children()) + elif not s.has_builder() and s.__class__==SCons.Node.FS.File: + sources.append(s) + else: + build_source(s.sources) + def FindSourceFiles(env, target=None, source=None ): """ returns a list of all children of the target nodes, which have no children. This selects all leaves of the DAG that gets build by SCons for @@ -185,17 +194,8 @@ def FindSourceFiles(env, target=None, source=None ): nodes = env.arg2nodes(target, env.fs.Entry) sources = [] - def build_source(ss): - for s in ss: - if s.__class__==SCons.Node.FS.Dir: - build_source(s.all_children()) - elif not s.has_builder() and s.__class__==SCons.Node.FS.File: - sources.append(s) - else: - build_source(s.sources) - for node in nodes: - build_source(node.all_children()) + build_source(node.all_children(), sources) # now strip the build_node from the sources by calling the srcnode # function diff --git a/src/engine/SCons/Tool/packaging/__init__.xml b/src/engine/SCons/Tool/packaging/__init__.xml index 8e7f652..bf4e96e 100644 --- a/src/engine/SCons/Tool/packaging/__init__.xml +++ b/src/engine/SCons/Tool/packaging/__init__.xml @@ -6,36 +6,46 @@ See its __doc__ string for a discussion of the format. --> <tool name="Packaging"> <summary> -TODO +Sets construction variables for the &b-Package; Builder. </summary> +<sets> +</sets> +<uses> +</uses> </tool> <builder name="Package"> -Builds software distribution packages. Packages consist of files to install -and packaging information. The former may be specified with the source -parameter and may be left out, in which case the &-bFindInstalledFiles; -function will collect all files that have an &-bInstall; or -&-bInstallAs; Builder attached. The target, if not specified will be deduced -from additional information given to this Builder. - -The packaging information is specified with the help of construction Variables -documented below. This information is called a tag to stress that some of them -can also be attached to files with the &-bTag; Builder.The mandatory ones will -complain if they were not specified. They vary depending on chosen target -packager. - -The target packager may be selected with the "PACKAGETYPE" command line option -or with the &-tPACKAGETYPE; construction variable. Currently there are six -packagers available: +<summary> +Builds software distribution packages. Packages consist of files +to install and packaging information. The former may be specified +with the &source; parameter and may be left out, in which case the +&FindInstalledFiles; function will collect all files that have an +&b-Install; or &b-InstallAs; Builder attached. If the ⌖, is +not specified it will be deduced from additional information given to +this Builder. + +The packaging information is specified with the help of construction +variables documented below. This information is called a tag to stress +that some of them can also be attached to files with the &Tag; function. +The mandatory ones will complain if they were not specified. They vary +depending on chosen target packager. + +The target packager may be selected with the "PACKAGETYPE" command line +option or with the &cv-PACKAGETYPE; construction variable. Currently +the following packagers available: * msi - Microsoft Installer * rpm - Redhat Package Manger * ipkg - Itsy Package Management System - * tarbz, tarbz and zip + * tarbz2 - compressed tar + * targz - compressed tar + * zip - zip file + * src_tarbz2 - compressed tar source + * src_targz - compressed tar source + * src_zip - zip file source An updated list is always available under the "package_type" option when running "scons --help" on a project that has packaging activated. - <example> env = Environment(tools=['default', 'packaging']) env.Install('/bin/', 'my_program') @@ -50,148 +60,229 @@ env.Package( NAME = 'foo', SOURCE_URL = 'http://foo.org/foo-1.2.3.tar.gz' ) </example> +</summary> +</builder> -<cvar name="DESCRIPTION"> +<cvar name="ARCHITECTURE"> <summary> -A long description of what the project is about. +Specifies the system architecture for which +the package is being built. +The default is the system architecture +of the machine on which SCons is running. +This is used to fill in the +<literal>Architecture:</literal> +field in an Ipkg +<filename>control</filename> file, +and as part of the name of a generated RPM file. </summary> </cvar> -<cvar name="DESCRIPTION_<lang>"> +<cvar name="CHANGE_SPECFILE"> <summary> -TODO +A hook for modifying the file that controls the packaging build +(the <filename>.spec</filename> for RPM, +the <filename>control</filename> for Ipkg, +the <filename>.wxs</filename> for MSI). +If set, the function will be called +after the SCons template for the file has been written. +XXX </summary> </cvar> +<cvar name="CHANGELOG"> +<summary> +The name of a file containing the change log text +to be included in the package. +This is included as the +<literal>%changelog</literal> +section of the RPM +<filename>.spec</filename> file. +</summary> +</cvar> -<cvar name="SUMMARY"> +<cvar name="DESCRIPTION"> <summary> -A short summary of what the project is about. +A long description of the project being packaged. +This is included in the relevant section +of the file that controls the packaging build. </summary> </cvar> -<cvar name="LICENSE"> +<cvar name="DESCRIPTION_lang"> <summary> -The shorthand of the license this project is under (gpl, lpgl, bsd etc.). +A language-specific long description for +the specified <varname>lang</varname>. +This is used to populate a +<literal>%description -l</literal> +section of an RPM +<filename>.spec</filename> file. </summary> </cvar> -<cvar name="NAME"> +<cvar name="LICENSE"> <summary> -Specfies the name of the project to package. +The abbreviated name of the license under which +this project is released (gpl, lpgl, bsd etc.). +See http://www.opensource.org/licenses/alphabetical +for a list of license names. </summary> </cvar> -<cvar name="VERSION"> +<cvar name="NAME"> <summary> -The version of the project, given as a string. +Specfies the name of the project to package. </summary> </cvar> -<cvar name="PACKAGEVERSION"> +<cvar name="PACKAGEROOT"> <summary> -The version of the package, if only changes in the package were done. Currently -only used by the rpm packager. +Specifies the directory where all files in resulting archive will be +placed if applicable. The default value is "$NAME-$VERSION". </summary> </cvar> <cvar name="PACKAGETYPE"> <summary> -Selects the package type to build. Currently those are available: +Selects the package type to build. Currently these are available: * msi - Microsoft Installer * rpm - Redhat Package Manger * ipkg - Itsy Package Management System - * tarbz2, targz and zip - tarball and zip packager - * src_tarbz2, src_targz and src_zip - source tarbarll and zip packager + * tarbz2 - compressed tar + * targz - compressed tar + * zip - zip file + * src_tarbz2 - compressed tar source + * src_targz - compressed tar source + * src_zip - zip file source This may be overridden with the "package_type" command line option. </summary> </cvar> -<cvar name="VENDOR"> +<cvar name="PACKAGEVERSION"> <summary> -TODO +The version of the package (not the underlying project). +This is currently only used by the rpm packager +and should reflect changes in the packaging, +not the underlying project code itself. </summary> </cvar> <cvar name="SOURCE_URL"> <summary> -TODO +The URL +(web address) +of the location from which the project was retrieved. +This is used to fill in the +<literal>Source:</literal> +field in the controlling information for Ipkg and RPM packages. </summary> </cvar> -<cvar name="ARCHITECURE"> +<cvar name="SUMMARY"> <summary> -TODO +A short summary of what the project is about. +This is used to fill in the +<literal>Summary:</literal> +field in the controlling information for Ipkg and RPM packages, +and as the +<literal>Description:</literal> +field in MSI packages. </summary> </cvar> -<cvar name="PACKAGEROOT"> +<cvar name="VENDOR"> <summary> -defines the directory where all files in resulting archive will be placed if -applicable. The default value is "$NAME-$VERSION". +The person or organization who supply the packaged software. +This is used to fill in the +<literal>Vendor:</literal> +field in the controlling information for RPM packages, +and the +<literal>Manufacturer:</literal> +field in the controlling information for MSI packages. </summary> </cvar> -<cvar name="LICENSE"> +<cvar name="VERSION"> <summary> -Short name of the license your package is under. Example: gpl, lgpl, bsd ... -See http://www.opensource.org/licenses/alphabetical +The version of the project, specified as a string. </summary> </cvar> -<cvar name="CHANGE_SPECFILE"> + +<cvar name="X_IPK_DEPENDS"> <summary> -TODO +This is used to fill in the +<literal>Depends:</literal> +field in the controlling information for Ipkg packages. </summary> </cvar> -<cvar name="CHANGELOG"> +<cvar name="X_IPK_DESCRIPTION"> <summary> -TODO +This is used to fill in the +<literal>Description:</literal> +field in the controlling information for Ipkg packages. +The default value is +<literal>$SUMMARY\n$DESCRIPTION</literal> </summary> </cvar> -<cvar name="X_RPM_PREINSTALL"> +<cvar name="X_IPK_MAINTAINER"> <summary> -TODO +This is used to fill in the +<literal>Maintainer:</literal> +field in the controlling information for Ipkg packages. </summary> </cvar> -<cvar name="X_RPM_DEFATTR"> +<cvar name="X_IPK_PRIORITY"> <summary> -TODO +This is used to fill in the +<literal>Priority:</literal> +field in the controlling information for Ipkg packages. </summary> </cvar> -<cvar name="X_RPM_POSTINSTALL"> +<cvar name="X_IPK_SECTION"> <summary> -TODO +This is used to fill in the +<literal>Section:</literal> +field in the controlling information for Ipkg packages. </summary> </cvar> -<cvar name="X_RPM_PREUNINSTALL"> + + +<cvar name="X_MSI_LANGUAGE"> <summary> -TODO +This is used to fill in the +<literal>Language:</literal> +attribute in the controlling information for MSI packages. </summary> </cvar> -<cvar name="X_RPM_POSTUNINSTALL"> +<cvar name="X_MSI_LICENSE_TEXT"> <summary> -TODO +The text of the software license in RTF format. +Carriage return characters will be +replaced with the RTF equivalent \\par. </summary> </cvar> -<cvar name="X_RPM_VERIFY"> +<cvar name="X_MSI_UPGRADE_CODE"> <summary> TODO </summary> </cvar> -<cvar name="X_RPM_PREP"> + +<cvar name="X_RPM_AUTOREQPROV"> <summary> -internal, but overridable +This is used to fill in the +<literal>AutoReqProv:</literal> +field in the RPM +<filename>.spec</filename> file. </summary> </cvar> @@ -201,188 +292,254 @@ internal, but overridable </summary> </cvar> -<cvar name="X_RPM_INSTALL"> +<cvar name="X_RPM_BUILDREQUIRES"> <summary> -internal, but overridable +This is used to fill in the +<literal>BuildRequires:</literal> +field in the RPM +<filename>.spec</filename> file. </summary> </cvar> -<cvar name="X_RPM_CLEAN"> +<cvar name="X_RPM_BUILDROOT"> <summary> internal, but overridable </summary> </cvar> -<cvar name="X_RPM_URL"> +<cvar name="X_RPM_CLEAN"> <summary> -TODO +internal, but overridable </summary> </cvar> -<cvar name="X_RPM_GROUP"> +<cvar name="X_RPM_CONFLICTS"> <summary> -TODO +This is used to fill in the +<literal>Conflicts:</literal> +field in the RPM +<filename>.spec</filename> file. </summary> </cvar> -<cvar name="X_RPM_GROUP_<lang>"> +<cvar name="X_RPM_DEFATTR"> <summary> -TODO +This value is used as the default attributes +for the files in the RPM package. +The default value is +<literal>(-,root,root)</literal>. </summary> </cvar> <cvar name="X_RPM_DISTRIBUTION"> <summary> -TODO +This is used to fill in the +<literal>Distribution:</literal> +field in the RPM +<filename>.spec</filename> file. </summary> </cvar> -<cvar name="X_RPM_ICON"> +<cvar name="X_RPM_EPOCH"> <summary> -TODO +This is used to fill in the +<literal>Epoch:</literal> +field in the controlling information for RPM packages. </summary> </cvar> -<cvar name="X_RPM_PACKAGER"> +<cvar name="X_RPM_EXCLUDEARCH"> <summary> -TODO +This is used to fill in the +<literal>ExcludeArch:</literal> +field in the RPM +<filename>.spec</filename> file. </summary> </cvar> -<cvar name="X_RPM_REQUIRES"> +<cvar name="X_RPM_EXLUSIVEARCH"> <summary> -TODO +This is used to fill in the +<literal>ExclusiveArch:</literal> +field in the RPM +<filename>.spec</filename> file. </summary> </cvar> -<cvar name="X_RPM_PROVIDES"> +<cvar name="X_RPM_GROUP"> <summary> -TODO +This is used to fill in the +<literal>Group:</literal> +field in the RPM +<filename>.spec</filename> file. </summary> </cvar> -<cvar name="X_RPM_CONFLICTS"> +<cvar name="X_RPM_GROUP_lang"> <summary> -TODO +This is used to fill in the +<literal>Group(lang):</literal> +field in the RPM +<filename>.spec</filename> file. +Note that +<varname>lang</varname> +is not literal +and should be replaced by +the appropriate language code. </summary> </cvar> -<cvar name="X_RPM_BUILDREQUIRES"> +<cvar name="X_RPM_ICON"> <summary> -TODO +This is used to fill in the +<literal>Icon:</literal> +field in the RPM +<filename>.spec</filename> file. </summary> </cvar> -<cvar name="X_RPM_SERIAL"> +<cvar name="X_RPM_INSTALL"> <summary> -TODO +internal, but overridable </summary> </cvar> -<cvar name="X_RPM_EPOCH"> +<cvar name="X_RPM_PACKAGER"> <summary> -TODO +This is used to fill in the +<literal>Packager:</literal> +field in the RPM +<filename>.spec</filename> file. </summary> </cvar> -<cvar name="X_RPM_AUTOREQPROV"> +<cvar name="X_RPM_PROVIDES"> <summary> -TODO +This is used to fill in the +<literal>Provides:</literal> +field in the RPM +<filename>.spec</filename> file. </summary> </cvar> -<cvar name="X_RPM_EXCLUDEARCH"> +<cvar name="X_RPM_POSTINSTALL"> <summary> -TODO +This is used to fill in the +<literal>%post:</literal> +section in the RPM +<filename>.spec</filename> file. </summary> </cvar> -<cvar name="X_RPM_EXLUSIVEARCH"> +<cvar name="X_RPM_PREINSTALL"> <summary> -TODO +This is used to fill in the +<literal>%pre:</literal> +section in the RPM +<filename>.spec</filename> file. </summary> </cvar> <cvar name="X_RPM_PREFIX"> <summary> -TODO +This is used to fill in the +<literal>Prefix:</literal> +field in the RPM +<filename>.spec</filename> file. </summary> </cvar> -<cvar name="X_RPM_CONFLICTS"> -<summary> -TODO -</summary> -</cvar> - -<cvar name="X_RPM_BUILDROOT"> +<cvar name="X_RPM_PREP"> <summary> internal, but overridable </summary> </cvar> -<cvar name="X_RPM_GROUP_<lang>"> +<cvar name="X_RPM_POSTUNINSTALL"> <summary> -TODO +This is used to fill in the +<literal>%postun:</literal> +section in the RPM +<filename>.spec</filename> file. </summary> </cvar> -<cvar name="X_MSI_LICENSE_TEXT"> +<cvar name="X_RPM_PREUNINSTALL"> <summary> -The text of the software license in rtf format. Carriage return chars will be -replaced with the rtf equivalent \\par. +This is used to fill in the +<literal>%preun:</literal> +section in the RPM +<filename>.spec</filename> file. </summary> </cvar> -<cvar name="X_MSI_LANGUAGE"> +<cvar name="X_RPM_REQUIRES"> <summary> -TODO +This is used to fill in the +<literal>Requires:</literal> +field in the RPM +<filename>.spec</filename> file. </summary> </cvar> -<cvar name="X_MSI_UPGRADE_CODE"> +<cvar name="X_RPM_SERIAL"> <summary> -TODO +This is used to fill in the +<literal>Serial:</literal> +field in the RPM +<filename>.spec</filename> file. </summary> </cvar> -<cvar name="X_IPK_PRIORITY"> +<cvar name="X_RPM_URL"> <summary> -TODO +This is used to fill in the +<literal>Url:</literal> +field in the RPM +<filename>.spec</filename> file. </summary> </cvar> -<cvar name="X_IPK_SECTION"> + + +<!-- + +THE FOLLOWING AREN'T CONSTRUCTION VARIABLES, +THEY'RE "TAGS" THAT CAN BE ATTACHED +TO DIFFERENT FILES OR DIRECTORIES. +NOT SURE YET WHAT TO DO ABOUT THESE. + +<cvar name="CONFIG"> <summary> TODO </summary> </cvar> -<cvar name="X_IPK_MAINTAINER"> +<cvar name="CONFIG_NOREPLACE"> <summary> TODO </summary> </cvar> -<cvar name="X_IPK_DEPENDS"> +<cvar name="DOC"> <summary> TODO </summary> </cvar> -<cvar name="X_IPK_DESCRIPTION"> +<cvar name="INSTALL_LOCATION"> <summary> -default is "$SUMMARY\n$DESCRIPTION" +internal, but overridable, TODO </summary> </cvar> -<cvar name="X_IPK_POSTRM"> +<cvar name="LANG_lang"> <summary> TODO </summary> </cvar> -<cvar name="X_IPK_PRERM"> +<cvar name="UNIX_ATTR"> <summary> TODO </summary> @@ -394,61 +551,49 @@ TODO </summary> </cvar> -<cvar name="X_IPK_PREINST"> +<cvar name="X_IPK_POSTRM"> <summary> TODO </summary> </cvar> -</builder> - -<builder name="Tag"> -Leaves hints for the Package() Builder on how specific should be packaged. -All those tags are optional. -<example> -Tag( Library( 'lib.c' ), unix-attr="0644" ) # makes sure the built library will - # be installed with 0644 file - # access mode -Tag( 'file2.txt', doc ) # marks file2.txt to be a documentation file -</example> - -<cvar name="INSTALL_LOCATION"> +<cvar name="X_IPK_PREINST"> <summary> -internal, but overridable, TODO +TODO </summary> </cvar> -<cvar name="CONFIG"> +<cvar name="X_IPK_PRERM"> <summary> TODO </summary> </cvar> -<cvar name="CONFIG_NOREPLACE"> +<cvar name="X_MSI_FEATURE"> <summary> TODO </summary> </cvar> -<cvar name="DOC"> +<cvar name="X_MSI_FILEID"> <summary> TODO </summary> </cvar> -<cvar name="UNIX_ATTR="> +<cvar name="X_MSI_LONGNAME"> <summary> TODO </summary> </cvar> -<cvar name="LANG_<lang>"> +<cvar name="X_MSI_SHORTNAME"> <summary> TODO </summary> </cvar> -<cvar name="X_RPM_VERIFY"> +<cvar name="X_MSI_VITAL"> <summary> TODO </summary> @@ -472,51 +617,42 @@ TODO </summary> </cvar> -<cvar name="X_MSI_FEATURE="> +<cvar name="X_RPM_VERIFY"> <summary> TODO </summary> </cvar> -<cvar name="X_MSI_VITAL"> -<summary> -TODO -</summary> -</cvar> +--> -<cvar name="X_MSI_FILEID"> -<summary> -TODO -</summary> -</cvar> -<cvar name="X_MSI_LONGNAME"> +<!-- +<builder name="Tag"> <summary> -TODO -</summary> -</cvar> +Leaves hints for the Package() Builder on how specific +files or directories should be packaged. +All tags are optional. -<cvar name="X_MSI_SHORTNAME"> -<summary> -TODO -</summary> -</cvar> +<example> +# makes sure the built library will be installed with 0644 file +# access mode +Tag( Library( 'lib.c' ), unix-attr="0644" ) -<cvar name="X_MSI_SHORTNAME"> -<summary> -TODO +# marks file2.txt to be a documentation file +Tag( 'file2.txt', doc ) +</example> </summary> -</cvar> - </builder> -<builder name="FindSourceFiles"> -A convenience function which returns all leafs of the build tree. -</builder> +<function name="FindSourceFiles"> +<summary> +A convenience function which returns all leaves of the build tree. +</summary> +</function> <builder name="FindInstalledFiles"> -Returns all files "build" by the install builder. -</builder> - -</tool> - +<summary> +Returns all files "built" by the &b-Install; or &b-InstallAs; builders. +</summary> +</function> +--> diff --git a/src/engine/SCons/Tool/packaging/rpm.py b/src/engine/SCons/Tool/packaging/rpm.py index 49b8ff8..94b7b7a 100644 --- a/src/engine/SCons/Tool/packaging/rpm.py +++ b/src/engine/SCons/Tool/packaging/rpm.py @@ -41,9 +41,7 @@ def package(env, target, source, PACKAGEROOT, NAME, VERSION, # initialize the rpm tool SCons.Tool.Tool('rpm').generate(env) - # create the neccesary builder bld = env['BUILDERS']['Rpm'] - env['RPMFLAGS'] = SCons.Util.CLVar('-ta') bld.push_emitter(targz_emitter) bld.push_emitter(specfile_emitter) @@ -67,8 +65,9 @@ def package(env, target, source, PACKAGEROOT, NAME, VERSION, if kw.has_key('ARCHITECTURE'): buildarchitecture = kw['ARCHITECTURE'] - srcrpm = '%s-%s-%s.src.rpm' % (NAME, VERSION, PACKAGEVERSION) - binrpm = string.replace(srcrpm, 'src', buildarchitecture) + fmt = '%s-%s-%s.%s.rpm' + srcrpm = fmt % (NAME, VERSION, PACKAGEVERSION, 'src') + binrpm = fmt % (NAME, VERSION, PACKAGEVERSION, buildarchitecture) target = [ srcrpm, binrpm ] diff --git a/src/engine/SCons/Tool/swig.py b/src/engine/SCons/Tool/swig.py index 5326e8d..8ca1b89 100644 --- a/src/engine/SCons/Tool/swig.py +++ b/src/engine/SCons/Tool/swig.py @@ -33,57 +33,30 @@ selection method. __revision__ = "__FILE__ __REVISION__ __DATE__ __DEVELOPER__" +import os.path +import re + import SCons.Action import SCons.Defaults +import SCons.Scanner import SCons.Tool import SCons.Util -from SCons.Scanner import Scanner -import os -import re SwigAction = SCons.Action.Action('$SWIGCOM', '$SWIGCOMSTR') def swigSuffixEmitter(env, source): - if '-c++' in SCons.Util.CLVar(env.subst("$SWIGFLAGS")): + if '-c++' in SCons.Util.CLVar(env.subst("$SWIGFLAGS", source=source)): return '$SWIGCXXFILESUFFIX' else: return '$SWIGCFILESUFFIX' -_reInclude = re.compile(r'%include\s+(\S+)') _reModule = re.compile(r'%module\s+(.+)') -def recurse(path, searchPath): - global _reInclude - f = open(path) - try: contents = f.read() - finally: f.close() - - found = [] - # Better code for when we drop Python 1.5.2. - #for m in _reInclude.finditer(contents): - # fname = m.group(1) - for fname in _reInclude.findall(contents): - for dpath in searchPath: - absPath = os.path.join(dpath, fname) - if os.path.isfile(absPath): - found.append(absPath) - break - - # Equivalent code for when we drop Python 1.5.2. - #for f in [f for f in found if os.path.splitext(f)[1] == ".i"]: - # found += recurse(f, searchPath) - for f in filter(lambda f: os.path.splitext(f)[1] == ".i", found): - found = found + recurse(f, searchPath) - return found - -def _scanSwig(node, env, path): - r = recurse(str(node), [os.path.abspath(os.path.dirname(str(node))), os.path.abspath(os.path.join("include", "swig"))]) - return r - def _swigEmitter(target, source, env): + swigflags = env.subst("$SWIGFLAGS", target=target, source=source) + flags = SCons.Util.CLVar(swigflags) for src in source: - src = str(src) - flags = SCons.Util.CLVar(env.subst("$SWIGFLAGS")) + src = str(src.rfile()) mnames = None if "-python" in flags and "-noproxy" not in flags: if mnames is None: @@ -94,9 +67,13 @@ def _swigEmitter(target, source, env): mnames = _reModule.findall(open(src).read()) java_files = map(lambda m: [m + ".java", m + "JNI.java"], mnames) java_files = SCons.Util.flatten(java_files) - outdir = env.subst('$SWIGOUTDIR') + outdir = env.subst('$SWIGOUTDIR', target=target, source=source) if outdir: java_files = map(lambda j, o=outdir: os.path.join(o, j), java_files) + java_files = map(env.fs.File, java_files) + for jf in java_files: + t_from_s = lambda t, p, s, x: t.dir + SCons.Util.AddMethod(jf, t_from_s, 'target_from_source') target.extend(java_files) return (target, source) @@ -112,13 +89,28 @@ def generate(env): cxx_file.add_action('.i', SwigAction) cxx_file.add_emitter('.i', _swigEmitter) + java_file = SCons.Tool.CreateJavaFileBuilder(env) + + java_file.suffix['.i'] = swigSuffixEmitter + + java_file.add_action('.i', SwigAction) + java_file.add_emitter('.i', _swigEmitter) + env['SWIG'] = 'swig' env['SWIGFLAGS'] = SCons.Util.CLVar('') env['SWIGCFILESUFFIX'] = '_wrap$CFILESUFFIX' env['SWIGCXXFILESUFFIX'] = '_wrap$CXXFILESUFFIX' - env['_SWIGOUTDIR'] = '${"-outdir " + SWIGOUTDIR}' - env['SWIGCOM'] = '$SWIG -o $TARGET ${_SWIGOUTDIR} $SWIGFLAGS $SOURCES' - env.Append(SCANNERS=Scanner(function=_scanSwig, skeys=[".i"])) + env['_SWIGOUTDIR'] = '${"-outdir " + str(SWIGOUTDIR)}' + env['SWIGPATH'] = [] + env['SWIGINCPREFIX'] = '-I' + env['SWIGINCSUFFIX'] = '' + env['_SWIGINCFLAGS'] = '$( ${_concat(SWIGINCPREFIX, SWIGPATH, SWIGINCSUFFIX, __env__, RDirs, TARGET, SOURCE)} $)' + env['SWIGCOM'] = '$SWIG -o $TARGET ${_SWIGOUTDIR} ${_SWIGINCFLAGS} $SWIGFLAGS $SOURCES' + + expr = '^[ \t]*%[ \t]*(?:include|import|extern)[ \t]*(<|"?)([^>\s"]+)(?:>|"?)' + scanner = SCons.Scanner.ClassicCPP("SWIGScan", ".i", "SWIGPATH", expr) + + env.Append(SCANNERS = scanner) def exists(env): return env.Detect(['swig']) diff --git a/src/engine/SCons/Tool/swig.xml b/src/engine/SCons/Tool/swig.xml index 679d683..580ef97 100644 --- a/src/engine/SCons/Tool/swig.xml +++ b/src/engine/SCons/Tool/swig.xml @@ -13,7 +13,11 @@ SWIG SWIGFLAGS SWIGCFILESUFFIX SWIGCXXFILESUFFIX +_SWIGINCFLAGS +SWIGINCPREFIX +SWIGINCSUFFIX SWIGCOM +SWIGPATH </sets> <uses> SWIGCOMSTR @@ -94,6 +98,36 @@ variable. </summary> </cvar> +<cvar name="_SWIGINCFLAGS"> +<summary> +An automatically-generated construction variable +containing the SWIG command-line options +for specifying directories to be searched for included files. +The value of &cv-_SWIGINCFLAGS; is created +by appending &cv-SWIGINCPREFIX; and &cv-SWIGINCSUFFIX; +to the beginning and end +of each directory in &cv-SWIGPATH;. +</summary> +</cvar> + +<cvar name="SWIGINCPREFIX"> +<summary> +The prefix used to specify an include directory on the SWIG command line. +This will be appended to the beginning of each directory +in the &cv-SWIGPATH; construction variable +when the &cv-_SWIGINCFLAGS; variable is automatically generated. +</summary> +</cvar> + +<cvar name="SWIGINCSUFFIX"> +<summary> +The suffix used to specify an include directory on the SWIG command line. +This will be appended to the end of each directory +in the &cv-SWIGPATH; construction variable +when the &cv-_SWIGINCFLAGS; variable is automatically generated. +</summary> +</cvar> + <cvar name="SWIGOUTDIR"> <summary> Specifies the output directory in which @@ -105,3 +139,55 @@ and translated into the <literal>swig -outdir</literal> option on the command line. </summary> </cvar> + +<cvar name="SWIGPATH"> +<summary> +The list of directories that the scripting language wrapper +and interface generate will search for included files. +The SWIG implicit dependency scanner will search these +directories for include files. +The default is to use the same path +specified as &cv-CPPPATH;. + +Don't explicitly put include directory +arguments in SWIGFLAGS; +the result will be non-portable +and the directories will not be searched by the dependency scanner. +Note: directory names in SWIGPATH will be looked-up relative to the SConscript +directory when they are used in a command. +To force +&scons; +to look-up a directory relative to the root of the source tree use #: + +<example> +env = Environment(SWIGPATH='#/include') +</example> + +The directory look-up can also be forced using the +&Dir;() +function: + +<example> +include = Dir('include') +env = Environment(SWIGPATH=include) +</example> + +The directory list will be added to command lines +through the automatically-generated +&cv-_SWIGINCFLAGS; +construction variable, +which is constructed by +appending the values of the +&cv-SWIGINCPREFIX; and &cv-SWIGINCSUFFIX; +construction variables +to the beginning and end +of each directory in &cv-SWIGPATH;. +Any command lines you define that need +the SWIGPATH directory list should +include &cv-_SWIGINCFLAGS;: + +<example> +env = Environment(SWIGCOM="my_swig -o $TARGET $_SWIGINCFLAGS $SORUCES") +</example> +</summary> +</cvar> diff --git a/src/engine/SCons/Tool/wix.py b/src/engine/SCons/Tool/wix.py index b133947..3aa3375 100644 --- a/src/engine/SCons/Tool/wix.py +++ b/src/engine/SCons/Tool/wix.py @@ -69,6 +69,9 @@ def exists(env): # add the install directory to light libpath. #for path in os.environ['PATH'].split(os.pathsep): for path in string.split(os.environ['PATH'], os.pathsep): + if not path: + continue + # workaround for some weird python win32 bug. if path[0] == '"' and path[-1:]=='"': path = path[1:-1] diff --git a/src/engine/SCons/Tool/yacc.py b/src/engine/SCons/Tool/yacc.py index cbccb29..34f60cb 100644 --- a/src/engine/SCons/Tool/yacc.py +++ b/src/engine/SCons/Tool/yacc.py @@ -43,7 +43,8 @@ import SCons.Util YaccAction = SCons.Action.Action("$YACCCOM", "$YACCCOMSTR") def _yaccEmitter(target, source, env, ysuf, hsuf): - flags = SCons.Util.CLVar(env.subst("$YACCFLAGS")) + yaccflags = env.subst("$YACCFLAGS", target=target, source=source) + flags = SCons.Util.CLVar(yaccflags) targetBase, targetExt = os.path.splitext(SCons.Util.to_String(target[0])) if '.ym' in ysuf: # If using Objective-C diff --git a/src/engine/SCons/Util.py b/src/engine/SCons/Util.py index 22aca08..3dfa287 100644 --- a/src/engine/SCons/Util.py +++ b/src/engine/SCons/Util.py @@ -312,16 +312,16 @@ def print_tree(root, child_func, prune=0, showtags=0, margin=[0], visited={}): return [" ","| "][m] margins = map(MMM, margin[:-1]) - if visited.has_key(rname): + children = child_func(root) + + if prune and visited.has_key(rname) and children: print string.join(tags + margins + ['+-[', rname, ']'], '') return print string.join(tags + margins + ['+-', rname], '') - if prune: - visited[rname] = 1 + visited[rname] = 1 - children = child_func(root) if children: margin.append(1) map(lambda C, cf=child_func, p=prune, i=IDX(showtags), m=margin, v=visited: @@ -404,6 +404,64 @@ def flatten(sequence, scalarp=is_Scalar, result=None): flatten(item, scalarp, result) return result + + +# The SCons "semi-deep" copy. +# +# This makes separate copies of lists (including UserList objects) +# dictionaries (including UserDict objects) and tuples, but just copies +# references to anything else it finds. +# +# A special case is any object that has a __semi_deepcopy__() method, +# which we invoke to create the copy, which is used by the BuilderDict +# class because of its extra initialization argument. +# +# The dispatch table approach used here is a direct rip-off from the +# normal Python copy module. + +_semi_deepcopy_dispatch = d = {} + +def _semi_deepcopy_dict(x): + copy = {} + for key, val in x.items(): + # The regular Python copy.deepcopy() also deepcopies the key, + # as follows: + # + # copy[semi_deepcopy(key)] = semi_deepcopy(val) + # + # Doesn't seem like we need to, but we'll comment it just in case. + copy[key] = semi_deepcopy(val) + return copy +d[types.DictionaryType] = _semi_deepcopy_dict + +def _semi_deepcopy_list(x): + return map(semi_deepcopy, x) +d[types.ListType] = _semi_deepcopy_list + +def _semi_deepcopy_tuple(x): + return tuple(map(semi_deepcopy, x)) +d[types.TupleType] = _semi_deepcopy_tuple + +def _semi_deepcopy_inst(x): + if hasattr(x, '__semi_deepcopy__'): + return x.__semi_deepcopy__() + elif isinstance(x, UserDict): + return x.__class__(_semi_deepcopy_dict(x)) + elif isinstance(x, UserList): + return x.__class__(_semi_deepcopy_list(x)) + else: + return x +d[types.InstanceType] = _semi_deepcopy_inst + +def semi_deepcopy(x): + copier = _semi_deepcopy_dispatch.get(type(x)) + if copier: + return copier(x) + else: + return x + + + class Proxy: """A simple generic Proxy class, forwarding all calls to subject. So, for the benefit of the python newbie, what does @@ -1044,4 +1102,35 @@ def RenameFunction(function, name): +# From Dinu C. Gherman, +# Python Cookbook, second edition, recipe 6.17, p. 277. +# Also: +# http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/68205 +# ASPN: Python Cookbook: Null Object Design Pattern + +class Null: + """ Null objects always and reliably "do nothging." """ + + def __new__(cls, *args, **kwargs): + if not '_inst' in vars(cls): + #cls._inst = type.__new__(cls, *args, **kwargs) + cls._inst = apply(type.__new__, (cls,) + args, kwargs) + return cls._inst + def __init__(self, *args, **kwargs): + pass + def __call__(self, *args, **kwargs): + return self + def __repr__(self): + return "Null()" + def __nonzero__(self): + return False + def __getattr__(self, mname): + return self + def __setattr__(self, name, value): + return self + def __delattr__(self, name): + return self + + + del __revision__ diff --git a/src/engine/SCons/UtilTests.py b/src/engine/SCons/UtilTests.py index db6d9f6..1149f35 100644 --- a/src/engine/SCons/UtilTests.py +++ b/src/engine/SCons/UtilTests.py @@ -106,7 +106,7 @@ class UtilTestCase(unittest.TestCase): return foo, expect, withtags - def tree_case_2(self): + def tree_case_2(self, prune=1): """Fixture for the render_tree() and print_tree() tests.""" stdlib_h = self.Node("stdlib.h") @@ -124,6 +124,10 @@ class UtilTestCase(unittest.TestCase): +-[stdlib.h] """ + if not prune: + expect = string.replace(expect, '[', '') + expect = string.replace(expect, ']', '') + lines = string.split(expect, '\n')[:-1] lines = map(lambda l: '[E BSPACN ]'+l, lines) withtags = string.join(lines, '\n') + '\n' @@ -163,7 +167,7 @@ class UtilTestCase(unittest.TestCase): actual = sys.stdout.getvalue() assert withtags == actual, (withtags, actual) - node, expect, withtags = self.tree_case_2() + node, expect, withtags = self.tree_case_2(prune=0) sys.stdout = StringIO.StringIO() print_tree(node, get_children, 1) |