diff options
author | Steven Knight <knight@baldmt.com> | 2009-01-09 16:43:32 (GMT) |
---|---|---|
committer | Steven Knight <knight@baldmt.com> | 2009-01-09 16:43:32 (GMT) |
commit | 7ab76c68556e5f6f142515872ea6334e959b8626 (patch) | |
tree | c4d23aed9df4381a24cac247b11dd1a4c245908a /doc | |
parent | e04fb604484cf37da383a38ef9b2bd8c9ef6c175 (diff) | |
download | SCons-7ab76c68556e5f6f142515872ea6334e959b8626.zip SCons-7ab76c68556e5f6f142515872ea6334e959b8626.tar.gz SCons-7ab76c68556e5f6f142515872ea6334e959b8626.tar.bz2 |
Issue 1086: add support for generic batch build actions, and
specific support for batched compilation for Microsoft Visual C/C++.
Merged revisions 3819-3851,3854-3869,3871-3877,3880 via svnmerge from
http://scons.tigris.org/svn/scons/branches/sgk_batch
........
r3820 | stevenknight | 2008-12-09 23:59:14 -0800 (Tue, 09 Dec 2008) | 6 lines
Issue 1086: Batch compilation support:
* $MSVC_BATCH to control Visual C/C++ batch compilation.
* New $CHANGED_SOURCES, $CHANGED_TARGETS, $UNCHANGED_SOURCES and
$UNCHANGED_TARGETS construction variables.
* New Action(batch_key=, targets=) keyword arguments.
........
r3880 | stevenknight | 2009-01-07 20:50:41 -0800 (Wed, 07 Jan 2009) | 3 lines
Use UniqueList objects to collect the all_children(), all_prerequisites()
and all_sources() lists instead of calling uniquer_hashables() by hand.
........
Diffstat (limited to 'doc')
-rw-r--r-- | doc/man/scons.1 | 171 | ||||
-rw-r--r-- | doc/scons.mod | 15 | ||||
-rw-r--r-- | doc/user/actions.in | 182 | ||||
-rw-r--r-- | doc/user/actions.xml | 182 | ||||
-rw-r--r-- | doc/user/builders-writing.in | 293 |
5 files changed, 682 insertions, 161 deletions
diff --git a/doc/man/scons.1 b/doc/man/scons.1 index 5526c5e..d1f9dcd 100644 --- a/doc/man/scons.1 +++ b/doc/man/scons.1 @@ -1481,11 +1481,15 @@ These warnings are enabled by default. --warn=reserved-variable, --warn=no-reserved-variable Enables or disables warnings about attempts to set the reserved construction variable names +.BR CHANGED_SOURCES , +.BR CHANGED_TARGETS , .BR TARGET , .BR TARGETS , -.BR SOURCE +.BR SOURCE , +.BR SOURCES , +.BR UNCHANGED_SOURCES or -.BR SOURCES . +.BR UNCHANGED_TARGETS . These warnings are disabled by default. .TP @@ -8644,16 +8648,25 @@ a = Action(build_it, varlist=['XXX']) The .BR Action () global function -also takes a +can be passed the following +optional keyword arguments +to modify the Action object's behavior: + +.IP .B chdir -keyword argument -which specifies that +The +.B chdir +keyword argument specifies that scons will execute the action after changing to the specified directory. -If the chdir argument is +If the +.B chdir +argument is a string or a directory Node, scons will change to the specified directory. -If the chdir argument +If the +.B chdir +argument is not a string or Node and is non-zero, then scons will change to the @@ -8688,6 +8701,8 @@ a = Action("build < ${SOURCE.file} > ${TARGET.file}", chdir=1) .EE +.IP +.B exitstatfunc The .BR Action () global function @@ -8703,6 +8718,7 @@ or modified value. This can be used, for example, to specify that an Action object's return value should be ignored +under special conditions and SCons should, therefore, consider that the action always suceeds: @@ -8714,6 +8730,111 @@ a = Action("build < ${SOURCE.file} > ${TARGET.file}", exitstatfunc=always_succeed) .EE +.IP +.B batch_key +The +.B batch_key +keyword argument can be used +to specify that the Action can create multiple target files +by processing multiple independent source files simultaneously. +(The canonical example is "batch compilation" +of multiple object files +by passing multiple source files +to a single invocation of a compiler +such as Microsoft's Visual C / C++ compiler.) +If the +.B batch_key +argument is any non-False, non-callable Python value, +the configured Action object will cause +.B scons +to collect all targets built with the Action object +and configured with the same construction environment +into single invocations of the Action object's +command line or function. +Command lines will typically want to use the +.BR CHANGED_SOURCES +construction variable +(and possibly +.BR CHANGED_TARGETS +as well) +to only pass to the command line those sources that +have actually changed since their targets were built. + +Example: + +.ES +a = Action('build $CHANGED_SOURCES', batch_key=True) +.EE + +The +.B batch_key +argument may also be +a callable function +that returns a key that +will be used to identify different +"batches" of target files to be collected +for batch building. +A +.B batch_key +function must take the following arguments: + +.IP action +The action object. + +.IP env +The construction environment +configured for the target. + +.IP target +The list of targets for a particular configured action. + +.IP source +The list of source for a particular configured action. + +The returned key should typically +be a tuple of values derived from the arguments, +using any appropriate logic to decide +how multiple invocations should be batched. +For example, a +.B batch_key +function may decide to return +the value of a specific construction +variable from the +.B env +argument +which will cause +.B scons +to batch-build targets +with matching values of that variable, +or perhaps return the +.BR id () +of the entire construction environment, +in which case +.B scons +will batch-build +all targets configured with the same construction environment. +Returning +.B None +indicates that +the particular target should +.I not +be part of any batched build, +but instead will be built +by a separate invocation of action's +command or function. +Example: + +.ES +def batch_key(action, env, target, source): + tdir = target[0].dir + if tdir.name == 'special': + # Don't batch-build any target + # in the special/ subdirectory. + return None + return (id(action), id(env), tdir) +a = Action('build $CHANGED_SOURCES', batch_key=batch_key) +.EE + .SS Miscellaneous Action Functions .B scons @@ -8906,20 +9027,42 @@ prefix. Besides construction variables, scons provides the following variables for each command execution: -.IP TARGET -The file name of the target being built, or the file name of the first -target if multiple targets are being built. +.IP CHANGED_SOURCES +The file names of all sources of the build command +that have changed since the target was last built. -.IP TARGETS -The file names of all targets being built. +.IP CHANGED_TARGETS +The file names of all targets that would be built +from sources that have changed since the target was last built. .IP SOURCE -The file name of the source of the build command, or the file name of the -first source if multiple sources are being built. +The file name of the source of the build command, +or the file name of the first source +if multiple sources are being built. .IP SOURCES The file names of the sources of the build command. +.IP TARGET +The file name of the target being built, +or the file name of the first target +if multiple targets are being built. + +.IP TARGETS +The file names of all targets being built. + +.IP UNCHANGED_SOURCES +The file names of all sources of the build command +that have +.I not +changed since the target was last built. + +.IP UNCHANGED_TARGETS +The file names of all targets that would be built +from sources that have +.I not +changed since the target was last built. + (Note that the above variables are reserved and may not be set in a construction environment.) diff --git a/doc/scons.mod b/doc/scons.mod index 767b40c..a2927cf 100644 --- a/doc/scons.mod +++ b/doc/scons.mod @@ -401,6 +401,8 @@ --> <!ENTITY buildfunc "<literal>builder function</literal>"> +<!ENTITY build_action "<literal>build action</literal>"> +<!ENTITY build_actions "<literal>build actions</literal>"> <!ENTITY builder_method "<literal>builder method</literal>"> <!ENTITY Configure_Contexts "<literal>Configure Contexts</literal>"> @@ -444,6 +446,19 @@ <!-- + Python keyword arguments + +--> + +<!ENTITY action "<literal>action=</literal>"> +<!ENTITY batch_key "<literal>batch_key=</literal>"> +<!ENTITY cmdstr "<literal>cmdstr=</literal>"> +<!ENTITY exitstatfunc "<literal>exitstatfunc=</literal>"> +<!ENTITY strfunction "<literal>strfunction=</literal>"> +<!ENTITY varlist "<literal>varlist=</literal>"> + +<!-- + File and program names used in examples. --> diff --git a/doc/user/actions.in b/doc/user/actions.in index 928b7ea..c1e5616 100644 --- a/doc/user/actions.in +++ b/doc/user/actions.in @@ -222,19 +222,183 @@ solutions to the above limitations. --> - <para> + <para> - XXX + &SCons; supports several types of &build_actions; + that can be performed to build one or more target files. + Usually, a &build_action; is a command-line string + that invokes an external command. + A build action can also be an external command + specified as a list of arguments, + or even a Python function. - </para> + </para> - <section> - <title>XXX</title> + <para> - <para> + Build action objects are created by the &Action; function. + This function is, in fact, what &SCons; uses + to interpret the &action; + keyword argument when you call the &Builder; function. + So the following line that creates a simple Builder: - XXX + </para> - </para> + <sconstruct> + b = Builder(action = 'build < $SOURCE > $TARGET') + </sconstruct> - </section> + <para> + + Is equivalent to: + + </para> + + <sconstruct> + b = Builder(action = Action('build < $SOURCE > $TARGET')) + </sconstruct> + + <para> + + The advantage of using the &Action; function directly + is that it can take a number of additional options + to modify the action's behavior in many useful ways. + + </para> + + <section> + <title>Command Strings as Actions</title> + + <section> + <title>Suppressing Command-Line Printing</title> + + <para> + + XXX + + </para> + + </section> + + <section> + <title>Ignoring Exit Status</title> + + <para> + + XXX + + </para> + + </section> + + </section> + + <section> + <title>Argument Lists as Actions</title> + + <para> + + XXX + + </para> + + </section> + + <section> + <title>Python Functions as Actions</title> + + <para> + + XXX + + </para> + + </section> + + <section> + <title>Modifying How an Action is Printed</title> + + <section> + <title>XXX: the &strfunction; keyword argument</title> + + <para> + + XXX + + </para> + + </section> + + <section> + <title>XXX: the &cmdstr; keyword argument</title> + + <para> + + XXX + + </para> + + </section> + + </section> + + <section> + <title>Making an Action Depend on Variable Contents: the &varlist; keyword argument</title> + + <para> + + XXX + + </para> + + </section> + + <section> + <title>chdir=1</title> + + <para> + + XXX + + </para> + + </section> + + <section> + <title>Batch Building of Multiple Targets from Separate Sources: the &batch_key; keyword argument</title> + + <para> + + XXX + + </para> + + </section> + + <section> + <title>Manipulating the Exit Status of an Action: the &exitstatfunc; keyword argument</title> + + <para> + + XXX + + </para> + + </section> + + <!-- + + ??? + + <section> + <title>presub=</title> + + <para> + + XXX + + </para> + + </section> + + --> diff --git a/doc/user/actions.xml b/doc/user/actions.xml index 928b7ea..04178b0 100644 --- a/doc/user/actions.xml +++ b/doc/user/actions.xml @@ -222,19 +222,183 @@ solutions to the above limitations. --> - <para> + <para> - XXX + &SCons; supports several types of &build_actions; + that can be performed to build one or more target files. + Usually, a &build_action; is a command-line string + that invokes an external command. + A build action can also be an external command + specified as a list of arguments, + or even a Python function. - </para> + </para> - <section> - <title>XXX</title> + <para> - <para> + Build action objects are created by the &Action; function. + This function is, in fact, what &SCons; uses + to interpret the &action; + keyword argument when you call the &Builder; function. + So the following line that creates a simple Builder: - XXX + </para> - </para> + <programlisting> + b = Builder(action = 'build < $SOURCE > $TARGET') + </programlisting> - </section> + <para> + + Is equivalent to: + + </para> + + <programlisting> + b = Builder(action = Action('build < $SOURCE > $TARGET')) + </programlisting> + + <para> + + The advantage of using the &Action; function directly + is that it can take a number of additional options + to modify the action's behavior in many useful ways. + + </para> + + <section> + <title>Command Strings as Actions</title> + + <section> + <title>Suppressing Command-Line Printing</title> + + <para> + + XXX + + </para> + + </section> + + <section> + <title>Ignoring Exit Status</title> + + <para> + + XXX + + </para> + + </section> + + </section> + + <section> + <title>Argument Lists as Actions</title> + + <para> + + XXX + + </para> + + </section> + + <section> + <title>Python Functions as Actions</title> + + <para> + + XXX + + </para> + + </section> + + <section> + <title>Modifying How an Action is Printed</title> + + <section> + <title>XXX: the &strfunction; keyword argument</title> + + <para> + + XXX + + </para> + + </section> + + <section> + <title>XXX: the &cmdstr; keyword argument</title> + + <para> + + XXX + + </para> + + </section> + + </section> + + <section> + <title>Making an Action Depend on Variable Contents: the &varlist; keyword argument</title> + + <para> + + XXX + + </para> + + </section> + + <section> + <title>chdir=1</title> + + <para> + + XXX + + </para> + + </section> + + <section> + <title>Batch Building of Multiple Targets from Separate Sources: the &batch_key; keyword argument</title> + + <para> + + XXX + + </para> + + </section> + + <section> + <title>Manipulating the Exit Status of an Action: the &exitstatfunc; keyword argument</title> + + <para> + + XXX + + </para> + + </section> + + <!-- + + ??? + + <section> + <title>presub=</title> + + <para> + + XXX + + </para> + + </section> + + --> diff --git a/doc/user/builders-writing.in b/doc/user/builders-writing.in index dc6e95b..2460b37 100644 --- a/doc/user/builders-writing.in +++ b/doc/user/builders-writing.in @@ -102,7 +102,7 @@ This functionality could be invoked as in the following example: programs, libraries, documents. you frequently want to be able to build some other type of file - not supported directly by &SCons; + not supported directly by &SCons;. Fortunately, &SCons; makes it very easy to define your own &Builder; objects for any custom file types you want to build. @@ -186,7 +186,8 @@ This functionality could be invoked as in the following example: <para> - With the &Builder; so attached to our &consenv; + With the &Builder; attached to our &consenv; + with the name &Foo;, we can now actually call it like so: </para> @@ -859,164 +860,198 @@ This functionality could be invoked as in the following example: </section> + <!-- + + <section> + <title>target_factor=, source_factory=</title> + + </section> + + <section> + <title>target_scanner=, source_scanner=</title> + + </section> + + <section> + <title>multi=</title> + + </section> + + <section> + <title>single_source=</title> + + </section> + + <section> + <title>src_builder=</title> + + </section> + + <section> + <title>ensure_suffix=</title> + + </section> + + --> + <section> <title>Where To Put Your Custom Builders and Tools</title> - <para> + <para> - The <filename>site_scons</filename> directory gives you a place to - put Python modules you can import into your SConscripts - (site_scons), add-on tools that can integrate into &SCons; - (site_scons/site_tools), and a site_scons/site_init.py file that - gets read before any &SConstruct; or &SConscript;, allowing you to - change &SCons;'s default behavior. + The <filename>site_scons</filename> directory gives you a place to + put Python modules you can import into your SConscripts + (site_scons), add-on tools that can integrate into &SCons; + (site_scons/site_tools), and a site_scons/site_init.py file that + gets read before any &SConstruct; or &SConscript;, allowing you to + change &SCons;'s default behavior. - </para> + </para> - <para> + <para> - If you get a tool from somewhere (the &SCons; wiki or a third party, - for instance) and you'd like to use it in your project, the - <filename>site_scons</filename> dir is the simplest place to put it. - Tools come in two flavors; either a Python function that operates on - an &Environment; or a Python file containing two functions, exists() - and generate(). + If you get a tool from somewhere (the &SCons; wiki or a third party, + for instance) and you'd like to use it in your project, the + <filename>site_scons</filename> dir is the simplest place to put it. + Tools come in two flavors; either a Python function that operates on + an &Environment; or a Python file containing two functions, exists() + and generate(). - </para> + </para> - <para> + <para> - A single-function Tool can just be included in your - <filename>site_scons/site_init.py</filename> file where it will be - parsed and made available for use. For instance, you could have a - <filename>site_scons/site_init.py</filename> file like this: + A single-function Tool can just be included in your + <filename>site_scons/site_init.py</filename> file where it will be + parsed and made available for use. For instance, you could have a + <filename>site_scons/site_init.py</filename> file like this: - </para> + </para> - <scons_example name="site1"> - <file name="site_scons/site_init.py" printme=1> - def TOOL_ADD_HEADER(env): - """A Tool to add a header from $HEADER to the source file""" - add_header = Builder(action=['echo "$HEADER" > $TARGET', - 'cat $SOURCE >> $TARGET']) - env.Append(BUILDERS = {'AddHeader' : add_header}) - env['HEADER'] = '' # set default value - </file> - <file name="SConstruct"> - env=Environment(tools=['default', TOOL_ADD_HEADER], HEADER="=====") - env.AddHeader('tgt', 'src') - </file> - <file name="src"> - hi there - </file> - </scons_example> + <scons_example name="site1"> + <file name="site_scons/site_init.py" printme=1> + def TOOL_ADD_HEADER(env): + """A Tool to add a header from $HEADER to the source file""" + add_header = Builder(action=['echo "$HEADER" > $TARGET', + 'cat $SOURCE >> $TARGET']) + env.Append(BUILDERS = {'AddHeader' : add_header}) + env['HEADER'] = '' # set default value + </file> + <file name="SConstruct"> + env=Environment(tools=['default', TOOL_ADD_HEADER], HEADER="=====") + env.AddHeader('tgt', 'src') + </file> + <file name="src"> + hi there + </file> + </scons_example> - <para> + <para> - and a &SConstruct; like this: + and a &SConstruct; like this: - </para> + </para> - <sconstruct> - # Use TOOL_ADD_HEADER from site_scons/site_init.py - env=Environment(tools=['default', TOOL_ADD_HEADER], HEADER="=====") - env.AddHeader('tgt', 'src') - </sconstruct> + <sconstruct> + # Use TOOL_ADD_HEADER from site_scons/site_init.py + env=Environment(tools=['default', TOOL_ADD_HEADER], HEADER="=====") + env.AddHeader('tgt', 'src') + </sconstruct> - <para> + <para> - The <function>TOOL_ADD_HEADER</function> tool method will be - called to add the <function>AddHeader</function> tool to the - environment. + The <function>TOOL_ADD_HEADER</function> tool method will be + called to add the <function>AddHeader</function> tool to the + environment. - </para> + </para> - <!-- - <scons_output example="site1" os="posix"> - <scons_output_command>scons -Q</scons_output_command> - </scons_output> - --> + <!-- + <scons_output example="site1" os="posix"> + <scons_output_command>scons -Q</scons_output_command> + </scons_output> + --> - <para> - Similarly, a more full-fledged tool with - <function>exists()</function> and <function>generate()</function> - methods can be installed in - <filename>site_scons/site_tools/toolname.py</filename>. Since - <filename>site_scons/site_tools</filename> is automatically added - to the head of the tool search path, any tool found there will be - available to all environments. Furthermore, a tool found there - will override a built-in tool of the same name, so if you need to - change the behavior of a built-in tool, site_scons gives you the - hook you need. - </para> + <para> + Similarly, a more full-fledged tool with + <function>exists()</function> and <function>generate()</function> + methods can be installed in + <filename>site_scons/site_tools/toolname.py</filename>. Since + <filename>site_scons/site_tools</filename> is automatically added + to the head of the tool search path, any tool found there will be + available to all environments. Furthermore, a tool found there + will override a built-in tool of the same name, so if you need to + change the behavior of a built-in tool, site_scons gives you the + hook you need. + </para> - <para> - Many people have a library of utility Python functions they'd like - to include in &SConscript;s; just put that module in - <filename>site_scons/my_utils.py</filename> or any valid Python module name of your - choice. For instance you can do something like this in - <filename>site_scons/my_utils.py</filename> to add build_id and MakeWorkDir functions: - </para> - - <scons_example name="site2"> - <file name="site_scons/my_utils.py" printme=1> - from SCons.Script import * # for Execute and Mkdir - def build_id(): - """Return a build ID (stub version)""" - return "100" - def MakeWorkDir(workdir): - """Create the specified dir immediately""" - Execute(Mkdir(workdir)) - </file> - <file name="SConscript"> - import my_utils - MakeWorkDir('/tmp/work') - print "build_id=" + my_utils.build_id() - </file> - </scons_example> + <para> + Many people have a library of utility Python functions they'd like + to include in &SConscript;s; just put that module in + <filename>site_scons/my_utils.py</filename> or any valid Python module name of your + choice. For instance you can do something like this in + <filename>site_scons/my_utils.py</filename> to add build_id and MakeWorkDir functions: + </para> + + <scons_example name="site2"> + <file name="site_scons/my_utils.py" printme=1> + from SCons.Script import * # for Execute and Mkdir + def build_id(): + """Return a build ID (stub version)""" + return "100" + def MakeWorkDir(workdir): + """Create the specified dir immediately""" + Execute(Mkdir(workdir)) + </file> + <file name="SConscript"> + import my_utils + MakeWorkDir('/tmp/work') + print "build_id=" + my_utils.build_id() + </file> + </scons_example> - <para> + <para> - And then in your &SConscript; or any sub-&SConscript; anywhere in - your build, you can import <filename>my_utils</filename> and use it: + And then in your &SConscript; or any sub-&SConscript; anywhere in + your build, you can import <filename>my_utils</filename> and use it: - </para> + </para> - <sconstruct> - import my_utils - print "build_id=" + my_utils.build_id() - my_utils.MakeWorkDir('/tmp/work') - </sconstruct> + <sconstruct> + import my_utils + print "build_id=" + my_utils.build_id() + my_utils.MakeWorkDir('/tmp/work') + </sconstruct> - <para> - Note that although you can put this library in - <filename>site_scons/site_init.py</filename>, - it is no better there than <filename>site_scons/my_utils.py</filename> - since you still have to import that module into your &SConscript;. - Also note that in order to refer to objects in the SCons namespace - such as &Environment; or &Mkdir; or &Execute; in any file other - than a &SConstruct; or &SConscript; you always need to do - </para> - <sconstruct> - from SCons.Script import * - </sconstruct> + <para> + Note that although you can put this library in + <filename>site_scons/site_init.py</filename>, + it is no better there than <filename>site_scons/my_utils.py</filename> + since you still have to import that module into your &SConscript;. + Also note that in order to refer to objects in the SCons namespace + such as &Environment; or &Mkdir; or &Execute; in any file other + than a &SConstruct; or &SConscript; you always need to do + </para> + <sconstruct> + from SCons.Script import * + </sconstruct> - <para> - This is true in modules in <filename>site_scons</filename> such as - <filename>site_scons/site_init.py</filename> as well. - </para> + <para> + This is true in modules in <filename>site_scons</filename> such as + <filename>site_scons/site_init.py</filename> as well. + </para> - <para> + <para> - If you have a machine-wide site dir you'd like to use instead of - <filename>./site_scons</filename>, use the - <literal>--site-dir</literal> option to point to your dir. - <filename>site_init.py</filename> and - <filename>site_tools</filename> will be located under that dir. - To avoid using a <filename>site_scons</filename> dir at all, even - if it exists, use the <literal>--no-site-dir</literal> option. + If you have a machine-wide site dir you'd like to use instead of + <filename>./site_scons</filename>, use the + <literal>--site-dir</literal> option to point to your dir. + <filename>site_init.py</filename> and + <filename>site_tools</filename> will be located under that dir. + To avoid using a <filename>site_scons</filename> dir at all, even + if it exists, use the <literal>--no-site-dir</literal> option. - </para> + </para> </section> |