<!--
Copyright (c) 2001, 2002, 2003 Steven Knight
Permission is hereby granted, free of charge, to any person obtaining
a copy of this software and associated documentation files (the
"Software"), to deal in the Software without restriction, including
without limitation the rights to use, copy, modify, merge, publish,
distribute, sublicense, and/or sell copies of the Software, and to
permit persons to whom the Software is furnished to do so, subject to
the following conditions:
The above copyright notice and this permission notice shall be included
in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY
KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE
WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
-->
<section id="sect-architecture">
<title>Architecture</title>
<para>
The heart of &SCons; is its <emphasis>Build Engine</emphasis>.
The &SCons; Build Engine is a Python module
that manages dependencies between
external objects
such as files or database records.
The Build Engine is designed to
be interface-neutral
and easily embeddable in any
software system that needs dependency
analysis between updatable objects.
</para>
<para>
The key parts of the Build Engine architecture
are captured in the following quasi-UML diagram:
</para>
<REMARK>
Including this figure makes our PDF build blow up.
The figure, however,
is left over from the Software Carpentry contest
and is therefore old, out-of-date, and needs to be redone anyway.
This is where it will go, anyway...
</REMARK>
<!--
YARG! THIS MAKES THE PDF BUILD BLOW UP. HELP!
<figure>
<title>&SCons; Architecture</title>
<graphic fileref="engine.jpg">
</figure>
-->
<para>
The point of &SCons; is to manage
dependencies between arbitrary external objects.
Consequently, the Build Engine does not restrict or specify
the nature of the external objects it manages,
but instead relies on subclass of the &Node;
class to interact with the external system or systems
(file systems, database management systems)
that maintain the objects being examined or updated.
</para>
<para>
The Build Engine presents to the software system in
which it is embedded
a Python API for specifying source (input) and target (output) objects,
rules for building/updating objects,
rules for scanning objects for dependencies, etc.
Above its Python API,
the Build Engine is completely
interface-independent,
and can be encapsulated by any other software
that supports embedded Python.
</para>
<para>
Software that chooses to use the Build Engine
for dependency management
interacts with it
through <emphasis>Construction Environments</emphasis>.
A Construction Environment consists
of a dictionary of environment variables,
and one or more associated
&Scanner; objects
and &Builder; objects.
The Python API is used to
form these associations.
</para>
<para>
A &Scanner; object specifies
how to examine a type of source object
(C source file, database record)
for dependency information.
A &Scanner; object may use
variables from the associated
Construction Environment
to modify how it scans an object:
specifying a search path for included files,
which field in a database record to consult,
etc.
</para>
<para>
A &Builder; object specifies
how to update a type of target object:
executable program, object file, database field, etc.
Like a &Scanner; object,
a &Builder; object may use
variables from the associated
Construction Environment
to modify how it builds an object:
specifying flags to a compiler,
using a different update function,
etc.
</para>
<para>
&Scanner; and &Builder; objects will return one or more
&Node; objects that represent external objects.
&Node; objects are the means by which the
Build Engine tracks dependencies:
A &Node; may represent a source (input) object that
should already exist,
or a target (output) object which may be built,
or both.
The &Node; class is sub-classed to
represent external objects of specific type:
files, directories, database fields or records, etc.
Because dependency information, however,
is tracked by the top-level &Node; methods and attributes,
dependencies can exist
between nodes representing different external object types.
For example,
building a file could be made
dependent on the value of a given
field in a database record,
or a database table could depend
on the contents of an external file.
</para>
<para>
The Build Engine uses a &Job; class (not displayed)
to manage the actual work of updating external target objects:
spawning commands to build files,
submitting the necessary commands to update a database record,
etc.
The &Job; class has sub-classes
to handle differences between spawning
jobs in parallel and serially.
</para>
<para>
The Build Engine also uses a
&Signature; class (not displayed)
to maintain information about whether
an external object is up-to-date.
Target objects with out-of-date signatures
are updated using the appropriate
&Builder; object.
</para>
<!-- BEGIN HTML -->
<!--
Details on the composition, methods,
and attributes of these classes
are available in the A HREF="internals.html" Internals /A page.
-->
<!-- END HTML -->
</section>
<section id="sect-engine">
<title>Build Engine</title>
<para>
More detailed discussion of some of the
Build Engine's characteristics:
</para>
<section>
<title>Python API</title>
<para>
The Build Engine can be embedded in any other software
that supports embedding Python:
in a GUI,
in a wrapper script that
interprets classic <filename>Makefile</filename> syntax,
or in any other software that
can translate its dependency representation
into the appropriate calls to the Build Engine API.
<!--<xref linkend="chap-native">--> describes in detail
the specification for a "Native Python" interface
that will drive the &SCons; implementation effort.
</para>
</section>
<section>
<title>Single-image execution</title>
<para>
When building/updating the objects,
the Build Engine operates as a single executable
with a complete Directed Acyclic Graph (DAG)
of the dependencies in the entire build tree.
This is in stark contrast to the
commonplace recursive use of Make
to handle hierarchical directory-tree builds.
</para>
</section>
<section>
<title>Dependency analysis</title>
<para>
Dependency analysis is carried out via digital signatures
(a.k.a. "fingerprints").
Contents of object are examined and reduced
to a number that can be stored and compared to
see if the object has changed.
Additionally, &SCons; uses the same
signature technique on the command-lines that
are executed to update an object.
If the command-line has changed since the last time,
then the object must be rebuilt.
</para>
</section>
<section>
<title>Customized output</title>
<para>
The output of Build Engine is customizable
through user-defined functions.
This could be used to print additional desired
information about what &SCons; is doing,
or tailor output to a specific build analyzer,
GUI, or IDE.
</para>
</section>
<section>
<title>Build failures</title>
<para>
&SCons; detects build failures via the exit status from the tools
used to build the target files. By default, a failed exit status
(non-zero on UNIX systems) terminates the build with an appropriate
error message. An appropriate class from the Python library will
interpret build-tool failures via an OS-independent API.
</para>
<para>
If multiple tasks are executing in a parallel build, and one tool
returns failure, &SCons; will not initiate any further build tasks,
but allow the other build tasks to complete before terminating.
</para>
<para>
A <option>-k</option> command-line option may be used to ignore
errors and continue building other targets. In no case will a target
that depends on a failed build be rebuilt.
</para>
</section>
</section>
<section id="sect-interfaces">
<title>Interfaces</title>
<para>
As previously described,
the &SCons; Build Engine
is interface-independent above its Python API,
and can be embedded in any software system
that can translate its dependency requirements
into the necessary Python calls.
</para>
<para>
The "main" &SCons; interface
for implementation purposes,
uses Python scripts as configuration files.
Because this exposes the Build Engine's Python API to the user,
it is current called the "Native Python" interface.
</para>
<para>
This section will also discuss
how &SCons; will function in the context
of two other interfaces:
the &Makefile; interface of the classic &Make; utility,
and a hypothetical graphical user interface (GUI).
</para>
<section>
<title>Native Python interface</title>
<para>
The Native Python interface is intended to be the primary interface
by which users will know &SCons;--that is,
it is the interface they will use
if they actually type &SCons; at a command-line prompt.
</para>
<para>
In the Native Python interface, &SCons; configuration files are simply
Python scripts that directly invoke methods from the Build Engine's
Python API to specify target files to be built, rules for building
the target files, and dependencies. Additional methods, specific to
this interface, are added to handle functionality that is specific to
the Native Python interface: reading a subsidiary configuration file;
copying target files to an installation directory; etc.
</para>
<para>
Because configuration files are Python scripts, Python flow control
can be used to provide very flexible manipulation of objects and
dependencies. For example, a function could be used to invoke a common
set of methods on a file, and called iteratively over an array of
files.
</para>
<para>
As an additional advantage, syntax errors in &SCons; Native Python
configuration files will be caught by the Python parser. Target-building
does not begin until after all configuration files are read, so a syntax
error will not cause a build to fail half-way.
</para>
</section>
<section>
<title>Makefile interface</title>
<para>
An alternate &SCons; interface would provide backwards
compatibility with the classic &Make utility.
This would be done by embedding the &SCons; Build Engine
in a Python script that can translate existing
&Makefile;s into the underlying calls to the
Build Engine's Python API
for building and tracking dependencies.
Here are approaches to solving some of the issues
that arise from marrying these two pieces:
</para>
<itemizedlist>
<listitem>
<para>
&Makefile; suffix rules can be translated
into an appropriate &Builder; object
with suffix maps from the Construction Environment.
</para>
</listitem>
<listitem>
<para>
Long lists of static dependences
appended to a &Makefile; by
various <command>"make depend"</command> schemes
can be preserved
but supplemented by
the more accurate dependency information
provided by &Scanner; objects.
</para>
</listitem>
<listitem>
<para>
Recursive invocations of &Make;
can be avoided by reading up
the subsidiary &Makefile; instead.
</para>
</listitem>
</itemizedlist>
<para>
Lest this seem like too outlandish an undertaking,
there is a working example of this approach:
Gary Holt's &Makepp; utility
is a Perl script that provides
admirably complete parsing of complicated &Makefile;s
around an internal build engine inspired,
in part, by the classic <application>Cons</application> utility.
</para>
</section>
<section>
<title>Graphical interfaces</title>
<para>
The &SCons; Build Engine
is designed from the ground up to be embedded
into multiple interfaces.
Consequently, embedding the dependency capabilities
of &SCons; into graphical interface
would be a matter of mapping the
GUI's dependency representation
(either implicit or explicit)
into corresponding calls to the Python API
of the &SCons; Build Engine.
</para>
<para>
Note, however, that this proposal leaves the problem of
designed a good graphical interface
for representing software build dependencies
to people with actual GUI design experience...
</para>
</section>
</section>