The &SCons; architecture consists of three layers:
The &SCons; Build Engine, a package of Python
modules that handle dependency management and updating out-of-date
objects.
The &SCons; API (applications programming
interface) between the Build Engine
and the user interface.
The &scons; script itself (note lower case
sc), which is the pre-provided interface to
the Build Engine.
Notice that this architecture separates the internal workings of
&SCons; (the Build Engine) from the
external user interface. The benefit is that the &SCons; Build Engine
can be imported into any other software package written in Python
to support a variety of user interfaces—or, to look at it
in reverse, other software interfaces can use the &SCons; Build
Engine to manage dependencies between their objects.
Because the
&SCons; package itself is modular, only those parts of the package
relevant to the embedding interface need be imported; for example,
a utility that wants to use only file timestamps for checking
whether a file is up-to-date
need not import the MD5 signature module.
The &SCons; Build Engine
The Build Engine is a package of Python modules that
form the heart of &SCons;.
The Build Engine can be broadly divided into five
architectural subsystems, each responsible
for a crucial part of &SCons; functionality:
A node subsystem, responsible for managing
the files (or other objects) to be built, and the dependency
relationships between them.
A scanner subsystem, responsible for
scanning various file types for implicit dependencies.
A signature subsystem, responsible for
deciding whether a given file (or other object) requires
rebuilding.
A builder subsystem, responsible for
actually executing the necessary command or function to
build a file (or other object).
A job/task subsystem, responsible for
handling parallelization of builds.
The rest of this section will provide a high-level overview of the
class structure of each of these Build Engine subsystems.
Node Subsystem
The node subsystem of the Build Engine is
responsible for managing the knowledge in &SCons; of
the relationships among the external objects
(files) it is responsible for updating.
The most important of these relationships is
the dependency relationship between various &Node; objects,
which &SCons; uses to determine the order
in which builds should be performed.
The &scons; script (or other
user interface)
tells the Build Engine
about dependencies
through its &consenv; API.
The Build Engine also discovers
dependencies automatically through the use of &Scanner; objects.
Subclasses of the &Node; class maintain additional
relationships that reflect the real-world
existence of these objects.
For example, the &Node_FS; subclass
is responsible for managing a
representation of the directory hierarchy
of a file system.
A &Walker; class is used by other subsystems
to walk the dependency tree maintained by the &Node; class.
The &Walker; class maintains a stack of &Node; objects
visited during its depth-first traversal of the
dependency tree,
and uses an intermediate node &Wrapper; class
to maintain state information about a
&Node; object's dependencies.
Scanner Subsystem
The scanner subsystem is responsible for maintaining
objects that can scan the contents of a &Node;'s
for implicit dependencies.
In practice, a given &Scanner; subclass object
functions as a prototype,
returning clones of itself
depending on the &consenv;
values governing how the &Node;
should be scanned.
Signature Subsystem
The signature subsystem is responsible for computing
signature information for &Node; objects.
The signature subsystem in &SCons;
supports multiple ways to
determine whether a &Node is up-to-date
by using an abstract &Sig; class
as a strategy wrapper:
By default, &SCons; tracks dependencies by computing and
maintaining MD5 signatures for the contents of each source file
(or other object). The signature of a derived
file consists of the aggregate of the signatures of all the source
files plus the command-line string used to
build the file. These signatures are stored in a &sconsign; file
in each directory.
If the contents of any of the source files changes, the change to its
MD5 signature is propogated to the signature of the derived file(s). The
simple fact that the new signature does not match the stored signature
indicates that the derived file is not up to date and must be rebuilt.
A separate &TimeStamp; subclass of the &Sig; class supports
the use of traditional file timestamps for
deciding whether files are up-to-date.
Builder Subsystem
The &SCons; Build Engine records how out-of-date files
(or other objects) should be rebuilt in &Builder; objects,
maintained by the builder subsystem:
The actual underlying class name is &BuilderBase;,
and there are subclasses that can encapsulate
multiple &Builder; objects for special purposes.
One subclass
(&CompositeBuilder;)
selects an appropriate encapsulated &Builder;
based on the file suffix of the target object.
The other
(&MultiStepBuilder;).
can chain together multiple
&Builder; objects,
for example,
to build an executable program from a source file
through an implicit intermediate object file.
A &BuilderBase; object has an associated
&ActionBase; object
responsible for actually executing
the appropriate steps
to update the target file.
There are three subclasses,
one for externally executable commands
(&CommandAction;),
one for Python functions
(&FunctionAction;),
and one for lists of
multiple &Action; objects
(&ListAction;).
Job/Task Subsystem
&SCons; supports parallel builds with a thread-based tasking
model, managed by the job/task subsystem.
Instead of performing an outer-loop recursive descent
of the dependency tree and then forking a task when it finds a
file that needs updating, &SCons; starts as many threads as are
requested, each thread managed by the &Jobs; class.
As a performance optimization,
the &Jobs; class maintains an internal
distinction between
&Serial; and &Parallel;
build jobs,
so that serial builds
don't pay any performance penalty
by using a multi-threaded implementation
written for &Parallel; builds.
Each &Jobs; object, running in its own thread,
then requests a &Task; from a central &Taskmaster;,
which is responsible
for handing out available &Task; objects for (re-)building
out-of-date nodes. A condition variable
makes sure that the &Jobs; objects
query the &Taskmaster; one at a time.
The &Taskmaster uses the node subsystem's
&Walker; class to walk the dependency tree,
and the &Sig; class to use the
appropriate method
of deciding if a &Node; is up-to-date.
This scheme has many advantages over the standard &Make;
implementation of .
Effective use of is difficult
with the usual recursive use of Make,
because the number of jobs started by multiply
at each level of the source tree.
This makes the actual number of jobs
executed at any moment very dependent on the size and layout of
the tree. &SCons;, in contrast, starts only as many jobs as are
requested, and keeps them constantly busy (excepting jobs that
block waiting for their dependency files to finish building).
The &SCons; API
This section provides an overview of the &SCons; interface. The
complete interface specification is both more detailed and flexible
than this overview.
&ConsVars;
In &SCons;, a &consenv; is an object through which an external
interface (such as the &scons; script) communicates dependency
information to the &SCons; Build Engine.
A construction environment is implemented as a dictionary
containing:
construction variables, string values that are substituted
into command lines or used by builder functions;
one or more &Builder; objects that can be invoked to update a
file or other object;
one or more &Scanner; objects that can be used to
scan a file automatically for dependencies (such as
files specified on #include lines).
&Consenvs; are instantiated as follows:
env = Environment()
env_debug = Environment(CCFLAGS = '-g')
&Builder; Objects
An &SCons; &Builder; object encapsulates information about how to
build a specific type of file: an executable program, an object
file, a library, etc. A &Builder; object is associated with a
file through an associated &consenv; method and later invoked to
actually build the file. The &Builder; object will typically use
construction variables (such as &CCFLAGS;, &LIBPATH;) to influence
the specific build execution.
&Builder; objects are instantiated as follows:
bld = Builder(name = 'Program', action = "$CC -o $TARGET $SOURCES")
In the above example, the action is a
command-line string in which the Build Engine will
interpolate the values of construction
variables before execution. The actual
action specified, though,
may be a function:
def update(dest):
# [code to update the object]
return 0
bld = Builder(name = 'Program', function = update)
Or a callable Python object (or class):
class class_a:
def __call__(self, kw):
# build the desired object
return 0
builder = SCons.Builder.Builder(action = class_a())
A &Builder; object may have the prefix and
suffix of its target file type specified
as keyword arguments at instantiation. Additionally, the
suffix of the source files used by this
&Builder; to build its target files may be specified using the
src_suffix keyword argument:
bld_lib = Builder(name = 'Library', action = "$AR r $TARGET $SOURCES",
prefix = 'lib', suffix = '.a', src_suffix = '.o')
The specified prefix and
suffix will be appended to the name of any
target file built by this &Builder; object, if they are not
already part of the file name. The src_suffix
is used by the &SCons; Build Engine to chain together
multiple &Builder; objects to create,
for example, a library from the original source
files without having to specify the
intermediate .o files.
&Builder; objects are associated with a &consenv; through a
&consvar; named &BUILDERS;, a list of the &Builder objects that
will be available for execution through the &consenv:
env = Environment(BUILDERS = [ Object, Library, WebPage, Program ])
&Scanner; Objects
&Scanner; objects perform automatic checking for dependencies
by scanning the contents of files. The canonical
example is scanning a C source file or header file for
files specified on #include lines.
A &Scanner; object is instantiated as follows:
def c_scan(contents):
# scan contents of file
return # list of files found
c_scanner = Scanner(name = 'CScan', function = c_scan,
argument = None,
skeys = ['.c', '.C', '.h', '.H')
The skeys argument specifies a list of file
suffixes for file types that this &Scanner; knows how to scan.
&Scanner; objects are associated with a &consenv; through a
&consvar; named &SCANNERS;, a list of the &Scanner; objects that
will be available through the &consenv:
env = Environment(SCANNERS = [ CScan, M4Scan ])
For utilities that will build files with a variety of file
suffixes, or which require unusual scanning rules, a &Scanner;
object may be associated explicitly with a &Builder; object as
follows:
def tool_scan(contents):
# scan contents of file
return # list of files found
tool_scanner = Scanner(name = 'TScan', function = tool_scan)
bld = Builder(name = 'Tool', scanner = tool_scanner)
&BuildDir;
&SCons; supports a flexible mechanism for building target
files in a separate build directory from the source files.
The &BuildDir; syntax is straightforward:
BuildDir(source = 'src', build = 'bld')
By
default, source files are linked or copied into the build
directory, because exactly replicating the source directory
is sometimes necessary for certain combinations of use of
#include "..." and search
paths.
An option exists to specify that only output files should be placed in
the build directory:
BuildDir(source = 'src', build = 'bld', no_sources = 1)
&Repository;
&SCons; supports the ability to search a list of code repositories
for source files and derived files. This works much like
&Make;'s VPATH feature, as implemented in
recent versions of GNU &Make;.
(The POSIX standard for &Make; specifies slightly
different behavior for VPATH.)
The syntax is:
Repository('/home/source/1.1', '/home/source/1.0')
A command-line option exists to allow
repositories to be specified on the command line, or in the
&SCONSFLAGS; environment variable (not construction variable!).
This avoids a chicken-and-egg situation and allows the top-level
&SConstruct; file to be found in a repository as well.
&Cache;
&SCons; supports a way for developers to share derived files. Again, the
syntax is straightforward:
Cache('/var/build.cache/i386')
Copies of any derived files built will be placed in the specified
directory with their MD5 signature. If another build results in an
out-of-date derived file with the same signature, the derived file
will be copied from the cache instead of being rebuilt.
The &scons; Script
The &scons; script provides an interface
that looks roughly equivalent to the
classic &Make; utility—that is, execution from the command
line, and dependency information read from configuration files.
The most noticeable difference between &scons; and &Make;, or most
other build tools, is that the configuration files are actually
Python scripts, generically called "SConscripts" (although the
top-level "Makefile" is named &SConstruct). Users do not have to
learn a new language syntax, but instead configure dependency
information by making direct calls to the Python API of the
&SCons; Build Engine. Here is an example &SConstruct file which
builds a program in side-by-side normal and debug versions:
env = Environment()
debug = env.Copy(CCFLAGS = '-g')
source_files = ['f1.c', 'f2.c', 'f3.c']
env.Program(target = 'foo', sources = source_files)
debug.Program(target = 'foo-debug', sources = source_files)
Notice the fact that this file is a Python script, which allows us
to define and re-use an array that lists the source files.
Because quoting individul strings in long
lists of files can get tedious and error-prone, the &SCons;
methods support a short-cut of listing multiple files in a single
string, separated by white space.
This would change
the assignment in the above example to a more easily-readable:
source_files = 'f1.c f2.c f3.c'
The mechanism to establish hierarchical builds is to "include" any
subsidiary configuration files in the build by listing them explicitly
in a call to the &SConscript; function:
SConscript('src/SConscript', 'lib/SConscript')
By convention, configuration files in subdirectories are named
&SConscript;.
The &scons; script has intentionally been made to look, from
the outside, as much like &Make; as is practical. To this
end, the &scons; script supports all of the same command-line
options supported by GNU &Make;: FILE,
, , ,
etc. For compatibility, &scons; ignores those GNU &Make; options
that don't make sense for the &SCons; architecture, such as
, , ,
and . The
intention is that, given an equivalent &SConstruct; file for a
&Makefile;, a user could use &SCons; as a drop-in replacement for
&Make;. Additional command-line options are, where possible, taken
from the Perl &Cons; utility on which the &SCons; design is based.