path: root/doc/user/separate.xml

<?xml version='1.0'?>
<!--
SPDX-License-Identifier: MIT

Copyright The SCons Foundation
-->

<!DOCTYPE sconsdoc [
    <!ENTITY % scons SYSTEM "../scons.mod">
    %scons;

    <!ENTITY % builders-mod SYSTEM "../generated/builders.mod">
    %builders-mod;
    <!ENTITY % functions-mod SYSTEM "../generated/functions.mod">
    %functions-mod;
    <!ENTITY % tools-mod SYSTEM "../generated/tools.mod">
    %tools-mod;
    <!ENTITY % variables-mod SYSTEM "../generated/variables.mod">
    %variables-mod;
]>

<chapter id="chap-separate"
         xmlns="http://www.scons.org/dbxsd/v1.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://www.scons.org/dbxsd/v1.0 http://www.scons.org/dbxsd/v1.0/scons.xsd">

<title>Separating Source and Build Trees: Variant Directories</title>

  <para>

  It is often useful to keep built files completely
  separate from the source files. Two main benefits are
  the ability to have different configurations simultaneously
  without build conflicts, and being version-control friendly.

  </para>

  <para>

  Consider if you have a project to build an embedded
  software system for a variety of different controller hardware.
  The system is able to share a lot of code,
  so it makes sense to use a common source tree,
  but certain build options in the source code
  and header files differ. For a regular in-place build,
  the build outputs go in the same place as the source code.
  If you build <emphasis>Controller A</emphasis> first,
  followed by <emphasis>Controller B</emphasis>,
  on the <emphasis>Controller B</emphasis> build everything that
  uses different build options has to be rebuilt since those
  objects will be different
  (the build lines, including preprocessor defines, are part of
  &SCons;'s out-of-date calculation for this reason).
  If you go back and build for <emphasis>Controller A</emphasis> again,
  things have to be rebuilt again for the same reason.
  However, if you can separate the locations of the output files,
  so each controller has its own location for build outputs,
  this problem can be avoided.

  </para>

  <para>

  Having a separated build tree also helps you keep your source tree clean -
  there is less chance of accidentally checking in build products
  to version control that were not intended to be checked in.
  You can add a separated build directory to your
  version control system's list of items not to track.
  You can even remove the whole build tree with a single command without
  risking removing any of the source code.

  </para>

  <para>

  The key to making this separation work is the ability to
  do out-of-tree builds: building under a separate root
  than the sources being built.
  You set up out of tree builds by establishing what &SCons;
  calls a <firstterm>variant directory</firstterm>,
  a place where you can build a single variant of your software
  (of course you can define more than one of these if you need to).
  Since &SCons; tracks targets by their path, it is able to distinguish
  build products like <filename>build/A/network.obj</filename>
  of the <emphasis>Controller A</emphasis> build
  from <filename>build/B/network.obj</filename>
  of the <emphasis>Controller B</emphasis> build,
  thus avoiding conflicts.

  </para>

  <para>

  &SCons; provides two ways to establish variant directories,
  one through the &f-link-SConscript; function that we have already seen,
  and the second through a more flexible &f-link-VariantDir; function.

  </para>

  <para>

  The variant directory mechanism does support doing multiple builds
  in one invocation of &SCons;, but the remainder of this chapter
  will focus on setting up a single build. You can combine these
  techniques with ones from the previous chapter and elsewhere
  in this Guide to set up more complex scenarios.

  </para>

  <note> <para>

  The &VariantDir; function used to be called &BuildDir;,
  a name which was changed because it turned out to be confusing:
  the &SCons; functionality
  differs from a familiar model of a "build directory"
  implemented by certain other build systems like GNU Autotools.
  You might still find references to the old name on
  the Internet in postings about &SCons;, but it no longer works.

  </para> </note>

  <section id="sect-variant-sconscript">
  <title>Specifying a Variant Directory Tree as Part of an &SConscript; Call</title>

    <para>

    The most straightforward way to establish a variant directory tree
    relies on the fact that the usual way to
    set up a build hierarchy is to have an
    &SConscript; file in the source directory.
    If you pass a &variant_dir; argument to the
    &f-link-SConscript; function call:

    </para>

    <scons_example name="separate_ex1">
      <file name="SConstruct" printme="1">
SConscript('src/SConscript', variant_dir='build')
      </file>
      <file name="src/SConscript">
env = Environment()
env.Program('hello.c')
      </file>
      <file name="src/hello.c">
int main() { printf("Hello, world!\n"); }
      </file>
    </scons_example>

    <para>

    &SCons; will then build all of the files in
    the &build; directory:

    </para>

    <scons_output example="separate_ex1" suffix="1">
      <scons_output_command>ls src</scons_output_command>
      <scons_output_command>scons -Q</scons_output_command>
      <scons_output_command>ls src</scons_output_command>
      <scons_output_command>ls build</scons_output_command>
    </scons_output>

    <para>

    No files were built in &src;:
    the object file
    <filename>build/hello.o</filename>
    and the executable file
    <filename>build/hello</filename>
    were built in the &build; directory, as expected.
    But notice that even though our &hello_c; file actually
    lives in the &src; directory, &SCons; has compiled a
    <filename>build/hello.c</filename> file
    to create the object file,
    and that file is now seen in &build;.

    </para>

    <para>

    You can ask &SCons; to show the dependency tree to illustrate
    a bit more:

    </para>

    <scons_output example="separate_ex1" suffix="2">
      <scons_output_command>scons -Q --tree=prune</scons_output_command>
    </scons_output>

    <para>

    What's happened is that &SCons; has <emphasis>duplicated</emphasis>
    the &hello_c; file from the &src; directory
    to the &build; directory,
    and built the program from there (it also duplicated &SConscript;).
    The next section explains why &SCons; does this.

    </para>

    <para>

    The nice thing about the &SConscript; approach is it is almost
    invisible to you:
    this build looks just like an ordinary in-place build
    except for the extra &variant_dir; argument in the
    &f-link-SConscript; call.
    &SCons; handles all the path adjustments for the
    out of tree &build; directory while it processes that SConscript file.

    </para>

  </section>

  <section id="sect-variant-duplication">
  <title>Why &SCons; Duplicates Source Files in a Variant Directory Tree</title>

    <para>

    When you set up a variant directory &SCons; conceptually behaves as
    if you requested a build in that directory.
    As noted in the previous chapter,
    all builds actually happen from the top level directory,
    but as an aid to understanding how &SCons; operates, think
    of it as <emphasis>build in place in the variant directory</emphasis>,
    not <emphasis>build in source but send build artifacts
    to the variant directory</emphasis>.
    It turns out in place builds are easier to get right than out
    of tree builds - so by default &SCons; simulates an in place build
    by making the variant directory look just like the source directory.
    The most straightforward way to do that is by making copies
    of the files needed for the build.

    </para>

    <para>

    The most direct reason to duplicate source files
    in variant directories
    is simply that some tools (mostly older versions)
    are written to only build their output files
    in the same directory as the source files - such tools often don't
    have any option to specify the output file, and the tool just
    uses a predefined output file name,
    or uses a derived variant of the source file name,
    dropping the result in the same directory.
    In this case, the choices are either
    to build the output file in the source directory
    and move it to the variant directory,
    or to duplicate the source files in the variant directory.

    </para>

    <para>

    Additionally,
    relative references between files
    can cause problems which are resolved by
    just duplicating the hierarchy of source files
    into the variant directory.
    You can see this at work in
    use of the C preprocessor <literal>#include</literal>
    mechanism with double quotes, not angle brackets:

    </para>

    <sconstruct>
#include "file.h"
    </sconstruct>

    <para>

    The <emphasis>de facto</emphasis> standard behavior
    for most C compilers in this case
    is to first look in the same directory
    as the source file that contains the <literal>#include</literal> line,
    then to look in the directories in the preprocessor search path.
    Add to this that the &SCons; implementation of
    support for code repositories
    (described below)
    means not all of the files
    will be found in the same directory hierarchy,
    and the simplest way to make sure
    that the right include file is found
    is to duplicate the source files into the variant directory,
    which provides a correct build
    regardless of the original location(s) of the source files.

    </para>

    <para>

    Although source-file duplication guarantees a correct build
    even in these edge cases,
    it can <emphasis>usually</emphasis> be safely disabled.
    The next section describes
    how you can disable the duplication of source files
    in the variant directory.

    </para>

  </section>

  <section id="sect-variant-no-duplication">
  <title>Telling &SCons; to Not Duplicate Source Files in the Variant Directory Tree</title>

    <para>

    In most cases and with most tool sets,
    &SCons; can use sources directly from the source directory
    <emphasis>without</emphasis>
    duplicating them into the variant directory before building,
    and everything will work just fine.
    You can disable the default &SCons; duplication behavior
    by specifying <literal>duplicate=False</literal>
    when you call the &f-link-SConscript; function:

    </para>

    <sconstruct>
SConscript('src/SConscript', variant_dir='build', duplicate=False)
    </sconstruct>

    <para>

    When this flag is specified, the results of a build
    look more like the mental model people may have from other
    build systems - that is,
    the output files end up in the variant directory
    while the source files do not.

    </para>

    <screen>
% <userinput>ls src</userinput>
SConscript
hello.c
% <userinput>scons -Q</userinput>
cc -c src/hello.c -o build/hello.o
cc -o build/hello build/hello.o
% <userinput>ls build</userinput>
hello
hello.o
    </screen>

    <para>

    If disabling duplication causes any problems,
    just return to the more cautious approach by letting
    &SCons; go back to duplicating files.

    </para>

  </section>

  <section id="sect-variantdir-function">
  <title>The &VariantDir; Function</title>

    <para>

    You can also use the &f-link-VariantDir; function to establish
    that target files should be built in a separate directory tree
    from the source files:

    </para>

    <scons_example name="separate_builddir">
      <file name="SConstruct" printme="1">
VariantDir('build', 'src')
env = Environment()
env.Program('build/hello.c')
      </file>
      <file name="src/hello.c">
int main() { printf("Hello, world!\n"); }
      </file>
    </scons_example>

    <para>

    When using this form, you have to tell &SCons; that
    sources and targets are in the variant directory,
    and those references will trigger the remapping,
    necessary file copying, etc. for an already established
    variant directory.  Here is the same example in a more
    spelled out form to show this more clearly:

    <programlisting language="python">
VariantDir('build', 'src')
env = Environment()
env.Program(target='build/hello', source=['build/hello.c'])
    </programlisting>

    </para>

    <para>

    When using the &VariantDir; function directly,
    &SCons; still duplicates the source files
    in the variant directory by default:

    </para>

    <scons_output example="separate_builddir" suffix="1">
      <scons_output_command>ls src</scons_output_command>
      <scons_output_command>scons -Q</scons_output_command>
      <scons_output_command>ls build</scons_output_command>
    </scons_output>

    <para>

    You can specify the same <literal>duplicate=False</literal> argument
    that you can specify for an &f-link-SConscript; call:

    </para>

    <scons_example name="separate_duplicate0">
      <file name="SConstruct" printme="1">
VariantDir('build', 'src', duplicate=False)
env = Environment()
env.Program('build/hello.c')
      </file>
      <file name="src/hello.c">
int main() { printf("Hello, world!\n"); }
      </file>
    </scons_example>

    <para>

    In which case &SCons;
    will disable duplication of the source files:

    </para>

    <scons_output example="separate_duplicate0" suffix="1">
      <scons_output_command>ls src</scons_output_command>
      <scons_output_command>scons -Q</scons_output_command>
      <scons_output_command>ls build</scons_output_command>
    </scons_output>

  </section>

  <section id="sect-variantdir-sconscript">
  <title>Using &VariantDir; With an &SConscript; File</title>

    <para>

    Even when using the &f-link-VariantDir; function,
    it is more natural to use it with
    a subsidiary &SConscript; file,
    because then you don't have to adjust your individual
    build instructions to use the variant directory path.
    For example, if the
    <filename>src/SConscript</filename>
    looks like this:

    </para>

    <scons_example name="separate_builddir_sconscript">
      <file name="SConstruct">
VariantDir('build', 'src')
SConscript('build/SConscript')
      </file>
      <file name="src/SConscript" printme="1">
env = Environment()
env.Program('hello.c')
      </file>
      <file name="src/hello.c">
int main() { printf("Hello, world!\n"); }
      </file>
    </scons_example>

    <para>

    Then our &SConstruct; file could look like:

    </para>

    <scons_example_file example="separate_builddir_sconscript" name="SConstruct">
    </scons_example_file>

    <para>

    Yielding the following output:

    </para>

    <scons_output example="separate_builddir_sconscript" suffix="1">
      <scons_output_command>ls src</scons_output_command>
      <scons_output_command>scons -Q</scons_output_command>
      <scons_output_command>ls build</scons_output_command>
    </scons_output>

    <para>

    This is completely equivalent
    to the use of &f-link-SConscript; with the
    <parameter>variant_dir</parameter> argument
    from earlier in this chapter,
    but did require callng the SConscript using the already established
    variant directory path to trigger that behavior.
    If you call <userinput>SConscript('src/SConscript')</userinput>
    you would get a normal in-place build in &src;.

    </para>

  </section>

  <section id="sect-variantdir-glob">
  <title>Using &Glob; with &VariantDir;</title>

    <para>

    The &f-link-Glob; file name pattern matching function
    works just as usual when using &f-link-VariantDir;.
    For example, if the
    <filename>src/SConscript</filename>
    looks like this:

    </para>

    <scons_example name="separate_glob_builddir_sconscript">
      <file name="SConstruct">
VariantDir('build', 'src')
SConscript('build/SConscript')
      </file>
      <file name="src/SConscript" printme="1">
env = Environment()
env.Program('hello', Glob('*.c'))
      </file>
      <file name="src/f1.c">
#include "f2.h"
int main() { printf(f2()); }
      </file>
      <file name="src/f2.c">
const char * f2() { return("Hello, world!\n"); }
      </file>
      <file name="src/f2.h">
const char * f2();
      </file>
    </scons_example>

    <para>

    Then with the same &SConstruct; file as in the previous section,
    and source files <filename>f1.c</filename>
    and <filename>f2.c</filename> in src,
    we would see the following output:

    </para>

    <scons_output example="separate_glob_builddir_sconscript" suffix="1">
      <scons_output_command>ls src</scons_output_command>
      <scons_output_command>scons -Q</scons_output_command>
      <scons_output_command>ls build</scons_output_command>
    </scons_output>

    <para>

    The &Glob; function returns Nodes in the
    <filename>build/</filename> tree, as you'd expect.

    </para>

  </section>

  <!--

  <section id="sect-variantdir-over-sconscript">
  <title>Why You'd Want to Call &VariantDir; Instead of &SConscript;</title>

    <para>

    XXX why call VariantDir() instead of SConscript(variant_dir=)

    </para>

  </section>

  -->

  <xi:include xmlns:xi="http://www.w3.org/2001/XInclude" href="variants.xml"/>

</chapter>