rebuilt docs

1 files changed, 506 insertions, 0 deletions

diff --git a/doc/user/caching.xml b/doc/user/caching.xml
index e69de29..2348d32 100644
--- a/doc/user/caching.xml
+++ b/doc/user/caching.xml
@@ -0,0 +1,506 @@
+<!--
+
+  __COPYRIGHT__
+
+  Permission is hereby granted, free of charge, to any person obtaining
+  a copy of this software and associated documentation files (the
+  "Software"), to deal in the Software without restriction, including
+  without limitation the rights to use, copy, modify, merge, publish,
+  distribute, sublicense, and/or sell copies of the Software, and to
+  permit persons to whom the Software is furnished to do so, subject to
+  the following conditions:
+
+  The above copyright notice and this permission notice shall be included
+  in all copies or substantial portions of the Software.
+
+  THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY
+  KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE
+  WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+  NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
+  LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
+  OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
+  WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
+
+-->
+
+  <para>
+
+  On multi-developer software projects,
+  you can sometimes speed up every developer's builds a lot by
+  allowing them to share the derived files that they build.
+  &SCons; makes this easy, as well as reliable.
+
+  </para>
+
+  <section>
+  <title>Specifying the Shared Cache Directory</title>
+
+    <para>
+
+    To enable sharing of derived files,
+    use the &CacheDir; function
+    in any &SConscript; file:
+
+    </para>
+
+    <programlisting>
+       CacheDir('/usr/local/build_cache')
+    </programlisting>
+
+    <para>
+
+    Note that the directory you specify must already exist
+    and be readable and writable by all developers
+    who will be sharing derived files.
+    It should also be in some central location
+    that all builds will be able to access.
+    In environments where developers are using separate systems
+    (like individual workstations) for builds,
+    this directory would typically be
+    on a shared or NFS-mounted file system.
+
+    </para>
+
+    <para>
+
+    Here's what happens:
+    When a build has a &CacheDir; specified,
+    every time a file is built,
+    it is stored in the shared cache directory
+    along with its MD5 build signature.
+      <footnote>
+      <para>
+      Actually, the MD5 signature is used as the name of the file
+      in the shared cache directory in which the contents are stored.
+      </para>
+      </footnote>
+    On subsequent builds,
+    before an action is invoked to build a file,
+    &SCons; will check the shared cache directory
+    to see if a file with the exact same build
+    signature already exists.
+    If so, the derived file will not be built locally,
+    but will be copied into the local build directory
+    from the shared cache directory,
+    like so:
+
+    </para>
+
+    <screen>
+      % <userinput>scons -Q</userinput>
+      cc -o hello.o -c hello.c
+      cc -o hello hello.o
+      % <userinput>scons -Q -c</userinput>
+      Removed hello.o
+      Removed hello
+      % <userinput>scons -Q</userinput>
+      Retrieved `hello.o' from cache
+      Retrieved `hello' from cache
+    </screen>
+
+    <para>
+
+    Note that the &CacheDir; feature still calculates
+    MD5 build sigantures for the shared cache file names
+    even if you configure &SCons; to use timestamps
+    to decide if files are up to date.
+    (See the <xref linkend="chap-depends"></xref>
+    chapter for information about the &Decider; function.)
+    Consequently, using &CacheDir; may reduce or eliminate any
+    potential performance improvements
+    from using timestamps for up-to-date decisions.
+
+    </para>
+
+  </section>
+
+  <section>
+  <title>Keeping Build Output Consistent</title>
+
+    <para>
+
+    One potential drawback to using a shared cache
+    is that the output printed by &SCons;
+    can be inconsistent from invocation to invocation,
+    because any given file may be rebuilt one time
+    and retrieved from the shared cache the next time.
+    This can make analyzing build output more difficult,
+    especially for automated scripts that
+    expect consistent output each time.
+
+    </para>
+
+    <para>
+
+    If, however, you use the <literal>--cache-show</literal> option,
+    &SCons; will print the command line that it
+    <emphasis>would</emphasis> have executed
+    to build the file,
+    even when it is retrieving the file from the shared cache.
+    This makes the build output consistent
+    every time the build is run:
+
+    </para>
+
+    <screen>
+      % <userinput>scons -Q</userinput>
+      cc -o hello.o -c hello.c
+      cc -o hello hello.o
+      % <userinput>scons -Q -c</userinput>
+      Removed hello.o
+      Removed hello
+      % <userinput>scons -Q --cache-show</userinput>
+      cc -o hello.o -c hello.c
+      cc -o hello hello.o
+    </screen>
+
+    <para>
+
+    The trade-off, of course, is that you no longer
+    know whether or not &SCons;
+    has retrieved a derived file from cache
+    or has rebuilt it locally.
+
+    </para>
+
+  </section>
+
+  <section>
+  <title>Not Using the Shared Cache for Specific Files</title>
+
+    <para>
+
+    You may want to disable caching for certain
+    specific files in your configuration.
+    For example, if you only want to put
+    executable files in a central cache,
+    but not the intermediate object files,
+    you can use the &NoCache;
+    function to specify that the
+    object files should not be cached:
+
+    </para>
+
+    <programlisting>
+       env = Environment()
+       obj = env.Object('hello.c')
+       env.Program('hello.c')
+       CacheDir('cache')
+       NoCache('hello.o')
+    </programlisting>
+
+    <para>
+
+    Then when you run &scons; after cleaning
+    the built targets,
+    it will recompile the object file locally
+    (since it doesn't exist in the shared cache directory),
+    but still realize that the shared cache directory
+    contains an up-to-date executable program
+    that can be retrieved instead of re-linking:
+
+    </para>
+
+    <!--
+
+    <scons_output example="ex1">
+      <scons_output_command>scons -Q</scons_output_command>
+      <scons_output_command>scons -Q -c</scons_output_command>
+      <scons_output_command>scons -Q</scons_output_command>
+    </scons_output>
+
+    -->
+
+    <screen>
+      % <userinput>scons -Q</userinput>
+      cc -o hello.o -c hello.c
+      cc -o hello hello.o
+      % <userinput>scons -Q -c</userinput>
+      Removed hello.o
+      Removed hello
+      % <userinput>scons -Q</userinput>
+      cc -o hello.o -c hello.c
+      Retrieved `hello' from cache
+    </screen>
+
+  </section>
+
+  <section>
+  <title>Disabling the Shared Cache</title>
+
+    <para>
+
+    Retrieving an already-built file
+    from the shared cache
+    is usually a significant time-savings
+    over rebuilding the file,
+    but how much of a savings
+    (or even whether it saves time at all)
+    can depend a great deal on your
+    system or network configuration.
+    For example, retrieving cached files
+    from a busy server over a busy network
+    might end up being slower than
+    rebuilding the files locally.
+
+    </para>
+
+    <para>
+
+    In these cases, you can specify
+    the <literal>--cache-disable</literal>
+    command-line option to tell &SCons;
+    to not retrieve already-built files from the
+    shared cache directory:
+
+    </para>
+
+    <screen>
+      % <userinput>scons -Q</userinput>
+      cc -o hello.o -c hello.c
+      cc -o hello hello.o
+      % <userinput>scons -Q -c</userinput>
+      Removed hello.o
+      Removed hello
+      % <userinput>scons -Q</userinput>
+      Retrieved `hello.o' from cache
+      Retrieved `hello' from cache
+      % <userinput>scons -Q -c</userinput>
+      Removed hello.o
+      Removed hello
+      % <userinput>scons -Q --cache-disable</userinput>
+      cc -o hello.o -c hello.c
+      cc -o hello hello.o
+    </screen>
+
+  </section>
+
+  <section>
+  <title>Populating a Shared Cache With Already-Built Files</title>
+
+    <para>
+
+    Sometimes, you may have one or more derived files
+    already built in your local build tree
+    that you wish to make available to other people doing builds.
+    For example, you may find it more effective to perform
+    integration builds with the cache disabled
+    (per the previous section)
+    and only populate the shared cache directory
+    with the built files after the integration build
+    has completed successfully.
+    This way, the cache will only get filled up
+    with derived files that are part of a complete, successful build
+    not with files that might be later overwritten
+    while you debug integration problems.
+
+    </para>
+
+    <para>
+
+    In this case, you can use the
+    the <literal>--cache-force</literal> option
+    to tell &SCons; to put all derived files in the cache,
+    even if the files already exist in your local tree
+    from having been built by a previous invocation:
+
+    </para>
+
+    <screen>
+      % <userinput>scons -Q --cache-disable</userinput>
+      cc -o hello.o -c hello.c
+      cc -o hello hello.o
+      % <userinput>scons -Q -c</userinput>
+      Removed hello.o
+      Removed hello
+      % <userinput>scons -Q --cache-disable</userinput>
+      cc -o hello.o -c hello.c
+      cc -o hello hello.o
+      % <userinput>scons -Q --cache-force</userinput>
+      scons: `.' is up to date.
+      % <userinput>scons -Q</userinput>
+      scons: `.' is up to date.
+    </screen>
+
+    <para>
+
+    Notice how the above sample run
+    demonstrates that the <literal>--cache-disable</literal>
+    option avoids putting the built
+    <filename>hello.o</filename>
+    and
+    <filename>hello</filename> files in the cache,
+    but after using the <literal>--cache-force</literal> option,
+    the files have been put in the cache
+    for the next invocation to retrieve.
+
+    </para>
+
+  </section>
+
+  <section>
+  <title>Minimizing Cache Contention:  the <literal>--random</literal> Option</title>
+
+    <para>
+
+    If you allow multiple builds to update the
+    shared cache directory simultaneously,
+    two builds that occur at the same time
+    can sometimes start "racing"
+    with one another to build the same files
+    in the same order.
+    If, for example,
+    you are linking multiple files into an executable program:
+
+    </para>
+
+    <programlisting>
+       Program('prog',
+               ['f1.c', 'f2.c', 'f3.c', 'f4.c', 'f5.c'])
+    </programlisting>
+
+    <para>
+
+    &SCons; will normally build the input object files
+    on which the program depends in their normal, sorted order:
+
+    </para>
+
+    <screen>
+      % <userinput>scons -Q</userinput>
+      cc -o f1.o -c f1.c
+      cc -o f2.o -c f2.c
+      cc -o f3.o -c f3.c
+      cc -o f4.o -c f4.c
+      cc -o f5.o -c f5.c
+      cc -o prog f1.o f2.o f3.o f4.o f5.o
+    </screen>
+
+    <para>
+
+    But if two such builds take place simultaneously,
+    they may each look in the cache at nearly the same
+    time and both decide that <filename>f1.o</filename>
+    must be rebuilt and pushed into the shared cache directory,
+    then both decide that <filename>f2.o</filename>
+    must be rebuilt (and pushed into the shared cache directory),
+    then both decide that <filename>f3.o</filename>
+    must be rebuilt...
+    This won't cause any actual build problems--both
+    builds will succeed,
+    generate correct output files,
+    and populate the cache--but
+    it does represent wasted effort.
+
+    </para>
+
+    <para>
+
+    To alleviate such contention for the cache,
+    you can use the <literal>--random</literal> command-line option
+    to tell &SCons; to build dependencies
+    in a random order:
+
+    </para>
+
+    <!--
+
+    The following <screen> output was generated by this:
+
+    <scons_output example="ex-random">
+      <scons_output_command>scons -Q - -random</scons_output_command>
+    </scons_output>
+
+    We captured it directly here to guarantee a "random" order,
+    guarding against the potential for - -random to happen
+    to return things in the original sorted order.
+
+    -->
+
+    <screen>
+      % <userinput>scons -Q --random</userinput>
+      cc -o f3.o -c f3.c
+      cc -o f1.o -c f1.c
+      cc -o f5.o -c f5.c
+      cc -o f2.o -c f2.c
+      cc -o f4.o -c f4.c
+      cc -o prog f1.o f2.o f3.o f4.o f5.o
+    </screen>
+
+    <para>
+
+    Multiple builds using the <literal>--random</literal> option
+    will usually build their dependencies in different,
+    random orders,
+    which minimizes the chances for a lot of
+    contention for same-named files
+    in the shared cache directory.
+    Multiple simultaneous builds might still race to try to build
+    the same target file on occasion,
+    but long sequences of inefficient contention
+    should be rare.
+
+    </para>
+
+    <para>
+
+    Note, of course,
+    the <literal>--random</literal> option
+    will cause the output that &SCons; prints
+    to be inconsistent from invocation to invocation,
+    which may be an issue when
+    trying to compare output from different build runs.
+
+    </para>
+
+    <para>
+
+    If you want to make sure dependencies will be built
+    in a random order without having to specify
+    the <literal>--random</literal> on very command line,
+    you can use the &SetOption; function to
+    set the <literal>random</literal> option
+    within any &SConscript; file:
+
+    </para>
+
+    <programlisting>
+       Program('prog',
+               ['f1.c', 'f2.c', 'f3.c', 'f4.c', 'f5.c'])
+
+       SetOption('random', 1)
+       Program('prog',
+               ['f1.c', 'f2.c', 'f3.c', 'f4.c', 'f5.c'])
+    </programlisting>
+
+  </section>
+
+  <!--
+
+  <section>
+  <title>Troubleshooting Shared Caching:  the &cache-debug; Option</title>
+
+    <para>
+
+    XXX describe the - - cache-debug option
+    XXX maybe point to the troubleshooting appendix?
+
+    </para>
+
+  </section>
+
+  -->
+
+  <!--
+
+  <section>
+
+    <para>
+
+    XXX describe CacheDir management:  monitoring, deleting, etc.
+
+    </para>
+
+  </section>
+
+  -->