path: root/doc/user/external.xml

<?xml version='1.0'?>
<!DOCTYPE sconsdoc [
        <!ENTITY % scons SYSTEM "../scons.mod">
        %scons;

        <!ENTITY % builders-mod SYSTEM "../generated/builders.mod">
        %builders-mod;
        <!ENTITY % functions-mod SYSTEM "../generated/functions.mod">
        %functions-mod;
        <!ENTITY % tools-mod SYSTEM "../generated/tools.mod">
        %tools-mod;
        <!ENTITY % variables-mod SYSTEM "../generated/variables.mod">
        %variables-mod;

        ]>

<chapter id="chap-external"
         xmlns="http://www.scons.org/dbxsd/v1.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://www.scons.org/dbxsd/v1.0 http://www.scons.org/dbxsd/v1.0/scons.xsd">
    <title>Using SCons with other build tools</title>

    <!--

     MIT License

     Copyright The SCons Foundation

     Permission is hereby granted, free of charge, to any person obtaining
     a copy of this software and associated documentation files (the
     "Software"), to deal in the Software without restriction, including
     without limitation the rights to use, copy, modify, merge, publish,
     distribute, sublicense, and/or sell copies of the Software, and to
     permit persons to whom the Software is furnished to do so, subject to
     the following conditions:

     The above copyright notice and this permission notice shall be included
     in all copies or substantial portions of the Software.

     THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY
     KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE
     WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
     NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
     LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
     OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
     WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

    -->

    <para>

        Sometimes a project needs to interact with other projects
        in various ways. For example, many open source projects
        make use of components from other open source projects,
        and want to use those in their released form, not recode
        their builds into &SCons;. As another example, sometimes
        the flexibility and power of &SCons; is useful for managing the
        overall project, but developers might like faster incremental
        builds when making small changes by using a different tool.

    </para>

    <para>

        This chapter shows some techniques for interacting with other
        projects and tools effectively from within &SCons;.

    </para>

    <section>
        <title>Creating a Compilation Database</title>

        <para>

            Tooling to perform analysis and modification
            of source code often needs to know not only the source code
            itself, but also how it will be compiled, as the compilation line
            affects the behavior of macros, includes, etc. &SCons; has a
            record of this information once it has run, in the form of
            Actions associated with the sources, and can emit this information
            so tools can use it.

        </para>

        <para>

            The Clang project has defined a <firstterm>JSON Compilation Database</firstterm>.
            This database is in common use as input into Clang tools
            and many IDEs and editors as well.
            See
            <ulink url="https://clang.llvm.org/docs/JSONCompilationDatabase.html">
                <citetitle>JSON Compilation Database Format Specification</citetitle>
            </ulink>
            for complete information. &SCons; can emit a
            compilation database in this format
            by enabling the &t-link-compilation_db; tool
            and calling the &b-link-CompilationDatabase; builder
            (<emphasis>available since &scons; 4.0</emphasis>).

        </para>

        <para>

            The compilation database can be populated with
            source and output files either with paths relative
            to the top of the build, or using absolute paths.
            This is controlled by
            <envar>COMPILATIONDB_USE_ABSPATH=(True|False)</envar>
            which defaults to <constant>False</constant>.
            The entries in this file can be filtered by using

            <envar>COMPILATIONDB_PATH_FILTER='pattern'</envar>
            where the filter pattern is a string following the Python
            <ulink url="https://docs.python.org/3/library/fnmatch.html">
                <systemitem>fnmatch</systemitem>
            </ulink>
            syntax.
            This filtering can be used for outputting different
            build variants to different compilation database files.

        </para>

        <para>

            The following example illustrates generating a compilation
            database containing absolute paths:

        </para>

        <scons_example name="external_cdb_ex1">
            <file name="SConstruct" printme="1">
                env = Environment(COMPILATIONDB_USE_ABSPATH=True)
                env.Tool('compilation_db')
                env.CompilationDatabase()
                env.Program('hello.c')
            </file>
            <file name="hello.c">
                int main( int argc, char* argv[] )
                {
                return 0;
                }
            </file>
        </scons_example>

        <scons_output example="external_cdb_ex1" suffix="1">
            <scons_output_command>scons -Q</scons_output_command>
        </scons_output>

        <para>
            <filename>compile_commands.json</filename>
            contains:
        </para>

        <programlisting language="json">
            [
            {
            "command": "gcc -o hello.o -c hello.c",
            "directory": "/home/user/sandbox",
            "file": "/home/user/sandbox/hello.c",
            "output": "/home/user/sandbox/hello.o"
            }
            ]
        </programlisting>

        <para>

            Notice that the generated database contains only an entry for
            the <filename>hello.c/hello.o</filename> pairing,
            and nothing for the generation of the final executable
            <filename>hello</filename>
            - the transformation of
            <filename>hello.o</filename>
            to
            <filename>hello</filename>
            does not have any information that affects interpretation
            of the source code,
            so it is not interesting to the compilation database.

        </para>

        <para>

            Although it can be a little surprising at first glance,
            a compilation database target is, like any other target,
            subject to &scons; target selection rules.
            This means if you set a default target (that does not
            include the compilation database), or use command-line
            targets, it might not be selected for building.
            This can actually be an advantage, since you don't
            necessarily want to regenerate the compilation database
            every build.
            The following example
            shows selecting relative paths (the default)
            for output and source,
            and also giving a non-default name to the database.
            In order to be able to generate the database separately from building,
            an alias is set referring to the database,
            which can then be used as a target - here we are only
            building the compilation database target, not the code.

        </para>

        <scons_example name="external_cdb_ex2">
            <file name="SConstruct" printme="1">
                env = Environment()
                env.Tool('compilation_db')
                cdb = env.CompilationDatabase('compile_database.json')
                Alias('cdb', cdb)
                env.Program('test_main.c')
            </file>
            <file name="test_main.c">
                #include "test_main.h"
                int main( int argc, char* argv[] )
                {
                return 0;
                }
            </file>
            <file name="test_main.h">
                /* dummy include file */
            </file>
        </scons_example>

        <scons_output example="external_cdb_ex2" suffix="1">
            <scons_output_command>scons -Q cdb</scons_output_command>
        </scons_output>

        <para>
            <filename>compile_database.json</filename>
            contains:
        </para>

        <programlisting language="json">
            [
            {
            "command": "gcc -o test_main.o -c test_main.c",
            "directory": "/home/user/sandbox",
            "file": "test_main.c",
            "output": "test_main.o"
            }
            ]
        </programlisting>

        <para>

            The following (incomplete) example shows using filtering
            to separate build variants.
            In the case of using variants,
            you want different compilation databases for each,
            since the build parameters differ, so the code analysis
            needs to see the correct build lines for the 32-bit build
            and 64-bit build hinted at here.
            For simplicity of presentation,
            the example omits the setup details of the variant directories:

        </para>

        <sconstruct>
            env = Environment()
            env.Tool('compilation_db')

            env1 = env.Clone()
            env1['COMPILATIONDB_PATH_FILTER'] = 'build/linux32/*'
            env1.CompilationDatabase('compile_commands-linux32.json')

            env2 = env.Clone()
            env2['COMPILATIONDB_PATH_FILTER'] = 'build/linux64/*'
            env2.CompilationDatabase('compile_commands-linux64.json')
        </sconstruct>

        <para>
            <filename>compile_commands-linux32.json</filename>
            contains:
        </para>

        <programlisting language="json">
            [
            {
            "command": "gcc -m32 -o build/linux32/test_main.o -c test_main.c",
            "directory": "/home/user/sandbox",
            "file": "test_main.c",
            "output": "build/linux32/test_main.o"
            }
            ]
        </programlisting>

        <para>
            <filename>compile_commands-linux64.json</filename>
            contains:
        </para>

        <programlisting language="json">
            [
            {
            "command": "gcc -m64 -o build/linux64/test_main.o -c test_main.c",
            "directory": "/home/user/sandbox",
            "file": "test_main.c",
            "output": "build/linux64/test_main.o"
            }
            ]
        </programlisting>

    </section>

    <section>
        <title>Ninja Build Generator</title>

        <note>
            <para>
                This is an experimental new feature. It is subject to change and/or removal without depreciation cycle
            </para>
            <para>
                To use this tool you must install pypi's <ulink url="https://pypi.org/project/ninja/">ninja
                package</ulink>.
                This can be done via
                <command>pip install ninja</command>
            </para>
            <para>
                To enable this feature you'll need to use one of the following
            </para>
            <example_commands>
                # On the command line
                --experimental=ninja

                # Or in your SConstruct
                SetOption('experimental', 'ninja')
            </example_commands>

        </note>

        <para>
            This tool will enabled creating a ninja build file from your SCons based build system. It can then invoke
            ninja to run your build. For most builds ninja will be significantly faster, but you may have to give up
            some accuracy. You are NOT advised to use this for production builds. It can however significantly speed up
            your build/debug/compile iterations.
        </para>
        <para>
            It's not expected that the ninja builder will work for all builds at this point. It's still under active
            development. If you find that your build doesn't work with ninja please bring this to the users mailing list
            or devel channel on our discord server.
        </para>
        <para>
            Specifically if your build has many (or even any) python function actions you may find that the ninja build
            will be slower as it will run ninja, which will then run SCons for each target created by a python action.
            To alleviate some of these, especially those python based actions built into SCons there is special logic to
            implement those actions via shell commands in the ninja build file.
        </para>
        <para>
            <ulink url="https://ninja-build.org/">
                <citetitle>Ninja Build System</citetitle>
            </ulink>
        </para>

        <para>
            <ulink url="https://ninja-build.org/manual.html#ref_ninja_file">
                <citetitle>Ninja File Format Specification</citetitle>
            </ulink>
        </para>
    </section>
</chapter>