From a8d1ef73713371da16300fd3bbd53cd993aa9e5d Mon Sep 17 00:00:00 2001 From: Mats Wichmann Date: Thu, 8 Dec 2022 10:18:47 -0700 Subject: uguide: variant dir tweaking [skip appveyor] Try to clarify some more about variant dirs (people are still getting confused), without getting too technical about it, in the spirit of the User Guide. Also some stuff about hierarchical builds - both manpage and user guide. User guide chunks that were modified anyway had section IDs added. This is part of a plan to gradually get rid of horrible-looking autogenerated links which are not constant (problem when Google archives something and then maybe it doesn't work). May look like this in a browser: https://scons.org/doc/production/HTML/scons-user.html#idp105548893220944 With assigned IDs, looks more like: .../HTML/scons-user.html#sect-variant-sconscript So far only three userguide files changed. Signed-off-by: Mats Wichmann --- doc/man/scons.xml | 51 +++++++--- doc/user/hierarchy.xml | 90 ++++++++--------- doc/user/separate.xml | 267 ++++++++++++++++++++++++++++++------------------- doc/user/variants.xml | 34 ++----- 4 files changed, 249 insertions(+), 193 deletions(-) diff --git a/doc/man/scons.xml b/doc/man/scons.xml index 8dda049..824cd41 100644 --- a/doc/man/scons.xml +++ b/doc/man/scons.xml @@ -145,33 +145,44 @@ to support additional input file types. Information about files involved in the build, -including a cryptographic hash of the contents, +including a cryptographic hash of the contents of source files, is cached for later reuse. By default this hash (the &contentsig;) is used to determine if a file has changed since the last build, -but this can be controlled by selecting an appropriate +although this can be controlled by selecting an appropriate &f-link-Decider; function. Implicit dependency files are also part of out-of-date computation. The scanned implicit dependency information can optionally be cached and used to speed up future builds. -A hash of each executed build action (the &buildsig; +A hash of each executed build action (the &buildsig;) is cached, so that changes to build instructions (changing flags, etc.) or to the build tools themselves (new version) can also trigger a rebuild. +&SCons; supports the concept of separated source and build +directories through the definition of +variant directories +(see the &f-link-VariantDir; function). + + + When invoked, &scons; looks for a file named &SConstruct; in the current directory and reads the build configuration from that file (other names are allowed, -see for more information). -The &SConstruct; +see +and the option +for more information). +The build may be structured in a hierarchical manner: +the &SConstruct; file may specify subsidiary configuration files by calling the -&f-link-SConscript; function. +&f-link-SConscript; function, +and these may, in turn, do the same. By convention, these subsidiary files are named &SConscript;, @@ -182,7 +193,14 @@ is used to refer generically to the complete set of configuration files for a project (including the &SConstruct; file), -regardless of the actual file names or number of such files. +regardless of the actual file names or number of such files. +A hierarchical build is not recursive - all of +the SConscript files are processed in a single pass, +although each is processed in a separate context so +as not to interfere with one another. &SCons; provides +mechanisms for information to be shared between +SConscript files when needed. + Before reading the &SConscript; files, &scons; @@ -870,9 +888,10 @@ files). duplicate -Print a line for each unlink/relink (or copy) of a variant file from -its source file. Includes debugging info for unlinking stale variant -files, as well as unlinking old targets before building them. +Print a line for each unlink/relink (or copy) of a file in +a variant directory from its source file. +Includes debugging info for unlinking stale variant directory files, +as well as unlinking old targets before building them. @@ -3686,9 +3705,9 @@ The default value is specifies a file which collects the output from commands that are executed to check for the existence of header files, libraries, etc. The default is #/config.log. -If you are using the -&VariantDir; function, -you may want to specify a subdirectory under your variant directory. +If you are using variant directories, +you may want to place the log file for a given build +under that build's variant directory. config_h specifies a C header file where the results of tests @@ -6720,7 +6739,7 @@ For example, the command line string: echo Last build occurred $TODAY. > $TARGET -but the build signature added to any target files would be computed from: +but the &buildsig; added to any target files would be computed from: echo Last build occurred . > $TARGET @@ -6741,8 +6760,8 @@ The callable must accept four arguments: env is the &consenv; to use for context, and for_signature is a boolean value that tells the callable -if it is being called for the purpose of generating a build signature. -Since the build signature is used for rebuild determination, +if it is being called for the purpose of generating a &buildsig;. +Since the &buildsig; is used for rebuild determination, variable elements that do not affect whether a rebuild should be triggered should be omitted from the returned string diff --git a/doc/user/hierarchy.xml b/doc/user/hierarchy.xml index e63d230..17874d0 100644 --- a/doc/user/hierarchy.xml +++ b/doc/user/hierarchy.xml @@ -1,4 +1,10 @@ + + %scons; @@ -18,32 +24,8 @@ xmlns="http://www.scons.org/dbxsd/v1.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.scons.org/dbxsd/v1.0 http://www.scons.org/dbxsd/v1.0/scons.xsd"> -Hierarchical Builds - - +Hierarchical Builds + %scons; @@ -17,98 +23,111 @@ xmlns="http://www.scons.org/dbxsd/v1.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.scons.org/dbxsd/v1.0 http://www.scons.org/dbxsd/v1.0/scons.xsd"> -Separating Source and Build Trees: Variant Directories - + - It's often useful to keep any built files completely - separate from the source files. Consider if you have a - project to build software for a variety of different - controller hardware. The boards are able to share a - lot of code, so it makes sense to keep them in the same - source tree, but certain build options in the source code - and header files differ. If you build "Controller A" first, - then "Controller B", on the "Controller B" build everything would - have to be rebuilt, because &SCons; recognizes that the build - instructions are different from those used in the "Controller A" - build for each target - the build instructions are part of - &SCons;'s out-of-date calculation. - Now when you go back and build for "Controller A", - things have to be rebuilt from scratch again for the same reason. - However, if you can separate the locations of the output files, - this problem can be avoided. - You can even set up to do both builds in one invocation of &SCons;. + Having a separated build also helps you keep your source tree clean - + there is less chance of accidentally checking in build products + to version control that were not intended to be checked in. + You can add a separated build directory to your + version control system's list of items not to track. + You can even remove the whole build tree with a single command without + risking removing any of the source code. - You can enable this separation by establishing one or more - variant directory trees - that are used to perform the build in, and thus provide a unique - home for object files, libraries, and executable programs, etc. - for a specific flavor, or variant, of build. &SCons; tracks - targets by their path, so when the variant directory is included, - objects belonging to "Controller A" can have different - build instructions than those belonging to "Controller B" without - triggering ping-ponging rebuilds. + The key to making this separation work is the ability to + do out-of-tree builds: building under a separate root + than the sources being built. + You set up out of tree builds by establishing what &SCons; + calls a variant directory, + a place where you can build a single variant of your software + (of course you can define more than one of these if you need to). + Since &SCons; tracks targets by their path, it is able to distinguish + build products like build/A/network.obj + of the Controller A build + from build/B/network.obj + of the Controller B build, + thus avoiding conflicts. - &SCons; provides two ways to do this, - one through the &f-link-SConscript; function that we've already seen, + &SCons; provides two ways to establish variant directories, + one through the &f-link-SConscript; function that we have already seen, and the second through a more flexible &f-link-VariantDir; function. + + The variant directory mechanism does support doing multiple builds + in one invocation of &SCons;, but the remainder of this chapter + will focus on setting up a single build. You can combine these + techniques with ones from the previous chapter and elsewhere + in this Guide to set up more complex scenarios. + + + + - Historical note: the &VariantDir; function - used to be called &BuildDir;, a name which was - removed because the &SCons; functionality + The &VariantDir; function used to be called &BuildDir;, + a name which was changed because it turned out to be confusing: + the &SCons; functionality differs from a familiar model of a "build directory" - implemented by other build systems like GNU Autotools. + implemented by certain other build systems like GNU Autotools. You might still find references to the old name on the Internet in postings about &SCons;, but it no longer works. - + -
+
Specifying a Variant Directory Tree as Part of an &SConscript; Call The most straightforward way to establish a variant directory tree - relies the fact that the usual way to + relies on the fact that the usual way to set up a build hierarchy is to have an - SConscript file in the source subdirectory. + &SConscript; file in the source directory. If you pass a &variant_dir; argument to the &f-link-SConscript; function call: @@ -130,7 +149,7 @@ int main() { printf("Hello, world!\n"); } &SCons; will then build all of the files in - the &build; subdirectory: + the &build; directory: @@ -143,46 +162,73 @@ int main() { printf("Hello, world!\n"); } - No files were built in &src;, they went to &build;. - The build output might show a bit of a surprise: + No files were built in &src;: the object file build/hello.o and the executable file build/hello - were built in the &build; subdirectory, - as expected. - But even though our &hello_c; file lives in the &src; subdirectory, - &SCons; has actually compiled a + were built in the &build; directory, as expected. + But notice that even though our &hello_c; file actually + lives in the &src; directory, &SCons; has compiled a build/hello.c file to create the object file, and that file is now seen in &build;. + + + + + + You can ask &SCons; to show the dependency tree to illustrate + a bit more: + + scons -Q --tree=prune + + What's happened is that &SCons; has duplicated - the &hello_c; file from the &src; subdirectory - to the &build; subdirectory, + the &hello_c; file from the &src; directory + to the &build; directory, and built the program from there (it also duplicated &SConscript;). The next section explains why &SCons; does this. + + + The nice thing about the &SConscript; approach is it is almost + invisible to you: + this build looks just like an ordinary in-place build + except for the extra &variant_dir; argument in the + &f-link-SConscript; call. + &SCons; handles all the path adjustments for the + out of tree &build; directory while it processes that SConscript file. + + +
-
+
Why &SCons; Duplicates Source Files in a Variant Directory Tree - The important thing to understand is that when you set up a variant directory, - &SCons; performs the build in that directory. - It turns out it's easiest to ensure where build products end up - by just building in place. - Since the build is happening in a place different from where the - sources are, the most straightforward way to guarantee a correct build - is for &SCons; to copy them there. + When you set up a variant directory &SCons; conceptually behaves as + if you requested a build in that directory. + As noted in the previous chapter, + all builds actually happen from the top level directory, + but as an aid to understanding how &SCons; operates, think + of it as build in place in the variant directory, + not build in source but send build artifacts + to the variant directory. + It turns out in place builds are easier to get right than out + of tree builds - so by default &SCons; simulates an in place build + by making the variant directory look just like the source directory. + The most straightforward way to do that is by making copies + of the files needed for the build. @@ -192,7 +238,11 @@ int main() { printf("Hello, world!\n"); } in variant directories is simply that some tools (mostly older versions) are written to only build their output files - in the same directory as the source files. + in the same directory as the source files - such tools often don't + have any option to specify the output file, and the tool just + uses a predefined output file name, + or uses a derived variant of the source file name, + dropping the result in the same directory. In this case, the choices are either to build the output file in the source directory and move it to the variant directory, @@ -204,9 +254,9 @@ int main() { printf("Hello, world!\n"); } Additionally, relative references between files - can cause problems if we don't - just duplicate the hierarchy of source files - in the variant directory. + can cause problems which are resolved by + just duplicating the hierarchy of source files + into the variant directory. You can see this at work in use of the C preprocessor #include mechanism with double quotes, not angle brackets: @@ -240,8 +290,8 @@ int main() { printf("Hello, world!\n"); } Although source-file duplication guarantees a correct build - even in these end-cases, - it can usually be safely disabled. + even in these edge cases, + it can usually be safely disabled. The next section describes how you can disable the duplication of source files in the variant directory. @@ -250,19 +300,19 @@ int main() { printf("Hello, world!\n"); }
-
+
Telling &SCons; to Not Duplicate Source Files in the Variant Directory Tree In most cases and with most tool sets, - &SCons; can place its target files in a build subdirectory + &SCons; can use sources directly from the source directory without - duplicating the source files + duplicating them into the variant directory before building, and everything will work just fine. - You can disable the default &SCons; behavior + You can disable the default &SCons; duplication behavior by specifying duplicate=False - when you call the &SConscript; function: + when you call the &f-link-SConscript; function: @@ -272,11 +322,11 @@ SConscript('src/SConscript', variant_dir='build', duplicate=False) - When this flag is specified, - &SCons; uses the variant directory - like most people expect--that is, - the output files are placed in the variant directory - while the source files stay in the source directory: + When this flag is specified, the results of a build + look more like the mental model people may have from other + build systems - that is, + the output files end up in the variant directory + while the source files do not. @@ -292,14 +342,22 @@ hello hello.o + + + If disabling duplication causes any problems, + just return to the more cautious approach by letting + &SCons; go back to duplicating files. + + +
-
+
The &VariantDir; Function - Use the &VariantDir; function to establish that target + Use the &f-link-VariantDir; function to establish that target files should be built in a separate directory from the source files: @@ -318,8 +376,8 @@ int main() { printf("Hello, world!\n"); } - Note that when you're not using - an &SConscript; file in the &src; subdirectory, + Note that when you are not using + an &SConscript; file in the &src; directory, you must actually specify that the program must be built from the build/hello.c @@ -345,7 +403,7 @@ int main() { printf("Hello, world!\n"); } You can specify the same duplicate=False argument - that you can specify for an &SConscript; call: + that you can specify for an &f-link-SConscript; call: @@ -375,12 +433,12 @@ int main() { printf("Hello, world!\n"); }
-
+
Using &VariantDir; With an &SConscript; File - Even when using the &VariantDir; function, + Even when using the &f-link-VariantDir; function, it's more natural to use it with a subsidiary &SConscript; file, because then you don't have to adjust your individual @@ -428,21 +486,26 @@ int main() { printf("Hello, world!\n"); } - Notice that this is completely equivalent - to the use of &SConscript; that we - learned about in the previous section. + This is completely equivalent + to the use of &f-link-SConscript; with the + variant_dir argument + from earlier in this chapter, + but did require callng the SConscript using the already established + variant directory path to trigger that behavior. + If you use SConscript('src/SConscript') + you would get a normal in-place build in &src;.
-
+
Using &Glob; with &VariantDir; The &f-link-Glob; file name pattern matching function - works just as usual when using &VariantDir;. + works just as usual when using &f-link-VariantDir;. For example, if the src/SConscript looks like this: @@ -496,7 +559,7 @@ const char * f2(); + %scons; @@ -13,36 +19,12 @@ %variables-mod; ]> -
-Variant Build Examples - - +Variant Build Examples -- cgit v0.12 From c544c4c2dc2c4fe7c04703479f088377ac7cb0cf Mon Sep 17 00:00:00 2001 From: Mats Wichmann Date: Wed, 14 Dec 2022 14:59:48 -0700 Subject: Clarify build tree separation wording [skip appveyor] Review pointed out that a use of the term "build" wasn't that clear, "build tree" would be more descriptive. Signed-off-by: Mats Wichmann --- doc/user/separate.xml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/user/separate.xml b/doc/user/separate.xml index f0636cc..4955d0e 100644 --- a/doc/user/separate.xml +++ b/doc/user/separate.xml @@ -61,7 +61,7 @@ Copyright The SCons Foundation - Having a separated build also helps you keep your source tree clean - + Having a separated build tree also helps you keep your source tree clean - there is less chance of accidentally checking in build products to version control that were not intended to be checked in. You can add a separated build directory to your -- cgit v0.12 From 889ce62c4c053a09bf9c8a029b848cc574297729 Mon Sep 17 00:00:00 2001 From: Mats Wichmann Date: Fri, 16 Dec 2022 07:49:43 -0700 Subject: uguide: clarify VariantDir example [skip appvyor] A little more tweaking to show all references have to be to the variant dir (the "implied target" form of the example might let readers miss it applies to targets too). Signed-off-by: Mats Wichmann --- doc/user/separate.xml | 37 +++++++++++++++++++++---------------- 1 file changed, 21 insertions(+), 16 deletions(-) diff --git a/doc/user/separate.xml b/doc/user/separate.xml index 4955d0e..0485a55 100644 --- a/doc/user/separate.xml +++ b/doc/user/separate.xml @@ -35,7 +35,7 @@ Copyright The SCons Foundation - + Consider if you have a project to build an embedded software system for a variety of different controller hardware. @@ -78,7 +78,7 @@ Copyright The SCons Foundation than the sources being built. You set up out of tree builds by establishing what &SCons; calls a variant directory, - a place where you can build a single variant of your software + a place where you can build a single variant of your software (of course you can define more than one of these if you need to). Since &SCons; tracks targets by their path, it is able to distinguish build products like build/A/network.obj @@ -98,7 +98,7 @@ Copyright The SCons Foundation - + The variant directory mechanism does support doing multiple builds in one invocation of &SCons;, but the remainder of this chapter will focus on setting up a single build. You can combine these @@ -173,7 +173,7 @@ int main() { printf("Hello, world!\n"); } build/hello.c file to create the object file, and that file is now seen in &build;. - + @@ -217,7 +217,7 @@ int main() { printf("Hello, world!\n"); } When you set up a variant directory &SCons; conceptually behaves as - if you requested a build in that directory. + if you requested a build in that directory. As noted in the previous chapter, all builds actually happen from the top level directory, but as an aid to understanding how &SCons; operates, think @@ -357,8 +357,8 @@ hello.o - Use the &f-link-VariantDir; function to establish that target - files should be built in a separate directory + You can also use the &f-link-VariantDir; function to establish + that target files should be built in a separate directory tree from the source files: @@ -376,13 +376,18 @@ int main() { printf("Hello, world!\n"); } - Note that when you are not using - an &SConscript; file in the &src; directory, - you must actually specify that - the program must be built from - the build/hello.c - file that &SCons; will duplicate in the - &build; subdirectory. + When using this form, you have to tell &SCons; that + sources and targets are in the variant directory, + and those references will trigger the remapping, + necessary file copying, etc. for an already established + variant directory. Here is the same example in a more + spelled out form to show this more clearly: + + +VariantDir('build', 'src') +env = Environment() +env.Program(target='build/hello', source=['build/hello.c']) + @@ -439,7 +444,7 @@ int main() { printf("Hello, world!\n"); } Even when using the &f-link-VariantDir; function, - it's more natural to use it with + it is more natural to use it with a subsidiary &SConscript; file, because then you don't have to adjust your individual build instructions to use the variant directory path. @@ -492,7 +497,7 @@ int main() { printf("Hello, world!\n"); } from earlier in this chapter, but did require callng the SConscript using the already established variant directory path to trigger that behavior. - If you use SConscript('src/SConscript') + If you call SConscript('src/SConscript') you would get a normal in-place build in &src;. -- cgit v0.12 From 755ab60ef33cf62e2f887cfddb1b084b7f44ba21 Mon Sep 17 00:00:00 2001 From: Mats Wichmann Date: Fri, 16 Dec 2022 08:30:40 -0700 Subject: Updated RELEASE to show VariantDir updated [ci skip] Signed-off-by: Mats Wichmann --- RELEASE.txt | 3 +++ 1 file changed, 3 insertions(+) diff --git a/RELEASE.txt b/RELEASE.txt index 2a32831..c158efb 100644 --- a/RELEASE.txt +++ b/RELEASE.txt @@ -105,6 +105,9 @@ DOCUMENTATION - Updated the User Guide chapter on installation: modernized the notes on Python installs, SCons installs, and having multiple SCons versions present on a single system. +- Updated the User Guide chapter on variant directories with more + explanation, and the introduction of terms like "out of tree" that + may help in forming a mental model. DEVELOPMENT ----------- -- cgit v0.12