Merge branch 'master' of github.com:SCons/scons

2 files changed, 137 insertions, 112 deletions

diff --git a/CONTRIBUTING.rst b/CONTRIBUTING.rst
index c179678..d4df14a 100644
--- a/CONTRIBUTING.rst
+++ b/CONTRIBUTING.rst
@@ -4,9 +4,9 @@ Contributing to SCons
 Introduction
 ===========
 
-Thanks for taking the time to contribute!
+Thanks for taking the time to contribute to SCons!
 
-This page will talk a bit about the development process,
+This will give a brief overview of the development process,
 and about the SCons tree (right here, if you're reading this
 in Github, or in a cloned repository).
 
@@ -24,7 +24,7 @@ there is a web interface.
 Resources
 =========
 
-Here are some important pointers:
+Here are some important pointers to other resources:
 
   * `SCons Home Page <https://scons.org>`_
   * `Github Project Page <https://github.com/scons/scons>`_
@@ -38,8 +38,8 @@ Reporting Bugs
 
 One of the easiest ways to contribute is by filing bugs.
 The SCons project welcomes bug reports and feature requests,
-but we have do a preference for having talked about it first -
-we request you send an email to the 
+but we *do* have a preference for having talked about them first -
+we request you send an email to the
 `SCons Users Mailing List <https://two.pairlist.net/mailman/listinfo/scons-users>`_
 or hop on the Discord channel (see link above), and if so
 instructed, then proceed to an issue report.
@@ -54,23 +54,29 @@ workarounds for the problem you've run into::
 About the Development Tree
 ==========================
 
-This tree has more logic than just the SCons code itself -
-there's quite a bit to do with packaging it in a couple
+This tree contains a lot more than just the SCons engine itself.
+Some of it has to do with packaging it in a couple
 of forms: a Python-installable package (source distribution
-and installable wheel file), a portable zip (or tar) distribution
+and installable wheel file, which get uploaded to the Python
+Package Index), a portable zip (or tar) distribution
 called "scons-local", and a full source bundle.  You usually
-don't need to worry about the packaging methods when working
-on a source or doc contribution.
-
-To the extent that this tree is about building SCons packages, the *full*
-development cycle is not just to test the code directly, but to package SCons,
-unpack the package, "install" SCons in a test subdirectory, and then to run
-the tests against the unpacked and installed software.  This helps eliminate
-problems caused by, for example, failure to update the list of files to be
-packaged.
+don't need to worry about the packaging parts when working
+on a source or doc contribution - unless you're adding an entirely
+new file, then the packaging bits may need to know about it. The
+project maintainers can usually help with that part.
+There are also tests and tools in the tree.
+
+The *full* development cycle is not just to test code changes directly,
+but also to package SCons, unpack the package, install SCons in a test
+location, and then run the tests against the unpacked and installed
+software.  This helps eliminate problems caused by the development
+tree being different than the tree anyone else would see (files
+not checked in to git, files not added to package manifest, etc.)
+Most of that is handled by the CI setup driven through GitHub,
+which runs checks for each update to a Pull Request.
 
 For just working on making an individual change to the SCons source, however,
-you don't actually need to build or install SCons; you *can* actually edit and
+you don't actually need to build or install SCons; you just edit and
 execute SCons in-place.  See the following sections below for more
 information:
 
@@ -91,10 +97,10 @@ information:
 Executing SCons Without Installing
 ==================================
 
-Being a project written in Python, you don't have to "build"
-SCons to run it, "build" actually refers to the steps taken to prepare
+Since SCons is written entirely in Python, you don't have to "build"
+SCons to run it. "Building" refers to the steps taken to prepare
 for constructing packages, the most time-consuing of which is
-building the documentation.  
+building the documentation.
 
 Documentation is written in a markup language which is a
 light extension of Docbook-XML, and the doc build consists
@@ -119,11 +125,45 @@ unless you are doing development that relates to the packaging.
 
 Often when doing SCons development, you have a sample project that
 isn't working right, and it's useful to be able to build that
-project from place you're working on changes to SCons. You
+project from the place you're working on changes to SCons. You
 can do that with the ``C`` (change directory) option::
 
     $ python scripts/scons.py -C /some/other/location [arguments]
 
+There is another approach that kind of reverses that order:
+construct a Python virtualenv and install the development tree in it.
+If you're not familiar with virtualenvs, there's an example here::
+
+    https://scons-cookbook.readthedocs.io/en/latest/#setting-up-a-python-virtualenv-for-scons
+
+Follow the initial steps, except omit installing ``scons`` as a package.
+When the virtualenv is contructed and activated, go to your working SCons
+git tree and perform a development install instead::
+
+    (myvenv) $ pip install --editable .
+
+Now while this virtualenv is activated, the command ``scons`` will refer
+to this editable version, and you don't have to be "in" this tree
+to run it.
+
+You can also arrange to execute ``scons.py`` from the command line
+by adding it to the ``PATH``, like::
+
+    # on Linux/Mac
+    $ export PATH=$PATH:`pwd`/scripts
+
+    # on Windows
+    C:\> set PATH="%PATH%;C:\path\to\scripts"
+
+Be careful on Windows, the path has a limit of 1024 characters which
+is pretty easy to exceed, and it will just truncate.
+
+You may first need to make ``scons.py`` executable (it should be
+by default, but sometimes things happen)::
+
+    $ chmod +x scripts/scons.py
+
+
 Other Required Software
 =======================
 
@@ -148,15 +188,7 @@ modules in the local ``SCons`` subdirectory tree and then run
 
     $ python scripts/scons.py [arguments]
 
-If you want to be able to just execute your modified version of SCons from the
-command line, you can make it executable and add its directory to your $PATH
-like so::
-
-    $ chmod 755 scripts/scons.py
-    $ export PATH=$PATH:`pwd`/scripts
-
-You should then be able to run this version of SCons by just typing ``scons.py``
-at your UNIX or Linux command line.
+(or, if using the virtualenv/editable approach, ``scons [arguents]``)
 
 Note that the regular SCons development process makes heavy use of automated
 testing.  See the `Testing`_ and `Typical Development Workflow`_ sections below for more
@@ -170,7 +202,7 @@ Debugging
 
 Python comes with a good interactive debugger.  When debugging changes by hand
 (i.e., when not using the automated tests), you can invoke SCons under control
-of the Python debugger by specifying the --debug=pdb option::
+of the Python debugger by specifying the ``--debug=pdb`` option::
 
     $ scons --debug=pdb [arguments]
     > /home/knight/scons/SCons/Script/Main.py(927)_main()
@@ -191,20 +223,21 @@ the debugger at that point.
 The debugger supports single stepping, stepping into functions, printing
 variables, etc.
 
-Trying to debug problems found by running the automated tests (see the
-`Testing`_ section, below) is more difficult, because the test automation
-harness re-invokes SCons and captures output. Consequently, there isn't an
-easy way to invoke the Python debugger in a useful way on any particular SCons
-call within a test script.
-
-The most effective technique for debugging problems that occur during an
-automated test is to use the good old tried-and-true technique of adding
-statements to print tracing information.  But note that you can't just use
-the ``print`` function, or even ``sys.stdout.write()`` because those change the
-SCons output, and the automated tests usually look for matches of specific
-output strings to decide if a given SCons invocation passes the test -
-so these additions may cause apparent failures different than the one you
-are trying to debug.
+When debugging unexpected behavior when running the test suite
+(see the `Testing`_ section, below), it can get a bit more complicated.
+For the Unit Tests, you will be running in-process, and so the
+``runtest.py`` script's debug option is helpful in getting things set up.
+
+Trying to debug problems found by running the end-to-end tests is
+more difficult, because the test automation harness re-invokes SCons and
+captures output - essentially, an instance of SCons is being runs as
+a "black box", and so it is considerably harder to interact with it
+effectively. The way forward is usually to add statements to trace progress.
+You can't just use the ``print`` function directly, or even ``sys.stdout.write()``
+because those change the SCons output, and the end-to-end tests usually
+look for matches of specific output strings to decide if a given SCons
+invocation has behaved as expected - so interleaving your trace information
+would cause lots of mismatches, and often obscure what you are trying to debug.
 
 To deal with this, SCons supports a ``Trace()`` function that (by default) will
 print messages to your console screen (``/dev/tty`` on UNIX or Linux, ``con`` on
@@ -441,8 +474,8 @@ debian/
     project (and thus inherited by projects which derive from it, if
     they haven't made their own packages). See:
 
-    - <https://packages.debian.org/bullseye/scons>_
-    - <https://packages.ubuntu.com/impish/scons>_
+    - `Debian scons packages <https://packages.debian.org/search?keywords=scons&searchon=names&suite=all&section=all>`_
+    - `Ubuntu scons packages <https://packages.ubuntu.com/search?keywords=scons&searchon=names&suite=all&section=all>`_
 
 doc/
     SCons documentation.  A variety of things here, in various stages of
@@ -491,12 +524,12 @@ SConstruct
     software.)
 
 SCons/
-    Where the actual source code is kept, of course.
+    This is the source code of the engine, plus unit tests and
+    documentation stubs kept together with pieces of the engine.
 
 test/
     End-to-end tests of the SCons utility itself.  These are separate from the
-    individual module unit tests, which live side-by-side with the modules
-    under SCons.
+    individual module unit tests.
 
 testing/
     SCons testing framework.
diff --git a/doc/user/external.xml b/doc/user/external.xml
index 0eab32e..ceeece0 100644
--- a/doc/user/external.xml
+++ b/doc/user/external.xml
@@ -129,13 +129,13 @@
 
         <scons_example name="external_cdb_ex1">
             <file name="SConstruct" printme="1">
-                env = Environment(COMPILATIONDB_USE_ABSPATH=True)
-                env.Tool('compilation_db')
-                env.CompilationDatabase()
-                env.Program('hello.c')
+env = Environment(COMPILATIONDB_USE_ABSPATH=True)
+env.Tool('compilation_db')
+env.CompilationDatabase()
+env.Program('hello.c')
             </file>
             <file name="hello.c">
-                int main( int argc, char* argv[] )
+                int main(int argc, char* argv[])
                 {
                 return 0;
                 }
@@ -152,14 +152,14 @@
         </para>
 
         <programlisting language="json">
-            [
-            {
-            "command": "gcc -o hello.o -c hello.c",
-            "directory": "/home/user/sandbox",
-            "file": "/home/user/sandbox/hello.c",
-            "output": "/home/user/sandbox/hello.o"
-            }
-            ]
+[
+    {
+        "command": "gcc -o hello.o -c hello.c",
+        "directory": "/home/user/sandbox",
+        "file": "/home/user/sandbox/hello.c",
+        "output": "/home/user/sandbox/hello.o"
+    }
+]
         </programlisting>
 
         <para>
@@ -202,15 +202,15 @@
 
         <scons_example name="external_cdb_ex2">
             <file name="SConstruct" printme="1">
-                env = Environment()
-                env.Tool('compilation_db')
-                cdb = env.CompilationDatabase('compile_database.json')
-                Alias('cdb', cdb)
-                env.Program('test_main.c')
+env = Environment()
+env.Tool('compilation_db')
+cdb = env.CompilationDatabase('compile_database.json')
+Alias('cdb', cdb)
+env.Program('test_main.c')
             </file>
             <file name="test_main.c">
                 #include "test_main.h"
-                int main( int argc, char* argv[] )
+                int main(int argc, char* argv[])
                 {
                 return 0;
                 }
@@ -225,23 +225,21 @@
         </scons_output>
 
         <para>
-            <filename>compile_database.json</filename>
-            contains:
+            <filename>compile_database.json</filename> contains:
         </para>
 
         <programlisting language="json">
-            [
-            {
-            "command": "gcc -o test_main.o -c test_main.c",
-            "directory": "/home/user/sandbox",
-            "file": "test_main.c",
-            "output": "test_main.o"
-            }
-            ]
+[
+    {
+        "command": "gcc -o test_main.o -c test_main.c",
+        "directory": "/home/user/sandbox",
+        "file": "test_main.c",
+        "output": "test_main.o"
+    }
+]
         </programlisting>
 
         <para>
-
             The following (incomplete) example shows using filtering
             to separate build variants.
             In the case of using variants,
@@ -251,20 +249,19 @@
             and 64-bit build hinted at here.
             For simplicity of presentation,
             the example omits the setup details of the variant directories:
-
         </para>
 
         <sconstruct>
-            env = Environment()
-            env.Tool('compilation_db')
+env = Environment()
+env.Tool("compilation_db")
 
-            env1 = env.Clone()
-            env1['COMPILATIONDB_PATH_FILTER'] = 'build/linux32/*'
-            env1.CompilationDatabase('compile_commands-linux32.json')
+env1 = env.Clone()
+env1["COMPILATIONDB_PATH_FILTER"] = "build/linux32/*"
+env1.CompilationDatabase("compile_commands-linux32.json")
 
-            env2 = env.Clone()
-            env2['COMPILATIONDB_PATH_FILTER'] = 'build/linux64/*'
-            env2.CompilationDatabase('compile_commands-linux64.json')
+env2 = env.Clone()
+env2["COMPILATIONDB_PATH_FILTER"] = "build/linux64/*"
+env2.CompilationDatabase('compile_commands-linux64.json')
         </sconstruct>
 
         <para>
@@ -273,30 +270,29 @@
         </para>
 
         <programlisting language="json">
-            [
-            {
-            "command": "gcc -m32 -o build/linux32/test_main.o -c test_main.c",
-            "directory": "/home/user/sandbox",
-            "file": "test_main.c",
-            "output": "build/linux32/test_main.o"
-            }
-            ]
+[
+    {
+        "command": "gcc -o hello.o -c hello.c",
+        "directory": "/home/mats/github/scons/exp/compdb",
+        "file": "hello.c",
+        "output": "hello.o"
+    }
+]
         </programlisting>
 
         <para>
-            <filename>compile_commands-linux64.json</filename>
-            contains:
+            <filename>compile_commands-linux64.json</filename> contains:
         </para>
 
         <programlisting language="json">
-            [
-            {
-            "command": "gcc -m64 -o build/linux64/test_main.o -c test_main.c",
-            "directory": "/home/user/sandbox",
-            "file": "test_main.c",
-            "output": "build/linux64/test_main.o"
-            }
-            ]
+[
+    {
+        "command": "gcc -m64 -o build/linux64/test_main.o -c test_main.c",
+        "directory": "/home/user/sandbox",
+        "file": "test_main.c",
+        "output": "build/linux64/test_main.o"
+    }
+]
         </programlisting>
 
     </section>
@@ -309,8 +305,6 @@
                 It is subject to change and/or removal without a depreciation cycle.
     </para>
 
-
-
     <para>
             Loading the &t-link-ninja; tool into SCons will make significant changes
             in SCons' normal functioning.
@@ -341,8 +335,6 @@ SetOption('experimental', 'ninja')
     </example_commands>
 </note>
 
-
-
         <para>
             Ninja is a small build system that tries to be fast
             by not making decisions. &SCons; can at times be slow