The &SCons; project has paid particular attention from day one to the development process. One of the first internal documents produced was a set of Developer's Guidelines to provide a loose framework for what we were trying to accomplish and how we would go about accomplishing it. These Guidelines cover things like: &SCons; will be written to Python version 2.4 (to ensure usability by a wide install base). How &SCons; is be tested: which infrastructure modules to use, what platforms to test on, etc. Expectations for developers (subscribe to the mailing list, encouraged to register at SourceForge). Brief outline of how to use the change management systems (Aegis and CVS) for &SCons; development;. Establishing these guidelines up front had two purposes: 1) Demonstrate the seriousness of the project to anyone wondering about joining the effort; 2) Give potential developers an idea up front as to whether their development style would mesh with the rest of the project.

One of the most important aspects of the &SCons; development process is the use of Peter Miller's Aegis change management system. I had been using Aegis for personal projects for several years, and found its development methodology vastly improved the quality of my programming. I was consequently committed to using it for &SCons; development. Aegis provides a number of things, including: A flexible source code control and branching model. A defined process with separate development, review and integration steps. A distributed development model based on distribution of atomic change sets. The single most important reason for using Aegis, however, is its management of automated tests as part of the development process.

The &SCons; project has made extensive use of automated tests from day one, taking inspiration mostly from Aegis, partly from the eXtreme Programming model, and with a little home-brew scripting for glue.

The underlying criteria for testing changes to the &SCons; code are taken from Aegis: Every change must have one or more new or modified tests checked in along with the code. The new code being checked in must pass all of the new and/or modified tests. The old, already checked-in code in must fail all of the new and/or modified tests. The new code being checked in must pass all unmodified, already checked-in tests. In practice, these restrictions can be overridden as necessary--for example, when changing comments or documentation. The criterion that surprises many people is having the old code fail the tests in the change. This makes sure that the new tests or modified tests really do exercise the bug fix or feature being added by the change. Together, these criteria ensure that every newly checked-in version &SCons; conforms to defined behavior, as defined by the tests. Whenever a bug is found, its fix is checked in with a new or modified test that guarantees the bug will not recur in the future. We have already built up a regression test base of almost 90 tests that cover the vast majority of &SCons;' functionality.

Testing standards are no good if they're too much of a burden for developers, who will at best work around or ignore the testing requirements, or at worst stop contributing code and go join a project that's more fun. To this end, good testing infrastructure that makes it easy to write tests is crucial. &SCons; development uses two development methodologies, one for the individual modules in the build engine, and the other for end-to-end tests of the &SCons; script. For the build engine modules, we use PyUnit. Every change to a build engine module must have a change to its corresponding unit tests, which live side-by-side in a separate file that imports module. As we build up a large body of unit tests, this ensures that the build engine will perform correctly whenever someone uses it in some application other than the &SCons; script itself. For end-to-end script tests, we have developed two modules to make writing tests easy. The first, TestCmd.py, is a generic module for testing commands or scripts (in any language, not just Python). The second module, TestScons.py, is a subclass of the generic TestCmd.py module. TestScons.py takes care of initialization and displaying error conditions specific to testing &SCons;. In practice, simple tests only need to initialize a test object, use the object to write some input files, run &SCons;, and then check whatever criteria determine whether the test passed or failed. A complete test of the &Program; method, for example, looks like this: test = TestSCons.TestSCons() test.write('SConstruct', """env = Environment() env.Program(target = 'foo', source = 'foo.c') """) test.write('foo.c', """ int main(int argc, char *argv[]) { argv[argc++] = "-"; /* dummy use of args */ printf("foo.c successfully compiled\\n"); exit (0); } """) test.run(arguments = 'foo') # runs SCons test.run(program = test.workpath('foo')) test.fail_test(test.stdout() != "foo.c successfully compiled\n") test.pass_test()

Registration of the &SCons; project was approved at SourceForge on 29 June 2001. Within a week, the initial code base was checked in, mailing lists were created, and the web site was set up. We started making use of the task-list manager to track what we had to finish for initial release. The obvious complication was how to use structured testing methodology of Aegis when SourceForge uses CVS for source control. Not using the SourceForge CVS tree would have had two significant disadvantages: one, missing out on the archiving and central location in the event of disaster; two, people coming to the SourceForge project page wouldn't be able to browse the source. The latter was particularly important in the early stages of development, in order to avoid any impression that this was Yet Another Project that starts with a bang and then dwindles as the initial enthusiasm starts to wear off. The solution was to use the SourceForge CVS repository for read-only access to the source. &SCons; developers are welcome to use CVS for their development, but the changes are not committed to the SourceForge repository. Instead, patches are sent to the integrator for processing through Aegis. When the change has been integrated into the Aegis repository, a home-brew script translates the Aegis change into a virtual shell script of commands that copy the necessary files from Aegis and check them in to CVS at SourceForge. (In practice, write access is not actually disabled for registered developers, but if they do make any changes directly at SourceForge, they can be overwritten at the next Aegis update.)