diff options
author | Evan Martin <martine@danga.com> | 2011-05-14 23:11:41 (GMT) |
---|---|---|
committer | Evan Martin <martine@danga.com> | 2011-05-14 23:11:41 (GMT) |
commit | 17a0335519638e3f821d38172d104bfad62eb01c (patch) | |
tree | 154ce2a1d75c9124394651068b12612d5a2342a2 /manual.asciidoc | |
parent | 2ec1b4262151a70cb694395dd38d99a937ba13e3 (diff) | |
download | Ninja-17a0335519638e3f821d38172d104bfad62eb01c.zip Ninja-17a0335519638e3f821d38172d104bfad62eb01c.tar.gz Ninja-17a0335519638e3f821d38172d104bfad62eb01c.tar.bz2 |
move various doc files out of top-level directory
Diffstat (limited to 'manual.asciidoc')
-rw-r--r-- | manual.asciidoc | 506 |
1 files changed, 0 insertions, 506 deletions
diff --git a/manual.asciidoc b/manual.asciidoc deleted file mode 100644 index 7aa000e..0000000 --- a/manual.asciidoc +++ /dev/null @@ -1,506 +0,0 @@ -Ninja -===== -Evan Martin <martine@danga.com> - - -Introduction ------------- - -Ninja is yet another build system. It takes as input the -interdependencies of files (typically source code and output -executables) and orchestrates building them, _quickly_. - -Ninja joins a sea of other build systems. Its distinguishing goal is -to be fast. It is born from my work on the Chromium browser project, -which has over 30,000 source files and whose other build systems -(including one built from custom non-recursive Makefiles) can take ten -seconds to start building after changing one file. Ninja is under a -second. - - -Philosophical overview -~~~~~~~~~~~~~~~~~~~~~~ - -Build systems get slow when they need to make decisions. When you are -in a edit-compile cycle you want it to be as fast as possible -- you -want the build system to do the minimum work necessary to figure out -what needs to be built immediately. - -Ninja contains the barest functionality necessary to describe -arbitrary dependency graphs. Its lack of syntax makes it impossible -to express complex decisions. Instead, Ninja is intended to be used -with a separate program generating its input files. The generator -program (like the `./configure` found in autotools projects) can -analyze system dependencies and make as many decisions as possible up -front so that incremental builds stay fast. Going beyond autotools, -even build-time decisions like "which compiler flags should I use?" -or "should I build a debug or release-mode binary?" belong in the -`.ninja` file generator. - -Conceptual overview -~~~~~~~~~~~~~~~~~~~ - -Ninja evaluates a graph of dependencies between files, and runs -whichever commands are necessary to make your build target up to date. -If you are familiar with Make, Ninja is very similar. - -A build file (default name: `build.ninja`) provides a list of _rules_ --- short names for longer commands, like how to run the compiler -- -along with a list of _build_ statements saying how to build files -using the rules -- which rule to apply to which inputs to produce -which outputs. - -Conceptually, `build` statements describe the dependency graph of your -project, while `rule` statements describe how to generate the files -along a given edge of the graph. - -Design goals -~~~~~~~~~~~~ - -Here are some of the design goals of Ninja: - -* very fast (i.e., instant) incremental builds, even for very large - projects. - -* very little policy about how code is built; "explicit is better than - implicit". - -* get dependencies correct, and in particular situations that are - difficult to get right with Makefiles (e.g. outputs need an implicit - dependency on the command line used to generate them; to build C - source code you need to use gcc's `-M` flags for header - dependencies). - -* when convenience and speed are in conflict, prefer speed. - -Some explicit _non-goals_: - -* convenient syntax for writing build files by hand. _You should - generate your ninja files using another program_. This is how we - can sidestep many policy decisions. - -* built-in rules. _Out of the box, Ninja has no rules for - e.g. compiling C code._ - -* build-time customization of the build. _Options belong in - the program that generates the ninja files_. - -* build-time decision-making ability such as conditionals or search - paths. _Making decisions is slow._ - -To restate, Ninja is faster than other build systems because it is -painfully simple. You must tell Ninja exactly what to do when you -create your project's `.ninja` files. - -Comparison to GNU make -~~~~~~~~~~~~~~~~~~~~~~ - -Ninja is closest in spirit and functionality to make. But -fundamentally, make has a lot of _features_: suffix rules, functions, -built-in rules that e.g. search for RCS files when building source. -Many projects find make alone adequate for their build problems. In -contrast, Ninja has almost no features; just those necessary to get -builds correct while punting most complexity to generation of the -ninja input files. Ninja by itself is unlikely to be useful for most -projects. - -Here are some of the features ninja adds to make. (These sorts of -features can often be implemented using more complicated Makefiles, -but they are not part of make itself.) - -* A Ninja rule may point at a path for extra implicit dependency - information. This makes it easy to get header dependencies correct - for C/C++ code. - -* A build edge may have multiple outputs. - -* Outputs implicitly depend on the command line that was used to generate - them, which means that changing e.g. compilation flags will cause - the outputs to rebuild. - -* Output directories are always implicitly created before running the - command that relies on them. - -* Rules can provide shorter descriptions of the command being run, so - you can print e.g. `CC foo.o` instead of a long command line while - building. - -* Command output is always buffered. This means commands running in - parallel don't interleave their output, and when a command fails we - can print its failure output next to the full command line that - produced the failure. - - -Getting started ---------------- - -The included `bootstrap.sh` should hopefully produce a working `ninja` -binary, by first blindly compiling all non-test files together then -re-building Ninja using itself. - -Usage is currently just - ----------------- -ninja target ----------------- - -where `target` is a known output described by `build.ninja` in the -current directory. - -There is no installation step; the only files of interest to a user -are the resulting binary and this manual. - - -Creating .ninja files ---------------------- -Here's a basic `.ninja` file that demonstrates most of the syntax. -It will be used as an example for the following sections. - ---------------------------------- -cflags = -Wall - -rule cc - command = gcc $cflags -c $in -o $out - -build foo.o: cc foo.c ---------------------------------- - -Variables -~~~~~~~~~ -Despite the non-goal of being convenient to write by hand, to keep -build files readable (debuggable), Ninja supports declaring shorter -reusable names for strings. A declaration like the following - ----------------- -cflags = -g ----------------- - -can be used on the right side of an equals sign, dereferencing it with -a dollar sign, like this: - ----------------- -rule cc - command = gcc $cflags -c $in -o $out ----------------- - -Variables can also be referenced using curly braces like `${in}`. - -Variables might better be called "bindings", in that a given variable -cannot be changed, only shadowed. There is more on how shadowing works -later in this document. - -Rules -~~~~~ - -Rules declare a short name for a command line. They begin with a line -consisting of the `rule` keyword and a name for the rule. Then -follows an indented set of `variable = value` lines. - -The basic example above declares a new rule named `cc`, along with the -command to run. (In the context of a rule, the `command` variable is -special and defines the command to run. A full list of special -variables is provided in <<ref_rule,the reference>>.) - -Within the context of a rule, two additional special variables are -available: `$in` expands to the list of input files (`foo.c`) and -`$out` to the output file (`foo.o`) for the command. - - -Build statements -~~~~~~~~~~~~~~~~ - -Build statements declare a relationship between input and output -files. They begin with the `build` keyword, and have the format -+build _outputs_: _rulename_ _inputs_+. Such a declaration says that -all of the output files are derived from the input files. When the -output files are missing or when the inputs change, Ninja will run the -rule to regenerate the outputs. - -The basic example above describes how to build `foo.o`, using the `cc` -rule. - -In the scope of a `build` block (including in the evaluation of its -associated `rule`), the variable `$in` is the list of inputs and the -variable `$out` is the list of outputs. - -A build statement may be followed by an indented set of `key = value` -pairs, much like a rule. These variables will shadow any variables -when evaluating the variables in the command. For example: - ----------------- -cflags = -Wall -Werror -rule cc - command = gcc $cflags -c $in -o $out - -# If left unspecified, builds get the outer $cflags. -build foo.o: cc foo.c - -# But you can can shadow variables like cflags for a particular build. -build special.o: cc special.c - cflags = -Wall - -# The variable was only shadowed for the scope of special.o; -# Subsequent build lines get the outer (original) cflags. -build bar.o: cc bar.c - ----------------- - -For more discussion of how scoping works, consult <<ref_scope,the -reference>>. - -If you need more complicated information passed from the build -statement to the rule (for example, if the rule needs "the file -extension of the first input"), pass that through as an extra -variable, like how `cflags` is passed above. - - -The `phony` rule -~~~~~~~~~~~~~~~~ - -The special rule name `phony` can be used to create aliases for other -targets. For example: - ----------------- -build all: phony some/file/in/a/faraway/subdir ----------------- - -This makes `ninja all` build the other files. Semantically, the -`phony` rule is equivalent to a plain rule where the `command` does -nothing, but phony rules are handled specially in that they aren't -printed when run, logged (see below), nor do they contribute to the -command count printed as part of the build process. - - -The Ninja log -~~~~~~~~~~~~~ - -For each built file, Ninja keeps a log of the command used to build -it. Using this log Ninja can know when an existing output was built -with a different command line than the build files specify (i.e., the -command line changed) and knows to rebuild the file. - -The log file is kept in the build root in a file called `.ninja_log`. -If you provide a variable named `builddir` in the outermost scope, -`.ninja_log` will be kept in that directory instead. - - -Generating Ninja files ----------------------- - -A work-in-progress patch to http://gyp.googlecode.com[gyp, the system -used to generate build files for the Chromium browser] to generate -ninja files for Linux is included in the source distribution. - -Conceptually, you could coax Automake into producing ninja files as -well, but I haven't tried it. It may very well be the case that most -projects use too much Makefile syntax in their `.am` files for this to -work. - -Extra tools ------------ - -The `-t` flag on the Ninja command line runs some tools that I have -found useful during Ninja's development. The current tools are: - -`query`:: dump the inputs and outputs of a given target. - -`browse`:: browse the dependency graph in a web browser. Clicking a -file focuses the view on that file, showing inputs and outputs. This -feature requires a Python installation. - -`graph`:: output a file in the syntax used by `graphviz`, a automatic -graph layout tool. Use it like: +ninja -t graph _target_ | dot -Tpng --ograph.png /dev/stdin+ . In the Ninja source tree, `ninja graph` -generates an image for Ninja itself. If no target is given generate a -graph for all root targets. - -`targets`:: output a list of targets either by rule or by depth. If used -like this +ninja -t targets rule _name_+ it prints the list of targets -using the given rule to be built. If no rule is given, it prints the source -files (the leaves of the graph). If used like this -+ninja -t targets depth _digit_+ it -prints the list of targets in a depth-first manner starting by the root -targets (the ones with no outputs). Indentation is used to mark dependencies. -If the depth is zero it prints all targets. If no arguments are provided -+ninja -t targets depth 1+ is assumed. In this mode targets may be listed -several times. If used like this +ninja -t targets all+ it -prints all the targets available without indentation and it is way faster -than the _depth_ mode. It returns non-zero if an error occurs. - -`rules`:: output the list of all rules with their description if they have -one. It can be used to know which rule name to pass to -+ninja -t targets rule _name_+. - -`clean`:: remove built files. If used like this +ninja -t clean+ it -removes all the built files. If used like this -+ninja -t clean _targets..._+ or like this -+ninja -t clean target _targets..._+ it removes the given targets and -recursively all files built for it. If used like this -+ninja -t clean rule _rules_+ it removes all files built using the given -rules. The depfiles are not removed. Files created but not referenced in -the graph are not removed. This tool takes in account the +-v+ and the -+-n+ options (note that +-n+ implies +-v+). It returns non-zero if an -error occurs. - -Ninja file reference --------------------- - -A file is a series of declarations. A declaration can be one of: - -1. A rule declaration, which begins with +rule _rulename_+, and - then has a series of indented lines defining variables. - -2. A build edge, which looks like +build _output1_ _output2_: - _rulename_ _input1_ _input2_+. + - Implicit dependencies may be tacked on the end with +| - _dependency1_ _dependency2_+. + - Order-only dependencies may be tacked on the end with +|| - _dependency1_ _dependency2_+. (See <<ref_dependencies,the reference on - dependency types>>.) - - -3. Variable declarations, which look like +_variable_ = _value_+. - -4. References to more files, which look like +subninja _path_+ or - +include _path_+. The difference between these is explained below - <<ref_scope,in the discussion about scoping>>. - -Comments begin with `#` and extend to the end of the line. - -Newlines are significant, but they can be escaped by putting a `\` -before them. - -Other whitespace is only significant if it's at the beginning of a -line. If a line is intended more than the previous one, it's -considered part of its parent's scope; if it is indented less than the -previous one, it closes the previous scope. - -Rule variables -~~~~~~~~~~~~~~ -[[ref_rule]] - -A `rule` block contains a list of `key = value` declarations that -affect the processing of the rule. Here is a full list of special -keys. - -`command` (_required_):: the command line to run. This string (after - $variables are expanded) is passed directly to `sh -c` without - interpretation by Ninja. - -`depfile`:: path to an optional `Makefile` that contains extra - _implicit dependencies_ (see <<ref_dependencies,the reference on - dependency types>>). This is explicitly to support `gcc` and its `-M` - family of flags, which output the list of headers a given `.c` file - depends on. -+ -Use it like in the following example: -+ ----- -rule cc - depfile = $out.d - command = gcc -MMD -MF $out.d [other gcc flags here] ----- -+ -When loading a `depfile`, Ninja implicitly adds edges such that it is -not an error if the listed dependency is missing. This allows you to -delete a depfile-discovered header file and rebuild, without the build -aborting due to a missing input. - - -`description`:: a short description of the command, used to pretty-print - the command as it's running. The `-v` flag controls whether to print - the full command or its description; if a command fails, the full command - line will always be printed before the command's output. - -Additionally, the special `$in` and `$out` variables expand to the -space-separated list of files provided to the `build` line referencing -this `rule`. - -Build dependencies -~~~~~~~~~~~~~~~~~~ -[[ref_dependencies]] - -There are three types of build dependencies which are subtly different. - -1. _Explicit dependencies_, as listed in a build line. These are - available as the `$in` variable in the rule. Changes in these files - cause the output to be rebuilt; if these file are missing and - Ninja doesn't know how to build them, the build is aborted. -+ -This is the standard form of dependency to be used for e.g. the -source file of a compile command. - -2. _Implicit dependencies_, either as picked up from - a `depfile` attribute on a rule or from the syntax +| _dep1_ - _dep2_+ on the end of a build line. The semantics are identical to - explicit dependencies, the only difference is that implicit dependencies - don't show up in the `$in` variable. -+ -This is for expressing dependencies that don't show up on the -command line of the command; for example, for a rule that runs a -script, the script itself should be an implicit dependency, as -changes to the script should cause the output to rebuild. -+ -Note that dependencies as loaded through depfiles have slightly different -semantics, as described in the <<ref_rule,rule reference>>. - -3. _Order-only dependencies_, expressed with the syntax +|| _dep1_ - _dep2_+ on the end of a build line. When these are missing, the - output is not rebuilt until they are built, but once they are - available further changes to the files do not affect the output. -+ -Order-only dependencies can be useful for bootstrapping dependencies -that are only discovered during build time: for example, to generate a -header file before starting a subsequent compilation step. (Once the -header is used in compilation, a generated dependency file will then -express the implicit dependency.) - -Evaluation and scoping -~~~~~~~~~~~~~~~~~~~~~~ -[[ref_scope]] - -Top-level variable declarations are scoped to the file they occur in. - -The `subninja` keyword, used to include another `.ninja` file, -introduces a new scope. The included `subninja` file may use the -variables from the parent file, and shadow their values for the file's -scope, but it won't affect values of the variables in the parent. - -To include another `.ninja` file in the current scope, much like a C -`#include` statement, use `include` instead of `subninja`. - -Variable declarations indented in a `build` block are scoped to the -`build` block. This scope is inherited by the `rule`. The full -lookup order for a variable referenced in a rule is: - -1. Rule-level variables (i.e. `$in`, `$command`). - -2. Build-level variables from the `build` that references this rule. - -3. File-level variables from the file that the `build` line was in. - -4. Variables from the file that included that file using the - `subninja` keyword. - -Future work ------------ - -Some pieces I'd like to add: - -_inotify_. I had originally intended to make Ninja be memory-resident -and to use `inotify` to keep the build state hot at all times. But -upon writing the code I found it was fast enough to run from scratch -each time. Perhaps a slower computer would still benefit from -inotify; the data structures are set up such that using inotify -shouldn't be hard. - -_build estimation and analysis_. As part of build correctness, Ninja -keeps a log of the time spent on each build statement. This log -format could be adjusted to instead store timing information across -multiple runs. From that, the total time necessary to build could be -estimated, allowing Ninja to print status like "3 minutes until -complete" when building. Additionally, a tool could output which -commands are the slowest or which directories take the most time -to build. - -_many others_. See the `todo` file included in the distribution. |