| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
A common problem in the Android build is inserting rules that perform
some sort of error checking that doesn't produce any artifacts needed
by the build, for example static analysis tools. There are a few
patterns currently used, both of which have downsides.
The first is to have a rule that depends on all of the static analysis
results. This ensures they run, but requires running static analysis
over everything, and not just the active parts of the build graph.
The second is to insert the static analysis rule into the build graph
between the artifact producing rule and anything that depends on it,
often copying the artifact as the output of the static analysis rule.
This increases the critical path of the build, often reducing
parallelism. In the case of copying the artifact, it also wastes
disk space.
This patch adds "validation nodes" to edges in Ninja. A build
statement can specify validation nodes using "|@" in the edge
inputs. The validation nodes are not used as an input to the edge
(the edge can run before the validation node is ready), but are
added to the initial nodes of the build graph whenever the edge
is part of the build graph. The edge that outputs the validation
node can depend on the output of the edge that is being validated
if desired.
Test: ninja_test
Change-Id: Ife27086c50c1b257a26509373199664680b2b247
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
-d stats enables instrumented profiling of key functions in ninja.
However, some of those functions are invoked 6+ million times in a NOP
build of Chromium and the cost of measuring those functions dwarfs the
cost of the functions. Here is typical -d stats output for a Chromium
build:
metric count avg (us) total (ms)
.ninja parse 6580 4197.5 27619.5
canonicalize str 6240450 0.0 47.3
canonicalize path 6251390 0.0 33.5
lookup node 6339402 0.0 37.2
.ninja_log load 1 176226.0 176.2
.ninja_deps load 1 465407.0 465.4
node stat 168997 8.8 1482.9
depfile load 327 352.7 115.3
99% of the measurements are in three functions. The total measurement
cost (per ETW sampled profiling) is 700-1200 ms, which is many times
greater than the costs of the functions.
With this change typical output looks like this:
metric count avg (us) total (ms)
.ninja parse 6580 3526.3 23203.2
.ninja_log load 1 227305.0 227.3
.ninja_deps load 1 485693.0 485.7
node stat 168997 9.6 1615.0
depfile load 327 383.1 125.3
This resolves issue #1998.
|
| |
|
|
|
|
|
|
| |
Edges are nominally ordered by order in the build manifest, but in
fact are ordered by memory address. In most cases the memory address
will be monontonically increasing. Since serialized build output will
need unique IDs, add a monotonically increasing ID to edges, and use
that for sorting instead of memory address.
|
| |
|
|
| |
header files
|
| |
|
|
|
|
| |
Track for each Edge whether depfile information has been loaded using an
explicit flag. This will allow RecomputeDirty to be repeated for an
edge without loading deps again.
|
| |
|
|
| |
Store a mark in each `Edge` to be updated as it is encountered by a walk.
|
| |
|
|
| |
The VS compiler complained about possible loss of data (and it was right!)
|
| | |
|
| |
|
|
|
|
|
|
| |
This will make it easier to optionally make this an error (because
ManifestParser has a way of printing errors), and it'll also make
it easier to make the tests quiet again.
No behavior change.
|
| |
|
|
|
|
|
|
|
|
|
| |
Patch #933 fixed a crash with duplicate edges by not adding edges to the
graph if all the edge's outputs are already built by other edges.
However, it added the edge to the out_edges of the edge's input nodes
before deleting it, letting inputs refer to dead edges.
To fix, move the check for deleting an edge above the code that adds
inputs. Expand VerifyGraph() to check that nodes don't refer to edges
that aren't present in the state.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Fixes #867, both the crashes and "[stuck]" issues.
The problem was that a duplicate edge would modify the in_edge of the
outputs of the new build rule, but the edge corresponding to the old
build rule would still think that the in_edge points to itself.
`old_edge->outputs_[0]->in_edge()` would not return `old_edge`, which
confused the scan logic.
As fix, let `State::AddOut()` reject changing in_edge if it's already
set. This changes behavior in a minor way: Previously, if there were
multiple edges for a single output, the last edge would be kept. Now,
the first edge is kept. This only had mostly-well-defined semantics if
all duplicate edges are the same (which is the only case I've seen in
practice), and for that case the behavior doesn't change.
For testing, add a VerifyGraph() function and call that every time any
test graph is parsed. That's a bit more code than just copying the test
cases from the bug into build_test.cc, but it also yields better test
coverage overall.
|
| |
|
|
|
|
|
|
| |
Ninja didn't support scoping rules through subninja and assumed
a unique rule name in the whole namespace. With this change, this
behavior is changed to allow scoping rules. Two rules can have the
same name if they belong to two different scopes. However, two
rules can NOT have the same name in the same scope.
|
| |
|
|
|
|
| |
The assert fires on cyclic manifests (found by afl-fuzz). Since there
was explicit error handing for this case already, just remove the
assert.
|
| | |
|
| | |
|
| | |
|
| |
|
|
|
|
|
| |
This is a pre-defined pool with a depth of 1. It has the special property
that any task in the pool has direct access to the console. This can be
useful for interactive tasks or long-running tasks which produce status
updates on the console (such as test suites).
|
| | |
|
| |
|
|
|
|
|
|
|
| |
Ideally we'd detect this at build time and only warn if your build
was affected, but that's hard to do. (Really we should just abort
when this scenario is detected rather than continuing, but now
users are relying on it.)
Hopefully improves issue #543.
|
| |
|
|
|
| |
DepsLog is a compact serialization of dependency information.
It can be used to replace depfiles for faster loading.
|
| | |
|
| | |
|
| |
|
|
|
| |
Now, a 'build' block can override any special binding like 'command'
or 'description' if it needs to.
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| |
|
|
|
|
|
| |
The build log is needed in computing whether an edge is
dirty, so I think it belongs here. (It's a bit weird
that Builder needs to reach into it to record completed
commands, maybe it will become cleaner with more thought.)
|
| | |
|
| | |
|
| | |
|
| |
|
|
| |
Fix a compilation error.
|
| | |
|
| | |
|
| | |
|
| |
|
|
|
|
| |
I think I had originally imagined StatCache would contain more state,
but at this point it's clear it was just managing a single map, which
could just as well be in the already-small State object.
|
| |
|
|
|
|
| |
The two were always one-to-one anyway. I started adding accessors
to FileStat and then realized most users wanted them on Node and
that forwarding them through was silly.
|
| |
|
|
|
|
|
|
|
|
|
| |
1. For targets, when invoking ninja to build a target.
2. For targets, when doing a "query" command.
3. For command names.
4. For the subcommands of the "targets" command.
Also change CmdTargets() to call LookupNode() instead of GetNode() --
since the result was checked for NULL, that's probably what was intended
here originally.
|
| |
|
|
|
| |
dirty_ is intended to remain static during the build (unless a restat
occurs), while outputs_ready_ reflects the dynamic state of the build.
|
|
|
This was a TODO in src/ninja_jumble.cc. Now this task is completed.
Signed-off-by: Thiago Farina <tfarina@chromium.org>
|
