summaryrefslogtreecommitdiffstats
path: root/src/build.cc
Commit message (Collapse)AuthorAgeFilesLines
* Fix confusing smart console output from concurrent buildsNico Weber2018-04-051-0/+13
| | | | | | | | | | | Developers tend to blame the last printed line when a build takes too long. Unfortunately, when building concurrently, the last printed line may have actually finished a long time ago. Under the current system, ninja does not update the status line to reflect what jobs are still running. This change makes ninja always print the oldest still running job instead. In other words, the likely build bottlenecks. Patch from David Zarzycki, originally uploaded at #1320.
* Merge pull request #1294 from bradking/plan-track-schedulingNico Weber2018-04-051-21/+23
|\ | | | | Track in Plan whether wanted edges have been scheduled
| * Track in Plan whether wanted edges have been scheduledBrad King2017-09-141-21/+23
| | | | | | | | | | | | | | | | Refactor the `want_` map to track for wanted edges whether they have been scheduled or not. This gives `ScheduleWork` a direct place to keep this information, making the logic more robust and easier to follow. It also future-proofs `ScheduleWork` to avoid repeat scheduling if it is called after an edge has been removed from `ready_` by `FindWork`.
* | make ninja build with -std=c++17Nico Weber2018-04-051-1/+6
|/ | | | | | Ninja is supposed to be able to build as C++98 so it can run on old systems, but it should also be possible to optionally build it with newer dialects.
* Drop unnecessary cycle detection in Plan::AddTargetBrad King2017-06-191-44/+5
| | | | | We now detect and reject cycles in DependencyScan::RecomputeDirty before Plan::AddTarget is called so we can assume DAG input to the Plan.
* Refactor RecomputeDirty to take a node instead of an edgeBrad King2017-06-191-2/+3
| | | | | | All call sites have a node on which they call `in_edge()` to call RecomputeDirty. Simplify call sites by taking the node directly and calling `in_edge()` internally.
* Merge pull request #1156 from cdbennett/windows-binary-mode-outputNico Weber2017-06-141-0/+15
|\ | | | | Write subprocess output to stdout in binary mode
| * Write subprocess output to stdout in binary modeColin D Bennett2016-05-121-0/+15
| | | | | | | | | | | | | | | | Set stdout to binary mode while writing subprocess output, so that the CR in a CR LF sequence is not replaced with CR LF itself, which would result in CR CR LF. Based on patch posted by nico in issue #773 comment.
* | Always rebuild on errorsColin Cross2017-05-221-5/+11
| | | | | | | | | | | | | | | | | | | | https://groups.google.com/forum/#!msg/ninja-build/YQuGNrECI-4/ti-lAs9SPv8J discusses a case where an rule updates its output file and then fails. The next run of ninja considers the ouptut file clean and doesn't rebuild it. Always stat output files after they are built, and write the mtime into .ninja_log. Consider output files dirty if the recorded mtime is older than the most recent input file.
* | Add a test that fails if a rule updates it output file but failsColin Cross2017-05-221-0/+7
| | | | | | | | | | | | | | | | https://groups.google.com/forum/#!msg/ninja-build/YQuGNrECI-4/ti-lAs9SPv8J discusses a case where an rule updates its output file and then fails. The next run of ninja considers the ouptut file clean and doesn't rebuild it. Add a test for this case, which currently fails.
* | Merge pull request #1181 from DanielWeber/issue-1161Nico Weber2017-01-241-1/+1
|\ \ | | | | | | Allow more path components
| * | Allow more path componentsDaniel Weber2016-08-221-1/+1
| | | | | | | | | | | | | | | | | | | | | - 60 instead of 30 path components - 64 instead of 32 backslashes in a path (windows only) Issue: 1161
* | | Tidy up rate snprintf'ing helperScott Graham2016-10-071-2/+2
|/ /
* | Fixed not initialized elapsed time.gkistanova2016-07-141-5/+8
|/
* Fix NINJA_STATUS %r on dumb terminalsColin Cross2016-04-291-7/+12
| | | | | | | | | | | | PR #999 made dumb terminals only output when edges finish. PrintStatus is called after finished_edges_ is incremented, which means the calculation for running edges will always return 1 less than the real number of running processes. This happens on smart terminals too, but ninja will immediately print the status for the next edge with starting_edges_ incremented, so the incorrect value is never visible. Pass a boolean specifying whether the status is being printed on an edge finishing, and if so count the edge that just finished as being running.
* Fix NINJA_STATUS %e on dumb terminalsColin Cross2016-04-281-4/+6
| | | | | | | | | | | | | PR #999 made dumb terminals only output when edges finish. BuildStatus::overall_rate_ stopwatch is only initialized to the current time when PrintStatus is called with finished_edges_ == 0, but on a dumb terminal it will be called for the first time when finished_edge_ = 1, which results in very long elapsed times: NINJA_STATUS="[%r processes, %f/%t @ %o/s : %es ] " [0 processes, 2/2 @ 0.0/s : 1461869902.367s ] Reset the stopwatches in BuildEdgeFinished before finshed_edges_ is incremented instead.
* Update %p to use finished edges after #1143.Nico Weber2016-04-281-1/+1
|
* Change default NINJA_STATUS to [%f/%t]Colin Cross2016-04-281-1/+1
| | | | | | | | | | | | | | | | | | PR #999 changed the status line to be printed when edges finish on dumb teerminals, but the default status message includes the number of started edges, resulting in sequential status lines with identical edge counts. Change the default status to show the number of finished edges, which will keep the count incrementing on every line. This will slightly change the output on smart terminals. Previously a build that was just starting would show a count equal to the number of concurrent jobs, and a build waiting for the final jobs to finish would show a count equal to the total number of edges. Now a starting build will show 0, and build waiting for the final jobs will show a count less than the total number of edges by the number of remaining jobs. Fixes: #1142
* Improve Plan::EdgeFinished signatureBrad King2016-04-191-6/+6
| | | | | | | Use an enumeration instead of a boolean to clarify the purpose of arguments at call sites. Suggested-by: Nico Weber <nicolasweber@gmx.de>
* Merge pull request #1126 from bradking/pool-release-on-failNico Weber2016-04-191-9/+16
|\ | | | | Release the pool slot held by an edge whether it succeeds or fails
| * Release the pool slot held by an edge whether it succeeds or failsFredrik Medley2016-04-061-9/+16
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When an edge finishes building, it should be release from its pool. Make sure that this also is the case when an edge fails to build. The bug can be shown with a pool has size N, then `ninja -k N+1` will still stop after N failing commands for that pool, even if there are many more jobs to be done for that pool: pool mypool depth = 1 rule bad_rule command = false pool = mypool build a : bad_rule build b : bad_rule Current behaviour: $ ninja -k 0 [1/2] false FAILED: false ninja: build stopped: cannot make progress due to previous errors. Expected behaviour: $ ninja -k 0 [1/2] false FAILED: false [2/2] false FAILED: false ninja: build stopped: cannot make progress due to previous errors. Signed-off-by: Fredrik Medley <fredrik.medley@gmail.com>
* | Merge pull request #1059 from bradking/avoid-double-scheduling-edgeNico Weber2016-04-191-7/+10
|\ \ | |/ |/| Avoid double-scheduling build edges in another case
| * Avoid double-scheduling build edges in another caseBrad King2016-02-031-7/+10
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The change in commit v1.2.0~3^2~3^2~3 (Fix duplicate edge Pool crash in the minimally invasive way, 2013-03-18) avoids double-scheduling in a case involving duplicate out edges. However, double-scheduling may also occur on a consistent graph when an edge and one of its dependencies share an order-only input: $ cat build.ninja ... build c: touch build b: touch || c build a: touch | b || c $ ninja a $ rm a c $ ninja a In this case 'c' will build first. When NodeFinished('c') loops over the out edges it will find AllInputsReady is true for 'b' and call EdgeFinished('b') since it is not wanted (up to date). This will call NodeFinished('b') which will loop over its out edges, find AllInputsReady is true for 'a', and call ScheduleEdge('a'). When we eventually return to the loop in NodeFinished('c') it will move on to its second output and find that AllInputsReady is true for 'a' and call ScheduleEdge('a') again. Teach ScheduleEdge to tolerate duplicate calls for an edge that has already been scheduled. Avoid calling EdgeScheduled more than once for the same edge.
* | Make deps=msvc experimentally available on non-Windows.Nico Weber2016-04-061-3/+1
|/ | | | | | | | | This makes it possible to run most of the clparser tests on non-Windows, and is potentially useful for cross-compiling on non-Windows hosts. Also, the manual didn't document this as Windows-only previously. If you use this on non-Windows, please let me know, else I might undo this change again in the future.
* Merge pull request #1033 from colincross/failedNico Weber2016-02-031-2/+9
|\ | | | | Print output file on failure
| * Print output file on failureColin Cross2015-10-051-2/+9
| | | | | | | | | | | | | | | | | | | | | | Modify the FAILED: output to provide the output files that failed to build, followed by the failed command on the next line. This makes the failure much easier to read, as you can immediately see much shorter name of the file that failed instead of trying to parse a very long command line. It also makes manually re-running the failed command much easier because you can copy the whole line without ending up with the FAILED: prefix.
* | Expose more details in FileReader::ReadFile signatureBrad King2016-02-031-2/+10
| | | | | | | | | | | | Return a status so callers can distinguish a missing file from an empty file. This allows our VirtualFileSystem test infrastructure to report as missing any file for which it has no entry.
* | Add -d keepdepfile to preserve depfilesShinichiro Hamaji2016-01-281-3/+5
| | | | | | | | | | This is useful when you are developing a tool which generates GCC-style depfiles.
* | Merge pull request #999 from colincross/dumbstatusNico Weber2015-11-121-2/+3
|\ \ | | | | | | Print status when edge finishes on dumb terminals
| * | Print status when edge finishes on dumb terminalsColin Cross2015-07-311-2/+3
| |/ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | On smart terminals ninja prints the status line both before and after running a command, reusing the same line if possible. On a dumb terminal that doesn't support reusing the line, it only prints the status before starting the command, but prints the output of the command when the command finishes, by which point other commands may have started and printed their status line. This makes it impossible to determine what command produced a line of output. Modify BuildEdgeStarted to only print the status line if the command is going to lock the console, or if ninja is running on a smart terminal. Modify BuildEdgeFinished to always print the status line unless the command locked the console, in which case the status was already printed and no other command can have printed any lines. The end result will be dumb terminal output that much more closely matches smart terminal output. One disadvantage is that dumb terminals won't show anything when starting a command, making it harder to tell what commands are currently running, but I expect most interactive uses of ninja will use a smart terminal.
* | Make links point to org pageNico Weber2015-11-111-1/+1
|/
* don't alias input/output in ExtractDeps (i.e. actually works now)Scott Graham2015-06-181-1/+3
|
* propagate include normalization failure to caller insteadScott Graham2015-06-181-1/+2
|
* simplify & inlineScott Graham2015-04-271-12/+7
|
* avoid calling ResumeDelayedJobs insteadScott Graham2015-04-241-6/+7
|
* Fix pool use count going unbalancedScott Graham2015-04-231-1/+3
|
* Don't get stuck on cyclic graphs where one edge has multiple outputs.Nico Weber2015-04-011-4/+21
| | | | | | | | | | | | | | | | | | | Fixes #934. Plan::AddSubTarget() tracks in want_ if each edge has been visited and visits every edge only once. But Plan::CheckDependencyCycle() worked on nodes however, so if a cycle was entered through an edge with multiple outputs, ninja would fail to detect that cycle. Move cycle detection to look for duplicate edges instead of nodes to fix this. The extra jump makes CheckDependencyCycle() a bit slower: for a synthetic build file with 10000 serial build steps, `ninja -n` now takes 0.32s instead of 0.26s before on my laptop. In practice, projects have a dependency change length on the order of 50, so there shouldn't be a noticeable slowdown in practice. (If this does end up being a problem: CheckDependencyCycle() currently does O(n) work and is called O(m) times from AddSubTarget(), so I think cycle checking is O(n^2) in the build depth. So instead of worrying about constant overhead, we could use a set<> instead of a stack<>. But it doesn't seem to be a problem in practice.)
* Cleanup: Don't search stack for cycle elements twice.Nico Weber2015-04-011-4/+2
| | | | | | | | | | | | | The common case is that there is no cycle. In that case, CheckDependencyCycle() searched the stack for a dupe from the back, wouldn't find one, and return false. If there was a cycle, it would then search again from the front (probably because the push_back() that used to be here would invalidate the ri iterator). Since the push_back() is gone, just search from the front once. No intended behavior change.
* Cleanup: Make stack a const ref now that it's no longer modified.Nico Weber2015-04-011-7/+8
|
* Cleanup: Don't modify during cycle checking.Nico Weber2015-04-011-6/+2
|
* Let Stat() have an err outparam instead of writing to stderr.Nico Weber2015-03-311-11/+16
| | | | | | | | | Also check for Stat() failure in a few more places. This way, ninja doesn't print two "ninja: error: " lines if stat() fails during a build. It also makes it easier to keep the stat tests quiet. Every caller of Stat() needs to explicitly log the error string if that's desired.
* Make failing stat() calls abort the build.Nico Weber2015-03-191-4/+17
| | | | | | | | | | | | | | | | | | | | | | | | Fixes #830, fixes #904. In practice, this either happens with 64-bit inodes and a 32-bit userspace when building without -D_FILE_OFFSET_BITS=64 in CFLAGS, or when a filename is longer than the system file length limit. Since DiskInterface::Stat() returns -1 on error, and Node used -1 on "stat state unknown", not aborting the build lead to ninja stat()ing the same file over and over again, until it finally ran out of stack. That's now fixed. * Change RecomputeOutputsDirty() to return success instead of dirty state (like RecomputeDirty()) and return the dirty state in a bool outparam * Node::Stat()s old return value wasn't used anywhere, change the function to return success instead and add an |err| outparam * Node::StatIfNecessary()'s old return value was used only in one place. Change that place to explicitly check status_known() and make StatIfNecessary() return success and add an |err| outparam * Plan::CleanNode() can now fail, make it return bool and add an |err| outparam
* Let DependencyScan::RecomputeDirty() work correclty with cyclic graphs.Nico Weber2014-12-081-1/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | RecomputDirty(edge) currently works roughly like this: RecomputeDirty(edge): LoadDeps(edge) for in in edge.inputs: if in.StatIfNecessary(): RecomputeDirty(in.in_edge) # recurse into inputs for out in edge.outputs: out.StatIfNecessary() # mark outputs as visited It uses the stat state of each node to mark nodes as visited and doesn't traverse across nodes that have been visited already. For cyclic graphs with an edge with multiple outputs on the cycle, nothing prevents an edge to be visited more than once if the cycle is entered through an output that isn't on the cycle. In other words, RecomputeDirty() for the same edge can be on the call stack more than once. This is bad for at least two reasons: 1. Deps are added multiple times, making the graph confusing to reason about. 2. LoadDeps() will insert into the inputs_ of an edge that's iterated over in a callframe higher up. This can invalidate the iterator, which causes a crash when the callframe with the loop over the now-invalidated iterator resumes. To fix this, let RecomputeDirty() mark all outputs of an edge as visited as the first thing it does. This way, even if the edge is on a cycle with several outputs, each output is already marked and no edge will have its deps loaded more than once. Fixes the crashes in #875. (In practice, it turns the crashes into "stuck [this is a bug]" messages for now, due to the example without duplicate rules in #867)
* Rename a few iterators. No behavior change.Nico Weber2014-12-071-58/+58
| | | | | | | | | It confused me that the iterator iterating over `outputs_` was called `i` -- this always made me think of "input", not "iterator". Call iterators over edge outputs "o", iterators over edge inputs "i", iterators over node input edges "oe", and general iterators over edges "e".
* make all GetNode explicit, add DepsLog canonicalize testScott Graham2014-11-101-1/+5
|
* fix order of args to CanonicalizePathScott Graham2014-11-091-2/+2
|
* remove CanonicalizePath overloads, test for toplevel behaviourScott Graham2014-11-091-2/+4
|
* Throttle the number of pending commands by the parallelism configurationtzik2014-09-221-1/+3
|
* Make "depfile=$out.d" work if $out contains escaped characters, rspfile too.Nico Weber2014-05-211-5/+5
| | | | | | | | | | | | | | Fixes #730. This has always been broken, but due to #690 more paths are now escaped (e.g. paths containing + characters, like file.c++). Also see discussion in #689. The approach is to give EdgeEnv an enum deciding on whether or not to escape file names, and provide functions that evaluate depfile and rspfile with that set that to kNoEscape. (depfile=$out.d doesn't make sense on edges with multiple outputs.) This should be relatively safe, as $in and $out can't be used on edges, only on rules (#687).
* Introduce the "console" poolPeter Collingbourne2014-02-031-2/+12
| | | | | | | This is a pre-defined pool with a depth of 1. It has the special property that any task in the pool has direct access to the console. This can be useful for interactive tasks or long-running tasks which produce status updates on the console (such as test suites).
generated by cgit v0.12 at 2024-07-22 16:42:37 (GMT)