Re: Why the Makefile is so eager to re-build & re-link
From: Johannes Sixt <hidden>
Date: 2021-06-24 21:30:25
Am 24.06.21 um 17:16 schrieb Jeff King:
On Thu, Jun 24, 2021 at 03:16:48PM +0200, Ævar Arnfjörð Bjarmason wrote:quoted
* {command,config}-list.h (and in-flight, my hook-list.h): Every time you touch a Documentation/git-*.txt we need to re-generate these, and since their mtime changes we re-compile and re-link all the way up to libgit and our other tools. I think the best solution here is to make the generate-*.sh shellscripts faster (just one takes ~300ms of nested shellscripting, just to grep out the first few lines of every git-*.txt, in e.g. Perl or a smarter awk script this would be <5ms).Yeah, I think Eric mentioned he had looked into doing this in perl, but we weren't entirely happy with the dependency. Here's another really odd thing I noticed: $ time sh ./generate-cmdlist.sh command-list.txt >one real 0m1.323s user 0m1.531s sys 0m0.834s $ time sh -x ./generate-cmdlist.sh command-list.txt >two [a bunch of trace output] real 0m0.513s user 0m0.754s sys 0m0.168s $ cmp one two [no output] Er, what? Running with "-x" makes it almost 3 times faster to generate the same output? I'd have said this is an anomaly, but it's repeatable (and swapping the order produces the same output, so it's not some weird priming thing). And then to top it all off, redirecting the trace is slow again: $ time sh -x ./generate-cmdlist.sh command-list.txt >two 2>/dev/null real 0m1.363s user 0m1.538s sys 0m0.902s A little mini-mystery that I think I may leave unsolved for now.
Strange, really. Reminds me of the case that the `read` built-in must read input byte by byte if stdin is not connected to a (searchable) file. I have two patches that speed up generate-cmdlist.sh a bit: generate-cmdlist.sh: replace for loop by printf's auto-repeat feature (https://github.com/j6t/git/commit/b6d05f653bede727bc001f299b57969f62d3bc03) generate-cmdlist.sh: spawn fewer processes (https://github.com/j6t/git/commit/fd8721ee8fae06d7b584fa5166f32bf5521ca304) that I can submit (give me some time, though) or interested parties could pick up. -- Hannes