Re: scripting speedups [was: [Summit topic] Crazy (and not so crazy) ideas]

On Tue, Oct 26 2021, Eric Wong wrote:

Johannes Schindelin [off-list ref] wrote:

quoted

* Test suite is slow. Shell scripts and process forking.

   * What if we had a special shell that interpreted the commands in a
     single process?

   * Even Git commands like rev-parse and hash-object, as long as that’s
     not the command you’re trying to test

This is something I've wanted in a very long time as a scripter.
fast-import has been great over the years, as is
"cat-file --batch(-check)", but there's gaps should be filled
(preferably without fragile linkage of shared libraries into a
script process)

quoted

   * Dscho wants to slip in a C-based solution

   * Jonathan tan commented: going back to your custom shell for tests
     idea, one thing we could do is have a custom command that generates
     the repo commits that we want (and that saves process spawns and
     might make the tests simpler too)

Perhaps a not-seriously-proposed patch from 2006 could be
modernized for our now-libified internals:

I think something very short of a "C-based solution" could give us most
of the wins here. Johannes was probably thinking of the scripting being
slow on Windows aspect of it.

But the main benefit of hypothetical C-based testing is that you can
connect it to the dependency tree we have in the Makefile, and only
re-run tests for code you needed to re-compile.

So e.g. we don't need to run tests that invoke "git tag" if the
dependency graph of builtin/tag.c didn't change.

With COMPUTE_HEADER_DEPENDENCIES we've got access to that dependency
information for our C code.

With trace2 we could record an initial test run, and know which built-in
commands are executed by which tests (even down to the sub-test level).

Connecting these two means that we can find all tests that say run "git
fsck", and if builtin/fsck.c is the only thing that changed in an
interactive rebase, that's the only tests we need to run.

Of course changes to things like cache.h or t/test-lib.sh would spoil
that cache entirely, but pretty much the same is true for re-compiling
things now, so would changing say builtin/init-db.c, as almost every
test does a "git init" somewhere.

But I think that approch is viable, and should take us from a huge
hypothetical project like "rewrite all the tests in C" to something
that's a viable weekend hacking project for someone who's interested.