scripting speedups [was: [Summit topic] Crazy (and not so crazy) ideas]

Johannes Schindelin [off-list ref] wrote:

* Test suite is slow. Shell scripts and process forking.

   * What if we had a special shell that interpreted the commands in a
     single process?

   * Even Git commands like rev-parse and hash-object, as long as that’s
     not the command you’re trying to test

This is something I've wanted in a very long time as a scripter.
fast-import has been great over the years, as is
"cat-file --batch(-check)", but there's gaps should be filled
(preferably without fragile linkage of shared libraries into a
script process)

   * Dscho wants to slip in a C-based solution

   * Jonathan tan commented: going back to your custom shell for tests
     idea, one thing we could do is have a custom command that generates
     the repo commits that we want (and that saves process spawns and
     might make the tests simpler too)

Perhaps a not-seriously-proposed patch from 2006 could be
modernized for our now-libified internals:

https://yhbt.net/lore/git/Pine.LNX.4.64.0602232229340.3771@g5.osdl.org/

      * We could replace several “setup repo” steps with “git fast-import”
        instead.

   * Dscho measured: 0.5 sec - 30 sec in setup steps. Can use fast-import,
     or can make a new format that helps us set up the test scenario

0.5s - 30s across the whole suite or individual tests?

Having a way to disable fsync globally should further improve
things, especially for people on slower storage.  libeatmydata
is available, but perhaps not widely available/known.

   * Elijah: test-lib-functions helpers could be built ins