subordinate trees ? we have %G-space, we should add %K-space

Started by jim.cromie@gmail.com on Jul 15, 2021

open the first message on its own page

subordinate trees ? we have %G-space, we should add %K-space

From: jim.cromie@gmail.com
Date: 2021-07-15 18:03:04

hi git people,

managing .configs, over time, and across work-trees, is an ongoing burden.
I wonder if/how git could help.

I imagine a subordinate/secondary tree,
which basically has only 1 tracked file ( .config ) in it.
Auto-commit on .config ( perhaps as 1st step of make ) would mean no
changes are missed

A tertiary tree could hold all of ./builds/*
or really any set of files that are ignored by the primary tree's
.gitignore file
its purpose would be to track build products; all *.o etc
its kind-of a superset of what might be found in your .ccache,
(it also includes *.cmd.o etc, and all other intermediate outputs)
but it is attributable to a TooChainCompilationTransferFn(.config, HEAD)

Since HEAD already has a sha*, we just need one for .config,
the secondary tree keeps that for us.

the tertiary tree doesnt REALLY need to keep *all* the build products,
it just needs to keep track of them (just the sha*, since they are
reproducible),
but every commit (to either source, or to .config) should result in 1 or more
build products changing.  It might be useful for QA purposes to predict and
verify what products should change.

Autocommit here means that make *anything* has known inputs,
so if results are useful/reportable, they have known inputs (no -dirty)

Also, it might be handy to be able to diff any 2 objdumps/readelfs of foo.o,
which would be easier with a cryptographically known DB


Heres how I think it could get leveraged for linux itself

We have %G space, expressed in the kernel name:
  5.14.0-rc1-lm1-00003-g4a7356173ef9-dirty

Lets add %K space, to extend that:
  5.14.0-rc1-lm1-00003-g4a7356173ef9-dirty-k6173ef96173

now posit a kverstr("%D-%G-%K") producing the 2nd name above.
    %T - tag :  "5.14.0-rc1"
    %L - localversion : "-lm1"
    %C - count : "00003", %C{5} default
    %G - HEAD-sha : "-g4a7356173ef9-dirty"
    %K - sha(.config) : "-k6c173efc961c", obviously 'c' is taken.
    %D - desc : "%T-%L-%C"

%L is seductive, but ultimately a nuisance; mine above is a
super-imaginitive "-localmod-1" abbreviation.

In contrast, %K would give:

- updates every time the .config is tweaked.

- each arch make defconfig yields single %Kval
  or a few (1 per salt-val)

- normalization done by make oldconfig narrows %K-space
  some %Kvals are "impossible" (aka: hand altered or fuzzed .configs)

- many .configs will get "well-known" %Kvals
  bots will report probs with %Gs against specific %Ks

- all well-known configs: defconfig, allmodconfig, etc...
  get unique %Ks, maybe become well-known
  esp as %K{5} which is short enough to recognize
  %Ks could be annotated
  %Ks change slowly as **/KBuild.* composition changes
  traceable to commits in %G

With (imagined) Git extensions:

1. subordinate trees, in same src/work-dir:

a configs subtree would have only .config file checked in, and
probably should auto-commit on changes.

ls ./.gitinclude.*

These actively define inclusions of files in the named subtree, rather
than ignores.  They dont see files that are not explicitly ignored by
primary tree.

so ./.gitinclude.1.configs:
# with
.config

could always, actively, track and commit changes to .config,
`history|grep config | tail -n5`, automatically added to commit-log,
would help (a little) explain why.  A string serialization of the
CONFIG_* changes, would completely characterize the .config diff, and

and ./.gitinclude.3.builds
# with
./builds/

captures all build product files written under ./builds/* into a
single subordinate tree.  This tree can only contain files ignored by
.gitignore (ie the primary/source tree)

obviously this will include all of ./**/*.o, which is essentially
a %G+%K versioned (and clean) ccache result set.

./builds/{*}   # distinct build-product (%G+%K dependent) sub-trees

./builds/foo # foo
./builds/bar

not really sure how the 2ndary tree (.config tracker) should play with
worktrees,
or even with multiple ./builds/ subdirs, each with a .config
those subdirs are our %K-space-explored,
ie those configs which we have at least tried

make randconfig is a rather low-quality way to expand the tested %K-space
but with %K handled a bit better, we could at least track how much of
the space gets tested this way.

ok, now Im meandering.
Keyboard shortcuts
hback out one level
jnext message in thread
kprevious message in thread
ldrill in
Escclose help / fold thread tree
?toggle this help