Re: [PATCH 0/2] Sparse index: fetch, pull, ls-files

On Tue, Nov 16 2021, Derrick Stolee via GitGitGadget wrote:

The 'ls-files' builtin is useful when debugging the index, and some scripts
use it, too. We are not changing the default behavior which expands a sparse
index in order to show all of the cached blobs. Instead, we add a '--sparse'
option that allows us to see the sparse directory entries upon request.
Combined with --debug, we can see a lot of index details, such as:

$ git ls-files --debug --sparse
LICENSE
  ctime: 1634910503:287405820
  mtime: 1634910503:287405820
  dev: 16777220 ino: 119325319
  uid: 501  gid: 20
  size: 1098    flags: 200000
README.md
  ctime: 1634910503:288090279
  mtime: 1634910503:288090279
  dev: 16777220 ino: 119325320
  uid: 501  gid: 20
  size: 934 flags: 200000
bin/index.js
  ctime: 1634910767:828434033
  mtime: 1634910767:828434033
  dev: 16777220 ino: 119325520
  uid: 501  gid: 20
  size: 7292    flags: 200000
examples/
  ctime: 0:0
  mtime: 0:0
  dev: 0    ino: 0
  uid: 0    gid: 0
  size: 0   flags: 40004000
package.json
  ctime: 1634910503:288676330
  mtime: 1634910503:288676330
  dev: 16777220 ino: 119325321
  uid: 501  gid: 20
  size: 680 flags: 200000


(In this example, the 'examples/' directory is sparse.)

Thanks!

That's useful, and seems to be closing the same feature gap that the RFC
series I sent back in March[1], but at the time you went with adding
this ability to t/helper/test-read-cache.c.

With your series diffing the two with the data used in your new test
shows that we had this with the helper beore:
    
     diff -u <(t/helper/test-tool -C sparse-index read-cache --table) <(t/helper/test-tool -C sparse-index read-cache --table --expand)
    --- /dev/fd/63  2021-11-23 02:57:00.980651400 +0100
    +++ /dev/fd/62  2021-11-23 02:57:00.980651400 +0100
    @@ -13,9 +13,13 @@
     100644 blob cebd1739abee3e524d72ca9f51465a94d5a71daf   e
     100644 blob e69de29bb2d1d6434b8b29ae775ad8c2e48c5391   folder1-
     100644 blob e69de29bb2d1d6434b8b29ae775ad8c2e48c5391   folder1.x
    -040000 tree e0f2d30b633eb781d675fedd78808135103fe1a0   folder1/
    +100644 blob 8b137891791fe96927ad78e64b0aad7bded08bdc   folder1/0/0/0
    +100644 blob e69de29bb2d1d6434b8b29ae775ad8c2e48c5391   folder1/0/1
    +100644 blob 78981922613b2afb6025042ff6bd878ac1994e85   folder1/a
     100644 blob e69de29bb2d1d6434b8b29ae775ad8c2e48c5391   folder10
    -040000 tree 123706f6fc38949628eaf0483edbf97ba21123ae   folder2/
    +100644 blob e69de29bb2d1d6434b8b29ae775ad8c2e48c5391   folder2/0/0/0
    +100644 blob 8b137891791fe96927ad78e64b0aad7bded08bdc   folder2/0/1
    +100644 blob 78981922613b2afb6025042ff6bd878ac1994e85   folder2/a
     100644 blob 18912c9a915632d7b3344ec25f349dc8b4b9bf27   g
    -040000 tree aaff74984cccd156a469afa7d9ab10e4777beb24   x/
    +100644 blob 78981922613b2afb6025042ff6bd878ac1994e85   x/a
     100644 blob 2d8c856aebeb20da61bd5112d6fa46ff3f56a9e8   z

Which we can now get out of ls-files:
    
    $ diff -u <(git -C sparse-index ls-files --stage --sparse) <(git -C sparse-index ls-files --stage)
    --- /dev/fd/63  2021-11-23 02:55:13.329613255 +0100
    +++ /dev/fd/62  2021-11-23 02:55:13.329613255 +0100
    @@ -13,9 +13,13 @@
     100644 cebd1739abee3e524d72ca9f51465a94d5a71daf 0      e
     100644 e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 0      folder1-
     100644 e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 0      folder1.x
    -040000 e0f2d30b633eb781d675fedd78808135103fe1a0 0      folder1/
    +100644 8b137891791fe96927ad78e64b0aad7bded08bdc 0      folder1/0/0/0
    +100644 e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 0      folder1/0/1
    +100644 78981922613b2afb6025042ff6bd878ac1994e85 0      folder1/a
     100644 e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 0      folder10
    -040000 123706f6fc38949628eaf0483edbf97ba21123ae 0      folder2/
    +100644 e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 0      folder2/0/0/0
    +100644 8b137891791fe96927ad78e64b0aad7bded08bdc 0      folder2/0/1
    +100644 78981922613b2afb6025042ff6bd878ac1994e85 0      folder2/a
     100644 18912c9a915632d7b3344ec25f349dc8b4b9bf27 0      g
    -040000 aaff74984cccd156a469afa7d9ab10e4777beb24 0      x/
    +100644 78981922613b2afb6025042ff6bd878ac1994e85 0      x/a
     100644 2d8c856aebeb20da61bd5112d6fa46ff3f56a9e8 0      z

So that's neat, but unless I'm wrong about there still being some
special-sauce in t/helper/test-read-cache.c that's needed this series
seems incomplete without migrating the existing test users of it to this
new ls-files --sparse, followed by a cherry-pick (or equivalent) of [2].

Is there any reason we wouldn't use ls-files instead of the test-tool in
the tests now? I understood from you at the time that the reason for the
that mode of the test-tool existing was an interim state until ls-files
learned to emit this sort of output, but that you wanted to add that
later. It seems we've arrived at that "later" :)

1. https://lore.kernel.org/git/20210317132814.30175-1-avarab@gmail.com/ (local)
2. https://lore.kernel.org/git/20210317132814.30175-5-avarab@gmail.com/ (local)