From: Junio C Hamano <hidden> Date: 2021-08-31 16:33:44
Jeff King [off-list ref] writes:
quoted
I think taking a look to see if ../config exists to use the data
might be helpful for some cases, but should not be a blocker for
completing the requested operation. The config from the non-alternate
repo should be sufficient for this (somewhat strange) case.
Yes, agreed. We have long supported these kind of "bare" alternates, and
I wouldn't be surprised if they are in wide use (though I do wonder how
folks actually modify them, since most commands that touch objects
really do want to be in a repository).
I kind of find the above two somewhat surprising, but I am willing
to go with the less safer option if that is what people want.
It has been perfectly OK in the pre-alternative-hash-algorithms
world, but we no longer live in such a world, so we'd need to come
up with a way to keep using alternates in a safer way.
I do not see the reasoning behind "should not be a blocker" from
Derrick substantiated. What's the reason why that raw object store
cannot come from an existing repository, and what's the benefit we
get from not having to have a repository there?
The natural extension for the hash-format problem would probably be to
call check_repository_format_gently() on the parent directory of the
alternate-objects dir. If it succeeds, then we can pull out the
hash_algo parameter from its repository_format struct. And if not, then
we just assume it matches the main repo.
But I suspect all of this is moot for now, beyond being able to return a
nicer error message. The rest of the code is not at all ready to handle
packs with two different hashes in the same process.
I do not think it is all that urgent to make it possible for packs
with different algorithms to be used. It is sufficient to _ignore_
(or error out) configured odb that is incompatible with the current
repository.
Thanks.
From: Taylor Blau <hidden> Date: 2021-08-31 16:43:21
On Tue, Aug 31, 2021 at 09:33:38AM -0700, Junio C Hamano wrote:
I do not see the reasoning behind "should not be a blocker" from
Derrick substantiated. What's the reason why that raw object store
cannot come from an existing repository, and what's the benefit we
get from not having to have a repository there?
I also didn't find the reasoning spelled out in his response, but I have
definitely had off-list discussions with Stolee where it was important to
be able to pass a value to `--object-dir` which does *not* belong to a
Git repository (but is used as a dumping ground for packs, a MIDX, and
loose objects).
It may be worthwhile to recapitulate that discussion here on the list.
(I'm hoping that Stolee won't mind filling in the details, since I seem
to have forgotten most of them).
Thanks,
Taylor
On Tue, Aug 31, 2021 at 09:33:38AM -0700, Junio C Hamano wrote:
quoted
I do not see the reasoning behind "should not be a blocker" from
Derrick substantiated. What's the reason why that raw object store
cannot come from an existing repository, and what's the benefit we
get from not having to have a repository there?
I also didn't find the reasoning spelled out in his response, but I have
definitely had off-list discussions with Stolee where it was important to
be able to pass a value to `--object-dir` which does *not* belong to a
Git repository (but is used as a dumping ground for packs, a MIDX, and
loose objects).
It may be worthwhile to recapitulate that discussion here on the list.
(I'm hoping that Stolee won't mind filling in the details, since I seem
to have forgotten most of them).
The way we have been using alternates in VFS for Git and Scalar is as a
"shared object cache" that is shared across multiple full Git repositories
with their own working trees. The shared object cache is located in a
location that can be found during "scalar clone" such as
~/.scalarCache/url_<hash-of-URL>/
This directory contains the same data as a .git/objects directory would.
Data is added to that cache using hooks during 'git fetch' or other
requests for remote data. This means that the second "scalar clone"
command is much faster than the first, because it already has most of
the commit and tree data required to satisfy the partial clone.
(Note: this feature does not exist in the current Scalar CLI RFC, but
would be contributed later.)
These caches were designed before the multi-pack-index -- in fact,
they were an inspiration for them because now deleting a repo would not
clean up old pack-files. The data would be added as a raw pack-file that
is processed with 'git index-pack' or as loose objects. The --object-dir
option was directly created as a way to target the creation and
maintenance of a multi-pack-index within one of these caches that don't
exist as full repositories. Clearly, there were some gaps in that
implementation and I regret creating those gaps.
If I were to redesign the shared object cache, then I would have created
the cache directories as bare repos and then create the "clone" repo as
a worktree linked to that base. That would allow all objects and refs to
be shared, achieving the same goals and an even better user experience.
I'm advocating for the position to continue allowing this feature to
exist without a necessary on-upgrade conversion of these non-repos to
full repos. Maybe that is the best thing to do in the long-term, but
will take some time to do. Keeping compatibility for now seems like it
won't hurt too much.
Thanks,
-Stolee
From: Jeff King <hidden> Date: 2021-09-01 10:03:09
On Tue, Aug 31, 2021 at 09:33:38AM -0700, Junio C Hamano wrote:
Jeff King [off-list ref] writes:
quoted
quoted
I think taking a look to see if ../config exists to use the data
might be helpful for some cases, but should not be a blocker for
completing the requested operation. The config from the non-alternate
repo should be sufficient for this (somewhat strange) case.
Yes, agreed. We have long supported these kind of "bare" alternates, and
I wouldn't be surprised if they are in wide use (though I do wonder how
folks actually modify them, since most commands that touch objects
really do want to be in a repository).
I kind of find the above two somewhat surprising, but I am willing
to go with the less safer option if that is what people want.
It has been perfectly OK in the pre-alternative-hash-algorithms
world, but we no longer live in such a world, so we'd need to come
up with a way to keep using alternates in a safer way.
I think the point is that most people _do_ still live in that world.
They have not started using the new hash algorithm yet, and what they
have been doing for years will continue to work. Likewise, once they
switch, things will continue to work as long as each repo's alternates
use the same hash.
So my reasoning was less "this is useful, and a good idea" and more "it
works now, and will probably continue to work OK in practice, so taking
it away will probably bother people".
Now if somebody wants to make an argument that they are not actually
workable now, I could buy that. ;) You cannot even run "pack-objects"
without a repository, though it is not too hard to copy the result
around.
quoted
But I suspect all of this is moot for now, beyond being able to return a
nicer error message. The rest of the code is not at all ready to handle
packs with two different hashes in the same process.
I do not think it is all that urgent to make it possible for packs
with different algorithms to be used. It is sufficient to _ignore_
(or error out) configured odb that is incompatible with the current
repository.
Yes, I think that would be an improvement. I just don't find it all that
urgent, since they're likely to get an error anyway (just probably one
that is more mysterious). Given the work involved to even detect the
situation, it doesn't seem like that high a priority to me.
-Peff