Linus Torvalds [off-list ref] writes:
So the problem may be that we basically send a totally unnecessary list of
all the objects we have, when the other end really only cares about the
fact that we have the objects that the tags point to. Which we know we do,
but we didn't say so, because "git-fetch" didn't really mark them that
way.
I think this speculation is correct. We should be able to do
better.
I almost suspect that we need to have a syntax where-by the local
fetch-list ends up doing
"$tagname:$tagname:$sha1wehave"
as the argument to fetch-pack, and then fetch-pack would be modified to
send those "$sha1wehave" objects early as "have" objects.
But this logic has to be a bit more involved.
A "have" object is not just has_sha1_file(), but it needs to be
reachable from one of our tips we have already verified as
complete, so either the caller of fetch-pack does the
verification and give a verified $sha1wehave, or fetch-pack
takes $sha1weseemtohave and does its own verification and then
send it as one of the "have" objects (the issue is the same as
the one in my previous message to Eric W. Biederman -- we trust
only refs not just having a single object).
It might be useful to have a helper script you can give N object
names and M refs (and/or --all flag to mean "all of the refs"),
which returns the ones that are reachable from the given refs.
It would be even more useful if it were a helper function, but
given that the computation would involve walking the ancestry
chain, I suspect it would have a bad interaction with any user
of such a helper function that wants to do its own ancestry
walking, because many of them seem to assume an object that has
already been parsed are the ones they parsed for their own
purpose.
On Wed, 24 May 2006, Junio C Hamano wrote:
A "have" object is not just has_sha1_file(), but it needs to be
reachable from one of our tips we have already verified as
complete
You're right.
And the strange part is that the commit we should give for the tag thing
_should_ actually be pretty recent, and I wonder why we end up walking the
whole damn tree history and saying "want" to basically them all.
IOW, I think there's something more fundamentally wrong with the tag
following. We _should_ have figured out much more quickly that we have it
all.
I'm starting to suspect that it's actually a tag-specific problem: we do
that reachability crud all by commit history, so the tags are a total
special case, and if we don't send the proper HAVE/WANT for those or mark
them properly with THEY_HAVE/COMMON etc, maybe the algorithm just gets
confused.
I need to go pick up my youngest, so I'll be off-line on this for a while.
Will try to think it through.
Linus
On Wed, 24 May 2006, Linus Torvalds wrote:
IOW, I think there's something more fundamentally wrong with the tag
following. We _should_ have figured out much more quickly that we have it
all.
Actually, maybe the problem is that Ralf's tree has two roots, because of
the old CVS history. It might be following the other root down for the
"have" part, since that one doesn't exist at all in the target and the
other side will never acknowledge any of it.
I'll play with it.
Linus
On Wed, May 24, 2006 at 04:43:02PM -0700, Linus Torvalds wrote:
Actually, maybe the problem is that Ralf's tree has two roots, because of
the old CVS history. It might be following the other root down for the
"have" part, since that one doesn't exist at all in the target and the
other side will never acknowledge any of it.
I'll play with it.
Interesting idea, so I went to play with it, too. I took a copy of the
tree and deleted all branches except the v2.6.16-stable tracking branch
which I pruned back to v2.6.16.17, then added a new branch starting at
the oldest commit, your initial import of the kernel tree:
$ git branch junk 1da177e4c3f41524e886b7f1b8a0c1fc7321cac2
$ git checkout junk
$ seq -f "%05.0f" 1 100 | while read i; do echo $i; echo $i > Makefile;\
git commit -s -m "Blah $i" Makefile; done
So with this I get:
$ git branch
* junk
v2.6.16-stable
$
If I now run
$ strace git-fetch-pack --thin git://www.kernel.org/pub/scm/linux/kernel/\
git/stable/linux-2.6.16.y.git \
refs/heads/master refs/tags/v2.6.16.18 2>&1 | grep have /tmp/xxx
I get:
write(3, "0032have ef686028603c291ba510c66"..., 50) = 50
write(3, "0032have 150384dac99eb263c4385c7"..., 50) = 50
write(3, "0032have 4df3afbfc2d8f6c22d41c63"..., 50) = 50
...
write(3, "0032have db119fba3d9495aa9cd5a63"..., 50) = 50
Where ef686028603c291ba510c66 = junk, 150384dac99eb263c4385c7 = junk~1 ...
db119fba3d9495aa9cd5a63 = junk~99 (first commit on the junk branch).
100 "have" lines upto this point, then:
write(3, "0032have d87319c3e4d908e157a462d"..., 50) = 50
write(3, "0032have 22ddf44d54d0b2326f7b233"..., 50) = 50
write(3, "0032have 90a03936acb1c3400a5833c"..., 50) = 50
write(3, "0032have bf7d8bacaaf241a0f015798"..., 50) = 50
write(3, "0032have a120571fbdfc8f543eea642"..., 50) = 50
write(3, "0032have 42a46c74c4520174b82a60a"..., 50) = 50
write(3, "0032have f66ab685594d49e570b2176"..., 50) = 50
write(3, "0032have 834f514019e01f87657a257"..., 50) = 50
write(3, "0032have 9d395d1961a0eeb9e8b1ef2"..., 50) = 50
write(3, "0032have aa48603d1ba772d0a2b28ab"..., 50) = 50
write(3, "0032have 54e5705fd460c7621a4d73c"..., 50) = 50
write(3, "0032have 37863c8a9b7b0261ec76daa"..., 50) = 50
write(3, "0032have a7603f9099869f9aeebd6c7"..., 50) = 50
write(3, "0032have 623c30d2ae22cd4b8703c77"..., 50) = 50
write(3, "0032have e2c78fb27dd13ab8c778a96"..., 50) = 50
write(3, "0032have dbb676d1214c181e6cde4ce"..., 50) = 50
write(3, "0032have 1ffe5e06461f72b9b6a2569"..., 50) = 50
These are the commits for which this test tree has the tags left:
$ ls .git/refs/tags/
v2.6.16.1 v2.6.16.12 v2.6.16.15 v2.6.16.2 v2.6.16.5 v2.6.16.8
v2.6.16.10 v2.6.16.13 v2.6.16.16 v2.6.16.3 v2.6.16.6 v2.6.16.9
v2.6.16.11 v2.6.16.14 v2.6.16.17 v2.6.16.4 v2.6.16.7
$
And finally:
write(3, "0032have 1da177e4c3f41524e886b7f"..., 50) = 50
which is your Linux-2.6.12-rc2 import.
Ralf