Junio C Hamano [off-list ref] writes:
But it seems to need some more work. I just tried to clone
git.git with --depth=1 and it cauterizes each branch with two
commits (I think that is what depth=1 means -- the latest and
one behind it), but it pulled almost the whole repository
anyway, and it turns out that "git log v1.4.3-rc1" gives me the
full history leading to it.
That's apparently because tags are not considered when truncating the
commit list. The patch below fixes it, and fetches the right number of
commits for each tag. However the correct fix is probably to not fetch
historical tags at all.
There's also a problem with the packing, a clone --depth 1 currently
results in a pack that's about 3 times as large as it should be.
---
diff --git a/shallow.c b/shallow.c
index 58a7b20..2db1dc4 100644
--- a/shallow.c
+++ b/shallow.c
@@ -1,5 +1,6 @@
#include "cache.h"
#include "commit.h"
+#include "tag.h"
static int is_shallow = -1;
@@ -54,7 +55,7 @@ struct commit_list *get_shallow_commits(
if (!commit) {
if (i < heads->nr) {
commit = (struct commit *)
- heads->objects[i++].item;
+ deref_tag(heads->objects[i++].item, NULL, 0);
if (commit->object.type != OBJ_COMMIT) {
commit = NULL;
continue;
--
Alexandre Julliard
Alexandre Julliard [off-list ref] writes:
There's also a problem with the packing, a clone --depth 1 currently
results in a pack that's about 3 times as large as it should be.
That's interesting.
: gitster; git clone -n --depth 1 git://127.0.0.1/git.git victim-001
remote: Generating pack...
remote: Done counting 6246 objects.
remote: Deltifying 6246 objects.
remote: 100% (6246/6246) done
Indexing 6246 objects.
remote: Total 6246, written 6246 (delta 3106), reused 4313 (delta 3106)
100% (6246/6246) done
Resolving 3106 deltas.
100% (3106/3106) done
: gitster; cd victim-001
: gitster; ls -lh .git/objects/pack/
total 9.6M
drwxrwsr-x 2 junio src 4.0K 2006-11-11 23:52 ./
drwxrwsr-x 4 junio src 4.0K 2006-11-11 23:52 ../
-r--r--r-- 1 junio src 148K 2006-11-11 23:52 pack-f5f88d83....idx
-r--r--r-- 1 junio src 9.5M 2006-11-11 23:52 pack-f5f88d83....pack
Repacking immediately after cloning brings it down to what is
expected.
: gitster; git repack -a -d -f
Generating pack...
Done counting 6246 objects.
Deltifying 6246 objects.
100% (6246/6246) done
Writing 6246 objects.
100% (6246/6246) done
Total 6246, written 6246 (delta 4815), reused 1407 (delta 0)
Pack pack-f5f88d83524213e3ab05697ff75f245b1ef9081a created.
: gitster; ls -lh .git/objects/pack/
total 2.8M
drwxrwsr-x 2 junio src 4.0K 2006-11-11 23:53 ./
drwxrwsr-x 4 junio src 4.0K 2006-11-11 23:52 ../
-rw-rw-r-- 1 junio src 148K 2006-11-11 23:53 pack-f5f88d83....idx
-rw-rw-r-- 1 junio src 2.6M 2006-11-11 23:53 pack-f5f88d83....pack
In any case, after this "shallow" stuff, repeated "fetch --depth
99" seems to fetch 0 object and 3400 objects alternately, and
the shallow file alternates between 900 bytes and 11000 bytes.
We would need to take a deeper look into what this series does,
before moving it to 'next'.
On Sun, 12 Nov 2006 00:16:40 -0800 Junio C Hamano wrote:
Alexandre Julliard [off-list ref] writes:
quoted
There's also a problem with the packing, a clone --depth 1 currently
results in a pack that's about 3 times as large as it should be.
That's interesting.
: gitster; git clone -n --depth 1 git://127.0.0.1/git.git victim-001
[...]
-r--r--r-- 1 junio src 9.5M 2006-11-11 23:52 pack-f5f88d83....pack
Repacking immediately after cloning brings it down to what is
expected.
: gitster; git repack -a -d -f
[...]
-rw-rw-r-- 1 junio src 2.6M 2006-11-11 23:53 pack-f5f88d83....pack
This is due to optimization in builtin-pack-objects.c:try_delta():
/*
* We do not bother to try a delta that we discarded
* on an earlier try, but only when reusing delta data.
*/
if (!no_reuse_delta && trg_entry->in_pack &&
trg_entry->in_pack == src_entry->in_pack)
return 0;
After removing this part the shallow pack after clone is 2.6M, as it
should be.
The problem with this optimization is that it is only valid if we are
repacking either the same set of objects as we did earlier, or its
superset. But if we are packing a subset of objects, there will be some
objects in that subset which were delta-compressed in the original pack,
but base objects for that deltas are not included in our subset -
therefore we will be unable to reuse existing deltas, and with that
optimization we will never try to use delta compression for these
objects. (The optimization assumes that if we will try to use delta
compression, we will try mostly the same base objects as we have tried
when we made the existing pack, and therefore will likely get the same
result - which is close to the truth when we are doing "repack -a", but
is badly wrong when we are doing "git-upload-pack" with a large number
of common commits, and therefore are excluding a lot of objects.)
So any partial fetch (shallow or not) from a mostly packed repository
currently results in a suboptimal pack. In fact, the fresh "repack -a
-d -f" is probably the worst case for subsequent fetch (not initial
clone) from that repository - objects for the most recent commit are
most likely to be stored without delta compression, and even if deltas
are used, they are likely in the wrong direction for someone who has an
older version and wants to update it.
In any case, after this "shallow" stuff, repeated "fetch --depth
99" seems to fetch 0 object and 3400 objects alternately, and
the shallow file alternates between 900 bytes and 11000 bytes.
I confirm this - different numbers, but the same problem...