git repack and dumb protocols

Hi there,

I've posted a git repository to the web that is an rsync copy of my 
development archive's .git directory, with update-server-info run on it. Now 
if I understand correctly, this contains a single file for every 
commit/file/whatever else I ever made, which is a paint over HTTP. I have then 
recently learned about the wonder that is git repack, which I would love to 
use on this repository, were it not for this comment in its manpage:

8< ----------------------------------------------------------------------
Especially useful when packing a repository that is used for private 
development and there is no need to worry about people fetching via dumb 
protocols from it. 8< 
----------------------------------------------------------------------

It says this relating to the '-a' option, but it makes me wonder if I'll break 
copies of the repo that other people have pulled if I use 'repack'? What are 
the possible interactions and things to keep in mind between repack and dumb 
protocols?

In any case, I would like to suggest that the rather ominous-sounding text in 
the manpage be replaced with something more concrete, i.e. "If you repack an 
archive that has people fetching via dumb protocols from it, X, Y, and Z will 
happen."

Thanks,
Andreas

(please cc--not subscribed)

The current text make some users feel uneasy, worrying whether
'-a' could lead to corrupt repositories. Clarify that '-a'
may lead to performance issues only for dumb protocols.

Signed-off-by: Michael J Gruber <redacted>
---
Is that clear enough but concise?

Applies to maint.

 Documentation/git-repack.txt |    5 ++++-
 1 files changed, 4 insertions(+), 1 deletions(-)

diff --git a/Documentation/git-repack.txt b/Documentation/git-repack.txt
index aaa8852..8af5505 100644
--- a/Documentation/git-repack.txt
+++ b/Documentation/git-repack.txt

@@ -32,7 +32,10 @@ OPTIONS
 	pack everything referenced into a single pack.
 	Especially useful when packing a repository that is used
 	for private development and there is no need to worry
-	about people fetching via dumb protocols from it.  Use
+	about people fetching via dumb protocols from it. 
+	(Over dumb protocols, one would have to fetch the whole new
+	pack in order to get any contained object, no matter how
+	many other objects in that pack exist locally already.) Use
 	with '-d'.  This will clean up the objects that `git prune`
 	leaves behind, but `git fsck --full` shows as
 	dangling.

-- 
1.6.3.1.268.g94d6d1

The current text makes some users feel uneasy, worrying whether
'-a' could lead to corrupt repositories. Clarify that '-a'
may lead to performance issues only for dumb protocols.

Signed-off-by: Michael J Gruber <redacted>
---
Is that clear enough but concise?

Applies to maint.

[Sorry for the resend, I managed to mistype the subject.]

 Documentation/git-repack.txt |    5 ++++-
 1 files changed, 4 insertions(+), 1 deletions(-)

diff --git a/Documentation/git-repack.txt b/Documentation/git-repack.txt
index aaa8852..8af5505 100644
--- a/Documentation/git-repack.txt
+++ b/Documentation/git-repack.txt

@@ -32,7 +32,10 @@ OPTIONS
 	pack everything referenced into a single pack.
 	Especially useful when packing a repository that is used
 	for private development and there is no need to worry
-	about people fetching via dumb protocols from it.  Use
+	about people fetching via dumb protocols from it. 
+	(Over dumb protocols, one would have to fetch the whole new
+	pack in order to get any contained object, no matter how
+	many other objects in that pack exist locally already.) Use
 	with '-d'.  This will clean up the objects that `git prune`
 	leaves behind, but `git fsck --full` shows as
 	dangling.

-- 
1.6.3.1.268.g94d6d1

On Wed, Jun 3, 2009 at 7:50 PM, Andreas Klöckner wrote:

8< ----------------------------------------------------------------------
Especially useful when packing a repository that is used for private
development and there is no need to worry about people fetching via dumb
protocols from it. 8<
----------------------------------------------------------------------

It says this relating to the '-a' option, but it makes me wonder if I'll break
copies of the repo that other people have pulled if I use 'repack'? What are
the possible interactions and things to keep in mind between repack and dumb
protocols?

It won't break any copies, but it may cause excessive network use.

Dumb protocols can only fetch or not-fetch the entire pack. So if
someone hasn't fetched since the last push, the next time they do a
"git pull", they will retrieve the entire repository. Even if they
only need a single small commit.

As you can imagine, people using dumb protocols over slow connections
may become somewhat annoyed with the admin that often does a repack
-a.

You do want to repack (usually without -a -- but see also ".keep")
periodically, though. Working out a schedule depends on the activity
level of the repository, and is left as an exercise for the reader.

In any case, I would like to suggest that the rather ominous-sounding text in
the manpage be replaced with something more concrete, i.e. "If you repack an
archive that has people fetching via dumb protocols from it, X, Y, and Z will
happen."

Patches welcome.

Peter Harris

Thanks, that's much better!

Andreas

On Donnerstag 04 Juni 2009, Michael J Gruber wrote:

On Thu, Jun 4, 2009 at 4:34 AM, Michael J
Gruber[off-list ref] wrote:

       pack everything referenced into a single pack.
       Especially useful when packing a repository that is used
       for private development and there is no need to worry
-       about people fetching via dumb protocols from it.  Use
+       about people fetching via dumb protocols from it.
+       (Over dumb protocols, one would have to fetch the whole new
+       pack in order to get any contained object, no matter how
+       many other objects in that pack exist locally already.) Use
       with '-d'.  This will clean up the objects that `git prune`
       leaves behind, but `git fsck --full` shows as
       dangling.

Instead of adding this, why don't we just remove the part about dumb
protocols? So just say that it's especially useful for private
development. Then, if we really want to keep this performance note
just add it after the paragraph in one of those "+" things (What are
those called?)

Stephen Boyd venit, vidit, dixit 04.06.2009 21:46:

On Thu, Jun 4, 2009 at 4:34 AM, Michael J
Gruber[off-list ref] wrote:

quoted

       pack everything referenced into a single pack.
       Especially useful when packing a repository that is used
       for private development and there is no need to worry
-       about people fetching via dumb protocols from it.  Use
+       about people fetching via dumb protocols from it.
+       (Over dumb protocols, one would have to fetch the whole new
+       pack in order to get any contained object, no matter how
+       many other objects in that pack exist locally already.) Use
       with '-d'.  This will clean up the objects that `git prune`
       leaves behind, but `git fsck --full` shows as
       dangling.

Instead of adding this, why don't we just remove the part about dumb
protocols? So just say that it's especially useful for private
development. Then, if we really want to keep this performance note
just add it after the paragraph in one of those "+" things (What are
those called?)

...because "repack -a" hurts *badly* on even midsized repos when dumb
protocols are used. On the other hand, using it isn't that much of an
improvement in terms of pack size, unless you repack manually very often
and, thus, produce many small packs. So, "gc --auto" users should be
fine in any case (dumb or not - the protocol, that is...) as long as
they don't use "-a".

Michael

On Fri, Jun 5, 2009 at 1:16 AM, Michael J
Gruber[off-list ref] wrote:

Stephen Boyd venit, vidit, dixit 04.06.2009 21:46:

quoted

Instead of adding this, why don't we just remove the part about dumb
protocols? So just say that it's especially useful for private
development. Then, if we really want to keep this performance note
just add it after the paragraph in one of those "+" things (What are
those called?)

...because "repack -a" hurts *badly* on even midsized repos when dumb
protocols are used. On the other hand, using it isn't that much of an
improvement in terms of pack size, unless you repack manually very often
and, thus, produce many small packs. So, "gc --auto" users should be
fine in any case (dumb or not - the protocol, that is...) as long as
they don't use "-a".

Maybe I spoke wrongly. I'm suggesting something like

       Especially useful when packing a repository that is used
       for private development.  Use with '-d'.  This will clean up
       the objects that `git prune` leaves behind, but
       `git fsck --full` shows as dangling.
+
Note: Users fetching over dumb protocols will have to fetch the
whole pack in order to get any contained object, no matter how
many other objects in that pack they already have locally.

I think this Note part probably needs some work though.

The current text makes some users feel uneasy, worrying whether
'-a' could lead to corrupt repositories. Clarify that '-a'
may lead to performance issues only for dumb protocols.

Signed-off-by: Michael J Gruber <redacted>
---
Helped-by: Stephen Boyd [off-list ref]

 Documentation/git-repack.txt |    7 +++++--
 1 files changed, 5 insertions(+), 2 deletions(-)

diff --git a/Documentation/git-repack.txt b/Documentation/git-repack.txt
index aaa8852..c9257a1 100644
--- a/Documentation/git-repack.txt
+++ b/Documentation/git-repack.txt

@@ -31,11 +31,14 @@ OPTIONS
 	Instead of incrementally packing the unpacked objects,
 	pack everything referenced into a single pack.
 	Especially useful when packing a repository that is used
-	for private development and there is no need to worry
-	about people fetching via dumb protocols from it.  Use
+	for private development. Use
 	with '-d'.  This will clean up the objects that `git prune`
 	leaves behind, but `git fsck --full` shows as
 	dangling.
++
+Note that users fetching over dumb protocols will have to fetch the
+whole new pack in order to get any contained object, no matter how many
+other objects in that pack they already have locally.
 
 -A::
 	Same as `-a`, unless '-d' is used.  Then any unreachable

-- 
1.6.3.2.278.gb6431.dirty