git auto-repack is broken...

4 messages, 3 authors, 2016-06-15 · open the first message on its own page

git auto-repack is broken...

From: Linus Torvalds <torvalds@linux-foundation.org>
Date: 2016-06-15 22:52:33

I actually tend to repack things pretty religiously (ok, not really,
but I do "git gc" reasonably regularly, so I was surprised to see
thig:

  Auto packing the repository for optimum performance. You may also
  run "git gc" manually. See "git help gc" for more information.

followed by this pitiful effort:

  Counting objects: 8, done.
  Delta compression using up to 4 threads.
  Compressing objects: 100% (8/8), done.
  Writing objects: 100% (8/8), done.
  Total 8 (delta 0), reused 0 (delta 0)

Ok, those 8 objects will *not* help anything at all, and the
autorepack is broken.

So what's going on? It turns out that I have a fair amount of
unreachable objects in this repository, because I do things like
fetching things without then merging them, etc. So the "git gc --auto"
will happily do "git repack -A" or whatever, and that in turn does
*nothing* what-so-ever (or rather, it packs my latest merge commit
like the above and generates that pack of a whopping 8 objects).

I can fix it with "git gc --prune=now", so it's not like I personally
really care, but since the whole point of "git gc --auto" is to allow
people who don't know what they are doing to ignore the whole issue of
GC and pruning, I do think this is a real UI bug.

I don't really have any suggestions for fixing it, though. Maybe we
should make "git gc --auto" remove any unreachable objects? That would
be potentially dangerous in shared repository situations, though. Or
have an extra option to "git repack -A" to also pack any loose objects
it finds at the end (whether reachable or not)?

                         Linus

Re: git auto-repack is broken...

From: Ævar Arnfjörð Bjarmason <hidden>
Date: 2016-06-15 22:52:33

On Fri, Dec 2, 2011 at 17:22, Linus Torvalds
[off-list ref] wrote:
Maybe we should make "git gc --auto" remove any unreachable objects?
Wouldn't that mean that any loose commit objects you have lying around
would be removed by the automatic git gc?

One feature of git that I personally rely on is that I can liberally
move heads around / make commits on detached heads and not have those
commits gc'd unless I explicitly ask for it for a while.

Re: git auto-repack is broken...

From: Linus Torvalds <torvalds@linux-foundation.org>
Date: 2016-06-15 22:52:33

On Fri, Dec 2, 2011 at 8:27 AM, Ævar Arnfjörð Bjarmason
[off-list ref] wrote:
quoted
Maybe we should make "git gc --auto" remove any unreachable objects?
Wouldn't that mean that any loose commit objects you have lying around
would be removed by the automatic git gc?

One feature of git that I personally rely on is that I can liberally
move heads around / make commits on detached heads and not have those
commits gc'd unless I explicitly ask for it for a while.
Well, with reflogs, you actually do have those objects reachable for
quite a while (90 days by default).

The "unreachable objects" tends to happen when you do fetches without
ever merging the result or actually remove branches (and/or expiring
the reflogs early etc). Not from the normal "use 'git reset' and
friends to move heads around".

That said, I do agree that removing loose objects is the much less
safe approach.

Of course, repacking the objects results in problems too: now you've
entirely lost the age information for that object, so now you cannot
prune it based on age any more.

But leaving the loose objects around and basically failing auto-gc
isn't good either.

                     Linus

Re: git auto-repack is broken...

From: Jeff King <hidden>
Date: 2016-06-15 22:52:33

On Fri, Dec 02, 2011 at 08:56:34AM -0800, Linus Torvalds wrote:
On Fri, Dec 2, 2011 at 8:27 AM, Ævar Arnfjörð Bjarmason
[off-list ref] wrote:
quoted
quoted
Maybe we should make "git gc --auto" remove any unreachable objects?
Wouldn't that mean that any loose commit objects you have lying around
would be removed by the automatic git gc?

One feature of git that I personally rely on is that I can liberally
move heads around / make commits on detached heads and not have those
commits gc'd unless I explicitly ask for it for a while.
Well, with reflogs, you actually do have those objects reachable for
quite a while (90 days by default).

The "unreachable objects" tends to happen when you do fetches without
ever merging the result or actually remove branches (and/or expiring
the reflogs early etc). Not from the normal "use 'git reset' and
friends to move heads around".

That said, I do agree that removing loose objects is the much less
safe approach.
We do remove loose objects that are totally unreferenced, but there is
still a time-delay, because we don't want to prune something like an
in-progress commit operation. The default delay for that is 2 weeks,
which I think is an arbitrary number that was "wow, if your git
operation takes longer than this, you're way too patient".

And in general, it works OK because people don't tend to accumulate more
than the auto-gc number of objects within a 2 week period. So perhaps
you're just special in your usage patterns.

One solution is just dropping that "2 weeks" down to something smaller,
but still conservative (say, 3 days?).

If you still have the repo in question, what is the date breakdown on
your loose objects?
Of course, repacking the objects results in problems too: now you've
entirely lost the age information for that object, so now you cannot
prune it based on age any more.
When the objects become unreferenced, we eject them from the pack into
loose form again. If they don't become referenced in the 2-week window,
they get pruned then. So yes, you drop the age information, but they do
eventually go away.

-Peff
Keyboard shortcuts
hback out one level
jnext message in thread
kprevious message in thread
ldrill in
Escclose help / fold thread tree
?toggle this help