Hi,
I have a shallow clone of linux-stable repo [1] on my computer. Now
I'm trying to repack with `git repack -A -d`.
Before repacking, here is the object counts on my clone
(`git count-objects -v`):
When I trigger repack operation, I expected that all objects on 17 packs
are consolidated into several 650M-sized packs. However, in my case, repacking
was hang at "Enumerating objects" stage, that is I stuck at:
"Eumerating objects: 902036"
I also try to reproduce the issue with other repo, such as with local copy
of GCC repo on my computer [2] using similar config, and the repack succeed
without any errors.
Am I missing something?
[1]: https://github.com/gregkh/linux
[2]: https://github.com/gcc-mirror/gcc
--
An old man doll... just what I always wanted! - Clara
From: Jeff King <hidden> Date: 2021-05-17 08:51:37
On Sun, May 16, 2021 at 08:09:56PM +0700, Bagas Sanjaya wrote:
I have a shallow clone of linux-stable repo [1] on my computer. Now
I'm trying to repack with `git repack -A -d`.
Before repacking, here is the object counts on my clone
(`git count-objects -v`):
When I trigger repack operation, I expected that all objects on 17 packs
are consolidated into several 650M-sized packs. However, in my case, repacking
was hang at "Enumerating objects" stage, that is I stuck at:
"Eumerating objects: 902036"
You could try using strace or gdb to see what it's doing.
But as a guess, one thing that sometimes causes a stall near the end of
"enumerating objects" is loosening unreachable objects that are
currently packed. You told repack to use "-A", which asks to loosen
those objects so they aren't lost when the old packs are deleted (as
opposed to "-a").
You'd probably want to at least say "--unpack-unreachable=some.time" to
avoid writing out ones which are not even recent (and which is what "git
gc" will do under the hood).
But if you don't care about keeping them at all (e.g., because this is
not an active repository where other simultaneous operations may be
taking place, so you know it is safe to delete even recent ones), then
just "git repack -a -d" is probably your best bet.
-Peff
You could try using strace or gdb to see what it's doing.
But as a guess, one thing that sometimes causes a stall near the end of
"enumerating objects" is loosening unreachable objects that are
currently packed. You told repack to use "-A", which asks to loosen
those objects so they aren't lost when the old packs are deleted (as
opposed to "-a").
I attached two strace logs, one for "git repack -A -d" and one for "git
repack -a -d".
For the former, I got following excerpt before I had to SIGINT the process:
I thought that in the case of "git repack -A -d", it is stucked at the
last read() before I ctrl-c'ed to trigger SIGINT.
You'd probably want to at least say "--unpack-unreachable=some.time" to
avoid writing out ones which are not even recent (and which is what "git
gc" will do under the hood).
quoted
But if you don't care about keeping them at all (e.g., because this is
not an active repository where other simultaneous operations may be
taking place, so you know it is safe to delete even recent ones), then
just "git repack -a -d" is probably your best bet.
-Peff
Using "-a" instead of "-A" on git repack works. Thanks.
--
An old man doll... just what I always wanted! - Clara
From: Jeff King <hidden> Date: 2021-05-18 12:08:00
On Tue, May 18, 2021 at 06:23:40PM +0700, Bagas Sanjaya wrote:
quoted
You could try using strace or gdb to see what it's doing.
But as a guess, one thing that sometimes causes a stall near the end of
"enumerating objects" is loosening unreachable objects that are
currently packed. You told repack to use "-A", which asks to loosen
those objects so they aren't lost when the old packs are deleted (as
opposed to "-a").
I attached two strace logs, one for "git repack -A -d" and one for "git
repack -a -d".
For the former, I got following excerpt before I had to SIGINT the process:
I thought that in the case of "git repack -A -d", it is stucked at the
last read() before I ctrl-c'ed to trigger SIGINT.
You need "strace -f". The parent repack process spawns pack-objects (via
the clone() call above), and then waits for it to print the name of the
generated pack at the end. So it will stall on that read() for quite a
while, even under normal circumstances.
-Peff
Hi Jeff, sorry for long reply.
On 18/05/21 19.07, Jeff King wrote:
You need "strace -f". The parent repack process spawns pack-objects (via
the clone() call above), and then waits for it to print the name of the
generated pack at the end. So it will stall on that read() for quite a
while, even under normal circumstances.
-Peff
OK, so this is writing out loose objects. It's exactly the case I
suspected.
The last write() sequence seems running repeatedly, what it means? Infinite loop?
If it's a lot of just write(), it's probably just a large object. If
it's a lot of writes interspersed with opens and links, as above, then
it's just a lot of objects. It probably would finish eventually, but
it's just slow.
-Peff