Re: git-repack: Outof memory

3 messages, 2 authors, 2016-06-15 · open the first message on its own page

Re: git-repack: Outof memory

From: Shawn Pearce <hidden>
Date: 2016-06-15 22:42:40

Dongsheng Song [off-list ref] wrote:
finished, thanks a lot.

$ ls -l .git/objects/pack/
total 1675466
-rw-r--r-- 1 www-data www-data    2964992 Sep 17 19:17
pack-b09c57c25e4459f6365b5d27139abfd93bf1c86f.idx
-rw-r--r-- 1 www-data www-data 1711037142 Sep 17 19:17
pack-b09c57c25e4459f6365b5d27139abfd93bf1c86f.pack

If the pack files larger than 2.5g,  how can  I repack it on i686 ?
*youch* A 1.5 GiB pack file?

Your files apparently do not delta compress very well.  I fear that
you are going to bump up against address space limitations soon
on 32 bit systems.  Then you will bump up against the 4 GiB pack
file size limit.  Which means you will need to use several packs
and avoid the '-a' flag when calling git-repack.

Nico was suggesting using the default window size (rather than
--window=64) as larger windows requires more memory during repack.
But you may also need the mmap window code I'm working on.  I better
hurry up and get that into Junio's testing branches.  :-)

-- 
Shawn.

Re: git-repack: Outof memory

From: Junio C Hamano <hidden>
Date: 2016-06-15 22:42:40

Shawn Pearce [off-list ref] writes:
...  I fear that
you are going to bump up against address space limitations soon
on 32 bit systems.  Then you will bump up against the 4 GiB pack
file size limit.  Which means you will need to use several packs
and avoid the '-a' flag when calling git-repack.
I presume that you are hinting that we would need to update
git-repack so that it is still useful without --all.

Which means "pack-objects --unpacked=active-pack" would need to
be pushed out so that git-repack can be updated to do the
"archived ones and repacking the active pack" we talked about
earlier.

I think pack-objects --unpacked=active-pack is ready, so I'll
push it out.

Re: git-repack: Outof memory

From: Shawn Pearce <hidden>
Date: 2016-06-15 22:42:40

Junio C Hamano [off-list ref] wrote:
Shawn Pearce [off-list ref] writes:
quoted
...  I fear that
you are going to bump up against address space limitations soon
on 32 bit systems.  Then you will bump up against the 4 GiB pack
file size limit.  Which means you will need to use several packs
and avoid the '-a' flag when calling git-repack.
I presume that you are hinting that we would need to update
git-repack so that it is still useful without --all.
Yes.  :-)

Apparently the Porcelain level is still open for debate.  Someone
just needs to post a reasonable implementation to get discussion
(or lack there of) going.  I'm too tired right now to try to roll
something up right, even though it looks to be rather simple.

As it is I'm having a difficult time navigating in vi to write this
email and normally vi navigation is as easy as breathing for me...
I think pack-objects --unpacked=active-pack is ready, so I'll
push it out.
Thanks; I noticed that in your "What's in git.git" announcement.

-- 
Shawn.
Keyboard shortcuts
hback out one level
jnext message in thread
kprevious message in thread
ldrill in
Escclose help / fold thread tree
?toggle this help