Initial push of a fully packed repo - why repack?

Pushing to a new empty repo, git spends a lot of time repacking, even
if we have just run git-repack. The process looks like

 - use parsecvs to import a large CVS repo
 - git-repack -a -d (packs to ~550MB, taking ~1hr)
 - git-push --all git+ssh://host/newlycreatedrepo.git

Does it make sense to detect and optimise for this case?

cheers,


martin

On Tue, 17 Apr 2007, Martin Langhoff wrote:

Pushing to a new empty repo, git spends a lot of time repacking, even
if we have just run git-repack. The process looks like

- use parsecvs to import a large CVS repo
- git-repack -a -d (packs to ~550MB, taking ~1hr)
- git-push --all git+ssh://host/newlycreatedrepo.git

Does it make sense to detect and optimise for this case?

Maybe...  Although the second repack during the push should be much much 
faster than the first one.


Nicolas

On 4/17/07, Nicolas Pitre [off-list ref] wrote:

On Tue, 17 Apr 2007, Martin Langhoff wrote:

quoted

Does it make sense to detect and optimise for this case?

Maybe...  Although the second repack during the push should be much much
faster than the first one.

It is - but it still burns through perhaps 1 minute of CPU and IO
rewriting the exact same pack as you can see:

Generating pack...
Done counting 290123 objects.
Deltifying 290123 objects.
 100% (290123/290123) done
Writing 290123 objects.
 100% (290123/290123) done
Total 290123 (delta 234293), reused 290123 (delta 234293)

cheers,


martin

On Tue, 17 Apr 2007, Martin Langhoff wrote:

On 4/17/07, Nicolas Pitre [off-list ref] wrote:

quoted

On Tue, 17 Apr 2007, Martin Langhoff wrote:

quoted

Does it make sense to detect and optimise for this case?

Maybe...  Although the second repack during the push should be much much
faster than the first one.

It is - but it still burns through perhaps 1 minute of CPU and IO
rewriting the exact same pack as you can see:

Sure.  On the IO you can't save.  You'll have to copy the packanyway and 
with all objects being "reused" the pack-objects code is basically not 
doing much more than a straight cp would do.

What is costly is figuring out if the single pack you have actually
contains all the objects you wish to push, and _only_ the objects you 
wish to push.  That is the real cost.  By the time all those objects are 
listed and accounted then repacking is basically copying the data 
over with almost no CPU usage.  In other words, the cost to determine if 
it is OK to simply send the pack you already have and actually 
sending it would be sensibly the same.


Nicolas