Re: First Git status takes 40+ minutes, when mounting fileystem/diskimage with 50G GIT repo + 900G of builds articles

2 messages, 2 authors, 2019-08-23 · open the first message on its own page

Re: First Git status takes 40+ minutes, when mounting fileystem/diskimage with 50G GIT repo + 900G of builds articles

From: Junio C Hamano <hidden>
Date: 2019-08-22 18:13:32

"Saravanan Shanmugham (sarvi)" [off-list ref] writes:
I suspect warming the filesystem caching is in play.
But so is the fact that walk every tree to find untracked files as well. 
Enable the untracked cache and "update-index --refresh", before
freezing the repository + working tree state in the diskimage?

Re: First Git status takes 40+ minutes, when mounting fileystem/diskimage with 50G GIT repo + 900G of builds articles

From: Saravanan Shanmugham (sarvi) <hidden>
Date: 2019-08-23 00:39:30

Thanks. That seems to help.

Just for context, we are using a copy-on-write/cloning solution to give developers  a prebuilt workspace with all the platforms fully built, as well as sources all cloned,
So they are ready for incremental development.

We create a ext4 diskimage, with a git clone workspace(50G) and 900G of fully built trees, then freeze that diskimage.
Developers have a way to clone that diskimage(into a copy-on-write) copy of the diskimage(in 30 seconds) that they can mount and use for incremental build development.

I did the following.

Mounted the existing filesystem
git update-index --untracked-cache
git update-index --refresh

Not sure what you meant by " working tree state in the diskimage"

I then detached the diskimage, dropped all the caches. 
Remounted the diskimage.


git status  - The very first git status after dropping the cache now returns in in 1.54 seconds, which seems very much acceptable.

I obviously need to do more read up on what/how git caches.

Can you point to any documentation on what sort of information GIT caches, and how to understand debug that?
I would like to understand what the above git update-index commands actually do, that is making this faster.
 
Thanks,
Sarvi
Occam’s Razor Rules

On 8/22/19, 11:13 AM, "Junio C Hamano" [off-list ref] wrote:

    "Saravanan Shanmugham (sarvi)" [off-list ref] writes:
    
    > I suspect warming the filesystem caching is in play.
    > But so is the fact that walk every tree to find untracked files as well. 
    
    Enable the untracked cache and "update-index --refresh", before
    freezing the repository + working tree state in the diskimage?
    
Keyboard shortcuts
hback out one level
jnext message in thread
kprevious message in thread
ldrill in
Escclose help / fold thread tree
?toggle this help