On Wed, Jan 07, 2026 at 10:58:36PM +0000, Martin Fick wrote:

quoted

From: Patrick Steinhardt <redacted> Sent: Wednesday, January 7, 2026 4:42 AM

On Tue, Jan 06, 2026 at 11:02:19PM +0000, Martin Fick wrote:

quoted

quoted

From: Patrick Steinhardt <redacted> Sent: Monday, January 5, 2026 11:53 PM

quoted

Did you try using perf(1) to profile the process and generate a flame
graph from it? That should likely make it immediately obvious where Git
is spending all of its time.

I will pursue this. Unfortunately this might be difficult on this
particular server.

True, on the server side this can be a bit tricky.

I ran perf, and got a flame graph, I am not sure what the best way to share that
is, but I will try to summarize what looked important:

About one third of the time is in this section:

libc-2.17.so 32.5%
 _memcmp_sse4_1 29.8%
 page_fault 7.23%
 ...

I am not really sure what that is doing?


Another third is doing:

unpack_object_header_buffer 30%
 page_fault 26.9%
 ...
 nfs_read_page 10%

Which could very well be looking at the headers of objects to see if they are 
tags needing to be peeled?

Both of these are lacking some information to be able to tell. Are you
by any chance able to share the whole flame graph? That'd make this a
bit easier to figure out.

And the remaining third was a bit all over the place with small sections,
the largest two of those sections being:

packed_refs_store_create ~8.7%
 unknown 4.4%
 memchr 4.4%
 page_fault 4.4%

We spend ~9% of time in `packed_refs_store_create()`? That looks
seriously broken to me, the function shouldn't even do much.

nth_packed_object_offset 7%
 page_fault 3.2%

This was way less informative (to me) then I would have hoped. :( Maybe
this means more to you? 

It does look like a lot of page_faults, likely due to the use of mmap?

Certainly looks like the page faults are to blame here overall. It's
still surprising to me it's _that_ slow. Quoting the other mail you sent:

On Wed, Jan 07, 2026 at 05:05:53PM +0000, Martin Fick wrote:

rw,intr,retrans=10,timeo=600,hard,rsize=32768,wsize=32768,tcp,noacl,_netdev

I know that back when we still supported NFS we recommended to use an
rsize and wsize of 1MB to reduce the round trip times.

Patrick