Mon, 11 Jul 2022 10:06:32 -0400 Jeff King [off-list ref]:
How did you measure? Are you sure that each one is allocating a
gigabyte itself, or might some of it be shared between the processes?
Git will mmap the packfiles on disk, which will count against RSS
(assuming the memory even gets faulted in). But multiple processes on
the same repository will share those read-only pages.
I ran top(1), all the git processes were competing for memory.
There was most of the time no memory shared, according to top.
This tool to process a single repository exists since a few years.
In the past I ran it on a 12cpu/64GB machine. Today it was running the
first time on a 96cpu/64GB machine. There is a slim chance the issue did
not show up because the 12 processes always had access to enough memory.
I will try your suggestions.
Thanks.
Olaf