mmap failure in master 1aa69c73577df21f5e37e47cc40cf44fc049121e

Hi,

when calling `git diff --quiet` on the gcc git repository
my system fails reproducible with a failed mmap. I have
noticed this possible bug with git in debian unstable and
then build git from source. But I am not able to get a core
or backtrace when gdb is attached.

The command I have used to build git is:

make clean; CFLAGS="-O0 -g -fsanitize=address" DEBUG=1 make -j24


Maybe you can give me some advice how to further debug this.

Jan

--------------------------------------------------------------

The output below was created with the system git not
the self-compiled version (that behaves in the same way).

» ulimit -a; git --version; git diff --quiet; git remote -v
core file size          (blocks, -c) unlimited
data seg size           (kbytes, -d) unlimited
scheduling priority             (-e) 0
file size               (blocks, -f) unlimited
pending signals                 (-i) 514868
max locked memory       (kbytes, -l) unlimited
max memory size         (kbytes, -m) unlimited
open files                      (-n) 20000
pipe size            (512 bytes, -p) 8
POSIX message queues     (bytes, -q) 819200
real-time priority              (-r) 0
stack size              (kbytes, -s) 8192
cpu time               (seconds, -t) unlimited
max user processes              (-u) 2000
virtual memory          (kbytes, -v) unlimited
file locks                      (-x) unlimited
git version 2.27.0.rc2
fatal: mmap failed: Cannot allocate memory
origin  git://gcc.gnu.org/git/gcc.git (fetch)
origin  git://gcc.gnu.org/git/gcc.git (push)

--------------------------------------------------------------

lstat("gcc/testsuite/gdc.test/fail_compilation/b19717a.d", {st_mode=S_IFREG|0644, st_size=494, ...}) = 0
lstat("gcc/testsuite/gdc.test/fail_compilation/b19717a.d", {st_mode=S_IFREG|0644, st_size=494, ...}) = 0
lstat("gcc/testsuite/gdc.test/fail_compilation/b19717a.d", {st_mode=S_IFREG|0644, st_size=494, ...}) = 0
lstat("gcc/testsuite/gdc.test/fail_compilation/b19717a.d", {st_mode=S_IFREG|0644, st_size=494, ...}) = 0
openat(AT_FDCWD, "gcc/testsuite/gdc.test/fail_compilation/b19717a.d", O_RDONLY) = 3
mmap(NULL, 494, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7f4a81a58000
close(3)                                = 0
lstat("gcc/testsuite/gdc.test/fail_compilation/b3841.d", {st_mode=S_IFREG|0644, st_size=2643, ...}) = 0
lstat("gcc/testsuite/gdc.test/fail_compilation/b3841.d", {st_mode=S_IFREG|0644, st_size=2643, ...}) = 0
lstat("gcc/testsuite/gdc.test/fail_compilation/b3841.d", {st_mode=S_IFREG|0644, st_size=2643, ...}) = 0
lstat("gcc/testsuite/gdc.test/fail_compilation/b3841.d", {st_mode=S_IFREG|0644, st_size=2643, ...}) = 0
lstat("gcc/testsuite/gdc.test/fail_compilation/b3841.d", {st_mode=S_IFREG|0644, st_size=2643, ...}) = 0
openat(AT_FDCWD, "gcc/testsuite/gdc.test/fail_compilation/b3841.d", O_RDONLY) = 3
mmap(NULL, 2643, PROT_READ, MAP_PRIVATE, 3, 0) = -1 ENOMEM (Cannot allocate memory)
openat(AT_FDCWD, "/usr/share/locale/en_US/LC_MESSAGES/libc.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/usr/share/locale/en/LC_MESSAGES/libc.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
write(2, "fatal: mmap failed: Cannot alloc"..., 43fatal: mmap failed: Cannot allocate memory
) = 43
exit_group(128)                         = ?

+++ exited with 128 +++

--------------------------------------------------------------

» uname -a; free -m
Linux edward 5.6.0-2-amd64 #1 SMP Debian 5.6.14-1 (2020-05-23) x86_64 GNU/Linux
               total        used        free      shared  buff/cache   available
Mem:         128842        6279       86976        1084       35586      120439
Swap:        122067           0      122067