Bizarro race conditions in the Git Makefile

   "Doctor, when I poke my eye like this it hurts"
   -"So don't do that then"

If you run the Git Makefile (with GNU Make 3.81) in parallel for long
enough you'll get some interesting breakages. Those interested in
poking their eyes can try:

    while nice -n 30 make -j 15 clean all CFLAGS=-O0 CC=gcc; do 1; done

These suggest that we have some bugs in our Makefile dependencies, but
since I haven't found what they are I'm just going to post the
symptoms.

The first example is the generation of the perl.mak from the
top-level. It seems that if you run make with -j $n the perl.mak will
get generated $n times:

        LINK git-fast-import
    Writing perl.mak for Git
    Writing perl.mak for Git
    Writing perl.mak for Git
    Writing perl.mak for Git
        LINK git-imap-send
    Writing perl.mak for Git
        LINK git-shell
        LINK git-show-index
        LINK git-upload-pack
    Writing perl.mak for Git
    [...]

This can lead to an error where they trip over each other:

    Writing perl.mak for Git
    Writing perl.mak for Git
        AR libgit.a
        AR xdiff/lib.a
        LINK git-fast-import
        LINK git-imap-send
    Writing perl.mak for Git
    Writing perl.mak for Git
    rename MakeMaker.tmp => perl.mak: No such file or directory at
/usr/share/perl/5.10/ExtUtils/MakeMaker.pm line 1004.
    make[2]: perl.mak: No such file or directory
    make[2]: *** No rule to make target `perl.mak'.  Stop.
    make[1]: *** [instlibdir] Error 2
    make: *** [git-svn] Error 2
    make: *** Waiting for unfinished jobs....
    make[2]: perl.mak: No such file or directory
    make[2]: *** No rule to make target `perl.mak'.  Stop.
    make[1]: *** [instlibdir] Error 2
    make: *** [git-relink] Error 2
    Writing perl.mak for Git
    Writing perl.mak for Git
    Writing perl.mak for Git
    Writing perl.mak for Git
    Writing perl.mak for Git

Here one make job seemingly had its MakeMaker.tmp file usurped by
another job.

Here's another one, it seems that the builtin/help.o dependency on
common-cmds.h is broken:

        CC builtin/help.o
        [...]
        CC builtin/ls-remote.o
    builtin/help.c:9:25: error: common-cmds.h: No such file or directory
    builtin/help.c: In function ‘list_common_cmds_help’:
    builtin/help.c:278: error: ‘common_cmds’ undeclared (first use in
this function)
    builtin/help.c:278: error: (Each undeclared identifier is reported only once
    builtin/help.c:278: error: for each function it appears in.)

Which is odd, because this seems OK:

    builtin/help.o: common-cmds.h
    builtin/help.s builtin/help.o: EXTRA_CPPFLAGS = \
        '-DGIT_HTML_PATH="$(htmldir_SQ)"' \
        '-DGIT_MAN_PATH="$(mandir_SQ)"' \
        '-DGIT_INFO_PATH="$(infodir_SQ)"'

A vaguely recall having run into some other make error, but I can't
reproduce it now. I'll keep poking my eye and reporting results to the
list.

Ævar Arnfjörð Bjarmason wrote:

Those interested in
poking their eyes can try:

    while nice -n 30 make -j 15 clean all CFLAGS=-O0 CC=gcc; do 1; done

You are asking make to simultaneously build and unbuild everything.
It does not really surprise me that it gets confused.

Does

 while
	nice -n 30 make -j 15 clean CFLAGS=-O0 CC=gcc &&
	nice -n 30 make -j 15 all CFLAGS=-O0 CC=gcc
 do
	:
 done

behave better?

    Writing perl.mak for Git
    Writing perl.mak for Git
    rename MakeMaker.tmp => perl.mak: No such file or directory at

Maybe something like the following is needed to protect against
interrupted builds (not the problem you reported, but this reminds
me).

diff --git a/perl/Makefile b/perl/Makefile
index 4ab21d6..eef7ee5 100644
--- a/perl/Makefile
+++ b/perl/Makefile

@@ -22,20 +22,21 @@ clean:
 ifdef NO_PERL_MAKEMAKER
 instdir_SQ = $(subst ','\'',$(prefix)/lib)
 $(makfile): ../GIT-CFLAGS Makefile
-	echo all: private-Error.pm Git.pm > $@
-	echo '	mkdir -p blib/lib' >> $@
-	echo '	$(RM) blib/lib/Git.pm; cp Git.pm blib/lib/' >> $@
-	echo '	$(RM) blib/lib/Error.pm' >> $@
+	echo all: private-Error.pm Git.pm > $@+
+	echo '	mkdir -p blib/lib' >> $@+
+	echo '	$(RM) blib/lib/Git.pm; cp Git.pm blib/lib/' >> $@+
+	echo '	$(RM) blib/lib/Error.pm' >> $@+
 	'$(PERL_PATH_SQ)' -MError -e 'exit($$Error::VERSION < 0.15009)' || \
-	echo '	cp private-Error.pm blib/lib/Error.pm' >> $@
-	echo install: >> $@
-	echo '	mkdir -p "$$(DESTDIR)$(instdir_SQ)"' >> $@
-	echo '	$(RM) "$$(DESTDIR)$(instdir_SQ)/Git.pm"; cp Git.pm "$$(DESTDIR)$(instdir_SQ)"' >> $@
-	echo '	$(RM) "$$(DESTDIR)$(instdir_SQ)/Error.pm"' >> $@
+	echo '	cp private-Error.pm blib/lib/Error.pm' >> $@+
+	echo install: >> $@+
+	echo '	mkdir -p "$$(DESTDIR)$(instdir_SQ)"' >> $@+
+	echo '	$(RM) "$$(DESTDIR)$(instdir_SQ)/Git.pm"; cp Git.pm "$$(DESTDIR)$(instdir_SQ)"' >> $@+
+	echo '	$(RM) "$$(DESTDIR)$(instdir_SQ)/Error.pm"' >> $@+
 	'$(PERL_PATH_SQ)' -MError -e 'exit($$Error::VERSION < 0.15009)' || \
-	echo '	cp private-Error.pm "$$(DESTDIR)$(instdir_SQ)/Error.pm"' >> $@
-	echo instlibdir: >> $@
-	echo '	echo $(instdir_SQ)' >> $@
+	echo '	cp private-Error.pm "$$(DESTDIR)$(instdir_SQ)/Error.pm"' >> $@+
+	echo instlibdir: >> $@+
+	echo '	echo $(instdir_SQ)' >> $@+
+	mv $@+ $@
 else
 $(makfile): Makefile.PL ../GIT-CFLAGS
 	$(PERL_PATH) $< PREFIX='$(prefix_SQ)'

-- 

On Fri, Aug 6, 2010 at 21:14, Jonathan Nieder [off-list ref] wrote:

Ęvar Arnfjörš Bjarmason wrote:

quoted

Those interested in
poking their eyes can try:

    while nice -n 30 make -j 15 clean all CFLAGS=-O0 CC=gcc; do 1; done

You are asking make to simultaneously build and unbuild everything.
It does not really surprise me that it gets confused.

Does

 while
       nice -n 30 make -j 15 clean CFLAGS=-O0 CC=gcc &&
       nice -n 30 make -j 15 all CFLAGS=-O0 CC=gcc
 do
       :
 done

behave better?

Yes that works. I thought that -j $n would mean that make would use $n
jobs to complete the first target, then move onto the next. Not
execute them all in paralell.

Thanks, and I have no idea about those Makefile/Perl changes.