Parallel make problem?

2 messages, 2 authors, 2016-06-15 · open the first message on its own page

Parallel make problem?

From: Bill Lear <hidden>
Date: 2016-06-15 22:44:10

I just downloaded 1.5.4 and am building with gnu make, passing the '-j5'
flag to invoke multiple jobs.  I get the following:

% make -j5 prefix=/opt/git-1.5.4 all doc info
[...]
make[1]: Entering directory `/home/blear/build/git-1.5.4/Documentation'
rm -f doc.dep+ doc.dep
/usr/bin/perl ./build-docdep.perl >doc.dep+
Writing perl.mak for Git
make -C ../ GIT-VERSION-FILE
mv doc.dep+ doc.dep
make -C ../ GIT-VERSION-FILE
mv doc.dep+ doc.dep
mv: cannot stat `doc.dep+': No such file or directory
[...]

it does not seem to fail from there, but I am concerned that this is
a problem and thought it worth reporting.


Bill

Re: Parallel make problem?

From: Jeff King <hidden>
Date: 2016-06-15 22:44:10

On Mon, Feb 04, 2008 at 02:15:49PM -0600, Bill Lear wrote:
I just downloaded 1.5.4 and am building with gnu make, passing the '-j5'
flag to invoke multiple jobs.  I get the following:

% make -j5 prefix=/opt/git-1.5.4 all doc info
I'm not sure this will ever be safe because of the recursive make
invocations ('make doc' invokes '$(MAKE) -C Documentation all', which in
turn does an '-include ../GIT-VERSION-FILE' which triggers a 'make -C
../ GIT-VERSION-FILE'; and that's just one of the possible problems). So
you actually have three different makes running, and nothing to prevent
them from clobbering each other if they hit the same file.
[...]
make[1]: Entering directory `/home/blear/build/git-1.5.4/Documentation'
rm -f doc.dep+ doc.dep
/usr/bin/perl ./build-docdep.perl >doc.dep+
Writing perl.mak for Git
make -C ../ GIT-VERSION-FILE
mv doc.dep+ doc.dep
make -C ../ GIT-VERSION-FILE
mv doc.dep+ doc.dep
mv: cannot stat `doc.dep+': No such file or directory
[...]
Actually, I think the particular one you're seeing is caused by the two
documentation makes going at once (both 'doc' and 'info' spawn a
recursive make in Documentation).

The solution is to avoid recursive make invocations, which is
potentially a lot of work and not likely to happen.

The workaround, as you have may have figured out already, is:

  make -j5 && make -j5 doc && make -j5 info

which should be equivalently fast, since there's plenty to parallelize
within each target.

-Peff
Keyboard shortcuts
hback out one level
jnext message in thread
kprevious message in thread
ldrill in
Escclose help / fold thread tree
?toggle this help