Thread (7 messages) 7 messages, 4 authors, 2019-09-25

Re: [PATCH v2] docs: Use make invocation's -j argument for parallelism

From: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>
Date: 2019-09-24 17:02:19
Also in: lkml

Em Mon, 23 Sep 2019 15:40:41 -0700
Kees Cook [off-list ref] escreveu:
On Sun, Sep 22, 2019 at 02:03:31PM -0600, Jonathan Corbet wrote:
quoted
On Thu, 19 Sep 2019 14:44:37 -0700
Kees Cook [off-list ref] wrote:
  
quoted
While sphinx 1.7 and later supports "-jauto" for parallelism, this
effectively ignores the "-j" flag used in the "make" invocation, which
may cause confusion for build systems. Instead, extract the available  
What sort of confusion might we expect?  Or, to channel akpm, "what are the
user-visible effects of this bug"?  
When I run "make htmldocs -j16" with a pre-1.7 sphinx, it is not
parallelized.
Sphinx supports parallel builds for a while. With pre-1.7, you could do
something like:

	make SPHINXOPTS="-j16" htmldocs

Yet, on my experiences on big machines (tested here with Xeon with 40 and 64
CPU threads), parallel build doesn't actually benefit with values higher than
-j5 to -j8, with pre-1.7.

Sphinx 1.7 and higher seem to have improved a lot with "-jauto"
(although I didn't time it comparing with -j5 or -j8 on a big server).
When I run "make htmldocs -j8" with 1.7+ sphinx, it uses
all my CPUs instead of 8. :)
This should do the trick:

	make SPHINXOPTS="-j8" htmldocs

But yeah, IMHO, the best is if it could honor the Makefile flag:

	make -j8 htmldocs

If SPHINXOPTS doesn't contain "-j".
quoted
quoted
+	-j $(shell python3 $(srctree)/scripts/jobserver-count $(SPHINX_PARALLEL)) \  
This (and the shebang line in the script itself) will cause the docs build
to fail on systems lacking Python 3.  While we have talked about requiring
Python 3 for the docs build, we have not actually taken that step yet.  We
probably shouldn't sneak it in here.  I don't see anything in the script
that should require a specific Python version, so I think it should be
tweaked to be version-independent and just invoke "python".  
Ah, no problem. I can fix this. In a quick scan it looked like sphinx
was python3, but I see now that's just my install. :)
On Fedora 30, both python2 and python3 versions are available:

	python2-sphinx.noarch : Python documentation generator
	python3-sphinx.noarch : Python documentation generator

However, if one tries to install it without specifying "2" or "3", it
defaults to python2 version:

	$ sudo dnf install python-sphinx
	...
	Installing:
	 python2-sphinx                  noarch 1:1.8.4-1.fc30            fedora  1.8 M

AFAIKT, this also applies when distro upgrades takes place: upgrading
a python2 sphinx from Fedora 30 to Rawhide will very likely keep
the python2 version.

Thanks,
Mauro
Keyboard shortcuts
hback out one level
jnext message in thread
kprevious message in thread
ldrill in
Escclose help / fold thread tree
?toggle this help