Re: [PATCH v2] docs: Use make invocation's -j argument for parallelism
From: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>
Date: 2019-09-24 17:02:19
Also in:
lkml
Em Mon, 23 Sep 2019 15:40:41 -0700 Kees Cook [off-list ref] escreveu:
On Sun, Sep 22, 2019 at 02:03:31PM -0600, Jonathan Corbet wrote:quoted
On Thu, 19 Sep 2019 14:44:37 -0700 Kees Cook [off-list ref] wrote:quoted
While sphinx 1.7 and later supports "-jauto" for parallelism, this effectively ignores the "-j" flag used in the "make" invocation, which may cause confusion for build systems. Instead, extract the availableWhat sort of confusion might we expect? Or, to channel akpm, "what are the user-visible effects of this bug"?When I run "make htmldocs -j16" with a pre-1.7 sphinx, it is not parallelized.
Sphinx supports parallel builds for a while. With pre-1.7, you could do something like: make SPHINXOPTS="-j16" htmldocs Yet, on my experiences on big machines (tested here with Xeon with 40 and 64 CPU threads), parallel build doesn't actually benefit with values higher than -j5 to -j8, with pre-1.7. Sphinx 1.7 and higher seem to have improved a lot with "-jauto" (although I didn't time it comparing with -j5 or -j8 on a big server).
When I run "make htmldocs -j8" with 1.7+ sphinx, it uses all my CPUs instead of 8. :)
This should do the trick: make SPHINXOPTS="-j8" htmldocs But yeah, IMHO, the best is if it could honor the Makefile flag: make -j8 htmldocs If SPHINXOPTS doesn't contain "-j".
quoted
quoted
+ -j $(shell python3 $(srctree)/scripts/jobserver-count $(SPHINX_PARALLEL)) \This (and the shebang line in the script itself) will cause the docs build to fail on systems lacking Python 3. While we have talked about requiring Python 3 for the docs build, we have not actually taken that step yet. We probably shouldn't sneak it in here. I don't see anything in the script that should require a specific Python version, so I think it should be tweaked to be version-independent and just invoke "python".Ah, no problem. I can fix this. In a quick scan it looked like sphinx was python3, but I see now that's just my install. :)
On Fedora 30, both python2 and python3 versions are available: python2-sphinx.noarch : Python documentation generator python3-sphinx.noarch : Python documentation generator However, if one tries to install it without specifying "2" or "3", it defaults to python2 version: $ sudo dnf install python-sphinx ... Installing: python2-sphinx noarch 1:1.8.4-1.fc30 fedora 1.8 M AFAIKT, this also applies when distro upgrades takes place: upgrading a python2 sphinx from Fedora 30 to Rawhide will very likely keep the python2 version. Thanks, Mauro