Em Thu, 19 Jun 2025 00:46:15 +0900
Akira Yokosawa [off-list ref] escreveu:

Hi Mauro,

On 2025/06/18 20:46, Mauro Carvalho Chehab wrote:

quoted

As discussed at:
   https://lore.kernel.org/all/20250610101331.62ba466f@foz.lan/ (local)

changeset f061c9f7d058 ("Documentation: Document each netlink family")
added a logic which generates *.rst files inside $(srctree). This is bad
when O=<BUILDDIR> is used.

A recent change renamed the yaml files used by Netlink, revealing a bad
side effect: as "make cleandocs" don't clean the produced files and symbols
appear duplicated for people that don't build the kernel from scratch.

This series adds an yaml parser extension and uses an index file with glob for
*. We opted to write such extension in a way that no actual yaml conversion
code is inside it. This makes it flexible enough to handle other types of yaml
files in the future. The actual yaml conversion logic were placed at 
netlink_yml_parser.py. 

As requested by YNL maintainers, this version has netlink_yml_parser.py
inside tools/net/ynl/pyynl/ directory. I don't like mixing libraries with
binaries, nor to have Python libraries spread all over the Kernel. IMO,
the best is to put all of them on a common place (scripts/lib, python/lib,
lib/python, ...) but, as this can be solved later, for now let's keep it this
way.

---

v6:
- YNL doc parser is now at tools/net/ynl/pyynl/lib/doc_generator.py;
- two patches got merged;
- added instructions to test docs with Sphinx 3.4.3 (minimal supported
  version);
- minor fixes.  

Quick tests against Sphinx 3.4.3 using container images based on
debian:bullseye and almalinux:9, both of which have 3.4.3 as their distro
packages, emits a *bunch* of warnings like the following:

/<srcdir>/Documentation/netlink/specs/conntrack.yaml:: WARNING: YAML parsing error: AttributeError("'Values' object has no attribute 'tab_width'")
/<srcdir>/Documentation/netlink/specs/devlink.yaml:: WARNING: YAML parsing error: AttributeError("'Values' object has no attribute 'tab_width'")
/<srcdir>/Documentation/netlink/specs/dpll.yaml:: WARNING: YAML parsing error: AttributeError("'Values' object has no attribute 'tab_width'")
/<srcdir>/Documentation/netlink/specs/ethtool.yaml:: WARNING: YAML parsing error: AttributeError("'Values' object has no attribute 'tab_width'")
/<srcdir>/Documentation/netlink/specs/fou.yaml:: WARNING: YAML parsing error: AttributeError("'Values' object has no attribute 'tab_width'")
[...]

I suspect there should be a minimal required minimal version of PyYAML.

Likely yes. From my side, I didn't change anything related to PyYAML, 
except by adding a loader at the latest patch to add line numbers.

The above warnings don't seem related. So, probably this was already
an issue.

Funny enough, I did, on my venv:

	$ pip install PyYAML==5.1
	$ tools/net/ynl/pyynl/ynl_gen_rst.py -i Documentation/netlink/specs/dpll.yaml -o Documentation/output/netlink/specs/dpll.rst -v
	...
	$ make clean; make SPHINXDIRS="netlink/specs" htmldocs
	...

but didn't get any issue (I have a later version installed outside
venv - not sure it it will do the right thing).

That's what I have at venv:

----------------------------- ---------
Package                       Version
----------------------------- ---------
alabaster                     0.7.13
babel                         2.17.0
certifi                       2025.6.15
charset-normalizer            3.4.2
docutils                      0.17.1
idna                          3.10
imagesize                     1.4.1
Jinja2                        2.8.1
MarkupSafe                    1.1.1
packaging                     25.0
pip                           25.1.1
Pygments                      2.19.1
PyYAML                        5.1
requests                      2.32.4
setuptools                    80.1.0
snowballstemmer               3.0.1
Sphinx                        3.4.3
sphinxcontrib-applehelp       1.0.4
sphinxcontrib-devhelp         1.0.2
sphinxcontrib-htmlhelp        2.0.1
sphinxcontrib-jsmath          1.0.1
sphinxcontrib-qthelp          1.0.3
sphinxcontrib-serializinghtml 1.1.5
urllib3                       2.4.0
----------------------------- ---------

"pip freeze" based on almalinux:9 says:

    PyYAML==5.4.1

"pip freeze" based on debian:bullseye says:

    PyYAML==5.3.1

What is the minimal required version here?

Breno, what's the minimal version? Please update requirements.txt
to ensure that, and modify ./scripts/sphinx-pre-install to
check for it.

And if users of those old distros need to manually upgrade PyYAML,
why don't you suggest them to upgrade Sphinx as well?

The criteria we used to define minimal version for python/sphinx
was having them released at the end of 2020/beginning 2021. So,
up to ~4 years old. We also double-checked latest LTS versions
from major distros.

With that, PyYAML 5.4.1 met the ~4 years old, and so 5.3.1 and
5.1.

funny enough:

	$ git grep tab_width Documentation/netlink/

doesn't return anything. Yet, tab_width is used by sphinx
extensions. The in-kernel ones do it the right way using
get:

	tab_width = self.options.get('tab-width',
				     self.state.document.settings.tab_width)

But perhaps some other extension you might have installed on your
environment has issues, or maybe Documentation/sphinx/parser_yaml.py
need to expand tabs with certain versions of docutils.

Please compare the versions that you're using on your test
environment with the ones I used here.

Regards,
Mauro