[GIT PULL for v6.17-rc2] add a generic yaml parser integrated with Netlink specs generation

8 messages, 6 authors, 2025-08-15 · open the first message on its own page

[GIT PULL for v6.17-rc2] add a generic yaml parser integrated with Netlink specs generation

From: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Date: 2025-08-12 09:33:37

Hi Jon/Jakub,

In case you both prefer to merge from a stable tag, please pull from:

	git://git.kernel.org/pub/scm/linux/kernel/git/mchehab/linux-docs.git docs/v6.17-1

For:

- An YAML parser Sphinx plugin, integrated with Netlink YAML doc
  parser.

The patch content is identical to my v10 submission:

	https://lore.kernel.org/linux-doc/cover.1753718185.git.mchehab+huawei@kernel.org/

The tag was rebased on the top of v6.17-rc1 to make easier for it to be
merged on both docs and netlink trees. 

No code changes since v10.

Regards,
Mauro

---

The following changes since commit 8f5ae30d69d7543eee0d70083daf4de8fe15d585:

  Linux 6.17-rc1 (2025-08-10 19:41:16 +0300)

are available in the Git repository at:

  git://git.kernel.org/pub/scm/linux/kernel/git/mchehab/linux-docs.git tags/docs/v6.17-1

for you to fetch changes up to 47459937be8031aae6aaa17ac5f60985f7c9e1bd:

  sphinx: parser_yaml.py: fix line numbers information (2025-08-12 07:47:31 +0200)

----------------------------------------------------------------
[GIT PULL for v6.17-rc2] add a generic yaml parser integrated with Netlink specs generation

----------------------------------------------------------------
Mauro Carvalho Chehab (14):
      docs: netlink: netlink-raw.rst: use :ref: instead of :doc:
      tools: ynl_gen_rst.py: Split library from command line tool
      docs: netlink: index.rst: add a netlink index file
      tools: ynl_gen_rst.py: cleanup coding style
      docs: sphinx: add a parser for yaml files for Netlink specs
      docs: use parser_yaml extension to handle Netlink specs
      docs: uapi: netlink: update netlink specs link
      tools: ynl_gen_rst.py: drop support for generating index files
      docs: netlink: remove obsolete .gitignore from unused directory
      MAINTAINERS: add netlink_yml_parser.py to linux-doc
      tools: netlink_yml_parser.py: add line numbers to parsed data
      docs: parser_yaml.py: add support for line numbers from the parser
      docs: parser_yaml.py: fix backward compatibility with old docutils
      sphinx: parser_yaml.py: fix line numbers information

 Documentation/Makefile                             |  17 -
 Documentation/conf.py                              |  20 +-
 Documentation/netlink/specs/index.rst              |  13 +
 Documentation/networking/index.rst                 |   2 +-
 Documentation/networking/netlink_spec/.gitignore   |   1 -
 Documentation/networking/netlink_spec/readme.txt   |   4 -
 Documentation/sphinx/parser_yaml.py                | 123 +++++++
 Documentation/userspace-api/netlink/index.rst      |   2 +-
 .../userspace-api/netlink/netlink-raw.rst          |   6 +-
 Documentation/userspace-api/netlink/specs.rst      |   2 +-
 MAINTAINERS                                        |   1 +
 tools/net/ynl/pyynl/lib/__init__.py                |   2 +
 tools/net/ynl/pyynl/lib/doc_generator.py           | 398 +++++++++++++++++++++
 tools/net/ynl/pyynl/ynl_gen_rst.py                 | 384 +-------------------
 14 files changed, 565 insertions(+), 410 deletions(-)
 create mode 100644 Documentation/netlink/specs/index.rst
 delete mode 100644 Documentation/networking/netlink_spec/.gitignore
 delete mode 100644 Documentation/networking/netlink_spec/readme.txt
 create mode 100755 Documentation/sphinx/parser_yaml.py
 create mode 100644 tools/net/ynl/pyynl/lib/doc_generator.py

Re: [GIT PULL for v6.17-rc2] add a generic yaml parser integrated with Netlink specs generation

From: Jakub Kicinski <kuba@kernel.org>
Date: 2025-08-12 17:43:13

On Tue, 12 Aug 2025 11:33:29 +0200 Mauro Carvalho Chehab wrote:
The patch content is identical to my v10 submission:

	https://lore.kernel.org/linux-doc/cover.1753718185.git.mchehab+huawei@kernel.org/
Thank you! We'll merge on Thu, FWIW, after we fast forward to Linus.

Re: [GIT PULL for v6.17-rc2] add a generic yaml parser integrated with Netlink specs generation

From: Jonathan Corbet <corbet@lwn.net>
Date: 2025-08-12 18:31:11

Mauro Carvalho Chehab [off-list ref] writes:
Hi Jon/Jakub,

In case you both prefer to merge from a stable tag, please pull from:

	git://git.kernel.org/pub/scm/linux/kernel/git/mchehab/linux-docs.git docs/v6.17-1

For:

- An YAML parser Sphinx plugin, integrated with Netlink YAML doc
  parser.
OK, I have done that.  I will note that it adds a warning:
Documentation/networking/netlink_spec/index.rst: WARNING: document isn't included in any toctree
...it might be nice to get that straightened out.

Thanks,

jon

Re: [GIT PULL for v6.17-rc2] add a generic yaml parser integrated with Netlink specs generation

From: Randy Dunlap <rdunlap@infradead.org>
Date: 2025-08-12 18:42:01


On 8/12/25 11:31 AM, Jonathan Corbet wrote:
Mauro Carvalho Chehab [off-list ref] writes:
quoted
Hi Jon/Jakub,

In case you both prefer to merge from a stable tag, please pull from:

	git://git.kernel.org/pub/scm/linux/kernel/git/mchehab/linux-docs.git docs/v6.17-1

For:

- An YAML parser Sphinx plugin, integrated with Netlink YAML doc
  parser.
OK, I have done that.  I will note that it adds a warning:
quoted
Documentation/networking/netlink_spec/index.rst: WARNING: document isn't included in any toctree
...it might be nice to get that straightened out.
I see it, at least in linux-next. However, its format is
"different," so that may have confused whatever printed
that message:

from Documentation/networking/index.rst:

   filter
   generic-hdlc
   generic_netlink
   netlink_spec/index
   gen_stats
   gtp
   ila


-- 
~Randy

Re: [GIT PULL for v6.17-rc2] add a generic yaml parser integrated with Netlink specs generation

From: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Date: 2025-08-12 19:24:07

On Tue, 12 Aug 2025 11:41:58 -0700
Randy Dunlap [off-list ref] wrote:
On 8/12/25 11:31 AM, Jonathan Corbet wrote:
quoted
Mauro Carvalho Chehab [off-list ref] writes:
  
quoted
Hi Jon/Jakub,

In case you both prefer to merge from a stable tag, please pull from:

	git://git.kernel.org/pub/scm/linux/kernel/git/mchehab/linux-docs.git docs/v6.17-1

For:

- An YAML parser Sphinx plugin, integrated with Netlink YAML doc
  parser.  
OK, I have done that.  I will note that it adds a warning:
  
quoted
Documentation/networking/netlink_spec/index.rst: WARNING: document isn't included in any toctree  
...it might be nice to get that straightened out.  
I see it, at least in linux-next. However, its format is
"different," so that may have confused whatever printed
that message:

from Documentation/networking/index.rst:

   filter
   generic-hdlc
   generic_netlink
   netlink_spec/index
   gen_stats
   gtp
   ila
Maybe some merge conflict/merge issue, as this was renamed:

$ git show 1ce4da3dd99e98bd4a8b396c291041080e0fe85e Documentation/networking/index.rst
commit 1ce4da3dd99e98bd4a8b396c291041080e0fe85e
Author: Mauro Carvalho Chehab [off-list ref]
Date:   Thu Jun 12 10:34:30 2025 +0200

    docs: use parser_yaml extension to handle Netlink specs
    
    Instead of manually calling ynl_gen_rst.py, use a Sphinx extension.
    This way, no .rst files would be written to the Kernel source
    directories.
    
    We are using here a toctree with :glob: property. This way, there
    is no need to touch the netlink/specs/index.rst file every time
    a new Netlink spec is added/renamed/removed.
    
    Signed-off-by: Mauro Carvalho Chehab [off-list ref]
    Reviewed-by: Donald Hunter [off-list ref]
diff --git a/Documentation/networking/index.rst b/Documentation/networking/index.rst
index ac90b82f3ce9..b7a4969e9bc9 100644
--- a/Documentation/networking/index.rst
+++ b/Documentation/networking/index.rst
@@ -57,7 +57,7 @@ Contents:
    filter
    generic-hdlc
    generic_netlink
-   netlink_spec/index
+   ../netlink/specs/index
    gen_stats
    gtp
    ila
Thanks,
Mauro

Re: [GIT PULL for v6.17-rc2] add a generic yaml parser integrated with Netlink specs generation

From: Akira Yokosawa <akiyks@gmail.com>
Date: 2025-08-13 02:49:57

[-CC: LKMM folks and list; this has nothing to do with the memory model]

Hi Jon,

On Tue, 12 Aug 2025 12:31:03 -0600, Jonathan Corbet wrote:
Mauro Carvalho Chehab [off-list ref] writes:
quoted
Hi Jon/Jakub,

In case you both prefer to merge from a stable tag, please pull from:

	git://git.kernel.org/pub/scm/linux/kernel/git/mchehab/linux-docs.git docs/v6.17-1

For:

- An YAML parser Sphinx plugin, integrated with Netlink YAML doc
  parser.
OK, I have done that.  I will note that it adds a warning:
quoted
Documentation/networking/netlink_spec/index.rst: WARNING: document isn't included in any toctree
...it might be nice to get that straightened out.
After the merge, "git status" complains:

    Untracked files:
      (use "git add <file>..." to include in what will be committed)
	Documentation/networking/netlink_spec/

So, I don't think there is anything you can do in the Git repo side ...

We need to remember to "rm -rf" the directory after crossing this merge
point.

In theory, such "rm -rf" could be added somewhere in Documentation/Makefile,
but that would not work well with write-protected shared kernel repos.

        Thanks, Akira

Re: [GIT PULL for v6.17-rc2] add a generic yaml parser integrated with Netlink specs generation

From: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Date: 2025-08-13 07:15:33

On Wed, 13 Aug 2025 11:49:50 +0900
Akira Yokosawa [off-list ref] wrote:
[-CC: LKMM folks and list; this has nothing to do with the memory model]

Hi Jon,

On Tue, 12 Aug 2025 12:31:03 -0600, Jonathan Corbet wrote:
quoted
Mauro Carvalho Chehab [off-list ref] writes:
  
quoted
Hi Jon/Jakub,

In case you both prefer to merge from a stable tag, please pull from:

	git://git.kernel.org/pub/scm/linux/kernel/git/mchehab/linux-docs.git docs/v6.17-1

For:

- An YAML parser Sphinx plugin, integrated with Netlink YAML doc
  parser.  
OK, I have done that.  I will note that it adds a warning:
  
quoted
Documentation/networking/netlink_spec/index.rst: WARNING: document isn't included in any toctree  
...it might be nice to get that straightened out.  
After the merge, "git status" complains:

    Untracked files:
      (use "git add <file>..." to include in what will be committed)
	Documentation/networking/netlink_spec/

So, I don't think there is anything you can do in the Git repo side ...

We need to remember to "rm -rf" the directory after crossing this merge
point.

In theory, such "rm -rf" could be added somewhere in Documentation/Makefile,
but that would not work well with write-protected shared kernel repos.
And this is actually what started this patch series: the original
approach of auto-generating and writing files under
Documentation/networking/netlink_spec/ is problematic: a patch
that renamed some files caused the past version of the generated
files to generate warnings.

So, yeah, with the old approach one needs to manually clean up
Documentation/networking/netlink_spec/ if this was not the first
time doc were built on a git clone instance.

Btw, before my patch series, there was the cleandocs target
was:

	YNL_INDEX:=$(srctree)/Documentation/networking/netlink_spec/index.rst
	YNL_RST_DIR:=$(srctree)/Documentation/networking/netlink_spec
	YNL_YAML_DIR:=$(srctree)/Documentation/netlink/specs
	YNL_RST_FILES_TMP := $(patsubst %.yaml,%.rst,$(wildcard $(YNL_YAML_DIR)/*.yaml))
	YNL_RST_FILES := $(patsubst $(YNL_YAML_DIR)%,$(YNL_RST_DIR)%, $(YNL_RST_FILES_TMP))

	cleandocs:
		$(Q)rm -f $(YNL_INDEX) $(YNL_RST_FILES)
		$(Q)rm -rf $(BUILDDIR)
		$(Q)$(MAKE) BUILDDIR=$(abspath $(BUILDDIR)) $(build)=Documentation/userspace-api/media clean

It means that, with the old approach, the safe way to apply
patches that touch YAML Netlink (YNL) is to do:

	make cleandocs			# remove old auto-generated patches from /Documentation/networking/netlink_spec
	<apply new series>
	make htmldocs

to ensure that the previous auto-generated files were removed.

Thanks,
Mauro

Re: [GIT PULL for v6.17-rc2] add a generic yaml parser integrated with Netlink specs generation

From: patchwork-bot+netdevbpf@kernel.org
Date: 2025-08-15 00:40:13

Hello:

This pull request was applied to netdev/net-next.git (main)
by Jakub Kicinski [off-list ref]:

On Tue, 12 Aug 2025 11:33:29 +0200 you wrote:
Hi Jon/Jakub,

In case you both prefer to merge from a stable tag, please pull from:

	git://git.kernel.org/pub/scm/linux/kernel/git/mchehab/linux-docs.git docs/v6.17-1

For:

[...]
Here is the summary with links:
  - [GIT,PULL,for,v6.17-rc2] add a generic yaml parser integrated with Netlink specs generation
    https://git.kernel.org/netdev/net-next/c/c4f72d3747ac

You are awesome, thank you!
-- 
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/patchwork/pwbot.html

Keyboard shortcuts
hback out one level
jnext message in thread
kprevious message in thread
ldrill in
Escclose help / fold thread tree
?toggle this help