Jakub Kicinski [off-list ref] writes:

On Sun, 19 Mar 2023 19:38:00 +0000 Donald Hunter wrote:

quoted

Add support for decoding nested arrays of scalars in netlink messages.

example?

OVS_VPORT_ATTR_UPCALL_PID is a C array of u32 values. I can provide that
as an example in the commit message.

quoted

Signed-off-by: Donald Hunter <donald.hunter@gmail.com>
---
 tools/net/ynl/lib/ynl.py | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/tools/net/ynl/lib/ynl.py b/tools/net/ynl/lib/ynl.py
index 32536e1f9064..077ba9e8dc98 100644
--- a/tools/net/ynl/lib/ynl.py
+++ b/tools/net/ynl/lib/ynl.py

@@ -93,6 +93,10 @@ class NlAttr:
     def as_bin(self):
         return self.raw
 
+    def as_array(self, type):
+        format, _ = self.type_formats[type]
+        return list({ x[0] for x in struct.iter_unpack(format, self.raw) })

So in terms of C this treats the payload of the attr as a packed array?
That's not what array-nest is, array-nest wraps every entry in another
nlattr:
https://docs.kernel.org/next/userspace-api/netlink/genetlink-legacy.html#array-nest

It's not a C array dumped into an attribute.

IIRC I was intending to use 'binary' for packed arrays. Still use
sub-type to carry the type, but main type should be 'binary'.

If that sounds reasonable could you document or remind me to document
this as the expected behavior? Sub-type appears completely undocumented
now :S

That sounds reasonable, yes. I will also rename the method to
'as_c_array'. I think it should just be restricted to scalar subtypes,
i.e. u16, u32, etc. Do you agree?

I will update the  documentation for this.