2025-11-03, 16:01:33 -0800, Jakub Kicinski wrote:

On Mon, 3 Nov 2025 11:13:08 +0100 Sabrina Dubroca wrote:

quoted

2025-10-30, 17:02:17 -0700, Jakub Kicinski wrote:

quoted

On Fri, 31 Oct 2025 00:13:59 +0100 Sabrina Dubroca wrote:  

quoted

I guess it's improving the situation, but I've got a system with an
ethtool that accepts the --json argument, but silently ignores it for
 -k (ie `ethtool --json -k $DEV` succeeds but doesn't produce a json
output), which will still cause the test to fail later.  

And --json was added to -k in Jan 2022, that's pretty long ago.
I'm not sure we need this aspect of the patch at all..  

Ok.  Then maybe a silly idea: for the tests that currently have some
form of "$TOOL is too old" check, do we want to remove those after a
while? If so, how long after the feature was introduced in $TOOL?

Or should we leave them, but not accept new checks to exclude
really-old versions of tools?  Do we need to document the cut-off ("we
don't support tool versions older than 2 years for networking
selftests" [or similar]) somewhere in Documentation/ ?

FWIW my current thinking is to prioritize test development and kernel
needs over the ability to run ksft on random old set of tools and have
clean skips. IOW avoid complicating writing tests by making the author
also responsible for testing versions of all tools.

I see. I liked Andrew's idea ("embed the date the requirement was
added into the test"), but it goes completely in the opposite
direction.

Figuring out why exactly a test failed in case of an old tool
(unexpected output passed to some pipe/parsing, exit with a non-zero
code, maybe other issues) is not always obvious. So without version
checks on the tools, I think we have to assume that the test requires
the latest version of all tools it calls (or at least a very recent
one). Which I guess is reasonable for upstream kernel development.

The list of tools which need to be updated or installed for all
networking tests to pass is rather long. My uneducated guess
is all these one off SKIP patches don't amount to much. Here for
example author is fixing one test, I'm pretty sure that far more
tests depend on -k --json.

A quick grep found only a few more (in python scripts under
drivers/net) for -k. But (also from a quick grep) many tests seem to
use jq without checking that the command is present.

So I guess you would lean toward not accepting any such patch, not
requiring new tests to have SKIP checks, but leaving any existing
checks in? (and I suspect removing all the existing ones wouldn't
actually reduce the flow of "add check for too old $tool" patches, so
it probably doesn't make sense to do that)

-- 
Sabrina