On Feb 23, 2023, at 10:19 AM, J. Bruce Fields [off-list ref] wrote:

On Wed, Feb 22, 2023 at 01:20:43PM -0500, Jeff Layton wrote:

quoted

The READMEs for v4.0 and v4.1 are inconsistent here. For v4.0, the "all"
flag is supposed to run all of the "standard" tests. For v4.1 "all" is
documented to run all of the tests, but it actually doesn't since not
every tests has "all" in its FLAGS: field.

I move that we change this. If I say that I want to run "all", then I
really do want to run _all_ of the tests. Ensure that every test has the
"all" flag set.

In some (all?) cases where the "all" flag was left off, it was
intentional.

We try not to flag spec-compliant servers as failing, because people are
sometimes a little careless about "fixing" failures that in their
particular case really shouldn't be fixed.  But sometimes it's still
useful to have a test that goes somewhat beyond the spec.

There might be other ways to handle that kind of test, but it would need
some more thought.

We could use a different name for "all" since it doesn't actually
run /all/ tests. Jeff suggested "standard", which seems sensible.

Also, we could add test categories specifically for particular server
implementations, if that's interesting to folks.

--b.

quoted

Signed-off-by: Jeff Layton <jlayton@kernel.org>
---
nfs4.1/testmod.py | 2 ++
1 file changed, 2 insertions(+)

If this is unacceptable, then an alternative could be to add a new
(similarly special-cased) "everything" flag.

diff --git a/nfs4.1/testmod.py b/nfs4.1/testmod.py
index 11e759d673fd..7b3bac543084 100644
--- a/nfs4.1/testmod.py
+++ b/nfs4.1/testmod.py

@@ -386,6 +386,8 @@ def createtests(testdir):
    for t in tests:

##         if not t.flags_list:
##             raise RuntimeError("%s has no flags" % t.fullname)
+        if "all" not in t.flags_list:
+            t.flags_list.append("all")
        for f in t.flags_list:
            if f not in flag_dict:
                flag_dict[f] = bit
-- 
2.39.2

--
Chuck Lever