Thread (9 messages) 9 messages, 5 authors, 2023-02-26

RE: [pynfs RFC PATCH] nfs4.0/testserver.py: don't return an error when tests fail

From: Frank Filz <hidden>
Date: 2023-02-23 16:22:58

From: Jeff Layton [mailto:jlayton@kernel.org]
 
This script was originally changed in eb3ba0b60055 ("Have testserver.py
have
non-zero exit code if any tests fail"), but the same change wasn't made to
the
4.1 testserver.py script.

There also wasn't much explanation for it, and it makes it difficult to
tell
whether the test harness itself failed, or whether there was a failure in
a
requested test.

Stop the 4.0 testserver.py from exiting with an error code when a test
fails, so
that a successful return means only that the test harness itself worked,
not that
every requested test passed.

Cc: Frank Filz <redacted>
Signed-off-by: Jeff Layton <jlayton@kernel.org>
---
 nfs4.0/testserver.py | 2 --
 1 file changed, 2 deletions(-)

I'm not sure about this one. I've worked around this in kdevops for now,
but it
would really be preferable if it worked this way, imo. If this isn't
acceptable,
maybe we can add a new option that enables this behavior?

Frank, what was the original rationale for eb3ba0b60055 ?
We needed a way for CI to easily detect failure of pynfs. I'm not sure how
helpful it is since Ganesha does fail some tests...

It might be helpful to have some helpers for CI to use, or an option that
causes pynfs to report in a way that's much easier for CI to determine if
pynfs succeeded or not.

Hmm, one thing that would help is to be able to flag a set of tests that
should not constitute a CI failure (known errors) but we want to keep
running them because of what they exercise, or to more readily detect that
they have been fixed.
quoted hunk ↗ jump to hunk
diff --git a/nfs4.0/testserver.py b/nfs4.0/testserver.py index
f2c41568e5c7..4f4286daa657 100755
--- a/nfs4.0/testserver.py
+++ b/nfs4.0/testserver.py
@@ -387,8 +387,6 @@ def main():

     if nfail < 0:
         sys.exit(3)
-    if nfail > 0:
-        sys.exit(2)

 if __name__ == "__main__":
     main()
--
2.39.2
  
Keyboard shortcuts
hback out one level
jnext message in thread
kprevious message in thread
ldrill in
Escclose help / fold thread tree
?toggle this help