RE: [pynfs RFC PATCH] nfs4.0/testserver.py: don't return an error when tests fail
From: Frank Filz <hidden>
Date: 2023-02-23 16:22:58
From: Jeff Layton [mailto:jlayton@kernel.org]
This script was originally changed in eb3ba0b60055 ("Have testserver.pyhave
non-zero exit code if any tests fail"), but the same change wasn't made to
the
4.1 testserver.py script. There also wasn't much explanation for it, and it makes it difficult to
tell
whether the test harness itself failed, or whether there was a failure in
a
requested test. Stop the 4.0 testserver.py from exiting with an error code when a test
fails, so
that a successful return means only that the test harness itself worked,
not that
every requested test passed. Cc: Frank Filz <redacted> Signed-off-by: Jeff Layton <jlayton@kernel.org> --- nfs4.0/testserver.py | 2 -- 1 file changed, 2 deletions(-) I'm not sure about this one. I've worked around this in kdevops for now,
but it
would really be preferable if it worked this way, imo. If this isn't
acceptable,
maybe we can add a new option that enables this behavior? Frank, what was the original rationale for eb3ba0b60055 ?
We needed a way for CI to easily detect failure of pynfs. I'm not sure how helpful it is since Ganesha does fail some tests... It might be helpful to have some helpers for CI to use, or an option that causes pynfs to report in a way that's much easier for CI to determine if pynfs succeeded or not. Hmm, one thing that would help is to be able to flag a set of tests that should not constitute a CI failure (known errors) but we want to keep running them because of what they exercise, or to more readily detect that they have been fixed.
quoted hunk ↗ jump to hunk
diff --git a/nfs4.0/testserver.py b/nfs4.0/testserver.py indexf2c41568e5c7..4f4286daa657 100755--- a/nfs4.0/testserver.py +++ b/nfs4.0/testserver.py@@ -387,8 +387,6 @@ def main(): if nfail < 0: sys.exit(3) - if nfail > 0: - sys.exit(2) if __name__ == "__main__": main() --2.39.2