This patch v2 series: (revised to address v1 comments)
Add kselftest framework for tests to use. This is a light
weight framework provides a set of interfaces to report test
results and test statistics on number of tests passed and failed.
Several tests are changed to use the framework to report results.
Shuah Khan (7):
selftests: add kselftest framework for uniform test reporting
selftests/breakpoints: change test to use ksft framework
selftests/ipc: change test to use ksft framework
selftests/kcmp: change test to use ksft framework
selftests/mount: change test to use ksft framework
selftests/ptrace: change test to use ksft framework
selftests/timers: change test to use ksft framework
.../selftests/breakpoints/breakpoint_test.c | 10 +++---
tools/testing/selftests/ipc/msgque.c | 26 +++++++-------
tools/testing/selftests/kcmp/kcmp_test.c | 27 ++++++++++----
tools/testing/selftests/kselftest.h | 41 ++++++++++++++++++++++
.../selftests/mount/unprivileged-remount-test.c | 8 +++--
tools/testing/selftests/ptrace/peeksiginfo.c | 14 ++++----
tools/testing/selftests/timers/posix_timers.c | 14 ++++----
7 files changed, 101 insertions(+), 39 deletions(-)
create mode 100644 tools/testing/selftests/kselftest.h
--
1.9.1
Add kselftest framework for tests to use. This is a light
weight framework provides a set of interfaces to report test
results. Tests can use these interfaces to report pass, and
fail cases as well as when failure is due to configuration
problems such as missing modules, or when a test that is should
fail, fails as expected, and a test that should fail, passes.
The framework uses POSIX standard return codes for reporting
results to address the needs of users that want to run the kernel
selftests from their user-space test suites and want to know why a
test failed. In addition, the framework includes interfaces to use
to report test statistics on number of tests passed and failed.
Signed-off-by: Shuah Khan <redacted>
---
tools/testing/selftests/kselftest.h | 41 +++++++++++++++++++++++++++++++++++++
1 file changed, 41 insertions(+)
create mode 100644 tools/testing/selftests/kselftest.h
@@ -42,7 +44,7 @@ static void set_breakpoint_addr(void *addr, int n)offsetof(structuser,u_debugreg[n]),addr);if(ret){perror("Can't set breakpoint addr\n");-exit(-1);+ksft_exit_fail();}}
@@ -105,7 +107,7 @@ static void toggle_breakpoint(int n, int type, int len,offsetof(structuser,u_debugreg[7]),dr7);if(ret){perror("Can't set dr7");-exit(-1);+ksft_exit_fail();}}
@@ -194,16 +196,16 @@ int main(int argc, char **argv)printf("based timers if other threads run on the CPU...\n");if(check_itimer(ITIMER_VIRTUAL)<0)-return-1;+returnksft_exit_fail();if(check_itimer(ITIMER_PROF)<0)-return-1;+returnksft_exit_fail();if(check_itimer(ITIMER_REAL)<0)-return-1;+returnksft_exit_fail();if(check_timer_create(CLOCK_THREAD_CPUTIME_ID)<0)-return-1;+returnksft_exit_fail();/**It'sunfortunatelyhardtoreliablytestatimerexpiration
@@ -215,7 +217,7 @@ int main(int argc, char **argv)*findabettersolution.*/if(check_timer_create(CLOCK_PROCESS_CPUTIME_ID)<0)-return-1;+returnksft_exit_fail();-return0;+returnksft_exit_pass();}
Change mount test to use kselftest framework to report
test results.
Signed-off-by: Shuah Khan <redacted>
---
tools/testing/selftests/mount/unprivileged-remount-test.c | 8 +++++---
1 file changed, 5 insertions(+), 3 deletions(-)
@@ -176,7 +178,7 @@ bool test_unpriv_remount(int mount_flags, int remount_flags, int invalid_flags)die("remount of /tmp with invalid flags ""succeeded unexpectedly\n");}-exit(EXIT_SUCCESS);+returnksft_exit_pass();}staticbooltest_unpriv_remount_simple(intmount_flags)
Change kcmp test to use kselftest framework to report
test results and test statistics.
Signed-off-by: Shuah Khan <redacted>
---
tools/testing/selftests/kcmp/kcmp_test.c | 27 ++++++++++++++++++++-------
1 file changed, 20 insertions(+), 7 deletions(-)
@@ -50,7 +52,7 @@ int main(int argc, char **argv)fd2=open(kpath,O_RDWR,0644);if(fd2<0){perror("Can't open file");-exit(1);+ksft_exit_fail();}/* An example of output and arguments */
@@ -74,23 +76,34 @@ int main(int argc, char **argv)if(ret){printf("FAIL: 0 expected but %d returned (%s)\n",ret,strerror(errno));+ksft_inc_fail_cnt();ret=-1;-}else+}else{printf("PASS: 0 returned as expected\n");+ksft_inc_pass_cnt();+}/* Compare with self */ret=sys_kcmp(pid1,pid1,KCMP_VM,0,0);if(ret){printf("FAIL: 0 expected but %d returned (%s)\n",ret,strerror(errno));+ksft_inc_fail_cnt();ret=-1;-}else+}else{printf("PASS: 0 returned as expected\n");+ksft_inc_pass_cnt();+}++ksft_print_cnts();-exit(ret);+if(ret)+ksft_exit_fail();+else+ksft_exit_pass();}waitpid(pid2,&status,P_ALL);-return0;+returnksft_exit_pass();}
Change ipc test to use kselftest framework to report
test results. With this change this test exits with
EXIT_FAIL instead of -errno. Changed print errno in
test fail messages to not loose that information.
Signed-off-by: Shuah Khan <redacted>
---
tools/testing/selftests/ipc/msgque.c | 26 ++++++++++++++------------
1 file changed, 14 insertions(+), 12 deletions(-)
@@ -195,58 +197,58 @@ int main(int argc, char **argv)if(getuid()!=0){printf("Please run the test as root - Exiting.\n");-exit(1);+returnksft_exit_fail();}msgque.key=ftok(argv[0],822155650);if(msgque.key==-1){-printf("Can't make key\n");-return-errno;+printf("Can't make key: %d\n",-errno);+returnksft_exit_fail();}msgque.msq_id=msgget(msgque.key,IPC_CREAT|IPC_EXCL|0666);if(msgque.msq_id==-1){err=-errno;-printf("Can't create queue\n");+printf("Can't create queue: %d\n",err);gotoerr_out;}err=fill_msgque(&msgque);if(err){-printf("Failed to fill queue\n");+printf("Failed to fill queue: %d\n",err);gotoerr_destroy;}err=dump_queue(&msgque);if(err){-printf("Failed to dump queue\n");+printf("Failed to dump queue: %d\n",err);gotoerr_destroy;}err=check_and_destroy_queue(&msgque);if(err){-printf("Failed to check and destroy queue\n");+printf("Failed to check and destroy queue: %d\n",err);gotoerr_out;}err=restore_queue(&msgque);if(err){-printf("Failed to restore queue\n");+printf("Failed to restore queue: %d\n",err);gotoerr_destroy;}err=check_and_destroy_queue(&msgque);if(err){-printf("Failed to test queue\n");+printf("Failed to test queue: %d\n",err);gotoerr_out;}-return0;+returnksft_exit_pass();err_destroy:if(msgctl(msgque.msq_id,IPC_RMID,0)){printf("Failed to destroy queue: %d\n",-errno);-return-errno;+returnksft_exit_fail();}err_out:-returnerr;+returnksft_exit_fail();}
On Tue, 2014-09-23 at 15:32 -0600, Shuah Khan wrote:
quoted hunk
Add kselftest framework for tests to use. This is a light
weight framework provides a set of interfaces to report test
results. Tests can use these interfaces to report pass, and
fail cases as well as when failure is due to configuration
problems such as missing modules, or when a test that is should
fail, fails as expected, and a test that should fail, passes.
The framework uses POSIX standard return codes for reporting
results to address the needs of users that want to run the kernel
selftests from their user-space test suites and want to know why a
test failed. In addition, the framework includes interfaces to use
to report test statistics on number of tests passed and failed.
Signed-off-by: Shuah Khan <redacted>
---
tools/testing/selftests/kselftest.h | 41 +++++++++++++++++++++++++++++++++++++
1 file changed, 41 insertions(+)
create mode 100644 tools/testing/selftests/kselftest.h
It would probably make sense to have the counters in a structures,
something like: struct ksft_counter { ... } ksft_cnt;
Then just pass it around the proposed functions as arguments. That also
minimizes a bit the global variables and would allow you to easily
change it in the future.
It would probably make sense to have the counters in a structures,
something like: struct ksft_counter { ... } ksft_cnt;
Then just pass it around the proposed functions as arguments. That also
minimizes a bit the global variables and would allow you to easily
change it in the future.
How does the following look?
struct ksft_count
{
unsigned int ksft_pass;
unsigned int ksft_fail;
unsigned int ksft_xfail;
unsigned int ksft_xpass;
unsigned int ksft_xskip;
};
static ksft_count ksft_cnt;
static inline void ksft_inc_pass_cnt(void) { ksft_cnt.ksft_pass++; }
static inline void ksft_inc_fail_cnt(void) { ksft_cnt.ksft_fail++; }
static inline void ksft_inc_xfail_cnt(void) { ksft_cnt.ksft_xfail++; }
static inline void ksft_inc_xpass_cnt(void) { ksft_cnt.ksft_xpass++; }
static inline void ksft_inc_xskip_cnt(void) { ksft_cnt.ksft_xskip++; }
With this approach, tests don't have to define their own counter
variable and pass it in. I am looking to abstract the framework
as much as possible.
thanks,
-- Shuah
--
Shuah Khan
Sr. Linux Kernel Developer
Samsung Research America (Silicon Valley)
shuahkh-JPH+aEBZ4P+UEJcrhfAQsw@public.gmane.org | (970) 217-8978