Re: [PATCH] test-lib: write test results to test-results/<basename>-<pid>

Hi,

On Sat, Mar 14, 2009 at 02:16:57PM +0100, Sverre Rabbelier wrote:

On Sat, Mar 14, 2009 at 13:28, SZEDER Gábor [off-list ref] wrote:

quoted

With my proposed change there would be no need to clean 'test-results'
before running the tests, because test-lib.sh would take care of that
(not by removing and recreating 'test-results/', but by overwriting
(IOW: removing and recreating, but in one step) individual test result
files).

Wouldn't that result in possible stale files being counted in the
result (e.g., if those tests were not run this time, but they were run
previously)?

It depends.

If you run only a few tests, then you do it with a command like 'make
t1234-foo.sh t5678-bar.sh'.

Currently this doesn't run the 'pre-clean' target, therefore if you
run different tests (e.g. 'make t1234-foo.sh ; make t5678-bar.sh'),
then a 'make aggregate-results' will include the result of both of
those tests.  The same happens with my proposal, too, because the test
of the last run will not overwrite the results of the test in the
first run.

Now suppose that you want to run the same set of tests twice (or more;
e.g. 'make t1234-foo.sh ; make t1234-foo.sh ; make
aggregate-results').  Since currently the pid of the test is included
in the test result file name, there will be two files (t1234-<pid1>
and t1234-<pid2>) created under 'test-results/', and _both_ will be
counted by 'aggregate-results'.  In this case my proposal is better,
because the last round will overwrite the result of the previous runs,
therefore no stale files will be counted.


Best,
Gábor