[PATCH] git-notes: Run partial expensive test everywhere

The git-notes expensive timing test is only expensive because it
either did 1k iterations or nothing. Change it to do 10 by default,
with an option to run the expensive version with the old
GIT_NOTES_TIMING_TESTS=ZomgYesPlease variable.

Since nobody was ostensibly running this test under TAP the code had
bitrotted so that it emitted invalid TAP. This change fixes that.

The old version would also mysteriously fail on systems without
/usr/bin/time, there's now a check for that using the test
prerequisite facility.

Signed-off-by: Ævar Arnfjörð Bjarmason <redacted>
---
 t/t3302-notes-index-expensive.sh |   39 ++++++++++++++++++++++++++-----------
 1 files changed, 27 insertions(+), 12 deletions(-)

diff --git a/t/t3302-notes-index-expensive.sh b/t/t3302-notes-index-expensive.sh
index 361a10a..3b8313c 100755
--- a/t/t3302-notes-index-expensive.sh
+++ b/t/t3302-notes-index-expensive.sh

@@ -7,11 +7,16 @@ test_description='Test commit notes index (expensive!)'
 
 . ./test-lib.sh
 
-test -z "$GIT_NOTES_TIMING_TESTS" && {
-	skip_all="Skipping timing tests"
-	test_done
-	exit
-}
+test_set_prereq NOT_EXPENSIVE
+test -n "$GIT_NOTES_TIMING_TESTS" && test_set_prereq EXPENSIVE
+
+if test -x /usr/bin/time
+then
+	# Hack around multiple test prerequisites not supporting AND-ing
+	# of terms
+	test_set_prereq USR_BIN_TIME+NOT_EXPENSIVE
+	test_have_prereq EXPENSIVE && test_set_prereq USR_BIN_TIME+EXPENSIVE
+fi
 
 create_repo () {
 	number_of_commits=$1

@@ -102,17 +107,27 @@ time_notes () {
 	done
 }
 
-for count in 10 100 1000 10000; do
+do_tests () {
+	pr=$1
+	count=$2
+
+	test_expect_success $pr 'setup / mkdir' '
+		mkdir $count &&
+		cd $count
+	'
 
-	mkdir $count
-	(cd $count;
+	test_expect_success $pr "setup $count" "create_repo $count"
 
-	test_expect_success "setup $count" "create_repo $count"
+	test_expect_success $pr 'notes work' "test_notes $count"
 
-	test_expect_success 'notes work' "test_notes $count"
+	test_expect_success USR_BIN_TIME+$pr 'notes timing with /usr/bin/time' "time_notes 100"
+
+	test_expect_success $pr 'teardown / cd ..' 'cd ..'
+}
 
-	test_expect_success 'notes timing' "time_notes 100"
-	)
+do_tests NOT_EXPENSIVE 10
+for count in 100 1000 10000; do
+	do_tests EXPENSIVE $count
 done
 
 test_done

-- 
1.7.2.1.295.gd03d

Heya,

On Tue, Aug 10, 2010 at 14:56, Ævar Arnfjörð Bjarmason [off-list ref] wrote:

The git-notes expensive timing test is only expensive because it
either did 1k iterations or nothing. Change it to do 10 by default,
with an option to run the expensive version with the old
GIT_NOTES_TIMING_TESTS=ZomgYesPlease variable.

Nice, why 10 though? Any motivation for that particular value?

Since nobody was ostensibly running this test under TAP the code had
bitrotted so that it emitted invalid TAP. This change fixes that.

Nice catch.

The old version would also mysteriously fail on systems without
/usr/bin/time, there's now a check for that using the test
prerequisite facility.

Should this patch be split up?

-- 
Cheers,

Sverre Rabbelier

On Tue, Aug 10, 2010 at 20:29, Sverre Rabbelier [off-list ref] wrote:

Heya,

On Tue, Aug 10, 2010 at 14:56, Ævar Arnfjörð Bjarmason [off-list ref] wrote:

quoted

The git-notes expensive timing test is only expensive because it
either did 1k iterations or nothing. Change it to do 10 by default,
with an option to run the expensive version with the old
GIT_NOTES_TIMING_TESTS=ZomgYesPlease variable.

Nice, why 10 though? Any motivation for that particular value?

The old version had "for count in 10 100 1000 10000; do". Mine has 10
as non-expensive, and "for count in 100 1000 10000; do" as expensive.

I.e. I'm running the first test batch from the old tests.

I have no idea whether it actually needs to run 10..10k times, I
didn't try to grok the actual test code.

quoted

The old version would also mysteriously fail on systems without
/usr/bin/time, there's now a check for that using the test
prerequisite facility.

Should this patch be split up?

It all touched the same bits, it'd be nastier to split it up IMO.

On Tue, Aug 10, 2010 at 19:56, Ævar Arnfjörð Bjarmason [off-list ref] wrote:

+       # Hack around multiple test prerequisites not supporting AND-ing
+       # of terms
+       test_set_prereq USR_BIN_TIME+NOT_EXPENSIVE
+       test_have_prereq EXPENSIVE && test_set_prereq USR_BIN_TIME+EXPENSIVE
+fi

In retrospect this may have been some brainfried code, I'll check it
out tomorrow.

The git-notes expensive timing test is only expensive because it
either did 10,100,1k and 10k iterations or nothing.

Change it to do 10 by default, with an option to run the expensive
version with the old GIT_NOTES_TIMING_TESTS=ZomgYesPlease variable.

Since nobody was ostensibly running this test under TAP the code had
bitrotted so that it emitted invalid TAP. This change fixes that.

The old version would also mysteriously fail on systems without
/usr/bin/time, there's now a check for that using the multiple test
prerequisite facility.

Signed-off-by: Ævar Arnfjörð Bjarmason <redacted>
---

On Tue, Aug 10, 2010 at 21:56, Ævar Arnfjörð Bjarmason [off-list ref] wrote:

On Tue, Aug 10, 2010 at 19:56, Ævar Arnfjörð Bjarmason [off-list ref] wrote:

quoted

+       # Hack around multiple test prerequisites not supporting AND-ing
+       # of terms
+       test_set_prereq USR_BIN_TIME+NOT_EXPENSIVE
+       test_have_prereq EXPENSIVE && test_set_prereq USR_BIN_TIME+EXPENSIVE
+fi

In retrospect this may have been some brainfried code, I'll check it
out tomorrow.

Here's a patch that's not crazy. In v1 I was hacking around not having
a facility I already added to the test-lib (tired).

This patch goes on top of my "test-lib: Multi-prereq support only
checked the last prereq" patch, which fixes up the test prereq
facility so that it actually works.

 t/t3302-notes-index-expensive.sh |   32 ++++++++++++++++++++------------
 1 files changed, 20 insertions(+), 12 deletions(-)

diff --git a/t/t3302-notes-index-expensive.sh b/t/t3302-notes-index-expensive.sh
index 361a10a..7c08e99 100755
--- a/t/t3302-notes-index-expensive.sh
+++ b/t/t3302-notes-index-expensive.sh

@@ -7,11 +7,9 @@ test_description='Test commit notes index (expensive!)'
 
 . ./test-lib.sh
 
-test -z "$GIT_NOTES_TIMING_TESTS" && {
-	skip_all="Skipping timing tests"
-	test_done
-	exit
-}
+test_set_prereq NOT_EXPENSIVE
+test -n "$GIT_NOTES_TIMING_TESTS" && test_set_prereq EXPENSIVE
+test -x /usr/bin/time && test_set_prereq USR_BIN_TIME
 
 create_repo () {
 	number_of_commits=$1

@@ -102,17 +100,27 @@ time_notes () {
 	done
 }
 
-for count in 10 100 1000 10000; do
+do_tests () {
+	pr=$1
+	count=$2
+
+	test_expect_success $pr 'setup / mkdir' '
+		mkdir $count &&
+		cd $count
+	'
 
-	mkdir $count
-	(cd $count;
+	test_expect_success $pr "setup $count" "create_repo $count"
 
-	test_expect_success "setup $count" "create_repo $count"
+	test_expect_success $pr 'notes work' "test_notes $count"
 
-	test_expect_success 'notes work' "test_notes $count"
+	test_expect_success USR_BIN_TIME,$pr 'notes timing with /usr/bin/time' "time_notes 100"
+
+	test_expect_success $pr 'teardown / cd ..' 'cd ..'
+}
 
-	test_expect_success 'notes timing' "time_notes 100"
-	)
+do_tests NOT_EXPENSIVE 10
+for count in 100 1000 10000; do
+	do_tests EXPENSIVE $count
 done
 
 test_done

-- 
1.7.2.1.295.gd03d

On Tue, Aug 10, 2010 at 23:37, Ævar Arnfjörð Bjarmason [off-list ref] wrote:

+test -x /usr/bin/time && test_set_prereq USR_BIN_TIME

It turns out that this fails on Solaris because its /usr/bin/time is different.

Maybe this should just check if /usr/bin/time --version is ^GNU ?

Ævar Arnfjörð Bjarmason wrote:

It turns out that this fails on Solaris because its /usr/bin/time is different.

Odd.  Different how?  As far as I can tell, all that test asks
of time is to execv() its arguments and pass on a 0 exit status.

Ah, maybe this is it: perhaps /usr/bin/time sh runs /bin/sh.  Does the
following help?

Patch is against next.  Untested except on Linux where it wouldn't
make a difference.
-- 8< --
Subject: t3302 (notes): Port to Solaris

The time_notes script, which uses POSIX shell features, is
currently sometimes run with a non-POSIX /bin/sh.

Reported-by: Ævar Arnfjörð Bjarmason <redacted>
Signed-off-by: Jonathan Nieder <redacted>
---

diff --git a/t/t3302-notes-index-expensive.sh b/t/t3302-notes-index-expensive.sh
index 7c08e99..e35d781 100755
--- a/t/t3302-notes-index-expensive.sh
+++ b/t/t3302-notes-index-expensive.sh

@@ -96,7 +96,7 @@ time_notes () {
 	for mode in no-notes notes
 	do
 		echo $mode
-		/usr/bin/time sh ../time_notes $mode $1
+		/usr/bin/time "$SHELL_PATH" ../time_notes $mode $1
 	done
 }
 

--