[PATCH v3 0/2] submodule update continue

STALE3716d

Revision v3 of 2 in this series.

5 messages, 2 authors, 2016-06-15 · open the first message on its own page

[PATCH v3 0/2] submodule update continue

From: Fredrik Gustafsson <hidden>
Date: 2016-06-15 22:51:27 

This series adds support to submodule update to continue if some
submodules fail to do so.

This is the third iteration of this patch series. 
The first can be found on:
http://thread.gmane.org/gmane.comp.version-control.git/171895
The second can be found on:
http://thread.gmane.org/gmane.comp.version-control.git/173629
Fredrik Gustafsson (2):
  Add new function die_with_status
  submodule update: continue when a checkout fails

 git-sh-setup.sh             |   12 +++-
 git-submodule.sh            |   48 +++++++++++++--
 t/t7406-submodule-update.sh |  144 +++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 196 insertions(+), 8 deletions(-)

-- 
1.7.6.rc0.14.g4fc55

[PATCH v3 1/2] Add new function die_with_status

From: Fredrik Gustafsson <hidden>
Date: 2016-06-15 22:51:27 

This function takes the exit code as a parameter.

This allows scripts to control the exit code when they need to die.

Make the old die-function to use die_with_status for simpler
maintenance.

die no longer prints empty die messages, this is a changed behavior.

Signed-off-by: Fredrik Gustafsson <redacted>
Mentored-by: Jens Lehmann [off-list ref]
Mentored-by: Heiko Voigt [off-list ref]
---
 git-sh-setup.sh |   12 ++++++++++--
 1 files changed, 10 insertions(+), 2 deletions(-)

diff --git a/git-sh-setup.sh b/git-sh-setup.sh
index 94e26ed..1a91f6e 100644
--- a/git-sh-setup.sh
+++ b/git-sh-setup.sh
@@ -40,8 +40,16 @@ git_broken_path_fix () {
 # @@BROKEN_PATH_FIX@@
 
 die() {
-	echo >&2 "$@"
-	exit 1
+	die_with_status 1 $@
+}
+die_with_status() {
+	stat=$1
+	shift
+	if test ${#@} -gt 0
+	then
+		echo >&2 "$@"
+	fi
+	exit $stat
 }
 
 GIT_QUIET=
-- 
1.7.6.rc0.14.g4fc55

[PATCH v3 2/2] submodule update: continue when a checkout fails

From: Fredrik Gustafsson <hidden>
Date: 2016-06-15 22:51:27 

"git submodule update" stops at the first error and gives control
back to the user. Only after the user fixes the problematic
submodule and runs "git submodule update" again, the second error
is found. And the user needs to repeat until all the problems are
found and fixed one by one. This is tedious.

Instead, the command can remember which submodules it had trouble with,
continue updating the ones it can, and report which ones had errors at
the end. The user can run "git submodule update", find all the ones that
need minor fixing (e.g. working tree was dirty) to fix them in a single
pass. Then another "git submodule update" can be run to update all.

Note that the problematic submodules are skipped only when they are to
be integrated with a safer value of submodule.<name>.update option,
namely "checkout". Fixing a failure in a submodule that uses "rebase" or
"merge" may need an involved conflict resolution by the user, and
leaving too many submodules in states that need resolution would not
reduce the mental burden on the user.

Signed-off-by: Fredrik Gustafsson <redacted>
Mentored-by: Jens Lehmann [off-list ref]
Mentored-by: Heiko Voigt [off-list ref]
---
 git-submodule.sh            |   48 +++++++++++++--
 t/t7406-submodule-update.sh |  144 +++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 186 insertions(+), 6 deletions(-)

diff --git a/git-submodule.sh b/git-submodule.sh
index d189a24..69780a8 100755
--- a/git-submodule.sh
+++ b/git-submodule.sh
@@ -444,7 +444,8 @@ cmd_update()
 	fi
 
 	cloned_modules=
-	module_list "$@" |
+	module_list "$@" | {
+	err=
 	while read mode sha1 stage path
 	do
 		if test "$stage" = U
@@ -525,17 +526,52 @@ cmd_update()
 				;;
 			esac
 
-			(clear_local_git_env; cd "$path" && $command "$sha1") ||
-			die "Unable to $action '$sha1' in submodule path '$path'"
-			say "Submodule path '$path': $msg '$sha1'"
+			if (clear_local_git_env; cd "$path" && $command "$sha1")
+			then
+				say "Submodule path '$path': $msg '$sha1'"
+			else
+				case $action in
+				rebase|merge)
+					die_with_status 2 "Unable to $action '$sha1' in submodule path '$path'"
+					;;
+				*)
+					err="${err} \"Failed to $action in submodule path '$path'\""
+					continue
+					;;
+				esac
+			fi
 		fi
 
 		if test -n "$recursive"
 		then
-			(clear_local_git_env; cd "$path" && eval cmd_update "$orig_flags") ||
-			die "Failed to recurse into submodule path '$path'"
+			(clear_local_git_env; cd "$path" && eval cmd_update "$orig_flags")
+			res=$?
+			if test $res -gt 0
+			then
+				if test $res -eq 1
+				then
+					err="${err} \"Failed to recurse into submodule path '$path'\""
+					continue
+				else
+					die_with_status $res "Failed to recurse into submodule path '$path'"
+				fi
+			fi
 		fi
 	done
+
+	if test -n "$err"
+	then
+		OIFS=$IFS
+		IFS=';'
+		for e in $err
+		do
+			echo $e
+		done
+		IFS=$OIFS
+		die
+	fi
+
+	}
 }
 
 set_name_rev () {
diff --git a/t/t7406-submodule-update.sh b/t/t7406-submodule-update.sh
index 4f16fcc..807f761 100755
--- a/t/t7406-submodule-update.sh
+++ b/t/t7406-submodule-update.sh
@@ -298,4 +298,148 @@ test_expect_success 'submodule update ignores update=rebase config for new submo
 	)
 '
 
+test_expect_success 'submodule update continues after checkout error' '
+	(cd super &&
+	 git reset --hard HEAD &&
+	 git submodule add ../submodule submodule2 &&
+	 git submodule init &&
+	 git commit -am "new_submodule" &&
+	 (cd submodule2 &&
+	  git rev-parse --max-count=1 HEAD > ../expect
+	 ) &&
+	 (cd submodule &&
+	  test_commit "update_submodule" file
+	 ) &&
+	 (cd submodule2 &&
+	  test_commit "update_submodule2" file
+	 ) &&
+	 git add submodule &&
+	 git add submodule2 &&
+	 git commit -m "two_new_submodule_commits" &&
+	 (cd submodule &&
+	  echo "" > file
+	 ) &&
+	 git checkout HEAD^ &&
+	 test_must_fail git submodule update &&
+	 (cd submodule2 &&
+	  git rev-parse --max-count=1 HEAD > ../actual
+	 ) &&
+	 test_cmp expect actual
+	)
+'
+test_expect_success 'submodule update continues after recursive checkout error' '
+	(cd super &&
+	 git reset --hard HEAD &&
+	 git checkout master &&
+	 git submodule update &&
+	 (cd submodule &&
+	  git submodule add ../submodule subsubmodule &&
+	  git submodule init &&
+	  git commit -m "new_subsubmodule"
+	 ) &&
+	 git add submodule &&
+	 git commit -m "update_submodule" &&
+	 (cd submodule &&
+	  (cd subsubmodule &&
+	   test_commit "update_subsubmodule" file
+	  ) &&
+	  git add subsubmodule &&
+	  test_commit "update_submodule_again" file &&
+	  (cd subsubmodule &&
+	   test_commit "update_subsubmodule_again" file
+	  ) &&
+	  test_commit "update_submodule_again_again" file
+	 ) &&
+	 (cd submodule2 &&
+	  git rev-parse --max-count=1 HEAD > ../expect &&
+	  test_commit "update_submodule2_again" file
+	 ) &&
+	 git add submodule &&
+	 git add submodule2 &&
+	 git commit -m "new_commits" &&
+	 git checkout HEAD^ &&
+	 (cd submodule &&
+	  git checkout HEAD^ &&
+	  (cd subsubmodule &&
+	   echo "" > file
+	  )
+	 ) &&
+	 test_must_fail git submodule update --recursive &&
+	 (cd submodule2 &&
+	  git rev-parse --max-count=1 HEAD > ../actual
+	 ) &&
+	 test_cmp expect actual
+	)
+'
+
+test_expect_success 'submodule update exit immediately in case of merge conflict' '
+	(cd super &&
+	 git checkout master &&
+	 git reset --hard HEAD &&
+	 (cd submodule &&
+	  (cd subsubmodule &&
+	   git reset --hard HEAD
+	  )
+	 ) &&
+	 git submodule update --recursive &&
+	 (cd submodule &&
+	  test_commit "update_submodule_2" file
+	 ) &&
+	 (cd submodule2 &&
+	  test_commit "update_submodule2_2" file
+	 ) &&
+	 git add submodule &&
+	 git add submodule2 &&
+	 git commit -m "two_new_submodule_commits" &&
+	 (cd submodule &&
+	  git checkout master &&
+	  test_commit "conflict" file &&
+	  echo "conflict" > file
+	 ) &&
+	 git checkout HEAD^ &&
+	 (cd submodule2 &&
+	  git rev-parse --max-count=1 HEAD > ../expect
+	 ) &&
+	 git config submodule.submodule.update merge &&
+	 test_must_fail git submodule update &&
+	 (cd submodule2 &&
+	  git rev-parse --max-count=1 HEAD > ../actual
+	 ) &&
+	 test_cmp expect actual
+	)
+'
+test_expect_success 'submodule update exit immediately after recursive rebase error' '
+	(cd super &&
+	 git checkout master &&
+	 git reset --hard HEAD &&
+	 (cd submodule &&
+	  git reset --hard HEAD &&
+	  git submodule update --recursive
+	 ) &&
+	 (cd submodule &&
+	  test_commit "update_submodule_3" file
+	 ) &&
+	 (cd submodule2 &&
+	  test_commit "update_submodule2_3" file
+	 ) &&
+	 git add submodule &&
+	 git add submodule2 &&
+	 git commit -m "two_new_submodule_commits" &&
+	 (cd submodule &&
+	  git checkout master &&
+	  test_commit "conflict2" file &&
+	  echo "conflict" > file
+	 ) &&
+	 git checkout HEAD^ &&
+	 (cd submodule2 &&
+	  git rev-parse --max-count=1 HEAD > ../expect
+	 ) &&
+	 git config submodule.submodule.update rebase &&
+	 test_must_fail git submodule update &&
+	 (cd submodule2 &&
+	  git rev-parse --max-count=1 HEAD > ../actual
+	 ) &&
+	 test_cmp expect actual
+	)
+'
 test_done
-- 
1.7.6.rc0.14.g4fc55

Re: [PATCH v3 2/2] submodule update: continue when a checkout fails

From: Junio C Hamano <hidden>
Date: 2016-06-15 22:51:27 

Fredrik Gustafsson [off-list ref] writes:

+				*)
+					err="${err} \"Failed to $action in submodule path '$path'\""
...
+					err="${err} \"Failed to recurse into submodule path '$path'\""
+					continue
+				else
+					die_with_status $res "Failed to recurse into submodule path '$path'"
+				fi
+			fi
 		fi
 	done
+
+	if test -n "$err"
+	then
+		OIFS=$IFS
+		IFS=';'
+		for e in $err
Hmm, I do not see anybody concatenating things with semicolons in between...
Am I reading a right patch?

Also I am puzzled what the double-quotes around each error message are
doing. Are they meant to be shown to the end user?

+		do
+			echo $e
Should this go to the standard error output?

+		done
+		IFS=$OIFS
+		die
Do we want to give an extra empty line here with "die"?

[PATCH v4 2/2] submodule update: continue when a checkout fails

From: Fredrik Gustafsson <hidden>
Date: 2016-06-15 22:51:28 

"git submodule update" stops at the first error and gives control
back to the user. Only after the user fixes the problematic
submodule and runs "git submodule update" again, the second error
is found. And the user needs to repeat until all the problems are
found and fixed one by one. This is tedious.

Instead, the command can remember which submodules it had trouble with,
continue updating the ones it can, and report which ones had errors at
the end. The user can run "git submodule update", find all the ones that
need minor fixing (e.g. working tree was dirty) to fix them in a single
pass. Then another "git submodule update" can be run to update all.

Note that the problematic submodules are skipped only when they are to
be integrated with a safer value of submodule.<name>.update option,
namely "checkout". Fixing a failure in a submodule that uses "rebase" or
"merge" may need an involved conflict resolution by the user, and
leaving too many submodules in states that need resolution would not
reduce the mental burden on the user.

Signed-off-by: Fredrik Gustafsson <redacted>
Mentored-by: Jens Lehmann [off-list ref]
Mentored-by: Heiko Voigt [off-list ref]
---
I'm sorry. You were reading a mix between two solutions that slipped
through. Here's a corrected patch.

I'll changed it to be printed to standard error output instead.

die will not print an empty line (as of the first patch in this series) but I
changed this to "exit 1" instead to be more clear.

 git-submodule.sh            |   50 +++++++++++++--
 t/t7406-submodule-update.sh |  144 +++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 188 insertions(+), 6 deletions(-)

diff --git a/git-submodule.sh b/git-submodule.sh
index d189a24..eb5eebc 100755
--- a/git-submodule.sh
+++ b/git-submodule.sh
@@ -444,7 +444,8 @@ cmd_update()
 	fi
 
 	cloned_modules=
-	module_list "$@" |
+	module_list "$@" | {
+	err=
 	while read mode sha1 stage path
 	do
 		if test "$stage" = U
@@ -525,17 +526,54 @@ cmd_update()
 				;;
 			esac
 
-			(clear_local_git_env; cd "$path" && $command "$sha1") ||
-			die "Unable to $action '$sha1' in submodule path '$path'"
-			say "Submodule path '$path': $msg '$sha1'"
+			if (clear_local_git_env; cd "$path" && $command "$sha1")
+			then
+				say "Submodule path '$path': $msg '$sha1'"
+			else
+				case $action in
+				rebase|merge)
+					die_with_status 2 "Unable to $action '$sha1' in submodule path '$path'"
+					;;
+				*)
+					err="${err};Failed to $action in submodule path '$path'"
+					continue
+					;;
+				esac
+			fi
 		fi
 
 		if test -n "$recursive"
 		then
-			(clear_local_git_env; cd "$path" && eval cmd_update "$orig_flags") ||
-			die "Failed to recurse into submodule path '$path'"
+			(clear_local_git_env; cd "$path" && eval cmd_update "$orig_flags")
+			res=$?
+			if test $res -gt 0
+			then
+				if test $res -eq 1
+				then
+					err="${err};Failed to recurse into submodule path '$path'"
+					continue
+				else
+					die_with_status $res "Failed to recurse into submodule path '$path'"
+				fi
+			fi
 		fi
 	done
+
+	if test -n "$err"
+	then
+		OIFS=$IFS
+		IFS=';'
+		for e in $err
+		do
+			if test -n "$e"
+			then
+				echo >&2 "$e"
+			fi
+		done
+		IFS=$OIFS
+		exit 1
+	fi
+	}
 }
 
 set_name_rev () {
diff --git a/t/t7406-submodule-update.sh b/t/t7406-submodule-update.sh
index 4f16fcc..807f761 100755
--- a/t/t7406-submodule-update.sh
+++ b/t/t7406-submodule-update.sh
@@ -298,4 +298,148 @@ test_expect_success 'submodule update ignores update=rebase config for new submo
 	)
 '
 
+test_expect_success 'submodule update continues after checkout error' '
+	(cd super &&
+	 git reset --hard HEAD &&
+	 git submodule add ../submodule submodule2 &&
+	 git submodule init &&
+	 git commit -am "new_submodule" &&
+	 (cd submodule2 &&
+	  git rev-parse --max-count=1 HEAD > ../expect
+	 ) &&
+	 (cd submodule &&
+	  test_commit "update_submodule" file
+	 ) &&
+	 (cd submodule2 &&
+	  test_commit "update_submodule2" file
+	 ) &&
+	 git add submodule &&
+	 git add submodule2 &&
+	 git commit -m "two_new_submodule_commits" &&
+	 (cd submodule &&
+	  echo "" > file
+	 ) &&
+	 git checkout HEAD^ &&
+	 test_must_fail git submodule update &&
+	 (cd submodule2 &&
+	  git rev-parse --max-count=1 HEAD > ../actual
+	 ) &&
+	 test_cmp expect actual
+	)
+'
+test_expect_success 'submodule update continues after recursive checkout error' '
+	(cd super &&
+	 git reset --hard HEAD &&
+	 git checkout master &&
+	 git submodule update &&
+	 (cd submodule &&
+	  git submodule add ../submodule subsubmodule &&
+	  git submodule init &&
+	  git commit -m "new_subsubmodule"
+	 ) &&
+	 git add submodule &&
+	 git commit -m "update_submodule" &&
+	 (cd submodule &&
+	  (cd subsubmodule &&
+	   test_commit "update_subsubmodule" file
+	  ) &&
+	  git add subsubmodule &&
+	  test_commit "update_submodule_again" file &&
+	  (cd subsubmodule &&
+	   test_commit "update_subsubmodule_again" file
+	  ) &&
+	  test_commit "update_submodule_again_again" file
+	 ) &&
+	 (cd submodule2 &&
+	  git rev-parse --max-count=1 HEAD > ../expect &&
+	  test_commit "update_submodule2_again" file
+	 ) &&
+	 git add submodule &&
+	 git add submodule2 &&
+	 git commit -m "new_commits" &&
+	 git checkout HEAD^ &&
+	 (cd submodule &&
+	  git checkout HEAD^ &&
+	  (cd subsubmodule &&
+	   echo "" > file
+	  )
+	 ) &&
+	 test_must_fail git submodule update --recursive &&
+	 (cd submodule2 &&
+	  git rev-parse --max-count=1 HEAD > ../actual
+	 ) &&
+	 test_cmp expect actual
+	)
+'
+
+test_expect_success 'submodule update exit immediately in case of merge conflict' '
+	(cd super &&
+	 git checkout master &&
+	 git reset --hard HEAD &&
+	 (cd submodule &&
+	  (cd subsubmodule &&
+	   git reset --hard HEAD
+	  )
+	 ) &&
+	 git submodule update --recursive &&
+	 (cd submodule &&
+	  test_commit "update_submodule_2" file
+	 ) &&
+	 (cd submodule2 &&
+	  test_commit "update_submodule2_2" file
+	 ) &&
+	 git add submodule &&
+	 git add submodule2 &&
+	 git commit -m "two_new_submodule_commits" &&
+	 (cd submodule &&
+	  git checkout master &&
+	  test_commit "conflict" file &&
+	  echo "conflict" > file
+	 ) &&
+	 git checkout HEAD^ &&
+	 (cd submodule2 &&
+	  git rev-parse --max-count=1 HEAD > ../expect
+	 ) &&
+	 git config submodule.submodule.update merge &&
+	 test_must_fail git submodule update &&
+	 (cd submodule2 &&
+	  git rev-parse --max-count=1 HEAD > ../actual
+	 ) &&
+	 test_cmp expect actual
+	)
+'
+test_expect_success 'submodule update exit immediately after recursive rebase error' '
+	(cd super &&
+	 git checkout master &&
+	 git reset --hard HEAD &&
+	 (cd submodule &&
+	  git reset --hard HEAD &&
+	  git submodule update --recursive
+	 ) &&
+	 (cd submodule &&
+	  test_commit "update_submodule_3" file
+	 ) &&
+	 (cd submodule2 &&
+	  test_commit "update_submodule2_3" file
+	 ) &&
+	 git add submodule &&
+	 git add submodule2 &&
+	 git commit -m "two_new_submodule_commits" &&
+	 (cd submodule &&
+	  git checkout master &&
+	  test_commit "conflict2" file &&
+	  echo "conflict" > file
+	 ) &&
+	 git checkout HEAD^ &&
+	 (cd submodule2 &&
+	  git rev-parse --max-count=1 HEAD > ../expect
+	 ) &&
+	 git config submodule.submodule.update rebase &&
+	 test_must_fail git submodule update &&
+	 (cd submodule2 &&
+	  git rev-parse --max-count=1 HEAD > ../actual
+	 ) &&
+	 test_cmp expect actual
+	)
+'
 test_done
-- 
1.7.6.rc0.14.gc8898.dirty
Keyboard shortcuts
hback out one level
jnext message in thread
kprevious message in thread
ldrill in
Escclose help / fold thread tree
?toggle this help