From: Imran M Yousuf <redacted>
The purpose of the recurse command in the git submodule is to recurse
a command in its submodule. For example if one wants to do a diff on its
project with submodules at once, one can simply do
git-submodule recurse diff HEAD
and would see the diff for all the modules it contains.
The recurse commands behavior can be customized with several arguments
that it accepts. The synopsis for the recurse command is:
git-submodule recurse [-q|--quiet] [-e|--exit-after-error]
[-d|--depth <recursion depth>] [-b|--breadth-first]
<git command> [<arguments> ...]
There are commands that can fail for a certain submodule but succeed for
others; if one wants to stop execution once the top level module's execution
fails, one can specify [-e|--exit-after-error]. It will ensure that once
execution of git <command> fails in the top level module it will not recurse
into its submodules.
If the project has submodule hierarchy upto n depth and we want to restrict
recursion to (n-p) depth; we can use the [-d|--depth <recursion depth>] option.
Value has to be greater than 0 and command will at least recurse into the first
depth. If depth is specified to p than all depths <= p will be recursed over.
While discussion on the recurse command one thing which was put forward
in several occassions is that there might be scenario where a command should be
executed over the child module before the parent module.
For such scenario [-b|--breadth-first] option can be used; one use case
in particular presented as an example is git commit; where almost everybody
mentioned that they prefer to commit the child module before the parent and
default will enable just that.
E.g. p -> a, b, c, e; a ->d is a module structure. If the following command is
used,
git submodule recurse commit -a
it will execute git commit -a in the following sequence - d, a, b, c, e, p.
Now if one want to instead go in a breadth first manner then one can
specify -b option. E.g. if the above command is -
git submodule recurse -b commit -a
it will execute git commit -a in the following sequence - p, a, d, b, c, e.
Signed-off-by: Imran M Yousuf <redacted>
---
git-submodule.sh | 132 +++++++++++++++++++++++++++++++++++++++++++++++++++++-
1 files changed, 130 insertions(+), 2 deletions(-)
diff --git a/git-submodule.sh b/git-submodule.sh
index a5ee2e5..8161d51 100755
--- a/git-submodule.sh
+++ b/git-submodule.sh
@@ -11,7 +11,8 @@ Use $0 -h for more details"
LONG_USAGE="$0 add [-q|--quiet] [-b|--branch branch] <repository> [<path>]
$0 [status] [-q|--quiet] [-c|--cached] [--] [<path>...]
$0 init|update [-q|--quiet] [--] [<path>...]
-$0 summary [--cached] [-n|--summary-limit <n>] [<commit>]"
+$0 summary [--cached] [-n|--summary-limit <n>] [<commit>]
+$0 recurse [-q|--quiet] [-e|--exit-after-error] [-d|--depth <recursion depth>] [-b|--breadth-first] <git command> [<args> ...]"
OPTIONS_SPEC=
. git-sh-setup
require_work_tree
@@ -20,6 +21,10 @@ command=
branch=
quiet=
cached=
+depth=0
+current_depth=0
+depth_first=1
+on_error=
#
# print stuff on stdout unless -q was specified
@@ -580,6 +585,129 @@ cmd_status()
done
}
+# Check whether the submodule is initialized or not
+initialize_sub_module()
+{
+ if test ! -d "$1"/.git
+ then
+ say "Submodule $1 is not initialized and skipped"
+ return 1
+ # Returns true if submodule is already initialized
+ elif test -d "$1"/.git
+ then
+ return 0
+ fi
+}
+
+# This function simply checks whether the depth is traverseable in terms of
+# depth and if so then it sequentially traverses its submodules
+traverse_submodules()
+{
+ # If current depth is the range specified than it will continue
+ # else return with success
+ if test "$depth" -gt 0 &&
+ test "$current_depth" -ge "$depth"
+ then
+ return 0;
+ fi
+ # If submodules exists than it will traverse over them
+ if test -f .gitmodules
+ then
+ # Incrementing the depth for the next level of submodules
+ current_depth=$(($current_depth + 1))
+ for mod_path in `sed -n -e 's/path = //p' .gitmodules`; do
+ traverse_module "$mod_path" "$@"
+ done
+ # Decremented the depth to bring it back to the depth of
+ # the current submodule
+ current_depth=$(($current_depth - 1))
+ fi
+}
+
+# This actually traverses a submodule; checks whether the its initialized
+# or not, does nothing if not initialized.
+traverse_module()
+{
+ # Will work in the submodule if and only if its initialized
+ initialize_sub_module "$1" &&
+ (
+ submod_path="$1"
+ shift
+ cd "$submod_path"
+ # If depth-first is specified in that case submodules are
+ # are traversed before executing the command on this submodule
+ test -n "$depth_first" && traverse_submodules "$@"
+ # pwd is mentioned in order to enable the ser to distinguish
+ # between same name modules, e.g. a/lib and b/lib.
+ say "git submodule recurse $submod_path $*"
+ git "$@"
+ # if exit on error is specifed than script will exit if any
+ # command fails. As there is no transaction there will be
+ # no rollback either
+ # TODO - If possible facilitate transaction
+ if test "$?" -ne 0 && test -n "$on_error"
+ then
+ die "FAILED: git submodule $submod_path $*"
+ fi
+ # If depth-first is not specified in that case submodules are
+ # are traversed after executing the command on this submodule
+ test -z "$depth_first" && traverse_submodules "$@"
+ )
+}
+
+# Propagates or recurses over all the submodules at any depth with any
+# git command, e.g. git-clone, git-status, git-commit etc., with the
+# arguments supplied exactly as it would have been supplied to the command
+# otherwise. This actually starts the recursive propagation.
+cmd_recurse() {
+ while :
+ do
+ case "$1" in
+ -q|--quiet)
+ quiet=1
+ ;;
+ -d|--depth)
+ shift
+ if test -z "$1"
+ then
+ echo "No <recursion depth> specified"
+ usage
+ # Arithmatic operation will give an error if depth is not number
+ # thus chose to check intergerness with regular expression.
+ # $1 is underquoted becuase the expr is in quotation
+ elif test "$(expr $1 : '[1-9][0-9]*')" -eq "$(expr $1 : '.*')"
+ then
+ depth="$1"
+ else
+ echo "<recursion depth> not an integer"
+ usage
+ fi
+ ;;
+ -b|--breadth-first)
+ depth_first=
+ ;;
+ -e|--exit-after-error)
+ on_error=1
+ ;;
+ -*)
+ usage
+ ;;
+ *)
+ break
+ ;;
+ esac
+ shift
+ done
+ test "$#" -le 0 && die "No git command specified"
+ project_home="$(pwd)"
+ if test -d "$project_home"/.git/
+ then
+ traverse_module . "$@"
+ else
+ die "$project_home not a git repo thus exiting"
+ fi
+}
+
# This loop parses the command line arguments to find the
# subcommand name to dispatch. Parsing of the subcommand specific
# options are primarily done by the subcommand implementations.@@ -589,7 +717,7 @@ cmd_status()
while test $# != 0 && test -z "$command"
do
case "$1" in
- add | init | update | status | summary)
+ add | init | update | status | summary |recurse)
command=$1
;;
-q|--quiet)
--
1.5.4.2
imyousuf@gmail.com writes:
The recurse commands behavior can be customized with several arguments
that it accepts. The synopsis for the recurse command is:
git-submodule recurse [-q|--quiet] [-e|--exit-after-error]
[-d|--depth <recursion depth>] [-b|--breadth-first]
<git command> [<arguments> ...]
Is there a reason to limit the command that can be run per submodule to
only "git" commands? To me, this "recurse" looks like a glorified "find"
command that can trigger its action only to submodule directories, but
limits what can be given to its -exec option to "git" commands. While it
would not make sense to give certain git command to recurse (e.g. neither
"git show 65ea3b8" nor "git clone $there" would make any sense), it would
be handy if we can give certain non-git commands to it (e.g. "du -sh").
quoted hunk
@@ -580,6 +585,129 @@ cmd_status()
done
}
+# Check whether the submodule is initialized or not
+initialize_sub_module()
Everybody else seems to spell "<do-something>_submodule"; should this be
any different?
+{
+ if test ! -d "$1"/.git
+ then
+ say "Submodule $1 is not initialized and skipped"
+ return 1
+ # Returns true if submodule is already initialized
Micronit; s/Returns/Return/. A sentence that begins with a capitalized
verb in comments is almost always in imperative mood, not third-person
singular present.
+ elif test -d "$1"/.git
+ then
+ return 0
+ fi
+}
Otherwise, what does it return? Do you need elif there, or just "else"?
+# This function simply checks whether the depth is traverseable in terms of
+# depth and if so then it sequentially traverses its submodules
+traverse_submodules()
+{
+ # If current depth is the range specified than it will continue
+ # else return with success
+ if test "$depth" -gt 0 &&
+ test "$current_depth" -ge "$depth"
+ then
+ return 0;
+ fi
+ # If submodules exists than it will traverse over them
+ if test -f .gitmodules
+ then
+ # Incrementing the depth for the next level of submodules
+ current_depth=$(($current_depth + 1))
+ for mod_path in `sed -n -e 's/path = //p' .gitmodules`; do
+ traverse_module "$mod_path" "$@"
+ done
+ # Decremented the depth to bring it back to the depth of
+ # the current submodule
+ current_depth=$(($current_depth - 1))
+ fi
+}
This makes me wonder if you should be iterating over .gitmodules, or
perhaps you may want to iterate over output of git-ls-files (picking
entries of gitlink type). How should a local change that adds a new
submodule or removes an existing submodule, or moves an existing submodule
interact with "submodule recurse"?
Also the same micronits (s/Incrementing/Increment/; s/Decremented/Decrement/).
Even if iterating over .gitmodules entries is a good idea, I suspect that
sed script is too fragile. Doesn't .gitmodules use the same format as git
configuration files, allowing spaces around values, value quoting and
trailing comments on the same line?
+# This actually traverses a submodule; checks whether the its initialized
+# or not, does nothing if not initialized.
s/the //;?
+traverse_module()
+{
+ # Will work in the submodule if and only if its initialized
+ initialize_sub_module "$1" &&
"initialize_sub_module" does not sound like a function that checks if it
is initialized, but more like a function to, eh, initialize the submodule.
Perhaps the function should be renamed to make it clearer that it is a
predicate?
+ (
+ submod_path="$1"
+ shift
+ cd "$submod_path"
+ # If depth-first is specified in that case submodules are
+ # are traversed before executing the command on this submodule
+ test -n "$depth_first" && traverse_submodules "$@"
+ # pwd is mentioned in order to enable the ser to distinguish
+ # between same name modules, e.g. a/lib and b/lib.
+ say "git submodule recurse $submod_path $*"
+ git "$@"
+ # if exit on error is specifed than script will exit if any
+ # command fails. As there is no transaction there will be
+ # no rollback either
s/than/then/;?
+ # TODO - If possible facilitate transaction
+ if test "$?" -ne 0 && test -n "$on_error"
+ then
+ die "FAILED: git submodule $submod_path $*"
Dying before doing further damage to the repository tree may be a good
idea, but I did not see the calling loop in traverse_submodules pay
attention to the exit code from here.
+ fi
+ # If depth-first is not specified in that case submodules are
+ # are traversed after executing the command on this submodule
+ test -z "$depth_first" && traverse_submodules "$@"
+ )
+}
+
+# Propagates or recurses over all the submodules at any depth with any
+# git command, e.g. git-clone, git-status, git-commit etc., with the
+# arguments supplied exactly as it would have been supplied to the command
+# otherwise. This actually starts the recursive propagation.
Is "git-clone" a good example to give here? What would that mean to
recurse into each submodule directories in a superproject to run "clone"?
+cmd_recurse() {
+ while :
+ do
+ case "$1" in
+ -q|--quiet)
+ quiet=1
+ ;;
+ -d|--depth)
+ shift
+ if test -z "$1"
+ then
+ echo "No <recursion depth> specified"
+ usage
+ # Arithmatic operation will give an error if depth is not number
+ # thus chose to check intergerness with regular expression.
+ # $1 is underquoted becuase the expr is in quotation
+ elif test "$(expr $1 : '[1-9][0-9]*')" -eq "$(expr $1 : '.*')"
Huh?
$ a='1 2 3'
$ expr $a : '[1-9]'
expr: syntax error
$ expr "$a" : '[1-9]'
1
$ z=$(expr $a : '[1-9]')
expr: syntax error
$ z=$(expr "$a" : '[1-9]')
$ echo $z
1
$ echo "$(expr $a : '[1-9]')"
expr: syntax error
$ echo "$(expr "$a" : '[1-9]')"
1
If you want to make sure that $(( ... )) would not choke with given "$1",
you can check by attempting to do a simple $(( ... )) to see if it errors
out, which would be simpler.
if test -z "$1"
then
...
elif ! echo $(( "$1" + 0 )) >/dev/null
then
die "$1 is not an integer"
...
On Mon, May 12, 2008 at 7:20 AM, Junio C Hamano [off-list ref] wrote:
imyousuf@gmail.com writes:
> The recurse commands behavior can be customized with several arguments
> that it accepts. The synopsis for the recurse command is:
>
> git-submodule recurse [-q|--quiet] [-e|--exit-after-error]
> [-d|--depth <recursion depth>] [-b|--breadth-first]
> <git command> [<arguments> ...]
Is there a reason to limit the command that can be run per submodule to
only "git" commands? To me, this "recurse" looks like a glorified "find"
command that can trigger its action only to submodule directories, but
limits what can be given to its -exec option to "git" commands. While it
would not make sense to give certain git command to recurse (e.g. neither
"git show 65ea3b8" nor "git clone $there" would make any sense), it would
be handy if we can give certain non-git commands to it (e.g. "du -sh").
I do agree how the recurse command looks, but considering that it is a
'git submodule' subcommand I thought having a general command might
have faced a greater criticism from the community. Similarly about not
allowing certain git commands is also in my list for the later version
as it would require a bigger discussion in the community.
> @@ -580,6 +585,129 @@ cmd_status()
> done
> }
>
> +# Check whether the submodule is initialized or not
> +initialize_sub_module()
Everybody else seems to spell "<do-something>_submodule"; should this be
any different?
> +{
> + if test ! -d "$1"/.git
> + then
> + say "Submodule $1 is not initialized and skipped"
> + return 1
> + # Returns true if submodule is already initialized
Micronit; s/Returns/Return/. A sentence that begins with a capitalized
verb in comments is almost always in imperative mood, not third-person
singular present.
Got it, thanks for the correction.
> + elif test -d "$1"/.git
> + then
> + return 0
> + fi
> +}
Otherwise, what does it return? Do you need elif there, or just "else"?
Yup, else would be sufficient. Sorry for the mistake
> +# This function simply checks whether the depth is traverseable in terms of
> +# depth and if so then it sequentially traverses its submodules
> +traverse_submodules()
> +{
> + # If current depth is the range specified than it will continue
> + # else return with success
> + if test "$depth" -gt 0 &&
> + test "$current_depth" -ge "$depth"
> + then
> + return 0;
> + fi
> + # If submodules exists than it will traverse over them
> + if test -f .gitmodules
> + then
> + # Incrementing the depth for the next level of submodules
> + current_depth=$(($current_depth + 1))
> + for mod_path in `sed -n -e 's/path = //p' .gitmodules`; do
> + traverse_module "$mod_path" "$@"
> + done
> + # Decremented the depth to bring it back to the depth of
> + # the current submodule
> + current_depth=$(($current_depth - 1))
> + fi
> +}
This makes me wonder if you should be iterating over .gitmodules, or
perhaps you may want to iterate over output of git-ls-files (picking
entries of gitlink type). How should a local change that adds a new
submodule or removes an existing submodule, or moves an existing submodule
interact with "submodule recurse"?
Actually once I am done with the recurse command I was planning to add
submodule mv and rm subcommands :). About the git-ls-files command yes
that is also an option, but in case of move it would require editing
.gitmodules and .git/config. AFAIK user need to currently manually
edit them for updating, hoping to write a shell script to get it done.
About it interacting with these changes, as long as the .gitmodules
file is updated correctly it should not be a problem, but if it
becomes inconsistent then it will chokes. In this regard, I checked
how 'git submodule update' works and it uses git-ls-fiiles --stage
with grep to find the gitlinks path and then search them through
.git/config, but it also faces the same problem if move is done
manually without changing the files. Also to be noted is the status
command also uses git-ls-files.
The reason why I did .gitsubmodule is I want to introduce auto-init
and update as an option, and plan to do it once the basic recurse
patches are accepted :). Then reading the .gitmodules would have been
necessary.
About the sed script another option would be to use -
git config -f ./.gitmodules --get-regexp '^submodule\..*\.path$' |
sed -n -e 's|^submodule\.\(.*\)\.path \(.*\)$|\2|p'
Will using this be preferable? I think so :).
Also the same micronits (s/Incrementing/Increment/; s/Decremented/Decrement/).
Even if iterating over .gitmodules entries is a good idea, I suspect that
sed script is too fragile. Doesn't .gitmodules use the same format as git
configuration files, allowing spaces around values, value quoting and
trailing comments on the same line?
I agree on the fragile point and I think I will replace it with the
one I mentioned above.
> +# This actually traverses a submodule; checks whether the its initialized
> +# or not, does nothing if not initialized.
s/the //;?
> +traverse_module()
> +{
> + # Will work in the submodule if and only if its initialized
> + initialize_sub_module "$1" &&
"initialize_sub_module" does not sound like a function that checks if it
is initialized, but more like a function to, eh, initialize the submodule.
Perhaps the function should be renamed to make it clearer that it is a
predicate?
I thought of renaming it but I was a bit lazy as I am writing another
patch for auto initialize :).
> + (
> + submod_path="$1"
> + shift
> + cd "$submod_path"
> + # If depth-first is specified in that case submodules are
> + # are traversed before executing the command on this submodule
> + test -n "$depth_first" && traverse_submodules "$@"
> + # pwd is mentioned in order to enable the ser to distinguish
> + # between same name modules, e.g. a/lib and b/lib.
> + say "git submodule recurse $submod_path $*"
> + git "$@"
> + # if exit on error is specifed than script will exit if any
> + # command fails. As there is no transaction there will be
> + # no rollback either
s/than/then/;?
> + # TODO - If possible facilitate transaction
> + if test "$?" -ne 0 && test -n "$on_error"
> + then
> + die "FAILED: git submodule $submod_path $*"
Dying before doing further damage to the repository tree may be a good
idea, but I did not see the calling loop in traverse_submodules pay
attention to the exit code from here.
Thanks for pointing out this bug, will fix it in the next version.
> + fi
> + # If depth-first is not specified in that case submodules are
> + # are traversed after executing the command on this submodule
> + test -z "$depth_first" && traverse_submodules "$@"
> + )
> +}
> +
> +# Propagates or recurses over all the submodules at any depth with any
> +# git command, e.g. git-clone, git-status, git-commit etc., with the
> +# arguments supplied exactly as it would have been supplied to the command
> +# otherwise. This actually starts the recursive propagation.
Is "git-clone" a good example to give here? What would that mean to
recurse into each submodule directories in a superproject to run "clone"?
I agree that git-clone is infact a bad example, will remove it :).
> +cmd_recurse() {
> + while :
> + do
> + case "$1" in
> + -q|--quiet)
> + quiet=1
> + ;;
> + -d|--depth)
> + shift
> + if test -z "$1"
> + then
> + echo "No <recursion depth> specified"
> + usage
> + # Arithmatic operation will give an error if depth is not number
> + # thus chose to check intergerness with regular expression.
> + # $1 is underquoted becuase the expr is in quotation
> + elif test "$(expr $1 : '[1-9][0-9]*')" -eq "$(expr $1 : '.*')"
Huh?
$ a='1 2 3'
$ expr $a : '[1-9]'
expr: syntax error
$ expr "$a" : '[1-9]'
1
$ z=$(expr $a : '[1-9]')
expr: syntax error
$ z=$(expr "$a" : '[1-9]')
$ echo $z
1
$ echo "$(expr $a : '[1-9]')"
expr: syntax error
$ echo "$(expr "$a" : '[1-9]')"
1
If you want to make sure that $(( ... )) would not choke with given "$1",
you can check by attempting to do a simple $(( ... )) to see if it errors
out, which would be simpler.
if test -z "$1"
then
...
elif ! echo $(( "$1" + 0 )) >/dev/null
then
die "$1 is not an integer"
...
This was what I was looking for a simpler and cleaner way :), thanks a
lot Junio.
BTW: its nice to see your emails once again :).
Best regards,
Imran
--
Imran M Yousuf
Email: imran@smartitengineering.com
Mobile: +880-1711402557