[BUG?] "git submodule foreach" when command is ssh

Hi All,

I just noticed something odd with "git submodule foreach". I was
running a script to create a backup of each submodule on a server I
have ssh access to. I was surprised to find that git submodule foreach
stopped silently after the first submodule.

A little debugging and I find that

git submodule foreach 'ssh localhost "ls /"' - stops silently after
the first module (note that the command does produce the expected
listing and there is no error about the command failing).

git submodule foreach 'echo foo' - works as expected

Any thoughts as to whats going on?

---
git version 1.7.3.2

On Thu, Jan 6, 2011 at 11:32 AM, Chris Packham [off-list ref] wrote:

Hi All,

I just noticed something odd with "git submodule foreach". I was
running a script to create a backup of each submodule on a server I
have ssh access to. I was surprised to find that git submodule foreach
stopped silently after the first submodule.

A little debugging and I find that

git submodule foreach 'ssh localhost "ls /"' - stops silently after
the first module (note that the command does produce the expected
listing and there is no error about the command failing).

git submodule foreach 'echo foo' - works as expected

Any thoughts as to whats going on?

---
git version 1.7.3.2

Actually this might be a ssh/bash bug (feature?). There is different
behaviour between

  find . -maxdepth 1 -type d -a ! -name '\.*' | while read; do echo
$REPLY && ssh localhost ls /; done

and

  find . -maxdepth 1 -type d -a ! -name '\.*' | while read; do echo
$REPLY && ls /; done

On Thu, Jan 06, 2011 at 11:50:58AM +1300, Chris Packham wrote:

Actually this might be a ssh/bash bug (feature?). There is different
behaviour between

  find . -maxdepth 1 -type d -a ! -name '\.*' | while read; do echo
$REPLY && ssh localhost ls /; done

and

  find . -maxdepth 1 -type d -a ! -name '\.*' | while read; do echo
$REPLY && ls /; done

Ssh will opportunistically eat data on stdin to send to the other side,
even though the command on the other side ("ls" in this case) will never
read it. Because of course ssh has no way of knowing that, and is trying
to be an interactive terminal. So it ends up eating some random amount
of the data you expected to go to the "read" call.

You can use the "-n" option to suppress it. For example:

  $ (echo foo; echo bar) |
    while read line; do
      echo local $line
      ssh host "echo remote $line"
    done

produces:

  local foo
  remote foo

but:

  $ (echo foo; echo bar) |
    while read line; do
      echo local $line
      ssh -n host "echo remote $line"
    done

produces:

  local foo
  remote foo
  local bar
  remote bar

which is what you want.

-Peff

On Thu, Jan 6, 2011 at 12:03 PM, Jeff King [off-list ref] wrote:

On Thu, Jan 06, 2011 at 11:50:58AM +1300, Chris Packham wrote:

quoted

Actually this might be a ssh/bash bug (feature?). There is different
behaviour between

  find . -maxdepth 1 -type d -a ! -name '\.*' | while read; do echo
$REPLY && ssh localhost ls /; done

and

  find . -maxdepth 1 -type d -a ! -name '\.*' | while read; do echo
$REPLY && ls /; done

Ssh will opportunistically eat data on stdin to send to the other side,
even though the command on the other side ("ls" in this case) will never
read it. Because of course ssh has no way of knowing that, and is trying
to be an interactive terminal. So it ends up eating some random amount
of the data you expected to go to the "read" call.

You can use the "-n" option to suppress it. For example:

 $ (echo foo; echo bar) |
   while read line; do
     echo local $line
     ssh host "echo remote $line"
   done

produces:

 local foo
 remote foo

but:

 $ (echo foo; echo bar) |
   while read line; do
     echo local $line
     ssh -n host "echo remote $line"
   done

produces:

 local foo
 remote foo
 local bar
 remote bar

which is what you want.

-Peff

Thanks that makes sense and adding -n to my ssh invocations solves the problem.

In message [off-list ref], Chri
s Packham writes:

    I just noticed something odd with "git submodule foreach". I was
    running a script to create a backup of each submodule on a server I
    have ssh access to. I was surprised to find that git submodule foreach
    stopped silently after the first submodule.

    A little debugging and I find that

    git submodule foreach 'ssh localhost "ls /"' - stops silently after

Putting some input redirection will work around the problem.
Presumably some pipe input is going into the ssh accidentally.

git submodule foreach 'ssh localhost "ls /" < /dev/null'

I'll also just take this moment to advertise gitslave
(http://gitslave.sf.net) as an alternate to submodules which may help
(or hinder) you--depending on your workflow.  It doesn't suffer from
this particular problem in any case.

					-Seth Robertson