Right now my major gripe with cogito is "cg-log" (which is actually the
only command I use right now, everything else I just do by hand with the
raw git archive) is that bash is being an ass about SIGPIPE, and when I
only look at the top part of the log, ie I do something like:
torvalds@ppc970:~/src/cogito> git log | head
it spews out the ten first lines of the log, and then it spews about a
million lines (well, 20, but anyway) of crap:
/home/torvalds/bin/cg-log: line 87: 6338 Done echo -n $color$key $rest
6339 Broken pipe | sed "s/>.*/> ${pdate/+0000/$tz}/"
/home/torvalds/bin/cg-log: line 87: 6328 Done cat-file commit $commit
6329 Broken pipe | while read key rest; do
case "$key" in
"author" | "committer")
if [ "$key" = "author" ]; then
color="$colauthor";
else
color="$colcommitter";
fi; date=(${rest#*> }); sec=${date[0]}; tz=${date[1]}; dtz=${tz/+/}; lsec=$(expr $dtz / 100 \* 3600 + $dtz % 100 \* 60 + $sec); pdate="$(date -Rud "1970-01-01 UTC + $lsec sec" 2>/dev/null)"; if [ "$pdate" ]; then
echo -n $color$key $rest | sed "s/>.*/> ${pdate/+0000/$tz}/"; echo $coldefault;
else
echo $color$key $rest $coldefault;
fi
;;
"")
echo; sed -re '
/
*Signed-off-by:.*/Is//'$colsignoff'&'$coldefault'/
s/./ &/
'
;;
*)
echo $colheader$key $rest $coldefault
;;
esac;
done
which is just incredibly annoying, since it makes the ten lines I actually
_wanted_ to get scroll off the screen..
Damn bash. What's the magic incantation that says SHUT UP!?
Linus
Right now my major gripe with cogito is "cg-log" (which is actually
the only command I use right now, everything else I just do by hand
with the raw git archive) is that bash is being an ass about SIGPIPE,
and when I only look at the top part of the log, ie I do something
like:
torvalds@ppc970:~/src/cogito> git log | head
I think you misspelled "cg-log". :-D
Doing "cg-log | head" gives me 10 lines of log and nothing else. Maybe
the problem has been fixed between 0.7 and the current version I'm using
(commit ID 49612c471eebd26efe926a71752e254c1cdc382d)?
Rene
My fingers have a hard time learning new patterns, so I've got:
torvalds@ppc970:~/git> cat ~/bin/git
#!/bin/sh
cmd="cg-$1"
shift
$cmd "$@"
until my fingers learn the new thing.
Doing "cg-log | head" gives me 10 lines of log and nothing else. Maybe
the problem has been fixed between 0.7 and the current version I'm using
(commit ID 49612c471eebd26efe926a71752e254c1cdc382d)?
no, this is current as of an hour ago, same head you have.
It's a bash-specific thing, and depends on how you compiled bash.
Defining "DONT_REPORT_SIGPIPE" in config-top.h when building bash gets rid
of it, but I really don't want to rebuild bash just because of this ;)
Linus
My fingers have a hard time learning new patterns, so I've got:
torvalds@ppc970:~/git> cat ~/bin/git
#!/bin/sh
cmd="cg-$1"
shift
$cmd "$@"
As a workaround, add this line after the shift to suppress the ugly
messages:
[ "$cmd" = cg-log ] && exec 2>/dev/null
Defining "DONT_REPORT_SIGPIPE" in config-top.h when building bash gets rid
of it, but I really don't want to rebuild bash just because of this ;)
Yes, I just compiled it and in the version from gnu.org this defaults to
not being defined. Man, am I glad my distro uncommented that line. :-)
Looking briefly at the source there doesn't seem to be a way to turn it
off besides this compile time setting. I really wonder why..
Rene
From: Ryan Anderson <hidden> Date: 2016-06-15 22:41:55
On Thu, Apr 28, 2005 at 12:21:26PM -0700, Linus Torvalds wrote:
On Thu, 28 Apr 2005, Rene Scharfe wrote:
quoted
I think you misspelled "cg-log". :-D
My fingers have a hard time learning new patterns, so I've got:
torvalds@ppc970:~/git> cat ~/bin/git
#!/bin/sh
cmd="cg-$1"
shift
$cmd "$@"
until my fingers learn the new thing.
quoted
Doing "cg-log | head" gives me 10 lines of log and nothing else. Maybe
the problem has been fixed between 0.7 and the current version I'm using
(commit ID 49612c471eebd26efe926a71752e254c1cdc382d)?
no, this is current as of an hour ago, same head you have.
It's a bash-specific thing, and depends on how you compiled bash.
Defining "DONT_REPORT_SIGPIPE" in config-top.h when building bash gets rid
of it, but I really don't want to rebuild bash just because of this ;)
Debian's bash seems to have that set, so it's a bit hard for me to test
(I don't really have any non-Debian machines around these days for some
reason.)
Try adding "set -e" to the beginning of cg-log.
That may cause some *other* problems, but I think it will fix this
problem.
--
Ryan Anderson
sometimes Pug Majere
Nope. I actually downloaded bash-3.0, and it literally seem sto be
impossible to get bash to not do it.
It has some noise there about signal trapping, but that doesn't actually
seem to work.
Linus
Nope. No difference what-so-ever.
Side note: it's timing-dependent, so apparently especially on SMP,
sometimes it doesn't complain, and then I think soemthing worked. Then I
try again, and it's always back.
I've made a bug-report against bash.
Linus
From: Joshua T. Corbin <hidden> Date: 2016-06-15 22:41:55
On 28 April 2005 14:28, you wrote:
Right now my major gripe with cogito is "cg-log" (which is actually the
only command I use right now, everything else I just do by hand with the
raw git archive) is that bash is being an ass about SIGPIPE, and when I
only look at the top part of the log, ie I do something like:
If cg-log is all you use, then you could get away with using yagf:
rsync://node1.wunjo.org/yagf.git
Features of cg-log missing in yagf log:
* colors
* sigpipe gripes ;)
* #!/bin/bash (or /usr/bin/env bash)
* ability to log between two commits with rev-tree (it's a planned feature
in the near future.)
Cheers,
Josh
From: Paul Jackson <hidden> Date: 2016-06-15 22:41:55
bash is being an ass about SIGPIPE
You have a multiprocessor, don't you.
The following silly little shell script will provoke the bash SIGPIPE
complaint reliably on a multiprocessor. It writes a big file, twice,
from a for-loop in a separate bash subshell through a pipe to a command
that exits after seeing one line.
Code Sample 1:
#!/bin/bash
for x in 1 2
do
cat /etc/termcap # a big text file
done | sed 1q
Adding a right trap _inside_ the shell loop that is _before_ the pipe
will reduce the verbosity of the complaint substantially (not show the
line number and text for each command inside the loop that is killed by
the SIGPIPE; rather just show the simple "Broken pipe" error message):
Code Sample 2:
#!/bin/bash
for x in 1 2
do
trap continue PIPE # reduce broken pipe screeching
cat /etc/termcap # a big text file
done | sed 1q
Then wrapping the entire pipeline (now that the bogus output is a
constant "Broken pipe" string) in the following manner will filter out
just that noise, leaving whatever else was on stdout and/or stderr
unscathed:
Code Sample 3:
#!/bin/bash
( ( (
for x in 1 2
do
trap continue PIPE # reduce broken pipe screeching
cat /etc/termcap # a big text file
done | sed 1q
) 1>&3 ) 2>&1 | grep -vxF 'Broken pipe' 1>&2 ) 3>&1
The following patch to bash jobs.c will enable "Code Sample 2" to do the
right thing, without depending (so much) on the DONT_REPORT_SIGPIPE
compile time flag. With this patch, you don't have to go all the way to
the baroque code in "Code Sample 3" to shut bash up. Just a well placed
trap is sufficient.
Whether or not this is actually worth persuing (or was even worth
reading ;) I don't know.
The following silly little shell script will provoke the bash SIGPIPE
complaint reliably on a multiprocessor.
Yup:
t: line 5: 3853 Broken pipe cat /etc/termcap
t: line 5: 3855 Broken pipe cat /etc/termcap
Adding a right trap _inside_ the shell loop that is _before_ the pipe
will reduce the verbosity of the complaint substantially (not show the
line number and text for each command inside the loop that is killed by
the SIGPIPE; rather just show the simple "Broken pipe" error message):
Code Sample 2:
#!/bin/bash
for x in 1 2
do
trap continue PIPE # reduce broken pipe screeching
cat /etc/termcap # a big text file
done | sed 1q
Didn't change anything for me. Same thing.
Then wrapping the entire pipeline (now that the bogus output is a
constant "Broken pipe" string) in the following manner will filter out
just that noise, leaving whatever else was on stdout and/or stderr
unscathed:
It will also grep out any occurrence of "Broken pipe", so if we're talking
about a kernel changelog, where we fix a pipe bug...
Whether or not this is actually worth persuing (or was even worth
reading ;) I don't know.
I don't know why the bash people have that stupid pipe reporting in the
first place, considering that no other shell seems to do it, and everybody
just wants to shut it up..
Linus
From: Paul Jackson <hidden> Date: 2016-06-15 22:41:55
Linus replied to pj:
quoted
Code Sample 2:
...
Didn't change anything for me. Same thing.
I don't believe you did what I did.
The source code for bash, both 2.x and 3.x versions, clearly displays a
simpler error message (no line number or redisplay of your script
commands) in the case that you set a trap. And I tested both shells on
a multiprocessor, to verify that they behaved as I expected, running
these silly little scripts.
To labour the point, just now on a multiprocessor near me, the following
six line script:
======================== begin ========================
#!/usr/people/pj/etc/bash/bash-3.0/bash
for x in 1 2
do
trap continue PIPE # reduce broken pipe screeching
cat /etc/termcap # a big text file
done | sed 1q
========================= end =========================
produced the following three lines of output:
======================== begin ========================
######## TERMINAL TYPE DESCRIPTIONS SOURCE FILE
Broken pipe
Broken pipe
========================= end =========================
whereas the following five line script (no trap):
======================== begin ========================
#!/usr/people/pj/etc/bash/bash-3.0/bash
for x in 1 2
do
cat /etc/termcap # a big text file
done | sed 1q
========================= end =========================
produced the following three __noisier__ lines of output:
======================== begin ========================
######## TERMINAL TYPE DESCRIPTIONS SOURCE FILE
foo: line 2: 11663 Broken pipe cat /etc/termcap
foo: line 2: 11665 Broken pipe cat /etc/termcap
========================= end =========================
quoted
just that noise, leaving whatever else was on stdout and/or stderr
unscathed:
It will also grep out any occurrence of "Broken pipe", so if we're talking
about a kernel changelog, where we fix a pipe bug...
No no no. You didn't read the code or the comment ;). That or my code
and comment were both unclear ... far more likely.
The following line noise is not plagarized from last years Obfuscated
Perl Contest:
( ( (
...
) 1>&3 ) 2>&1 | grep -vxF 'Broken pipe' 1>&2 ) 3>&1
It's the magic shell incantation to run grep only on the stderr stream,
while passing through the stdout stream untouched. Even on the stderr
stream, it only zaps lines that are exactly the eleven characters
'Broken pipe' (plus newline).
I don't know why the bash people have that stupid pipe reporting in the
Now there we agree. I might speculate that they were trying to get an
early lead in the Linus Git of the Year contest. But this is relatively
mild compared to some of the crap I've seen on other projects I won't
name here. So I too am at a loss to know why.
--
I won't rest till it's the best ...
Programmer, Linux Scalability
Paul Jackson [off-list ref] 1.650.933.1373, 1.925.600.0401
On Fri, Apr 29, 2005 at 11:29:22PM -0700, Paul Jackson wrote:
Linus replied to pj:
quoted
quoted
Code Sample 2:
...
Didn't change anything for me. Same thing.
I don't believe you did what I did.
The source code for bash, both 2.x and 3.x versions, clearly displays a
simpler error message (no line number or redisplay of your script
commands) in the case that you set a trap. And I tested both shells on
a multiprocessor, to verify that they behaved as I expected, running
these silly little scripts.
I don't have a multiprocessor and I see the same. Are you sure it's SMP
dependant?
Your solution (trapping _inside_ the job, too) works for me, btw. Here's
a patch for cg-log that reduces the clutter to two "Broken pipe" lines
(pun not intended).
Rene
From: Herbert Xu <herbert@gondor.apana.org.au> Date: 2016-06-15 22:41:55
Ryan Anderson [off-list ref] wrote:
On Thu, Apr 28, 2005 at 12:21:26PM -0700, Linus Torvalds wrote:
quoted
Defining "DONT_REPORT_SIGPIPE" in config-top.h when building bash gets rid
of it, but I really don't want to rebuild bash just because of this ;)
Debian's bash seems to have that set, so it's a bit hard for me to test
This issue has been around for years. The discussion that led to
Debian setting this option may be helpful in understanding it:
http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=10494
A brief time line:
11 Jun 1997 The issue was reported to Debian.
20 Nov 1999 Chet Ramey remarks that bash's default will not change.
4 Sep 2004 Debian sets DONT_REPORT_SIGPIPE.
Chet Ramey claims that setting DONT_REPORT_SIGPIPE by default
would make bash incompatible with every other shell out there.
Interestingly, all the non-bash shells that I've tried disagree
with him.
Cheers,
--
Visit Openswan at http://www.openswan.org/
Email: Herbert Xu ~{PmV>HI~} [off-list ref]
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
Thanks for the pointer. That discussion points
to some alternative fixes that may be more useful
(instead of installing a repaired shell).
One approach is to install a trap for SIGPIPE in
non-terminating command in a pipeline where the
later items might not process all the data, e.g.:
(trap {} SIGPIPE; find .) | head -1
<rant>
THIS IS A REALLY, REALLY BAD DECISION BY THE BASH TEAM.
Why should the default be "create annoying spurious error report?".
This should at least be a run-time settable option,
with the OPPOSITE default.
</rant>
Which is sad, bash is generally reasonably good.
I guess now I have to say "bash, once properly configured
using DONT_REPORT_SIGPIPE, is reasonably good."
From: Paul Jackson <hidden> Date: 2016-06-15 22:41:55
David wrote:
One approach is to install a trap for SIGPIPE in
non-terminating command in a pipeline where the
later items might not process all the data, e.g.:
(trap {} SIGPIPE; find .) | head -1
Both the versions of bash that I looked at (2.05 and 3.0) _still_
complain even if SIGPIPE is trapped - they just complain with
a more terse message, unless DONT_REPORT_SIGPIPE is not defined.
Linus's version apparently isn't even more terse with this trap.
What bash do you have that this trap silences?
... rant ...
agreed
--
I won't rest till it's the best ...
Programmer, Linux Scalability
Paul Jackson [off-list ref] 1.650.933.1373, 1.925.600.0401
From: Paul Jackson <hidden> Date: 2016-06-15 22:41:55
Rene wrote:
Are you sure it's SMP dependant?
No - I'm not sure. It just happened to be that way on the couple of
systems I looked at (and I figured that in any case, it was a good bet
that Linus had multiple processors ;).
Here's a patch for cg-log
Looks plausible to me.
--
I won't rest till it's the best ...
Programmer, Linux Scalability
Paul Jackson [off-list ref] 1.650.933.1373, 1.925.600.0401
From: Petr Baudis <hidden> Date: 2016-06-15 22:41:55
Dear diary, on Sat, Apr 30, 2005 at 01:04:10PM CEST, I got a letter
where Rene Scharfe [off-list ref] told me that...
On Fri, Apr 29, 2005 at 11:29:22PM -0700, Paul Jackson wrote:
quoted
Linus replied to pj:
quoted
quoted
Code Sample 2:
...
Didn't change anything for me. Same thing.
I don't believe you did what I did.
The source code for bash, both 2.x and 3.x versions, clearly displays a
simpler error message (no line number or redisplay of your script
commands) in the case that you set a trap. And I tested both shells on
a multiprocessor, to verify that they behaved as I expected, running
these silly little scripts.
I don't have a multiprocessor and I see the same. Are you sure it's SMP
dependant?
Your solution (trapping _inside_ the job, too) works for me, btw. Here's
a patch for cg-log that reduces the clutter to two "Broken pipe" lines
(pun not intended).
Could you elaborate on how exactly is it supposed to help? I see
identical behaviour with the traps and without them (UP, bash-2.05b).
--
Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
C++: an octopus made by nailing extra legs onto a dog. -- Steve Taylor
From: Paul Jackson <hidden> Date: 2016-06-15 22:41:56
Could you elaborate on how exactly is it supposed to help?
The key code is in bash/jobs.c.
If you have a bash while or for loop feeding a pipe that shuts down
while the loop is still running commands that try to write the pipe
(perhaps you were pipe'ing to "head -1", and it exit'd, having read its
one line), then the next command to attempt to write that pipe will die,
and the bash instance that is handling that loop (and forked that
command that just died) will notice the command died with a SIGPIPE
signal.
At this point, one of three possible things happens:
1) If your bash is compiled with DONT_REPORT_SIGPIPE defined, then
that bash instance quietly leaves. The concensus around here
is that is "good(tm)."
2) If not so compiled, then:
2a) If you set a trap on SIGPIPE in that shell, it prints:
fprintf (stderr, "%s", j_strsignal (termsig));
2b) Else if you did not trap SIGPIPE, it prints:
fprintf (stderr, "%s: line %d: ", get_name_for_error (),
(line_number == 0) ? 1 : line_number);
pretty_print_job (job, JLIST_NONINTERACTIVE, stderr);
The pretty_print_job() in (2b) can be a multi-line confusion.
Sample output for (2a):
Broken pipe
Sample output in simple one line case for (2b):
foo: line 2: 11663 Broken pipe cat /etc/termcap
A couple of others are reporting different behaviour than what I report
above - including Linus.
So it is almost certain that I don't understand all I know about this.
What behaviour do you see?
--
I won't rest till it's the best ...
Programmer, Linux Scalability
Paul Jackson [off-list ref] 1.650.933.1373, 1.925.600.0401
From: David A. Wheeler <hidden> Date: 2016-06-15 22:41:56
I reported:
quoted
One approach is to install a trap for SIGPIPE in
non-terminating command in a pipeline where the
later items might not process all the data, e.g.:
(trap {} SIGPIPE; find .) | head -1
Paul Jackson wrote:
Both the versions of bash that I looked at (2.05 and 3.0) _still_
complain even if SIGPIPE is trapped - they just complain with
a more terse message, unless DONT_REPORT_SIGPIPE is not defined.
...
What bash do you have that this trap silences?
Actually, I have a working bash, so I can't test any of
these work-arounds. I was merely foolish enough to quote
the Debian discussion, where someone reported that as a
work-around. I had hopes that it would help.
That'll teach me to pay attention to documentation :-).
I wonder, does a top-level trap work? E.g.:
trap "" SIGPIPE
...
Anyone, good luck to those with broken bashes...
I wonder, does a top-level trap work? E.g.:
trap "" SIGPIPE
No.
But putting the traps _inside_ the loops seems to help. So something like
the appended at least makes it somewhat useful
And yes, you need them at both levels, it appears. Or maybe that just
changes the timing enough. Whatever.
Linus
----
Index: cg-log
===================================================================
From: Herbert Xu <herbert@gondor.apana.org.au> Date: 2016-06-15 22:41:56
Linus Torvalds [off-list ref] wrote:
But putting the traps _inside_ the loops seems to help. So something like
the appended at least makes it somewhat useful
It's not the loop that's important, but the subshell where the
signal is caught. Every time you enter a subshell all traps are
reset. Since anything that comes (before or) after a pipe symbol
goes into a subshell...