Re: data loss when doing ls-remote and piped to command

Am Mittwoch, 15. September 2021, 20:17:42 CEST schrieb Junio C Hamano:

Rolf Eike Beer [off-list ref] writes:

quoted

The given repository is a clone of the vanilla kernel.

/usr/bin/git --git-dir=/home/ebeer/repos/upstream/linux/.git ls-remote
origin 2>&1 | less

And I then see things like this:

6f38b5d6cfd43dde3058a10c68baae9cf17af912        refs/tags/v5.0-rc2
1c7fc5cbc33980acd13ae83d0b416db002fe95601e7f97f64b59514d936    
refs/tags/v5.7-rc2^{} d0709bb6da2ab6d49b11643e98abdf79b1a2817f       
refs/tags/v5.7-rc3

Not offering any solution, just an observation of the problem and
annotating the report.

What we see on the second line is the beginning of peeled
v5.0-rc2^{} up to the "acd13" (that is, the first 19 bytes of the
line), followed by the full line for peeled v5.7-rc2^{} (which
begins with "ae83d").  12407 bytes in between are missing, which
is even more puzzling as it is not a nice round number.

I wonder if this is "less" misconfigured and misbehaving.  Did the
user after seeing v5.7-* tags scroll back with 'b' or something?

To quote myself:

quoted

This is not bound to less, we originally observed the error on a 
entirely different tool that tried to parse the output of ls-remote.

In fact when less opened I just started to scroll down until I visually 
noticed an error.

If the output (including the 2>&1 redirection) is sent to a file and
then "cat <that-file" is invoked, does the same thing happen?  How
about "cat <that-file | less"?

The redirection seems to be an important part of it. I now did:

git ... 2>&1 | sha256sum

This gives different results basically on every run. I also noticed that 
having more tags makes it easier to reproduce, so a stable kernel in contrast 
to vanilla is a better trigger. Doing that without the stderr redirection gave 
the same result every time I tried.

Regards,

Eike
-- 
Rolf Eike Beer, emlix GmbH, http://www.emlix.com
Fon +49 551 30664-0, Fax +49 551 30664-11
Gothaer Platz 3, 37083 Göttingen, Germany
Sitz der Gesellschaft: Göttingen, Amtsgericht Göttingen HR B 3160
Geschäftsführung: Heike Jordan, Dr. Uwe Kracke – Ust-IdNr.: DE 205 198 055

emlix - smart embedded open source