On Sun, 8 Nov 2026 at 09:54, Johannes Sixt [off-list ref] wrote:
Am 07.11.26 um 15:59 schrieb Kristofer Karlsson via GitGitGadget:
quoted
Several tests create fixture data consisting of a single repeated
character with "test-tool genzeros <count> | tr '\000' <char>". This
works well on platforms where tr(1) is implemented efficiently.
The implementation of tr(1) on macOS supports multi-byte characters,
and processes its input one character at a time through
getwchar() and putwchar(), even in the C locale. For small inputs
that does not matter, but t4205 generates more than 2GB of data this
way, which makes that test much slower on macOS than on Linux: on the
macOS CI runners, tr(1) alone spends 95 to 123 seconds of CPU time on
it, compared to about 6 seconds on Linux.
Fix this by adding "test-tool genbytes <byte> [<count>]", which writes
the requested byte directly, and turn "genzeros" into a thin
wrapper around the same code.
All cases that do not need a particular character can use
printf %0.500d 0
Though, on my Linux box,
printf %0.2147483649d 0
produces nothing, but
printf 000000000%0.2147483640d 0
has 2147483649 bytes of output.
If this isn't insanely slow on macOS, use this pattern instead of a
highly engineered helper program? ;-)
-- Hannes
Nice trick, and it would work for t4205, which does not care about
the byte value. I tried it for the 2147483649 bytes there on Linux:
printf time peak memory
dash 6.5s 2.1GB
bash 6.0s 3MB
coreutils 3.7s 2.1GB
It is definitely fast enough (on the variants I tested).
The annoying thing (but not a real blocker) is the extra memory
usage required and I am not sure I fully trust that all
implementations will behave the same for absurdly large padding.
(Particularly for paddings larger than 2^31, as you noted too.)
It feels a bit like a hack, and tweaking the existing test-tool
feels like a more sane option that is easier to reason about.
A gotcha here is that the printf hack only works for producing
"0", so for the cases we truly need NUL characters it does not
help us, and I think that is also a point in favor of
simply generalizing test-tool genzeros instead.
Thanks,
Kristofer