Re: [PATCH 3/9] test-mergesort: add test subcommand

René Scharfe [off-list ref] writes:

Repeatable tests are not essential (the original paper didn't mention
seeding), but shouldn't be much trouble to implement and would simplify
comparisons across versions, systems and among testers.

The only downside I can think of is that it may perhaps also simplify
over-fitting, i.e. I might find micro-tweaks that only work for our
specific rand() sequence and then misinterpret them as general
improvements..

Yup, I think you summarized the pros-and-cons nicely.