Thread (8 messages) 8 messages, 6 authors, 13d ago

Stop compressing manual pages (was: Bug#1123959: manpages: Please consider shipping uncompressed man pages)

From: Alejandro Colomar <alx@kernel.org>
Date: 2025-12-25 14:15:07

Hi,

On Thu, Dec 25, 2025 at 02:47:33PM +0100, Dr. Tobias Quathamer wrote:
Am 25.12.25 um 12:20 schrieb Alejandro Colomar:
quoted
Hello Helge, Tobias,

On Thu, Dec 25, 2025 at 06:07:57AM +0000, Helge Kreutzmann wrote:
quoted
Hello Tobias,
if you look at mansect(1), the example given does not work in Debian.
I reported this upstream and got the following reply:
quoted
The issue is that Debian compresses manual pages.  Please consider
changing the policy to not compress manual pages.  The storage savings
are irrelevant in this age.
Could you consider this?
Thanks!

Indeed, compressed manual pages are a pain to work with.  You can't use
regular Unix tools to work with them.  With uncompressed manual pages,
You can go to /usr/share/man, and run a pipe of programs to do a complex
search.  With tools like zgrep(1) and zcat(1), you can do some stuff,
but not everything.
Hi Helge and Alex,

thanks for your bug report and the provided statistics. I haven't thought
about this up until now, because it violates Debian Policy. Quoting from
Section 12.1
(https://www.debian.org/doc/debian-policy/ch-docs.html#manual-pages):

"Manual pages should be installed compressed using gzip -9."

And regarding the terminology using the word "should", this is defined in
section 1.1 (https://www.debian.org/doc/debian-policy/ch-scope.html#scope):

"The terms should and should not, and the adjective recommended, denote best
practices. Non-conformance with these guidelines will generally be
considered a bug, but will not necessarily render a package unsuitable for
distribution. These statements correspond to bug severities of important,
normal, and minor. They are collectively called Policy recommendations."

So by not compressing the man pages, the Debian package would introduce a
bug. Moreover, I'd have to explicitely opt out of automatic compression in
the build stage of the package.

All of this is doable, of course. But I'm a bit hesitant with just making
the switch, given that the manpages package is certainly the package with
the most man pages in the Debian ecosystem -- by a large margin.

So it might be better to discuss the pros and cons in a broader audience,
trying to understand why the compression has been chosen initially. Maybe
only due to disk space limitations back then, but maybe there are other
reasons as well -- which might still be valid today.
Yup, I'd like that policy to change.  I've added debian-policy@ to this
mail (and also linux-man@).

For those reading only since this email, please have a look at
<https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1123959>, which
itself also references an discussion in the Linux man-pages project from
a couple of years ago:
<https://lore.kernel.org/linux-man/c8cf5be0-04e7-f0a1-179f-eada6182c33e@gmail.com/T/#m272e6ee8939d0836999dd8bb28f2e0e94f48dfc7 (local)>.

I'll paste again the numbers:

        $ sudo make install-man prefix=/opt/local/man/gz__1 -j LINK_PAGES=symlink Z=.gz  GZIPFLAGS=-1  | wc -l
        2571
        $ sudo make install-man prefix=/opt/local/man/gz__9 -j LINK_PAGES=symlink Z=.gz  GZIPFLAGS=-9  | wc -l
        2571
        $ sudo make install-man prefix=/opt/local/man/man__ -j LINK_PAGES=symlink Z=                   | wc -l
        2571

        $ du -sh /opt/local/man/*
        5.7M    /opt/local/man/gz__1
        5.5M    /opt/local/man/gz__9
        5.5M    /opt/local/man/gz___
        9.4M    /opt/local/man/man__

        $ export MANPATH=/opt/local/man/gz__1/share/man
        $ /bin/time -f %e dash -c "man -Kaw RLIMIT_NOFILE | wc -l | xargs printf '%s; '"
        17; 0.21
        $ /bin/time -f %e dash -c "find $MANPATH -type f | while read f; do gzip -d - <\$f | grep -l RLIMIT_NOFILE >/dev/null && echo \$f; done | wc -l | xargs printf '%s; '"
        17; 1.16

        $ export MANPATH=/opt/local/man/gz__9/share/man
        $ /bin/time -f %e dash -c "man -Kaw RLIMIT_NOFILE | wc -l | xargs printf '%s; '"
        17; 0.20
        $ /bin/time -f %e dash -c "find $MANPATH -type f | while read f; do gzip -d - <\$f | grep -l RLIMIT_NOFILE >/dev/null && echo \$f; done | wc -l | xargs printf '%s; '"
        17; 1.17

        $ export MANPATH=/opt/local/man/man__/share/man
        $ /bin/time -f %e dash -c "man -Kaw RLIMIT_NOFILE | wc -l | xargs printf '%s; '"
        17; 0.55
        $ /bin/time -f %e dash -c "find $MANPATH -type f | xargs -P0 grep -l RLIMIT_NOFILE | wc -l | xargs printf '%s; '"
        17; 0.01

Using uncompressed manual pages is both faster and simpler, by orders of
magnitude, when doing complex searches with pipelines.  In the simple
cases where man(1) is enough, the speed is in the same order of
magnitude.

Also, the compression only cuts storage by half, so not even an order of
magnitude.  In this age, where storage is relatively cheap, systems that
have manual pages installed most likely have room enough for the
uncompressed pages.

Please change Debian policy 12.1 ("Manual pages") to recommend
uncompressed pages.


Have a lovely day!
Alex
Regards,
Tobias
-- 
<https://www.alejandro-colomar.es>

Attachments

Keyboard shortcuts
hback out one level
jnext message in thread
kprevious message in thread
ldrill in
Escclose help / fold thread tree
?toggle this help