Let me quote the original mail (I had to dig it out of the archives as
well, no nice threading, too lazy, sorry):
It still doesn't say why. I did see a reference to fleeting reference
to <stdatomic.h> and <float.h>.
My conjecture is that the real reason is avoid atomic emulation
(softatomic?) and softfloat code. It's not related to <stdarg.h> at
all: this header is replaced so that GCC's include subdirectory can be
dropped from the include search path. What I don't know if this is to
avoid obscure linker failures related to libatomic/softfloat (obviously
not great) or run-time failures (worse).
In any case, it would be nice to know what the real motivation is.
After all, <stdatomic.h> is exactly like <stdarg.h> in that it's
possible to use its functionality even without the header file. The
__atomic builtins are even documented in the GCC manual (unlike
<stdatomic.h>), which is why some programmers prefer them over the
standard interface. And then there's the _Atomic keyword itself, whose
use can easily result in calls to libatomic functions, too. So blocking
<stdatomic.h> makes little sense to me.
I don't know enough about softfloat if blocking the inclusion of
<float.h> is worth it.
Thanks,
Florian
On Sat, Sep 04, 2021 at 05:19:21PM +0200, Florian Weimer wrote:
* Segher Boessenkool:
quoted
Let me quote the original mail (I had to dig it out of the archives as
well, no nice threading, too lazy, sorry):
It still doesn't say why. I did see a reference to fleeting reference
to <stdatomic.h> and <float.h>.
Yeah... I dug out the actual patch from linux-kbuild:
https://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild.git/commit/?h=kbuild-v5.15&id=c0891ac15f0428ffa81b2e818d416bdf3cb74ab6
The reasoning in there is completely wrong. <stdarg.h> is not a
"userspace header". Instead, it is a standard header, required for some
functionality in C.
It also says
"GPL 2 version of <stdarg.h> can be extracted from
http://archive.debian.org/debian/pool/main/g/gcc-4.2/gcc-4.2_4.2.4.orig.tar.gz"
which seems to suggest you cannot use stuff from GPLv3-licensed GCC.
This is just wrong. The header in question says
"""
Under Section 7 of GPL version 3, you are granted additional
permissions described in the GCC Runtime Library Exception, version
3.1, as published by the Free Software Foundation.
"""
And <https://gcc.gnu.org/git/?p=gcc.git;a=blob;f=COPYING.RUNTIME>
reads in part
"""
1. Grant of Additional Permission.
You have permission to propagate a work of Target Code formed by
combining the Runtime Library with Independent Modules, even if such
propagation would otherwise violate the terms of GPLv3, provided that
all Target Code was generated by Eligible Compilation Processes. You
may then convey such a combination under terms of your choice,
consistent with the licensing of the Independent Modules.
"""
which says that if you compile with GCC, then it is perfectly fine if it
uses the standard C headers, it does not make your work GPL-licenced.
After all, <stdatomic.h> is exactly like <stdarg.h> in that it's
possible to use its functionality even without the header file. The
__atomic builtins are even documented in the GCC manual (unlike
<stdatomic.h>), which is why some programmers prefer them over the
standard interface. And then there's the _Atomic keyword itself, whose
use can easily result in calls to libatomic functions, too. So blocking
<stdatomic.h> makes little sense to me.
I don't know enough about softfloat if blocking the inclusion of
<float.h> is worth it.
Blocking the use of <float.h> is pretty useless: it is possible to do
millions of things in the kernel source that are frowned upon, or
actively bad, or anything in between or more extreme. That is what code
review is for. If it would be a common mistake (it is not afaik) you
can warn for it from checkpatch.pl or something.
The patch is just re-implementing part of the standard GCC <stdarg.h>,
so that it will only work with recent GCC (and maybe clang as well if it
implements the GCC internal interfaces correctly (i.e. compatibly) here,
and the same for other compilers). Almost all of the <stdarg.h> GCC
itself uses is the same, but it also is compatible to the various C
standards if this header is included indirectly. That is all just some
ifdeffery anyway, so doesn't influence compilation times noticeably, and
all that.
- * -
So as far as I can see the motivation behind the patch is a) a
misunderstanding of what standard C headers are, are for, etc.; and b)
a misunderstanding of the GPL and the GCC runtime exception to it. The
patch makes things worse than they were. If on the contrary Linux would
use *more* standard compiler headers, say <stddef.h>, then insidious
bugs like that fixed by c46bbf5d2def would be prevented.
Segher
On Sat, Sep 4, 2021 at 8:19 AM Florian Weimer [off-list ref] wrote:
In any case, it would be nice to know what the real motivation is.
I don't know about the original motivation, but the reason I like that
patch after-the-fact is that I've actually been in situations where I
test out self-built compilers without installing them.
Then it's convenient to have a completely standalone kernel tree.
Nobody cares about things like <stdatomic.h> They are completely
irrelevant for the kernel, exactly because we've always just done our
own, or used __builtin_xyz() for things.
<stdarg.h> is the only actual core header file that we have always
taken from the installed system headers - because historically it
really was required. Everything else we can just do ourselves.
Linus
On Sat, Sep 04, 2021 at 10:22:25AM -0700, Linus Torvalds wrote:
On Sat, Sep 4, 2021 at 8:19 AM Florian Weimer [off-list ref] wrote:
quoted
In any case, it would be nice to know what the real motivation is.
I don't know about the original motivation, but the reason I like that
patch after-the-fact is that I've actually been in situations where I
test out self-built compilers without installing them.
Then you probably know that that is quite hard to do correctly. And
installing the compiler (into a temporary dir that you can just blow
away after you are done with it) is trivial (see for example
<https://gcc.gnu.org/install/finalinstall.html> line 11 or so).
Also, the standard headers are *part of* the compiler. Including those
(like <stdarg.h> here) that are required for freestanding
implementations.
Then it's convenient to have a completely standalone kernel tree.
Yes, and you certainly do not want to mix in anything userspace,
accidentally or not. But these "freestanding" headers already have to
make sure they do not do that, and they are used by many OSes and
embedded-style things, so you can have some confidence that this
actually works.
And the C standard requires you to use these headers to use some
features of the C language (variable arguments in this specific case).
You can of course mimic whatever some implementation does and hope that
will work on some other implementations, and on future versions of that
first implementation as well. But that is imo not a good idea, just
using the standard headers (which are required, and are there on all
implementations I have heard of, without bugs) is a much simpler idea,
much more future-proof, much less maintenance.
Nobody cares about things like <stdatomic.h> They are completely
irrelevant for the kernel, exactly because we've always just done our
own, or used __builtin_xyz() for things.
The Linux kernel needs (or wants) stronger primitives than provided
there, yeah, so you could not even use it as the underlying
implementation for the semantics the kernel wants (which it could do
with <stdint.h> to implement u64 etc.)
<stdarg.h> is the only actual core header file that we have always
taken from the installed system headers - because historically it
really was required. Everything else we can just do ourselves.
You also need <stdint.h> and <stddef.h>. These are much simpler to
implement of course, but see for example the c46bbf5d2def commit I
mentioned before. Some of these compiler headers might have been buggy
(or non-existent) historically, but that is very long ago, you really
can expect at least C99 to be there (and work correctly) nowadays :-)
Segher
On Sat, Sep 4, 2021 at 12:18 PM Segher Boessenkool
[off-list ref] wrote:
And the C standard requires you to use these headers to use some
features of the C language
That's really the point here: we don't care AT ALL.
The C standard headers are simply not relevant for the kernel. Never
have been, never will be.
We care about the low-level compiler infrastructure, not the standard C headers.
Those standards are designed for - and try to cater to - a completely
different audience.
We do atomics, and we do not care at all about the mis-designed C++
standard atomics.
We do our own type system, and again, we don't care at all about the C
"official" type system and odd name rules that change from one version
to the other.
That has always been the case.
We generally *cannot* use the system header files, because they bring
in things that the kernel simply cannot handle.
That's entirely obvious for things like <stdio.h>, but it's actually
true even for other things.
You also need <stdint.h> and <stddef.h>.
No, you really don't.
We avoid those intentionally, and always have. Because the system
header files have never been a good match.
Now, <stdarg.h> is actually special for the kernel, exactly because
unlike other header files, that one really ended up being something
that historically wasn't exposed in any other way and wasn't doable
sanely inside the kernel.
But it does look like gcc and clang have both ended up exposing the
interfaces the same way, using the __buildin_xyz model to just wrap
the standard names in a namespace-clean way.
That really wasn't historically true. Back in the bad old days of
varargs etc, you literally had "va_list" be a "char *" and the
accessor functions were magic macros that updated things by hand by
the size of the arguments etc...
So <stdarg.h> is historically very very special, and tied to compiler
implementation details.
In ways that a lot of other "standard C header files" very much are
*not* - many of those are about the types that the system provides.
And in fact often the kernel is the *source* and definition of parts of them.
Linus
On Sat, Sep 04, 2021 at 12:58:50PM -0700, Linus Torvalds wrote:
On Sat, Sep 4, 2021 at 12:18 PM Segher Boessenkool
[off-list ref] wrote:
quoted
And the C standard requires you to use these headers to use some
features of the C language
That's really the point here: we don't care AT ALL.
The C standard headers are simply not relevant for the kernel. Never
have been, never will be.
Yes they are. It is the single standard way to get the functionality
you want.
These are not library headers. These headers are provided by the
compiler itself. (The one exception is limits.h, but you have no use
for that in the kernel anyway).
We care about the low-level compiler infrastructure, not the standard C headers.
I don't know why you think you can separate that. Take <stdarg.h> --
there is no other (portable, standard) way to implement receiving
variadic arguments.
Yes you can mimic what GCC currently does in its <stdarg.h>, and hope
that keeps working on later versions of GCC, and on all older compiler
versions you care about, and that it works on other compilers as well.
Those standards are designed for - and try to cater to - a completely
different audience.
"""
This International Standard specifies the form and establishes the
interpretation of programs expressed in the programming language C.
Its purpose is to promote portability, reliability, maintainability,
and efficient execution of C language programs on a variety of
computing systems.
"""
Is the kernel not written in C? The C standard is not the POSIX
standard, or anything like it.
We do our own type system, and again, we don't care at all about the C
"official" type system and odd name rules that change from one version
to the other.
<stdint.h> has existed since C99, and has not relevantly changed since.
And you *do* care about this feature, deeply. That is why the kernel
has u8, for the exact same reason standard C has uint8_t: to get exact
width integer types. The kernel could just include the standard header
and then
typedef uint8_t u8;
instead of the dance it currently goes through.
We generally *cannot* use the system header files, because they bring
in things that the kernel simply cannot handle.
Like?
These header files are part of the compiler. The compiler is required
to provide these to be a C compiler. Without using those header files
there is no portable way to use some important features of C.
These are not library headers. These are not headers provided by some
third party. These headers are an intricate part of the compiler
itself.
quoted
You also need <stdint.h> and <stddef.h>.
No, you really don't.
We avoid those intentionally, and always have. Because the system
header files have never been a good match.
Linux re-implements all of it. It *does* need it, it just has its own
implementation if it. It needs the features it provides.
But it does look like gcc and clang have both ended up exposing the
interfaces the same way, using the __buildin_xyz model to just wrap
the standard names in a namespace-clean way.
That really wasn't historically true. Back in the bad old days of
varargs etc, you literally had "va_list" be a "char *" and the
accessor functions were magic macros that updated things by hand by
the size of the arguments etc...
Before C90 there was <varargs.h> instead, and that was like that indeed.
The standard C <stdarg.h> header *cannot* be implemented in terms of
other building blocks in C. This is the underlying theme of all
standard C headers.
In ways that a lot of other "standard C header files" very much are
*not* - many of those are about the types that the system provides.
And in fact often the kernel is the *source* and definition of parts of them.
That is not true at all. These headers in no way depend on Linux or any
other OS. Some of it is mandated by the C standard, some of it is
mandated by the low-level ABI used (the size of an int for example, that
kind of thing). But none of it is from any kernel.
I get that long ago in the dark ages Linux had to implement many things
from scratch. And that implementing things from scratch is more fun
anyway. Also, everything seems to work now, tinkering with it is not
necessarily the best idea.
But that does not mean that if you had to start over you should not use
the standard C headers, or that it would be a bad idea to use them more
often now.
Segher
On Mon, Sep 6, 2021 at 8:50 AM Segher Boessenkool
[off-list ref] wrote:
Yes they are. It is the single standard way to get the functionality
you want.
Sorry, but 'standard' is the part that the kernel simply doesn't do.
Just face it. The kernel isn't written with all those standards that
are designed for entirely different things in mind.
I don't know why you think you can separate that. Take <stdarg.h> --
there is no other (portable, standard) way to implement receiving
variadic arguments.
You clearly haven't actually read the thread.
That was my whole argument. For the _historical_ situation.
Linus
On Mon, Sep 06, 2021 at 09:30:01AM -0700, Linus Torvalds wrote:
On Mon, Sep 6, 2021 at 8:50 AM Segher Boessenkool
[off-list ref] wrote:
quoted
Yes they are. It is the single standard way to get the functionality
you want.
Sorry, but 'standard' is the part that the kernel simply doesn't do.
Just face it. The kernel isn't written with all those standards that
are designed for entirely different things in mind.
The kernel is written in C, and C is meant in part for implementing
things like kernels.
When I say "standard C" I mean "without the GNU C extensions". All C
compilers implement standard C. Only GCC implements all the GNU C
extensions, and implements most of them correctly.
quoted
I don't know why you think you can separate that. Take <stdarg.h> --
there is no other (portable, standard) way to implement receiving
variadic arguments.
That was my whole argument. For the _historical_ situation.
But this whole thread is about removing uses of <stdarg.h>, and
eventually removing the
-nostdinc -isystem $(shell $(CC) -print-file-name=include)
thing (except in some places where it is next to impossible to
reimplement the compiler's implementation (instead of just very hard
and useless work, making all this more fragile in the process)).
That is not the past, that is the future. A bleak future, if technical
decisions are based on FUD like "compiler headers are dangerous".
Segher
On Mon, Sep 6, 2021 at 10:30 AM Segher Boessenkool
[off-list ref] wrote:
But this whole thread is about removing uses of <stdarg.h>, and
eventually removing the
-nostdinc -isystem $(shell $(CC) -print-file-name=include)
Yes.
But your argument against it was based on that past argument of it
being compiler-specific, in ways that are not true any more.
Happily, it clearly is the case that compilers have standardized their
internal implementation of this in ways that means that we actually
_could_ do this for <stdarg.h>.
Exactly the way we do it for pretty much every other header file.
IOW, <stdarg.h> is no longer the special thing it used to be.
We use the compiler intrinsics without the C library header files for
everything else, so doing so for <stdarg.h> seems to actually be a
clarification and improvement.
Linus