FWIW, I didn't receive it, because you excluded
linux-kernel@vger.kernel.org
me neither :(
Maybe just repost it to LKML ?
Isn't it sufficient to start Cc:ing the list?
You mention the interface *was* designed with cpu features remaining
constant over a process' lifetime. Between the line I'm reading that
this might not be the case anymore.
How could that happen ? Process migration on a different CPU (or perhaps
on a different host) ?
AMX will be shown as enabled in the hardware, but trap into the kernel
on first use. The kernel developers prefer a model where it is checked
that the process has previously enabled the feature explicitly, instead
relying on lazy initialization as part of the trap (as intended by the
hardware design). This means that the usual CPUID/XCR0 approach (which
is reflected in the glibc feature) will not work.
Now it turns out that we can still support this in glibc because of the
pointer indirection, but only if the kernel provides a bit we can read
in thread-specific data.
Damn, how could the cpu designers come up with such weird concepts
in the first place ? :o
It's not the CPU designers. The CPU behaves according to the old model.
(I consider the old model a success, despite all the challenges, but not
everyone agrees, obviosly.)
Thanks,
Florian
FWIW, I didn't receive it, because you excluded
linux-kernel@vger.kernel.org
me neither :(
Maybe just repost it to LKML ?
Isn't it sufficient to start Cc:ing the list?
Well, in that case people probably missed the original mail.
(maybe, I'm too lazy for searching the web for archives ... :P)
quoted
You mention the interface *was* designed with cpu features remaining
constant over a process' lifetime. Between the line I'm reading that
this might not be the case anymore.
How could that happen ? Process migration on a different CPU (or perhaps
on a different host) ?
AMX will be shown as enabled in the hardware, but trap into the kernel
on first use. The kernel developers prefer a model where it is checked
that the process has previously enabled the feature explicitly, instead
relying on lazy initialization as part of the trap (as intended by the
hardware design). This means that the usual CPUID/XCR0 approach (which
is reflected in the glibc feature) will not work.
Ah, now I'm beginning to get it:
* this feature needs to be initialized first, before it can be used
* on first use (when not initialized yet), it traps into the kernel
* we don't want to always initialize it at boot
Correct ?
What I'm wondering: why shall the process explicitly ask for it and
why isn't the initialization be done either on bootup or on first use ?
quoted
Damn, how could the cpu designers come up with such weird concepts
in the first place ? :o
It's not the CPU designers. The CPU behaves according to the old model.
(I consider the old model a success, despite all the challenges, but not
everyone agrees, obviosly.)
I'm still claiming already this old model is a horrible misdesign and
(most of) the extensions made over the decades are anything but well
designed - there had been many changes to do it much, much better.
For example there would have been ways to introduce new opcodes in a way
that they can be easily emulated in kernel or userland, w/o going
through a full trap.
But that's gonna be a long discussion on its own, probably getting
offtopic here.
--mtx
--
---
Hinweis: unverschlüsselte E-Mails können leicht abgehört und manipuliert
werden ! Für eine vertrauliche Kommunikation senden Sie bitte ihren
GPG/PGP-Schlüssel zu.
---
Enrico Weigelt, metux IT consult
Free software and Linux embedded engineering
info@metux.net -- +49-151-27565287
On Wed, Jun 30, 2021 at 02:22:19PM +0200, Enrico Weigelt, metux IT consult wrote:
Ah, now I'm beginning to get it:
* this feature needs to be initialized first, before it can be used
* on first use (when not initialized yet), it traps into the kernel
* we don't want to always initialize it at boot
Correct ?
Not exactly. It's available but comes with a huge context-switch
cost for each task using it.
What I'm wondering: why shall the process explicitly ask for it and
why isn't the initialization be done either on bootup or on first use ?
I'm still claiming already this old model is a horrible misdesign and
(most of) the extensions made over the decades are anything but well
designed - there had been many changes to do it much, much better.
For example there would have been ways to introduce new opcodes in a way
that they can be easily emulated in kernel or userland, w/o going
through a full trap.
It's not a matter of opcodes but of context switch cost which not
everyone wants to inflict to every single task that opportunistically
uses these instructions without realizing what this subsequently
implies for the rest of their life. All this is discussed in the
thread above. I don't remember seeing anybody criticize the choice
of instruction encoding hence it's irrelevant to this discussion.
Hoping this helps,
Willy
From: Arjan van de Ven <hidden> Date: 2021-06-30 13:59:08
On 6/30/2021 5:22 AM, Enrico Weigelt, metux IT consult wrote:
quoted
AMX will be shown as enabled in the hardware, but trap into the kernel
on first use. The kernel developers prefer a model where it is checked
that the process has previously enabled the feature explicitly, instead
relying on lazy initialization as part of the trap (as intended by the
hardware design). This means that the usual CPUID/XCR0 approach (which
is reflected in the glibc feature) will not work.
Ah, now I'm beginning to get it:
* this feature needs to be initialized first, before it can be used
* on first use (when not initialized yet), it traps into the kernel
* we don't want to always initialize it at boot
Correct ?
not really, the init is PER PROCESS
and then there is a per thread 8Kb state allocation that needs to be context switched/etc
once you actually use AMX.
What I'm wondering: why shall the process explicitly ask for it and
why isn't the initialization be done either on bootup or on first use ?
the kernel needs to be able to say "no" in a graceful way, there are several scenarios
(from the sysadmin wanting to manage power/performance/resources to outright compatibility where
the kernel wants or needs to say "no". Most obvious example: if a process asked for an sigaltstack,
we can't let the process use AMX since that stack will be too small most likely to hold
the stackframe)
If you do this on "first use of the instruction" there is no graceful way to say "no".
From: Len Brown <lenb@kernel.org> Date: 2021-06-30 15:21:13
The latest proposal for kernel AMX support (updated today) is here:
https://lore.kernel.org/lkml/20210630060226.24652-1-chang.seok.bae@intel.com/
The main challenge for AMX is not context switch performance.
Hardware recognizes INIT state (the common case) and skips that data
transfer when it is not needed.
The main challenge for AMX is compatibility. Specifically, user
signal stack growth.
The legacy ABI is that we put an uncompacted XSTATE image on the signal stack.
In the default stack case, this isn't a problem, but when a user
allocates an alternative signal stack,
the 8K of XSTATE growth that AMX can exceed what the user allocated.
The new system call tells the kernel that the application can handle it.
(it can do this by not using altsigstack, or by using the updated
stack size advertised
by glibc 2.34 and later, or some other means)
From: Enrico Weigelt, metux IT consult <hidden> Date: 2021-06-30 15:26:16
On 30.06.21 15:55, Arjan van de Ven wrote:
quoted
* this feature needs to be initialized first, before it can be used
* on first use (when not initialized yet), it traps into the kernel
* we don't want to always initialize it at boot
Correct ?
not really, the init is PER PROCESS
IIRC there had been some discussion here whether it should be done per
thread. But now that I've learned that the major problem is saving the
register state, I wouldn't dare thinking about how a working per-thread
solution really would need to look like :o
(by the way: is sighandler stack per thread or per process ?)
the kernel needs to be able to say "no" in a graceful way, there are
several scenarios
(from the sysadmin wanting to manage power/performance/resources to
outright compatibility where
the kernel wants or needs to say "no". Most obvious example: if a
process asked for an sigaltstack,
we can't let the process use AMX since that stack will be too small most
likely to hold
the stackframe)
Ah okay, when I wrote that mail, didn't know yet that so much state
needs to be saved.
--mtx
--
---
Hinweis: unverschlüsselte E-Mails können leicht abgehört und manipuliert
werden ! Für eine vertrauliche Kommunikation senden Sie bitte ihren
GPG/PGP-Schlüssel zu.
---
Enrico Weigelt, metux IT consult
Free software and Linux embedded engineering
info@metux.net -- +49-151-27565287