Re: x86 CPU features detection for applications (and AMX)

On Monday, 28 June 2021 05:40:32 PDT Enrico Weigelt, metux IT consult wrote:

quoted

The first problem is the cross-platformness need. Because we library and
application developers need to support other OSes, we'll need to deploy
our
own CPUID-based detection. It's far better to use common code everywhere,
where one developer working on Linux can fix bugs in FreeBSD, macOS or
Windows or any of the permutations. Every platform-specific deviation
adds to maintenance requirements and is a source of potential latent
bugs, now or in the future due to refactoring. That is why doing
everything in the form of instructions would be far better and easier,
rather than system calls.

hmm, maybe some libcpuid ?

Indeed. I'm querying inside Intel to see if I can get buy-in to create such a 
library.

quoted

The second problem is going to be backwards compatibility. Applications
and
libraries may want to ship precompiled binaries that make use of the new
CPU features, whether they are open source or not.

Since we're talking about GNU libc here, binary-only stuff is probably
out of scope here. OTOH, using differnt libc versions in those special
cases isn't such a big deal.

Shipping a libc is not trivial, either technically or due to licensing 
requirements. Most applications want to link against whatever libc the system 
already provides, if that's possible.

quoted

It comes as no surprise to anyone that we CPU makers will have made
software that use those features and
want to have it ready on Day 1 of the HW being available for the market
(if
we're doing our jobs right). That often involves precompiling because
everyone > who installed their compilers more than one year ago will not
have the necessary tools to build.

Actually, you should talk to the compiler folks much more early, at the
point where you know how those features look like.

We do, but it's not enough.

GCC releases once a year, so it's easy to miss the feature freeze. Then there 
are Linux distros that do LTS every 2 years or so. Worse, those two are 
usually out of phase. For example, if you're using the current Ubuntu LTS 
today (almost July 2021), you're using 20.04, which was released one month 
before the GCC 10 release. So you're using GCC 9, released May 2019, which 
means its features were frozen on December 2018. That's an incredibly long 
lead time.

As a consequence, you will see precompiled binaries.

For using certain new CPU specific features, the need for a compiler
upgrade really should be no excuse. And at least for vast majority of
cases, a proper compiler could do it much better than the average
programmer.

To compile the software that uses those instructions, undoubtedly. But what if 
I did that for you and you could simply download the binary for the library 
and/or plugins such that you could slot into your existing systems and CI? 
This could make a difference between adoption or not.

quoted

And by "recently", I mean "anything since the glibc that came with Red Hat
Enterprise Linux 7" (2.17).

Uh, that's really ancient. Nobody can seriously expect modern features
on such an ancient distro. If people really insist spending so much
money for running such old code, instead of just doing a dist upgrade,
then I can only reply with "not our problem".

Yes and no.

Red Hat has been incredibly successful in backporting kernel features to the 
old 3.10 that came with RHEL 7. Whether they will do that for AMX state saving 
and the system call that we're discussing here, I can't say. AFAIU, they did 
backport the AVX512 state-saving to that 3.10, so they may.

Even if they don't, the *software* that people deploy may be the same build 
for RHEL 7 and for a modern distro that will have a 5.14 kernel. That software 
may have non-AVX, AVX2, AVX512 and AMX-specific code paths and would do 
runtime detection of which one is best to use. If a system call is needed, the 
system call needs to be issued even on that 3.10 and if it responds with 
-ENOSYS or -EINVAL, then it will fall back to the next best option.

So my point is: this shouldn't be in glibc because the glibc will not have the 
new system call wrappers or TLS fields.

What we SW engineers need is an easy and fast method to act depending on
whether some CPU supports some feature (eg. a new opcode). Things like
cpuinfo are only a tiny piece of that. What we could really use is a
conditional jump/call based on whether feature X is supported - without
any kernel intervention. Then the machine code could be easily layed out
to support both cases with our without some feature X. Alternatively we
could have a fast trapping in useland - hw generated call already would
be a big help.

That's what cpuid is for. With GCC function multi-versioning or equivalent 
manually-rolled solutions, you can get exactly what you're asking for.

Yes, the checking became far more complex with the need to check XCR0 after 
AVX came along, but since the instruction itself is a slow and serialising, 
any library will just cache the results. And as a result, the level of CPU 
features is not expected to change. It never has in the past, so this hasn't 
been an issue.

If we had something in that direction, we wouldn't have to have this
kind discussion here anymore - it would be entirely up to compiler and
library folks, no need for any kernel support at all.

For most features, there isn't. You don't see us discussing 
AVX512VP2INTERSECT, for example. This discussion only exists because AMX 
requires more state to be saved during context switches and signal delivery. 
See Peter's email.

And one point that immediately jumps into my mind (w/o looking deeper
into it): it introduces completely new registers - do we now need extra
code for tasks switching etc ?

Yes, this is the crux of this discussion.

-- 
Thiago Macieira - thiago.macieira (AT) intel.com
  Software Architect - Intel DPG Cloud Engineering