On Wed, Mar 26, 2025 at 2:56 PM Arnd Bergmann [off-list ref] wrote:

On Wed, Mar 26, 2025, at 07:07, Guo Ren wrote:

quoted

On Tue, Mar 25, 2025 at 9:18 PM Arnd Bergmann [off-list ref] wrote:

quoted

On Tue, Mar 25, 2025, at 13:26, Peter Zijlstra wrote:

quoted

On Tue, Mar 25, 2025 at 08:15:41AM -0400, guoren@kernel.org wrote:

You declare the syscall ABI to be the native 64-bit ABI, but this
is fundamentally not true because a many uapi structures are
defined in terms of 'long' or pointer values, in particular in
the ioctl call.

I modified uapi with
void __user *msg_name;
->
union {void __user *msg_name; u64 __msg_name;};
to make native 64-bit ABI.

I would look at compat stuff instead of using __riscv_xlen macro.

The problem I see here is that there are many more drivers
that you did not modify than drivers that you did change this
way.  The union is particularly ugly, but even if you find
a nicer method of doing this, you now also put the burden
on future driver writers to do this right for your platform.

Got it.

quoted

quoted

As far as I can tell, there is no way to rectify this design flaw
other than to drop support for 64-bit userspace and only support
regular rv32 userspace. I'm also skeptical that supporting rv64
userspace helps in practice other than for testing, since
generally most memory overhead is in userspace rather than the
kernel, and there is much more to gain from shrinking the larger
userspace by running rv32 compat mode binaries on a 64-bit kernel
than the other way round.

The lp64-abi userspace rootfs works fine in this patch set, which
proves the technique is valid. But the modification on uapi is raw,
and I'm looking at compat stuff.

There is a big difference between making it work for a particular
set of userspace binaries and making it correct for the entire
kernel ABI.

I agree that limiting the hacks to the compat side while keeping
the native ABI as ilp32 as in your previous versions is better,
but I also don't think this can be easily done without major
changes to how compat mode works in general, and that still
seems like a show-stopper for two reasons:

- it still puts the burden on driver writers to get it right
  for your platform. The scope is a bit smaller than in the
  current version because that would be limited to the compat
  handlers and not change the native codepath, but that's
  still a lot of drivers.

- the way that I would imagine this to be implemented in
  practice would require changing the compat code in a way that
  allows multiple compat ABIs, so drivers can separate the
  normal 32-on-64 handling from the 64-on-32 version you need.
  We have discussed something like this in the past, but Linus
  has already made it very clear that he doesn't want it done
  that way. Whichever way you do it, this is unlikely to
  find consensus.

Got it, thanks for analysing.

quoted

Supporting lp64-abi userspace is essential because riscv lp64-abi and
ilp32-abi userspace are hybrid deployments when the target is
ilp32-abi userspace. The lp64-abi provides a good supplement to
ilp32-abi which eases the development.

I'm not following here, please clarify. I do understand that
having a mixed 32/64 userspace can help for development, but
that can already be done on a 64-bit kernel and it doesn't
seem to be useful for deployment because having two sets of
support libraries makes this counterproductive for the goal
of saving RAM.

In my case, most binaries and libraries are based on 32-bit, but a
small part would remain on 64-bit, which may be statically linked.
For RISC-V, the rv64 ecosystem is more complete than the rv32's. So,
rv64-abi is always necessary, and rv32-abi is a supplement.

quoted

quoted

If you remove the CONFIG_64BIT changes that Peter mentioned and
the support for ilp64 userland from your series, you end up
with a kernel that is very similar to a native rv32 kernel
but executes as rv64ilp32 and runs rv32 userspace. I don't have
any objections to that approach, and the same thing has come
up on arm64 as a possible idea as well, but I don't know if
that actually brings any notable advantage over an rv32 kernel.

Are there CPUs that can run rv64 kernels and rv32 userspace
but not rv32 kernels, similar to what we have on Arm Cortex-A76
and Cortex-A510?

Yes, there is, and it only supports rv32 userspace, not rv32 kernel.
https://www.xrvm.com/product/xuantie/C908

Ok, thanks for the link.

       Arnd

-- 
Best Regards
 Guo Ren