Unable to load any kernel modules (compiled with the kernel) in 5.9-rc2?

Hi,

Recently I'm trying to build my own kernel with 64K page size for arm64
to do some testing.

Although the kernel and its modules (plain make -j8 on my RK3399 board,
with native toolchain) are compiled without problem, and the kernel can
boot, it can't load any kernel modules.

I originally thought it's a problem in my env setup or kernel config,
but it turns out that, some distro even has the same bug, namely
Manjaro-ARM (I guess archlinuxarm is also affected).

Is there any known bug related to this problem?
Or how can I further debug the problem?
The dmesg doesn't show any useful info about the unable to load problem,
and insmod/modprobe just shows not-that-meaningful error message like
'exec format error'.

Thanks,
Qu