On Sat, Sep 4, 2021 at 7:06 AM Markus Mayer [off-list ref] wrote:

Hi,

We are running into build issues with some external kernel modules if
the GNUmake version is 3.x and the kernel is recent(-ish). The culprit
seems to be the sub-make invocation when GNUmake < 4 is detected:
    https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=688931a5ad4e

The module build works fine for older kernels (those not containing
this patch) even with GNUmake 3.81. And it works fine with GNUmake >=
4 with any kernel, even those with the above patch. In other words, if
the sub-make invocation doesn't take place (either because make is new
enough or because the kernel doesn't have the version check yet), our
module build does work as intended.

Due to how the build is integrated with other components, we need to
be calling "make -e" to have environment variables take precedence,
which seems to be another piece of the puzzle. The problem doesn't
seem to be happening without "-e".

The ingredients for our problem, therefore, seem to be:
    * old GNUmake (<4)
    * newish kernel  (>=5.1)
    * run make -e

I should also mention that the kernel module is being cross-compiled
for ARM and ARM64, although that does not seem to be playing a role
here. In my example below, I was using the native compiler.

The problem we are observing is that the contents of $(M) and
$(CFLAGS_MODULE) are lost during the extra sub-make invocation that is
conditional upon GNUmake < 4. There might be other lost variables,
too, but we haven't noticed any effects if there are.

Losing $(M) causes the build to go off the rails completely. This is
easily detected and can be worked around by setting $(KBUILD_EXTMOD)
directly and foregoing $(M) on the command line altogether. The loss
of $(CFLAGS_MODULE) is a little more insidious, since the build does
succeed even when it's empty. However, required defines remain unset
despite being set in the top-level makefile. The resulting kernel
module doesn't work (which can lead to a lot of head scratching). I
also don't know of a way of working around losing CFLAGS_MODULE's
contents.

I sometimes test GNU make 3.81 for kernel builds, but I have not tested
the -e option.

Now I tested the -e option, and it worked for me.
Both $(M) and $(KBUILD_EXTMOD) were correctly set.

So, I did not observe anything you claim.

I was able to reproduce the loss of $(M) quite easily doing something like this:

obj-m += hello.o

all:
        ${MAKE} -C $(KERNEL_DIR) -e M=$(PWD) modules

clean:
        make -C $(KERNEL_DIR) M=$(PWD) clean

I ran this Makefile with GNU Make 3.81


masahiro@oscar:~/workspace/hello$ cat Makefile
obj-m += hello.o

KERNEL_DIR := $(HOME)/ref/linux

all:
        ${MAKE} -C $(KERNEL_DIR) -e M=$(PWD) modules

clean:
        make -C $(KERNEL_DIR) M=$(PWD) clean
masahiro@oscar:~/workspace/hello$ make-3.81 --version
GNU Make 3.81
Copyright (C) 2006  Free Software Foundation, Inc.
This is free software; see the source for copying conditions.
There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A
PARTICULAR PURPOSE.

This program built for x86_64-unknown-linux-gnu
masahiro@oscar:~/workspace/hello$ make-3.81
make-3.81 -C /home/masahiro/ref/linux -e
M=/home/masahiro/workspace/hello modules
make-3.81[1]: Entering directory `/home/masahiro/ref/linux'
make-3.81[2]: Entering directory `/home/masahiro/ref/linux'
  CC [M]  /home/masahiro/workspace/hello/hello.o
  MODPOST /home/masahiro/workspace/hello/Module.symvers
  CC [M]  /home/masahiro/workspace/hello/hello.mod.o
  LD [M]  /home/masahiro/workspace/hello/hello.ko
make-3.81[2]: Leaving directory `/home/masahiro/ref/linux'
make-3.81[1]: Leaving directory `/home/masahiro/ref/linux'
masahiro@oscar:~/workspace/hello$ ls hello*
hello.c  hello.ko  hello.mod  hello.mod.c  hello.mod.o  hello.o





hello.ko was successfully built.


Entering/Leaving directory is eye-sores,
but presumably it is because MAKEFLAGS is overridden
by the environment since you gave -e.

I do not understand your motivation for using -e, though.

Instead of building the out-of-tree hello.ko, which we are asking it
to do, it'll go off and build all the in-kernel modules instead. Since
it sees $(M) as empty, it just executes "make modules".

Unfortunately, I have NOT been successful reproducing losing the
contents of $(CFLAGS_MODULE) in a simple test environment. In my
tests, it was always retained. Nonetheless, in the actual build
environment, it does get lost. And only in the combination of new-ish
kernel and old-ish make, i.e. whenever the sub-make invocation happens
due to the make version.

BTW, commenting out the make version test does make our module build
work. So, it is definitely related to that code snippet. (Of course,
building on a reasonably recent Linux distro also makes everything
work, but that isn't possible for us in all circumstances.)

Do you have any thoughts on this or any pointers? Is there a way this
issue could be resolved? It does seem like the version check has some
unintended side-effects, even if the scenario in which one would come
across them is fairly uncommon and most developers will never
experience them.

I am willing to try out any suggestions or provide further information
if needed.

Regards,
-Markus

-- 
Best Regards
Masahiro Yamada