Re: [PATCH v4] livepatch: Clear relocation targets on a module removal

On Fri, Aug 5, 2022 at 1:58 PM Joe Lawrence [off-list ref] wrote:

On Mon, Aug 01, 2022 at 02:21:29PM -0700, Song Liu wrote:

quoted

From: Miroslav Benes <mbenes@suse.cz>

Josh reported a bug:

  When the object to be patched is a module, and that module is
  rmmod'ed and reloaded, it fails to load with:

  module: x86/modules: Skipping invalid relocation target, existing value is nonzero for type 2, loc 00000000ba0302e9, val ffffffffa03e293c
  livepatch: failed to initialize patch 'livepatch_nfsd' for module 'nfsd' (-8)
  livepatch: patch 'livepatch_nfsd' failed for module 'nfsd', refusing to load module 'nfsd'

  The livepatch module has a relocation which references a symbol
  in the _previous_ loading of nfsd. When apply_relocate_add()
  tries to replace the old relocation with a new one, it sees that
  the previous one is nonzero and it errors out.

  On ppc64le, we have a similar issue:

  module_64: livepatch_nfsd: Expected nop after call, got e8410018 at e_show+0x60/0x548 [livepatch_nfsd]
  livepatch: failed to initialize patch 'livepatch_nfsd' for module 'nfsd' (-8)
  livepatch: patch 'livepatch_nfsd' failed for module 'nfsd', refusing to load module 'nfsd'

He also proposed three different solutions. We could remove the error
check in apply_relocate_add() introduced by commit eda9cec4c9a1
("x86/module: Detect and skip invalid relocations"). However the check
is useful for detecting corrupted modules.

We could also deny the patched modules to be removed. If it proved to be
a major drawback for users, we could still implement a different
approach. The solution would also complicate the existing code a lot.

We thus decided to reverse the relocation patching (clear all relocation
targets on x86_64). The solution is not
universal and is too much arch-specific, but it may prove to be simpler
in the end.

Reported-by: Josh Poimboeuf <redacted>
Signed-off-by: Miroslav Benes <mbenes@suse.cz>
Signed-off-by: Song Liu <song@kernel.org>

---

NOTE: powerpc code has not be tested.

Hi Song,

I just want to provide a quick check in on this patch...

First -- what tree / commit should this be based on?  When I add this
patch on top of a v5.19 based tree, I see:

arch/powerpc/kernel/module_64.c: In function ‘clear_relocate_add’:
arch/powerpc/kernel/module_64.c:781:52: error: incompatible type for argument 1 of ‘instr_is_relative_link_branch’
  781 |                 if (!instr_is_relative_link_branch(*instruction))
      |                                                    ^~~~~~~~~~~~
      |                                                    |
      |                                                    u32 {aka unsigned int}
In file included from arch/powerpc/kernel/module_64.c:20:
./arch/powerpc/include/asm/code-patching.h:122:46: note: expected ‘ppc_inst_t’ but argument is of type ‘u32’ {aka ‘unsigned int’}
  122 | int instr_is_relative_link_branch(ppc_inst_t instr);
      |                                   ~~~~~~~~~~~^~~~~
arch/powerpc/kernel/module_64.c:785:32: error: ‘PPC_INST_NOP’ undeclared (first use in this function); did you mean ‘PPC_INST_COPY’?
  785 |                 *instruction = PPC_INST_NOP;
      |                                ^~~~~~~~~~~~
      |                                PPC_INST_COPY
arch/powerpc/kernel/module_64.c:785:32: note: each undeclared identifier is reported only once for each function it appears in
make[2]: *** [scripts/Makefile.build:249: arch/powerpc/kernel/module_64.o] Error 1
make[1]: *** [scripts/Makefile.build:466: arch/powerpc/kernel] Error 2
make: *** [Makefile:1849: arch/powerpc] Error 2

I am sorry that I didn't build the PPC code. (I did fix some code, but
I guess that's
not enough. ) I was hoping kernel test bot to run build tests on the
patch, but I
guess the bot is not following live-patching mail list?

The code was based Linus' tree, probably 5.19-rc7.

Second, I rebased the klp-convert-tree on top of v5.19 here:
https://github.com/joe-lawrence/klp-convert-tree/tree/klp-convert-v7-devel

and I can confirm that at least the x86_64 livepatching selftests
(including the klp-relocation tests added by this tree) do pass.  I
haven't had a chance to try writing new tests to verify this specific
patch, but I'll take a look next week.

I also got the selftests pass for another patch. Checking dmesg is
a little tricky, btw. I will take a look at klp-convert.

Thanks,
Song