powerpc/bug: Cast to unsigned long before passing to inline asm

Started by Michael Ellerman on Aug 31, 2021 · 5 messages from 3 authors · Last reply Sep 1, 2021

Subsystems (2)

Maintainers (2)

STALE1903d

open the first message on its own page

Message details and related content
In commit 1e688dd2a3d6 ("powerpc/bug: Provide better flexibility to
WARN_ON/__WARN_FLAGS() with asm goto") we changed WARN_ON(). Previously
it would take the warning condition, x, and double negate it before
converting the result to int, and passing that int to the underlying
inline asm. ie:

  #define WARN_ON(x) ({
  	int __ret_warn_on = !!(x);
  	if (__builtin_constant_p(__ret_warn_on)) {
  	...
  	} else {
  		BUG_ENTRY(PPC_TLNEI " %4, 0",
  			  BUGFLAG_WARNING | BUGFLAG_TAINT(TAINT_WARN),
  			  "r" (__ret_warn_on));

The asm then does a full register width comparison with zero and traps
if it is non-zero (PPC_TLNEI).

The new code instead passes the full expression, x, with some unknown
type, to the inline asm:

  #define WARN_ON(x) ({
	...
	do {
		if (__builtin_constant_p((x))) {
		...
		} else {
			...
			WARN_ENTRY(PPC_TLNEI " %4, 0",
				   BUGFLAG_WARNING | BUGFLAG_TAINT(TAINT_WARN),
				   __label_warn_on, "r" (x));

This was not seen to cause any issues with GCC, however with clang in at
least one case it leads to a WARN_ON() that fires incorrectly and
repeatedly at boot, as reported[1] by Nathan:

  WARNING: CPU: 0 PID: 1 at lib/klist.c:62 .klist_add_tail+0x3c/0x110
  Modules linked in:
  CPU: 0 PID: 1 Comm: swapper/0 Tainted: G        W         5.14.0-rc7-next-20210825 #1
  NIP:  c0000000007ff81c LR: c00000000090a038 CTR: 0000000000000000

80 more lines

diff --git a/arch/powerpc/include/asm/bug.h b/arch/powerpc/include/asm/bug.h
index 1ee0f22313ee..02c08d1492f8 100644
--- a/arch/powerpc/include/asm/bug.h
+++ b/arch/powerpc/include/asm/bug.h
@@ -119,7 +119,8 @@ __label_warn_on:						\
 								\
 			WARN_ENTRY(PPC_TLNEI " %4, 0",		\
 				   BUGFLAG_WARNING | BUGFLAG_TAINT(TAINT_WARN),	\
-				   __label_warn_on, "r" (x));	\
+				   __label_warn_on,		\
+				   "r" ((__force long)(x)));	\
 			break;					\
 __label_warn_on:						\
 			__ret_warn_on = true;			\
base-commit: e1ab9a730b426fadc018f91b7c98412473e542fb
prerequisite-patch-id: 942553bda7d83bbae8bf6b2b718033d488ee2410
prerequisite-patch-id: a14c44e671eba8648c4fe385a2552fd57875ec8a
prerequisite-patch-id: 94f5c890f54da2b46f06c60562e879171fab2be3
prerequisite-patch-id: 330af32f2aa34a432d450acc9f6e9fd1cec96417
prerequisite-patch-id: b46c65afa63944f3fb02f4b9bdf940507bb25de6
prerequisite-patch-id: c4ba00ee949f70d7745f75bad11bbb2416f329f1
prerequisite-patch-id: f479601944d0aa615716d5349d93bd6e3d5619c1
prerequisite-patch-id: 9523cde933393b2d68648cecb740efdba9dd8601
prerequisite-patch-id: 034afc97c841a6dcd2b9932406f391d65d18bf87
prerequisite-patch-id: effd7ac8a7db6b59a2677c9c3a7ef8b3ef8bdaf8
prerequisite-patch-id: 23883cf116ee69b452db3c6e10dd49e756e7b5d5
prerequisite-patch-id: 37b6695321c96db466b0faba9308bacfb79c7ced
prerequisite-patch-id: 83420e68ca4476c9ba5a67aa19e1fdc0b6d656a4
prerequisite-patch-id: 362219acf820b78b83c6c09071a636b28976a1ce
prerequisite-patch-id: 857513c5f431887d16a59d193834dcec636c73dc
prerequisite-patch-id: 49f6879a819e205b5361280ab923664fcd29daaf
prerequisite-patch-id: 5a37bcf70c5cb44d78de63a64e5ce920a0a7e419
prerequisite-patch-id: 2c06dd3833117b0498baa198694f6c7e84975840
prerequisite-patch-id: 5794a211ebbf7f0d416ae882443201621c00f615
prerequisite-patch-id: 19ed5ae34e233079c7f66376b8d309cac2b57dbc
prerequisite-patch-id: 1d4c82277473e8dbecf83faf6c4a6788538b064d
prerequisite-patch-id: 8cb5ecc4fe23dafb4a43192f93b669c80a548985
prerequisite-patch-id: 763b8d98c3aefd120862154b94814e3ef3578b5c
prerequisite-patch-id: f45e04e6d030eb157be550976b07dc891fa0836d
prerequisite-patch-id: 07b6fb682675845aca694deff1847bc7a40e1fec
prerequisite-patch-id: 7f1082effa12b1eba445cef90e4749155662888c
prerequisite-patch-id: 76743814dd8e6151c27676ae2e318579d658bf8b
prerequisite-patch-id: 8a6b12c11dbbcd5dda0ccc9877dee1be785e0173
prerequisite-patch-id: e98f013ce41c27d16f75ac3eb1c7eec4235cca0a
prerequisite-patch-id: 285e11f96169ec82702a69b2fca5318c0e307508
prerequisite-patch-id: 9fa89fb9f4ac839177307891bb240009f1d55e88
prerequisite-patch-id: feebaed3f6e0c15e8fa468d64129fe9aa4411d57
prerequisite-patch-id: 8f1093cf40180a623439d82e563e1dd18194cc19
prerequisite-patch-id: d0466662674595d0678e71e5258d55b93d54b5c4
prerequisite-patch-id: 286812aaed6630139583fd21d388137b8d5a6931
prerequisite-patch-id: 54af8aa735a12282bb40a0ed87455e268ae356d9
prerequisite-patch-id: cc5ee85759d99a6ebf18e39634dde65f15476f84
prerequisite-patch-id: 3f8437c8bfda23c45839596ec432d81a95505061
prerequisite-patch-id: f30d6fa2c7c7c417ee4bee0827c0ce587570db34

116 more lines

Message details and related content
On Tue, Aug 31, 2021 at 11:27:20PM +1000, Michael Ellerman wrote:
In commit 1e688dd2a3d6 ("powerpc/bug: Provide better flexibility to
WARN_ON/__WARN_FLAGS() with asm goto") we changed WARN_ON(). Previously
it would take the warning condition, x, and double negate it before
converting the result to int, and passing that int to the underlying
inline asm. ie:

  #define WARN_ON(x) ({
  	int __ret_warn_on = !!(x);
  	if (__builtin_constant_p(__ret_warn_on)) {
  	...
  	} else {
  		BUG_ENTRY(PPC_TLNEI " %4, 0",
  			  BUGFLAG_WARNING | BUGFLAG_TAINT(TAINT_WARN),
  			  "r" (__ret_warn_on));

The asm then does a full register width comparison with zero and traps
if it is non-zero (PPC_TLNEI).

The new code instead passes the full expression, x, with some unknown
type, to the inline asm:

  #define WARN_ON(x) ({
	...
	do {
		if (__builtin_constant_p((x))) {
		...
		} else {
			...
			WARN_ENTRY(PPC_TLNEI " %4, 0",
				   BUGFLAG_WARNING | BUGFLAG_TAINT(TAINT_WARN),
				   __label_warn_on, "r" (x));

This was not seen to cause any issues with GCC, however with clang in at
least one case it leads to a WARN_ON() that fires incorrectly and
repeatedly at boot, as reported[1] by Nathan:

  WARNING: CPU: 0 PID: 1 at lib/klist.c:62 .klist_add_tail+0x3c/0x110
  Modules linked in:
  CPU: 0 PID: 1 Comm: swapper/0 Tainted: G        W         5.14.0-rc7-next-20210825 #1
  NIP:  c0000000007ff81c LR: c00000000090a038 CTR: 0000000000000000

76 more quoted lines

Reviewed-by: Nathan Chancellor <nathan@kernel.org>
Tested-by: Nathan Chancellor <nathan@kernel.org>
quoted hunk
---
 arch/powerpc/include/asm/bug.h | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/arch/powerpc/include/asm/bug.h b/arch/powerpc/include/asm/bug.h
index 1ee0f22313ee..02c08d1492f8 100644
--- a/arch/powerpc/include/asm/bug.h
+++ b/arch/powerpc/include/asm/bug.h
@@ -119,7 +119,8 @@ __label_warn_on:						\
 								\
 			WARN_ENTRY(PPC_TLNEI " %4, 0",		\
 				   BUGFLAG_WARNING | BUGFLAG_TAINT(TAINT_WARN),	\
-				   __label_warn_on, "r" (x));	\
+				   __label_warn_on,		\
+				   "r" ((__force long)(x)));	\
 			break;					\
 __label_warn_on:						\
 			__ret_warn_on = true;			\
-- 
2.25.1
Message details and related content
Hi!

On Tue, Aug 31, 2021 at 11:27:20PM +1000, Michael Ellerman wrote:
Nathan filed an LLVM bug [2], in which Eli Friedman explained that "if
you pass a value of a type that's narrower than a register to an inline
asm, the high bits are undefined". In this case we are passing a bool
to the inline asm, which is a single bit value, and so the compiler
thinks it can leave the high bits of r30 unmasked, because only the
value of bit 0 matters.

Because the inline asm compares the full width of the register (32 or
64-bit) we need to ensure the value passed to the inline asm has all
bits defined. So fix it by casting to long.

We also already cast to long for the similar case in BUG_ENTRY(), which
it turns out was added to fix a similar bug in 2005 in commit
32818c2eb6b8 ("[PATCH] ppc64: Fix issue with gcc 4.0 compiled kernels").
That points to <https://gcc.gnu.org/PR23422>, which shows the correct
explanation.

The code as it was did **not** pass a bool.  It perhaps passed an int
(so many macros in play, it is hard to tell for sure, but it is int or
long int, perhaps unsigned (which does not change anything here).  But
td wants a 64-bit register, not a 32-bit one (which are the only two
possibilities for the "r" constraint on PowerPC).  The cast to "long" is
fine for powerpc64, but it has nothing to do with "narrower than a
register".

If this is not the correct explanation for LLVM, it needs to solve some
other bug.


Segher
Message details and related content
Segher Boessenkool [off-list ref] writes:
On Tue, Aug 31, 2021 at 11:27:20PM +1000, Michael Ellerman wrote:
quoted
Nathan filed an LLVM bug [2], in which Eli Friedman explained that "if
you pass a value of a type that's narrower than a register to an inline
asm, the high bits are undefined". In this case we are passing a bool
to the inline asm, which is a single bit value, and so the compiler
thinks it can leave the high bits of r30 unmasked, because only the
value of bit 0 matters.

Because the inline asm compares the full width of the register (32 or
64-bit) we need to ensure the value passed to the inline asm has all
bits defined. So fix it by casting to long.

We also already cast to long for the similar case in BUG_ENTRY(), which
it turns out was added to fix a similar bug in 2005 in commit
32818c2eb6b8 ("[PATCH] ppc64: Fix issue with gcc 4.0 compiled kernels").
That points to <https://gcc.gnu.org/PR23422>, which shows the correct
explanation.
That's a pity because I don't understand that explanation ^_^

Why does sign extension matter when we're comparing against zero?
The code as it was did **not** pass a bool.  It perhaps passed an int
(so many macros in play, it is hard to tell for sure, but it is int or
long int, perhaps unsigned (which does not change anything here).
I don't understand that. It definitely is passing a bool at the source
level. Are you saying it's getting promoted somehow?

It expands to:

      asm goto(
              "1:   "
              "tdnei"
              "
              " " % 4,
              0 " "\n " ".section __ex_table,\"a\";"
                                              " "
                                              ".balign 4;"
                                              " "
                                              ".long (1b) - . ;"
                                              " "
                                              ".long (%l[__label_warn_on]) - . ;"
                                              " "
                                              ".previous"
                                              " "
                                              ".section __bug_table,\"aw\"\n"
                                              "2:\t.4byte 1b - 2b, %0 - 2b\n"
                                              "\t.short %1, %2\n"
                                              ".org 2b+%3\n"
                                              ".previous\n"
              :
              : "i"("lib/klist.c"), "i"(62),
                "i"((1 << 0) | ((9) << 8)),
                "i"(sizeof(struct bug_entry)),
                "r"(knode_dead(knode))
                ^^^^^^^^^^^^^^^^^^^^^
              :
              : __label_warn_on);

And knode_dead() returns bool:

  static bool knode_dead(struct klist_node *knode)
  {
  	return (unsigned long)knode->n_klist & KNODE_DEAD;

5 more lines

But td wants a 64-bit register, not a 32-bit one (which are the only two
possibilities for the "r" constraint on PowerPC).  The cast to "long" is
fine for powerpc64, but it has nothing to do with "narrower than a
register".
If it's 32-bit vs 64-bit, and the clang explanation is correct, then
we'd expect the low 32-bits of the value passed to the asm to have the
correct value, ie. have been masked with KNODE_DEAD.
If this is not the correct explanation for LLVM, it needs to solve some
other bug.
OK. I just need to get this fixed in the kernel, so I'll do a new
version with a changelog that is ~= "shrug not sure what's going on" and
merge that. Then we can argue about what is really going on later :)

cheeers
Message details and related content
On Wed, Sep 01, 2021 at 05:17:26PM +1000, Michael Ellerman wrote:
Segher Boessenkool [off-list ref] writes:
quoted
On Tue, Aug 31, 2021 at 11:27:20PM +1000, Michael Ellerman wrote:
quoted
Nathan filed an LLVM bug [2], in which Eli Friedman explained that "if
you pass a value of a type that's narrower than a register to an inline
asm, the high bits are undefined". In this case we are passing a bool
to the inline asm, which is a single bit value, and so the compiler
thinks it can leave the high bits of r30 unmasked, because only the
value of bit 0 matters.

Because the inline asm compares the full width of the register (32 or
64-bit) we need to ensure the value passed to the inline asm has all
bits defined. So fix it by casting to long.

We also already cast to long for the similar case in BUG_ENTRY(), which
it turns out was added to fix a similar bug in 2005 in commit
32818c2eb6b8 ("[PATCH] ppc64: Fix issue with gcc 4.0 compiled kernels").
That points to <https://gcc.gnu.org/PR23422>, which shows the correct
explanation.
That's a pity because I don't understand that explanation ^_^

Why does sign extension matter when we're comparing against zero?
The "td" insn wants a 64-bit quantity.  You have to provide one, the
compiler will not do an extend itself, it does not try to understand the
asm template in any way.
quoted
The code as it was did **not** pass a bool.  It perhaps passed an int
(so many macros in play, it is hard to tell for sure, but it is int or
long int, perhaps unsigned (which does not change anything here).
I don't understand that. It definitely is passing a bool at the source
level. Are you saying it's getting promoted somehow?

It expands to:
<snip>
And knode_dead() returns bool:

  static bool knode_dead(struct klist_node *knode)
  {
  	return (unsigned long)knode->n_klist & KNODE_DEAD;
  }

So in my book that means the type there is bool. But I'm not a compiler
guy so I guessing I'm missing something.
I was confused by all the macros and stuff.  And "bool" in the kernel
means "_Bool" now (so it is a character type, with GCC).
quoted
If this is not the correct explanation for LLVM, it needs to solve some
other bug.
OK. I just need to get this fixed in the kernel, so I'll do a new
version with a changelog that is ~= "shrug not sure what's going on" and
merge that. Then we can argue about what is really going on later :)
One thing you should probably do is never pass expressions as asm
operands that are "r".  Instead, make a temporary var and assign to that,
so it will have the type you want, without being able to forget to add
a cast :-)


Segher
Keyboard shortcuts
hback out one level
jnext message in thread / next series-diff position
kprevious message in thread / previous series-diff position
ldrill in
Escclose help / fold thread tree
?toggle this help