Re: [PATCH v2 1/5] [ppc] Process dynamic relocations for kernel

On Tue, 2011-10-25 at 17:23 +0530, Suzuki K. Poulose wrote: 

The following patch implements the dynamic relocation processing for
PPC32 kernel. relocate() accepts the target virtual address and relocates
 the kernel image to the same.

Hi Suzuki,

Thanks for the patches.  I've been testing them on a 440-based card, and
encountered TLB error exceptions because the BSS section wasn't getting
properly cleared in early_init().

It turns out that some of the instructions which were modified in
relocate() weren't then getting flushed out of the d-cache into memory.
After that, early_init() executed the stale (non-modified) instructions
for the BSS area.  Those instructions just accessed offset 0 instead of
the actual BSS-related offsets.  That resulted in BSS not getting`
zeroed.

I was able to verify this on my 440 by comparing the d-cache and i-cache
entries for the BSS-accessing instructions in early_init() using a
RISCWatch.  As I suspected, the instructions in the d-cache showed the
corrected offsets, but the i-cache showed the old, non-relocated
offsets.

To fix the issue, I wrote the following patch, applied on top of your
patches.  Suggestions and comments are welcome.

From c88ae39da0c0352f411aca8d9636990a442d47da Mon Sep 17 00:00:00 2001

From: Josh Poimboeuf <redacted>
Date: Wed, 2 Nov 2011 16:41:24 -0500
Subject: [PATCH] Flush relocated instructions from data cache

After updating instructions with relocated addresses, flush them from
the data cache and invalidate the icache line so we don't execute stale
instructions.

Signed-off-by: Josh Poimboeuf <redacted>
---
 arch/powerpc/kernel/reloc_32.S |   11 ++++++++++-
 1 files changed, 10 insertions(+), 1 deletions(-)

diff --git a/arch/powerpc/kernel/reloc_32.S

b/arch/powerpc/kernel/reloc_32.S
index 045d61e..a92857d 100644

--- a/arch/powerpc/kernel/reloc_32.S
+++ b/arch/powerpc/kernel/reloc_32.S

@@ -137,6 +137,9 @@ get_type:
 	lwz	r0, 8(r9)	/* r_addend */
 	add	r0, r0, r3	/* final addend */
 	stwx	r0, r4, r7	/* memory[r4+r7]) = (u32)r0 */
+	dcbst	r4,r7		/* flush dcache line to memory */
+	sync			/* wait for flush to complete */
+	icbi	r4,r7		/* invalidate icache line */
 	b	nxtrela		/* continue */
 
 	/* R_PPC_ADDR16_HI */

@@ -177,6 +180,9 @@ lo16:
 	/* Store half word */
 store_half:
 	sthx	r0, r4, r7	/* memory[r4+r7] = (u16)r0 */
+	dcbst	r4,r7		/* flush dcache line to memory */
+	sync			/* wait for flush to complete */
+	icbi	r4,r7		/* invalidate icache line */
 
 nxtrela:
 	cmpwi	r8, 0		/* relasz = 0 ? */

@@ -185,7 +191,10 @@ nxtrela:
 	subf	r8, r6, r8	/* relasz -= relaent */
 	b	applyrela
 
-done:	blr
+done:
+	sync			/* wait for icache invalidates to complete */
+	isync			/* discard any prefetched instructions */
+	blr
 
 
 p_dyn:		.long	__dynamic_start - 0b

-- 
1.7.4.1