This series is a follow on to the series posted earlier (archived at [1]).
Patches 01/22 .. 09/22 of this series have been pretty intensively reviewed;
thanks to all who helped. We've modified per feedback, and these should be in
reasonable shape.
Patches 10/22 .. 19/22 of this series have not been very widely reviewed.
We'd very much appreciate eyeballs here.
Patches 20/22 .. 22/22 of this series are specific to the TI Keystone platform.
These are not ready to be merged in. These are being provided here for the sake
of completeness, and to better illustrate the other patches in this series.
These are dependent on the smpops patches (see [2]).
These patches are also available on the following git repository:
git://arago-project.org/git/projects/linux-keystone.git keystone-v2
[1] - http://thread.gmane.org/gmane.linux.kernel/1336081
[2] - http://permalink.gmane.org/gmane.linux.ports.arm.kernel/171540
Series changelog:
(01/22) ARM: add mechanism for late code patching
(v2) pulled runtime patching code into separate source files
(v2) reordered arguments to patch macros for consistency with assembly
"Rd, Rt, imm" ordering
(v2) added support for mov immediate patching
(v2) cache flush patched instructions instead of entire kernel code
(v2) pack patch table to reduce table volume
(v2) add to module vermagic to reflect abi change
(v2) misc. cleanups in naming and structure
(02/22) ARM: add self test for runtime patch mechanism
(v2) added init-time tests to verify instruction encoding
(03/22) ARM: use late patch framework for phys-virt patching
(v2) move __pv_offset and __pv_phys_offset to C code
(v2) restore conditional init of __pv_offset and __pv_phys_offset
(04/22) ARM: LPAE: use phys_addr_t on virt <--> phys conversion
(v2) fix patched __phys_to_virt() to use 32-bit operand
(v2) convert non-patch __phys_to_virt and __virt_to_phys to inlines to retain
type checking
(05/22) ARM: LPAE: support 64-bit virt_to_phys patching
(v2) use phys_addr_t instead of split high/low phys_offsets
(v2) use mov immediate instead of add to zero when patching in high order
physical address bits
(v2) fix __pv_phys_offset handling for big-endian
(v2) remove set_phys_offset()
(06/22) ARM: LPAE: use signed arithmetic for mask definitions
(07/22) ARM: LPAE: use phys_addr_t in alloc_init_pud()
(08/22) ARM: LPAE: use phys_addr_t in free_memmap()
(v2) unchanged from v1
(09/22) ARM: LPAE: use phys_addr_t for initrd location and size
(v2) revert to unsigned long for initrd size
(10/22) ARM: LPAE: use phys_addr_t in switch_mm()
(v2) use phys_addr_t instead of u64 in switch_mm()
(v2) revert on changes to v6 and v7-2level
(v2) fix register mapping for big-endian in v7-3level
(11/22) ARM: LPAE: use 64-bit accessors for TTBR registers
(v2) restore comment in cpu_set_reserved_ttbr0()
(12/22) ARM: LPAE: define ARCH_LOW_ADDRESS_LIMIT for bootmem
(13/22) ARM: LPAE: factor out T1SZ and TTBR1 computations
(v2) unchanged from v1
(14/22) ARM: LPAE: accomodate >32-bit addresses for page table base
(v2) apply arch_pgd_shift only on lpae
(v2) move arch_pgd_shift definition to asm/memory.h
(v2) revert on changes to non-lpae procs
(v2) add check to ensure that the pgd physical address is aligned at an
ARCH_PGD_SHIFT boundary
(15/22) ARM: mm: use physical addresses in highmem sanity checks
(16/22) ARM: mm: cleanup checks for membank overlap with vmalloc area
(17/22) ARM: mm: clean up membank size limit checks
(18/22) ARM: add virt_to_idmap for interconnect aliasing
(v2) unchanged from v1
(19/22) ARM: recreate kernel mappings in early_paging_init()
(v2) disable on !lpae at compile time
arch/arm/Kconfig | 36 ++++
arch/arm/Makefile | 1 +
arch/arm/boot/dts/keystone-sim.dts | 77 +++++++
arch/arm/configs/keystone_defconfig | 23 +++
arch/arm/include/asm/mach/arch.h | 1 +
arch/arm/include/asm/memory.h | 94 ++++++---
arch/arm/include/asm/module.h | 7 +
arch/arm/include/asm/page.h | 2 +-
arch/arm/include/asm/pgtable-3level-hwdef.h | 10 +
arch/arm/include/asm/pgtable-3level.h | 6 +-
arch/arm/include/asm/proc-fns.h | 28 ++-
arch/arm/include/asm/runtime-patch.h | 175 ++++++++++++++++
arch/arm/kernel/Makefile | 1 +
arch/arm/kernel/armksyms.c | 4 -
arch/arm/kernel/head.S | 107 +++-------
arch/arm/kernel/module.c | 7 +-
arch/arm/kernel/runtime-patch.c | 230 +++++++++++++++++++++
arch/arm/kernel/setup.c | 18 ++
arch/arm/kernel/smp.c | 11 +-
arch/arm/kernel/vmlinux.lds.S | 13 +-
arch/arm/mach-keystone/Makefile | 2 +
arch/arm/mach-keystone/Makefile.boot | 1 +
arch/arm/mach-keystone/include/mach/debug-macro.S | 44 ++++
arch/arm/mach-keystone/include/mach/memory.h | 47 +++++
arch/arm/mach-keystone/include/mach/timex.h | 21 ++
arch/arm/mach-keystone/include/mach/uncompress.h | 24 +++
arch/arm/mach-keystone/keystone.c | 124 +++++++++++
arch/arm/mach-keystone/keystone.h | 23 +++
arch/arm/mach-keystone/platsmp.c | 84 ++++++++
arch/arm/mm/context.c | 9 +-
arch/arm/mm/idmap.c | 4 +-
arch/arm/mm/init.c | 19 +-
arch/arm/mm/mmu.c | 114 +++++++---
arch/arm/mm/proc-v7-3level.S | 50 +++--
34 files changed, 1214 insertions(+), 203 deletions(-)
create mode 100644 arch/arm/boot/dts/keystone-sim.dts
create mode 100644 arch/arm/configs/keystone_defconfig
create mode 100644 arch/arm/include/asm/runtime-patch.h
create mode 100644 arch/arm/kernel/runtime-patch.c
create mode 100644 arch/arm/mach-keystone/Makefile
create mode 100644 arch/arm/mach-keystone/Makefile.boot
create mode 100644 arch/arm/mach-keystone/include/mach/debug-macro.S
create mode 100644 arch/arm/mach-keystone/include/mach/memory.h
create mode 100644 arch/arm/mach-keystone/include/mach/timex.h
create mode 100644 arch/arm/mach-keystone/include/mach/uncompress.h
create mode 100644 arch/arm/mach-keystone/keystone.c
create mode 100644 arch/arm/mach-keystone/keystone.h
create mode 100644 arch/arm/mach-keystone/platsmp.c
--
1.7.9.5
This patch redefines the early boot time use of the R4 register to steal a few
low order bits (ARCH_PGD_SHIFT bits) on LPAE systems. This allows for up to
38-bit physical addresses.
Signed-off-by: Cyril Chemparathy <redacted>
Signed-off-by: Vitaly Andrianov <redacted>
---
arch/arm/include/asm/memory.h | 15 +++++++++++++++
arch/arm/kernel/head.S | 10 ++++------
arch/arm/kernel/smp.c | 11 +++++++++--
arch/arm/mm/proc-v7-3level.S | 8 ++++++++
4 files changed, 36 insertions(+), 8 deletions(-)
@@ -306,6 +306,7 @@ __create_page_tables:#endif#ifdef CONFIG_ARM_LPAEsubr4,r4,#0x1000 @ point to the PGD table+movr4,r4,lsr#ARCH_PGD_SHIFT#endifmovpc,lrENDPROC(__create_page_tables)
@@ -379,7 +380,7 @@ __secondary_data:*r0=cp#15 control register*r1=machineID*r2=atagsordtbpointer-*r4=pagetablepointer+*r4=pagetable (seeARCH_PGD_SHIFTinasm/memory.h)*r9=processorID*r13=*virtual*addresstojumptouponcompletion*/
This patch cleans up the highmem sanity check code by simplifying the range
checks with a pre-calculated size_limit. This patch should otherwise have no
functional impact on behavior.
This patch also removes a redundant (bank->start < vmalloc_limit) check, since
this is already covered by the !highmem condition.
Signed-off-by: Cyril Chemparathy <redacted>
Signed-off-by: Vitaly Andrianov <redacted>
---
arch/arm/mm/mmu.c | 19 +++++++++++--------
1 file changed, 11 insertions(+), 8 deletions(-)
This patch replaces the original physical offset patching implementation
with one that uses the newly added patching framework. In the process, we now
unconditionally initialize the __pv_phys_offset and __pv_offset globals in the
head.S code.
Signed-off-by: Cyril Chemparathy <redacted>
---
arch/arm/Kconfig | 1 +
arch/arm/include/asm/memory.h | 26 +++--------
arch/arm/kernel/armksyms.c | 4 --
arch/arm/kernel/head.S | 95 +++++++----------------------------------
arch/arm/kernel/module.c | 5 ---
arch/arm/kernel/setup.c | 12 ++++++
arch/arm/kernel/vmlinux.lds.S | 5 ---
7 files changed, 36 insertions(+), 112 deletions(-)
@@ -511,92 +511,29 @@ ENDPROC(fixup_smp)#ifdef CONFIG_ARM_PATCH_PHYS_VIRT-/*__fixup_pv_table-patchthestubinstructionswiththedeltabetween-*PHYS_OFFSETandPAGE_OFFSET,whichisassumedtobe16MiBalignedand-*canbeexpressedbyanimmediateshifteroperand.Thestubinstruction-*hasaformof'(add|sub) rd, rn, #imm'.+/*+*__fixup_pv_offsets-update__pv_offsetand__pv_phys_offsetbasedonthe+*runtimelocationofthekernel.*/__HEAD-__fixup_pv_table:+__fixup_pv_offsets:adrr0,1f-ldmiar0,{r3-r5,r7}+ldmiar0,{r3-r6}subr3,r0,r3@PHYS_OFFSET-PAGE_OFFSET-addr4,r4,r3@adjusttablestartaddress-addr5,r5,r3@adjusttableendaddress-addr7,r7,r3@adjust__pv_phys_offsetaddress-strr8,[r7]@savecomputedPHYS_OFFSETto__pv_phys_offset-movr6,r3,lsr#24 @ constant for add/sub instructions-teqr3,r6,lsl#24 @ must be 16MiB aligned-THUMB(itne@crosssectionbranch)-bne__error-strr6,[r7,#4] @ save to __pv_offset-b__fixup_a_pv_table-ENDPROC(__fixup_pv_table)+addr4,r4,r3@virt_to_phys(__pv_phys_offset)+addr5,r5,r3@virt_to_phys(__pv_offset)+addr6,r6,r3@virt_to_phys(PAGE_OFFSET)=PHYS_OFFSET+strr6,[r4]@save__pv_phys_offset+strr3,[r5]@save__pv_offset+movpc,lr+ENDPROC(__fixup_pv_offsets).align1:.long.-.long__pv_table_begin-.long__pv_table_end-2:.long__pv_phys_offset--.text-__fixup_a_pv_table:-#ifdef CONFIG_THUMB2_KERNEL-lslsr6,#24-beq2f-clzr7,r6-lsrr6,#24-lslr6,r7-bicr6,#0x0080-lsrsr7,#1-orrcsr6,#0x0080-orrr6,r6,r7,lsl#12-orrr6,#0x4000-b2f-1:addr7,r3-ldrhip,[r7,#2]-andip,0x8f00-orrip,r6@maskinoffsetbits31-24-strhip,[r7,#2]-2:cmpr4,r5-ldrccr7,[r4],#4 @ use branch for delay slot-bcc1b-bxlr-#else-b2f-1:ldrip,[r7,r3]-bicip,ip,#0x000000ff-orrip,ip,r6@maskinoffsetbits31-24-strip,[r7,r3]-2:cmpr4,r5-ldrccr7,[r4],#4 @ use branch for delay slot-bcc1b-movpc,lr+.long__pv_phys_offset+.long__pv_offset+.longPAGE_OFFSET#endif-ENDPROC(__fixup_a_pv_table)--ENTRY(fixup_pv_table)-stmfdsp!,{r4-r7,lr}-ldrr2,2f@getaddressof__pv_phys_offset-movr3,#0 @ no offset-movr4,r0@r0=tablestart-addr5,r0,r1@r1=tablesize-ldrr6,[r2,#4] @ get __pv_offset-bl__fixup_a_pv_table-ldmfdsp!,{r4-r7,pc}-ENDPROC(fixup_pv_table)-.align-2:.long__pv_phys_offset--.data-.globl__pv_phys_offset-.type__pv_phys_offset,%object-__pv_phys_offset:-.long0-.size__pv_phys_offset,.-__pv_phys_offset-__pv_offset:-.long0-#endif#include "head-common.S"
From: Nicolas Pitre <hidden> Date: 2012-08-12 03:03:13
On Fri, 10 Aug 2012, Cyril Chemparathy wrote:
This patch replaces the original physical offset patching implementation
with one that uses the newly added patching framework. In the process, we now
unconditionally initialize the __pv_phys_offset and __pv_offset globals in the
head.S code.
What was wrong with the former PHYS_OFFSET = __pv_phys_offset ?
If you really want to have it optimized at run time, you could simply
use your new stub to patch a mov instruction instead of going through
__virt_to_phys which uses and add on top of a constant.
quoted hunk
static inline unsigned long __virt_to_phys(unsigned long x)
{
unsigned long t;
- __pv_stub(x, t, "add", __PV_BITS_31_24);
+ early_patch_imm8("add", t, x, __pv_offset, 0);
return t;
}
static inline unsigned long __phys_to_virt(unsigned long x)
{
unsigned long t;
- __pv_stub(x, t, "sub", __PV_BITS_31_24);
+ early_patch_imm8("sub", t, x, __pv_offset, 0);
return t;
}
#else
@@ -511,92 +511,29 @@ ENDPROC(fixup_smp)#ifdef CONFIG_ARM_PATCH_PHYS_VIRT-/*__fixup_pv_table-patchthestubinstructionswiththedeltabetween-*PHYS_OFFSETandPAGE_OFFSET,whichisassumedtobe16MiBalignedand-*canbeexpressedbyanimmediateshifteroperand.Thestubinstruction-*hasaformof'(add|sub) rd, rn, #imm'.+/*+*__fixup_pv_offsets-update__pv_offsetand__pv_phys_offsetbasedonthe+*runtimelocationofthekernel.*/__HEAD-__fixup_pv_table:+__fixup_pv_offsets:adrr0,1f-ldmiar0,{r3-r5,r7}+ldmiar0,{r3-r6}subr3,r0,r3@PHYS_OFFSET-PAGE_OFFSET-addr4,r4,r3@adjusttablestartaddress-addr5,r5,r3@adjusttableendaddress-addr7,r7,r3@adjust__pv_phys_offsetaddress-strr8,[r7]@savecomputedPHYS_OFFSETto__pv_phys_offset-movr6,r3,lsr#24 @ constant for add/sub instructions-teqr3,r6,lsl#24 @ must be 16MiB aligned-THUMB(itne@crosssectionbranch)-bne__error-strr6,[r7,#4] @ save to __pv_offset-b__fixup_a_pv_table-ENDPROC(__fixup_pv_table)+addr4,r4,r3@virt_to_phys(__pv_phys_offset)+addr5,r5,r3@virt_to_phys(__pv_offset)+addr6,r6,r3@virt_to_phys(PAGE_OFFSET)=PHYS_OFFSET+strr6,[r4]@save__pv_phys_offset+strr3,[r5]@save__pv_offset+movpc,lr+ENDPROC(__fixup_pv_offsets).align1:.long.-.long__pv_table_begin-.long__pv_table_end-2:.long__pv_phys_offset--.text-__fixup_a_pv_table:-#ifdef CONFIG_THUMB2_KERNEL-lslsr6,#24-beq2f-clzr7,r6-lsrr6,#24-lslr6,r7-bicr6,#0x0080-lsrsr7,#1-orrcsr6,#0x0080-orrr6,r6,r7,lsl#12-orrr6,#0x4000-b2f-1:addr7,r3-ldrhip,[r7,#2]-andip,0x8f00-orrip,r6@maskinoffsetbits31-24-strhip,[r7,#2]-2:cmpr4,r5-ldrccr7,[r4],#4 @ use branch for delay slot-bcc1b-bxlr-#else-b2f-1:ldrip,[r7,r3]-bicip,ip,#0x000000ff-orrip,ip,r6@maskinoffsetbits31-24-strip,[r7,r3]-2:cmpr4,r5-ldrccr7,[r4],#4 @ use branch for delay slot-bcc1b-movpc,lr+.long__pv_phys_offset+.long__pv_offset+.longPAGE_OFFSET#endif-ENDPROC(__fixup_a_pv_table)--ENTRY(fixup_pv_table)-stmfdsp!,{r4-r7,lr}-ldrr2,2f@getaddressof__pv_phys_offset-movr3,#0 @ no offset-movr4,r0@r0=tablestart-addr5,r0,r1@r1=tablesize-ldrr6,[r2,#4] @ get __pv_offset-bl__fixup_a_pv_table-ldmfdsp!,{r4-r7,pc}-ENDPROC(fixup_pv_table)-.align-2:.long__pv_phys_offset--.data-.globl__pv_phys_offset-.type__pv_phys_offset,%object-__pv_phys_offset:-.long0-.size__pv_phys_offset,.-__pv_phys_offset-__pv_offset:-.long0-#endif#include "head-common.S"
I missed this in the previous patch, but could you fail the module
loading by returning an error if runtime_patch() fails? That would take
care of not accepting modules that might have been compiled with future
runtime patching extensions that are not yet supported in an earlier
kernel.
You also should remove the MODULE_ARCH_VERMAGIC_P2V definitions now that
the corresponding code is no longer there.
Nicolas
This patch replaces the original physical offset patching implementation
with one that uses the newly added patching framework. In the process, we now
unconditionally initialize the __pv_phys_offset and __pv_offset globals in the
head.S code.
This last sentence is now wrong.
Removed.
[...]
quoted
-extern unsigned long __pv_phys_offset;
-#define PHYS_OFFSET __pv_phys_offset
[...]
quoted
+#define PHYS_OFFSET __virt_to_phys(PAGE_OFFSET)
What was wrong with the former PHYS_OFFSET = __pv_phys_offset ?
If you really want to have it optimized at run time, you could simply
use your new stub to patch a mov instruction instead of going through
__virt_to_phys which uses and add on top of a constant.
The intent was to optimize out the load(s) on references to PHYS_OFFSET,
but is it worth it? If so, we could go with a patched mov (or two) with
the necessary endian fixups. If not, we could revert to
__pv_phys_offset loads as before.
[...]
quoted
s = find_mod_section(hdr, sechdrs, ".runtime.patch.table");
if (s)
runtime_patch((void *)s->sh_addr, s->sh_size);
I missed this in the previous patch, but could you fail the module
loading by returning an error if runtime_patch() fails? That would take
care of not accepting modules that might have been compiled with future
runtime patching extensions that are not yet supported in an earlier
kernel.
Sure. Thanks.
You also should remove the MODULE_ARCH_VERMAGIC_P2V definitions now that
the corresponding code is no longer there.
Hmm...
"rt-patch" needs to be in vermagic to prevent modules built against the
new code from being loaded on older kernels that used the traditional
patch code.
"p2v" needs to be in there as well, because it should be possible to
build without PATCH_PHYS_VIRT, but with RUNTIME_PATCH as and when there
are other users for this.
Thanks
-- Cyril.
From: Nicolas Pitre <hidden> Date: 2012-08-13 03:32:48
On Sun, 12 Aug 2012, Cyril Chemparathy wrote:
On 08/11/12 23:03, Nicolas Pitre wrote:
quoted
On Fri, 10 Aug 2012, Cyril Chemparathy wrote:
quoted
-extern unsigned long __pv_phys_offset;
-#define PHYS_OFFSET __pv_phys_offset
[...]
quoted
quoted
+#define PHYS_OFFSET __virt_to_phys(PAGE_OFFSET)
What was wrong with the former PHYS_OFFSET = __pv_phys_offset ?
If you really want to have it optimized at run time, you could simply
use your new stub to patch a mov instruction instead of going through
__virt_to_phys which uses and add on top of a constant.
The intent was to optimize out the load(s) on references to PHYS_OFFSET, but
is it worth it? If so, we could go with a patched mov (or two) with the
necessary endian fixups. If not, we could revert to __pv_phys_offset loads as
before.
If you want to do better than the load, then you'd better go all the way
with the patched move.
quoted
You also should remove the MODULE_ARCH_VERMAGIC_P2V definitions now that
the corresponding code is no longer there.
Hmm...
"rt-patch" needs to be in vermagic to prevent modules built against the new
code from being loaded on older kernels that used the traditional patch code.
Right.
"p2v" needs to be in there as well, because it should be possible to build
without PATCH_PHYS_VIRT, but with RUNTIME_PATCH as and when there are other
users for this.
That doesn't matter if there are other users. As soon as there is a
.init.runtime_patch_table that needs to be processed then "rt-patch"
flags it. Whether this is used for PATCH_PHYS_VIRT or other purposes is
irrelevant.
However there isn't any pv_table anymore, so "p2v" should go as there is
no more code to process those tables if they're ever encountered.
Nicolas
This patch adds a step in the init sequence, in order to recreate the kernel
code/data page table mappings prior to full paging initialization. This is
necessary on LPAE systems that run out of a physical address space outside the
4G limit. On these systems, this implementation provides a machine descriptor
hook that allows the PHYS_OFFSET to be overridden in a machine specific
fashion.
Signed-off-by: Cyril Chemparathy <redacted>
Signed-off-by: Vitaly Andrianov <redacted>
---
arch/arm/include/asm/mach/arch.h | 1 +
arch/arm/kernel/setup.c | 3 ++
arch/arm/mm/mmu.c | 65 ++++++++++++++++++++++++++++++++++++++
3 files changed, 69 insertions(+)
This patch adds an architecture defined override for ARCH_LOW_ADDRESS_LIMIT.
On PAE systems, the absence of this override causes bootmem to incorrectly
limit itself to 32-bit addressable physical memory.
Signed-off-by: Cyril Chemparathy <redacted>
Signed-off-by: Vitaly Andrianov <redacted>
---
arch/arm/include/asm/memory.h | 2 ++
1 file changed, 2 insertions(+)
This patch modifies the highmem sanity checking code to use physical addresses
instead. This change eliminates the wrap-around problems associated with the
original virtual address based checks, and this simplifies the code a bit.
The one constraint imposed here is that low physical memory must be mapped in
a monotonically increasing fashion if there are multiple banks of memory,
i.e., x < y must => pa(x) < pa(y).
Signed-off-by: Cyril Chemparathy <redacted>
Signed-off-by: Vitaly Andrianov <redacted>
---
arch/arm/mm/mmu.c | 22 ++++++++++------------
1 file changed, 10 insertions(+), 12 deletions(-)
From: Nicolas Pitre <hidden> Date: 2012-08-12 04:29:58
On Fri, 10 Aug 2012, Cyril Chemparathy wrote:
This patch modifies the highmem sanity checking code to use physical addresses
instead. This change eliminates the wrap-around problems associated with the
original virtual address based checks, and this simplifies the code a bit.
The one constraint imposed here is that low physical memory must be mapped in
a monotonically increasing fashion if there are multiple banks of memory,
i.e., x < y must => pa(x) < pa(y).
Signed-off-by: Cyril Chemparathy <redacted>
Signed-off-by: Vitaly Andrianov <redacted>
This patch modifies the switch_mm() processor functions to use phys_addr_t.
On LPAE systems, we now honor the upper 32-bits of the physical address that
is being passed in, and program these into TTBR as expected.
Signed-off-by: Cyril Chemparathy <redacted>
Signed-off-by: Vitaly Andrianov <redacted>
---
arch/arm/include/asm/proc-fns.h | 4 ++--
arch/arm/mm/proc-v7-3level.S | 26 ++++++++++++++++++++++----
2 files changed, 24 insertions(+), 6 deletions(-)
From: Nicolas Pitre <hidden> Date: 2012-08-12 04:05:06
On Fri, 10 Aug 2012, Cyril Chemparathy wrote:
quoted hunk
This patch modifies the switch_mm() processor functions to use phys_addr_t.
On LPAE systems, we now honor the upper 32-bits of the physical address that
is being passed in, and program these into TTBR as expected.
Signed-off-by: Cyril Chemparathy <redacted>
Signed-off-by: Vitaly Andrianov <redacted>
---
arch/arm/include/asm/proc-fns.h | 4 ++--
arch/arm/mm/proc-v7-3level.S | 26 ++++++++++++++++++++++----
2 files changed, 24 insertions(+), 6 deletions(-)
@@ -0,0 +1,20 @@+CONFIG_EXPERIMENTAL=y+CONFIG_BLK_DEV_INITRD=y+CONFIG_ARCH_KEYSTONE=y+CONFIG_ARM_ARCH_TIMER=y+CONFIG_AEABI=y+CONFIG_HIGHMEM=y+CONFIG_VFP=y+CONFIG_NEON=y+# CONFIG_SUSPEND is not set+CONFIG_UEVENT_HELPER_PATH="/sbin/hotplug"+CONFIG_BLK_DEV_RAM=y+CONFIG_SERIAL_8250=y+CONFIG_SERIAL_8250_CONSOLE=y+CONFIG_SERIAL_OF_PLATFORM=y+CONFIG_PRINTK_TIME=y+CONFIG_DEBUG_KERNEL=y+CONFIG_DEBUG_INFO=y+CONFIG_DEBUG_USER=y+CONFIG_DEBUG_LL=y+CONFIG_EARLY_PRINTK=y
This patch adds support for 64-bit physical addresses in virt_to_phys()
patching. This does not do real 64-bit add/sub, but instead patches in the
upper 32-bits of the phys_offset directly into the output of virt_to_phys.
There is no corresponding change on the phys_to_virt() side, because
computations on the upper 32-bits would be discarded anyway.
Signed-off-by: Cyril Chemparathy <redacted>
---
arch/arm/include/asm/memory.h | 22 ++++++++++++++++++----
arch/arm/kernel/head.S | 4 ++++
arch/arm/kernel/setup.c | 2 +-
3 files changed, 23 insertions(+), 5 deletions(-)
From: Nicolas Pitre <hidden> Date: 2012-08-12 03:39:12
On Fri, 10 Aug 2012, Cyril Chemparathy wrote:
quoted hunk
This patch adds support for 64-bit physical addresses in virt_to_phys()
patching. This does not do real 64-bit add/sub, but instead patches in the
upper 32-bits of the phys_offset directly into the output of virt_to_phys.
There is no corresponding change on the phys_to_virt() side, because
computations on the upper 32-bits would be discarded anyway.
Signed-off-by: Cyril Chemparathy <redacted>
---
arch/arm/include/asm/memory.h | 22 ++++++++++++++++++----
arch/arm/kernel/head.S | 4 ++++
arch/arm/kernel/setup.c | 2 +-
3 files changed, 23 insertions(+), 5 deletions(-)
Hmmm... I'm afraid this is going to be suboptimal when LPAE is not
selected.
First of all, you do not need to cast tlo to a u64 in the return value.
Then, I'm not sure if the compiler is smart enough to see that the
returned value is a phys_addr_t which can be a u32, and in this case the
(u64)thi << 32 is going to be truncated right away, and therefore there
is no point in emiting the corresponding instructions.
Furthermore, if LPAE is not defined, then thi never gets initialized and
should produce a warning. Did you test compilation of the code with LPAE
turned off?
I'd prefer something like this where more stuff is validated by the
compiler:
static inline phys_addr_t __virt_to_phys(unsigned long x)
{
unsigned long tlo, thi;
phys_addr_t ret;
early_patch_imm8("add", tlo, x, __pv_offset, 0);
ret = tlo;
if (sizeof(phys_addr_t) > 4) {
#ifdef __ARMEB__
early_patch_imm8_mov("mov", thi, __pv_phys_offset, 0);
#else
early_patch_imm8_mov("mov", thi, __pv_phys_offset, 4);
#endif
ret |= ((u64)thi) << 32;
}
return ret);
}
This should let the compiler optimize things whether LPAE is enabledor
not while validating both cases.
Nicolas
This patch adds support for 64-bit physical addresses in virt_to_phys()
patching. This does not do real 64-bit add/sub, but instead patches in the
upper 32-bits of the phys_offset directly into the output of virt_to_phys.
There is no corresponding change on the phys_to_virt() side, because
computations on the upper 32-bits would be discarded anyway.
Signed-off-by: Cyril Chemparathy <redacted>
---
arch/arm/include/asm/memory.h | 22 ++++++++++++++++++----
arch/arm/kernel/head.S | 4 ++++
arch/arm/kernel/setup.c | 2 +-
3 files changed, 23 insertions(+), 5 deletions(-)
Hmmm... I'm afraid this is going to be suboptimal when LPAE is not
selected.
I understand your concern, but I don't see the sub-optimality. I tested
the following function with GCC versions 4.3.3 and 4.7. This is after
the other changes that I mentioned in my previous email, but with the
__virt_to_phys() code itself unchanged:
phys_addr_t ____test_virt_to_phys(unsigned long addr)
{
return __virt_to_phys(addr);
}
The resulting code in both cases looks like:
<____test_virt_to_phys>:
b c04f0528
bx lr
[the branch of course gets patched to an add]
First of all, you do not need to cast tlo to a u64 in the return value.
True enough.
Then, I'm not sure if the compiler is smart enough to see that the
returned value is a phys_addr_t which can be a u32, and in this case the
(u64)thi << 32 is going to be truncated right away, and therefore there
is no point in emiting the corresponding instructions.
In this case, it appears to be smart enough. However, I agree that
relying on compiler smarts is probably not the best thing for us to do.
Furthermore, if LPAE is not defined, then thi never gets initialized and
should produce a warning. Did you test compilation of the code with LPAE
turned off?
Sure. One of our test platforms is non-LPAE. The compiler does not
produce warnings on this, and this is consistent across both compiler
versions.
I'd prefer something like this where more stuff is validated by the
compiler:
static inline phys_addr_t __virt_to_phys(unsigned long x)
{
unsigned long tlo, thi;
phys_addr_t ret;
early_patch_imm8("add", tlo, x, __pv_offset, 0);
ret = tlo;
if (sizeof(phys_addr_t) > 4) {
#ifdef __ARMEB__
early_patch_imm8_mov("mov", thi, __pv_phys_offset, 0);
#else
early_patch_imm8_mov("mov", thi, __pv_phys_offset, 4);
#endif
ret |= ((u64)thi) << 32;
}
return ret);
}
This should let the compiler optimize things whether LPAE is enabledor
not while validating both cases.
Agreed on the principal, but more below...
I've meanwhile been chasing down another problem - the code generated
for the LPAE case. The original code resulted in the following:
<____test_virt_to_phys>:
mov r2, #0
b c01bc800 # patch: add r1, r0, __pv_offset
b c01bc810 # patch: mov r0, __phys_offset_high
orr r2, r2, r1
mov r3, r0
mov r1, r3
mov r0, r2
bx lr
Yikes! This code does a bunch of futile register shuffling and a
pointless or, all in the name of generating the result in a 64-bit
register-pair from the 32-bit halves.
In order to get past this, I tried adding operand qualifiers (R = upper
32-bits, Q = lower 32-bits) in the patch macros, in the hope that
treating these as native 64-bit register pairs would eliminate the need
to shuffle them around after the inline assembly blocks. This then
allows us to implement __virt_to_phys() as follows:
static inline phys_addr_t __virt_to_phys(unsigned long x)
{
phys_addr_t t;
if (sizeof(t) == 4) {
t = x;
early_patch_imm8("add", t, "", t, __pv_offset, 0);
return t;
}
/*
* On LPAE, we do not _need_ to do 64-bit arithmetic because
* the high order 32 bits are never changed by the phys-virt
* offset. We simply patch in the high order physical address
* bits instead.
*
* Note: the mov _must_ be first here. From the compiler's
* perspective, this is the initializer for the variable. The
* mov itself initializes only the upper half. The subsequent
* add treats t as a read/write operand and initializes the
* lower half.
*/
#ifdef __ARMEB__
early_patch_imm8_mov("mov", t, "R", __pv_phys_offset, 0);
#else
early_patch_imm8_mov("mov", t, "R", __pv_phys_offset, 4);
#endif
early_patch_imm8("add", t, "Q", x, __pv_offset, 0);
return t;
}
With this, we get significantly better looking generated code:
<____test_virt_to_phys>:
b c01d519c # patch: mov r3, __phys_offset_high
b c01d51ac # patch: add r2, r0, __phys_offset_high
mov r0, r2
mov r1, r3
bx lr
This is about as far along as I've been able to proceed. I still
haven't figured out a way to get it to patch in place without an extra
register pair.
Overall, this is still a bit too kludgy for my liking. In particular,
the read/write operand forces add/sub/... users to initialize the result
variable. I am currently leaning towards adding native support for
64-bit operations in the runtime patch code, instead of having to hack
around it with 32-bit primitives. Better ideas, any one?
Thanks
-- Cyril.
From: Nicolas Pitre <hidden> Date: 2012-08-13 04:03:51
On Sun, 12 Aug 2012, Cyril Chemparathy wrote:
On 08/11/12 23:39, Nicolas Pitre wrote:
quoted
On Fri, 10 Aug 2012, Cyril Chemparathy wrote:
quoted
This patch adds support for 64-bit physical addresses in virt_to_phys()
patching. This does not do real 64-bit add/sub, but instead patches in
the
upper 32-bits of the phys_offset directly into the output of virt_to_phys.
There is no corresponding change on the phys_to_virt() side, because
computations on the upper 32-bits would be discarded anyway.
Signed-off-by: Cyril Chemparathy <redacted>
---
arch/arm/include/asm/memory.h | 22 ++++++++++++++++++----
arch/arm/kernel/head.S | 4 ++++
arch/arm/kernel/setup.c | 2 +-
3 files changed, 23 insertions(+), 5 deletions(-)
Hmmm... I'm afraid this is going to be suboptimal when LPAE is not
selected.
I understand your concern, but I don't see the sub-optimality. I tested the
following function with GCC versions 4.3.3 and 4.7. This is after the other
changes that I mentioned in my previous email, but with the __virt_to_phys()
code itself unchanged:
phys_addr_t ____test_virt_to_phys(unsigned long addr)
{
return __virt_to_phys(addr);
}
The resulting code in both cases looks like:
<____test_virt_to_phys>:
b c04f0528
bx lr
[the branch of course gets patched to an add]
OK. Please add such information and results in your commit log.
quoted
First of all, you do not need to cast tlo to a u64 in the return value.
True enough.
quoted
Then, I'm not sure if the compiler is smart enough to see that the
returned value is a phys_addr_t which can be a u32, and in this case the
(u64)thi << 32 is going to be truncated right away, and therefore there
is no point in emiting the corresponding instructions.
In this case, it appears to be smart enough. However, I agree that relying on
compiler smarts is probably not the best thing for us to do.
Well, we do rely on a lot of compiler smarts in the kernel. As long as
sufficiently old compiler versions do get it right then it should be
fine.
quoted
Furthermore, if LPAE is not defined, then thi never gets initialized and
should produce a warning. Did you test compilation of the code with LPAE
turned off?
Sure. One of our test platforms is non-LPAE. The compiler does not produce
warnings on this, and this is consistent across both compiler versions.
That is odd. I still prefer my version nevertheless so to avoid this
potential issue in the future.
quoted
I'd prefer something like this where more stuff is validated by the
compiler:
static inline phys_addr_t __virt_to_phys(unsigned long x)
{
unsigned long tlo, thi;
phys_addr_t ret;
early_patch_imm8("add", tlo, x, __pv_offset, 0);
ret = tlo;
if (sizeof(phys_addr_t) > 4) {
#ifdef __ARMEB__
early_patch_imm8_mov("mov", thi, __pv_phys_offset, 0);
#else
early_patch_imm8_mov("mov", thi, __pv_phys_offset, 4);
#endif
ret |= ((u64)thi) << 32;
}
return ret);
}
This should let the compiler optimize things whether LPAE is enabledor
not while validating both cases.
Agreed on the principal, but more below...
I've meanwhile been chasing down another problem - the code generated for the
LPAE case. The original code resulted in the following:
<____test_virt_to_phys>:
mov r2, #0
b c01bc800 # patch: add r1, r0, __pv_offset
b c01bc810 # patch: mov r0, __phys_offset_high
orr r2, r2, r1
mov r3, r0
mov r1, r3
mov r0, r2
bx lr
Yikes! This code does a bunch of futile register shuffling and a pointless or,
all in the name of generating the result in a 64-bit register-pair from the
32-bit halves.
In order to get past this, I tried adding operand qualifiers (R = upper
32-bits, Q = lower 32-bits) in the patch macros, in the hope that treating
these as native 64-bit register pairs would eliminate the need to shuffle them
around after the inline assembly blocks. This then allows us to implement
__virt_to_phys() as follows:
static inline phys_addr_t __virt_to_phys(unsigned long x)
{
phys_addr_t t;
if (sizeof(t) == 4) {
t = x;
early_patch_imm8("add", t, "", t, __pv_offset, 0);
return t;
}
/*
* On LPAE, we do not _need_ to do 64-bit arithmetic because
* the high order 32 bits are never changed by the phys-virt
* offset. We simply patch in the high order physical address
* bits instead.
*
* Note: the mov _must_ be first here. From the compiler's
* perspective, this is the initializer for the variable. The
* mov itself initializes only the upper half. The subsequent
* add treats t as a read/write operand and initializes the
* lower half.
*/
#ifdef __ARMEB__
early_patch_imm8_mov("mov", t, "R", __pv_phys_offset, 0);
#else
early_patch_imm8_mov("mov", t, "R", __pv_phys_offset, 4);
#endif
early_patch_imm8("add", t, "Q", x, __pv_offset, 0);
return t;
}
With this, we get significantly better looking generated code:
<____test_virt_to_phys>:
b c01d519c # patch: mov r3, __phys_offset_high
b c01d51ac # patch: add r2, r0, __phys_offset_high
mov r0, r2
mov r1, r3
bx lr
This is about as far along as I've been able to proceed. I still haven't
figured out a way to get it to patch in place without an extra register pair.
Overall, this is still a bit too kludgy for my liking. In particular, the
read/write operand forces add/sub/... users to initialize the result variable.
I am currently leaning towards adding native support for 64-bit operations in
the runtime patch code, instead of having to hack around it with 32-bit
primitives. Better ideas, any one?
The best I've come up with is this:
//typedef unsigned long phys_addr_t;
typedef unsigned long long phys_addr_t;
phys_addr_t __virt_to_phys(unsigned long v)
{
phys_addr_t p;
if (sizeof(phys_addr_t) > 4) {
asm ("add %Q0, %1, #PV_OFFSET" : "=r" (p) : "r" (v));
asm ("mov %R0, #PHYS_HIGHBITS" : "+r" (p));
} else {
asm ("add %0, %1, #PV_OFFSET" : "=r" (p) : "r" (v));
}
return p;
}
The add is done first as this reduce the life span of the v variable.
Because p is listed as an early clobber, then in theory the register
holding v could be used to store the low part of p right away. If you
put the mov first then you can't do that especially on big endian.
It seems that there is no clean way to combine the LPAE and non LPAE
cases.
Nicolas
The original phys_to_virt/virt_to_phys patching implementation relied on early
patching prior to MMU initialization. On PAE systems running out of >4G
address space, this would have entailed an additional round of patching after
switching over to the high address space.
The approach implemented here conceptually extends the original PHYS_OFFSET
patching implementation with the introduction of "early" patch stubs. Early
patch code is required to be functional out of the box, even before the patch
is applied. This is implemented by inserting functional (but inefficient)
load code into the .runtime.patch.code init section. Having functional code
out of the box then allows us to defer the init time patch application until
later in the init sequence.
In addition to fitting better with our need for physical address-space
switch-over, this implementation should be somewhat more extensible by virtue
of its more readable (and hackable) C implementation. This should prove
useful for other similar init time specialization needs, especially in light
of our multi-platform kernel initiative.
This code has been boot tested in both ARM and Thumb-2 modes on an ARMv7
(Cortex-A8) device.
Note: the obtuse use of stringified symbols in patch_stub() and
early_patch_stub() is intentional. Theoretically this should have been
accomplished with formal operands passed into the asm block, but this requires
the use of the 'c' modifier for instantiating the long (e.g. .long %c0).
However, the 'c' modifier has been found to ICE certain versions of GCC, and
therefore we resort to stringified symbols here.
Signed-off-by: Cyril Chemparathy <redacted>
---
arch/arm/Kconfig | 3 +
arch/arm/include/asm/module.h | 7 ++
arch/arm/include/asm/runtime-patch.h | 175 +++++++++++++++++++++++++++++++
arch/arm/kernel/Makefile | 1 +
arch/arm/kernel/module.c | 4 +
arch/arm/kernel/runtime-patch.c | 189 ++++++++++++++++++++++++++++++++++
arch/arm/kernel/setup.c | 3 +
arch/arm/kernel/vmlinux.lds.S | 10 ++
8 files changed, 392 insertions(+)
create mode 100644 arch/arm/include/asm/runtime-patch.h
create mode 100644 arch/arm/kernel/runtime-patch.c
From: Nicolas Pitre <hidden> Date: 2012-08-12 02:22:16
On Fri, 10 Aug 2012, Cyril Chemparathy wrote:
The original phys_to_virt/virt_to_phys patching implementation relied on early
patching prior to MMU initialization. On PAE systems running out of >4G
address space, this would have entailed an additional round of patching after
switching over to the high address space.
The approach implemented here conceptually extends the original PHYS_OFFSET
patching implementation with the introduction of "early" patch stubs. Early
patch code is required to be functional out of the box, even before the patch
is applied. This is implemented by inserting functional (but inefficient)
load code into the .runtime.patch.code init section. Having functional code
out of the box then allows us to defer the init time patch application until
later in the init sequence.
In addition to fitting better with our need for physical address-space
switch-over, this implementation should be somewhat more extensible by virtue
of its more readable (and hackable) C implementation. This should prove
useful for other similar init time specialization needs, especially in light
of our multi-platform kernel initiative.
This code has been boot tested in both ARM and Thumb-2 modes on an ARMv7
(Cortex-A8) device.
Note: the obtuse use of stringified symbols in patch_stub() and
early_patch_stub() is intentional. Theoretically this should have been
accomplished with formal operands passed into the asm block, but this requires
the use of the 'c' modifier for instantiating the long (e.g. .long %c0).
However, the 'c' modifier has been found to ICE certain versions of GCC, and
therefore we resort to stringified symbols here.
Signed-off-by: Cyril Chemparathy <redacted>
The original phys_to_virt/virt_to_phys patching implementation relied on early
patching prior to MMU initialization. On PAE systems running out of >4G
address space, this would have entailed an additional round of patching after
switching over to the high address space.
The approach implemented here conceptually extends the original PHYS_OFFSET
patching implementation with the introduction of "early" patch stubs. Early
patch code is required to be functional out of the box, even before the patch
is applied. This is implemented by inserting functional (but inefficient)
load code into the .runtime.patch.code init section. Having functional code
out of the box then allows us to defer the init time patch application until
later in the init sequence.
In addition to fitting better with our need for physical address-space
switch-over, this implementation should be somewhat more extensible by virtue
of its more readable (and hackable) C implementation. This should prove
useful for other similar init time specialization needs, especially in light
of our multi-platform kernel initiative.
This code has been boot tested in both ARM and Thumb-2 modes on an ARMv7
(Cortex-A8) device.
Note: the obtuse use of stringified symbols in patch_stub() and
early_patch_stub() is intentional. Theoretically this should have been
accomplished with formal operands passed into the asm block, but this requires
the use of the 'c' modifier for instantiating the long (e.g. .long %c0).
However, the 'c' modifier has been found to ICE certain versions of GCC, and
therefore we resort to stringified symbols here.
Signed-off-by: Cyril Chemparathy <redacted>
Reviewed-by: Nicolas Pitre <redacted>
Thanks.
I've been looking at the compiler emitted code, and had to make a couple
of changes to keep things streamlined...
[...]
First, the "m" operand modifier for "sym" forces GCC to emit code to
load the address of the symbol into a register. I've replaced this with
"i" (&(sym) to make that go away. With this, the emitted code doesn't
contain any such unexpected nonsense.
Second, marking the "to" operand as early clobber makes the compiler
generate horrid register moves around the assembly block, even when it
has registers to spare. Simply adding a temporary variable does a much
much better job, especially since this temporary register is used only
in the patched-out "early" code.
Thanks
-- Cyril.
This patch fixes up the types used when converting back and forth between
physical and virtual addresses.
Signed-off-by: Vitaly Andrianov <redacted>
Signed-off-by: Cyril Chemparathy <redacted>
---
arch/arm/include/asm/memory.h | 26 ++++++++++++++++++--------
1 file changed, 18 insertions(+), 8 deletions(-)
From: Nicolas Pitre <hidden> Date: 2012-08-12 03:04:55
On Fri, 10 Aug 2012, Cyril Chemparathy wrote:
This patch fixes up the types used when converting back and forth between
physical and virtual addresses.
Signed-off-by: Vitaly Andrianov <redacted>
Signed-off-by: Cyril Chemparathy <redacted>
This patch moves the TTBR1 offset calculation and the T1SZ calculation out
of the TTB setup assembly code. This should not affect functionality in
any way, but improves code readability as well as readability of subsequent
patches in this series.
Signed-off-by: Cyril Chemparathy <redacted>
Signed-off-by: Vitaly Andrianov <redacted>
---
arch/arm/include/asm/pgtable-3level-hwdef.h | 10 ++++++++++
arch/arm/mm/proc-v7-3level.S | 16 ++++------------
2 files changed, 14 insertions(+), 12 deletions(-)
From: Nicolas Pitre <hidden> Date: 2012-08-12 04:19:18
On Fri, 10 Aug 2012, Cyril Chemparathy wrote:
This patch moves the TTBR1 offset calculation and the T1SZ calculation out
of the TTB setup assembly code. This should not affect functionality in
any way, but improves code readability as well as readability of subsequent
patches in this series.
Signed-off-by: Cyril Chemparathy <redacted>
Signed-off-by: Vitaly Andrianov <redacted>
From: Vitaly Andrianov <redacted>
This patch fixes the alloc_init_pud() function to use phys_addr_t instead of
unsigned long when passing in the phys argument.
This is an extension to commit 97092e0c56830457af0639f6bd904537a150ea4a (ARM:
pgtable: use phys_addr_t for physical addresses), which applied similar changes
elsewhere in the ARM memory management code.
Signed-off-by: Vitaly Andrianov <redacted>
Signed-off-by: Cyril Chemparathy <redacted>
Acked-by: Nicolas Pitre <redacted>
---
arch/arm/mm/mmu.c | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
Keystone platforms have their physical memory mapped at an address outside the
32-bit physical range. A Keystone machine with 16G of RAM would find its
memory at 0x0800000000 - 0x0bffffffff.
For boot purposes, the interconnect supports a limited alias of some of this
memory within the 32-bit addressable space (0x80000000 - 0xffffffff). This
aliasing is implemented in hardware, and is not intended to be used much
beyond boot. For instance, DMA coherence does not work when running out of
this aliased address space.
Therefore, we've taken the approach of booting out of the low physical address
range, and subsequently we switch over to the high range once we're safely
inside machine specific territory. This patch implements this switch over
mechanism, which involves rewiring the TTBRs and page tables to point to the
new physical address space.
Signed-off-by: Vitaly Andrianov <redacted>
Signed-off-by: Cyril Chemparathy <redacted>
---
arch/arm/Kconfig | 1 +
arch/arm/boot/dts/keystone-sim.dts | 8 +++---
arch/arm/configs/keystone_defconfig | 1 +
arch/arm/mach-keystone/include/mach/memory.h | 25 +++++++++++++++++
arch/arm/mach-keystone/keystone.c | 39 ++++++++++++++++++++++++++
arch/arm/mach-keystone/platsmp.c | 16 +++++++++--
6 files changed, 83 insertions(+), 7 deletions(-)
@@ -74,6 +74,41 @@ static const char *keystone_match[] __initconst = {NULL,};+staticvoid__initkeystone_init_meminfo(void)+{+boollpae=IS_ENABLED(CONFIG_ARM_LPAE);+boolpvpatch=IS_ENABLED(CONFIG_ARM_PATCH_PHYS_VIRT);+phys_addr_toffset=PHYS_OFFSET-KEYSTONE_LOW_PHYS_START;+phys_addr_tmem_start,mem_end;++BUG_ON(meminfo.nr_banks<1);+mem_start=meminfo.bank[0].start;+mem_end=mem_start+meminfo.bank[0].size-1;++/* nothing to do if we are running out of the <32-bit space */+if(mem_start>=KEYSTONE_LOW_PHYS_START&&+mem_end<=KEYSTONE_LOW_PHYS_END)+return;++if(!lpae||!pvpatch){+panic("Enable %s%s%s to run outside 32-bit space\n",+!lpae?__stringify(CONFIG_ARM_LPAE):"",+(!lpae&&!pvpatch)?" and ":"",+!pvpatch?__stringify(CONFIG_ARM_PATCH_PHYS_VIRT):"");+}++if(mem_start<KEYSTONE_HIGH_PHYS_START||+mem_end>KEYSTONE_HIGH_PHYS_END){+panic("Invalid address space for memory (%08llx-%08llx)\n",+mem_start,mem_end);+}++offset+=KEYSTONE_HIGH_PHYS_START;+pr_info("switching to high address space at 0x%llx\n",offset);+__pv_phys_offset=offset;+__pv_offset=offset-PAGE_OFFSET;+}+DT_MACHINE_START(KEYSTONE,"Keystone")smp_ops(keystone_smp_ops).map_io=keystone_map_io,
@@ -51,17 +52,26 @@ static void __init keystone_smp_prepare_cpus(unsigned int max_cpus)/* nothing for now */}+staticvoid__cpuinitkeystone_secondary_initmem(void)+{+#ifdef CONFIG_ARM_LPAE+pgd_t*pgd0=pgd_offset_k(0);+cpu_set_ttbr(1,__pa(pgd0)+TTBR1_OFFSET);+local_flush_tlb_all();+#endif+}+staticvoid__cpuinitkeystone_secondary_init(unsignedintcpu){gic_secondary_init(0);+keystone_secondary_initmem();}staticint__cpuinitkeystone_boot_secondary(unsignedintcpu,structtask_struct*idle){-unsignedlong*ptr;--ptr=phys_to_virt(0x800001f0);+unsignedlong*ptr=(unsignedlong*)(PAGE_OFFSET+0x1f0);+ptr[cpu]=virt_to_idmap(&secondary_startup);__cpuc_flush_dcache_area(ptr,sizeof(ptr)*4);
On Keystone platforms, physical memory is entirely outside the 32-bit
addressible range. Therefore, the (bank->start > ULONG_MAX) check below marks
the entire system memory as highmem, and this causes unpleasentness all over.
This patch eliminates the extra bank start check (against ULONG_MAX) by
checking bank->start against the physical address corresponding to vmalloc_min
instead.
In the process, this patch also cleans up parts of the highmem sanity check
code by removing what has now become a redundant check for banks that entirely
overlap with the vmalloc range.
Signed-off-by: Cyril Chemparathy <redacted>
Signed-off-by: Vitaly Andrianov <redacted>
---
arch/arm/mm/mmu.c | 19 +------------------
1 file changed, 1 insertion(+), 18 deletions(-)
From: Nicolas Pitre <hidden> Date: 2012-08-12 04:36:25
On Fri, 10 Aug 2012, Cyril Chemparathy wrote:
On Keystone platforms, physical memory is entirely outside the 32-bit
addressible range. Therefore, the (bank->start > ULONG_MAX) check below marks
the entire system memory as highmem, and this causes unpleasentness all over.
This patch eliminates the extra bank start check (against ULONG_MAX) by
checking bank->start against the physical address corresponding to vmalloc_min
instead.
In the process, this patch also cleans up parts of the highmem sanity check
code by removing what has now become a redundant check for banks that entirely
overlap with the vmalloc range.
Are you sure of this? The code that you removed not only checks for
banks that fall into the vmalloc area, but it also skipp them. This is
now lost.
On Keystone platforms, physical memory is entirely outside the 32-bit
addressible range. Therefore, the (bank->start > ULONG_MAX) check below marks
the entire system memory as highmem, and this causes unpleasentness all over.
This patch eliminates the extra bank start check (against ULONG_MAX) by
checking bank->start against the physical address corresponding to vmalloc_min
instead.
In the process, this patch also cleans up parts of the highmem sanity check
code by removing what has now become a redundant check for banks that entirely
overlap with the vmalloc range.
Are you sure of this? The code that you removed not only checks for
banks that fall into the vmalloc area, but it also skipp them. This is
now lost.
I almost missed out on this email...
The check is not quite lost, we still have the following in the
!CONFIG_HIGHMEM block:
if (highmem) {
printk(...);
continue;
}
The change is that highmem is now set outside the #ifdef when
(bank->start >= vmalloc_limit), and therefore the check is truly redundant.
From: Nicolas Pitre <hidden> Date: 2012-09-10 18:07:39
On Mon, 10 Sep 2012, Cyril Chemparathy wrote:
On 8/12/2012 12:36 AM, Nicolas Pitre wrote:
quoted
On Fri, 10 Aug 2012, Cyril Chemparathy wrote:
quoted
On Keystone platforms, physical memory is entirely outside the 32-bit
addressible range. Therefore, the (bank->start > ULONG_MAX) check below
marks
the entire system memory as highmem, and this causes unpleasentness all
over.
This patch eliminates the extra bank start check (against ULONG_MAX) by
checking bank->start against the physical address corresponding to
vmalloc_min
instead.
In the process, this patch also cleans up parts of the highmem sanity
check
code by removing what has now become a redundant check for banks that
entirely
overlap with the vmalloc range.
Are you sure of this? The code that you removed not only checks for
banks that fall into the vmalloc area, but it also skipp them. This is
now lost.
I almost missed out on this email...
The check is not quite lost, we still have the following in the
!CONFIG_HIGHMEM block:
if (highmem) {
printk(...);
continue;
}
The change is that highmem is now set outside the #ifdef when (bank->start >=
vmalloc_limit), and therefore the check is truly redundant.
This patch adds TTBR accessor macros, and modifies cpu_get_pgd() and
the LPAE version of cpu_set_reserved_ttbr0() to use these instead.
In the process, we also fix these functions to correctly handle cases
where the physical address lies beyond the 4G limit of 32-bit addressing.
Signed-off-by: Cyril Chemparathy <redacted>
Signed-off-by: Vitaly Andrianov <redacted>
---
arch/arm/include/asm/proc-fns.h | 24 +++++++++++++++++++-----
arch/arm/mm/context.c | 9 ++-------
2 files changed, 21 insertions(+), 12 deletions(-)
From: Nicolas Pitre <hidden> Date: 2012-08-12 04:11:38
On Fri, 10 Aug 2012, Cyril Chemparathy wrote:
quoted hunk
This patch adds TTBR accessor macros, and modifies cpu_get_pgd() and
the LPAE version of cpu_set_reserved_ttbr0() to use these instead.
In the process, we also fix these functions to correctly handle cases
where the physical address lies beyond the 4G limit of 32-bit addressing.
Signed-off-by: Cyril Chemparathy <redacted>
Signed-off-by: Vitaly Andrianov <redacted>
---
arch/arm/include/asm/proc-fns.h | 24 +++++++++++++++++++-----
arch/arm/mm/context.c | 9 ++-------
2 files changed, 21 insertions(+), 12 deletions(-)
I don't think you have to clobber the condition code (cc) here. It has
been a long while since the ability to pass a condition code inside an
inline assembly statement was removed from gcc.
Other that that...
Acked-by: Nicolas Pitre <redacted>
Nicolas
This patch adds basic sanity tests to ensure that the instruction patching
results in valid instruction encodings. This is done by verifying the output
of the patch process against a vector of assembler generated instructions at
init time.
Signed-off-by: Cyril Chemparathy <redacted>
---
arch/arm/Kconfig | 12 ++++++++++++
arch/arm/kernel/runtime-patch.c | 41 +++++++++++++++++++++++++++++++++++++++
2 files changed, 53 insertions(+)
From: Nicolas Pitre <hidden> Date: 2012-08-12 02:36:00
On Fri, 10 Aug 2012, Cyril Chemparathy wrote:
quoted hunk
This patch adds basic sanity tests to ensure that the instruction patching
results in valid instruction encodings. This is done by verifying the output
of the patch process against a vector of assembler generated instructions at
init time.
Signed-off-by: Cyril Chemparathy <redacted>
---
arch/arm/Kconfig | 12 ++++++++++++
arch/arm/kernel/runtime-patch.c | 41 +++++++++++++++++++++++++++++++++++++++
2 files changed, 53 insertions(+)
Why this masking? With Thumb2 you may find functions starting at
halfword aligned addresses. Only the LSB (indicating thumb mode) should
be masked out.
quoted hunk
+ int i, ret;
+ u32 ninsn, insn;
+
+ insn = test_code[0];
+ for (i = 0; test_code[i]; i++) {
+ ret = do_patch_imm8(insn, 0x41 << i, &ninsn);
+ if (ret < 0)
+ pr_err("runtime patch (imm8): failed at shift %d\n", i);
+ else if (ninsn != test_code[i])
+ pr_err("runtime patch (imm8): failed, need %x got %x\n",
+ test_code[i], ninsn);
+ }
+}
+
+static void __init runtime_patch_test(void)
+{
+ test_patch_imm8();
+}
+#endif
+
int runtime_patch(const void *table, unsigned size)
{
const struct patch_info *p = table, *end = (table + size);
This patch adds basic sanity tests to ensure that the instruction patching
results in valid instruction encodings. This is done by verifying the output
of the patch process against a vector of assembler generated instructions at
init time.
Signed-off-by: Cyril Chemparathy <redacted>
---
Why this masking? With Thumb2 you may find functions starting at
halfword aligned addresses. Only the LSB (indicating thumb mode) should
be masked out.
From: Nicolas Pitre <hidden> Date: 2012-08-13 03:19:52
On Sun, 12 Aug 2012, Cyril Chemparathy wrote:
On 08/11/12 22:35, Nicolas Pitre wrote:
quoted
On Fri, 10 Aug 2012, Cyril Chemparathy wrote:
quoted
This patch adds basic sanity tests to ensure that the instruction patching
results in valid instruction encodings. This is done by verifying the
output
of the patch process against a vector of assembler generated instructions
at
init time.
Signed-off-by: Cyril Chemparathy <redacted>
---
From: Vitaly Andrianov <redacted>
The free_memmap() was mistakenly using unsigned long type to represent
physical addresses. This breaks on PAE systems where memory could be placed
above the 32-bit addressible limit.
This patch fixes this function to properly use phys_addr_t instead.
Signed-off-by: Vitaly Andrianov <redacted>
Signed-off-by: Cyril Chemparathy <redacted>
Acked-by: Nicolas Pitre <redacted>
---
arch/arm/mm/init.c | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
@@ -469,8 +469,8 @@ free_memmap(unsigned long start_pfn, unsigned long end_pfn)*Converttophysicaladdresses,and*roundstartupwardsandenddownwards.*/-pg=(unsignedlong)PAGE_ALIGN(__pa(start_pg));-pgend=(unsignedlong)__pa(end_pg)&PAGE_MASK;+pg=PAGE_ALIGN(__pa(start_pg));+pgend=__pa(end_pg)&PAGE_MASK;/**Iftherearefreepagesbetweenthese,
From: Vitaly Andrianov <redacted>
On some PAE systems (e.g. TI Keystone), memory is above the 32-bit addressible
limit, and the interconnect provides an aliased view of parts of physical
memory in the 32-bit addressible space. This alias is strictly for boot time
usage, and is not otherwise usable because of coherency limitations.
On such systems, the idmap mechanism needs to take this aliased mapping into
account. This patch introduces a virt_to_idmap() macro, which can be used on
such sub-architectures to represent the interconnect supported boot time
alias. Most other systems would leave this macro untouched, i.e., do a simply
virt_to_phys() and nothing more.
Signed-off-by: Vitaly Andrianov <redacted>
Signed-off-by: Cyril Chemparathy <redacted>
---
arch/arm/include/asm/memory.h | 9 +++++++++
arch/arm/kernel/smp.c | 2 +-
arch/arm/mm/idmap.c | 4 ++--
3 files changed, 12 insertions(+), 3 deletions(-)
@@ -85,8 +85,8 @@ static int __init init_static_idmap(void)return-ENOMEM;/* Add an identity mapping for the physical address of the section. */-idmap_start=virt_to_phys((void*)__idmap_text_start);-idmap_end=virt_to_phys((void*)__idmap_text_end);+idmap_start=virt_to_idmap((void*)__idmap_text_start);+idmap_end=virt_to_idmap((void*)__idmap_text_end);pr_info("Setting up static identity map for 0x%llx - 0x%llx\n",(longlong)idmap_start,(longlong)idmap_end);
@@ -0,0 +1,74 @@+/*+*Copyright2012TexasInstruments,Inc.+*+*Basedonplatsmp.c,Copyright2010-2011Calxeda,Inc.+*Basedonplatsmp.c,Copyright(C)2002ARMLtd.+*+*Thisprogramisfreesoftware;youcanredistributeitand/ormodifyit+*underthetermsandconditionsoftheGNUGeneralPublicLicense,+*version2,aspublishedbytheFreeSoftwareFoundation.+*+*Thisprogramisdistributedinthehopeitwillbeuseful,butWITHOUT+*ANYWARRANTY;withouteventheimpliedwarrantyofMERCHANTABILITYor+*FITNESSFORAPARTICULARPURPOSE.SeetheGNUGeneralPublicLicensefor+*moredetails.+*+*YoushouldhavereceivedacopyoftheGNUGeneralPublicLicensealongwith+*thisprogram.Ifnot,see<http://www.gnu.org/licenses/>.+*/+#include<linux/init.h>+#include<linux/smp.h>+#include<linux/io.h>++#include<asm/smp_plat.h>+#include<asm/smp_ops.h>+#include<asm/hardware/gic.h>+#include<asm/cacheflush.h>+#include<asm/memory.h>++#include"keystone.h"++staticvoid__initkeystone_smp_init_cpus(void)+{+unsignedinti,ncores;++ncores=4;++/* sanity check */+if(ncores>NR_CPUS){+pr_warn("restricted to %d cpus\n",NR_CPUS);+ncores=NR_CPUS;+}++for(i=0;i<ncores;i++)+set_cpu_possible(i,true);++set_smp_cross_call(gic_raise_softirq);+}++staticvoid__initkeystone_smp_prepare_cpus(unsignedintmax_cpus)+{+/* nothing for now */+}++staticvoid__cpuinitkeystone_secondary_init(unsignedintcpu)+{+gic_secondary_init(0);+}++staticint__cpuinit+keystone_boot_secondary(unsignedintcpu,structtask_struct*idle)+{+unsignedlong*ptr;++ptr=phys_to_virt(0x800001f0);+ptr[cpu]=virt_to_idmap(&secondary_startup);+__cpuc_flush_dcache_area(ptr,sizeof(ptr)*4);++return0;+}++structsmp_opskeystone_smp_ops__initdata={+smp_init_ops(keystone)+smp_secondary_ops(keystone)+};
From: Vitaly Andrianov <redacted>
This patch fixes the initrd setup code to use phys_addr_t instead of assuming
32-bit addressing. Without this we cannot boot on systems where initrd is
located above the 4G physical address limit.
Signed-off-by: Vitaly Andrianov <redacted>
Signed-off-by: Cyril Chemparathy <redacted>
---
arch/arm/mm/init.c | 13 +++++++------
1 file changed, 7 insertions(+), 6 deletions(-)
@@ -347,14 +348,14 @@ void __init arm_memblock_init(struct meminfo *mi, struct machine_desc *mdesc)#ifdef CONFIG_BLK_DEV_INITRDif(phys_initrd_size&&!memblock_is_region_memory(phys_initrd_start,phys_initrd_size)){-pr_err("INITRD: 0x%08lx+0x%08lx is not a memory region - disabling initrd\n",-phys_initrd_start,phys_initrd_size);+pr_err("INITRD: 0x%08llx+0x%08lx is not a memory region - disabling initrd\n",+(u64)phys_initrd_start,phys_initrd_size);phys_initrd_start=phys_initrd_size=0;}if(phys_initrd_size&&memblock_is_region_reserved(phys_initrd_start,phys_initrd_size)){-pr_err("INITRD: 0x%08lx+0x%08lx overlaps in-use memory region - disabling initrd\n",-phys_initrd_start,phys_initrd_size);+pr_err("INITRD: 0x%08llx+0x%08lx overlaps in-use memory region - disabling initrd\n",+(u64)phys_initrd_start,phys_initrd_size);phys_initrd_start=phys_initrd_size=0;}if(phys_initrd_size){
From: Nicolas Pitre <hidden> Date: 2012-08-12 03:58:46
On Fri, 10 Aug 2012, Cyril Chemparathy wrote:
From: Vitaly Andrianov <redacted>
This patch fixes the initrd setup code to use phys_addr_t instead of assuming
32-bit addressing. Without this we cannot boot on systems where initrd is
located above the 4G physical address limit.
Signed-off-by: Vitaly Andrianov <redacted>
Signed-off-by: Cyril Chemparathy <redacted>
@@ -347,14 +348,14 @@ void __init arm_memblock_init(struct meminfo *mi, struct machine_desc *mdesc)#ifdef CONFIG_BLK_DEV_INITRDif(phys_initrd_size&&!memblock_is_region_memory(phys_initrd_start,phys_initrd_size)){-pr_err("INITRD: 0x%08lx+0x%08lx is not a memory region - disabling initrd\n",-phys_initrd_start,phys_initrd_size);+pr_err("INITRD: 0x%08llx+0x%08lx is not a memory region - disabling initrd\n",+(u64)phys_initrd_start,phys_initrd_size);phys_initrd_start=phys_initrd_size=0;}if(phys_initrd_size&&memblock_is_region_reserved(phys_initrd_start,phys_initrd_size)){-pr_err("INITRD: 0x%08lx+0x%08lx overlaps in-use memory region - disabling initrd\n",-phys_initrd_start,phys_initrd_size);+pr_err("INITRD: 0x%08llx+0x%08lx overlaps in-use memory region - disabling initrd\n",+(u64)phys_initrd_start,phys_initrd_size);phys_initrd_start=phys_initrd_size=0;}if(phys_initrd_size){
This patch applies to PAGE_MASK, PMD_MASK, and PGDIR_MASK, where forcing
unsigned long math truncates the mask at the 32-bits. This clearly does bad
things on PAE systems.
This patch fixes this problem by defining these masks as signed quantities.
We then rely on sign extension to do the right thing.
Signed-off-by: Cyril Chemparathy <redacted>
Signed-off-by: Vitaly Andrianov <redacted>
---
arch/arm/include/asm/page.h | 2 +-
arch/arm/include/asm/pgtable-3level.h | 6 +++---
2 files changed, 4 insertions(+), 4 deletions(-)
From: Nicolas Pitre <hidden> Date: 2012-08-12 03:58:00
On Fri, 10 Aug 2012, Cyril Chemparathy wrote:
This patch applies to PAGE_MASK, PMD_MASK, and PGDIR_MASK, where forcing
unsigned long math truncates the mask at the 32-bits. This clearly does bad
things on PAE systems.
This patch fixes this problem by defining these masks as signed quantities.
We then rely on sign extension to do the right thing.
Signed-off-by: Cyril Chemparathy <redacted>
Signed-off-by: Vitaly Andrianov <redacted>