From: Jonathan Neuschäfer <j.neuschaefer@gmx.net> Date: 2018-02-20 16:16:17
This patchset adds support for DISCONTIGMEM on 32-bit PowerPC. This is
required to properly support the Nintendo Wii's memory layout, in which
there are two blocks of RAM and MMIO in the middle.
Previously, this memory layout was handled by code that joins the two
RAM blocks into one, reserves the MMIO hole, and permits allocations of
reserved memory in ioremap. This hack didn't work with resource-based
allocation (as used for example in the GPIO driver for Wii[1]), however.
After this patchset, users of the Wii can either select CONFIG_FLATMEM
to get the old behaviour, or CONFIG_DISCONTIGMEM to get the new
behaviour.
Some parts of this patchset are probably not ideal (I'm thinking of my
implementation of pfn_to_nid here), and will require some discussion/
changes.
[1]: https://www.spinics.net/lists/devicetree/msg213956.html
Jonathan Neuschäfer (6):
powerpc/mm/32: Use pfn_valid to check if pointer is in RAM
powerpc: numa: Fix overshift on PPC32
powerpc: numa: Use the right #ifdef guards around functions
powerpc: numa: Restrict fake NUMA enulation to CONFIG_NUMA systems
powerpc: Implement DISCONTIGMEM and allow selection on PPC32
powerpc: wii: Don't rely on reserved memory hack if DISCONTIGMEM is
set
arch/powerpc/Kconfig | 5 ++++-
arch/powerpc/include/asm/mmzone.h | 21 +++++++++++++++++++++
arch/powerpc/mm/numa.c | 18 +++++++++++++++---
arch/powerpc/mm/pgtable_32.c | 2 +-
arch/powerpc/platforms/embedded6xx/wii.c | 10 +++++++---
5 files changed, 48 insertions(+), 8 deletions(-)
--
2.16.1
From: Jonathan Neuschäfer <j.neuschaefer@gmx.net> Date: 2018-02-20 16:14:23
The implementation of pfn_to_nid and pfn_valid in mmzone.h is based on
arch/metag's implementation.
Signed-off-by: Jonathan Neuschäfer <j.neuschaefer@gmx.net>
---
NOTE: Checking NODE_DATA(nid) in pfn_to_nid appears to be uncommon.
Running up to MAX_NUMNODES instead of checking NODE_DATA(nid) would
require the node_data array to be filled with valid pointers.
---
arch/powerpc/Kconfig | 5 ++++-
arch/powerpc/include/asm/mmzone.h | 21 +++++++++++++++++++++
arch/powerpc/mm/numa.c | 7 +++++++
3 files changed, 32 insertions(+), 1 deletion(-)
@@ -96,6 +99,7 @@ void __init wii_memory_fixups(void)/* allow ioremapping the address space in the hole */__allow_ioremap_reserved=1;+#endif}unsignedlong__initwii_mmu_mapin_mem2(unsignedlongtop)
From: Jonathan Neuschäfer <j.neuschaefer@gmx.net> Date: 2018-02-20 16:17:25
of_node_to_nid and dump_numa_cpu_topology are declared inline in their
respective header files, if CONFIG_NUMA is not set. Thus it is only
valid to define these functions in numa.c if CONFIG_NUMA is set.
(numa.c, despite the name, isn't conditionalized on CONFIG_NUMA, but
CONFIG_NEED_MULTIPLE_NODES.)
Signed-off-by: Jonathan Neuschäfer <j.neuschaefer@gmx.net>
---
arch/powerpc/mm/numa.c | 4 ++++
1 file changed, 4 insertions(+)
@@ -254,6 +254,7 @@ static int of_node_to_nid_single(struct device_node *device)returnnid;}+#ifdef CONFIG_NUMA/* Walk the device tree upwards, looking for an associativity id */intof_node_to_nid(structdevice_node*device){
@@ -272,6 +273,7 @@ int of_node_to_nid(struct device_node *device)returnnid;}EXPORT_SYMBOL(of_node_to_nid);+#endifstaticint__initfind_min_common_depth(void){
@@ -778,6 +781,7 @@ void __init dump_numa_cpu_topology(void)pr_cont("\n");}}+#endif/* Initialize NODE_DATA for a node on the local memory */staticvoid__initsetup_node_data(intnid,u64start_pfn,u64end_pfn)
From: Jonathan Neuschäfer <j.neuschaefer@gmx.net> Date: 2018-02-20 16:17:29
The Nintendo Wii has a memory layout that places two chunks of RAM at
non-adjacent addresses, and MMIO between them. Currently, the allocation
of these MMIO areas is made possible by declaring the MMIO hole as
reserved memory and allowing reserved memory to be allocated (cf.
wii_memory_fixups).
This patch is the first step towards proper support for discontiguous
memory on PPC32 by using pfn_valid to check if a pointer points into
RAM, rather than open-coding the check. It should result in no
functional difference.
Signed-off-by: Jonathan Neuschäfer <j.neuschaefer@gmx.net>
---
arch/powerpc/mm/pgtable_32.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
@@ -147,7 +147,7 @@ __ioremap_caller(phys_addr_t addr, unsigned long size, unsigned long flags,*Don'tallowanybodytoremapnormalRAMthatwe'reusing.*mem_init()setshigh_memorysoonlydothecheckafterthat.*/-if(slab_is_available()&&(p<virt_to_phys(high_memory))&&+if(slab_is_available()&&pfn_valid(__phys_to_pfn(p))&&!(__allow_ioremap_reserved&&memblock_is_region_reserved(p,size))){printk("__ioremap(): phys addr 0x%llx is RAM lr %ps\n",(unsignedlonglong)p,__builtin_return_address(0));
From: Jonathan Neuschäfer <j.neuschaefer@gmx.net> Date: 2018-02-20 16:18:49
When read_n_cells is compiled for PPC32, "result << 32" causes an
overshift, which GCC doesn't like. Fix this by using u64 instead of
unsigned long.
Signed-off-by: Jonathan Neuschäfer <j.neuschaefer@gmx.net>
---
arch/powerpc/mm/numa.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
Le 20/02/2018 à 17:14, Jonathan Neuschäfer a écrit :
quoted hunk
The Nintendo Wii has a memory layout that places two chunks of RAM at
non-adjacent addresses, and MMIO between them. Currently, the allocation
of these MMIO areas is made possible by declaring the MMIO hole as
reserved memory and allowing reserved memory to be allocated (cf.
wii_memory_fixups).
This patch is the first step towards proper support for discontiguous
memory on PPC32 by using pfn_valid to check if a pointer points into
RAM, rather than open-coding the check. It should result in no
functional difference.
Signed-off-by: Jonathan Neuschäfer <j.neuschaefer@gmx.net>
---
arch/powerpc/mm/pgtable_32.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
@@ -147,7 +147,7 @@ __ioremap_caller(phys_addr_t addr, unsigned long size, unsigned long flags,*Don'tallowanybodytoremapnormalRAMthatwe'reusing.*mem_init()setshigh_memorysoonlydothecheckafterthat.*/-if(slab_is_available()&&(p<virt_to_phys(high_memory))&&+if(slab_is_available()&&pfn_valid(__phys_to_pfn(p))&&
I'm not sure this is equivalent:
high_memory = (void *) __va(max_low_pfn * PAGE_SIZE);
#define ARCH_PFN_OFFSET ((unsigned long)(MEMORY_START >> PAGE_SHIFT))
#define pfn_valid(pfn) ((pfn) >= ARCH_PFN_OFFSET && (pfn) < max_mapnr)
set_max_mapnr(max_pfn);
So in the current implementation it checks against max_low_pfn while
your patch checks against max_pfn
max_low_pfn = max_pfn = memblock_end_of_DRAM() >> PAGE_SHIFT;
#ifdef CONFIG_HIGHMEM
max_low_pfn = lowmem_end_addr >> PAGE_SHIFT;
#endif
Christophe
!(__allow_ioremap_reserved && memblock_is_region_reserved(p, size))) {
printk("__ioremap(): phys addr 0x%llx is RAM lr %ps\n",
(unsigned long long)p, __builtin_return_address(0));
---
L'absence de virus dans ce courrier électronique a été vérifiée par le logiciel antivirus Avast.
https://www.avast.com/antivirus
include/linux/mmzone.h:1239:19: error: conflicting types for 'pfn_valid'
static inline int pfn_valid(unsigned long pfn)
^~~~~~~~~
In file included from include/linux/mmzone.h:912:0,
from include/linux/gfp.h:6,
from include/linux/mm.h:10,
from include/linux/mman.h:5,
from arch/powerpc/kernel/asm-offsets.c:22:
arch/powerpc/include/asm/mmzone.h:40:19: note: previous definition of 'pfn_valid' was here
static inline int pfn_valid(int pfn)
^~~~~~~~~
make[2]: *** [arch/powerpc/kernel/asm-offsets.s] Error 1
make[2]: Target '__build' not remade because of errors.
make[1]: *** [prepare0] Error 2
make[1]: Target 'prepare' not remade because of errors.
make: *** [sub-make] Error 2
vim +/pfn_valid +1239 include/linux/mmzone.h
c4e1be9e Dave Hansen 2017-07-06 1237
7b7bf499 Will Deacon 2011-05-19 1238 #ifndef CONFIG_HAVE_ARCH_PFN_VALID
d41dee36 Andy Whitcroft 2005-06-23 @1239 static inline int pfn_valid(unsigned long pfn)
d41dee36 Andy Whitcroft 2005-06-23 1240 {
d41dee36 Andy Whitcroft 2005-06-23 1241 if (pfn_to_section_nr(pfn) >= NR_MEM_SECTIONS)
d41dee36 Andy Whitcroft 2005-06-23 1242 return 0;
29751f69 Andy Whitcroft 2005-06-23 1243 return valid_section(__nr_to_section(pfn_to_section_nr(pfn)));
d41dee36 Andy Whitcroft 2005-06-23 1244 }
7b7bf499 Will Deacon 2011-05-19 1245 #endif
d41dee36 Andy Whitcroft 2005-06-23 1246
:::::: The code at line 1239 was first introduced by commit
:::::: d41dee369bff3b9dcb6328d4d822926c28cc2594 [PATCH] sparsemem memory model
:::::: TO: Andy Whitcroft [off-list ref]
:::::: CC: Linus Torvalds [off-list ref]
---
0-DAY kernel test infrastructure Open Source Technology Center
https://lists.01.org/pipermail/kbuild-all Intel Corporation
Le 20/02/2018 à 17:14, Jonathan Neuschäfer a écrit :
This patchset adds support for DISCONTIGMEM on 32-bit PowerPC. This is
required to properly support the Nintendo Wii's memory layout, in which
there are two blocks of RAM and MMIO in the middle.
Previously, this memory layout was handled by code that joins the two
RAM blocks into one, reserves the MMIO hole, and permits allocations of
reserved memory in ioremap. This hack didn't work with resource-based
allocation (as used for example in the GPIO driver for Wii[1]), however.
After this patchset, users of the Wii can either select CONFIG_FLATMEM
to get the old behaviour, or CONFIG_DISCONTIGMEM to get the new
behaviour.
My question might me stupid, as I don't know PCC64 in deep, but when
looking at page_is_ram() in arch/powerpc/mm/mem.c, I have the feeling
the PPC64 implements ram by blocks. Isn't it what you are trying to
achieve ? Wouldn't it be feasible to map to what's done in PPC64 for PPC32 ?
Christophe
Some parts of this patchset are probably not ideal (I'm thinking of my
implementation of pfn_to_nid here), and will require some discussion/
changes.
[1]: https://www.spinics.net/lists/devicetree/msg213956.html
Jonathan Neuschäfer (6):
powerpc/mm/32: Use pfn_valid to check if pointer is in RAM
powerpc: numa: Fix overshift on PPC32
powerpc: numa: Use the right #ifdef guards around functions
powerpc: numa: Restrict fake NUMA enulation to CONFIG_NUMA systems
powerpc: Implement DISCONTIGMEM and allow selection on PPC32
powerpc: wii: Don't rely on reserved memory hack if DISCONTIGMEM is
set
arch/powerpc/Kconfig | 5 ++++-
arch/powerpc/include/asm/mmzone.h | 21 +++++++++++++++++++++
arch/powerpc/mm/numa.c | 18 +++++++++++++++---
arch/powerpc/mm/pgtable_32.c | 2 +-
arch/powerpc/platforms/embedded6xx/wii.c | 10 +++++++---
5 files changed, 48 insertions(+), 8 deletions(-)
From: Jonathan Neuschäfer <j.neuschaefer@gmx.net> Date: 2018-02-21 13:52:30
Hello Christophe,
On Tue, Feb 20, 2018 at 06:45:09PM +0100, christophe leroy wrote:
[...]
quoted
- if (slab_is_available() && (p < virt_to_phys(high_memory)) &&
+ if (slab_is_available() && pfn_valid(__phys_to_pfn(p)) &&
I'm not sure this is equivalent:
high_memory = (void *) __va(max_low_pfn * PAGE_SIZE);
#define ARCH_PFN_OFFSET ((unsigned long)(MEMORY_START >> PAGE_SHIFT))
#define pfn_valid(pfn) ((pfn) >= ARCH_PFN_OFFSET && (pfn) < max_mapnr)
set_max_mapnr(max_pfn);
So in the current implementation it checks against max_low_pfn while your
patch checks against max_pfn
max_low_pfn = max_pfn = memblock_end_of_DRAM() >> PAGE_SHIFT;
#ifdef CONFIG_HIGHMEM
max_low_pfn = lowmem_end_addr >> PAGE_SHIFT;
#endif
Good point, I haven't considered CONFIG_HIGHMEM before.
As far as I understand it, in the non-CONFIG_HIGHMEM case:
- max_low_pfn is set to the same value as max_pfn, so the ioremap
check should detect the same PFNs as RAM.
and with CONFIG_HIGHMEM:
- max_low_pfn is set to lowmem_end_addr >> PAGE_SHIFT
- but max_pfn isn't
So, I think you're right.
While looking through arch/powerpc/mm, I noticed that there's a
page_is_ram function, which simply uses the memblocks directly, on
PPC32. It seems like a good candidate for the RAM check in
__ioremap_caller, except that there's this code, which apparently
trashes memblock 0 completely on non-CONFIG_NEED_MULTIPLE_NODES:
https://elixir.bootlin.com/linux/v4.16-rc2/source/arch/powerpc/mm/mem.c#L223
Thanks,
Jonathan Neuschäfer
From: Jonathan Neuschäfer <j.neuschaefer@gmx.net> Date: 2018-02-21 14:42:48
Hi,
On Wed, Feb 21, 2018 at 08:06:10AM +0100, Christophe LEROY wrote:
Le 20/02/2018 à 17:14, Jonathan Neuschäfer a écrit :
quoted
This patchset adds support for DISCONTIGMEM on 32-bit PowerPC. This is
required to properly support the Nintendo Wii's memory layout, in which
there are two blocks of RAM and MMIO in the middle.
Previously, this memory layout was handled by code that joins the two
RAM blocks into one, reserves the MMIO hole, and permits allocations of
reserved memory in ioremap. This hack didn't work with resource-based
allocation (as used for example in the GPIO driver for Wii[1]), however.
After this patchset, users of the Wii can either select CONFIG_FLATMEM
to get the old behaviour, or CONFIG_DISCONTIGMEM to get the new
behaviour.
My question might me stupid, as I don't know PCC64 in deep, but when looking
at page_is_ram() in arch/powerpc/mm/mem.c, I have the feeling the PPC64
implements ram by blocks. Isn't it what you are trying to achieve ? Wouldn't
it be feasible to map to what's done in PPC64 for PPC32 ?
Using page_is_ram in __ioremap_caller and the same memblock-based
approach that's used on PPC64 on PPC32 *should* work, but I think due to
the following line in initmem_init, it won't:
memblock_set_node(0, (phys_addr_t)ULLONG_MAX, &memblock.memory, 0);
Thanks,
Jonathan Neuschäfer
Le 21/02/2018 à 15:42, Jonathan Neuschäfer a écrit :
Hi,
On Wed, Feb 21, 2018 at 08:06:10AM +0100, Christophe LEROY wrote:
quoted
Le 20/02/2018 à 17:14, Jonathan Neuschäfer a écrit :
quoted
This patchset adds support for DISCONTIGMEM on 32-bit PowerPC. This is
required to properly support the Nintendo Wii's memory layout, in which
there are two blocks of RAM and MMIO in the middle.
Previously, this memory layout was handled by code that joins the two
RAM blocks into one, reserves the MMIO hole, and permits allocations of
reserved memory in ioremap. This hack didn't work with resource-based
allocation (as used for example in the GPIO driver for Wii[1]), however.
After this patchset, users of the Wii can either select CONFIG_FLATMEM
to get the old behaviour, or CONFIG_DISCONTIGMEM to get the new
behaviour.
My question might me stupid, as I don't know PCC64 in deep, but when looking
at page_is_ram() in arch/powerpc/mm/mem.c, I have the feeling the PPC64
implements ram by blocks. Isn't it what you are trying to achieve ? Wouldn't
it be feasible to map to what's done in PPC64 for PPC32 ?
Using page_is_ram in __ioremap_caller and the same memblock-based
approach that's used on PPC64 on PPC32 *should* work, but I think due to
the following line in initmem_init, it won't:
memblock_set_node(0, (phys_addr_t)ULLONG_MAX, &memblock.memory, 0);
From: Jonathan Neuschäfer <j.neuschaefer@gmx.net> Date: 2018-02-21 16:08:15
On Wed, Feb 21, 2018 at 07:46:28AM +0800, kbuild test robot wrote:
[...]
quoted
quoted
include/linux/mmzone.h:1239:19: error: conflicting types for 'pfn_valid'
static inline int pfn_valid(unsigned long pfn)
^~~~~~~~~
In file included from include/linux/mmzone.h:912:0,
from include/linux/gfp.h:6,
from include/linux/mm.h:10,
from include/linux/mman.h:5,
from arch/powerpc/kernel/asm-offsets.c:22:
arch/powerpc/include/asm/mmzone.h:40:19: note: previous definition of 'pfn_valid' was here
static inline int pfn_valid(int pfn)
^~~~~~~~~
make[2]: *** [arch/powerpc/kernel/asm-offsets.s] Error 1
make[2]: Target '__build' not remade because of errors.
make[1]: *** [prepare0] Error 2
make[1]: Target 'prepare' not remade because of errors.
make: *** [sub-make] Error 2
Oops, I'll fix this in the next version (and compile-test on ppc64...).
Weirdly enough, x86-32 and parisc define pfn_valid with an int
parameter, too (both of them since the Beginning Of Time, aka.
v2.6.12-rc2).
From: Jonathan Neuschäfer <j.neuschaefer@gmx.net> Date: 2018-02-21 16:53:08
On Wed, Feb 21, 2018 at 04:02:25PM +0100, Christophe LEROY wrote:
[...]
quoted
quoted
My question might me stupid, as I don't know PCC64 in deep, but when looking
at page_is_ram() in arch/powerpc/mm/mem.c, I have the feeling the PPC64
implements ram by blocks. Isn't it what you are trying to achieve ? Wouldn't
it be feasible to map to what's done in PPC64 for PPC32 ?
Using page_is_ram in __ioremap_caller and the same memblock-based
approach that's used on PPC64 on PPC32 *should* work, but I think due to
the following line in initmem_init, it won't:
memblock_set_node(0, (phys_addr_t)ULLONG_MAX, &memblock.memory, 0);
Le 21/02/2018 à 17:08, Jonathan Neuschäfer a écrit :
On Wed, Feb 21, 2018 at 07:46:28AM +0800, kbuild test robot wrote:
[...]
quoted
quoted
quoted
include/linux/mmzone.h:1239:19: error: conflicting types for 'pfn_valid'
static inline int pfn_valid(unsigned long pfn)
^~~~~~~~~
In file included from include/linux/mmzone.h:912:0,
from include/linux/gfp.h:6,
from include/linux/mm.h:10,
from include/linux/mman.h:5,
from arch/powerpc/kernel/asm-offsets.c:22:
arch/powerpc/include/asm/mmzone.h:40:19: note: previous definition of 'pfn_valid' was here
static inline int pfn_valid(int pfn)
^~~~~~~~~
make[2]: *** [arch/powerpc/kernel/asm-offsets.s] Error 1
make[2]: Target '__build' not remade because of errors.
make[1]: *** [prepare0] Error 2
make[1]: Target 'prepare' not remade because of errors.
make: *** [sub-make] Error 2
Oops, I'll fix this in the next version (and compile-test on ppc64...).
Weirdly enough, x86-32 and parisc define pfn_valid with an int
parameter, too (both of them since the Beginning Of Time, aka.
v2.6.12-rc2).
Behind the fact that the pfn type is different, my understanding is that
you have to define CONFIG_HAVE_ARCH_PFN_VALID in the Kconfig in order to
avoid it being included in include/linux/mmzone.h
Christophe
---
L'absence de virus dans ce courrier électronique a été vérifiée par le logiciel antivirus Avast.
https://www.avast.com/antivirus
From: Jonathan Neuschäfer <j.neuschaefer@gmx.net> Date: 2018-02-21 23:32:07
On Wed, Feb 21, 2018 at 04:02:25PM +0100, Christophe LEROY wrote:
[...]
quoted
quoted
My question might me stupid, as I don't know PCC64 in deep, but when looking
at page_is_ram() in arch/powerpc/mm/mem.c, I have the feeling the PPC64
implements ram by blocks. Isn't it what you are trying to achieve ? Wouldn't
it be feasible to map to what's done in PPC64 for PPC32 ?
Using page_is_ram in __ioremap_caller and the same memblock-based
approach that's used on PPC64 on PPC32 *should* work, but I think due to
the following line in initmem_init, it won't:
memblock_set_node(0, (phys_addr_t)ULLONG_MAX, &memblock.memory, 0);
Can't we just fix that ?
Turns out I was completely wrong about this. memblock_set_node as called
above only assigns all memory to node 0 and merges *adjacent* memblocks.
It doesn't merge the memblocks on the Wii, which are far apart.
So now I actually have a working patchset (coming soon), that's a good
deal shorter than this patchset, and hopefully won't break
CONFIG_HIGHMEM in the same way.
Thanks for your input! :)
Jonathan Neuschäfer