On Fri, Jan 19, 2001 at 09:27:44PM -0500, Dan Malek wrote:
I just heard about the bug Tom Gall fixed in "context_overflow"
by testing for current->mm == NULL.
I believe the proper solution is to use 'current->active_mm'
instead of 'current->mm' (and you never get a null pointer).
This way, the proper 'active' context is updated with a new
context even though a kernel thread has stolen it from somewhere
else to use. I think skipping the selection of a new context
in this case could be logically incorrect for some PowerPC cores.
Since this got no response, I'm cross-posting to linuxppc-commit.
So, does anyone else have any comments on this? This appears to be something
that's going to be quite hard to track down if we don't get it right..
So what's the right answer before we forget about it again and go on?
And on that note... what do we have for bug-tracking systems for ppc kernel
related stuff? The collective memory of people on these to mailing lists?
I know someone mentioned a sourceforge page at one point.. is anyone useing that?
--
--------------------------------------------------------------------------
Troy Benjegerdes 'da hozer' hozer@drgw.net
** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/
On Fri, Jan 19, 2001 at 09:27:44PM -0500, Dan Malek wrote:
quoted
I just heard about the bug Tom Gall fixed in "context_overflow"
by testing for current->mm == NULL.
I believe the proper solution is to use 'current->active_mm'
instead of 'current->mm' (and you never get a null pointer).
This way, the proper 'active' context is updated with a new
context even though a kernel thread has stolen it from somewhere
else to use. I think skipping the selection of a new context
in this case could be logically incorrect for some PowerPC cores.
Since this got no response, I'm cross-posting to linuxppc-commit.
I will look into this tomorrow. It's an important fix, I don't want to rush.
So, does anyone else have any comments on this? This appears to be something
that's going to be quite hard to track down if we don't get it right..
So what's the right answer before we forget about it again and go on?
And on that note... what do we have for bug-tracking systems for ppc kernel
related stuff? The collective memory of people on these to mailing lists?
I know someone mentioned a sourceforge page at one point.. is anyone useing that?
There is a sourceforge page. It hasn't been active which probably isn't a good
thing. I know I'm keeping a bug list on paper that I know about. It's not a long
list. Happy to put it out on sourceforge... just don't want it all to be a waste
of time tho. I'm sure Olaf might have a bug or two to contribute 8-)
https://sourceforge.net/projects/ppclinux/
--
Hakuna Matata,
Tom
-----------------------------------------------------------
PPC Linux Guy "My heart is human, my blood is boiling,
my brain IBM" -- Mr Roboto, Styxx
tgall@rochcivictheatre.org
** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/
From: Larry McVoy <hidden> Date: 2001-01-22 04:55:57
On Sun, Jan 21, 2001 at 10:28:42PM -0600, Troy Benjegerdes wrote:
And on that note... what do we have for bug-tracking systems for ppc kernel
related stuff? The collective memory of people on these to mailing lists?
We have a fairly simple bug system we use for BitKeeper; if there was interest
I could polish up a version for the PPC team.
--
---
Larry McVoy lm at bitmover.com http://www.bitmover.com/lm
** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/
From: Dan Malek <hidden> Date: 2001-01-22 18:10:29
Tom Gall wrote:
I will look into this tomorrow. It's an important fix, I don't want to rush.
This was a bug that appeared during the 2.3 development, and (the
collective :-) we have been running systems like this for a long
time. I'm running all of mine with my proposed (proper :-) bug
fix, but since they never crashed before......I'll at least check
it into the 2_5 baseline along with some other stuff.
-- Dan
** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/
On Fri, Jan 19, 2001 at 09:27:44PM -0500, Dan Malek wrote:
quoted
I just heard about the bug Tom Gall fixed in "context_overflow"
by testing for current->mm == NULL.
I believe the proper solution is to use 'current->active_mm'
instead of 'current->mm' (and you never get a null pointer).
This way, the proper 'active' context is updated with a new
context even though a kernel thread has stolen it from somewhere
else to use. I think skipping the selection of a new context
in this case could be logically incorrect for some PowerPC cores.
Since this got no response, I'm cross-posting to linuxppc-commit.
I will look into this tomorrow. It's an important fix, I don't want to rush.
Hi All,
Ok here's the explaination and I beg forgivness if this isn't clear or needs
more filled in.
current->mm I believe is correct. active_mm for tasks in user space just point
back to mm. kernel space tasks will have an mm of NULL yet their active_mm will
point back to the last user space task they ran.
The reason for this patch is in the case where the idle task comes in on one
processor and on another processor it has encountered a context overflow. The
idle task on processor 0 detects the overflow as well and that's when things get
interesting, and why the change.
So anyway that's the situation from my neck of the woods. These other
processor cores make me worried. What cores are you refering too? Are they SMP?
How and in what ways are they different?
Many thanks to Pat McCarthy who is my local guru on this topic and also the
author of the fix in question.
Regards,
Tom
--
Tom Gall - PowerPC Linux Team "Where's the ka-boom? There was
Linux Technology Center supposed to be an earth
(w) tom_gall@vnet.ibm.com shattering ka-boom!"
(w) 507-253-4558 -- Marvin Martian
(h) tgall@rochcivictheatre.org
http://oss.software.ibm.com/developerworks/opensource/linux
** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/
From: Dan Malek <hidden> Date: 2001-01-22 19:59:49
tom_gall@vnet.ibm.com wrote:
current->mm I believe is correct. active_mm for tasks in user space just point
back to mm. kernel space tasks will have an mm of NULL yet their active_mm will
point back to the last user space task they ran.
Not exactly. Every task running on a CPU must have an active_mm, and
it represents the current context for the MMU. This active_mm comes
from a single threaded application's 'mm', or in the case of a
thread without an 'mm' from the previous application that ran, or
from somewhere else depending upon VM_CLONE games.
The point you are missing is 'active_mm' represents the current
context for the MMU. If you get a context overflow, you can't skip
getting and setting a context for an active task just because it
doesn't have a 'current->mm'. Your modification to do this
results in a task running on a CPU with a "NO CONTEXT" mm, and worse
and incorrect VSID/ASID/PID/whatever for the task running on that MMU.
The reason for this patch is in the case where the idle task comes in on one
processor and on another processor it has encountered a context overflow.
It's not just the idle task. It could be any task that is supposed
to get an active_mm from someone else.
The patch is just logically incorrect. There should be no
'if current->mm' and it should get/set context on current->active_mm.
-- Dan
** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/
current->mm I believe is correct. active_mm for tasks in user space just point
back to mm. kernel space tasks will have an mm of NULL yet their active_mm will
point back to the last user space task they ran.
Not exactly. Every task running on a CPU must have an active_mm, and
it represents the current context for the MMU. This active_mm comes
from a single threaded application's 'mm', or in the case of a
thread without an 'mm' from the previous application that ran, or
from somewhere else depending upon VM_CLONE games.
Hi Dan,
Pat and I huddled around your note and gave this some more thought. Still not
convinced it's wrong tho.
The point you are missing is 'active_mm' represents the current
context for the MMU. If you get a context overflow, you can't skip
getting and setting a context for an active task just because it
doesn't have a 'current->mm'. Your modification to do this
results in a task running on a CPU with a "NO CONTEXT" mm, and worse
and incorrect VSID/ASID/PID/whatever for the task running on that MMU.
active_mm represents the current context of USER space, not kernel space. I
think that's the important point here. If a task doesn't have a current->mm it's
a kernel task. It shouldn't be using the Segment Registers in the context.
Right? A kernel task should only be concerned with addresses in the range of
C0000000-FFFFFFFF which aren't in the context.
If what you say is true that incorrect VSID/ASID etc could be handed out, I'm
wondering how my box has been up running and stable since last week. It's not
proof that it's right but I would think something would have melted down by
now....
quoted
The reason for this patch is in the case where the idle task comes in on one
processor and on another processor it has encountered a context overflow.
It's not just the idle task. It could be any task that is supposed
to get an active_mm from someone else.
Since current->mm is NULL, it's a kernel task... granted it doesn't have to be
the idle task but it shouldn't matter. Or when you say any task, are you saying
that user tasks as well?
Correct me if I'm wrong but from the code we were looking at the sched.c when
you pass through switch_mm from a kernel task to a user task, it catches it and
you go from state of NO CONTEXT to the correct context.
Beat me over the head with a crowbar please if I'm missing something.
Regards,
Tom
--
Tom Gall - PowerPC Linux Team "Where's the ka-boom? There was
Linux Technology Center supposed to be an earth
(w) tom_gall@vnet.ibm.com shattering ka-boom!"
(w) 507-253-4558 -- Marvin Martian
(h) tgall@rochcivictheatre.org
http://oss.software.ibm.com/developerworks/opensource/linux
** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/
From: Dan Malek <hidden> Date: 2001-01-23 00:10:04
tom_gall@vnet.ibm.com wrote:
active_mm represents the current context of USER space,
Not exactly. The mm object may only contain page tables for a
user thread, but it also contains information about the MMU context
in general for any thread running on the processor.
..........If a task doesn't have a current->mm it's
a kernel task. It shouldn't be using the Segment Registers in the context.
Right? A kernel task should only be concerned with addresses in the range of
C0000000-FFFFFFFF which aren't in the context.
No, you are confusing MMU context with kernel memory mapping and
our (mostly incorrect) use of VSIDs on the 7xx processors.
If what you say is true that incorrect VSID/ASID etc could be handed out, I'm
wondering how my box has been up running and stable since last week.
Because you are not running something like an MPC8xx or IBM4xx that
cares whether it is correct in kernel space. Some processors do care.
Since current->mm is NULL, it's a kernel task... granted it doesn't have to be
the idle task but it shouldn't matter. Or when you say any task, are you saying
that user tasks as well?
The MMU context switching logic doesn't make any assumptions about
the meaning of current->mm. If there is a current->mm, it switches
to that as the active_mm for the thread. If there isn't a current->mm,
it locates something to use as the active_mm. The rules for selecting
an active_mm can be whatever makes sense for reducing MMU management
or implementing features.
Correct me if I'm wrong but from the code we were looking at the sched.c when
you pass through switch_mm from a kernel task to a user task, it catches it and
you go from state of NO CONTEXT to the correct context.
Yes, but the problem is the context overflowed you did not select a
new one. You allowed the thread to run on the processor (regardless of
what it was) with an expired context, that doesn't match the context
of active_mm. Then later, you find yet another context to switch to
for the same thread that was using the wrong one.
Beat me over the head with a crowbar please if I'm missing something.
What's the big deal? I'm going to say it for the third time. The
active_mm is supposed to represent the mmu context for the thread
currently running on the processor. When the context overflows, we
should get (pick a number) and set (in the MMU) the new context for
the active_mm running on each processor. It is logically incorrect
to test current->mm and skip the get/set. By doing so, you have a
stale MMU hardware context and an mm object that shouldn't be running
on a processor.
-- Dan
** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/
From: Gabriel Paubert <hidden> Date: 2001-01-23 10:00:28
On Mon, 22 Jan 2001, Dan Malek wrote:
tom_gall@vnet.ibm.com wrote:
quoted
active_mm represents the current context of USER space,
Not exactly. The mm object may only contain page tables for a
user thread, but it also contains information about the MMU context
in general for any thread running on the processor.
quoted
..........If a task doesn't have a current->mm it's
a kernel task. It shouldn't be using the Segment Registers in the context.
Right? A kernel task should only be concerned with addresses in the range of
C0000000-FFFFFFFF which aren't in the context.
No, you are confusing MMU context with kernel memory mapping and
our (mostly incorrect) use of VSIDs on the 7xx processors.
Finally, somebody had to say it. But saying mostly incorrect is the
understatement of the week.
quoted
If what you say is true that incorrect VSID/ASID etc could be handed out, I'm
wondering how my box has been up running and stable since last week.
Because you are not running something like an MPC8xx or IBM4xx that
cares whether it is correct in kernel space. Some processors do care.
Yes they do care, but let us noi add unnecessary baggage to processors who
do not need it. All these games with current_mm and active_mm were
introduced for x86 because of their stupid MMU. 8xx/4xx are unfortunately
almost as braindead while 6xx/7xx do get it right.
The MMU code is so different for 4xx/8xx and 6xx/7xx that adding a few
conditionals for this mm management won't hurt. No kernel will ever run
without a recompile on both kind of MMUs anyawy.
quoted
Since current->mm is NULL, it's a kernel task... granted it doesn't have to be
the idle task but it shouldn't matter. Or when you say any task, are you saying
that user tasks as well?
The MMU context switching logic doesn't make any assumptions about
the meaning of current->mm. If there is a current->mm, it switches
to that as the active_mm for the thread. If there isn't a current->mm,
it locates something to use as the active_mm. The rules for selecting
an active_mm can be whatever makes sense for reducing MMU management
or implementing features.
For 6xx/7xx active_mm has not even any reason to exist in the first place,
there are no TLB flush issues when switching tasks when TLB directly
map through an intermediate 52 bit address space. Changing segment
registers if necessary is _cheap_, the operation is local to a processor
and only needs a couple of isync.
Yes, but the problem is the context overflowed you did not select a
new one. You allowed the thread to run on the processor (regardless of
what it was) with an expired context, that doesn't match the context
of active_mm. Then later, you find yet another context to switch to
for the same thread that was using the wrong one.
A proper implementation on 6xx/7xx would not even know what an expired
context is. It would not even be possible to happen in the first place.
Regards,
Gabriel.
** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/
From: Dan Malek <hidden> Date: 2001-01-23 18:21:50
Gabriel Paubert wrote:
Finally, somebody had to say it. But saying mostly incorrect is the
understatement of the week.
It's been mumbled about for a while, and now that I am merging the
latest 4xx code and updating 8xx (and getting some performance
results on the 82xx and 74xx) I want to create more PowerPC generic
functions for all of the variants. Since I am doing the work here,
I can also try some alternatives better suited to PowerPC.
Yes they do care, but let us noi add unnecessary baggage to processors who
do not need it.
I could make them all not care (I think), but I just found it interesting
the generic Linux memory management now has the concept of context
management, and we seem to be the only ones using it (and not very
effectively).
..... All these games with current_mm and active_mm were
introduced for x86 because of their stupid MMU.
I guess I am not familiar enough with that stupid MMU to see why
it had to be done that way. To me, it looked like a neat solution
to sharing mm contexts when it was useful. I would like to use
the lazy TLB management, but it's implementation doesn't seem quite
right (or more likely I don't fully understand it :-).
..... 8xx/4xx are unfortunately
almost as braindead while 6xx/7xx do get it right.
I'll take issue with that. The 8xx/4xx is actually quite nice for
32-bit only embedded processors. It is quite flexible and with some
large page size enhancements I am considering, it should be efficient
as well. The 6xx/7xx have to solve the very large virtual address
space challenge, and unfortunately require lots of software management
because the Linux page tables don't map into their hardware assist
very well. The 74xx (or at least 7450) allows some additional
flexibility that we may want to consider as well. Anyway, there
are other solutions that should work better and I want to work
toward a more generic VM interface for all of PowerPC.
The MMU code is so different for 4xx/8xx and 6xx/7xx that adding a few
conditionals for this mm management won't hurt. No kernel will ever run
without a recompile on both kind of MMUs anyawy.
Right, the MMUs are different, and they share common exception
vectors (a good thing....hint to IBM). Except for the TLB miss
handler and the hash table management, all of the PowerPC code
should be identical (except for the IBM 4xx, which seems to think
it is necessary to invent new registers to do the same thing...).
For 6xx/7xx active_mm has not even any reason to exist....
Perhaps, but it does and we should make sure it is used correctly
(hence, my complaining about not updating it correctly during a
context overflow). I'll be using it to properly manage contexts/VSIDs
in the future.
A proper implementation on 6xx/7xx would not even know what an expired
context is. It would not even be possible to happen in the first place.
Heh :-)....Right, but it will require a context, and active_mm is
the proper place to store/retrieve it. Doesn't the HIGHMEM option
require we properly use a kernel context? Or, do we just get lucky
there, too :-).
What you see today won't be there much longer......
-- Dan
** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/
From: Paul Mackerras <hidden> Date: 2001-02-06 10:50:35
Gabriel Paubert writes:
On Mon, 22 Jan 2001, Dan Malek wrote:
[snip]
quoted
No, you are confusing MMU context with kernel memory mapping and
our (mostly incorrect) use of VSIDs on the 7xx processors.
Finally, somebody had to say it. But saying mostly incorrect is the
understatement of the week.
What's incorrect about it?
Paul.
--
Paul Mackerras, Open Source Research Fellow, Linuxcare, Inc.
+61 2 6262 8990 tel, +61 2 6262 8991 fax
paulus@linuxcare.com.au, http://www.linuxcare.com.au/
Linuxcare. Putting Open Source to work.
** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/
From: Paul Mackerras <hidden> Date: 2001-02-06 10:55:29
Dan Malek writes:
Heh :-)....Right, but it will require a context, and active_mm is
the proper place to store/retrieve it. Doesn't the HIGHMEM option
require we properly use a kernel context? Or, do we just get lucky
there, too :-).
The way we do things on 6xx/7xx, kernel accesses to addresses >=
0xc0000000 don't depend on active_mm or on the currently selected
context at all. We can effectively choose a different context for
each 256MB segment of the address space, and we always choose context
0 for segments 0xc to 0xf. Further, the hash_page routine looks in
swapper_pg_dir for linux PTEs for addresses >= 0xc0000000, rather than
in the page tables associated with current->active_mm or current->mm.
--
Paul Mackerras, Open Source Research Fellow, Linuxcare, Inc.
+61 2 6262 8990 tel, +61 2 6262 8991 fax
paulus@linuxcare.com.au, http://www.linuxcare.com.au/
Linuxcare. Putting Open Source to work.
** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/
From: Dan Malek <hidden> Date: 2001-02-06 21:11:51
Paul Mackerras wrote:
The way we do things on 6xx/7xx, .....
That's the way all PowerPCs currently do it.
.... We can effectively choose a different context for
each 256MB segment of the address space, and we always choose context
0 for segments 0xc to 0xf.
That's not what MMU context means, well at least the way I have
learned to use it in the past. An MMU context is supposed to represent
the virtual mapping of memory objects. Linux has memory objects
and the ability to map these through VM areas, which is interesting
considering (IMHO) the TLB management and the terms (like context)
banted about are such a big hack. Normally, it is the other way around.
You have some legacy hunk of code designed around arcane two level
page tables that tries to represent VM areas and memory objects
with TLB management doing its best to implement real MMU context.
-- Dan
** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/
From: Dan Malek <hidden> Date: 2001-02-06 21:32:57
Paul Mackerras wrote:
What's incorrect about it?
The PowerPC gives us the ability to utilize a huge virtual address
space, and with newer processors a larger physical address space.
Although on 32-bit processors we can't see all of this at once, it
is there for us to use. The only difference between a 32- and 64-bit
PowerPC kernel implementation is that on the 64-bit you get to see
all of the VM space all of the time, while the 32-bit system is
scurrying around behind the scenes ensuring what you need to see
is currently mapped.
We are currently using VSIDs as a lazy TLB reloader, not for
a large VM space context manager. There shouldn't be any need
for CONFIG_HIGHMEM on PowerPC, and we should be able to map huge
PCI spaces we see on the multiple bridge CPCI systems without
any worry of running out of VM space.
The reason this is hard to realize today is we have built up a
system with the assumption there is only a 32-bit virtual space,
half of it consumed by a user task. When I saw the VM changes
happening during 2.3, I saw hoping we could take the opportunity
to improve PowerPC performance with better MMU management. Maybe
someday I can work on it.
-- Dan
** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/
From: Paul Mackerras <hidden> Date: 2001-02-06 21:50:20
Dan Malek writes:
That's not what MMU context means, well at least the way I have
learned to use it in the past. An MMU context is supposed to represent
the virtual mapping of memory objects. Linux has memory objects
No, an MMU context represents an address space, or more precisely the
set of virtual to physical mappings in an address space, which will
typically include mappings of many objects. That's the way the term
is used in the Linux kernel, that's why it's the mm_struct (which
represents an address space) which has the MMU context in it.
and the ability to map these through VM areas, which is interesting
considering (IMHO) the TLB management and the terms (like context)
banted about are such a big hack. Normally, it is the other way around.
You have some legacy hunk of code designed around arcane two level
page tables that tries to represent VM areas and memory objects
with TLB management doing its best to implement real MMU context.
On machines like the x86 where the MMU doesn't know about MMU contexts
you have to basically context-switch the whole MMU including the TLB.
Fortunately we don't have to do that. :)
Paul.
--
Paul Mackerras, Open Source Research Fellow, Linuxcare, Inc.
+61 2 6262 8990 tel, +61 2 6262 8991 fax
paulus@linuxcare.com.au, http://www.linuxcare.com.au/
Linuxcare. Putting Open Source to work.
** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/
From: Paul Mackerras <hidden> Date: 2001-02-06 22:08:06
Dan Malek writes:
The PowerPC gives us the ability to utilize a huge virtual address
space, and with newer processors a larger physical address space.
Although on 32-bit processors we can't see all of this at once, it
is there for us to use. The only difference between a 32- and 64-bit
PowerPC kernel implementation is that on the 64-bit you get to see
all of the VM space all of the time, while the 32-bit system is
scurrying around behind the scenes ensuring what you need to see
is currently mapped.
I'm not sure what "scurrying around behind the scenes" you're talking
about.
Are you saying that when two tasks both have an object mapped, we
should be using the same virtual address in PPC terms (i.e. the 52 or
80-bit intermediate VA), and set up the segment registers for the 2
tasks with the same VSIDs? I think that would be a really really bad
idea (for a start it limits the number of objects you can map to
around 10 or so). Doing that would reduce the number of hash table
entries slightly but believe me, hash table space is not a problem.
We are currently using VSIDs as a lazy TLB reloader, not for
a large VM space context manager. There shouldn't be any need
You're intent on solving a problem that doesn't exist, just because
the hardware can solve it. On a 32-bit system, linux processes don't
want and indeed have no notion of a large VM space, if by large you
mean more than a few gigabytes.
And saying "lazy TLB reloader" is misleading. Lazy TLB reloading is
what you do when your MMU doesn't have any support for MMU contexts,
in order to try to minimize TLB flushes. Fortunately Linux knows
about MMUs that support MMU contexts.
for CONFIG_HIGHMEM on PowerPC, and we should be able to map huge
PCI spaces we see on the multiple bridge CPCI systems without
any worry of running out of VM space.
On a 32-bit PPC you can't ever see more than 4GB of virtual address
space at once without changing mappings. We really don't want to go
down the path of trying to use the segment registers like the bank
registers you used to get on 16-bit systems 25 years ago. For
example, if we mapped in the highmem pages like this, it would
drastically limit the number of highmem pages that we could have
mapped in at once.
The reason this is hard to realize today is we have built up a
system with the assumption there is only a 32-bit virtual space,
half of it consumed by a user task. When I saw the VM changes
happening during 2.3, I saw hoping we could take the opportunity
to improve PowerPC performance with better MMU management. Maybe
someday I can work on it.
Let me know when you've rewritten the whole of the Linux generic MM
layer. :-)
The reason for having the user task space visible in the kernel is so
that copy_to_user/copy_from_user can go fast, without having to set up
any mappings or do any more than a range-check on the address.
--
Paul Mackerras, Open Source Research Fellow, Linuxcare, Inc.
+61 2 6262 8990 tel, +61 2 6262 8991 fax
paulus@linuxcare.com.au, http://www.linuxcare.com.au/
Linuxcare. Putting Open Source to work.
** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/
From: Dan Malek <hidden> Date: 2001-02-06 22:29:30
Paul Mackerras wrote:
quoted
That's not what MMU context means, well at least the way I have
learned to use it in the past. An MMU context is supposed to represent
the virtual mapping of memory objects. Linux has memory objects
No, an MMU context represents an address space, or more precisely the
set of virtual to physical mappings in an address space,...
Isn't that what I just said above :-)? Your original message said
you want to map some context to just a few of the VSIDs, that is what
I said isn't correct.
On machines like the x86 where the MMU doesn't know about MMU contexts
you have to basically context-switch the whole MMU including the TLB.
Fortunately we don't have to do that. :)
Well, an MMU doesn't have to know about contexts (or have something
called a 'context register') for you to implement MMU context management.
-- Dan
** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/
From: Paul Mackerras <hidden> Date: 2001-02-06 22:45:57
Dan Malek writes:
quoted
quoted
That's not what MMU context means, well at least the way I have
learned to use it in the past. An MMU context is supposed to represent
the virtual mapping of memory objects. Linux has memory objects
No, an MMU context represents an address space, or more precisely the
set of virtual to physical mappings in an address space,...
Isn't that what I just said above :-)? Your original message said
That is one possible interpretation of what you said above. :) You
went on to talk about memory objects which is why I thought you were
saying that a context represented a mapping of a single object. I
think we're actually in violent agreement.
you want to map some context to just a few of the VSIDs, that is what
I said isn't correct.
What does "correct" mean in this context (excuse the pun :) ?
We can set up the VSIDs however we darn well please. We set up the
VSIDs so that the kernel portion of each context is the same. Where's
the problem?
In saying "we can effectively choose a different context for each
256MB segment" I was trying to say "if you think about MMUs in terms
of MMU contexts, then you can think of the PPC MMU as being able to
select a different context, not just per task, but per 256MB segment
of the address space of that task as well".
Well, an MMU doesn't have to know about contexts (or have something
called a 'context register') for you to implement MMU context management.
Sure. But it needs to know about more address bits (in some sense)
than the 32 bits of VA that the cpu core gives you. The x86 MMU
doesn't.
--
Paul Mackerras, Open Source Research Fellow, Linuxcare, Inc.
+61 2 6262 8990 tel, +61 2 6262 8991 fax
paulus@linuxcare.com.au, http://www.linuxcare.com.au/
Linuxcare. Putting Open Source to work.
** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/
From: Dan Malek <hidden> Date: 2001-02-06 23:14:40
Paul Mackerras wrote:
Are you saying that when two tasks both have an object mapped, we
should be using the same virtual address in PPC terms
No. Simply that the PowerPC allows us to utilize a greater than
32-bit address and an MMU context should represent that and allow
us to manage it.
You're intent on solving a problem that doesn't exist, just because
the hardware can solve it.
I think the CONFIG_HIGHMEM is clearly a problem that wasn't
implemented very well. We are seeing embedded PowerPCs with 1G or
larger real memory, along with lots of PCI busses with devices
that require mapping. This simply doesn't work today without some
kernel modifications. Clearly problems that need solutions.
And saying "lazy TLB reloader" is misleading. Lazy TLB reloading is
what you do when your MMU doesn't have any support for MMU contexts,
in order to try to minimize TLB flushes.
That's exactly what we are doing on PowerPC. We use the VSIDs as a
kind of context register, not as an extension of virtual space addressing.
We minimize TLB flushes by using different VSIDs for Linux MMU contexts.
When switching MMU contexts, we simply allow the new VSIDs to miss
and cause a TLB reload. It's not misleading, it is exactly what
we are doing. We aren't actually minimizing anything, just postponing
the TLB reloads. Actively switching MMU contexts allows the ability
to preload TLB entries, minimizing misses upon the context switch,
and also provides a more predictable context switch behavior. Do it
right, get more features and flexibility.
On a 32-bit PPC you can't ever see more than 4GB of virtual address
space at once without changing mappings.
Yep.
.... We really don't want to go
down the path of trying to use the segment registers like the bank
registers you used to get on 16-bit systems 25 years ago.
We already have. That is basically what CONFIG_HIGHMEM does...just
a horse of a different color.
....... For
example, if we mapped in the highmem pages like this, it would
drastically limit the number of highmem pages that we could have
mapped in at once.
But that is effectively what we are (Linus is :-) doing. It is
competition for a finite resource. My only suggestion is we take
advantage of the PowerPC hardware to do it for us, and then have
a cleaner path to a 64-bit (or simply larger physical address space)
implementations (like the 7450).
Let me know when you've rewritten the whole of the Linux generic MM
layer. :-)
Linus is learning :-). We don't have to rewrite much of it anymore.
We just need to use it more wisely in our processor specific porting.
The reason for having the user task space visible....
I've known about that trade off for about 25 years, and I am sure it
was discussed long before that. From experience, I tend to favor
the other direction. The user/kernel spaces are separate and you
should design a system that treats them as such. If you want to
share the space, you end up with empty functions/macros for mapping.
Much easier than trying to go the other way. You also uncover lots
of programming errors, especially the pointers are not integers, that
makes it much easier to scale up to larger systems. There are just
way too many advantages to separate user/kernel spaces and the trivial
data copy argument should just never prevail.
-- Dan
** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/
From: Paul Mackerras <hidden> Date: 2001-02-07 00:23:58
Dan Malek writes:
No. Simply that the PowerPC allows us to utilize a greater than
32-bit address and an MMU context should represent that and allow
us to manage it.
I think would need to see a more concrete proposal to really
understand what you're getting at.
quoted
And saying "lazy TLB reloader" is misleading. Lazy TLB reloading is
what you do when your MMU doesn't have any support for MMU contexts,
in order to try to minimize TLB flushes.
That's exactly what we are doing on PowerPC. We use the VSIDs as a
kind of context register, not as an extension of virtual space addressing.
No, let's be clear, we are *not* doing lazy TLB switching as the term
is used in the linux kernel. As I understand it, on x86 the kernel
avoids switching the page tables on a context switch if possible, so
in fact a process running in the kernel can in some cases be running
using the page tables of another task. This is a win in the cases
where you switch to a task which does a small amount of processing
(and doesn't access its userspace area) then goes back to sleep.
We minimize TLB flushes by using different VSIDs for Linux MMU contexts.
When switching MMU contexts, we simply allow the new VSIDs to miss
and cause a TLB reload. It's not misleading, it is exactly what
we are doing. We aren't actually minimizing anything, just postponing
the TLB reloads. Actively switching MMU contexts allows the ability
to preload TLB entries, minimizing misses upon the context switch,
Preloading TLB entries just means you take the hit now instead of
later, how does that help? In general it's better to take the hit
later because you can't predict with 100% accuracy which TLB entries
will be needed. If you preload an entry which isn't then used, you've
wasted time.
quoted
.... We really don't want to go
down the path of trying to use the segment registers like the bank
registers you used to get on 16-bit systems 25 years ago.
We already have. That is basically what CONFIG_HIGHMEM does...just
a horse of a different color.
No, the point about bank registers, or segment registers, is that
there are only a few of them, whereas there are lots of pages. That's
why it is better to map in high memory pages using PTEs instead of
segment registers.
You talk about "utilizing a greater than 32 bit address". But on a
32-bit platform you have to do some kind of mapping in order to access
that. You can do that mapping with PTEs or with segment registers,
but the trouble with segment registers is that there are far too few
of them to be useful.
But that is effectively what we are (Linus is :-) doing. It is
competition for a finite resource. My only suggestion is we take
advantage of the PowerPC hardware to do it for us, and then have
a cleaner path to a 64-bit (or simply larger physical address space)
implementations (like the 7450).
Sounds nice, got a concrete proposal? It's easy to criticise.
I've known about that trade off for about 25 years, and I am sure it
was discussed long before that. From experience, I tend to favor
the other direction. The user/kernel spaces are separate and you
should design a system that treats them as such. If you want to
Linux does, that's why there is copy_to/from_user, get/put_user etc.
The sparc64 port has separate user and kernel spaces, but there at
least you have the lda and sta instructions (load/store to/from
alternate address space) which can access the user space with no
overhead.
--
Paul Mackerras, Open Source Research Fellow, Linuxcare, Inc.
+61 2 6262 8990 tel, +61 2 6262 8991 fax
paulus@linuxcare.com.au, http://www.linuxcare.com.au/
Linuxcare. Support for the revolution.
** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/
From: Roman Zippel <hidden> Date: 2001-02-07 09:18:46
Hi,
On Tue, 6 Feb 2001, Dan Malek wrote:
We are currently using VSIDs as a lazy TLB reloader, not for
a large VM space context manager. There shouldn't be any need
for CONFIG_HIGHMEM on PowerPC, and we should be able to map huge
PCI spaces we see on the multiple bridge CPCI systems without
any worry of running out of VM space.
Somewhere I lost you, could you explain it a bit more (maybe with an
example)? What has CONFIG_HIGHMEM to do with the pci mapping???
bye, Roman
** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/
From: Dan Malek <hidden> Date: 2001-02-07 17:46:25
Roman Zippel wrote:
Somewhere I lost you, could you explain it a bit more (maybe with an
example)? What has CONFIG_HIGHMEM to do with the pci mapping???
The only thing they have in common is the desire to consume kernel
virtual space. They were just two examples of the reason I am trying
to find methods of increasing the kernel virtual space.
-- Dan
** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/
From: Dan Malek <hidden> Date: 2001-02-07 18:02:22
Paul Mackerras wrote:
Sounds nice, got a concrete proposal? It's easy to criticise.
Sorry, I don't mean to criticize (or criticise :-). Well, maybe
just a little......in general some of the Linux design seems to
be just a little short sighted (not picking on you Paul, I even
do this :-). We tend to fix an immediate problem today without
stepping back for 30 seconds to apply some knowledge gained over
the past 50 years of computer development.
Over the past month or so, I have been part of some new processor
ports and trying to find solutions for large memory systems. It
has pushed me deep into the VM design again, I am trying to clean
up some of the redundant functions we have and find some longer term
solutions instead of providing a "hack of the day" that will just
need to be thrown away in favor of something else tomorrow.
I don't yet have a proposal, and in retrospect it does seem the
discussion started on a negative tone. I'm thinking about flying
out to some Caribbean island for a week or so to work on this (among
other things :-).
I have a couple of things to update for the other processor work I
am doing, and after that I will experiment with a few things. I'll
let you know if I can find something useful.
-- Dan
** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/
From: Roman Zippel <hidden> Date: 2001-02-07 18:39:23
Hi,
On Wed, 7 Feb 2001, Dan Malek wrote:
quoted
Somewhere I lost you, could you explain it a bit more (maybe with an
example)? What has CONFIG_HIGHMEM to do with the pci mapping???
The only thing they have in common is the desire to consume kernel
virtual space. They were just two examples of the reason I am trying
to find methods of increasing the kernel virtual space.
CONFIG_HIGHMEM has little to do with the kernel virtual space. The main
purpose is to get the memory into user space, where user programs can use
them. I really don't understand, what problem you're trying to solve. The
kernel only wants to manage high memory and rarely access it (and uses
kmap for this), most of the time these pages should be in the user virtual
address space only.
bye, Roman
** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/
From: Gabriel Paubert <hidden> Date: 2001-02-07 21:16:28
Hi,
CONFIG_HIGHMEM has little to do with the kernel virtual space. The main
purpose is to get the memory into user space, where user programs can use
them. I really don't understand, what problem you're trying to solve. The
kernel only wants to manage high memory and rarely access it (and uses
kmap for this), most of the time these pages should be in the user virtual
address space only.
Confused. If you have up to 64 Gb or RAM (the 7450 has 36 address lines)
and can use not even use one gigabyte for cache, you end up with a
seriously unbalanced system. Depending on what you do of course.
Hey 64 Gb of RAM is not even that expensive today (the only problem is
buffering and additional latency caused by the additional layers
you need to access 100+ modules).
Gabriel.
** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/
From: Paul Mackerras <hidden> Date: 2001-02-08 00:34:24
Gabriel Paubert writes:
quoted
CONFIG_HIGHMEM has little to do with the kernel virtual space. The main
purpose is to get the memory into user space, where user programs can use
them. I really don't understand, what problem you're trying to solve. The
kernel only wants to manage high memory and rarely access it (and uses
kmap for this), most of the time these pages should be in the user virtual
address space only.
Confused. If you have up to 64 Gb or RAM (the 7450 has 36 address lines)
and can use not even use one gigabyte for cache, you end up with a
seriously unbalanced system. Depending on what you do of course.
The highmem pages are used mainly as page-cache pages and anonymous
user pages. At the moment the kernel uses bounce buffers for doing
I/O to/from highmem pages but that is on the list of things to be
fixed. :)
Paul.
--
Paul Mackerras, Open Source Research Fellow, Linuxcare, Inc.
+61 2 6262 8990 tel, +61 2 6262 8991 fax
paulus@linuxcare.com.au, http://www.linuxcare.com.au/
Linuxcare. Putting open source to work.
** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/
From: Paul Mackerras <hidden> Date: 2001-02-08 00:48:49
Dan Malek writes:
I don't yet have a proposal, and in retrospect it does seem the
discussion started on a negative tone. I'm thinking about flying
Well, it did start with a remark of yours about our "mostly incorrect
use of the MMU". :)
I think that it is seductive but misleading when the PPC docs say that
the PPC supports a "large virtual address space". Yes it does, in a
sense, but it's not one that you can access directly on a 32-bit PPC;
you have to go changing segment registers to get at all of it - i.e.,
there's no such thing as a pointer into this space that the CPU
understands natively. You could have such a pointer as a 64-bit
value, but to use it you have to split it apart and put one part in a
segment register and make a pointer that the CPU understands out of
the other part. And the other thing is that there just aren't enough
segment registers on 32-bit PPCs to be really useful (except as a way
of coarsely partitioning the address space).
And of course when you go to a 64-bit PPC all of those sort of
problems go away; you just have 64-bit pointers and set up your
logical -> physical translation however you want, and you can then
basically ignore the MMU's "virtual" address space except as something
internal to the translation process that only really affects what
values you have to put in the segment and hash tables.
Paul.
--
Paul Mackerras, Open Source Research Fellow, Linuxcare, Inc.
+61 2 6262 8990 tel, +61 2 6262 8991 fax
paulus@linuxcare.com.au, http://www.linuxcare.com.au/
Linuxcare. Putting open source to work.
** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/
From: Frank Rowand <hidden> Date: 2001-02-08 01:39:21
Paul Mackerras wrote:
I think that it is seductive but misleading when the PPC docs say that
the PPC supports a "large virtual address space". Yes it does, in a
sense, but it's not one that you can access directly on a 32-bit PPC;
you have to go changing segment registers to get at all of it - i.e.,
there's no such thing as a pointer into this space that the CPU
understands natively. You could have such a pointer as a 64-bit
value, but to use it you have to split it apart and put one part in a
segment register and make a pointer that the CPU understands out of
the other part. And the other thing is that there just aren't enough
segment registers on 32-bit PPCs to be really useful (except as a way
of coarsely partitioning the address space).
Then there are the PPC processors that don't even have segment registers....
Paul.
-Frank
--
Frank Rowand [off-list ref]
MontaVista Software, Inc
** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/