Hi list
I have been researching why I can't use dcbz and friends on 8xx CPU.
Till Strauman found out long ago that the dcxx instuctions did not update
the DAR register and that was the key I needed.
After a lot of testing/reading I found a 2 instruction workaround
that lets me use dcbz in copy_tofrom_user().
I can also used dcbz in a user space app without problems. I even
removed the TLB change bit hack by Tom Rini and everything works.
By copying the MD_EPN register to the DAR register in the DTLB Miss handler
just before MD_RPN is written everything just works. The only explanation that
makes sense to me is that the MD_RPN has a undocumented dependency to the DAR
register, the 20 msb of MD_EPN and DAR needs to be the same before MD_RPN is written.
Does this make any sense to anyone?
I can prepare a patch if somebody is interested to try it on their 8xx CPU.
Jocke
** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/
Does this make any sense to anyone?
I can prepare a patch if somebody is interested to try it on their 8xx CPU.
Jocke
I forgot:
Why is the Guarded flag set in DTLB Miss handler for the pte?
Won't performance degrade?
Jocke
PS.
Here is the patch if anyone wants to test. Please report your results.
Don't forget to enable dcbz in arch/ppc/kernel/misc.S and arch/ppc/lib/string.S
for 8xx.
Index: arch/ppc/kernel/head_8xx.S
===================================================================
RCS file: /home/cvsadmin/cvsroot/kernel/linuxppc/arch/ppc/kernel/head_8xx.S,v
retrieving revision 1.1.1.1
diff -u -r1.1.1.1 head_8xx.S
--- arch/ppc/kernel/head_8xx.S 1 Nov 2002 13:44:02 -0000 1.1.1.1+++ arch/ppc/kernel/head_8xx.S 24 Mar 2003 10:46:22 -0000
@@ -407,6 +407,13 @@rlwinm.r20,r21,0,0,19/*Extractpagedescriptorpageaddress*/beq2f/*Ifzero,don't try to find a pte */+/*Forsomereasonthe20msbofMD_EPNandDARmustbethesamewhen+*MD_RPNiswritten.Thedcxxinstructionsdon't set DAR when they+*causeaDTLBMisssocopythemfromMD_EPN.+*/+mfsprr20,MD_EPN+mtsprDAR,r20+/*Wehaveaptetable,soloadfetchtheptefromthetable.*/tophys(r21,r21)
Hi list
I have been researching why I can't use dcbz and friends on 8xx CPU.
Till Strauman found out long ago that the dcxx instuctions did not update
the DAR register and that was the key I needed.
After a lot of testing/reading I found a 2 instruction workaround
that lets me use dcbz in copy_tofrom_user().
I can also used dcbz in a user space app without problems. I even
removed the TLB change bit hack by Tom Rini and everything works.
By copying the MD_EPN register to the DAR register in the DTLB Miss handler
just before MD_RPN is written everything just works. The only explanation that
makes sense to me is that the MD_RPN has a undocumented dependency to the DAR
register, the 20 msb of MD_EPN and DAR needs to be the same before MD_RPN is written.
hmm, this is not quite correct. I can copy MD_EPN to DAR after the write to MD_RPN
and it still works. Seems like DAR must be correct when rfi is executed. I wonder
why?
Jocke
** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/
Hi list
I have been researching why I can't use dcbz and friends on 8xx CPU.
Till Strauman found out long ago that the dcxx instuctions did not update
the DAR register and that was the key I needed.
After a lot of testing/reading I found a 2 instruction workaround
that lets me use dcbz in copy_tofrom_user().
I can also used dcbz in a user space app without problems. I even
removed the TLB change bit hack by Tom Rini and everything works.
By copying the MD_EPN register to the DAR register in the DTLB Miss handler
just before MD_RPN is written everything just works. The only explanation that
makes sense to me is that the MD_RPN has a undocumented dependency to the DAR
register, the 20 msb of MD_EPN and DAR needs to be the same before MD_RPN is written.
hmm, this is not quite correct. I can copy MD_EPN to DAR after the write to MD_RPN
and it still works. Seems like DAR must be correct when rfi is executed. I wonder
why?
hmm, can it be that when you return from the DTLB Miss, you sometimes get an DTLB error
immediately and that DTLB Error "inherits" the DAR from the DTLB Miss?
I removed the "copy DAR to MD_EPN" part in the DTLB Error handler and my systems
runs fine.
Jocke
** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/
From: Till Straumann <hidden> Date: 2003-03-24 17:09:52
Joakim Tjernlund wrote:
quoted
quoted
Hi list
-- snip --
quoted
quoted
I even
removed the TLB change bit hack by Tom Rini and everything works.
-- snip --
I removed the "copy DAR to MD_EPN" part in the DTLB Error handler and my systems
runs fine.
How exactly do you know that "everything works fine" ???
Please note that
a) you may have bug-free hardware.
b) you may have a valid TLB entry at the time 'dcbz' is executed.
c) you may end up with a bogus mapping but not notice it.
You really should first confirm that your hardware suffers from the bug by
running a _simple_ test program in a _simple_ environment (NOT linux).
Then, you should confirm that your workaround solution is correct by
testing it in the same simple environment.
Just because you are able to use 'dcbz' on linux repeatedly without
crashing does'nt prove that your hardware is buggy but linux handles
it correctly.
-- Till
PS: please CC me on any replies; I'm not subscribed to this list
I even
removed the TLB change bit hack by Tom Rini and everything works.
-- snip --
quoted
I removed the "copy DAR to MD_EPN" part in the DTLB Error handler and my systems
runs fine.
How exactly do you know that "everything works fine" ???
Please note that
a) you may have bug-free hardware.
b) you may have a valid TLB entry at the time 'dcbz' is executed.
c) you may end up with a bogus mapping but not notice it.
Every time I tried to use dcbz on user space it failed HARD directly.
Whenever I tried to "undo" any hacks that were there to compensate for
buggy dcxx instructions, the kernel would not boot.
Now I can boot and run my app and even run a user space app which clears
memory with dcbz. I can remove HWWRITE bit setting on kernel space address and
remove the copying of DAR to MD_EPN in the DTLB Error handler.
I think I am on to something here, don't you?
You really should first confirm that your hardware suffers from the bug by
running a _simple_ test program in a _simple_ environment (NOT linux).
I don't have a simple environment. I have ppcboot on my board which doesn't
use the MMU at all.
Then, you should confirm that your workaround solution is correct by
testing it in the same simple environment.
Just because you are able to use 'dcbz' on linux repeatedly without
crashing does'nt prove that your hardware is buggy but linux handles
it correctly.
Yes, I know that and that's why I write to this list to seek input from
others that know more than I do and have a simple test environment(hint, hint :-)
-- Till
PS: please CC me on any replies; I'm not subscribed to this list
On Mon, Mar 24, 2003 at 11:19:29AM +0100, Joakim Tjernlund wrote:
Hi list
I have been researching why I can't use dcbz and friends on 8xx CPU.
Till Strauman found out long ago that the dcxx instuctions did not update
the DAR register and that was the key I needed.
After a lot of testing/reading I found a 2 instruction workaround
that lets me use dcbz in copy_tofrom_user().
I can also used dcbz in a user space app without problems. I even
removed the TLB change bit hack by Tom Rini and everything works.
By copying the MD_EPN register to the DAR register in the DTLB Miss handler
just before MD_RPN is written everything just works. The only explanation that
makes sense to me is that the MD_RPN has a undocumented dependency to the DAR
register, the 20 msb of MD_EPN and DAR needs to be the same before MD_RPN is written.
Does this make any sense to anyone?
I can prepare a patch if somebody is interested to try it on their 8xx CPU.
Can you post a patch which does everything (remove the TLB change bit,
etc)? Thanks.
Sure, but I don't want to restore the old TLB change bit. I removed it as a test only to see if
my system could take it. The kernel will do a lot DTLB errors if it is restored.
I will removed the Guarded flag though since I don't see why it's there. Speak up
now if you want to have it left.
Jocke
** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/
On Mon, Mar 24, 2003 at 09:57:43PM +0100, Joakim Tjernlund wrote:
quoted
Can you post a patch which does everything (remove the TLB change bit,
etc)? Thanks.
Sure, but I don't want to restore the old TLB change bit. I removed it as a test only to see if
my system could take it. The kernel will do a lot DTLB errors if it is restored.
I will removed the Guarded flag though since I don't see why it's there. Speak up
now if you want to have it left.
Well, if I understand what you think you've got working correctly, what
you've got replaces the current workaround for a similar (related?)
issue.
--
Tom Rini
http://gate.crashing.org/~trini/
** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/
From: Dan Malek <hidden> Date: 2003-03-24 21:42:16
Joakim Tjernlund wrote:
I will removed the Guarded flag though since I don't see why it's there. Speak up
now if you want to have it left.
Everything is there for a reason, including setting the guarded flag if
the PTE indicates this should be done. Since it doesn't seem to be clear
from messages I've sent in the past, the guarded flag is used to ensure
the proper order of operation on I/O spaces. Any ioremap will request
this feature.
Thanks.
-- Dan
** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/
I will removed the Guarded flag though since I don't see why it's there. Speak up
now if you want to have it left.
Everything is there for a reason, including setting the guarded flag if
the PTE indicates this should be done. Since it doesn't seem to be clear
from messages I've sent in the past, the guarded flag is used to ensure
the proper order of operation on I/O spaces. Any ioremap will request
this feature.
I got the impression that the Guarded flag was ALWAYS set. Reading
the comment again I think I missunderstood(my assembly is rusty), right?
Jocke
On Mon, Mar 24, 2003 at 09:57:43PM +0100, Joakim Tjernlund wrote:
quoted
quoted
Can you post a patch which does everything (remove the TLB change bit,
etc)? Thanks.
Sure, but I don't want to restore the old TLB change bit. I removed it as a test only to see if
my system could take it. The kernel will do a lot DTLB errors if it is restored.
I will removed the Guarded flag though since I don't see why it's there. Speak up
now if you want to have it left.
Well, if I understand what you think you've got working correctly, what
you've got replaces the current workaround for a similar (related?)
issue.
I think it makes the "copy DAR to MD_EPN" part in the DTLB error handler needless.
Anyway it works with and without that part.
Jocke
** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/
Can you post a patch which does everything (remove the TLB change bit,
etc)? Thanks.
--
Tom Rini
OK, here it goes. I had do this blind since I am still on 2.4.20. Let me know
if it does not compile.
Jocke
===== arch/ppc/kernel/head_8xx.S 1.41 vs edited =====
--- 1.41/arch/ppc/kernel/head_8xx.S Thu Feb 27 20:40:15 2003+++ edited/arch/ppc/kernel/head_8xx.S Mon Mar 24 23:31:27 2003
@@ -451,6 +451,13 @@#endifmtsprMD_RPN,r20/*UpdateTLBentry*/+/*The20msbofMD_EPNandDARmustbethesamewhenrfiis+*executed.Thedcxxinstructionsdon't set DAR when they+*causeaDTLBMisssocopythemfromMD_EPN.+*/+mfsprr20,MD_EPN+mtsprDAR,r20+mfsprr20,M_TW/*Restoreregisters*/lwzr21,0(r0)mtcrr21
From: Dan Malek <hidden> Date: 2003-03-24 22:59:48
Joakim Tjernlund wrote:
I think it makes the "copy DAR to MD_EPN" part in the DTLB error handler needless.
Anyway it works with and without that part.
No, you need to leave that there as well. Read the comment.....
If it appears to "work" for you, what happens is you have a DTLB Error
immediately following a DTLB miss (the common case lots of the time), so
the MD_EPN just happens to contain the proper information. When you generate
the case where a DTLB Error does not match a previous DTLB miss, you will
end up tracking down the wrong PTEs. Even in this case it is likely to
appear things work correctly, but they do not. There is one case with the dcbi
where there is no proper operation of the code, so it was necessary to solve
the problem another way.
This is exactly why people are telling you to carefully generate test cases
the prove an error exists 100% of the time, and that your modifications
properly correct the condition 100% of the time. When you run something
like Linux you get pathological cases that do not exercise all conditions
with a particular configurations, or you could be generating errors that
are not precisely tested as failure conditions.
Thanks.
-- Dan
** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/
From: Dan Malek <hidden> Date: 2003-03-24 23:09:11
Joakim Tjernlund wrote:
OK, here it goes.
No, this isn't correct......
+ /* The 20 msb of MD_EPN and DAR must be the same when rfi is
+ * executed. The dcxx instructions don't set DAR when they
+ * cause a DTLB Miss so copy them from MD_EPN.
No, don't be screwing around with these registers.
-#ifdef CONFIG_8xx
+#ifdef CONFIG_8xx_CPU6
What is this all about? The CPU6 errata is a clearly defined
problem on well known silicon revisions. Don't be using it where
it isn't intended.
Please try again..........thanks.
-- Dan
** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/
+ /* The 20 msb of MD_EPN and DAR must be the same when rfi is
+ * executed. The dcxx instructions don't set DAR when they
+ * cause a DTLB Miss so copy them from MD_EPN.
No, don't be screwing around with these registers.
Did you read my mails? This is the key.
What harm can it do? This is much less intrusive than the
"copy DAR to MD_EPN" part in the TLB Error handler.
quoted
-#ifdef CONFIG_8xx
+#ifdef CONFIG_8xx_CPU6
What is this all about? The CPU6 errata is a clearly defined
problem on well known silicon revisions. Don't be using it where
it isn't intended.
OK, I have removed all CONFIG_8xx_CPU6 and CONFIG_8xx
I restored the "copy DAR to MD_EPN" in the TLB Error hanler as well.
Please try again..........thanks.
Here it is:
===== arch/ppc/kernel/head_8xx.S 1.41 vs edited =====
--- 1.41/arch/ppc/kernel/head_8xx.S Thu Feb 27 20:40:15 2003+++ edited/arch/ppc/kernel/head_8xx.S Tue Mar 25 00:43:39 2003
@@ -451,6 +451,13 @@#endifmtsprMD_RPN,r20/*UpdateTLBentry*/+/*The20msbofMD_EPNandDARmustbethesamewhenrfiis+*executed.Thedcxxinstructionsdon't set DAR when they+*causeaDTLBMisssocopythemfromMD_EPN.+*/+mfsprr20,MD_EPN+mtsprDAR,r20+mfsprr20,M_TW/*Restoreregisters*/lwzr21,0(r0)mtcrr21
I think it makes the "copy DAR to MD_EPN" part in the DTLB error handler needless.
Anyway it works with and without that part.
No, you need to leave that there as well. Read the comment.....
If it appears to "work" for you, what happens is you have a DTLB Error
immediately following a DTLB miss (the common case lots of the time), so
the MD_EPN just happens to contain the proper information. When you generate
the case where a DTLB Error does not match a previous DTLB miss, you will
end up tracking down the wrong PTEs. Even in this case it is likely to
appear things work correctly, but they do not. There is one case with the dcbi
where there is no proper operation of the code, so it was necessary to solve
the problem another way.
What case is that(when dcbi does not work)? Give me an example so I can test
and see what happens, with and without my patch.
This is exactly why people are telling you to carefully generate test cases
the prove an error exists 100% of the time, and that your modifications
properly correct the condition 100% of the time. When you run something
like Linux you get pathological cases that do not exercise all conditions
with a particular configurations, or you could be generating errors that
are not precisely tested as failure conditions.
I know all that and thats why post to the list with questions, ideas and patches to try out.
If you had read my mails you would had understood that too.
Your yelling at me for trying isn't very helpful.
Jocke
** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/
From: Dan Malek <hidden> Date: 2003-03-25 00:48:24
Joakim Tjernlund wrote:
Did you read my mails?
I did. From this I gathered that you don't understand why some
of this code is there in the first place, you removed code that
would have guaranteed to cause an error if you would have run
the proper test, and you can't explain why the changes you made
cause the results you see. Not something that instills much
confidence.
All of these MMU and cache control registers have effects on
one another. Writing one register can have the side effect of
updating another. The DAR must have valid addresses in order
to properly rfi, we know some cache operations may not set this
properly, and we also know that in these cases the MD_EPN is also
not set properly. The reason it "works" for you is you found the
most likely case where you took a DTLB miss, followed by a DTLB Error
to the same page (or at least to valid pages). If the miss/error
sequence was to the same page, it will likely work correctly. If the
miss/error sequence is not to the same pages, it could appear
to "work" but chances are the dcbz was not executed properly and
you didn't notice the side effects of that. If you happen to
find the case where you generate the DTLB miss/error/error with
intervening TLB exceptions for other pages (like when swapping)
and the instructions involved are cache instructions (like the
dcbi in the comment example), you will end up with a big mess.
It's really hard to generate all of these test cases, and you
can't always use Linux to do so, but eventually Linux will stumble
across them.
For a long time, we "solved" these problems on 8xx by not properly
implementing copy-on-write pages and always marking pages dirty.
It reduced the likelyhood of the problem, reduced the MMU overhead,
but cost lots more in memory and processing time during forks.
This still may not be a bad configuration option for selected
applications.
Like I said from the beginning, the 100% guaranteed working solution
is to not use dcbz/dcbt on the 8xx. Anything else will eventually
stumble across the error conditions. If I'm developing a product
that uses Linux (or any other OS that will dynamically update the MMU
and take DTLB exceptions) I won't take the risk of failure using
these instructions.
-- Dan
** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/
From: Dan Malek <hidden> Date: 2003-03-25 01:22:54
After reading this I guess I need to further clarify what happens.....
Dan Malek wrote:
... The reason it "works" for you is you found the
most likely case where you took a DTLB miss, followed by a DTLB Error
to the same page...
What happens in these cases (which occurs like almost all of the time),
is there is "leftover" information in the control/status registers from
previous exceptions that causes the fault processing to take place
properly. In some error conditions, the leftover information isn't
exactly correct, causing the page fault handler to look up wrong
information or the dcbz/dcbt to do their thing on the wrong address.
If you are not running a specific test case and precisely testing
for the results of the operation, you may not notice this or at some
time in the future notice erratic system behavior that is impossible
to explain. The "normal" instructions have exactly one well documented
failure case, in the case of DTLBError, the MD_EPN sometimes doesn't
get the proper status, but we know the DAR does, so we can copy the
bits we need from the DAR to the EPN to properly process the fault.
The cache instructions have never been normal instructions. In some
versions of silicon they won't generate a fault, some will allow cache
operations to uncached spaces, some will fault but not provide the
proper status in any MMU register. Sometimes they work, it all depends
upon the state of the cache line and TLB. The cases most likely to
fail are the ones that are the hardest to create.
I've spent way too much time working on this on many versions of
silicon with the assistance of people from Motorola. It isn't a trivial
problem to detect, quantify and document so someone else can repeat
the exercise. Through the first four major versions of 860 silicon
(and countless minor versions) the wrong behavior I discussed was
fixed in the next release, only to have something else unexpected appear.
Since I didn't want a career in cache instruction debugging (and I had
products to get to market), the solution was to simply not use these
instructions.
Thanks.
-- Dan
** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/
I did. From this I gathered that you don't understand why some
of this code is there in the first place, you removed code that
No I don't understand this code fully. I don't think anybody but you understand
this code since it contains undocumented workarounds.
I asked the list before I removed it. Had you objected before I got to the
patch I had left it alone. It was just a test.
would have guaranteed to cause an error if you would have run
the proper test, and you can't explain why the changes you made
cause the results you see.
But I can explain(didn't I do that earlier?). We agree that dcxx will
cause DTLB Misses without setting DAR, right?. By copying the MD_EPN to
DAR before rfi, DAR will have address whose EPN is correct but with a
semirandom page offset(the last bits in MD_EPN that doesn't referrer to
the EPN). This must be better than just leaving dar as is, especially if
rfi needs a valid addess to operate properly as you write below.
The second part:
+ /* Copy 20 msb from EPN to DAR since the dcxx instuctions fails
+ * update the DAR when they cause a DTLB Miss.
+ */
+ mfspr r21, MD_EPN
+ rlwinm r21, r21, 0, 0, 19
+ mfspr r20, DAR
+ rlwinm r20, r20, 0, 20, 31
+ or r20, r20, r21
+ mtspr DAR, r20
Just replaces the EPN part to generate a valid address within the page. The lower bits
are untouched. This is a nop if DAR already contains the right address, otherwise DAR
will at least contain a valid page address.
This could be move to before error test if you want play safe(the 2 instructions before
rfi should be removed in this case)
Not something that instills much confidence.
After the above explanation I hope that will improve somewhat.
All of these MMU and cache control registers have effects on
one another. Writing one register can have the side effect of
updating another. The DAR must have valid addresses in order
to properly rfi, we know some cache operations may not set this
properly, and we also know that in these cases the MD_EPN is also
not set properly.
If MD_EPN is not set propely in the DTLB Miss, you are lost regardless of
my patch.
[SNIP]
For a long time, we "solved" these problems on 8xx by not properly
implementing copy-on-write pages and always marking pages dirty.
It reduced the likelyhood of the problem, reduced the MMU overhead,
but cost lots more in memory and processing time during forks.
This still may not be a bad configuration option for selected
applications.
Is it easy to patch/hard code? I would like to test the stability of my system.
Like I said from the beginning, the 100% guaranteed working solution
is to not use dcbz/dcbt on the 8xx. Anything else will eventually
stumble across the error conditions. If I'm developing a product
that uses Linux (or any other OS that will dynamically update the MMU
and take DTLB exceptions) I won't take the risk of failure using
these instructions.
I still that the patch to head_8xx.S has merit on its own. It will workaround
the case where DAR has an invalid value in the DTLB Miss handler.
Still, it would be nice if you could give the patch a go in your lab(with recent silicon).
Time to sleep for me
Jocke
** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/
After reading this I guess I need to further clarify what happens.....
Dan Malek wrote:
quoted
... The reason it "works" for you is you found the
most likely case where you took a DTLB miss, followed by a DTLB Error
to the same page...
What happens in these cases (which occurs like almost all of the time),
is there is "leftover" information in the control/status registers from
previous exceptions that causes the fault processing to take place
properly. In some error conditions, the leftover information isn't
exactly correct, causing the page fault handler to look up wrong
information or the dcbz/dcbt to do their thing on the wrong address.
If you are not running a specific test case and precisely testing
for the results of the operation, you may not notice this or at some
time in the future notice erratic system behavior that is impossible
to explain. The "normal" instructions have exactly one well documented
failure case, in the case of DTLBError, the MD_EPN sometimes doesn't
get the proper status, but we know the DAR does, so we can copy the
bits we need from the DAR to the EPN to properly process the fault.
The cache instructions have never been normal instructions. In some
versions of silicon they won't generate a fault, some will allow cache
operations to uncached spaces, some will fault but not provide the
proper status in any MMU register. Sometimes they work, it all depends
upon the state of the cache line and TLB. The cases most likely to
fail are the ones that are the hardest to create.
So maybe there still is silicon bugs which is visible in the DTLB Error handler.
Is there a problem with MD_EPN in DTLB Miss context?
We do know that the dcxx instructions (including those that are used by
xxx_dcache_range functions) does not update the DAR register. You say that
the rfi depends on DAR(I would like to know in what way, I can't find any info
about that) so should we not try to make sure that DAR contains a valid address
by copying the EPN from MD_EPN?
I have another idea: If you get a DTLB Error direcly after DTLB Miss for the
same page, DAR will NOT be updated(at least in some cases) and you "inherit"
the DAR from the preceeding DTLB Miss. I have tried to verify that but I have
not been able to construct a test case that will prove this. Any ideas on that one?
Either way the patch makes it possible to use dcbz in user space, I can remove
the "copy DAR to MD_EPN" part in the DTLB Error handler, restore the Change bit in
the pte for kernel space and still have a usable system, wheras doing any one of
these modifications without my patch resulted in hard lookups during boot.
That suggests to me that the DTLB Miss patch does some good.
Can you present a case where will be worse off with the DTLB Miss part of
my patch applied?
I've spent way too much time working on this on many versions of
silicon with the assistance of people from Motorola. It isn't a trivial
problem to detect, quantify and document so someone else can repeat
the exercise. Through the first four major versions of 860 silicon
(and countless minor versions) the wrong behavior I discussed was
fixed in the next release, only to have something else unexpected appear.
Since I didn't want a career in cache instruction debugging (and I had
products to get to market), the solution was to simply not use these
instructions.
I do understand your position, but please consider that the current impl.
may still have bugs or can be improved upon so please don't dismiss any
attempt to make the 8xx CPUs perform better under linux.
Again, you don't have to enable the use of dcbz(yet :-). The DTLB Miss modification
should IMHO go in regardless, unless you find a case were it does not work.
Jocke
** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/
For a long time, we "solved" these problems on 8xx by not properly
implementing copy-on-write pages and always marking pages dirty.
It reduced the likelyhood of the problem, reduced the MMU overhead,
but cost lots more in memory and processing time during forks.
This still may not be a bad configuration option for selected
applications.
Is it easy to patch/hard code? I would like to test the stability of my system.
I am still intrested in trying copy-on-write but I can't figure out
how. Is it a big job? Any pointers?
Jocke
** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/
From: Till Straumann <hidden> Date: 2003-03-26 02:17:11
Hi guys.
I revitalized my old test environment on a
mpc860t (XPC860TZP50B5 / 3J21M to be precise).
I found that 'dcbz' (while failing to set DAR)
indeed sets MD_EPN correctly. Hence, Jocke's fix
(copy EPN[0:19]->DAR) would handle that.
My older idea (fixing up MD_EPN and DAR based
on the faulting instruction opcode and the involved
GPR contents) should work even if we have neither
a valid MD_EPN nor DAR.
-- Till.
** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/
Thanks Till, you are the first one who actually tested
my fix.
I revitalized my old test environment on a
mpc860t (XPC860TZP50B5 / 3J21M to be precise).
hmm, B5 rev is pretty buggy. Good that it works on
this silicon as well.
I found that 'dcbz' (while failing to set DAR)
indeed sets MD_EPN correctly. Hence, Jocke's fix
(copy EPN[0:19]->DAR) would handle that.
Did you try any of the other dcxx instructions as well?
My older idea (fixing up MD_EPN and DAR based
on the faulting instruction opcode and the involved
GPR contents) should work even if we have neither
a valid MD_EPN nor DAR.
Yes, I tried to do that. I never got past how to convert the
virtual address in SRR0 to physical so I could read the instruction
and decode it.
From: Dan Malek <hidden> Date: 2003-03-26 15:18:11
Till Straumann wrote:
I found that 'dcbz' (while failing to set DAR)
indeed sets MD_EPN correctly. Hence, Jocke's fix
(copy EPN[0:19]->DAR) would handle that.
After sleeping for a couple of days and consuming large
amounts of medicine to cure a cold, I think I understand
why copying these bits around seems to "fix" problems.
It's all related to the sequence of TLB miss/error exceptions
that I had been describing all along. The first thing that
is going to most likely happen is you will get a TLB miss to
load a PTE into the TLB. It will be marked valid but not
dirty (not writable). Immediately upon performing the rfi
you will get a TLB Error to handle the dirty PTE update.
By copying the bits from MD_EPN to the DAR in the miss handler,
the Error handler will have at least a 4K boundary aligned DAR
and it will execute correctly to update the dirty state. At
this point, it will appear to "work" properly (even though
it is likely the dcbz didn't execute) because the system will
at least keep running (for a while).
If you have a situation where you get a TLB Error without
a matching TLB miss (very rare, but they can happen as the
result of swapping, copy on write, certain other page table
updates), then you are hosed. The DAR will contain some information
from a previous exception, we will likely end up on a "hung"
system continually taking TLB Error exceptions because we
can't fix them properly. This is basically what happens
without the bit copying "fix".
My older idea (fixing up MD_EPN and DAR based
on the faulting instruction opcode and the involved
GPR contents) should work even if we have neither
a valid MD_EPN nor DAR.
All of the TLB exception handlers must have minimal instructions.
The ones in Linux are too big already. The very little you would
gain from making a dcbz/dcbt work correctly would be lost many,
many, many times over in a more complex TLB exception handler.
Copying bits from MD_EPN to DAR doesn't set the DAR "correctly",
it only gives you the page boundary. This is going to further
confuse debuggers or signal handlers if you actually have an
addressing bug that is detected by one of these instructions.
The only update I would like to see to TLB exception handlers is
the removal of code due to streamlining of the page table organization.
Thanks.
-- Dan
** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/
From: Dan Malek <hidden> Date: 2003-03-26 15:22:18
Joakim Tjernlund wrote:
I am still intrested in trying copy-on-write but I can't figure out
how. Is it a big job? Any pointers?
The 8xx properly implements copy-on-write now. I don't remember when
I updated this, maybe somewhere in the mid 2.3.xxx timeframe. All of
the processor specific MM functions have to be changed, it isn't
something that is likely going to be a configuration option. due to
all of the common #define, macros, and page table functions. It
does remove three instructions and a branch from the TLBmiss handler :-)
-- Dan
** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/
I found that 'dcbz' (while failing to set DAR)
indeed sets MD_EPN correctly. Hence, Jocke's fix
(copy EPN[0:19]->DAR) would handle that.
After sleeping for a couple of days and consuming large
amounts of medicine to cure a cold, I think I understand
why copying these bits around seems to "fix" problems.
Sleeping is also good, I wish I got some more of that too :-)
It's all related to the sequence of TLB miss/error exceptions
that I had been describing all along. The first thing that
is going to most likely happen is you will get a TLB miss to
load a PTE into the TLB. It will be marked valid but not
dirty (not writable). Immediately upon performing the rfi
you will get a TLB Error to handle the dirty PTE update.
By copying the bits from MD_EPN to the DAR in the miss handler,
the Error handler will have at least a 4K boundary aligned DAR
and it will execute correctly to update the dirty state.
Yes, that's the conclusion I came to too.
At this point, it will appear to "work" properly (even though
it is likely the dcbz didn't execute)
hmm, why would not dcbz execute properly? Surely the instruction is restarted
when the DTLB Miss/Error handler return?
because the system will
at least keep running (for a while).
If you have a situation where you get a TLB Error without
a matching TLB miss (very rare, but they can happen as the
result of swapping, copy on write, certain other page table
updates), then you are hosed. The DAR will contain some information
from a previous exception, we will likely end up on a "hung"
system continually taking TLB Error exceptions because we
can't fix them properly. This is basically what happens
without the bit copying "fix".
In a earlier mail you said the there was a case where the "normal" instructions
would not set MD_EPN in the DTLB Error handler and that it was documented.
What case is this and where can I find this documentation?
What this boils down to is that you don't known what register, DAR or MD_EPN to
trust in the DTLB Error handler, right?
Maybe Till Straumann's suggestion to set the DAR register to a known "bad" address
when leaving an exception. Then test for this bad address in the DTLB error handler
to decide what register to trust?
Note that the other dcxx instructions(according to Till) does not set DAR either
so if a dcbi ever causes a
1) DTLB Miss followed by a DTLB Error
2) DTLB Miss taking the DataAccess path
3) DTLB Error taking the DataAccess path*
4) DTLB Error not the DataAccess path*
* = I assume that these does not update DAR either, can anyone confirm this?
then you are also hosed. My patch will at least fix 1 and 2.
Jocke
** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/
I am still intrested in trying copy-on-write but I can't figure out
how. Is it a big job? Any pointers?
The 8xx properly implements copy-on-write now. I don't remember when
I updated this, maybe somewhere in the mid 2.3.xxx timeframe. All of
the processor specific MM functions have to be changed, it isn't
something that is likely going to be a configuration option. due to
all of the common #define, macros, and page table functions. It
does remove three instructions and a branch from the TLBmiss handler :-)
Ahh, that explanins why I had such a hard time trying to find
where it didn't do copy-on-write :-)
Thanks
Jocke
** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/
From: Till Straumann <hidden> Date: 2003-03-26 18:11:34
Dan Malek wrote:
Till Straumann wrote:
quoted
I found that 'dcbz' (while failing to set DAR)
indeed sets MD_EPN correctly. Hence, Jocke's fix
(copy EPN[0:19]->DAR) would handle that.
After sleeping for a couple of days and consuming large
amounts of medicine to cure a cold, I think I understand
why copying these bits around seems to "fix" problems.
It's all related to the sequence of TLB miss/error exceptions
that I had been describing all along. The first thing that
is going to most likely happen is you will get a TLB miss to
load a PTE into the TLB. It will be marked valid but not
dirty (not writable). Immediately upon performing the rfi
you will get a TLB Error to handle the dirty PTE update.
By copying the bits from MD_EPN to the DAR in the miss handler,
the Error handler will have at least a 4K boundary aligned DAR
and it will execute correctly to update the dirty state. At
this point, it will appear to "work" properly (even though
it is likely the dcbz didn't execute) because the system will
at least keep running (for a while).
I'm not sure whether to agree with this analysis. I thought
you said that TLB Error correctly set DAR? Hence you would
get
- TLB Miss, copy EPN to DAR (DAR is semi-correct, i.e. to
a page boundary).
- RFI, dcbz is restarted
--- here, task switching could occur ---
- TLB Error because the page was not writable (but, according
to your earlier mail, DAR should be correct now).
- copy DAR-> EPN, lookup PTE, replace TLB
- RFI, dcbz is restarted and eventually executed.
Under the premise that TLB Miss always sets EPN correctly
and TLB Error always sets DAR correctly, this should work
even if a task switch (or IRQ) sneaks in.
If you have a situation where you get a TLB Error without
a matching TLB miss (very rare, but they can happen as the
result of swapping, copy on write, certain other page table
updates), then you are hosed.
Again: I thought you said that TLB Error _does_ set DAR?
The DAR will contain some information
from a previous exception, we will likely end up on a "hung"
system continually taking TLB Error exceptions because we
can't fix them properly. This is basically what happens
without the bit copying "fix".
quoted
My older idea (fixing up MD_EPN and DAR based
on the faulting instruction opcode and the involved
GPR contents) should work even if we have neither
a valid MD_EPN nor DAR.
All of the TLB exception handlers must have minimal instructions.
The ones in Linux are too big already. The very little you would
gain from making a dcbz/dcbt work correctly would be lost many,
many, many times over in a more complex TLB exception handler.
From a performance standpoint, perhaps (although I believe that
working around the problem is more of a 'size' than execution path
length increase). But you gain the huge advantage that you don't
have to carefully screen any software you intend to port for cache
instructions.
Copying bits from MD_EPN to DAR doesn't set the DAR "correctly",
it only gives you the page boundary. This is going to further
confuse debuggers or signal handlers if you actually have an
addressing bug that is detected by one of these instructions.
The 'fixup' workaround would provide a correct DAR, however.
-- Till
The only update I would like to see to TLB exception handlers is
the removal of code due to streamlining of the page table organization.
Thanks.
-- Dan
From: Dan Malek <hidden> Date: 2003-03-26 18:52:45
Joakim Tjernlund wrote:
hmm, why would not dcbz execute properly? Surely the instruction is restarted
when the DTLB Miss/Error handler return?
Write a test to verify it. I know there were cases in the past when
it didn't work. You would notice it when you expected to have some
zero initialized spaces that weren't.
In a earlier mail you said the there was a case where the "normal" instructions
would not set MD_EPN in the DTLB Error handler and that it was documented.
What case is this and where can I find this documentation?
Well, it's more of an interpretation of the documentation and discussions
with Motorola engineers.
What this boils down to is that you don't known what register, DAR or MD_EPN to
trust in the DTLB Error handler, right?
The registers we are discussing belong to two distinct functional areas.
The Mx_xxx registers are all part of the mmu functional unit and can serve
to be hardware assist to the software TLB management. The DAR, DSISR, SRRx
and so on are part of the PowerPC core, set during exceptions and used by
fault handlers to determine the course of action for fault repair. The
latter function is PowerPC generic, the former is unique to the 8xx
implementation.
I do not know why DAR is sometimes not set properly during some types
cache operation exceptions. In general, cache operation exceptions
have been unpredictable in various ways across all of the different
silicon revisions. Sometimes a dcbz to uncached space was ignored,
sometimes it generated an alignment exception (which is the correct
behavior), sometimes it generated tlb error exceptions, for example.
In all cases except some cache instructions, the DAR is properly set
for tlb error exceptions. It has to be or the system won't function
properly, so there isn't any question as to whether we can trust it.
For TLB miss exceptions, the Mx_xxx hardware assist registers work
properly, as documented, to minimize the number of instructions needed
to determine what PTE to load. The TLB miss exception has exactly
one function, to as quickly as possible locate the PTE from the table
hierarchy and stuff the bits into the MMU. That's it and it should
really be implemented in five lines of code.
The interpretation of the value of the Mx_xxx registers during TLB Error
exceptions has always been under discussion. Sometimes you can read they
should work like the TLB miss case, but since we aren't processing a
TLB miss exception, perhaps they aren't set. Perhaps when it appears to
work it is the result of residual information from a previous TLB miss,
other times we know it has been set as the result of the exception. In
any case, we know if we copy the bits from the DAR into the MD_EPN, we
can then use the hardware assist of these registers to perform the PTE
lookup just like the TLB miss function.
We optimize the update of the usage and dirty indicators in the PTE
right in the TLB Error exception function to more closely emulate "real"
PPC processors. This way, we can use the same fault processing functions.
If we didn't wish to do this, we could simply call an 8xx specific version
of do_fault, in which case it doesn't matter what is in the Mx_xxx registers
because we would do a software lookup using DAR. It wouldn't be as efficient
and we couldn't use a common PPC function, but we wouldn't have to question
the workings of the silicon in this case.
So, we can always trust DAR, it's just that MD_EPN is left to your interpretation
of the documentation and depends upon the implementation of the functions.
Maybe Till Straumann's suggestion to set the DAR register to a known "bad" address
when leaving an exception. Then test for this bad address in the DTLB error handler
to decide what register to trust?
Not necessary.
By not "correctly" tracking usage and dirty pages, as we did in the past,
it caused us to never generate DTLB Error exceptions unless it was truly
an access to a non-recoverable bad address or if the page was swapped out.
I would just set 'used' on every instruction page and 'used + dirty' on
every data page allocation. A static embedded system would just converge on
this point anyway, but in more dynamic systems it cost more real memory.
Of course, the 8xx has custom cache management/control functions that
have always seemed to work properly.
-- Dan
** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/
From: Till Straumann <hidden> Date: 2003-03-26 18:52:49
Jocke.
There's of course a trivial test to check Dan's suspicion
that it might just be a lucky sequence of TLBMiss-TLBError
exceptions helping you out.
Just load some bogus values (a la 0xdeadbeef) into
MD_EPN and DAR shortly prior to returning from the
TLBMiss exception (but after walking the tables, of
course). Thereby you make sure that any 'leftover'
values seen by a subsequent TLBError are invalidated.
-- Till
Dan Malek wrote:
Till Straumann wrote:
quoted
I found that 'dcbz' (while failing to set DAR)
indeed sets MD_EPN correctly. Hence, Jocke's fix
(copy EPN[0:19]->DAR) would handle that.
After sleeping for a couple of days and consuming large
amounts of medicine to cure a cold, I think I understand
why copying these bits around seems to "fix" problems.
It's all related to the sequence of TLB miss/error exceptions
that I had been describing all along. The first thing that
is going to most likely happen is you will get a TLB miss to
load a PTE into the TLB. It will be marked valid but not
dirty (not writable). Immediately upon performing the rfi
you will get a TLB Error to handle the dirty PTE update.
By copying the bits from MD_EPN to the DAR in the miss handler,
the Error handler will have at least a 4K boundary aligned DAR
and it will execute correctly to update the dirty state. At
this point, it will appear to "work" properly (even though
it is likely the dcbz didn't execute) because the system will
at least keep running (for a while).
If you have a situation where you get a TLB Error without
a matching TLB miss (very rare, but they can happen as the
result of swapping, copy on write, certain other page table
updates), then you are hosed. The DAR will contain some information
from a previous exception, we will likely end up on a "hung"
system continually taking TLB Error exceptions because we
can't fix them properly. This is basically what happens
without the bit copying "fix".
quoted
My older idea (fixing up MD_EPN and DAR based
on the faulting instruction opcode and the involved
GPR contents) should work even if we have neither
a valid MD_EPN nor DAR.
All of the TLB exception handlers must have minimal instructions.
The ones in Linux are too big already. The very little you would
gain from making a dcbz/dcbt work correctly would be lost many,
many, many times over in a more complex TLB exception handler.
Copying bits from MD_EPN to DAR doesn't set the DAR "correctly",
it only gives you the page boundary. This is going to further
confuse debuggers or signal handlers if you actually have an
addressing bug that is detected by one of these instructions.
The only update I would like to see to TLB exception handlers is
the removal of code due to streamlining of the page table organization.
Thanks.
-- Dan
From: Dan Malek <hidden> Date: 2003-03-26 19:06:34
Till Straumann wrote:
- TLB Miss, copy EPN to DAR (DAR is semi-correct, i.e. to
a page boundary).
- RFI, dcbz is restarted
The EPN has never been copied to DAR. Some bits of DAR are copied
to EPN so we can use the hardware assist in the TLB Error exception.
quoted hunk
--- here, task switching could occur ---
No, it can't because the TLB Miss rfi is likely to immediately result in
a TLB Error. The TLB Miss doesn't return through a path that causes
a context switch.
Under the premise that TLB Miss always sets EPN correctly
and TLB Error always sets DAR correctly, this should work
even if a task switch (or IRQ) sneaks in.
The current implementation relies on TLB Miss always setting EPN
(and other hardware assist registers) correctly, and TLB Error
relies on DAR being set correctly. Everything works fine until
you get a TLB Error on a dcbz/dcbt that doesn't properly update DAR.
In this case EPN isn't set correctly either.
Again: I thought you said that TLB Error _does_ set DAR?
It does, except in the case of some cache instruction exceptions.
... But you gain the huge advantage that you don't
have to carefully screen any software you intend to port for cache
instructions.
The instructions that don't work properly are dcbz/dcbt. We have
corrected this in the kernel and all GNU libraries long ago. There
are other reasons to need a unique 8xx (and 4xx) set of libraries
other than the use of these cache instructions.
-- Dan
** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/
From: Till Straumann <hidden> Date: 2003-03-26 22:42:45
Here are the results of some tests I ran
on my mpc860t (XPC860TZP50B5 / 3J21M).
I use mpc8bug on a FADS board to load motorola's
'init860.S' example. The example was slightly
modified to include (1-4 use differend EAs, of
course).
1) a 1:1 TLB to DRAM, R/W access (already present
in the vanilla code)
2) a TLB to map DRAM r/w, caching inhibited
3) a TLB to map DRAM r/o (behavior is the same
if for a 'real RO' mapping and for a RW
mapping with the 'CHANGE' bit cleared).
4) a TLB to map DRAM r/w but with the VALID
bit clear.
5) a 'main' routine which executes the
instructions under test through one
of the aforementioned TLB mappings.
Here's what I found (I generally load bogus
values into MD_EPN/DAR prior to executing any
of the cache instructions under test)
A) All of dcbf, dcbi, dcbst, dcbz (dcbt and dcbtst
should not and do not) raise a TLBMiss exception
with MD_EPN set correctly but failing to set DAR
(i.e. DAR is left unmodified) when trying to
access an address not in any of the TLBs.
B) All of dcbf, dcbi, dcbst, dcbz raise a TLBError
exception when accessing mapping 4) [as they should].
Unlike regular loads/stores (non-cache instructions),
DAR is NOT SET, however (i.e. left unmodified).
MD_EPN is set to 0x00000000 which does NOT point to
the faulting EA.
C) dcbi and dcbz behave like B) when accessing
through 3) [RO mapping]. Note that dcbf and
dcbst are treated like 'loads' and hence do
not raise a TLBError due to protection violation
(in accordance with the docs).
D) dcbz through a cache inhibited mapping (2)) does
not generate an alignment exception but
transparently clears the memory (either is allowed
according to the PPC specs).
Note: dcbt/dcbtst were included in all of the
tests but they never raise an exception which is
the correct behavior.
CONCLUSION:
- the only correct workaround for TLBError
is the one I suggested earlier: TLBError
handler has to inspect the faulting opcode
and fixup DAR and MD_EPN based on the GPR
values if the faulting instruction is any
of dcbf, dcbi, dcbst or dcbz.
Performance of this solution could be
improved (eliminate opcode-check in the
vast majority of the cases) by storing
a 'tag' value in DAR.
- the aforementioned workaround also could handle
TLBMiss - although performance is more of an issue
there.
Alternatives are
a) Jocke's workaround: copy MD_EPN -> DAR
(disadvantage: DAR truncated to page
boundary).
b) like a) but do this only if the faulting
instruction is any dcbxx (disadvantage:
performance loss).
c) merge significant bits of MD_EPN into
DAR. Thus, only cache instructions suffer
from an incorrect page offset in DAR.
At the same time, it's cheap.
Based on the analysis, I'd suggest to implement
the 'fixup' workaround for TLBError and alternative
c) for TLBMiss
-- Till
PS. due to unclear re-distribution terms of init860.S,
I refrain from appending it to this message. Send me
email for details about the test software.
** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/
From: Till Straumann <hidden> Date: 2003-03-27 01:25:27
My earlier message was not completely
correct. The TLBError exception raised
by any of the cache instructions
(dcbf, dcbst, dcbi, dcbz) does actually
leave the EPN bits in MD_EPN unmodified,
clears the reserved bits and sets CASID.
Here are the (corrected) results of some tests I ran
on my mpc860t (XPC860TZP50B5 / 3J21M).
I use mpc8bug on a FADS board to load motorola's
'init860.S' example. The example was slightly
modified to include (1-4 use differend EAs, of
course).
1) a 1:1 TLB to DRAM, R/W access (already present
in the vanilla code)
2) a TLB to map DRAM r/w, caching inhibited
3) a TLB to map DRAM r/o (behavior is the same
if for a 'real RO' mapping and for a RW
mapping with the 'CHANGE' bit cleared).
4) a TLB to map DRAM r/w but with the VALID
bit clear.
5) a 'main' routine which executes the
instructions under test through one
of the aforementioned TLB mappings.
Here's what I found (I generally load bogus
values into MD_EPN/DAR prior to executing any
of the cache instructions under test)
A) All of dcbf, dcbi, dcbst, dcbz (dcbt and dcbtst
should not and do not) raise a TLBMiss exception
with MD_EPN set correctly but failing to set DAR
(i.e. DAR is left unmodified) when trying to
access an address not in any of the TLBs.
B) All of dcbf, dcbi, dcbst, dcbz raise a TLBError
exception when accessing mapping 4) [as they should].
Unlike regular loads/stores (non-cache instructions),
NEITHER DAR not MD_EPN are set, however. DAR is left
unmodified MD_EPN has the CASID and reserved bits set/reset
but leaves the EPN bits unmodified.
C) dcbi and dcbz behave like B) when accessing
through 3) [RO mapping]. Note that dcbf and
dcbst are treated like 'loads' and hence do
not raise a TLBError due to protection violation
(in accordance with the docs).
D) dcbz through a cache inhibited mapping (2)) does
not generate an alignment exception but
transparently clears the memory (either is allowed
according to the PPC specs).
Note: dcbt/dcbtst were included in all of the
tests but they never raise an exception which is
the correct behavior.
CONCLUSION:
- the only correct workaround for TLBError
is the one I suggested earlier: TLBError
handler has to inspect the faulting opcode
and fixup DAR and MD_EPN based on the GPR
values if the faulting instruction is any
of dcbf, dcbi, dcbst or dcbz.
Performance of this solution could be
improved (eliminate opcode-check in the
vast majority of the cases) by storing
a 'tag' value in DAR.
- the aforementioned workaround also could handle
TLBMiss - although performance is more of an issue
there.
Alternatives are
a) Jocke's workaround: copy MD_EPN -> DAR
(disadvantage: DAR truncated to page
boundary).
b) like a) but do this only if the faulting
instruction is any dcbxx (disadvantage:
performance loss).
c) merge significant bits of MD_EPN into
DAR. Thus, only cache instructions suffer
from an incorrect page offset in DAR.
At the same time, it's cheap.
Based on the analysis, I'd suggest to implement
the 'fixup' workaround for TLBError and alternative
c) for TLBMiss
-- Till
PS. due to unclear re-distribution terms of init860.S,
I refrain from appending it to this message. Send me
email for details about the test software.
** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/
Great job Till!
I have forwarded this to Motorola. Primarly I want to know if never silicon
behaves the same way.
Here are the (corrected) results of some tests I ran
on my mpc860t (XPC860TZP50B5 / 3J21M).
I use mpc8bug on a FADS board to load motorola's
'init860.S' example. The example was slightly
modified to include (1-4 use differend EAs, of
course).
1) a 1:1 TLB to DRAM, R/W access (already present
in the vanilla code)
2) a TLB to map DRAM r/w, caching inhibited
3) a TLB to map DRAM r/o (behavior is the same
if for a 'real RO' mapping and for a RW
mapping with the 'CHANGE' bit cleared).
4) a TLB to map DRAM r/w but with the VALID
bit clear.
Level1 or Level2 valid bit? Does it matter which one?
5) a 'main' routine which executes the
instructions under test through one
of the aforementioned TLB mappings.
hmm, I wonder if the Guarded bit would change anything.
Here's what I found (I generally load bogus
values into MD_EPN/DAR prior to executing any
of the cache instructions under test)
A) All of dcbf, dcbi, dcbst, dcbz (dcbt and dcbtst
should not and do not) raise a TLBMiss exception
with MD_EPN set correctly but failing to set DAR
(i.e. DAR is left unmodified) when trying to
access an address not in any of the TLBs.
B) All of dcbf, dcbi, dcbst, dcbz raise a TLBError
exception when accessing mapping 4) [as they should].
Unlike regular loads/stores (non-cache instructions),
NEITHER DAR not MD_EPN are set, however. DAR is left
unmodified MD_EPN has the CASID and reserved bits set/reset
but leaves the EPN bits unmodified.
C) dcbi and dcbz behave like B) when accessing
through 3) [RO mapping]. Note that dcbf and
dcbst are treated like 'loads' and hence do
not raise a TLBError due to protection violation
(in accordance with the docs).
D) dcbz through a cache inhibited mapping (2)) does
not generate an alignment exception but
transparently clears the memory (either is allowed
according to the PPC specs).
Note: dcbt/dcbtst were included in all of the
tests but they never raise an exception which is
the correct behavior.
CONCLUSION:
- the only correct workaround for TLBError
is the one I suggested earlier: TLBError
handler has to inspect the faulting opcode
and fixup DAR and MD_EPN based on the GPR
values if the faulting instruction is any
of dcbf, dcbi, dcbst or dcbz.
Performance of this solution could be
improved (eliminate opcode-check in the
vast majority of the cases) by storing
a 'tag' value in DAR.
Yes, but I can't even figure out how to read the instruction. Address translation
is off so how do I figure out the real address?
hmm, what does mfspr r20, M_TWB return when MD_EPN contain old EPN? The old EPN in
MD_EPN or does M_TWB know real address that generated the fault?
- the aforementioned workaround also could handle
TLBMiss - although performance is more of an issue
there.
Alternatives are
a) Jocke's workaround: copy MD_EPN -> DAR
(disadvantage: DAR truncated to page
boundary).
b) like a) but do this only if the faulting
instruction is any dcbxx (disadvantage:
performance loss).
c) merge significant bits of MD_EPN into
DAR. Thus, only cache instructions suffer
from an incorrect page offset in DAR.
At the same time, it's cheap.
Based on the analysis, I'd suggest to implement
the 'fixup' workaround for TLBError and alternative
c) for TLBMiss
The TLB Miss workaround only needs to be in the slow path(DataAccess path) and
yes, it should be alternative c).
** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/
hmm, why would not dcbz execute properly? Surely the instruction is restarted
when the DTLB Miss/Error handler return?
Write a test to verify it. I know there were cases in the past when
it didn't work. You would notice it when you expected to have some
zero initialized spaces that weren't.
I think Till's superb analysis show that this is not a problem.
quoted
In a earlier mail you said the there was a case where the "normal" instructions
would not set MD_EPN in the DTLB Error handler and that it was documented.
What case is this and where can I find this documentation?
Well, it's more of an interpretation of the documentation and discussions
with Motorola engineers.
So this case is mere speculation and has never been proven?
[SNIP]
For TLB miss exceptions, the Mx_xxx hardware assist registers work
properly, as documented, to minimize the number of instructions needed
to determine what PTE to load. The TLB miss exception has exactly
one function, to as quickly as possible locate the PTE from the table
hierarchy and stuff the bits into the MMU. That's it and it should
really be implemented in five lines of code.
Five lines of code, that I would like to see :-) Seriously, why do the
DTLB handlers have two tables, one for kernel space and one for the rest?
Can the DataAccess path be removed in the DTLB Miss handler?
[SNIP]
quoted
Maybe Till Straumann's suggestion to set the DAR register to a known "bad" address
when leaving an exception. Then test for this bad address in the DTLB error handler
to decide what register to trust?
Not necessary.
By not "correctly" tracking usage and dirty pages, as we did in the past,
it caused us to never generate DTLB Error exceptions unless it was truly
an access to a non-recoverable bad address or if the page was swapped out.
I would just set 'used' on every instruction page and 'used + dirty' on
every data page allocation. A static embedded system would just converge on
this point anyway, but in more dynamic systems it cost more real memory.
hmm, don't you loose copy-on-write by doing so?
Would the suggested instruction decoding in the DLTB Error handler
be too ugly for your liking?
Since dcbi also causes DTLB Errors with undefined values in DAR and MD_EPN it
might be a good idea.
Jocke
** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/
The current implementation relies on TLB Miss always setting EPN
(and other hardware assist registers) correctly, and TLB Error
relies on DAR being set correctly. Everything works fine until
you get a TLB Error on a dcbz/dcbt that doesn't properly update DAR.
In this case EPN isn't set correctly either.
I just want to point out(again) that TLB Miss caused by most of the
dcxx(dcbf, dcbi, dcbst, dcbz) instructions that end up in the slow path(DataAccess)
don't work with the current impl. since DAR isn't set. The code fragment below
will fix that(from my earlier patch). This won't affect the fast path at all
since it all of it can be in the slow path.
+ /* Copy 20 msb from EPN to DAR since the dcxx instructions fails
+ * update the DAR when they cause a DTLB Miss.
+ */
+ mfspr r21, MD_EPN
+ rlwinm r21, r21, 0, 0, 19
+ mfspr r20, DAR
+ rlwinm r20, r20, 0, 20, 31
+ or r20, r20, r21
+ mtspr DAR, r20
It is hard to end up in slow path for kernel space addresses, but
it's not impossible I guess. Comments?
Jocke
** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/
CONCLUSION:
- the only correct workaround for TLBError
is the one I suggested earlier: TLBError
handler has to inspect the faulting opcode
and fixup DAR and MD_EPN based on the GPR
values if the faulting instruction is any
of dcbf, dcbi, dcbst or dcbz.
Performance of this solution could be
improved (eliminate opcode-check in the
vast majority of the cases) by storing
a 'tag' value in DAR.
Hi again
I have been hacking on dcxx address decoder. Since assembler isn't my cup of tea,
I used C mixed with asm statements. The resulting assembler isn't too bad either IMHO.
To load the instruction into r21 I used:
mfspr r20,SRR0
andis. r21, r20, 0x8000
beq 56f
tophys(r21, r20)
lwz r21,0(r21)
56:
This only works for kernel space addresses. I can't figure out how to get to user space as well.
I can live without user space anyway.
I am still thinking about the 'tag'. Since MD_EPN isn't set as well as DAR I thinking about
storing a tag in MD_EPN instead. It's less intrusive. Maybe it is enough to look at the
valid bit in MD_EPN?
What do you think so far?
Oh, this should go into the DTLB Error handler.
Jocke
======================= dcxx address decoder =========================
#define RA(inst) (((inst) & 0x001F0000) >> 16)
#define RA_MASK 0x001F0000
#define RB(inst) (((inst) & 0x0000F800) >> 11)
#define RB_MASK 0x0000F800
/* Compile with ppc_8xx-gcc -S -O2 -mregnames dcbz.c -fcall-used-r20 -fcall-used-r21
* to see the resulting assembler */
/* Assumes dcxx instruction in reg 21 when called */
decode_dcxx_and_sum(void)
{
register unsigned long r21 asm("r21"); /* make it live in reg 21 */
register unsigned long r20 asm("r20"); /* make it live in reg 20 */
asm("mfctr %0\n\t"
"stw %0, 16(0)":: "r" (r20)); /* save ctr reg on stack */
r20 = RB(r21) * 8; /* offset into jump table for reg RB */
asm("addi %0, %0, 100f@l":: "r" (r20)); /* add start of table */
asm("mtctr %0":: "r" (r20)); /* load ctr with jump address */
r20 = 0; /* sum starts at zero */
asm("bctr"::); /* jump into table */
/* Below is the jump table. */
asm("100:\n\t"
"add %0, %0, 0\n\t"
"b 99f\n\t"
"add %0, %0, 1\n\t"
"b 99f\n\t"
"add %0, %0, 2\n\t"
"b 99f\n\t"
"add %0, %0, 3\n\t"
"b 99f\n\t"
"add %0, %0, 4\n\t"
"b 99f\n\t"
"add %0, %0, 5\n\t"
"b 99f\n\t"
"add %0, %0, 6\n\t"
"b 99f\n\t"
"add %0, %0, 7\n\t"
"b 99f\n\t"
"add %0, %0, 8\n\t"
"b 99f\n\t"
"add %0, %0, 9\n\t"
"b 99f\n\t"
"add %0, %0, 10\n\t"
"b 99f\n\t"
"add %0, %0, 11\n\t"
"b 99f\n\t"
"add %0, %0, 12\n\t"
"b 99f\n\t"
"99:\n\t"
: : "r" (r20));
r21 = RA(r21) * 8; /* offset into jump table for reg RA */
if(r21){ /* if reg zero, don't add it */
asm("addi %0, %0, 100b@l":: "r" (r21)); /* add start of table */
asm("mtctr %0":: "r" (r21)); /* load ctr with jump address */
r21 &= ~RA_MASK; /* make sure we don't execute this mre than once */
asm("bctr":: "r" (r21)); /* jump into table */
}
asm("mtdar %0": : "r" (r20)); /* save sum to DAR */
asm("lwz %0, 16(0)\n\t"
"mtctr %0" :: "r" (r21)); /* restore ctr reg from stack */
}
This is the resulting assembler:
.file "dcbz.c"
gcc2_compiled.:
.section ".text"
.align 2
.globl decode_dcxx_and_sum
.type decode_dcxx_and_sum,@function
decode_dcxx_and_sum:
mfctr %r20
stw %r20, 16(0)
rlwinm %r20,%r21,24,24,28
addi %r20, %r20, 100f@l
mtctr %r20
li %r20,0
bctr
100:
add %r20, %r20, 0
b 99f
add %r20, %r20, 1
b 99f
add %r20, %r20, 2
b 99f
add %r20, %r20, 3
b 99f
add %r20, %r20, 4
b 99f
add %r20, %r20, 5
b 99f
add %r20, %r20, 6
b 99f
add %r20, %r20, 7
b 99f
add %r20, %r20, 8
b 99f
add %r20, %r20, 9
b 99f
add %r20, %r20, 10
b 99f
add %r20, %r20, 11
b 99f
add %r20, %r20, 12
b 99f
99:
rlwinm. %r21,%r21,19,24,28
bc 12,2,.L3
addi %r21, %r21, 100b@l
mtctr %r21
rlwinm %r21,%r21,0,16,10
bctr
.L3:
mtdar %r20
lwz %r21, 16(0)
mtctr %r21
blr
.Lfe1:
.size decode_dcxx_and_sum,.Lfe1-decode_dcxx_and_sum
.ident "GCC: (GNU) 2.95.3 20010315 (release/MontaVista)"
** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/
The current implementation relies on TLB Miss always setting EPN
(and other hardware assist registers) correctly, and TLB Error
relies on DAR being set correctly. Everything works fine until
you get a TLB Error on a dcbz/dcbt that doesn't properly update DAR.
In this case EPN isn't set correctly either.
I just want to point out(again) that TLB Miss caused by most of the
dcxx(dcbf, dcbi, dcbst, dcbz) instructions that end up in the slow path(DataAccess)
don't work with the current impl. since DAR isn't set. The code fragment below
will fix that(from my earlier patch). This won't affect the fast path at all
since it all of it can be in the slow path.
+ /* Copy 20 msb from EPN to DAR since the dcxx instructions fails
+ * update the DAR when they cause a DTLB Miss.
+ */
+ mfspr r21, MD_EPN
+ rlwinm r21, r21, 0, 0, 19
+ mfspr r20, DAR
+ rlwinm r20, r20, 0, 20, 31
+ or r20, r20, r21
+ mtspr DAR, r20
It is hard to end up in slow path for kernel space addresses, but
it's not impossible I guess. Comments?
hmm, no response from the maintainer(s). You don't agree?
Did another test. I tagged DAR with a bad address(0xdead0000) in
the TLB Miss handler(after the mtspr MD_RPN, r20) shortly before
rfi.
Now the kernel won't boot. It turns out it's consistent_alloc that is
doing a invalidate_dcache_range(). According to a comment, memory is allocated
from the vmalloc memory space. Doing a invalidate_dcache_range() on this memory
causes a DTLB error with the change bit cleared and since DAR isn't set by
dcbi, linux hangs.
Clearly, the DTLB exeception procedure still is buggy.
Jocke
** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/
From: Dan Malek <hidden> Date: 2003-04-04 15:43:38
Joakim Tjernlund wrote:
hmm, no response from the maintainer(s). You don't agree?
It's interesting to watch these hacks, but I can't justify
complicating a general purpose function with more bus cycles by
emulating a functional problem. By not using these instructions
we have a working system that costs just a few more cycles during
the memory copy/zero operations. If we had _working_ dcbz
instructions, it would be a gain to use them, but from a system
perspective it is going to cost more to "fix up" these than
the code that already exists.
As I said in the past, I'm sensitive to the code in the TLB exception
processing. So do something to remove code and streamline the process
and I'm really interested. Do something to add more code and it's
going to get placed pretty low in my pile of things to do.
Thanks.
-- Dan
** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/
hmm, no response from the maintainer(s). You don't agree?
It's interesting to watch these hacks, but I can't justify
complicating a general purpose function with more bus cycles by
emulating a functional problem. By not using these instructions
we have a working system that costs just a few more cycles during
the memory copy/zero operations. If we had _working_ dcbz
instructions, it would be a gain to use them, but from a system
perspective it is going to cost more to "fix up" these than
the code that already exists.
Look a little harder. I am not talking just about dcbz, dcbi has
the same problem. As it stands today, there are real bugs w.r.t dcbi.
consistent_alloc() causes DTLB errors with a incorrect address in DAR.
Just force DAR to an invalid address in the DTLB Miss handler before
you leave it and see if you kernel boots.
As I said in the past, I'm sensitive to the code in the TLB exception
processing. So do something to remove code and streamline the process
and I'm really interested. Do something to add more code and it's
going to get placed pretty low in my pile of things to do.
Yes I know, but you are not reading what am writing. For instance,
the change below does not add to the fast path of the DTLB Miss handler and
it fixes a real problem:
I just want to point out(again) that TLB Miss caused by most of the
dcxx(dcbf, dcbi, dcbst, dcbz) instructions that end up in the slow path(DataAccess)
don't work with the current impl. since DAR isn't set. The code fragment below
will fix that(from my earlier patch). This won't affect the fast path at all
since it all of it can be in the slow path.
+ /* Copy 20 msb from EPN to DAR since the dcxx instructions fails
+ * update the DAR when they cause a DTLB Miss.
+ */
+ mfspr r21, MD_EPN
+ rlwinm r21, r21, 0, 0, 19
+ mfspr r20, DAR
+ rlwinm r20, r20, 0, 20, 31
+ or r20, r20, r21
+ mtspr DAR, r20
Go back and think dcbi instead of dcbz.
Jocke
** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/
CONCLUSION:
- the only correct workaround for TLBError
is the one I suggested earlier: TLBError
handler has to inspect the faulting opcode
and fixup DAR and MD_EPN based on the GPR
values if the faulting instruction is any
of dcbf, dcbi, dcbst or dcbz.
Performance of this solution could be
improved (eliminate opcode-check in the
vast majority of the cases) by storing
a 'tag' value in DAR.
Hi again
I have been hacking on dcxx address decoder. Since assembler isn't my cup of tea,
I used C mixed with asm statements. The resulting assembler isn't too bad either IMHO.
To load the instruction into r21 I used:
mfspr r20,SRR0
andis. r21, r20, 0x8000
beq 56f
tophys(r21, r20)
lwz r21,0(r21)
56:
This only works for kernel space addresses. I can't figure out how to get to user space as well.
I can live without user space anyway.
I am still thinking about the 'tag'. Since MD_EPN isn't set as well as DAR I thinking about
storing a tag in MD_EPN instead. It's less intrusive. Maybe it is enough to look at the
valid bit in MD_EPN?
What do you think so far?
Oh, this should go into the DTLB Error handler.
Me again :-)
I have completed and tested my workaround for the dcbx instructions. The workaround
handles ALL dcbx instructions, ANY register combination and works both on
kernel space and user space addresses.
During my testing I noticed that memory allocated with
consistent_alloc() or kmalloc() causes TLB Errors while vmalloc does not.
If this is true(confirmation wanted) it means that the current impl. is fragile.
Both dcbi and dcbz does not update DAR when a TLB error happens, instead the
the previous setting of DAR is used.
I also did some benchmarking using copy_page(dcbz enabled) and memcpy to
memory allocated with kmalloc and/or vmalloc. copy_page is about 30% faster
than memcpy even with the workaround applied.
There is one concern left. I tag DAR with a "bad address" just before
an exception is finished. In the TLB Error handler, check if DAR contains
the "bad address" and if it does then the workaround is executed.
I need find all exceptions where DAR is modified. Currently I tag DAR in
STD_EXCEPTON(), DataAccess, Alignment, DataStore and DataTLBError. Have I
missed any exception?
I also need to find a good value for the "bad address". Currently I use
0xdead0000 and that's probably not the best value.
Tagging with this value is a two instruction operation:
lis r20, 0xdead
mtspr DAR, r20
The test in the TLB Error handler look like this:
mfspr r21, DAR
lis r20, 0xdead
cmpw cr0, r20, r21
beq- <workaround address>
I can not see any reason NOT to add this to the BK tree(after some minor modifications
mentioned above and a little cleanup). It fixes a real problem
with dcbi and as a bonus you can use dcbz as well since it has the same problem
that dcbi has and the fix is generic for all dcbx instructions. Dan's argument,
"It's interesting to watch these hacks, but I can't justify
complicating a general purpose function with more bus cycles by
emulating a functional problem. By not using these instructions
we have a working system that costs just a few more cycles during
the memory copy/zero operations. If we had _working_ dcbz
instructions, it would be a gain to use them, but from a system
perspective it is going to cost more to "fix up" these than
the code that already exists", is not valid. This is not only about
optimization, but also about correctness of existing use of dcbi.
Patch against 2.4.20 devel available on request until I have cleaned it up a little.
Jocke
** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/
Joakim Tjernlund wrote:
Why not tag MD_EPN with an unused CASID?
That won't work, will it? Your test report said that the CASID
bits were set/reset by dcbi and dcbz? Did I missunderstand something?
Also, the tagging procedure will require more than 2 instructions, I think.
I tried to use the VALID bit in MD_EPN, but that does not work.
One thing I noticed is that I can remove M_CASID and M_EVALID copy in the DTLB Error
handler without any visible problems. Why are they needed?
Jocke
Hi again
I am wonder if it is possible to reduce the number of instructions in:
mfspr r20, DAR
lis r21, 0xdead
cmpw cr0, r20, r21
beq- 100f
Jocke
** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/
Patch against 2.4.20 devel available on request until I have cleaned it up a little.
Jocke
Here is the patch, against 2.4.20 devel but I think it will apply
against BK HEAD. Comments welcome :-)
Jocke
Index: arch/ppc/kernel/head_8xx.S
===================================================================
RCS file: /home/cvsadmin/cvsroot/kernel/linuxppc/arch/ppc/kernel/head_8xx.S,v
retrieving revision 1.1
diff -u -r1.1 head_8xx.S
@@ -624,6 +665,146 @@STD_EXCEPTION(0x1f00,Trap_1f,UnknownException).=0x2000+/*ThisistheworkaroundproceduretocalculatethedataEAforabuggydcbxinstruction+*bydecodingtheregistersusedbythedcbxinstructionandaddingthem.+*DARissettothecalculatedaddressandr20alsoholdstheEAonexit.+*/+139:/*fetchinstructionfromuserspacememory*/+DO_8xx_CPU6(0x3780,r3)+mtsprMD_EPN,r20/*HavetouseMD_EPNforwalk,MI_EPNcan't */+mfsprr21,M_TWB/*Getlevel1tableentryaddress*/+lwzr21,0(r21)/*Getthelevel1entry*/+tophys (r21,r21)+DO_8xx_CPU6(0x3b80,r3)+mtsprMD_TWC,r21/*Loadptetablebaseaddress*/+mfsprr21,MD_TWC/*....andgetthepteaddress*/+lwzr21,0(r21)/*Getthepte*/+/*concatphysicalpageaddressandpageoffset*/+rlwinmr21,r21,0,0,19+rlwinmr20,r20,0,20,31+orr21,r20,r21+b140f+100:/*Entrypointfordcbxworkaround.*/+/*fetchinstructionfrommemory.*/+mfsprr20,SRR0+andis.r21,r20,0x8000+beq-139b/*Branchifuserspaceaddress*/+tophys (r21,r20)+140:lwzr21,0(r21)+#if DEBUG_DCBX_INSTRUCTIONS+/*Checkifitreallyisadcbxinstruction*/+rlwinmr20,r21,0,21,30+cmpwicr0,r20,940/*Isdcbi?*/+bc12,2,142f+cmpwicr0,r20,2028/*Isdcbz?*/+bc12,2,142f+cmpwicr0,r20,492/*Isdcbtst?*/+bc12,2,142f+cmpwicr0,r20,556/*Isdcbt?*/+bc12,2,142f+cmpwicr0,r20,108/*Isdcbst?*/+bc12,2,142f+cmpwicr0,r20,172/*Isdcbf?*/+bc12,2,142f++141:b141b/*Stophereifnodcbxinstruction*/+142:/*continue,itwasadcbxinstruction.*/+#endif++#ifdef CONFIG_8xx_CPU6+lwzr3,8(r0)/*restorer3frommemory*/+#endif+mfctrr20+mtdarr20/*savectrreginDAR*/+rlwinmr20,r21,24,24,28/*offsetintojumptableforregRB*/+addir20,r20,150f@l/*addstartoftable*/+mtctrr20/*loadctrwithjumpaddress*/+xorr20,r20,r20/*sumstartsatzero*/+bctr/*jumpintotable*/+150:+addr20,r20,r0+b151f+addr20,r20,r1+b151f+addr20,r20,r2+b151f+addr20,r20,r3+b151f+addr20,r20,r4+b151f+addr20,r20,r5+b151f+addr20,r20,r6+b151f+addr20,r20,r7+b151f+addr20,r20,r8+b151f+addr20,r20,r9+b151f+addr20,r20,r10+b151f+addr20,r20,r11+b151f+addr20,r20,r12+b151f+addr20,r20,r13+b151f+addr20,r20,r14+b151f+addr20,r20,r15+b151f+addr20,r20,r16+b151f+addr20,r20,r17+b151f+addr20,r20,r18+b151f+addr20,r20,r19+b151f+mtctrr21/*reg20needsspecialhandling*/+b154f+mtctrr21/*reg21needsspecialhandling*/+b153f+addr20,r20,r22+b151f+addr20,r20,r23+b151f+addr20,r20,r24+b151f+addr20,r20,r25+b151f+addr20,r20,r25+b151f+addr20,r20,r27+b151f+addr20,r20,r28+b151f+addr20,r20,r29+b151f+addr20,r20,r30+b151f+addr20,r20,r31+151:+rlwinm.r21,r21,19,24,28/*offsetintojumptableforregRA*/+bc12,2,152f/*ifregRAiszero,don't add it */+addir21,r21,150b@l/*addstartoftable*/+mtctrr21/*loadctrwithjumpaddress*/+rlwinmr21,r21,0,16,10/*makesurewedon't execute this more than once */+bctr/*jumpintotable*/+152:+mfdarr21+mtctrr21/*restorectrregfromDAR*/+mtdarr20/*savefaultEAtoDAR*/+b101b/*GobacktonormalTLBhandling*/++/*specialhandlingforr20,r21sincethesearemodifiedalready*/+153:lwzr21,4(r0)/*loadr21frommemory*/+b155f+154:mfsprr21,M_TW/*loadr20fromM_TW*/+155:addr20,r20,r21/*addit*/+mfctrr21/*restorer21*/+b151b/**Thiscodefinishessavingtheregisterstotheexceptionframe
On Tue, Apr 08, 2003 at 07:25:37PM +0200, Joakim Tjernlund wrote:
quoted
Patch against 2.4.20 devel available on request until I have cleaned it up a little.
Jocke
Here is the patch, against 2.4.20 devel but I think it will apply
against BK HEAD. Comments welcome :-)
Can you let me know if this applies cleanly to the current linux-2.4
tree (Or, 2.4.21-pre7 tarball)? If not, can you generate a patch
against this? Thanks.
--
Tom Rini
http://gate.crashing.org/~trini/
** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/
On Tue, Apr 08, 2003 at 07:25:37PM +0200, Joakim Tjernlund wrote:
quoted
quoted
Patch against 2.4.20 devel available on request until I have cleaned it up a little.
Jocke
Here is the patch, against 2.4.20 devel but I think it will apply
against BK HEAD. Comments welcome :-)
Can you let me know if this applies cleanly to the current linux-2.4
tree (Or, 2.4.21-pre7 tarball)? If not, can you generate a patch
against this? Thanks.
--
Tom Rini
pulled the latest linuxppc_2_4, patched it and inspected the result.
Current patch will apply with some offset and looks good.
I cannot run standard 2.4 without major patching, so I can't be sure
that it works.
Anyhow here is a version against linuxppc_2_4:
===== arch/ppc/kernel/head_8xx.S 1.33 vs edited =====
@@ -633,6 +674,146 @@STD_EXCEPTION(0x1f00,Trap_1f,UnknownException).=0x2000+/*ThisistheworkaroundproceduretocalculatethedataEAforabuggydcbxinstruction+*bydecodingtheregistersusedbythedcbxinstructionandaddingthem.+*DARissettothecalculatedaddressandr20alsoholdstheEAonexit.+*/+139:/*fetchinstructionfromuserspacememory*/+DO_8xx_CPU6(0x3780,r3)+mtsprMD_EPN,r20/*HavetouseMD_EPNforwalk,MI_EPNcan't */+mfsprr21,M_TWB/*Getlevel1tableentryaddress*/+lwzr21,0(r21)/*Getthelevel1entry*/+tophys (r21,r21)+DO_8xx_CPU6(0x3b80,r3)+mtsprMD_TWC,r21/*Loadptetablebaseaddress*/+mfsprr21,MD_TWC/*....andgetthepteaddress*/+lwzr21,0(r21)/*Getthepte*/+/*concatphysicalpageaddressandpageoffset*/+rlwinmr21,r21,0,0,19+rlwinmr20,r20,0,20,31+orr21,r20,r21+b140f+100:/*Entrypointfordcbxworkaround.*/+/*fetchinstructionfrommemory.*/+mfsprr20,SRR0+andis.r21,r20,0x8000+beq-139b/*Branchifuserspaceaddress*/+tophys (r21,r20)+140:lwzr21,0(r21)+#if DEBUG_DCBX_INSTRUCTIONS+/*Checkifitreallyisadcbxinstruction*/+rlwinmr20,r21,0,21,30+cmpwicr0,r20,940/*Isdcbi?*/+bc12,2,142f+cmpwicr0,r20,2028/*Isdcbz?*/+bc12,2,142f+cmpwicr0,r20,492/*Isdcbtst?*/+bc12,2,142f+cmpwicr0,r20,556/*Isdcbt?*/+bc12,2,142f+cmpwicr0,r20,108/*Isdcbst?*/+bc12,2,142f+cmpwicr0,r20,172/*Isdcbf?*/+bc12,2,142f++141:b141b/*Stophereifnodcbxinstruction*/+142:/*continue,itwasadcbxinstruction.*/+#endif++#ifdef CONFIG_8xx_CPU6+lwzr3,8(r0)/*restorer3frommemory*/+#endif+mfctrr20+mtdarr20/*savectrreginDAR*/+rlwinmr20,r21,24,24,28/*offsetintojumptableforregRB*/+addir20,r20,150f@l/*addstartoftable*/+mtctrr20/*loadctrwithjumpaddress*/+xorr20,r20,r20/*sumstartsatzero*/+bctr/*jumpintotable*/+150:+addr20,r20,r0+b151f+addr20,r20,r1+b151f+addr20,r20,r2+b151f+addr20,r20,r3+b151f+addr20,r20,r4+b151f+addr20,r20,r5+b151f+addr20,r20,r6+b151f+addr20,r20,r7+b151f+addr20,r20,r8+b151f+addr20,r20,r9+b151f+addr20,r20,r10+b151f+addr20,r20,r11+b151f+addr20,r20,r12+b151f+addr20,r20,r13+b151f+addr20,r20,r14+b151f+addr20,r20,r15+b151f+addr20,r20,r16+b151f+addr20,r20,r17+b151f+addr20,r20,r18+b151f+addr20,r20,r19+b151f+mtctrr21/*reg20needsspecialhandling*/+b154f+mtctrr21/*reg21needsspecialhandling*/+b153f+addr20,r20,r22+b151f+addr20,r20,r23+b151f+addr20,r20,r24+b151f+addr20,r20,r25+b151f+addr20,r20,r25+b151f+addr20,r20,r27+b151f+addr20,r20,r28+b151f+addr20,r20,r29+b151f+addr20,r20,r30+b151f+addr20,r20,r31+151:+rlwinm.r21,r21,19,24,28/*offsetintojumptableforregRA*/+bc12,2,152f/*ifregRAiszero,don't add it */+addir21,r21,150b@l/*addstartoftable*/+mtctrr21/*loadctrwithjumpaddress*/+rlwinmr21,r21,0,16,10/*makesurewedon't execute this more than once */+bctr/*jumpintotable*/+152:+mfdarr21+mtctrr21/*restorectrregfromDAR*/+mtdarr20/*savefaultEAtoDAR*/+b101b/*GobacktonormalTLBhandling*/++/*specialhandlingforr20,r21sincethesearemodifiedalready*/+153:lwzr21,4(r0)/*loadr21frommemory*/+b155f+154:mfsprr21,M_TW/*loadr20fromM_TW*/+155:addr20,r20,r21/*addit*/+mfctrr21/*restorer21*/+b151b/**Thiscodefinishessavingtheregisterstotheexceptionframe
As I said in the past, I'm sensitive to the code in the TLB exception
processing. So do something to remove code and streamline the process
and I'm really interested. Do something to add more code and it's
going to get placed pretty low in my pile of things to do.
Thanks.
If you make swapper_pg_dir 64KB aligned, you can removed
ori r21, r21, swapper_pg_dir@l instructions in
DataStoreTLBMiss, DataTLBError and InstructionTLBMiss
I did it with the following, ugly, hack in vmliunx.lds:
swapper_pg_dir = ALIGN(0x10000);
. = . + 4096;
I guess there is a better way to this.
Jocke
** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/
Me again :-)
I have completed and tested my workaround for the dcbx instructions. The workaround
handles ALL dcbx instructions, ANY register combination and works both on
kernel space and user space addresses.
I also did some benchmarking using copy_page(dcbz enabled) and memcpy to
memory allocated with kmalloc and/or vmalloc. copy_page is about 30% faster
than memcpy even with the workaround applied.
Here I go again :)
I have been running this patch on some 20-30 custom MPC860/862 boards in our test lab
since I posted this message and it is stable.
I made some changes since then:
- Made it configurable, #define CONFIG_8xx_DCBxFIXED to enable it.
- Tagging in the fast path in the DLBMiss handler is just one(1) instruction.
- Test and branch if TAG present is two instructions in the DTLB Error handler.
- Enabled the use of the dcbz instruction in copy_tofrom_user(), cacheable_memzero(),
cacheable_memcpy(), clear_page() and copy_page()
Feedback most welcome!
Patch against linuxppc_2_4_devel follows.
Jocke
@@ -457,6 +481,13 @@#endifmtsprMD_RPN,r20/*UpdateTLBentry*/+#ifdef CONFIG_8xx_DCBxFIXED+#if TAG_VAL == 0x00f0 /* Save 1 instr. by reusing the val loaded in r21 above */+mtsprDAR,r21+#else+TAG_DAR_R20+#endif+#endifmfsprr20,M_TW/*Restoreregisters*/lwzr21,0(r0)mtcrr21
@@ -591,6 +641,13 @@#endifmtsprMD_RPN,r20/*UpdateTLBentry*/+#ifdef CONFIG_8xx_DCBxFIXED+#if TAG_VAL == 0x00f0 /* Save 1 instr. by reusing the val loaded in r21 above */+mtsprDAR,r21+#else+TAG_DAR_R20+#endif+#endifmfsprr20,M_TW/*Restoreregisters*/lwzr21,0(r0)mtcrr21
@@ -628,6 +685,149 @@.=0x2000+#ifdef CONFIG_8xx_DCBxFIXED+/*ThisistheworkaroundproceduretocalculatethedataEAforabuggydcbxinstruction+*bydecodingtheregistersusedbythedcbxinstructionandaddingthem.+*DARissettothecalculatedaddressandr20alsoholdstheEAonexit.+*/+139:/*fetchinstructionfromuserspacememory*/+DO_8xx_CPU6(0x3780,r3)+mtsprMD_EPN,r20+mfsprr21,M_TWB/*Getlevel1tableentryaddress*/+lwzr21,0(r21)/*Getthelevel1entry*/+tophys (r21,r21)+DO_8xx_CPU6(0x3b80,r3)+mtsprMD_TWC,r21/*Loadptetablebaseaddress*/+mfsprr21,MD_TWC/*....andgetthepteaddress*/+lwzr21,0(r21)/*Getthepte*/+/*concatphysicalpageaddress(r21)andpageoffset(r20)*/+rlwimir21,r20,0,20,31+b140f+100:/*Entrypointfordcbxworkaround.*/+/*fetchinstructionfrommemory.*/+mfsprr20,SRR0+andis.r21,r20,0x8000+beq-139b/*Branchifuserspaceaddress*/+tophys (r21,r20)+140:lwzr21,0(r21)++/*Checkifitreallyisadcbxinstruction*/+rlwinmr20,r21,0,21,30+cmpwicr0,r20,2028/*Isdcbz?*/+beq+142f+cmpwicr0,r20,940/*Isdcbi?*/+beq+142f+cmpwicr0,r20,556/*Isdcbt?*/+beq+142f+cmpwicr0,r20,172/*Isdcbf?*/+beq+142f+#ifdef DEBUG_DCBX_INSTRUCTIONS+cmpwicr0,r20,108/*Isdcbst?ShouldnevercauseaDTLBMiss/Error*/+beq+142f+cmpwicr0,r20,492/*Isdcbtst?ShouldnevercauseaDTLBMiss/Error*/+beq+142f++141:b141b/*Stophereifnodcbxinstruction*/+#endif+mfsprr20,DAR/*r20mustholdDARatexit*/+b101b/*Noneoftheabove,gobacktonormalTLBprocessing*/+142:/*continue,itwasadcbxinstruction.*/++#ifdef CONFIG_8xx_CPU6+lwzr3,8(r0)/*restorer3frommemory*/+#endif+mfctrr20+mtdarr20/*savectrreginDAR*/+rlwinmr20,r21,24,24,28/*offsetintojumptableforregRB*/+addir20,r20,150f@l/*addstartoftable*/+mtctrr20/*loadctrwithjumpaddress*/+xorr20,r20,r20/*sumstartsatzero*/+bctr/*jumpintotable*/+150:+addr20,r20,r0+b151f+addr20,r20,r1+b151f+addr20,r20,r2+b151f+addr20,r20,r3+b151f+addr20,r20,r4+b151f+addr20,r20,r5+b151f+addr20,r20,r6+b151f+addr20,r20,r7+b151f+addr20,r20,r8+b151f+addr20,r20,r9+b151f+addr20,r20,r10+b151f+addr20,r20,r11+b151f+addr20,r20,r12+b151f+addr20,r20,r13+b151f+addr20,r20,r14+b151f+addr20,r20,r15+b151f+addr20,r20,r16+b151f+addr20,r20,r17+b151f+addr20,r20,r18+b151f+addr20,r20,r19+b151f+mtctrr21/*reg20needsspecialhandling*/+b154f+mtctrr21/*reg21needsspecialhandling*/+b153f+addr20,r20,r22+b151f+addr20,r20,r23+b151f+addr20,r20,r24+b151f+addr20,r20,r25+b151f+addr20,r20,r25+b151f+addr20,r20,r27+b151f+addr20,r20,r28+b151f+addr20,r20,r29+b151f+addr20,r20,r30+b151f+addr20,r20,r31+151:+rlwinm.r21,r21,19,24,28/*offsetintojumptableforregRA*/+beq152f/*ifregRAiszero,don't add it */+addir21,r21,150b@l/*addstartoftable*/+mtctrr21/*loadctrwithjumpaddress*/+rlwinmr21,r21,0,16,10/*makesurewedon't execute this more than once */+bctr/*jumpintotable*/+152:+mfdarr21+mtctrr21/*restorectrregfromDAR*/+mtdarr20/*savefaultEAtoDAR*/+b101b/*GobacktonormalTLBhandling*/++/*specialhandlingforr20,r21sincethesearemodifiedalready*/+153:lwzr21,4(r0)/*loadr21frommemory*/+b155f+154:mfsprr21,M_TW/*loadr20fromM_TW*/+155:addr20,r20,r21/*addit*/+mfctrr21/*restorer21*/+b151b+#endif/**Thiscodefinishessavingtheregisterstotheexceptionframe*andjumpstotheappropriatehandlerfortheexception,turning---a/arch/ppc/lib/string.STueApr2900:45:352003+++b/arch/ppc/lib/string.SFriMay914:17:072003