405 TLB miss reduction

Hello,

we are suffering from TLB misses on a 405GP processor, eating up to
10% of the CPU power when running our (rather big) application. We
can regain a few percent by using the kernel option CONFIG_PIN_TLB
but we are thinking about further kernel modifications to reduce
TLB misses. What comes into my mind is:

 - using a kernel PAGE_SIZE of 8KB (instead of 4KB).
 - using large-page TLB entries.

Has anybody already investigated the effort or benefit of such
changes or knows about other (simple) measures (apart from
replacing the hardware)?

TIA.

Wolfgang.


** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/

Wolfgang Grandegger wrote:

....We
can regain a few percent by using the kernel option CONFIG_PIN_TLB
but we are thinking about further kernel modifications to reduce
TLB misses. What comes into my mind is:

If you have a large application I doubt any kernel modification
will gain much.  It's the application causing the huge amounts
of tlb misses, you probably need to evaluate changes that will
reduce that.

It's always easy to pick on the kernel and make some changes
becaue it is a very static and well behaved application.  It
seems your biggest performance increase would come from the
analysis of the application and some redesign to improve its
use of system resources.

Thanks.


	-- Dan


** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/

On 12/10/2003 06:08 PM Dan Malek wrote:

Wolfgang Grandegger wrote:

quoted

....We
can regain a few percent by using the kernel option CONFIG_PIN_TLB
but we are thinking about further kernel modifications to reduce
TLB misses. What comes into my mind is:

If you have a large application I doubt any kernel modification
will gain much.  It's the application causing the huge amounts
of tlb misses, you probably need to evaluate changes that will
reduce that.

It's always easy to pick on the kernel and make some changes
becaue it is a very static and well behaved application.  It
seems your biggest performance increase would come from the
analysis of the application and some redesign to improve its
use of system resources.

We have been surprised, that CONFIG_PIN_TLB was able to reduce the page
miss rate already by approx. 40%. We are also working on the
optimization/tuning of our application and likely there we can gain more
than by further squeezing the TBL management of the Linux kernel, I agree.

Thanks.

Wolfgang.


** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/

Wolfgang Grandegger wrote:

| Has anybody already investigated the effort or benefit of such
| changes or knows about other (simple) measures (apart from
| replacing the hardware)?

I have been thinking about doing something slightly more intelligent
with tLB miss handling and selecting the next entry to write.

At the moment I am doing stuff like setting up PTXdist (which does
indeed rule by the way once you get the right configuration) however I
could do with something for the holiday period so please keep me updated
~ and I will try to fit bits in when I get time with a board to do it.

There is an EPPC405 which I am hoping to borrow for a while because it
is not particularly being used but I will see what happens about it.

Cheers,

Jon.

P.S. I am going to be at FOSDEM and welcome the idea of meeting other
ppc 405 or Virtex II Pro or indeed anyone from this list.


** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/

Wolfgang Grandegger wrote:

| We have been surprised, that CONFIG_PIN_TLB was able to reduce the page
| miss rate already by approx. 40%. We are also working on the
| optimization/tuning of our application and likely there we can gain more
| than by further squeezing the TBL management of the Linux kernel, I agree.

I noticed a possible improvement in performance however have not yet
done a set of tests to back this up. I saw the huge TLB stuff in
2.6.0-test11 and nearly fell off my chair - can someone here provide
some reference to read on this so I can be more useful talking about it?

Jon.


** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/

On 12/11/2003 05:48 PM Jon Masters wrote:

Wolfgang Grandegger wrote:

| We have been surprised, that CONFIG_PIN_TLB was able to reduce the page
| miss rate already by approx. 40%. We are also working on the
| optimization/tuning of our application and likely there we can gain more
| than by further squeezing the TBL management of the Linux kernel, I agree.

I noticed a possible improvement in performance however have not yet
done a set of tests to back this up. I saw the huge TLB stuff in
2.6.0-test11 and nearly fell off my chair - can someone here provide
some reference to read on this so I can be more useful talking about it?

I found "Documentation/vm/hugetlbpage.txt" in linuxppc-2.5 from BK.

Wolfgang.


** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/

Wolfgang Grandegger wrote:

| I found "Documentation/vm/hugetlbpage.txt" in linuxppc-2.5 from BK.

Yeah I have that too but want to know whether there is a resource I am
missing where this stuff is getting discussed and sorted.

Jon.


** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/