From: Graham Stoney <hidden> Date: 2000-08-16 07:26:24
Casting our minds back to July, Dan Malek wrote:
OK, I think I am bailing out here. For some reason, if I remove
the 'dcbz' instructions on the MPC8xx processor the world is just
a better place. I don't know why, maybe because of some of the
TLB mapping, but I can't find a reason.
What was the eventual outcome of this? I've been doing some 2.2.13
kernel profiling on the 860, and __copy_tofrom_user is coming up as a
hotspot.
I tried dropping in the new improved version from
linux-2.4.0-test7-pre4, and none of the 8xx mods are in there: it'l only
work for 32 byte cache lines.
I hacked it around and found the same as you: it won't work with the
dcbz in there, and of course it doesn't run any faster than the old
version without it. It's certainly getting more complex in there, and I
see your point about whether the extra code will actually make it run
any faster, especially on 8xx CPUs with small I-caches. I'd be keen to
test whatever you've come up with to see if it's actually better than
the old 2.2 code on 8xx CPUs.
It sounds like a few people have at least had a shot at adding support
for other than 32 byte cache lines, but none have propagated into the
official kernels; how does that happen anyway?
Thanks,
Graham
** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/
OK, I think I am bailing out here. For some reason, if I remove
the 'dcbz' instructions on the MPC8xx processor the world is just
a better place. I don't know why, maybe because of some of the
TLB mapping, but I can't find a reason.
What was the eventual outcome of this? I've been doing some 2.2.13
kernel profiling on the 860, and __copy_tofrom_user is coming up as a
hotspot.
I tried dropping in the new improved version from
linux-2.4.0-test7-pre4, and none of the 8xx mods are in there: it'l only
work for 32 byte cache lines.
Graham,
That's not right. Which version of linux-2.4.0-x do you have? I'm sure Paul's,
Cort's and and I know for sure mine work for more than 32 byte cache lines
because 2 of my boxes have 128 byte cache lines and there's support in there for
it. (Paul wrote it... awesome stuff)
I hacked it around and found the same as you: it won't work with the
dcbz in there, and of course it doesn't run any faster than the old
version without it. It's certainly getting more complex in there, and I
see your point about whether the extra code will actually make it run
any faster, especially on 8xx CPUs with small I-caches. I'd be keen to
test whatever you've come up with to see if it's actually better than
the old 2.2 code on 8xx CPUs.
This isn't the only spot too by the way ... glibc has the some problem.
It sounds like a few people have at least had a shot at adding support
for other than 32 byte cache lines, but none have propagated into the
official kernels; how does that happen anyway?
By official are you meaning Linus' ?
Regards,
Tom
--
Tom Gall - PowerPC Linux Team "Where's the ka-boom? There was
Linux Technology Center supposed to be an earth
(w) tom_gall@vnet.ibm.com shattering ka-boom!"
(w) 507-253-4558 -- Marvin Martian
(h) tgall@uswest.net
http://oss.software.ibm.com/developerworks/opensource/linux
** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/
From: Graham Stoney <hidden> Date: 2000-08-17 00:50:57
Hi Tom,
Thanks for the response...
Graham Stoney wrote:
I tried dropping in the new improved version from linux-2.4.0-test7-pre4,
and none of the 8xx mods are in there: it'l only work for 32 byte cache
lines.
tom_gall@vnet.ibm.com writes:
That's not right. Which version of linux-2.4.0-x do you have? I'm sure Paul's,
Cort's and and I know for sure mine work for more than 32 byte cache lines
because 2 of my boxes have 128 byte cache lines and there's support in there
for it. (Paul wrote it... awesome stuff)
This isn't the only spot too by the way ... glibc has the some problem.
Yes, in sysdeps/powerpc/memset.S. This is easily enough solved by nuking it,
and allowing glibc to use the generic C version.
quoted
It sounds like a few people have at least had a shot at adding support
for other than 32 byte cache lines, but none have propagated into the
official kernels; how does that happen anyway?
By official are you meaning Linus' ?
Yes. Do I need to run BitKeeper to keep up with the latest ppc changes?
They don't seem to be making it in to Linus's kernel...
Thanks,
Graham
--
Graham Stoney
Principal Hardware/Software Engineer
Canon Information Systems Research Australia
Ph: +61 2 9805 2909 Fax: +61 2 9805 2929
** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/
From: Dan Malek <hidden> Date: 2000-08-17 19:28:23
Graham Stoney wrote:
..... I've been doing some 2.2.13
kernel profiling on the 860, and __copy_tofrom_user is coming up as a
hotspot.
In what kind of test?
I tried dropping in the new improved version from
linux-2.4.0-test7-pre4, and none of the 8xx mods are in there: it'l only
work for 32 byte cache lines.
Hmmm....I check it into the FSM BK tree a long time ago.
-- Dan
** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/
From: Graham Stoney <hidden> Date: 2000-08-18 02:10:57
Graham Stoney wrote:
..... I've been doing some 2.2.13
kernel profiling on the 860, and __copy_tofrom_user is coming up as a
hotspot.
Dan Malek writes:
In what kind of test?
Reading data from a TCP connected socket at full speed over the FEC and just
dropping it, to measure the maximum theoretical TCP throughput. I added
/proc/profile support to the ppc kernel (it was missing) to see where the
time was going; the patch to do this is available at:
http://members.xoom.com/greyhams/linux/patches/2.2/profile.patch
Here are the top ten functions output from readprofile:
3490 total 0.0050
972 csum_partial_copy_generic 6.5676
754 __copy_to_user 2.4481
129 do_lost_interrupts 2.0156
113 kfree 0.1519
94 tcp_recvmsg 0.0625
85 kmalloc 0.1250
74 alloc_skb 0.2534
73 fec_enet_rx 0.1393
64 tcp_rcv_established 0.0370
62 tcp_v4_rcv 0.0686
The count values on the left are in jiffies, and those on the right are in
jiffies per instruction (I think real time would be more useful!). I split
__copy_tofrom_user so it would appear seperately in the profile as
__copy_to_user and __copy_from_user.
It shows that almost half the time in TCP reception is consumed in:
1. checksuming & copying the data into the socket buffer in
csum_partial_copy_generic (called from fec_enet_rx->eth_copy_and_sum->
csum_partial_copy->csum_partial_copy_generic),
and
2. copying the result out to the user (called from sys_read->sock_read->
sock_recvmsg->tcp_recvmsg->memcpy_toiovec->copy_to_user->__copy_to_user)
I've got this crazy idea that the FEC could DMA directly to the skb to
eliminate the first copy. Pity the FEC can't calculate IP checksums for us,
but eliminating the copy should make it go faster even though tcp_v4_rcv
would then need to calculate the checksum in software. Would you like to tell
me why this won't work before I spend hours trying to implement it? :-)
quoted
I tried dropping in the new improved version from
linux-2.4.0-test7-pre4, and none of the 8xx mods are in there: it'l only
work for 32 byte cache lines.
Hmmm....I check it into the FSM BK tree a long time ago.
Anyone know how/when these propagate to the stuff on kernel.org?
Thanks,
Graham
--
Graham Stoney
Principal Hardware/Software Engineer
Canon Information Systems Research Australia
Ph: +61 2 9805 2909 Fax: +61 2 9805 2929
** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/