On Tue, 29 Mar 2005 14:32:33 -0500 (EST)
John Heffner [off-list ref] wrote:

The motivation for my question is that I get very unpredictable
performance over loopback with UP for all architectures, often varying by
more than a factor of two.  I haven't really tried to track down the
cause, but an important characteristic seems to be that the greater the
differential between the CPU utilization of the sender and the receiver,
the slower the throughput.  (But I'm not sure if there's a causal relation
here.)  Maybe this is simply scheduler strangeness, since it doesn't seem
to be an issue that I've noticed on SMP.  Has anyone seen this or know
offhand what's going on?

It could be L2 cache-coloring effects as well.  Try to keep the working
set size smaller than the L2 cache size of the cpu you are on.