Thread (15 messages) 15 messages, 6 authors, 2005-10-11

Re: Timeline of IPoIB performance

From: Matt Leininger <hidden>
Date: 2005-10-08 02:25:49

Possibly related (same subject, not in this thread)

I'm adding netdev to this thread to see if they can help.

I'm seeing an IPoIB (IP over InfiniBand) netperf performance drop off,
of up to 90 MB/s, when using kernels newer than 2.6.11.  This doesn't
appear to be an OpenIB IPoIB issue since the older in-kernel IB for
2.6.11 and a recent svn3687 snapshot both have the same performance (464
MB/s) with 2.6.11.  I used the same kernel config file as a starting
point for each of these kernel builds.  Have there been any changes in
Linux that would explain these results?

Here is the hardware setup and netperf results using 'netperf -f -M -c
-C -H IPoIB_ADDRESS

All benchmarks are with RHEL4 x86_64 with HCA FW v4.7.0
dual EM64T 3.2 GHz PCIe IB HCA (memfull)

Kernel           OpenIB    msi_x  netperf (MB/s)  
2.6.14-rc3      in-kernel    1     374 
2.6.13.2        svn3627      1     386 
2.6.13.2        in-kernel    1     394 
2.6.12.5-lustre in-kernel    1     399  
2.6.12.5        in-kernel    1     402 
2.6.12          in-kernel    1     406 
2.6.12-rc6      in-kernel    1     407
2.6.12-rc5      in-kernel    1     405   <<<<<
2.6.12-rc4      in-kernel    1     470   <<<<<
2.6.12-rc3      in-kernel    1     466 
2.6.12-rc2      in-kernel    1     469 
2.6.12-rc1      in-kernel    1     466
2.6.11          in-kernel    1     464 
2.6.11          svn3687      1     464 
2.6.9-11.ELsmp  svn3513      1     425  (Woody's results, 3.6Ghz EM64T) 

 Thanks,

	- Matt
Keyboard shortcuts
hback out one level
jnext message in thread
kprevious message in thread
ldrill in
Escclose help / fold thread tree
?toggle this help