Re: IPoIB performance
From: Atchley, Scott <hidden>
Date: 2012-09-05 17:59:23
On Sep 5, 2012, at 1:50 PM, Reeted wrote:
On 08/29/12 21:35, Atchley, Scott wrote:quoted
Hi all, I am benchmarking a sockets based application and I want a sanity check on IPoIB performance expectations when using connected mode (65520 MTU).....I have read that with newer cards the datagram (unconnected) mode is faster at IPoIB than connected mode. Do you want to check?
I have read that the latency is lower (better) but the bandwidth is lower. Using datagram mode limits the MTU to 2044 and the throughput to ~3 Gb/s on these machines/cards. Connected mode at the same MTU performs roughly the same. The win in connected mode comes with larger MTUs. With a 9000 MTU, I see ~6 Gb/s. Pushing the MTU to 655120 (the maximum for ipoib), I can get ~13 Gb/s.
What benchmark program are you using?
netperf with process binding (-T). I tune sysctl per the DOE FasterData specs: http://fasterdata.es.net/host-tuning/linux/ Scott-- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org More majordomo info at http://vger.kernel.org/majordomo-info.html