RE: [PATCH] An argument for allowing applications to manually send RMPP packets if desired
From: Mike Heinz <hidden>
Date: 2011-09-16 18:28:16
There hasn't been any discussion of this patch since Monday, I'd like to see if we can keep this conversation moving forward. Right now, as I see it, we have a scalability issue with how rmpp packets are currently handled. While the current method works great on small to mid-sized fabrics - and vastly simplifies life for the application programmer - as fabrics grow in size, it risks memory exhaustion when many nodes make similar queries to a server at the same time. In addition, it eliminates the flexibility provided in the RMPP protocol; if the SA/SM handles RMPP packets individually it can not only stash packets for re-use, it can take advantage of the timing and windowing in the protocol to throttle its responses during periods of heavy load. Note that no one is arguing that the existing model isn't appropriate for client applications - I doubt we will be seeing fabrics so large that a single NodeInfo response would exhaust all RAM on the client. Roland, you suggested developing a new interface that avoids the copying without requiring the full protocol to be handled in user space, possibly using writev(). But is it really worthwhile to add complexity to ib_mad, ib_umad, libibumad and libibmad for something that's only needed on (for example) one machine out of several thousand? In addition, such an interface still would not allow a sophisticated SM implementation to do load management of the queries. Trying to put load management of MAD queries in the kernel itself would result in a constantly changing kernel module as we try to adapt to emerging issues as the fabrics become ever larger. The patch at hand avoids all that complexity by simply allowing the SM application handle RMPP itself in whatever manner is considered best. If the SM wants to use the current RMPP implementation it can, turning this ability on is completely optional. This message and any attached documents contain information from QLogic Corporation or its wholly-owned subsidiaries that may be confidential. If you are not the intended recipient, you may not read, copy, distribute, or use this information. If you have received this transmission in error, please notify the sender immediately by reply e-mail and then delete this message. -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org More majordomo info at http://vger.kernel.org/majordomo-info.html