Re: [PATCH v2 2/5] IB/core, RDMA RW API: Do not exceed QP SGE send limit
From: Sagi Grimberg <hidden>
Date: 2016-07-13 09:32:20
-static inline u32 rdma_rw_max_sge(struct ib_device *dev,
- enum dma_data_direction dir)
-{
- return dir == DMA_TO_DEVICE ?
- dev->attrs.max_sge : dev->attrs.max_sge_rd;
-}
-Wait, this looks wrong... iWARP devices have max_sge_rd = 1, Steve, did you get to test this?
quoted hunk
static inline u32 rdma_rw_fr_page_list_len(struct ib_device *dev) { /* arbitrary limit to avoid allocating gigantic resources */@@ -186,7 +179,7 @@ static int rdma_rw_init_map_wrs(struct rdma_rw_ctx *ctx, struct ib_qp *qp, u64 remote_addr, u32 rkey, enum dma_data_direction dir) { struct ib_device *dev = qp->pd->device; - u32 max_sge = rdma_rw_max_sge(dev, dir); + u32 max_sge = qp->max_send_sge;
Here, rdma_read will use qp.max_send_sge = max_sge which is set by the ULP. If the ULP used max(max_sge, max_sge_rd) which is the right thing, then we get a bug, if the ULP set min(max_sge, max_sge_rd) then it has effectively a single sge even for writes (which is not good). The QP user needs to set the maximum sge allowed for reads or writes, but for reads use max_sge_rd and for writes use max_sge. Otherwise this will break. -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org More majordomo info at http://vger.kernel.org/majordomo-info.html