IB_MR_TYPE_SG_GAPS support

Hi-

I have a report that net/sunrpc/xprtrdma/frwr_ops.c::frwr_map()
is attempting to register a mix of GPU and CPU pages with a
single MR during an NFSv4 READ.

We think we understand where both pages are coming from, and
why frwr_map() is attempting to coalesce them (the verbs
provider happens to support IB_DEVICE_SG_GAPS_REG).

frwr_map() does this:

305         for (i = 0; i < nsegs;) {
306                 sg_set_page(&mr->mr_sg[i], seg->mr_page,
307                             seg->mr_len, seg->mr_offset);
308 
309                 ++seg;
310                 ++i;
311                 if (ep->re_mrtype == IB_MR_TYPE_SG_GAPS)
312                         continue;
313                 if ((i < nsegs && seg->mr_offset) ||
314                     offset_in_page((seg-1)->mr_offset + (seg-1)->mr_len))
315                         break;
316         }

If GPU and CPU pages are not considered contiguous, then some
additional loop termination logic is needed here so that at
a boundary between page types, xprtrdma will simply move
into a fresh MR instead of coalescing.

Does anyone have suggestions of where to look for an
appropriate check?

TIA

--
Chuck Lever