On Fri, Mar 12, 2021 at 4:04 PM Zhu Yanjun [off-list ref] wrote:

On Fri, Mar 12, 2021 at 8:25 AM Jason Gunthorpe [off-list ref] wrote:

quoted

On Thu, Mar 11, 2021 at 06:41:43PM +0800, Zhu Yanjun wrote:

quoted

On Mon, Mar 8, 2021 at 8:16 PM Jason Gunthorpe [off-list ref] wrote:

quoted

On Mon, Mar 08, 2021 at 06:13:52PM +0800, Zhu Yanjun wrote:

quoted

And I delved into the source code of __sg_alloc_table_from_pages. I
found that this function is related with ib_dma_max_seg_size. So
when ib_dma_max_seg_size is set to UINT_MAX, the sg dma address is
4K (one page). When ib_dma_max_seg_size is set to SZ_2M, the sg dma
address is 2M now.

That seems like a bug, you should fix it

Hi, Jason && Leon

I compared the function __sg_alloc_table_from_pages with ib_umem_add_sg_table.
In __sg_alloc_table_from_pages:

"
 449         if (prv) {
 450                 unsigned long paddr = (page_to_pfn(sg_page(prv))
* PAGE_SIZE +
 451                                        prv->offset + prv->length) /
 452                                       PAGE_SIZE;
 453
 454                 if (WARN_ON(offset))
 455                         return ERR_PTR(-EINVAL);
 456
 457                 /* Merge contiguous pages into the last SG */
 458                 prv_len = prv->length;
 459                 while (n_pages && page_to_pfn(pages[0]) == paddr) {
 460                         if (prv->length + PAGE_SIZE > max_segment)
 461                                 break;
 462                         prv->length += PAGE_SIZE;
 463                         paddr++;
 464                         pages++;
 465                         n_pages--;
 466                 }
 467                 if (!n_pages)
 468                         goto out;
 469         }

"
if prv->length + PAGE_SIZE > max_segment, then set another sg.
In the commit "RDMA/umem: Move to allocate SG table from pages",
max_segment is dma_get_max_seg_size.
Normally it is UINT_MAX. So in my host, prv->length + PAGE_SIZE is
usually less than max_segment
since length is unsigned int.

I don't understand what you are trying to say

  460                         if (prv->length + PAGE_SIZE > max_segment)

max_segment should be a very big number and "prv->length + PAGE_SIZE" should
always be < max_segment so it should always be increasing the size of
prv->length and 'rpv' here is the sgl.

Sure.

When max_segment is UINT_MAX, prv->length is UINT_MAX - PAGE_SIZE.
It is big. That is, segment size is UINT_MAX - PAGE_SIZE.

But from the function ib_umem_add_sg_table, the prv->length is SZ_2M.
That is, segment size if SZ_2M.

It is the difference between the 2 functions.

BTW, my commit is to fix this difference.

Zhu Yanjun

Zhu Yanjun

quoted

The other loops are the same.

Jason