On Mon, Apr 19, 2021 at 8:43 AM Ilias Apalodimas
[off-list ref] wrote:

[...]

quoted

Pages mapped into the userspace have their refcnt elevated, so the
page_ref_count() check by the drivers indicates to not reuse such
pages.

When tcp_zerocopy_receive() is invoked it will call tcp_zerocopy_vm_insert_batch()
which will end up doing a get_page().
What you are saying is that once the zerocopy is done though, skb_release_data()
won't be called, but instead put_page() will be? If that's the case then we are
indeed leaking DMA mappings and memory. That sounds weird though, since the
refcnt will be one in that case (zerocopy will do +1/-1 once it's done), so who
eventually frees the page?
If kfree_skb() (or any wrapper that calls skb_release_data()) is called
eventually, we'll end up properly recycling the page into our pool.

From what I understand (Eric, please correct me if I'm wrong) for
simple cases there are 3 page references taken. One by the driver,
second by skb and third by page table.

In tcp_zerocopy_receive(), tcp_zerocopy_vm_insert_batch() gets one
page ref through insert_page_into_pte_locked(). However before
returning from tcp_zerocopy_receive(), the skb references are dropped
through tcp_recv_skb(). So, whenever the user unmaps the page and
drops the page ref only then that page can be reused by the driver.

In my understanding, for zerocopy rx the skb_release_data() is called
on the pages while they are still mapped into the userspace. So,
skb_release_data() might not be the right place to recycle the page
for zerocopy. The email chain at [1] has some discussion on how to
bundle the recycling of pages with their lifetime.

[1] https://lore.kernel.org/linux-mm/20210316013003.25271-1-arjunroy.kdev@gmail.com/ (local)