Hi Jakub,

On Thu, 8 Aug 2024 at 16:52, Jakub Kicinski [off-list ref] wrote:

On Thu, 8 Aug 2024 14:12:34 +0300 Ilias Apalodimas wrote:

quoted

In any case why do you want to hold on the IOMMU? The network
interface -- at least in theory -- should be down and we wont be
processing any more packets.

I should have you a link to Yonglong's report:

https://lore.kernel.org/all/8743264a-9700-4227-a556-5f931c720211@huawei.com/ (local)

Ah I read that bug report during my vacation but completely forgot about it...
Thanks this helps.

we get_device() hoping that it will keep the IOMMU machinery active
(even if the device won't use the page we need to unmap it when it's
freed), but it sounds like IOMMU dies when driver is unbound. Even if
there are outstanding references to the device.

I occasionally hit this problem reloading drivers during development,
TBH, too. And we have been told we "use the API wrong" so let's fix
it on our end?..

It's been a while since I looked at the use cases, but I don't love
the idea of stalling the netdev removal until sockets process all
packets. There's a chance that the device will stay there forever.
I'll have to take a closer look but the first thing that comes to mind
is to unmap the pages early, before page_pool_destroy() is called and
perhaps add a flag that says "the pool is there only to process
existing packets, but you can't DMA into it anymore".

Thanks
/Ilias