Re: [PATCH net v1 2/2] gve: use max allowed ring size for ZC page_pools

On Mon, Nov 10, 2025 at 12:36:46PM +0000, Pavel Begunkov wrote:

On 11/7/25 13:35, Dragos Tatulea wrote:

quoted

On Thu, Nov 06, 2025 at 05:18:33PM -0800, Jakub Kicinski wrote:

quoted

On Thu, 6 Nov 2025 17:25:43 +0000 Dragos Tatulea wrote:

quoted

On Wed, Nov 05, 2025 at 06:56:46PM -0800, Mina Almasry wrote:

quoted

On Wed, Nov 5, 2025 at 6:22 PM Jakub Kicinski [off-list ref] wrote:

quoted

Increasing cache sizes to the max seems very hacky at best.
The underlying implementation uses genpool and doesn't even
bother to do batching.

OK, my bad. I tried to think through downsides of arbitrarily
increasing the ring size in a ZC scenario where the underlying memory
is pre-pinned and allocated anyway, and I couldn't think of any, but I
won't argue the point any further.

I see a similar issue with io_uring as well: for a 9K MTU with 4K ring
size there are ~1% allocation errors during a simple zcrx test.

mlx5 calculates 16K pages and the io_uring zcrx buffer matches exactly
that size (16K * 4K). Increasing the buffer doesn't help because the
pool size is still what the driver asked for (+ also the
internal pool limit). Even worse: eventually ENOSPC is returned to the
application. But maybe this error has a different fix.

Hm, yes, did you trace it all the way to where it comes from?
page pool itself does not have any ENOSPC AFAICT. If the cache
is full we free the page back to the provider via .release_netmem

Yes I did. It happens in io_cqe_cache_refill() when there are no more
CQEs:
https://elixir.bootlin.com/linux/v6.17.7/source/io_uring/io_uring.c#L775

-ENOSPC here means io_uring's CQ got full. It's non-fatal, the user
is expected to process completions and reissue the request. And it's
best to avoid that for performance reasons, e.g. by making the CQ
bigger as you already noted.

Got it. Thanks Pavel!

Thanks,
Dragos