Re: [PATCH net-next 10/10] net/mlx5e: Use the 'num_doorbells' devlink param
From: Cosmin Ratiu <hidden>
Date: 2025-09-16 08:32:25
Also in:
bpf, linux-doc, linux-rdma, lkml
On Wed, 2025-09-10 at 09:16 -0700, Stanislav Fomichev wrote:
quoted
+ - 0: No channel-specific doorbells, use the global one for everything. + - [1, max_num_channels]: Spread netdev channels equally across these + doorbells.Do you have any guidance on this number? Why would the user want `num_doorbells < num_doorbells` vs `num_doorbells == num_channels`? IOW, why not allocate the same number of doorbells as the number of channels and do it unconditionally without devlink param? Are extra doorbells causing any overhead in the non-contended case?
In most cases, additional doorbells are an overhead and not required. For the last 10+ years, mlx5 has been running with a single doorbell for all channels. But as the number of cores and channels grew, bottlenecks were discovered on some platforms. Thus the need for this series. This series proposes 8 as the new default and we expect nobody would need to touch this knob except in extreme cases, but I think it's nice for it to exist. Regression testing showed little to no impact on most platforms where 1 doorbell was enough, and a significant improvement on platforms that showed MMIO bottlenecks. I'll add some numbers in the next version's cover letter. Cosmin.