Re: [net-next 10/14] net/mlx5: Let user configure io_eq_size param

On Tue, 26 Oct 2021 15:54:28 +0000 Saeed Mahameed wrote:

On Tue, 2021-10-26 at 08:05 -0700, Jakub Kicinski wrote:

quoted

On Mon, 25 Oct 2021 13:54:27 -0700 Saeed Mahameed wrote:  

quoted

From: Shay Drory <redacted>

Currently, each I/O EQ is taking 128KB of memory. This size
is not needed in all use cases, and is critical with large scale.
Hence, allow user to configure the size of I/O EQs.

For example, to reduce I/O EQ size to 64, execute:
$ devlink resource set pci/0000:00:0b.0 path /io_eq_size/ size 64
$ devlink dev reload pci/0000:00:0b.0  

This sort of config is needed by more drivers,
we need a standard way of configuring this.

We had a debate internally about the same thing, Jiri and I thought
that EQ might be a ConnectX only thing (maybe some other vendors have
it) but it is not really popular

I thought it's a RDMA thing. At least according to grep there's 
a handful of non-MLX drivers which have eqs. Are these not actual
event queues? (huawei/hinic, ibm/ehea, microsoft/mana, qlogic/qed)

we thought, until other vendors start contributing or asking for 
the same thing, maybe then we can standardize.

Yeah, like the standardization part ever happens :/ 

Look at the EQE/CQE interrupt generation thing. New vendor comes in and
copies best known practice (which is some driver-specific garbage,
ethtool priv-flags in that case). The maintainer (me) has to be the
policeman remember all those knobs with prior art and push back. Most
of the time the vendor decides to just keep the knob out of tree at
this point, kudos to Hauwei for not giving up. New vendor implements
the API, none of the existing vendors provide reviews or feedback.
Then none of the existing vendors implements the now-standard API.
Someone working for a large customer (me, again) has to go and ask 
for the API to be implemented. Which takes months even tho the patches
should be trivial.

If the initial patches adding the cqe/eqe interrupt modes to priv-flags
were nacked and the standard API created we'd all have saved much time.

quoted

Sorry, I didn't have the time to look thru your patches
yesterday, I'm sending a revert for all your new devlink
params.  

Sure, we will submit a RFC to give other vendors a chance to comment,
it will be basically the same patch (devlink resource) while making the
parameters vendor generic.

IDK if resource is a right fit (as mentioned to Parav in the discussion
on the revert).