> The eHCA driver can now handle multiple event queues (read: interrupt
> sources) instead of one. The number of available EQs is selected via the
> nr_eqs module parameter.
> CQs are either assigned to the EQs based on the comp_vector index or, if the
> dist_eqs module parameter is supplied, using a round-robin scheme.
Do you have any data on how well this round-robin assignment works?
It seems not quite right to me for the driver to advertise nr_eqs
completion vectors, but then if round-robin is turned on to ignore the
consumer's decision about which vector to use.
Maybe if round-robin is turned on you should report 0 as the number of
completion vectors? Or maybe we should allow well-known values for
the completion vector passed to ib_create_cq to allow consumers to
specify a policy (like round robin) instead of a particular vector?
Maybe the whole interface is broken and we should only be exposing
policies to consumers instead of the specific vector?
I think I would rather hold off on multiple EQs for this merge window
and plan on having something really solid and thought-out for 2.6.24.
- R.
Roland Dreier [off-list ref] wrote on 16.07.2007 18:04:26:
It seems not quite right to me for the driver to advertise nr_eqs
completion vectors, but then if round-robin is turned on to ignore the
consumer's decision about which vector to use.
The round-robin feature was primarily meant as a debug/evaluation feature;
it is not supposed to be active by default. ULP programmers can, for
example, quickly evaluate the performance increase that comp_vectors could
give them, without changing their code. Without this debug option, the
comp_vector policy is still up to the ULPs.
Maybe if round-robin is turned on you should report 0 as the number of
completion vectors?
That sounds like a reasonable idea -- I'll change that right away.
Maybe the whole interface is broken and we should only be exposing
policies to consumers instead of the specific vector?
If so, I think the policies should be handled by the IB core code instead
of being re-invented by each driver. The IB core would then again pass
actual comp_vector values to the driver.
I think I would rather hold off on multiple EQs for this merge window
and plan on having something really solid and thought-out for 2.6.24.
It's your call, but the code is there and I don't expect it to change a
lot later, so it could be used by others to get a first impression of
what's possible using comp_vectors and to gather some experience with
them.
Regards,
Joachim
Roland Dreier [off-list ref] wrote on 16.07.2007 18:04:26:
Do you have any data on how well this round-robin assignment works?
It seems not quite right to me for the driver to advertise nr_eqs
completion vectors, but then if round-robin is turned on to ignore the
consumer's decision about which vector to use.
No, I've no figures to provide here. The background of this dist_eqs
option is actually to allow us testing across all event queues
without to change the testcases resp consumers to use certain
event queue number. Thus, I should comment it as EXPERIMENTAL?
Maybe if round-robin is turned on you should report 0 as the number of
completion vectors? Or maybe we should allow well-known values for
the completion vector passed to ib_create_cq to allow consumers to
specify a policy (like round robin) instead of a particular vector?
Maybe the whole interface is broken and we should only be exposing
policies to consumers instead of the specific vector?
Agree in that device driver should not overwrite consumer's policy
of event queue assigment. Since dist_eqs is disabled as default,
there's no issue, isn't it?
Regarding ib_verbs: perhaps we should provide create/destroy_eq()
and let upper level protocols or consumers dictate the assignment
to cq by passing an event queue pointer to create_cq()...
I think I would rather hold off on multiple EQs for this merge window
and plan on having something really solid and thought-out for 2.6.24.
Fair enough. However why don't let us gather experience with this
feature now? Should we remove dist_eqs option for more consistency?
Thanks
Nam