Re: [PATCHv2 2/2] nvme-pci: remove cached shadow doorbell offsets

On Thu, Oct 14, 2021 at 10:49:38AM -0700, Keith Busch wrote:

On Thu, Oct 14, 2021 at 06:09:30PM +0100, John Levon wrote:

quoted

On Thu, Oct 14, 2021 at 09:45:43AM -0700, Keith Busch wrote:

quoted

And when this feature is in use, the specification requires all queue
updates use this mechanism, so don't don't treat the admin queue
differently.

This would cause implementation difficulties for a controller, right?

If the controller is not maintaining the admin queue shadow, then the
event index will always tell the host to ring the doorbell, right? So
the same host behvaior would ultimately happen.

The eventidx only causes the additional BAR0 writes if the doorbell update
crosses it (for example, a 4->6 transition when host sees eventidx of 5). So we
couldn't just leave eventidx at zero, and neither can we maintain it to force
BAR0 writes, as there are races that means we'd have to look at the shadow
doorbell value - and make assumptions - anyway.

This would all be a lot easier if zero wasn't a valid doorbell value, but we're
stuck now.

quoted

Currently we presume that the admin queue's shadow doorbells aren't valid, so
always look at the BAR0 location. Given we need to support older Linux versions,
we don't have any non-hacky way to handle this: we'd need something horrible
that presumes those Linux versions - and any other driver implementations -
always leave the admin queue shadows at zero, so if we see a non-zero value, it
must be doing shadow doorbells.

It's unclear to me what the advantage of fixing this is, given Linux has been
out of spec for so long.

Ah, I skimmed the part that said we are not up to spec compliance, so
aligning with the spec with any feature usually justifies itself. I see
your point was the opposite :)

Exactly. The test SPDK client is out of spec in exactly the same way.

I'm trying to follow up on the spec side so it reflects reality (I can't find a
driver that behaves differently here), but that's obviously a slow process.

thanks
john