First, avoid double request completion for nvmf_fail_nonready_command if
the request is freed asynchronously.
Second, avoid IO error. If work with nvme native multipath, rdma HBA
drivers may failed due to failed hardware, queue_rq should failed the
request instead of returning error to block layer, So upper layer can
failover the request if another path is available.
Need set the state of request to MQ_RQ_COMPLETE when queue_rq directly
complete the request. So introduce blk_mq_set_request_complete.
V5:
- add comment for blk_mq_set_request_complete.
- fail the request just for -EIO returned by rdma HBA drivers.
- delete the helper: nvme_complete_failed_req.
V4:
- add status parameter for nvme_complete_failed_req.
- export nvme_complete_failed_req instead of inline function.
V3:
- complete the request just for HBA driver path related error.
V2:
- use "switch" instead "if" to check return status.
Chao Leng (3):
blk-mq: introduce blk_mq_set_request_complete
nvme-fabrics: avoid double request completion for
nvmf_fail_nonready_command
nvme-rdma: avoid IO error for nvme native multipath
drivers/nvme/host/fabrics.c | 2 +-
drivers/nvme/host/rdma.c | 13 ++++++++++++-
include/linux/blk-mq.h | 9 +++++++++
3 files changed, 22 insertions(+), 2 deletions(-)
--
2.16.4
Work with nvme native multipath, if a path related error occurs when
queue_rq call HBA drive to send request, queue_rq will return
BLK_STS_IOERR to blk-mq. The request is completed with BLK_STS_IOERR
instead of fail over to retry.
queue_rq need complete the request with NVME_SC_HOST_PATH_ERROR and set
the state of request to MQ_RQ_COMPLETE, the request will fail over to
retry if needed.
Signed-off-by: Chao Leng <redacted>
---
drivers/nvme/host/rdma.c | 13 ++++++++++++-
1 file changed, 12 insertions(+), 1 deletion(-)
When reconnect, the request may be completed with NVME_SC_HOST_PATH_ERROR
in nvmf_fail_nonready_command. The state of request will be changed to
MQ_RQ_IN_FLIGHT before call nvme_complete_rq. If free the request
asynchronously such as in nvme_submit_user_cmd, in extreme scenario
the request may be completed again in tear down process.
nvmf_fail_nonready_command do not need calling blk_mq_start_request
before complete the request. nvmf_fail_nonready_command should set
the state of request to MQ_RQ_COMPLETE before complete the request.
Signed-off-by: Chao Leng <redacted>
---
drivers/nvme/host/fabrics.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
nvme drivers need to set the state of request to MQ_RQ_COMPLETE when
directly complete request in queue_rq.
So add blk_mq_set_request_complete.
Signed-off-by: Chao Leng <redacted>
---
include/linux/blk-mq.h | 9 +++++++++
1 file changed, 9 insertions(+)
From: Christoph Hellwig <hch@lst.de> Date: 2021-02-03 16:16:12
So I think this is conceptually fine, but I still find the API a little
arcane. What do you think about the following incremental patch?
If that looks good and tests good for you I can apply the series with
the modifications:
So I think this is conceptually fine, but I still find the API a little
arcane. What do you think about the following incremental patch?
If that looks good and tests good for you I can apply the series with
the modifications:
This looks good to me.
Thank you very much.
On 2021/2/4 0:14, Christoph Hellwig wrote:
quoted hunk
So I think this is conceptually fine, but I still find the API a little
arcane. What do you think about the following incremental patch?
If that looks good and tests good for you I can apply the series with
the modifications:
From: Christoph Hellwig <hch@lst.de> Date: 2021-02-04 15:31:54
On Thu, Feb 04, 2021 at 08:27:46AM -0700, Jens Axboe wrote:
On 1/31/21 8:49 PM, Chao Leng wrote:
quoted
nvme drivers need to set the state of request to MQ_RQ_COMPLETE when
directly complete request in queue_rq.
So add blk_mq_set_request_complete.
This is a bit iffy, but looks ok for the intended use case. We just have
to be careful NOT to add frivolous users of it...
Yes, that is my main issue with it. The current use case looks fine,
but I suspect every time someone else uses it it's probly going to be
as misuse. I've applied this to nvme-5.12 with the rest of the patches,
it should be on its way to you soon.
On Thu, Feb 04, 2021 at 08:27:46AM -0700, Jens Axboe wrote:
quoted
On 1/31/21 8:49 PM, Chao Leng wrote:
quoted
nvme drivers need to set the state of request to MQ_RQ_COMPLETE when
directly complete request in queue_rq.
So add blk_mq_set_request_complete.
This is a bit iffy, but looks ok for the intended use case. We just have
to be careful NOT to add frivolous users of it...
Yes, that is my main issue with it. The current use case looks fine,
but I suspect every time someone else uses it it's probly going to be
as misuse. I've applied this to nvme-5.12 with the rest of the patches,
it should be on its way to you soon.
Might benefit from a big fat comment on why you probably shouldn't
be using it...
--
Jens Axboe
From: Christoph Hellwig <hch@lst.de> Date: 2021-02-04 15:38:47
On Thu, Feb 04, 2021 at 08:32:17AM -0700, Jens Axboe wrote:
On 2/4/21 8:30 AM, Christoph Hellwig wrote:
quoted
On Thu, Feb 04, 2021 at 08:27:46AM -0700, Jens Axboe wrote:
quoted
On 1/31/21 8:49 PM, Chao Leng wrote:
quoted
nvme drivers need to set the state of request to MQ_RQ_COMPLETE when
directly complete request in queue_rq.
So add blk_mq_set_request_complete.
This is a bit iffy, but looks ok for the intended use case. We just have
to be careful NOT to add frivolous users of it...
Yes, that is my main issue with it. The current use case looks fine,
but I suspect every time someone else uses it it's probly going to be
as misuse. I've applied this to nvme-5.12 with the rest of the patches,
it should be on its way to you soon.
Might benefit from a big fat comment on why you probably shouldn't
be using it...
On Thu, Feb 04, 2021 at 08:32:17AM -0700, Jens Axboe wrote:
quoted
On 2/4/21 8:30 AM, Christoph Hellwig wrote:
quoted
On Thu, Feb 04, 2021 at 08:27:46AM -0700, Jens Axboe wrote:
quoted
On 1/31/21 8:49 PM, Chao Leng wrote:
quoted
nvme drivers need to set the state of request to MQ_RQ_COMPLETE when
directly complete request in queue_rq.
So add blk_mq_set_request_complete.
This is a bit iffy, but looks ok for the intended use case. We just have
to be careful NOT to add frivolous users of it...
Yes, that is my main issue with it. The current use case looks fine,
but I suspect every time someone else uses it it's probly going to be
as misuse. I've applied this to nvme-5.12 with the rest of the patches,
it should be on its way to you soon.
Might benefit from a big fat comment on why you probably shouldn't
be using it...