Thread (50 messages) 50 messages, 2 authors, 2021-05-17

Re: nvme tcp receive errors

From: Keith Busch <kbusch@kernel.org>
Date: 2021-04-21 14:29:14

On Tue, Apr 20, 2021 at 10:33:30PM -0700, Sagi Grimberg wrote:
Can you retry with the following applied on top of what I sent you?
Thanks, we'll give this a try. Just a quick question on the first patch:
quoted hunk ↗ jump to hunk
--
diff --git a/drivers/nvme/host/tcp.c b/drivers/nvme/host/tcp.c
index c60c1dcfb587..ff39d37e9793 100644
--- a/drivers/nvme/host/tcp.c
+++ b/drivers/nvme/host/tcp.c
@@ -63,6 +63,7 @@ struct nvme_tcp_request {
        /* send state */
        size_t                  offset;
        size_t                  data_sent;
+       size_t                  data_recvd;
        enum nvme_tcp_send_state state;
        enum nvme_tcp_cmd_state cmd_state;
 };
@@ -769,6 +770,7 @@ static int nvme_tcp_recv_data(struct nvme_tcp_queue
*queue, struct sk_buff *skb,
                *len -= recv_len;
                *offset += recv_len;
                queue->data_remaining -= recv_len;
+               req->data_recvd += recv_len;
Does this need to get reset to 0 during the initial setup? It looks like
a "req->data_recvd = 0" in nvme_tcp_setup_cmd_pdu() should happen, no?
quoted hunk ↗ jump to hunk
        }

        if (!queue->data_remaining) {
@@ -776,6 +778,7 @@ static int nvme_tcp_recv_data(struct nvme_tcp_queue
*queue, struct sk_buff *skb,
                        nvme_tcp_ddgst_final(queue->rcv_hash,
&queue->exp_ddgst);
                        queue->ddgst_remaining = NVME_TCP_DIGEST_LENGTH;
                } else {
+                       BUG_ON(req->data_recvd != req->data_len);
                        req->cmd_state = NVME_TCP_CMD_DATA_DONE;
                        if (pdu->hdr.flags & NVME_TCP_F_DATA_SUCCESS) {
                                req->cmd_state = NVME_TCP_CMD_DONE;
--

There might be a hidden assumption here that may cause this if multiple
c2hdata pdus will come per request...

If that is the case, you can try the following (on top):
--
diff --git a/drivers/nvme/host/tcp.c b/drivers/nvme/host/tcp.c
index ff39d37e9793..aabec8e6810a 100644
--- a/drivers/nvme/host/tcp.c
+++ b/drivers/nvme/host/tcp.c
@@ -773,19 +773,20 @@ static int nvme_tcp_recv_data(struct nvme_tcp_queue
*queue, struct sk_buff *skb,
                req->data_recvd += recv_len;
        }

-       if (!queue->data_remaining) {
+       if (!queue->data_remaining)
+               nvme_tcp_init_recv_ctx(queue);
+
+       if (req->data_recvd == req->data_len) {
                if (queue->data_digest) {
                        nvme_tcp_ddgst_final(queue->rcv_hash,
&queue->exp_ddgst);
                        queue->ddgst_remaining = NVME_TCP_DIGEST_LENGTH;
                } else {
-                       BUG_ON(req->data_recvd != req->data_len);
                        req->cmd_state = NVME_TCP_CMD_DATA_DONE;
                        if (pdu->hdr.flags & NVME_TCP_F_DATA_SUCCESS) {
                                req->cmd_state = NVME_TCP_CMD_DONE;
                                nvme_tcp_end_request(rq, NVME_SC_SUCCESS);
                                queue->nr_cqe++;
                        }
-                       nvme_tcp_init_recv_ctx(queue);
                }
        }
--
_______________________________________________
Linux-nvme mailing list
Linux-nvme@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-nvme
Keyboard shortcuts
hback out one level
jnext message in thread
kprevious message in thread
ldrill in
Escclose help / fold thread tree
?toggle this help