On Thu, Jul 31, 2025 at 2:10 AM Hannes Reinecke [off-list ref] wrote:

On 7/30/25 22:08, Olga Kornievskaia wrote:

quoted

Revert kvec msg iterator before trying to process a TLS alert
when possible.

In nvmet_tcp_try_recv_data(), it's assumed that no msg control
message buffer is set prior to sock_recvmsg(). Hannes suggested
that upon detecting that TLS control message is received log a
message and error out. Left comments in the code for the future
improvements.

Fixes: a1c5dd8355b1 ("nvmet-tcp: control messages for recvmsg()")
Suggested-by: Hannes Reinecke <hare@suse.de>
Reviewed-by: Hannes Reinecky <redacted>
Signed-off-by: Olga Kornievskaia <okorniev@redhat.com>
---
  drivers/nvme/target/tcp.c | 30 +++++++++++++++++++-----------
  1 file changed, 19 insertions(+), 11 deletions(-)

diff --git a/drivers/nvme/target/tcp.c b/drivers/nvme/target/tcp.c
index 688033b88d38..055e420d3f2e 100644
--- a/drivers/nvme/target/tcp.c
+++ b/drivers/nvme/target/tcp.c

@@ -1161,6 +1161,7 @@ static int nvmet_tcp_try_recv_pdu(struct nvmet_tcp_queue *queue)
      if (unlikely(len < 0))
              return len;
      if (queue->tls_pskid) {
+             iov_iter_revert(&msg.msg_iter, len);
              ret = nvmet_tcp_tls_record_ok(queue, &msg, cbuf);
              if (ret < 0)
                      return ret;

@@ -1217,19 +1218,28 @@ static void nvmet_tcp_prep_recv_ddgst(struct nvmet_tcp_cmd *cmd)
  static int nvmet_tcp_try_recv_data(struct nvmet_tcp_queue *queue)
  {
      struct nvmet_tcp_cmd  *cmd = queue->cmd;
-     int len, ret;
+     int len;

      while (msg_data_left(&cmd->recv_msg)) {
+             /* to detect that we received a TlS alert, we assumed that
+              * cmg->recv_msg's control buffer is not setup. kTLS will
+              * return an error when no control buffer is set and
+              * non-tls-data payload is received.
+              */
              len = sock_recvmsg(cmd->queue->sock, &cmd->recv_msg,
                      cmd->recv_msg.msg_flags);
+             if (cmd->recv_msg.msg_flags & MSG_CTRUNC) {
+                     if (len == 0 || len == -EIO) {
+                             pr_err("queue %d: unhandled control message\n",
+                                    queue->idx);
+                             /* note that unconsumed TLS control message such
+                              * as TLS alert is still on the socket.
+                              */

Hmm. Will it get cleared when we close the socket?

If the socket is closed then any data on that socket would be freed.

Or shouldn't we rather introduce proper cmsg handling?

That would be what I have originally proposed (I know that was on the
private list). But yes, we can setup a dedicated kvec to receive the
TLS control message once its been detected and then call
nvme_tcp_tls_record_ok().

Let me know if proper cmsg handling is what's desired for this patch.

(If we do, we'll need it to do on the host side, too)

I can see that the host doesn't have any TLS alert handling now. If
the only place where (TLS) traffic is read from is in host/tcp.c
nvme_tcp_init_connection(), then that's seems like an easy case
because it uses a kvec to back the kernel_recvmsg() msg structure. If
the ctype is tls alert, you can call tls_alert_recv() and pass in the
"iov".  -- assuming patch#4 already went in by that time)

quoted

+                             return -EAGAIN;
+                     }
+             }
              if (len <= 0)
                      return len;
-             if (queue->tls_pskid) {
-                     ret = nvmet_tcp_tls_record_ok(cmd->queue,
-                                     &cmd->recv_msg, cmd->recv_cbuf);
-                     if (ret < 0)
-                             return ret;
-             }

              cmd->pdu_recv += len;
              cmd->rbytes_done += len;

@@ -1267,6 +1277,7 @@ static int nvmet_tcp_try_recv_ddgst(struct nvmet_tcp_queue *queue)
      if (unlikely(len < 0))
              return len;
      if (queue->tls_pskid) {
+             iov_iter_revert(&msg.msg_iter, len);
              ret = nvmet_tcp_tls_record_ok(queue, &msg, cbuf);
              if (ret < 0)
                      return ret;

@@ -1453,10 +1464,6 @@ static int nvmet_tcp_alloc_cmd(struct nvmet_tcp_queue *queue,
      if (!c->r2t_pdu)
              goto out_free_data;

-     if (queue->state == NVMET_TCP_Q_TLS_HANDSHAKE) {
-             c->recv_msg.msg_control = c->recv_cbuf;
-             c->recv_msg.msg_controllen = sizeof(c->recv_cbuf);
-     }

As you delete this you can also remove the definition of 'recv_msg'
from nvmet_tcp_cmd structure.

You mean 'recv_cbuf', right? recv_msg would still be needed by the code.

I can send v2 with that change. Whether or not cmsg handling is needed
in v2 I'd need a confirmation on. Given I'm working on compile only
mode, I'd rather keep changes to minimal.

quoted

      c->recv_msg.msg_flags = MSG_DONTWAIT | MSG_NOSIGNAL;

      list_add_tail(&c->entry, &queue->free_list);

@@ -1736,6 +1743,7 @@ static int nvmet_tcp_try_peek_pdu(struct nvmet_tcp_queue *queue)
              return len;
      }

+     iov_iter_revert(&msg.msg_iter, len);
      ret = nvmet_tcp_tls_record_ok(queue, &msg, cbuf);
      if (ret < 0)
              return ret;

Cheers,

Hannes
--
Dr. Hannes Reinecke                  Kernel Storage Architect
hare@suse.de                                +49 911 74053 688
SUSE Software Solutions GmbH, Frankenstr. 146, 90461 Nürnberg
HRB 36809 (AG Nürnberg), GF: I. Totev, A. McDonald, W. Knoblich