On Tue, Sep 16, 2025 at 2:31 AM Olga Kornievskaia [off-list ref] wrote:

On Mon, Sep 15, 2025 at 7:46 AM Hannes Reinecke [off-list ref] wrote:

quoted

On 9/5/25 04:46, alistair23@gmail.com wrote:

quoted

From: Alistair Francis <redacted>

The TLS 1.3 specification allows the TLS client or server to send a
KeyUpdate. This is generally used when the sequence is about to
overflow or after a certain amount of bytes have been encrypted.

The TLS spec doesn't mandate the conditions though, so a KeyUpdate
can be sent by the TLS client or server at any time. This includes
when running NVMe-OF over a TLS 1.3 connection.

As such Linux should be able to handle a KeyUpdate event, as the
other NVMe side could initiate a KeyUpdate.

Upcoming WD NVMe-TCP hardware controllers implement TLS support
and send KeyUpdate requests.

This series builds on top of the existing TLS EKEYEXPIRED work,
which already detects a KeyUpdate request. We can now pass that
information up to the NVMe layer (target and host) and then pass
it up to userspace.

Userspace (ktls-utils) will need to save the connection state
in the keyring during the initial handshake. The kernel then
provides the key serial back to userspace when handling a
KeyUpdate. Userspace can use this to restore the connection
information and then update the keys, this final process
is similar to the initial handshake.

Link: https://datatracker.ietf.org/doc/html/rfc8446#section-4.6.3

v2:
  - Change "key-serial" to "session-id"
  - Fix reported build failures
  - Drop tls_clear_err() function
  - Stop keep alive timer during KeyUpdate
  - Drop handshake message decoding in the NVMe layer

Alistair Francis (7):
   net/handshake: Store the key serial number on completion
   net/handshake: Make handshake_req_cancel public
   net/handshake: Expose handshake_sk_destruct_req publically
   nvmet: Expose nvmet_stop_keep_alive_timer publically
   net/handshake: Support KeyUpdate message types
   nvme-tcp: Support KeyUpdate
   nvmet-tcp: Support KeyUpdate

  Documentation/netlink/specs/handshake.yaml |  19 +++-
  Documentation/networking/tls-handshake.rst |   4 +-
  drivers/nvme/host/tcp.c                    |  88 +++++++++++++++--
  drivers/nvme/target/core.c                 |   1 +
  drivers/nvme/target/tcp.c                  | 104 +++++++++++++++++++--
  include/net/handshake.h                    |  17 +++-
  include/uapi/linux/handshake.h             |  14 +++
  net/handshake/genl.c                       |   5 +-
  net/handshake/handshake.h                  |   1 -
  net/handshake/request.c                    |  18 ++++
  net/handshake/tlshd.c                      |  46 +++++++--
  net/sunrpc/svcsock.c                       |   3 +-
  net/sunrpc/xprtsock.c                      |   3 +-
  13 files changed, 289 insertions(+), 34 deletions(-)

Hey Alistair,
thanks for doing this. While the patchset itself looks okay-ish, there
are some general ideas/concerns for it:

- I have posted a patch for replacing the current 'read_sock()'
interface with a recvmsg() base workflow. That should give us
access to the 'real' control message, so it would be good if you
could fold it in.

Thanks for sending that. I'll rebase my changes on top of the patch
and update it all.

quoted

- Olga has send a patchset fixing a security issue with control
messages; the gist is that the network code expects a 'kvec' based
msg buffer when receiving a control message. So essentially one
has to receive a message _without_ a control buffer, check for
MSG_CTRUNC, and then read the control message via kvec.

Oh interesting. I'll see if I can find the patchset and update my
series to follow that.

quoted

Can you ensure that your patchset follows these guidelines?
- There is no method to trigger a KeyUpdate, making it really hard
to test this feature (eg by writin a blktest for it). Ideally we

I have some patches that do send a KeyUpdate [1] which is what I'm
using to test. It allows me to send a KeyUpdate from either side.

quoted

should be able to trigger it from both directions, but having just
one (eg on the target side) should be enough for starters.
A possible interface would be to implement write support to the
'tls_key' debugfs attribute; when writing the same key ID as
the one currently in use the KeyUpdate mechanism could be started.

That's a good point about allowing userspace and blktest to initiate a
KeyUpdate. I'll look at adding support for a debugfs attribute

quoted

But thanks for doing the work!

Hi Alistart,

I would like to pingy-pack on this message and ask a few questions as
I'm a bit confused about this implemenation.

NFS is also interested in being able to handle KeyUpdate functionality
of TLS and having NvME doing it serves as an example. But the general
approach confuses me.

All messages go thru a TLS (kernel) layer portion of sock_recvmsg
(kernel_recvmsg). When the TLS kernel layer detects that it's
non-TLS-data payload, it does various things depending on whether or
not control buffer was set up prior to the call to sock_recvmsg.
KeyUpdate message is a type of HANDSHAKE message and thus non-TLS-data
payload. While I was doing my changes to NvME code I noticed that
there are multiple places NvME (target) calls into kernel_recvmsg()
and thus those places would need to handle receiving non-TLS-data
payloads. Previously there was a TLS alert which is non-data but now
there is Handshake (specifically Keyupdate, but not others).

I guess where I'm going is I don't see how NvMe is connecting
receiving KeyUpdate (ie, identifying that it received specifically
that and not other handshake type) and its handling of KeyUpdate from
kernel_recvmsg the when NvME is just normally receiving data.

The kernel TLS layer is handling the KeyUpdate [2]. The current
upstream Linux TLS layer will decode a KeyUpdate and mark a
`key_update_pending`. Note that upstream doesn't actually do a
KeyUpdate, hence this series.

This patch series reads to me as it is expecting KeyUpdate to be a
part of the Handshake process (ie., there is a patch to "cancel" an
ongoing handshake, there is an upcall to tlshd with the KeyUpdate?).

No, KeyUpdate can't be part of the handshake process

This doesn't make sense to me. KeyUpdate, while a type of Handshake
message, is not done during the handshake -- it is done after sending
the Finished message which concludes the handshake flow (and
involvement of tlshd) and can happen at any time during normal TLS
encrypted message exchange after the handshake. Here's a snippet from
the TLS spec:

Exactly, KeyUpdate can happen at any time after the handshake.

The general idea is that the TLS layer will detect a KeyUpdate [2],
then report EKEYEXPIRED to the NVMe layer which then asks
ktls-utils/gnutls in user space to update the keys. Does that make
more sense?

The KeyUpdate handshake message is used to indicate that the sender
   is updating its sending cryptographic keys.  This message can be sent
   by either peer after it has sent a Finished message.  Implementations
   that receive a KeyUpdate message prior to receiving a Finished
   message MUST terminate the connection with an "unexpected_message"
   alert.

1: https://github.com/alistair23/linux/commit/714d58a0aed5d49fb24ea22497024c3d958a60b8
2: https://github.com/torvalds/linux/blob/46a51f4f5edade43ba66b3c151f0e25ec8b69cb6/net/tls/tls_sw.c#L1775

Alistair