kcm: reject attaching sockets with a ULP, and TLS on KCM sockets

Started by Rishikesh Jethwani on Nov 2, 2026 · 3 messages from 3 authors · Last reply Nov 4, 2026

Subsystems (2)

Maintainers (5)

Show 1 more
COLD17d REVIEWED · 1 review, 0 by maintainers

Status: Reviewed.

1 review trailer.

open the first message on its own page

Message details and related content
KCM and kTLS RX do not work together, in either order.

KCM's psock_data_ready() is the lower socket's data_ready callback. It
can run in softirq context under read_lock_bh(&sk->sk_callback_lock),
and calls strp_data_ready(), which invokes sock->ops->read_sock()
inline. With kTLS RX, that is tls_sw_read_sock(), which can sleep: it
allocates with sk->sk_allocation, waits for async crypto, flushes the
socket backlog, and may wait in tls_rx_reader_acquire().

Found by code review. On an unmodified kernel with
CONFIG_DEBUG_ATOMIC_SLEEP, sending TLS records over loopback to a socket
with software kTLS RX attached to a KCM mux reports:

  BUG: sleeping function called from invalid context at include/linux/sched/mm.h:321
  in_atomic(): 1, irqs_disabled(): 0, non_block: 0, pid: 189, name: ksoftirqd/34
  Call Trace:
   __might_resched
   kmem_cache_alloc_node_noprof
   __alloc_skb
   alloc_skb_with_frags
   tls_rx_one_record
   tls_sw_read_sock
   strp_read_sock
   strp_data_ready
   psock_data_ready [kcm]

Before commit 662fbcec32f4 ("net/tls: implement ->read_sock()"), TLS
inherited tcp_read_sock(), so KCM parsed the ciphertext in the TCP
receive queue as messages. HW TLS RX uses the same proto_ops as SW
RX, so it is affected the same way.

The two also stack socket callbacks and neither restores the other's:
with TLS RX configured after the attach, kcm_unattach() overwrites
tls_data_ready(), and releasing TLS RX later reinstalls
psock_data_ready() on a socket no longer attached to KCM. On TX,
kcm_unattach() restores its saved sk_write_space, dropping
tls_write_space().

There is no known user of this combination, so reject it, as was done
for sockmap in commit 460e6486617c ("tls: reject the combination of TLS

65 more lines

diff --git a/net/kcm/kcmsock.c b/net/kcm/kcmsock.c
index a7f94d42036f..1f8bcbb8b9c3 100644
--- a/net/kcm/kcmsock.c
+++ b/net/kcm/kcmsock.c
@@ -30,11 +30,26 @@
 #include <net/kcm.h>
 #include <net/netns/generic.h>
 #include <net/sock.h>
+#include <net/tcp.h>
 #include <uapi/linux/kcm.h>
 #include <trace/events/sock.h>
 
 unsigned int kcm_net_id;
 
+/* Not registered, so TCP_ULP can't select it. It only holds
+ * icsk_ulp_ops so tcp_set_ulp() refuses other ULPs like kTLS. Like any
+ * ULP it stays until the socket is destroyed, pinning this module, and
+ * tcp_cleanup_ulp() releases it. Clearing it on a live socket would race
+ * with lockless readers such as do_tcp_getsockopt() and tcp_diag.
+ *
+ * No .clone: children copy icsk_ulp_ops without a module reference, so
+ * a socket that was once attached must not listen().
+ */
+static const struct tcp_ulp_ops kcm_ulp_ops = {
+	.name = "kcm",
+	.owner = THIS_MODULE,
+};
+
 static struct kmem_cache *kcm_psockp __read_mostly;
 static struct kmem_cache *kcm_muxp __read_mostly;
 static struct workqueue_struct *kcm_wq;
@@ -1242,6 +1257,7 @@ static int kcm_attach(struct socket *sock, struct socket *csock,
 {
 	struct kcm_sock *kcm = kcm_sk(sock->sk);
 	struct kcm_mux *mux = kcm->mux;
+	struct inet_connection_sock *icsk;
 	struct sock *csk;
 	struct kcm_psock *psock = NULL, *tpsock;
 	struct list_head *head;
@@ -1259,12 +1275,14 @@ static int kcm_attach(struct socket *sock, struct socket *csock,
 
 	lock_sock(csk);
 
-	/* Only allow TCP sockets to be attached for now */
-	if ((csk->sk_family != AF_INET && csk->sk_family != AF_INET6) ||
-	    csk->sk_protocol != IPPROTO_TCP) {
+	/* Only allow TCP sockets to be attached for now. A raw socket
+	 * created with IPPROTO_TCP has that sk_protocol but no icsk.
+	 */
+	if (!sk_is_tcp(csk)) {
 		err = -EOPNOTSUPP;
 		goto out;
 	}
+	icsk = inet_csk(csk);
 
 	/* Don't allow listeners or closed sockets */
 	if (csk->sk_state == TCP_LISTEN || csk->sk_state == TCP_CLOSE) {
@@ -1272,6 +1290,15 @@ static int kcm_attach(struct socket *sock, struct socket *csock,
 		goto out;
 	}
 
+	/* KCM takes over read_sock() and the socket callbacks like a ULP
+	 * does, the two can't stack. Our own marker left by an earlier
+	 * attach is fine, a concurrent attach is caught by sk_user_data.
+	 */
+	if (icsk->icsk_ulp_ops && icsk->icsk_ulp_ops != &kcm_ulp_ops) {
+		err = -EOPNOTSUPP;
+		goto out;
+	}
+
 	psock = kmem_cache_zalloc(kcm_psockp, GFP_KERNEL);
 	if (!psock) {
 		err = -ENOMEM;
@@ -1305,6 +1332,11 @@ static int kcm_attach(struct socket *sock, struct socket *csock,
 	psock->save_write_space = csk->sk_write_space;
 	psock->save_state_change = csk->sk_state_change;
 	csk->sk_user_data = psock;
+	if (!icsk->icsk_ulp_ops) {
+		/* Paired with module_put() in tcp_cleanup_ulp() */
+		__module_get(THIS_MODULE);
+		icsk->icsk_ulp_ops = &kcm_ulp_ops;
+	}
 	WRITE_ONCE(csk->sk_data_ready, psock_data_ready);
 	WRITE_ONCE(csk->sk_write_space, psock_write_space);
 	csk->sk_state_change = psock_state_change;
-- 
2.50.1
Message details and related content
Thank you for your contribution! Sashiko AI review found 3 potential
issue(s) to consider.

Critical: 0 · High: 0 · Medium: 1 · Low: 2

If you believe none of the issues are legitimate, please reply to each
issue of Medium severity or higher and explain why the AI is wrong.
If any of them are legitimate and you plan to respin, please let patchwork
know by including "pw-bot: cr" as a separate line at the end of your reply
(one such reply per series is enough).

- [Medium] The commit message understates one user-visible regression.
- [Low] The commit message says 'rmmod kcm waits until such sockets are
  destroyed'.
- [Low] Documentation/networking/kcm.rst ('Attach transport sockets' /
  'Unattach transport sockets', lines 144-190) is not updated for the…

--

kcm: reject attaching sockets with a ULP, and TLS on KCM sockets

With this commit, kcm_attach() refuses TCP sockets that already have a
ULP, and it now refuses raw IPPROTO_TCP sockets too. Attached sockets get
an unregistered dummy kcm_ulp_ops, so tcp_set_ulp() refuses kTLS on them
afterwards. The marker stays until the socket is destroyed, and it pins
the kcm module.

Link: https://lore.kernel.org/netdev/3108e934-ec00-458b-8720-450554b35b55@linux.dev/
Holding a ULP also has visible side effects: getsockopt(TCP_ULP) and
inet_diag report "kcm" on a socket that was ever attached,
sk_has_decrypt_user() is true so PSP RX association is refused after the
attach, and adding an attached socket to a sockmap fails with -EINVAL
rather than -EBUSY.
[Severity: Medium]

Is the sockmap effect really limited to attached sockets?
SIOCKCMUNATTACH does not remove the kcm_ulp_ops marker, because
kcm_unattach() only clears sk_user_data:

net/kcm/kcmsock.c:kcm_unattach() {
    ...
	write_lock_bh(&csk->sk_callback_lock);
	csk->sk_user_data = NULL;
	WRITE_ONCE(csk->sk_data_ready, psock->save_data_ready);
    ...
}

sk_psock_init() checks for a ULP before it looks at sk_user_data:

net/core/skmsg.c:sk_psock_init() {
    ...
	if (sk_is_inet(sk) && inet_csk_has_ulp(sk)) {
		psock = ERR_PTR(-EINVAL);
		goto out;
	}
    ...
}

Before this patch, this sequence succeeded:

  SIOCKCMATTACH -> SIOCKCMUNATTACH -> BPF_MAP_UPDATE_ELEM on a sockmap

With this patch, the map update fails with -EINVAL for the rest of the
socket's life.

Could the stable-facing paragraph that lists the user-visible changes also
say that a socket that was ever attached can no longer be added to a
sockmap or sockhash?
listen() after a disconnect fails with -EINVAL, as
the dummy has no .clone, and rmmod kcm waits until such sockets are
destroyed.
[Severity: Low]

Is "waits" accurate here? Unloading does not block on a non-zero module
refcount:

kernel/module/main.c:try_stop_module() {
	/* If it's not unused, quit unless we're forcing. */
	if (try_release_module_ref(mod) != 0) {
		*forced = try_force_unload(flags);
		if (!(*forced))
			return -EWOULDBLOCK;
	}
    ...
}

The reference taken by __module_get() in kcm_attach() is only dropped by
tcp_cleanup_ulp() when the TCP socket is destroyed. As long as any TCP
socket that was once attached still exists, rmmod kcm fails at once with
"Module kcm is in use". This holds even after every KCM socket is closed
and every socket has been unattached.
quoted hunk
diff --git a/net/kcm/kcmsock.c b/net/kcm/kcmsock.c
index a7f94d42036fb..1f8bcbb8b9c3b 100644
--- a/net/kcm/kcmsock.c
+++ b/net/kcm/kcmsock.c
@@ -30,11 +30,26 @@
[ ... ]
quoted hunk
+/* Not registered, so TCP_ULP can't select it. It only holds
+ * icsk_ulp_ops so tcp_set_ulp() refuses other ULPs like kTLS. Like any
+ * ULP it stays until the socket is destroyed, pinning this module, and
+ * tcp_cleanup_ulp() releases it. Clearing it on a live socket would race
+ * with lockless readers such as do_tcp_getsockopt() and tcp_diag.
+ *
+ * No .clone: children copy icsk_ulp_ops without a module reference, so
+ * a socket that was once attached must not listen().
+ */
+static const struct tcp_ulp_ops kcm_ulp_ops = {
+	.name = "kcm",
+	.owner = THIS_MODULE,
+};
[Severity: Low]

This isn't a bug, but should Documentation/networking/kcm.rst be updated
as well? The "Attach transport sockets" section only says:

  - fd: file descriptor for TCP socket being attached

The "Unattach transport sockets" section says unattaching "is
straightforward".

After this patch, SIOCKCMATTACH refuses raw IPPROTO_TCP sockets and
sockets that already have a ULP, both with -EOPNOTSUPP.

The kcm marker also stays after SIOCKCMUNATTACH. For the rest of the
socket's life:

  - setting TCP_ULP fails with -EEXIST
  - getsockopt(TCP_ULP) and inet_diag report "kcm"
  - listen() fails with -EINVAL
  - sockmap insertion fails with -EINVAL
  - PSP RX association is refused

[ ... ]

-- 
Sashiko AI review · https://netdev-ai.bots.linux.dev/sashiko/#/patchset/20261102220310.12649-1-rjethwani%40purestorage.com
Message details and related content
On Tue, Nov 3, 2026 at 2:05 PM [off-list ref] wrote:
Thank you for your contribution! Sashiko AI review found 3 potential
issue(s) to consider.

Critical: 0 · High: 0 · Medium: 1 · Low: 2

If you believe none of the issues are legitimate, please reply to each
issue of Medium severity or higher and explain why the AI is wrong.
If any of them are legitimate and you plan to respin, please let patchwork
know by including "pw-bot: cr" as a separate line at the end of your reply
(one such reply per series is enough).

- [Medium] The commit message understates one user-visible regression.
- [Low] The commit message says 'rmmod kcm waits until such sockets are
  destroyed'.
- [Low] Documentation/networking/kcm.rst ('Attach transport sockets' /
  'Unattach transport sockets', lines 144-190) is not updated for the…

--

kcm: reject attaching sockets with a ULP, and TLS on KCM sockets

With this commit, kcm_attach() refuses TCP sockets that already have a
ULP, and it now refuses raw IPPROTO_TCP sockets too. Attached sockets get
an unregistered dummy kcm_ulp_ops, so tcp_set_ulp() refuses kTLS on them
afterwards. The marker stays until the socket is destroyed, and it pins
the kcm module.

Link: https://lore.kernel.org/netdev/3108e934-ec00-458b-8720-450554b35b55@linux.dev/
quoted
Holding a ULP also has visible side effects: getsockopt(TCP_ULP) and
inet_diag report "kcm" on a socket that was ever attached,
sk_has_decrypt_user() is true so PSP RX association is refused after the
attach, and adding an attached socket to a sockmap fails with -EINVAL
rather than -EBUSY.
[Severity: Medium]

Is the sockmap effect really limited to attached sockets?
SIOCKCMUNATTACH does not remove the kcm_ulp_ops marker, because

31 more quoted lines

Good catch, I'll list the sockmap case in the stable paragraph,
and I'll fix the earlier paragraph that said the -EINVAL only applied to
attached sockets.
quoted
listen() after a disconnect fails with -EINVAL, as
the dummy has no .clone, and rmmod kcm waits until such sockets are
destroyed.
[Severity: Low]

Is "waits" accurate here? Unloading does not block on a non-zero module
refcount:

kernel/module/main.c:try_stop_module() {
        /* If it's not unused, quit unless we're forcing. */
        if (try_release_module_ref(mod) != 0) {
                *forced = try_force_unload(flags);
                if (!(*forced))
                        return -EWOULDBLOCK;
        }
    ...
}

The reference taken by __module_get() in kcm_attach() is only dropped by
tcp_cleanup_ulp() when the TCP socket is destroyed. As long as any TCP
socket that was once attached still exists, rmmod kcm fails at once with
"Module kcm is in use". This holds even after every KCM socket is closed
and every socket has been unattached.
Right, rmmod fails immediately rather than waiting. I'll reword it to say
it fails while any once-attached socket exists.
quoted
diff --git a/net/kcm/kcmsock.c b/net/kcm/kcmsock.c
index a7f94d42036fb..1f8bcbb8b9c3b 100644
--- a/net/kcm/kcmsock.c
+++ b/net/kcm/kcmsock.c
@@ -30,11 +30,26 @@
[ ... ]
quoted
+/* Not registered, so TCP_ULP can't select it. It only holds
+ * icsk_ulp_ops so tcp_set_ulp() refuses other ULPs like kTLS. Like any
+ * ULP it stays until the socket is destroyed, pinning this module, and
+ * tcp_cleanup_ulp() releases it. Clearing it on a live socket would race
+ * with lockless readers such as do_tcp_getsockopt() and tcp_diag.
+ *
+ * No .clone: children copy icsk_ulp_ops without a module reference, so
+ * a socket that was once attached must not listen().
+ */
+static const struct tcp_ulp_ops kcm_ulp_ops = {
+     .name = "kcm",
+     .owner = THIS_MODULE,
+};
[Severity: Low]

This isn't a bug, but should Documentation/networking/kcm.rst be updated
as well? The "Attach transport sockets" section only says:

  - fd: file descriptor for TCP socket being attached

The "Unattach transport sockets" section says unattaching "is
straightforward".

After this patch, SIOCKCMATTACH refuses raw IPPROTO_TCP sockets and
sockets that already have a ULP, both with -EOPNOTSUPP.

The kcm marker also stays after SIOCKCMUNATTACH. For the rest of the
socket's life:

  - setting TCP_ULP fails with -EEXIST
  - getsockopt(TCP_ULP) and inet_diag report "kcm"

3 more quoted lines

Yes. The next version updates kcm.rst.
[ ... ]

--
Sashiko AI review · https://netdev-ai.bots.linux.dev/sashiko/#/patchset/20261102220310.12649-1-rjethwani%40purestorage.com
pw-bot: cr
Keyboard shortcuts
hback out one level
jnext message in thread / next series-diff position
kprevious message in thread / previous series-diff position
ldrill in
Escclose help / fold thread tree
?toggle this help