On 7/2/26 4:49 PM, David Howells wrote:

Currently, when a CHALLENGE packet comes in, it's queued in an OOB queue on
the AF_RXRPC socket that generated one of the calls on that connection for
the application (which might be in userspace) to service.  The application
then picks up the CHALLENGE and requests a RESPONSE packet be generated,
allowing the app to include app-specific data in it if appropriate.  There
is, however, no actual limit on the capacity of the CHALLENGE queue, and
this could be abused remotely - and also getting the OOB mechanism right
has proven tricky.

Further, by analogy with other AFS codebases, it's not actually necessary
to generate the application data in response to the CHALLENGE.  The reason
I did this was to set the encryption on the app-data to be the same as that
specified in the CHALLENGE as the server must be able to handle that.
However, it's sufficient to use the encoding type set in the token that is
going to be sent to the server; presumably the kerberos server knows that
the fileserver can handle that type - otherwise why tell the client to use
it?

With this in mind, make the following changes to the rxrpc server:

 (1) Remove the OOB queuing stuff and retire the related CMSG values and
     sockopt.

 (2) Revert to making the connection event processor work item parse the
     CHALLENGE and generate the RESPONSE directly.

 (3) Add another (optional) parameter that is passed in when an rxrpc
     client call is created and ends up attached to the connection bundle
     and is a user-type key containing the application data.

 (4) RESPONSE generation looks at the bundle and if the app-data is there,
     it will include it (if the security class is YFS-RxGK; RxKAD ignores
     it).

 (5) The AFS filesystem driver will create an app-data key when it probes a
     fileserver and will attach it to the afs_server struct.  This is then
     picked up when a call is made to that server and passed to rxrpc.

 (6) Direct userspace users of AF_RXRPC can partake by creating a user-type
     key containing the app-data they want to use and passing its serial ID
     in a CMSG of type RXRPC_RESPONSE_APPDATA in the initial sendmsg() of a
     call.

To support this, I've made a change outside of afs and rxrc:

 (7) Add a refcount to the user-type key payload.  The problem is that the
     RESPONSE packet generator needs to look at the length of the app-data,
     allocate a buffer for the packet and then copy the app-data in - but
     the RCU read lock cannot be held across the allocation and the key
     might get updated.  To get around this, a ref is taken on the payload
     before the allocation and then put afterwards.

 (8) Since the DNS resolver makes use of the user-defined type's code, but
     allocates the payload itself, make it initialise the refcount.

Fixes: 5800b1cf3fd8 ("rxrpc: Allow CHALLENGEs to the passed to the app for a RESPONSE")
Link: https://sashiko.dev/#/patchset/20260624163819.3017002-1-dhowells%40redhat.com
Signed-off-by: David Howells <dhowells@redhat.com>
cc: Marc Dionne <marc.dionne@auristor.com>
cc: Jeffrey Altman <redacted>
cc: Eric Dumazet <edumazet@google.com>
cc: "David S. Miller" <davem@davemloft.net>
cc: Jakub Kicinski <kuba@kernel.org>
cc: Paolo Abeni <pabeni@redhat.com>
cc: Simon Horman <horms@kernel.org>
cc: Jarkko Sakkinen <jarkko@kernel.org>
cc: linux-afs@lists.infradead.org
cc: keyrings@vger.kernel.org
cc: stable@kernel.org
---
 fs/afs/cm_security.c         | 151 ++++++--------
 fs/afs/fs_probe.c            |   5 +
 fs/afs/internal.h            |  37 ++--
 fs/afs/main.c                |   1 -
 fs/afs/rxrpc.c               |  39 ++--
 fs/afs/server.c              |   2 +-
 include/keys/user-type.h     |   2 +
 include/net/af_rxrpc.h       |  20 +-
 include/trace/events/afs.h   |   1 +
 include/trace/events/rxrpc.h |   2 -
 include/uapi/linux/rxrpc.h   |   6 +-
 net/dns_resolver/dns_key.c   |   1 +
 net/rxrpc/Makefile           |   1 -
 net/rxrpc/af_rxrpc.c         |  49 +----
 net/rxrpc/ar-internal.h      |  22 +-
 net/rxrpc/call_object.c      |   4 +-
 net/rxrpc/conn_client.c      |   2 +
 net/rxrpc/conn_event.c       |  68 +-----
 net/rxrpc/key.c              |  36 ++++
 net/rxrpc/oob.c              | 387 -----------------------------------
 net/rxrpc/recvmsg.c          |  84 +-------
 net/rxrpc/rxgk.c             | 128 +++---------
 net/rxrpc/rxkad.c            |  27 ---
 net/rxrpc/sendmsg.c          |  15 ++
 net/rxrpc/server_key.c       |  40 ----
 security/keys/user_defined.c |  23 ++-
 26 files changed, 238 insertions(+), 915 deletions(-)
 delete mode 100644 net/rxrpc/oob.c

This is quite big and difficult to review, could you please someout
break it in smaller chunk?

/P