Re: [PATCH 4/6] nfsd: allocate new session-based DRC slots on demand.
From: Jeff Layton <jlayton@kernel.org>
Date: 2024-12-06 01:04:17
On Fri, 2024-12-06 at 11:43 +1100, NeilBrown wrote:
quoted hunk ↗ jump to hunk
If a client ever uses the highest available slot for a given session, attempt to allocate more slots so there is room for the client to use them if wanted. GFP_NOWAIT is used so if there is not plenty of free memory, failure is expected - which is what we want. It also allows the allocation while holding a spinlock. Each time we increase the number of slots by 20% (rounded up). This allows fairly quick growth while avoiding excessive over-shoot. We would expect to stablise with around 10% more slots available than the client actually uses. Signed-off-by: NeilBrown <redacted> --- fs/nfsd/nfs4state.c | 40 +++++++++++++++++++++++++++++++++++----- 1 file changed, 35 insertions(+), 5 deletions(-)diff --git a/fs/nfsd/nfs4state.c b/fs/nfsd/nfs4state.c index 67dfc699e411..ec4468ebbd40 100644 --- a/fs/nfsd/nfs4state.c +++ b/fs/nfsd/nfs4state.c@@ -4235,11 +4235,6 @@ nfsd4_sequence(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate, slot = xa_load(&session->se_slots, seq->slotid); dprintk("%s: slotid %d\n", __func__, seq->slotid); - /* We do not negotiate the number of slots yet, so set the - * maxslots to the session maxreqs which is used to encode - * sr_highest_slotid and the sr_target_slot id to maxslots */ - seq->maxslots = session->se_fchannel.maxreqs; - trace_nfsd_slot_seqid_sequence(clp, seq, slot); status = check_slot_seqid(seq->seqid, slot->sl_seqid, slot->sl_flags & NFSD4_SLOT_INUSE);@@ -4289,6 +4284,41 @@ nfsd4_sequence(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate, cstate->session = session; cstate->clp = clp; + /* + * If the client ever uses the highest available slot, + * gently try to allocate another 20%. This allows + * fairly quick growth without grossly over-shooting what + * the client might use. + */
20% seems like a reasonable place to start, but I do wonder if this might need to be tunable under some workloads. Oh well, we can cross that bridge if/when someone complains.
+ if (seq->slotid == session->se_fchannel.maxreqs - 1 &&
+ session->se_fchannel.maxreqs < NFSD_MAX_SLOTS_PER_SESSION) {
+ int s = session->se_fchannel.maxreqs;
+ int cnt = DIV_ROUND_UP(s, 5);
+
+ do {
+ /*
+ * GFP_NOWAIT is a low-priority non-blocking
+ * allocation which can be used under
+ * client_lock and only succeeds if there is
+ * plenty of memory.
+ * Use GFP_ATOMIC which is higher priority for
+ * xa_store() so we are less likely to waste the
+ * effort of the first allocation.
+ */I don't know here. Why not just use GFP_NOWAIT for the xa_store too? If we're so memory constrained that that fails, we're probably better off releasing the slot.
+ slot = kzalloc(slot_bytes(&session->se_fchannel),
+ GFP_NOWAIT);
+ if (slot &&
+ !xa_is_err(xa_store(&session->se_slots, s, slot,
+ GFP_ATOMIC | __GFP_NOWARN))) {
+ s += 1;
+ session->se_fchannel.maxreqs = s;
+ } else {
+ kfree(slot);
+ }
+ } while (slot && --cnt > 0);
+ }
+ seq->maxslots = session->se_fchannel.maxreqs;
+
out:
switch (clp->cl_cb_state) {
case NFSD4_CB_DOWN:-- Jeff Layton [off-list ref]