Re: [PATCH RFC net-next 2/3] crypto: caam/qi2 - skip isolated CPUs when setting up queue pairs

On Thu, Oct 08, 2026 at 10:49:57AM +0000, Josua Mayer wrote:

Hi Ioana,

Thank you for taking a look at my patch!

On 10/6/26 11:27, Ioana Ciornei wrote:

quoted

On Wed, Sep 30, 2026 at 06:54:59PM +0200, Josua Mayer wrote:

quoted

The driver looks up the affine DPIO and sets up one Rx/Tx queue pair for
each online CPU, up to the number of queues. Response notifications and
NAPI polling are handled on the DPIO object's assigned CPU. All online
may enqueue requests.

When isolcpus=managed_irq is used to reserve CPUs for latency-sensitive
applications, those CPUs are expected to be free of routine device
interrupts.

The DPIO driver assigns affine CPUs only from the HK_TYPE_MANAGED_IRQ
housekeeping mask, leaving isolated CPUs without an affine DPIO. This
defers caam qi2 probe indefinitely once DPIO setup reaches an isolated
CPU.

Restrict queue pairs, notifications and NAPI polling to the online CPUs
in the HK_TYPE_MANAGED_IRQ housekeeping mask, so that isolated CPUs are
skipped and don't defer probe. Isolated CPUs may still enqueue requests,
with responses handled on housekeeping CPUs.

Don't you need to also change the for_each_online_cpu() from
dpaa2_dpseci_setup() to take into account only the housekeeping
managed_irq cores?

I initially thought no, because of the "Allow all cores to enqueue"
comment in code.

quoted

Something like:

--- a/drivers/crypto/caam/caamalg_qi2.c
+++ b/drivers/crypto/caam/caamalg_qi2.c

@@ -5101,7 +5101,7 @@ static int __cold dpaa2_dpseci_setup(struct fsl_mc_device *ls_dev)
         }

         i = 0;
-       for_each_online_cpu(cpu) {
+       for_each_cpu_and(cpu, cpu_online_mask, hk_mask) {
                 u8 j;

                 j = i % priv->num_pairs;

Correct.

The loop is doing two things in one: assign tx queue to each cpu,
and assign rx queue to each cpu.

Only rx side is an issue for isolcpus. Do you have any idea why each cpu
got a tx queue assigned, and if this may be changed?


Here is a copy of the complete loop, with inline comments on the
control flow:

quoted

	i = 0;
	for_each_online_cpu(cpu) {
		u8 j;

		j = i % priv->num_pairs;

		ppriv = per_cpu_ptr(priv->ppriv, cpu);
		ppriv->req_fqid = priv->tx_queue_attr[j].fqid;

Assigns a tx queue to the cpu, one tx queue may be assigned to
multiple cores due to modulo above (if i >= num_pairs).

quoted

		/*
		 * Allow all cores to enqueue, while only some of them
		 * will take part in dequeuing.
		 */
		if (++i > priv->num_pairs)
			continue;

After this i <= num_pairs.

quoted

		ppriv->rsp_fqid = priv->rx_queue_attr[j].fqid;
		ppriv->prio = j;

Assigns a rx queue to the cpu.
This is the case which causes issues with isolcpus, and it should
be avoided for non-housekeeping cpus.

quoted

		dev_dbg(dev, "pair %d: rx queue %d, tx queue %d\n", j,
			priv->rx_queue_attr[j].fqid,
			priv->tx_queue_attr[j].fqid);

		ppriv->net_dev = alloc_netdev_dummy(0);
		if (!ppriv->net_dev) {
			err = -ENOMEM;
			goto err_alloc_netdev;
		}
		cpumask_set_cpu(cpu, priv->clean_mask);
		ppriv->net_dev->dev = *dev;

		netif_napi_add_tx_weight(ppriv->net_dev, &ppriv->napi,
					 dpaa2_dpseci_poll,
					 DPAA2_CAAM_NAPI_WEIGHT);
	}

dpaa2_caam_enqueue unconditionally accesses ppriv->dpio per-cpu
and ppriv->req_fqid, not gated by any checks, percpu (any cpu).

This causes two issues:
1. each cpu is still expected to have a dpio object
2. each cpu is still expected to have an assigned tx queue

I don't quite understand how we get into dpaa2_caam_enqueue,
I suppose it can legitimately happen on any cpu core whatsoever?

I don't have any experience in the crypto area but from the looks of it,
there is nothing limiting on what cpus crypto_aead_encrypt() could run on.

For example, one of crypto_aead_encrypt() callers is the macsec software
implementation. In that case, I would expect that the macsec's
.ndo_start_xmit() callback could be run on a non-housekeeping, which in
turn means that caam_qi2 needs to be prepared to run the enqueue() from
any cpu.

Ioana