On Fri, Aug 16, 2024 at 02:53:31PM +0100, Will Deacon wrote:

On Tue, Aug 06, 2024 at 07:11:47PM -0700, Nicolin Chen wrote:

quoted

There is an existing arm_smmu_cmdq_build_sync_cmd() so the driver should
call it at all places other than going through arm_smmu_cmdq_build_cmd()
separately. This helps the following patch that adds a CS_NONE option.

Note that this changes the type of CMD_SYNC in __arm_smmu_cmdq_skip_err,
in ARM_SMMU_OPT_MSIPOLL=true case, from previously a non-MSI one to now
an MSI one that is proven to still work using a hacking test:
  nvme: Adding to iommu group 10
  nvme: --------hacking-----------
  arm-smmu-v3: unexpected global error reported (0x00000001),
               this could be serious
  arm-smmu-v3: CMDQ error (cons 0x01000022): Illegal command
  arm-smmu-v3: skipping command in error state:
  arm-smmu-v3:  0x0000000000000000
  arm-smmu-v3:  0x0000000000000000
  nvme: -------recovered----------
  nvme nvme0: 72/0/0 default/read/poll queues
   nvme0n1: p1 p2

Hmm, I'm still a little wary of this. The only reason we emit a CMD_SYNC
in __arm_smmu_cmdq_skip_err() is because the SMMU doesn't have a NOP
command. So I'd really like the SYNC to have as little functionality
as possible in this case.

I think we could either propapate the 'cs' field down to
arm_smmu_cmdq_build_cmd() or just open-code the command-creation in
__arm_smmu_cmdq_skip_err().

OK. Let's go with your approach then:
+       if (arm_smmu_cmdq_needs_busy_polling(smmu, cmdq))
+               u64p_replace_bits(cmd, CMDQ_SYNC_0_CS_NONE, CMDQ_SYNC_0_CS);

Thanks
Nicolin