-----Original Message-----
From: Steven Price [mailto:steven.price@arm.com]
Sent: 21 May 2021 13:55
To: Shameerali Kolothum Thodi <redacted>;
linux-arm-kernel@lists.infradead.org; linux-acpi@vger.kernel.org;
iommu@lists.linux-foundation.org
Cc: Linuxarm <redacted>; lorenzo.pieralisi@arm.com;
joro@8bytes.org; robin.murphy@arm.com; wanghuiqiang
[off-list ref]; Guohanjun (Hanjun Guo)
[off-list ref]; Sami.Mujawar@arm.com; jon@solid-run.com;
eric.auger@redhat.com; yangyicong [off-list ref]
Subject: Re: [PATCH v4 0/8] ACPI/IORT: Support for IORT RMR node

On 13/05/2021 14:45, Shameer Kolothum wrote:

quoted

Hi,

v3 -->v4
-Included the SMMUv2 SMR bypass install changes suggested by
 Steve(patch #7)
-As per Robin's comments, RMR reserve implementation is now
 more generic  (patch #8) and dropped v3 patches 8 and 10.
-Rebase to 5.13-rc1

The whole series is available here,
https://github.com/hisilicon/kernel-dev/tree/private-v5.13-rc1-rmr-v4-ext

RFC v2 --> v3
 -Dropped RFC tag as the ACPICA header changes are now ready to be
  part of 5.13[0]. But this series still has a dependency on that patch.
 -Added IORT E.b related changes(node flags, _DSM function 5 checks for
  PCIe).
 -Changed RMR to stream id mapping from M:N to M:1 as per the spec and
  discussion here[1].
 -Last two patches add support for SMMUv2(Thanks to Jon Nettleton!)

Sanity tested on a HiSilicon D06. Further testing and feedback is greatly
appreciated.

With the updated SMMUv2 support this works fine on my Juno with EFIFB
(and corresponding patches to the firmware to expose the ACPI tables).
Feel free to add

Tested-by: Steven Price <steven.price@arm.com>

Thanks Steve. I am in the process of incorporating the comments from Joerg/Robin
to reuse the struct iommu_resv_region. I will post a v5 soon with that and a couple
of other minor fixes.

Thanks,
Shameer

Thanks,

Steve

quoted

https://github.com/hisilicon/kernel-dev/tree/private-v5.12-rc8-rmr-v3

Thanks,
Shameer

[0]

https://lore.kernel.org/linux-acpi/20210406213028.718796-22-erik.kaneda@i
ntel.com/

quoted

[1]

https://op-lists.linaro.org/pipermail/linaro-open-discussions/2021-April/00015
0.html

quoted

RFC v1 --> v2:
 - Added a generic interface for IOMMU drivers to retrieve all the
   RMR info associated with a given IOMMU.
 - SMMUv3 driver gets the RMR list during probe() and installs
   bypass STEs for all the SIDs in the RMR list. This is to keep
   the ongoing traffic alive(if any) during SMMUv3 reset. This is
   based on the suggestions received for v1 to take care of the
   EFI framebuffer use case. Only sanity tested for now.
 - During the probe/attach device, SMMUv3 driver reserves any
   RMR region associated with the device such that there is a unity
   mapping for them in SMMU.
---

From RFC v1:
-------------
The series adds support to IORT RMR nodes specified in IORT
Revision E -ARM DEN 0049E[0]. RMR nodes are used to describe memory
ranges that are used by endpoints and require a unity mapping
in SMMU.

We have faced issues with 3408iMR RAID controller cards which
fail to boot when SMMU is enabled. This is because these controllers
make use of host memory for various caching related purposes and when
SMMU is enabled the iMR firmware fails to access these memory regions
as there is no mapping for them. IORT RMR provides a way for UEFI to
describe and report these memory regions so that the kernel can make
a unity mapping for these in SMMU.

Tests:

With a UEFI, that reports the RMR for the dev,
....
[16F0h 5872   1]                         Type : 06
[16F1h 5873   2]                       Length : 007C
[16F3h 5875   1]                     Revision : 00
[1038h 0056   2]                     Reserved : 00000000
[1038h 0056   2]                   Identifier : 00000000
[16F8h 5880   4]                Mapping Count : 00000001
[16FCh 5884   4]               Mapping Offset : 00000040

[1700h 5888   4]    Number of RMR Descriptors : 00000002
[1704h 5892   4]        RMR Descriptor Offset : 00000018

[1708h 5896   8]          Base Address of RMR : 0000E6400000
[1710h 5904   8]                Length of RMR : 000000100000
[1718h 5912   4]                     Reserved : 00000000

[171Ch 5916   8]          Base Address of RMR : 0000000027B00000
[1724h 5924   8]                Length of RMR : 0000000000C00000
[172Ch 5932   4]                     Reserved : 00000000

[1730h 5936   4]                   Input base : 00000000
[1734h 5940   4]                     ID Count : 00000001
[1738h 5944   4]                  Output Base : 00000003
[173Ch 5948   4]             Output Reference : 00000064
[1740h 5952   4]        Flags (decoded below) : 00000001
                               Single Mapping : 1
...

Without the series the RAID controller initialization fails as
below,

...
[   12.631117] megaraid_sas 0000:03:00.0: FW supports sync

cache        : Yes

quoted

[   12.637360] megaraid_sas 0000:03:00.0: megasas_disable_intr_fusion is

called outbound_intr_mask:0x40000009

quoted

[   18.776377] megaraid_sas 0000:03:00.0: Init cmd return status FAILED

for SCSI host 0

quoted

[   23.019383] megaraid_sas 0000:03:00.0: Waiting for FW to come to

ready state

quoted

[  106.684281] megaraid_sas 0000:03:00.0: FW in FAULT state, Fault

code:0x10000 subcode:0x0 func:megasas_transition_to_ready

quoted

[  106.695186] megaraid_sas 0000:03:00.0: System Register set:
[  106.889787] megaraid_sas 0000:03:00.0: Failed to transition controller

to ready for scsi0.

quoted

[  106.910475] megaraid_sas 0000:03:00.0: Failed from megasas_init_fw

6407

quoted

estuary:/$

With the series, now the kernel has direct mapping for the dev as
below,

estuary:/$ cat /sys/kernel/iommu_groups/0/reserved_regions
0x0000000008000000 0x00000000080fffff msi
0x0000000027b00000 0x00000000286fffff direct
0x00000000e6400000 0x00000000e64fffff direct
estuary:/$

....
[   12.254318] megaraid_sas 0000:03:00.0: megasas_disable_intr_fusion is

called outbound_intr_mask:0x40000009

quoted

[   12.739089] megaraid_sas 0000:03:00.0: FW provided

supportMaxExtLDs: 0      max_lds: 32

quoted

[   12.746628] megaraid_sas 0000:03:00.0: controller type       :

iMR(0MB)

quoted

[   12.752694] megaraid_sas 0000:03:00.0: Online Controller Reset(OCR)  :

Enabled

quoted

[   12.759798] megaraid_sas 0000:03:00.0: Secure JBOD support   : Yes
[   12.765778] megaraid_sas 0000:03:00.0: NVMe passthru support : Yes
[   12.771931] megaraid_sas 0000:03:00.0: FW provided TM

TaskAbort/Reset timeou: 6 secs/60 secs

quoted

[   12.780503] megaraid_sas 0000:03:00.0: JBOD sequence map

support     : Yes

quoted

[   12.787000] megaraid_sas 0000:03:00.0: PCI Lane Margining

support    : No

quoted

[   12.819179] megaraid_sas 0000:03:00.0: NVME page size        :

(4096)

quoted

[   12.825672] megaraid_sas 0000:03:00.0: megasas_enable_intr_fusion is

called outbound_intr_mask:0x40000000

quoted

[   12.835199] megaraid_sas 0000:03:00.0: INIT adapter done
[   12.873932] megaraid_sas 0000:03:00.0: pci id                :

(0x1000)/(0x0017)/(0x19e5)/(0xd213)

quoted

[   12.881644] megaraid_sas 0000:03:00.0: unevenspan support    : no
[   12.887451] megaraid_sas 0000:03:00.0: firmware crash dump   : no
[   12.893344] megaraid_sas 0000:03:00.0: JBOD sequence map     :

enabled

quoted

RAID controller init is now success and can detect the drives
attached as well.

Jon Nettleton (1):
  iommu/arm-smmu: Get associated RMR info and install bypass SMR

Shameer Kolothum (7):
  ACPI/IORT: Add support for RMR node parsing
  iommu/dma: Introduce generic helper to retrieve RMR info
  ACPI/IORT: Add a helper to retrieve RMR memory regions
  iommu/arm-smmu-v3: Introduce strtab init helper
  iommu/arm-smmu-v3: Add bypass flag to arm_smmu_write_strtab_ent()
  iommu/arm-smmu-v3: Get associated RMR info and install bypass STE
  iommu/dma: Reserve any RMR regions associated with a dev

 drivers/acpi/arm64/iort.c                   | 144

+++++++++++++++++++-

quoted

 drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c |  69 +++++++---
 drivers/iommu/arm/arm-smmu/arm-smmu.c       |  64 +++++++++
 drivers/iommu/dma-iommu.c                   | 102 +++++++++++++-
 include/linux/acpi_iort.h                   |   7 +
 include/linux/dma-iommu.h                   |  10 ++
 include/linux/iommu.h                       |  19 +++
 7 files changed, 392 insertions(+), 23 deletions(-)

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel