-----Original Message-----
From: Jason Gunthorpe <jgg@nvidia.com>
Sent: Wednesday, September 4, 2024 4:00 PM
To: Shameerali Kolothum Thodi <redacted>
Cc: acpica-devel@lists.linux.dev; Guohanjun (Hanjun Guo)
[off-list ref]; iommu@lists.linux.dev; Joerg Roedel
[off-list ref]; Kevin Tian [off-list ref]; kvm@vger.kernel.org;
Len Brown [off-list ref]; linux-acpi@vger.kernel.org; linux-arm-
kernel@lists.infradead.org; Lorenzo Pieralisi [off-list ref]; Rafael J.
Wysocki [off-list ref]; Robert Moore [off-list ref]; Robin
Murphy [off-list ref]; Sudeep Holla [off-list ref];
Will Deacon [off-list ref]; Alex Williamson
[off-list ref]; Eric Auger [off-list ref]; Jean-
Philippe Brucker [off-list ref]; Moritz Fischer [off-list ref];
Michael Shavit [off-list ref]; Nicolin Chen [off-list ref];
patches@lists.linux.dev; Mostafa Saleh [off-list ref]
Subject: Re: [PATCH v2 2/8] iommu/arm-smmu-v3: Use S2FWB when available

On Wed, Sep 04, 2024 at 02:20:36PM +0000, Shameerali Kolothum Thodi wrote:

quoted

This should be added to arm_64_lpae_alloc_pgtable_s2(), not here.

Woops! Yes:

-       /* The NS quirk doesn't apply at stage 2 */
-       if (cfg->quirks)
+       if (cfg->quirks & ~(IO_PGTABLE_QUIRK_ARM_S2FWB))
                return NULL;

quoted

With the above fixed, I was able to assign a n/w VF dev to a Guest on
a test hardware that supports S2FWB.

Okay great

quoted

However host kernel has this WARN message:
[ 1546.165105] WARNING: CPU: 5 PID: 7047 at
drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c:1086
arm_smmu_entry_qword_diff+0x124/0x138
....

Yes, my dumb mistake again, thanks for testing

@@ -1009,7 +1009,8 @@ void arm_smmu_get_ste_used(const __le64 *ent,

__le64 *used_bits)
        /* S2 translates */
        if (cfg & BIT(1)) {
                used_bits[1] |=
-                       cpu_to_le64(STRTAB_STE_1_EATS | STRTAB_STE_1_SHCFG);
+                       cpu_to_le64(STRTAB_STE_1_S2FWB | STRTAB_STE_1_EATS |
+                                   STRTAB_STE_1_SHCFG);

quoted

root@localhost:/# ping 150.0.124.42
PING 150.0.124.42 (150.0.124.42): 56 data bytes
64 bytes from 150.0.124.42: seq=0 ttl=64 time=47.648 ms

So DMA is not totally broken if a packet flowed.

quoted

[ 1395.958630] hns3 0000:c2:00.0 eth1: NETDEV WATCHDOG: CPU: 1:
transmit queue 10 timed out 5260 ms

Timeout? Maybe interrupts are not working? Does /proc/interrupts suggest
that? That would point at the ITS mapping