-----Original Message-----
From: Nicolin Chen <redacted>
Sent: Wednesday, August 28, 2024 6:15 PM
To: Shameerali Kolothum Thodi <redacted>
Cc: Jason Gunthorpe <jgg@nvidia.com>; acpica-devel@lists.linux.dev;
Guohanjun (Hanjun Guo) [off-list ref];
iommu@lists.linux.dev; Joerg Roedel [off-list ref]; Kevin Tian
[off-list ref]; kvm@vger.kernel.org; Len Brown
[off-list ref]; linux-acpi@vger.kernel.org; linux-arm-
kernel@lists.infradead.org; Lorenzo Pieralisi [off-list ref]; Rafael J.
Wysocki [off-list ref]; Robert Moore [off-list ref];
Robin Murphy [off-list ref]; Sudeep Holla
[off-list ref]; Will Deacon [off-list ref]; Alex Williamson
[off-list ref]; Eric Auger [off-list ref]; Jean-
Philippe Brucker [off-list ref]; Moritz Fischer
[off-list ref]; Michael Shavit [off-list ref];
patches@lists.linux.dev; Mostafa Saleh [off-list ref]
Subject: Re: [PATCH v2 0/8] Initial support for SMMUv3 nested translation

Hi Shameer,

On Wed, Aug 28, 2024 at 04:31:36PM +0000, Shameerali Kolothum Thodi
wrote:

quoted

Hi Nicolin,

quoted

-----Original Message-----
From: Nicolin Chen <redacted>
Sent: Tuesday, August 27, 2024 10:31 PM
To: Jason Gunthorpe <jgg@nvidia.com>
Cc: acpica-devel@lists.linux.dev; Guohanjun (Hanjun Guo)
[off-list ref]; iommu@lists.linux.dev; Joerg Roedel
[off-list ref]; Kevin Tian [off-list ref];
kvm@vger.kernel.org; Len Brown [off-list ref]; linux-
acpi@vger.kernel.org; linux-arm-kernel@lists.infradead.org; Lorenzo
Pieralisi [off-list ref]; Rafael J. Wysocki
[off-list ref]; Robert Moore [off-list ref]; Robin
Murphy [off-list ref]; Sudeep Holla

[off-list ref];

quoted

quoted

Will Deacon [off-list ref]; Alex Williamson
[off-list ref]; Eric Auger [off-list ref];
Jean-Philippe Brucker <jean- philippe@linaro.org>; Moritz Fischer
[off-list ref]; Michael Shavit [off-list ref];
patches@lists.linux.dev; Shameerali Kolothum Thodi
[off-list ref]; Mostafa Saleh
[off-list ref]
Subject: Re: [PATCH v2 0/8] Initial support for SMMUv3 nested
translation

quoted

As mentioned above, the VIOMMU series would be required to test the
entire nesting feature, which now has a v2 rebasing on this series.
I tested it with a paring QEMU branch. Please refer to:
https://lore.kernel.org/linux-
iommu/cover.1724776335.git.nicolinc@nvidia.com/

Thanks for this. I haven't gone through the viommu and its Qemu branch
yet.  The way we present nested-smmuv3/iommufd to the Qemu seems to
have changed  with the above Qemu branch(multiple nested SMMUs).
The old Qemu command line for nested setup doesn't work anymore.

Could you please share an example Qemu command line  to verify this
series(Sorry, if I missed it in the links/git).

My bad. I updated those two "for_iommufd_" QEMU branches with a
README commit on top of each for the reference command.

Thanks. I did give it a go and this is my command line based on above,

./qemu-system-aarch64-nicolin-viommu -object iommufd,id=iommufd0 \
-machine hmat=on \
-machine virt,accel=kvm,gic-version=3,iommu=nested-smmuv3,ras=on \
-cpu host -smp cpus=61 -m size=16G,slots=4,maxmem=256G -nographic \
-object memory-backend-ram,size=8G,id=m0 \
-object memory-backend-ram,size=8G,id=m1 \
-numa node,memdev=m0,cpus=0-60,nodeid=0  -numa node,memdev=m1,nodeid=1 \
-device vfio-pci-nohotplug,host=0000:75:00.1,iommufd=iommufd0 \
-bios QEMU_EFI.fd \
-drive if=none,file=ubuntu-18.04-old.img,id=fs \
-device virtio-blk-device,drive=fs \
-kernel Image \
-append "rdinit=init console=ttyAMA0 root=/dev/vda rw earlycon=pl011,0x9000000 kpti=off" \
-nographic

But it fails to boot very early:

root@ubuntu:/home/shameer/qemu-test# ./qemu_run-simple-iommufd-nicolin-2
qemu-system-aarch64-nicolin-viommu: Illegal numa node 2
 
Any idea what am I missing? Do you any special config enabled while building Qemu?

Thanks,
Shameer