On Fri, 17 May 2024 19:55:21 +0530, Nilay Shroff wrote:

On NUMA aware system, we make a numa-node online only if that node is
attached to cpu/memory. However it's possible that we have some PCI/IO
device affinitized to a numa-node which is not currently online. In such
case we set the numa-node id of the corresponding PCI device to -1
(NUMA_NO_NODE). Not assigning the correct numa-node id to PCI device may
impact the performance of such device. For instance, we have a multi
controller NVMe disk where each controller of the disk is attached to
different PHB (PCI host bridge). Each of these PHBs has numa-node id
assigned during PCI enumeration. During PCI enumeration if we find that
the numa-node is not online then we set the numa-node id of the PHB to -1.
If we create shared namespace and attach to multi controller NVMe disk
then that namespace could be accessed through each controller and as each
controller is connected to different PHBs, it's possible to access the
same namespace using multiple PCI channel. While sending IO to a shared
namespace, NVMe driver would calculate the optimal IO path using numa-node
distance. However if the numa-node id is not correctly assigned to NVMe
PCIe controller then it's possible that driver would calculate incorrect
NUMA distance and hence select the non-optimal path for sending IO. If
this happens then we could potentially observe the degraded IO performance.

[...]

Applied to powerpc/next.

[1/1] powerpc/numa: Online a node if PHB is attached.
      https://git.kernel.org/powerpc/c/11981816e3614156a1fe14a1e8e77094ea46c7d5

cheers