[PATCH 1/1] vfio-pci/nvlink2: Allow fallback to ibm,mmio-atsd[0]

Subsystems: the rest, vfio driver

STALE2380d

4 messages, 2 authors, 2020-02-07 · open the first message on its own page

[PATCH 1/1] vfio-pci/nvlink2: Allow fallback to ibm,mmio-atsd[0]

From: Sam Bobroff <hidden>
Date: 2020-02-06 03:37:02 

Older versions of skiboot only provide a single value in the device
tree property "ibm,mmio-atsd", even when multiple Address Translation
Shoot Down (ATSD) registers are present. This prevents NVLink2 devices
(other than the first) from being used with vfio-pci because vfio-pci
expects to be able to assign a dedicated ATSD register to each NVLink2
device.

However, ATSD registers can be shared among devices. This change
allows vfio-pci to fall back to sharing the register at index 0 if
necessary.

Signed-off-by: Sam Bobroff <redacted>
---
 drivers/vfio/pci/vfio_pci_nvlink2.c | 13 +++++++++++--
 1 file changed, 11 insertions(+), 2 deletions(-)

diff --git a/drivers/vfio/pci/vfio_pci_nvlink2.c b/drivers/vfio/pci/vfio_pci_nvlink2.c
index f2983f0f84be..851ba673882b 100644
--- a/drivers/vfio/pci/vfio_pci_nvlink2.c
+++ b/drivers/vfio/pci/vfio_pci_nvlink2.c
@@ -420,8 +420,17 @@ int vfio_pci_ibm_npu2_init(struct vfio_pci_device *vdev)
 
 	if (of_property_read_u64_index(hose->dn, "ibm,mmio-atsd", nvlink_index,
 			&mmio_atsd)) {
-		dev_warn(&vdev->pdev->dev, "No available ATSD found\n");
-		mmio_atsd = 0;
+		dev_warn(&vdev->pdev->dev,
+			 "No ibm,mmio-atsd[%d] found: trying ibm,mmio-atsd[0]\n",
+			 nvlink_index);
+		if (of_property_read_u64_index(hose->dn, "ibm,mmio-atsd", 0,
+				&mmio_atsd)) {
+			dev_warn(&vdev->pdev->dev, "No available ATSD found\n");
+			mmio_atsd = 0;
+		} else {
+			dev_warn(&vdev->pdev->dev,
+				 "Using fallback ibm,mmio-atsd[0] for ATSD.\n");
+		}
 	}
 
 	if (of_property_read_u64(npu_node, "ibm,device-tgt-addr", &tgt)) {
-- 
2.22.0.216.g00a2a96fc9

Re: [PATCH 1/1] vfio-pci/nvlink2: Allow fallback to ibm,mmio-atsd[0]

From: Alexey Kardashevskiy <hidden>
Date: 2020-02-06 04:24:57 


On 06/02/2020 14:17, Sam Bobroff wrote:
quoted hunk
Older versions of skiboot only provide a single value in the device	
tree property "ibm,mmio-atsd", even when multiple Address Translation
Shoot Down (ATSD) registers are present. This prevents NVLink2 devices
(other than the first) from being used with vfio-pci because vfio-pci
expects to be able to assign a dedicated ATSD register to each NVLink2
device.

However, ATSD registers can be shared among devices. This change
allows vfio-pci to fall back to sharing the register at index 0 if
necessary.

Signed-off-by: Sam Bobroff <redacted>
---
 drivers/vfio/pci/vfio_pci_nvlink2.c | 13 +++++++++++--
 1 file changed, 11 insertions(+), 2 deletions(-)

diff --git a/drivers/vfio/pci/vfio_pci_nvlink2.c b/drivers/vfio/pci/vfio_pci_nvlink2.c
index f2983f0f84be..851ba673882b 100644
--- a/drivers/vfio/pci/vfio_pci_nvlink2.c
+++ b/drivers/vfio/pci/vfio_pci_nvlink2.c
@@ -420,8 +420,17 @@ int vfio_pci_ibm_npu2_init(struct vfio_pci_device *vdev)
 
 	if (of_property_read_u64_index(hose->dn, "ibm,mmio-atsd", nvlink_index,
 			&mmio_atsd)) {
-		dev_warn(&vdev->pdev->dev, "No available ATSD found\n");
-		mmio_atsd = 0;
+		dev_warn(&vdev->pdev->dev,
+			 "No ibm,mmio-atsd[%d] found: trying ibm,mmio-atsd[0]\n",
+			 nvlink_index);

We do not really need this warning (nvlink_index doesn't matter that
much, we can work out from the device tree what happened), warnings
below are enough (if you really want, you can print nvlink_index there).

Either way,

Reviewed-by: Alexey Kardashevskiy <redacted>




+		if (of_property_read_u64_index(hose->dn, "ibm,mmio-atsd", 0,
+				&mmio_atsd)) {
+			dev_warn(&vdev->pdev->dev, "No available ATSD found\n");
+			mmio_atsd = 0;
+		} else {
+			dev_warn(&vdev->pdev->dev,
+				 "Using fallback ibm,mmio-atsd[0] for ATSD.\n");
+		}
 	}
 
 	if (of_property_read_u64(npu_node, "ibm,device-tgt-addr", &tgt)) {
-- 
Alexey

Re: [PATCH 1/1] vfio-pci/nvlink2: Allow fallback to ibm,mmio-atsd[0]

From: Sam Bobroff <hidden>
Date: 2020-02-07 02:41:06 

On Thu, Feb 06, 2020 at 03:23:03PM +1100, Alexey Kardashevskiy wrote:

On 06/02/2020 14:17, Sam Bobroff wrote:
quoted
Older versions of skiboot only provide a single value in the device	
tree property "ibm,mmio-atsd", even when multiple Address Translation
Shoot Down (ATSD) registers are present. This prevents NVLink2 devices
(other than the first) from being used with vfio-pci because vfio-pci
expects to be able to assign a dedicated ATSD register to each NVLink2
device.

However, ATSD registers can be shared among devices. This change
allows vfio-pci to fall back to sharing the register at index 0 if
necessary.

Signed-off-by: Sam Bobroff <redacted>
---
 drivers/vfio/pci/vfio_pci_nvlink2.c | 13 +++++++++++--
 1 file changed, 11 insertions(+), 2 deletions(-)

diff --git a/drivers/vfio/pci/vfio_pci_nvlink2.c b/drivers/vfio/pci/vfio_pci_nvlink2.c
index f2983f0f84be..851ba673882b 100644
--- a/drivers/vfio/pci/vfio_pci_nvlink2.c
+++ b/drivers/vfio/pci/vfio_pci_nvlink2.c
@@ -420,8 +420,17 @@ int vfio_pci_ibm_npu2_init(struct vfio_pci_device *vdev)
 
 	if (of_property_read_u64_index(hose->dn, "ibm,mmio-atsd", nvlink_index,
 			&mmio_atsd)) {
-		dev_warn(&vdev->pdev->dev, "No available ATSD found\n");
-		mmio_atsd = 0;
+		dev_warn(&vdev->pdev->dev,
+			 "No ibm,mmio-atsd[%d] found: trying ibm,mmio-atsd[0]\n",
+			 nvlink_index);

We do not really need this warning (nvlink_index doesn't matter that
much, we can work out from the device tree what happened), warnings
below are enough (if you really want, you can print nvlink_index there).

Either way,

Reviewed-by: Alexey Kardashevskiy <redacted>
Thanks,

I'll change it if there's some reason to do another version but
otherwise leave it as is.

Sam.
quoted
+		if (of_property_read_u64_index(hose->dn, "ibm,mmio-atsd", 0,
+				&mmio_atsd)) {
+			dev_warn(&vdev->pdev->dev, "No available ATSD found\n");
+			mmio_atsd = 0;
+		} else {
+			dev_warn(&vdev->pdev->dev,
+				 "Using fallback ibm,mmio-atsd[0] for ATSD.\n");
+		}
 	}
 
 	if (of_property_read_u64(npu_node, "ibm,device-tgt-addr", &tgt)) {
-- 
Alexey

Re: [PATCH 1/1] vfio-pci/nvlink2: Allow fallback to ibm,mmio-atsd[0]

From: Sam Bobroff <hidden>
Date: 2020-02-07 04:44:54 

On Fri, Feb 07, 2020 at 01:39:14PM +1100, Sam Bobroff wrote:
On Thu, Feb 06, 2020 at 03:23:03PM +1100, Alexey Kardashevskiy wrote:
quoted

On 06/02/2020 14:17, Sam Bobroff wrote:
quoted
Older versions of skiboot only provide a single value in the device	
tree property "ibm,mmio-atsd", even when multiple Address Translation
Shoot Down (ATSD) registers are present. This prevents NVLink2 devices
(other than the first) from being used with vfio-pci because vfio-pci
expects to be able to assign a dedicated ATSD register to each NVLink2
device.

However, ATSD registers can be shared among devices. This change
allows vfio-pci to fall back to sharing the register at index 0 if
necessary.

Signed-off-by: Sam Bobroff <redacted>
---
 drivers/vfio/pci/vfio_pci_nvlink2.c | 13 +++++++++++--
 1 file changed, 11 insertions(+), 2 deletions(-)

diff --git a/drivers/vfio/pci/vfio_pci_nvlink2.c b/drivers/vfio/pci/vfio_pci_nvlink2.c
index f2983f0f84be..851ba673882b 100644
--- a/drivers/vfio/pci/vfio_pci_nvlink2.c
+++ b/drivers/vfio/pci/vfio_pci_nvlink2.c
@@ -420,8 +420,17 @@ int vfio_pci_ibm_npu2_init(struct vfio_pci_device *vdev)
 
 	if (of_property_read_u64_index(hose->dn, "ibm,mmio-atsd", nvlink_index,
 			&mmio_atsd)) {
-		dev_warn(&vdev->pdev->dev, "No available ATSD found\n");
-		mmio_atsd = 0;
+		dev_warn(&vdev->pdev->dev,
+			 "No ibm,mmio-atsd[%d] found: trying ibm,mmio-atsd[0]\n",
+			 nvlink_index);

We do not really need this warning (nvlink_index doesn't matter that
much, we can work out from the device tree what happened), warnings
below are enough (if you really want, you can print nvlink_index there).

Either way,

Reviewed-by: Alexey Kardashevskiy <redacted>
Thanks,

I'll change it if there's some reason to do another version but
otherwise leave it as is.

Sam.
Oh! I almost forgot: maybe this should carry a fixes tag?
Fixes: 7f92891778df ("vfio_pci: Add NVIDIA GV100GL [Tesla V100 SXM2] subdriver")
quoted
quoted
+		if (of_property_read_u64_index(hose->dn, "ibm,mmio-atsd", 0,
+				&mmio_atsd)) {
+			dev_warn(&vdev->pdev->dev, "No available ATSD found\n");
+			mmio_atsd = 0;
+		} else {
+			dev_warn(&vdev->pdev->dev,
+				 "Using fallback ibm,mmio-atsd[0] for ATSD.\n");
+		}
 	}
 
 	if (of_property_read_u64(npu_node, "ibm,device-tgt-addr", &tgt)) {
-- 
Alexey




    
  

  
    

      

        Keyboard shortcuts
      

      

        
          
hback out one level

          
jnext message in thread

          
kprevious message in thread

          
ldrill in

          
Escclose help / fold thread tree

          
?toggle this help