[PATCH-RESEND] cxl: Disable prefault_mode in Radix mode

Subsystems: char and misc drivers, the rest

6 messages, 5 authors, 2018-06-01 · open the first message on its own page

[PATCH-RESEND] cxl: Disable prefault_mode in Radix mode

From: Vaibhav Jain <hidden>
Date: 2018-05-18 09:42:45 

From: Vaibhav Jain <redacted>

Currently we see a kernel-oops reported on Power-9 while attaching a
context to an AFU, with radix-mode and sysfs attr 'prefault_mode' set
to anything other than 'none'. The backtrace of the oops is of this
form:

Unable to handle kernel paging request for data at address 0x00000080
Faulting instruction address: 0xc00800000bcf3b20
cpu 0x1: Vector: 300 (Data Access) at [c00000037f003800]
    pc: c00800000bcf3b20: cxl_load_segment+0x178/0x290 [cxl]
    lr: c00800000bcf39f0: cxl_load_segment+0x48/0x290 [cxl]
    sp: c00000037f003a80
   msr: 9000000000009033
   dar: 80
 dsisr: 40000000
  current = 0xc00000037f280000
  paca    = 0xc0000003ffffe600   softe: 3        irq_happened: 0x01
    pid   = 3529, comm = afp_no_int
<snip>
[c00000037f003af0] c00800000bcf4424 cxl_prefault+0xfc/0x248 [cxl]
[c00000037f003b50] c00800000bcf8a40 process_element_entry_psl9+0xd8/0x1a0 [cxl]
[c00000037f003b90] c00800000bcf944c cxl_attach_dedicated_process_psl9+0x44/0x130 [cxl]
[c00000037f003bd0] c00800000bcf5448 native_attach_process+0xc0/0x130 [cxl]
[c00000037f003c50] c00800000bcf16cc afu_ioctl+0x3f4/0x5e0 [cxl]
[c00000037f003d00] c00000000039d98c do_vfs_ioctl+0xdc/0x890
[c00000037f003da0] c00000000039e1a8 ksys_ioctl+0x68/0xf0
[c00000037f003df0] c00000000039e270 sys_ioctl+0x40/0xa0
[c00000037f003e30] c00000000000b320 system_call+0x58/0x6c
--- Exception: c01 (System Call) at 0000000010053bb0

The issue is caused as on Power-8 the AFU attr 'prefault_mode' was
used to improve initial storage fault performance by prefaulting
process segments. However on Power-9 with radix mode we don't have
Storage-Segments that we can prefault. Also prefaulting process Pages
will be too costly and fine-grained.

Hence, since the prefaulting mechanism doesn't makes sense of
radix-mode, this patch updates prefault_mode_store() to not allow any
other value apart from CXL_PREFAULT_NONE when radix mode is enabled.

Cc: <redacted>
Fixes: f24be42aab37 ("cxl: Add psl9 specific code")
Signed-off-by: Vaibhav Jain <redacted>
---
Change-log:

Resend  ->  Updated the commit description to add more info on the
	    issue seen [Andrew]
---
 Documentation/ABI/testing/sysfs-class-cxl |  4 +++-
 drivers/misc/cxl/sysfs.c                  | 16 ++++++++++++----
 2 files changed, 15 insertions(+), 5 deletions(-)

diff --git a/Documentation/ABI/testing/sysfs-class-cxl b/Documentation/ABI/testing/sysfs-class-cxl
index 640f65e79ef1..267920a1874b 100644
--- a/Documentation/ABI/testing/sysfs-class-cxl
+++ b/Documentation/ABI/testing/sysfs-class-cxl
@@ -69,7 +69,9 @@ Date:           September 2014
 Contact:        linuxppc-dev@lists.ozlabs.org
 Description:    read/write
                 Set the mode for prefaulting in segments into the segment table
-                when performing the START_WORK ioctl. Possible values:
+                when performing the START_WORK ioctl. Only applicable when
+                running under hashed page table mmu.
+                Possible values:
                         none: No prefaulting (default)
                         work_element_descriptor: Treat the work element
                                  descriptor as an effective address and
diff --git a/drivers/misc/cxl/sysfs.c b/drivers/misc/cxl/sysfs.c
index 4b5a4c5d3c01..629e2e156412 100644
--- a/drivers/misc/cxl/sysfs.c
+++ b/drivers/misc/cxl/sysfs.c
@@ -353,12 +353,20 @@ static ssize_t prefault_mode_store(struct device *device,
 	struct cxl_afu *afu = to_cxl_afu(device);
 	enum prefault_modes mode = -1;
 
-	if (!strncmp(buf, "work_element_descriptor", 23))
-		mode = CXL_PREFAULT_WED;
-	if (!strncmp(buf, "all", 3))
-		mode = CXL_PREFAULT_ALL;
 	if (!strncmp(buf, "none", 4))
 		mode = CXL_PREFAULT_NONE;
+	else {
+		if (!radix_enabled()) {
+
+			/* only allowed when not in radix mode */
+			if (!strncmp(buf, "work_element_descriptor", 23))
+				mode = CXL_PREFAULT_WED;
+			if (!strncmp(buf, "all", 3))
+				mode = CXL_PREFAULT_ALL;
+		} else {
+			dev_err(device, "Cannot prefault with radix enabled\n");
+		}
+	}
 
 	if (mode == -1)
 		return -EINVAL;
-- 
2.17.0

Re: [PATCH-RESEND] cxl: Disable prefault_mode in Radix mode

From: Frederic Barrat <hidden>
Date: 2018-05-18 12:43:17 


Le 18/05/2018 à 11:42, Vaibhav Jain a écrit :
quoted hunk
From: Vaibhav Jain <redacted>

Currently we see a kernel-oops reported on Power-9 while attaching a
context to an AFU, with radix-mode and sysfs attr 'prefault_mode' set
to anything other than 'none'. The backtrace of the oops is of this
form:

Unable to handle kernel paging request for data at address 0x00000080
Faulting instruction address: 0xc00800000bcf3b20
cpu 0x1: Vector: 300 (Data Access) at [c00000037f003800]
     pc: c00800000bcf3b20: cxl_load_segment+0x178/0x290 [cxl]
     lr: c00800000bcf39f0: cxl_load_segment+0x48/0x290 [cxl]
     sp: c00000037f003a80
    msr: 9000000000009033
    dar: 80
  dsisr: 40000000
   current = 0xc00000037f280000
   paca    = 0xc0000003ffffe600   softe: 3        irq_happened: 0x01
     pid   = 3529, comm = afp_no_int
<snip>
[c00000037f003af0] c00800000bcf4424 cxl_prefault+0xfc/0x248 [cxl]
[c00000037f003b50] c00800000bcf8a40 process_element_entry_psl9+0xd8/0x1a0 [cxl]
[c00000037f003b90] c00800000bcf944c cxl_attach_dedicated_process_psl9+0x44/0x130 [cxl]
[c00000037f003bd0] c00800000bcf5448 native_attach_process+0xc0/0x130 [cxl]
[c00000037f003c50] c00800000bcf16cc afu_ioctl+0x3f4/0x5e0 [cxl]
[c00000037f003d00] c00000000039d98c do_vfs_ioctl+0xdc/0x890
[c00000037f003da0] c00000000039e1a8 ksys_ioctl+0x68/0xf0
[c00000037f003df0] c00000000039e270 sys_ioctl+0x40/0xa0
[c00000037f003e30] c00000000000b320 system_call+0x58/0x6c
--- Exception: c01 (System Call) at 0000000010053bb0

The issue is caused as on Power-8 the AFU attr 'prefault_mode' was
used to improve initial storage fault performance by prefaulting
process segments. However on Power-9 with radix mode we don't have
Storage-Segments that we can prefault. Also prefaulting process Pages
will be too costly and fine-grained.

Hence, since the prefaulting mechanism doesn't makes sense of
radix-mode, this patch updates prefault_mode_store() to not allow any
other value apart from CXL_PREFAULT_NONE when radix mode is enabled.

Cc: <redacted>
Fixes: f24be42aab37 ("cxl: Add psl9 specific code")
Signed-off-by: Vaibhav Jain <redacted>
---
Thanks!
Acked-by: Frederic Barrat <redacted>
quoted hunk
Change-log:

Resend  ->  Updated the commit description to add more info on the
	    issue seen [Andrew]
---
  Documentation/ABI/testing/sysfs-class-cxl |  4 +++-
  drivers/misc/cxl/sysfs.c                  | 16 ++++++++++++----
  2 files changed, 15 insertions(+), 5 deletions(-)

diff --git a/Documentation/ABI/testing/sysfs-class-cxl b/Documentation/ABI/testing/sysfs-class-cxl
index 640f65e79ef1..267920a1874b 100644
--- a/Documentation/ABI/testing/sysfs-class-cxl
+++ b/Documentation/ABI/testing/sysfs-class-cxl
@@ -69,7 +69,9 @@ Date:           September 2014
  Contact:        linuxppc-dev@lists.ozlabs.org
  Description:    read/write
                  Set the mode for prefaulting in segments into the segment table
-                when performing the START_WORK ioctl. Possible values:
+                when performing the START_WORK ioctl. Only applicable when
+                running under hashed page table mmu.
+                Possible values:
                          none: No prefaulting (default)
                          work_element_descriptor: Treat the work element
                                   descriptor as an effective address and
diff --git a/drivers/misc/cxl/sysfs.c b/drivers/misc/cxl/sysfs.c
index 4b5a4c5d3c01..629e2e156412 100644
--- a/drivers/misc/cxl/sysfs.c
+++ b/drivers/misc/cxl/sysfs.c
@@ -353,12 +353,20 @@ static ssize_t prefault_mode_store(struct device *device,
  	struct cxl_afu *afu = to_cxl_afu(device);
  	enum prefault_modes mode = -1;

-	if (!strncmp(buf, "work_element_descriptor", 23))
-		mode = CXL_PREFAULT_WED;
-	if (!strncmp(buf, "all", 3))
-		mode = CXL_PREFAULT_ALL;
  	if (!strncmp(buf, "none", 4))
  		mode = CXL_PREFAULT_NONE;
+	else {
+		if (!radix_enabled()) {
+
+			/* only allowed when not in radix mode */
+			if (!strncmp(buf, "work_element_descriptor", 23))
+				mode = CXL_PREFAULT_WED;
+			if (!strncmp(buf, "all", 3))
+				mode = CXL_PREFAULT_ALL;
+		} else {
+			dev_err(device, "Cannot prefault with radix enabled\n");
+		}
+	}

  	if (mode == -1)
  		return -EINVAL;

Re: [PATCH-RESEND] cxl: Disable prefault_mode in Radix mode

From: Andrew Donnellan <hidden>
Date: 2018-05-21 04:56:48 

On 18/05/18 19:42, Vaibhav Jain wrote:
quoted hunk
From: Vaibhav Jain <redacted>

Currently we see a kernel-oops reported on Power-9 while attaching a
context to an AFU, with radix-mode and sysfs attr 'prefault_mode' set
to anything other than 'none'. The backtrace of the oops is of this
form:

Unable to handle kernel paging request for data at address 0x00000080
Faulting instruction address: 0xc00800000bcf3b20
cpu 0x1: Vector: 300 (Data Access) at [c00000037f003800]
     pc: c00800000bcf3b20: cxl_load_segment+0x178/0x290 [cxl]
     lr: c00800000bcf39f0: cxl_load_segment+0x48/0x290 [cxl]
     sp: c00000037f003a80
    msr: 9000000000009033
    dar: 80
  dsisr: 40000000
   current = 0xc00000037f280000
   paca    = 0xc0000003ffffe600   softe: 3        irq_happened: 0x01
     pid   = 3529, comm = afp_no_int
<snip>
[c00000037f003af0] c00800000bcf4424 cxl_prefault+0xfc/0x248 [cxl]
[c00000037f003b50] c00800000bcf8a40 process_element_entry_psl9+0xd8/0x1a0 [cxl]
[c00000037f003b90] c00800000bcf944c cxl_attach_dedicated_process_psl9+0x44/0x130 [cxl]
[c00000037f003bd0] c00800000bcf5448 native_attach_process+0xc0/0x130 [cxl]
[c00000037f003c50] c00800000bcf16cc afu_ioctl+0x3f4/0x5e0 [cxl]
[c00000037f003d00] c00000000039d98c do_vfs_ioctl+0xdc/0x890
[c00000037f003da0] c00000000039e1a8 ksys_ioctl+0x68/0xf0
[c00000037f003df0] c00000000039e270 sys_ioctl+0x40/0xa0
[c00000037f003e30] c00000000000b320 system_call+0x58/0x6c
--- Exception: c01 (System Call) at 0000000010053bb0

The issue is caused as on Power-8 the AFU attr 'prefault_mode' was
used to improve initial storage fault performance by prefaulting
process segments. However on Power-9 with radix mode we don't have
Storage-Segments that we can prefault. Also prefaulting process Pages
will be too costly and fine-grained.

Hence, since the prefaulting mechanism doesn't makes sense of
radix-mode, this patch updates prefault_mode_store() to not allow any
other value apart from CXL_PREFAULT_NONE when radix mode is enabled.

Cc: <redacted>
Fixes: f24be42aab37 ("cxl: Add psl9 specific code")
Signed-off-by: Vaibhav Jain <redacted>
---
Change-log:

Resend  ->  Updated the commit description to add more info on the
	    issue seen [Andrew]
Thanks for redoing the commit message.

This looks good to me.

Acked-by: Andrew Donnellan <redacted>

-- 
Andrew Donnellan              OzLabs, ADL Canberra
andrew.donnellan@au1.ibm.com  IBM Australia Limited

Re: [PATCH-RESEND] cxl: Disable prefault_mode in Radix mode

From: Michael Ellerman <mpe@ellerman.id.au>
Date: 2018-05-29 01:18:38 

Vaibhav Jain [off-list ref] writes:
quoted hunk
From: Vaibhav Jain <redacted>

Currently we see a kernel-oops reported on Power-9 while attaching a
context to an AFU, with radix-mode and sysfs attr 'prefault_mode' set
to anything other than 'none'. The backtrace of the oops is of this
form:

Unable to handle kernel paging request for data at address 0x00000080
Faulting instruction address: 0xc00800000bcf3b20
cpu 0x1: Vector: 300 (Data Access) at [c00000037f003800]
    pc: c00800000bcf3b20: cxl_load_segment+0x178/0x290 [cxl]
    lr: c00800000bcf39f0: cxl_load_segment+0x48/0x290 [cxl]
    sp: c00000037f003a80
   msr: 9000000000009033
   dar: 80
 dsisr: 40000000
  current = 0xc00000037f280000
  paca    = 0xc0000003ffffe600   softe: 3        irq_happened: 0x01
    pid   = 3529, comm = afp_no_int
<snip>
[c00000037f003af0] c00800000bcf4424 cxl_prefault+0xfc/0x248 [cxl]
[c00000037f003b50] c00800000bcf8a40 process_element_entry_psl9+0xd8/0x1a0 [cxl]
[c00000037f003b90] c00800000bcf944c cxl_attach_dedicated_process_psl9+0x44/0x130 [cxl]
[c00000037f003bd0] c00800000bcf5448 native_attach_process+0xc0/0x130 [cxl]
[c00000037f003c50] c00800000bcf16cc afu_ioctl+0x3f4/0x5e0 [cxl]
[c00000037f003d00] c00000000039d98c do_vfs_ioctl+0xdc/0x890
[c00000037f003da0] c00000000039e1a8 ksys_ioctl+0x68/0xf0
[c00000037f003df0] c00000000039e270 sys_ioctl+0x40/0xa0
[c00000037f003e30] c00000000000b320 system_call+0x58/0x6c
--- Exception: c01 (System Call) at 0000000010053bb0
  ^^^
This tells patch/git-am to drop the rest of the change log, which is not
what we want.

I tend to indent stack traces etc with two spaces, which avoids the
problem. Or in this case we can just drop the line as it's not really
that informative.

I've fixed it up.

cheers

Re: [PATCH-RESEND] cxl: Disable prefault_mode in Radix mode

From: Vaibhav Jain <hidden>
Date: 2018-05-29 01:56:01 

Michael Ellerman [off-list ref] writes:

Vaibhav Jain [off-list ref] writes:
quoted
From: Vaibhav Jain <redacted>

Currently we see a kernel-oops reported on Power-9 while attaching a
context to an AFU, with radix-mode and sysfs attr 'prefault_mode' set
to anything other than 'none'. The backtrace of the oops is of this
form:

Unable to handle kernel paging request for data at address 0x00000080
Faulting instruction address: 0xc00800000bcf3b20
cpu 0x1: Vector: 300 (Data Access) at [c00000037f003800]
    pc: c00800000bcf3b20: cxl_load_segment+0x178/0x290 [cxl]
    lr: c00800000bcf39f0: cxl_load_segment+0x48/0x290 [cxl]
    sp: c00000037f003a80
   msr: 9000000000009033
   dar: 80
 dsisr: 40000000
  current = 0xc00000037f280000
  paca    = 0xc0000003ffffe600   softe: 3        irq_happened: 0x01
    pid   = 3529, comm = afp_no_int
<snip>
[c00000037f003af0] c00800000bcf4424 cxl_prefault+0xfc/0x248 [cxl]
[c00000037f003b50] c00800000bcf8a40 process_element_entry_psl9+0xd8/0x1a0 [cxl]
[c00000037f003b90] c00800000bcf944c cxl_attach_dedicated_process_psl9+0x44/0x130 [cxl]
[c00000037f003bd0] c00800000bcf5448 native_attach_process+0xc0/0x130 [cxl]
[c00000037f003c50] c00800000bcf16cc afu_ioctl+0x3f4/0x5e0 [cxl]
[c00000037f003d00] c00000000039d98c do_vfs_ioctl+0xdc/0x890
[c00000037f003da0] c00000000039e1a8 ksys_ioctl+0x68/0xf0
[c00000037f003df0] c00000000039e270 sys_ioctl+0x40/0xa0
[c00000037f003e30] c00000000000b320 system_call+0x58/0x6c
--- Exception: c01 (System Call) at 0000000010053bb0
  ^^^
This tells patch/git-am to drop the rest of the change log, which is not
what we want.

I tend to indent stack traces etc with two spaces, which avoids the
problem. Or in this case we can just drop the line as it's not really
that informative.

I've fixed it up.

cheers
Sorry for missing that and thanks for fixing it in the patch.

-- 
Vaibhav Jain [off-list ref]
Linux Technology Center, IBM India Pvt. Ltd.

Re: [PATCH-RESEND] cxl: Disable prefault_mode in Radix mode

From: Michael Ellerman <hidden>
Date: 2018-06-01 15:54:59 

On Fri, 2018-05-18 at 09:42:23 UTC, Vaibhav Jain wrote:
quoted hunk
From: Vaibhav Jain <redacted>

Currently we see a kernel-oops reported on Power-9 while attaching a
context to an AFU, with radix-mode and sysfs attr 'prefault_mode' set
to anything other than 'none'. The backtrace of the oops is of this
form:

Unable to handle kernel paging request for data at address 0x00000080
Faulting instruction address: 0xc00800000bcf3b20
cpu 0x1: Vector: 300 (Data Access) at [c00000037f003800]
    pc: c00800000bcf3b20: cxl_load_segment+0x178/0x290 [cxl]
    lr: c00800000bcf39f0: cxl_load_segment+0x48/0x290 [cxl]
    sp: c00000037f003a80
   msr: 9000000000009033
   dar: 80
 dsisr: 40000000
  current = 0xc00000037f280000
  paca    = 0xc0000003ffffe600   softe: 3        irq_happened: 0x01
    pid   = 3529, comm = afp_no_int
<snip>
[c00000037f003af0] c00800000bcf4424 cxl_prefault+0xfc/0x248 [cxl]
[c00000037f003b50] c00800000bcf8a40 process_element_entry_psl9+0xd8/0x1a0 [cxl]
[c00000037f003b90] c00800000bcf944c cxl_attach_dedicated_process_psl9+0x44/0x130 [cxl]
[c00000037f003bd0] c00800000bcf5448 native_attach_process+0xc0/0x130 [cxl]
[c00000037f003c50] c00800000bcf16cc afu_ioctl+0x3f4/0x5e0 [cxl]
[c00000037f003d00] c00000000039d98c do_vfs_ioctl+0xdc/0x890
[c00000037f003da0] c00000000039e1a8 ksys_ioctl+0x68/0xf0
[c00000037f003df0] c00000000039e270 sys_ioctl+0x40/0xa0
[c00000037f003e30] c00000000000b320 system_call+0x58/0x6c
--- Exception: c01 (System Call) at 0000000010053bb0

The issue is caused as on Power-8 the AFU attr 'prefault_mode' was
used to improve initial storage fault performance by prefaulting
process segments. However on Power-9 with radix mode we don't have
Storage-Segments that we can prefault. Also prefaulting process Pages
will be too costly and fine-grained.

Hence, since the prefaulting mechanism doesn't makes sense of
radix-mode, this patch updates prefault_mode_store() to not allow any
other value apart from CXL_PREFAULT_NONE when radix mode is enabled.

Cc: <redacted>
Fixes: f24be42aab37 ("cxl: Add psl9 specific code")
Signed-off-by: Vaibhav Jain <redacted>
Acked-by: Frederic Barrat <redacted>
Acked-by: Andrew Donnellan <redacted>
Applied to powerpc next, thanks.

https://git.kernel.org/powerpc/c/b6c84ba22ff3a198eb8d5552cf9b8f

cheers
Keyboard shortcuts
hback out one level
jnext message in thread
kprevious message in thread
ldrill in
Escclose help / fold thread tree
?toggle this help