On Fri, Jun 13, 2014 at 08:12:08AM +0100, Denis Kirjanov wrote:
On 6/12/14, Catalin Marinas [off-list ref] wrote:
quoted
On Thu, Jun 12, 2014 at 01:00:57PM +0100, Denis Kirjanov wrote:
quoted
On 6/12/14, Denis Kirjanov [off-list ref] wrote:
quoted
On 6/12/14, Catalin Marinas [off-list ref] wrote:
quoted
On 11 Jun 2014, at 21:04, Denis Kirjanov [off-list ref]
wrote:
quoted
On 6/11/14, Catalin Marinas [off-list ref] wrote:
quoted
On Wed, Jun 11, 2014 at 04:13:07PM +0400, Denis Kirjanov wrote:
quoted
I got a trace while running 3.15.0-08556-gdfb9454:
[ 104.534026] Unable to handle kernel paging request for data at
address 0xc00000007f000000
Were there any kmemleak messages prior to this, like "kmemleak
disabled"? There could be a race when kmemleak is disabled because
of
some fatal (for kmemleak) error while the scanning is taking place
(which needs some more thinking to fix properly).
No. I checked for the similar problem and didn't find anything
relevant.
I'll try to bisect it.
Does this happen soon after boot? I guess it’s the first scan
(scheduled at around 1min after boot). Something seems to be telling
kmemleak that there is a valid memory block at 0xc00000007f000000.
Yeah, it happens after a while with a booted system so that's the
first kmemleak scan.
I've bisected to this commit: d4c54919ed86302094c0ca7d48a8cbd4ee753e92
"mm: add !pte_present() check on existing hugetlb_entry callbacks".
Reverting the commit fixes the issue
I can't figure how this causes the problem but I have more questions. Is
0xc00000007f000000 address always the same in all crashes? If yes, you
could comment out start_scan_thread() in kmemleak_late_init() to avoid
the scanning thread starting. Once booted, you can run:
echo dump=0xc00000007f000000 > /sys/kernel/debug/kmemleak
and check the dmesg for what kmemleak knows about that address, when it
was allocated and whether it should be mapped or not.
OK, so that's the DART table allocated via alloc_dart_table(). Is
dart_tablebase removed from the kernel linear mapping after allocation?
If that's the case, we need to tell kmemleak to ignore this block (see
patch below, untested). But I still can't explain how commit
d4c54919ed863020 causes this issue.
(also cc'ing the powerpc list and maintainers)
---------------8<--------------------------
From 09a7f1c97166c7bdca7ca4e8a4ff2774f3706ea3 Mon Sep 17 00:00:00 2001
From: Catalin Marinas <catalin.marinas@arm.com>
Date: Fri, 13 Jun 2014 09:44:21 +0100
Subject: [PATCH] powerpc/kmemleak: Do not scan the DART table
The DART table allocation is registered to kmemleak via the
memblock_alloc_base() call. However, the DART table is later unmapped
and dart_tablebase VA no longer accessible. This patch tells kmemleak
not to scan this block and avoid an unhandled paging request.
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Cc: Paul Mackerras <redacted>
---
arch/powerpc/sysdev/dart_iommu.c | 5 +++++
1 file changed, 5 insertions(+)
On Fri, Jun 13, 2014 at 08:12:08AM +0100, Denis Kirjanov wrote:
quoted
On 6/12/14, Catalin Marinas [off-list ref] wrote:
quoted
On Thu, Jun 12, 2014 at 01:00:57PM +0100, Denis Kirjanov wrote:
quoted
On 6/12/14, Denis Kirjanov [off-list ref] wrote:
quoted
On 6/12/14, Catalin Marinas [off-list ref] wrote:
quoted
On 11 Jun 2014, at 21:04, Denis Kirjanov [off-list ref]
wrote:
quoted
On 6/11/14, Catalin Marinas [off-list ref] wrote:
quoted
On Wed, Jun 11, 2014 at 04:13:07PM +0400, Denis Kirjanov wrote:
quoted
I got a trace while running 3.15.0-08556-gdfb9454:
[ 104.534026] Unable to handle kernel paging request for data
at
address 0xc00000007f000000
Were there any kmemleak messages prior to this, like "kmemleak
disabled"? There could be a race when kmemleak is disabled
because
of
some fatal (for kmemleak) error while the scanning is taking
place
(which needs some more thinking to fix properly).
No. I checked for the similar problem and didn't find anything
relevant.
I'll try to bisect it.
Does this happen soon after boot? I guess it=E2=80=99s the first s=
can
quoted
quoted
quoted
quoted
quoted
(scheduled at around 1min after boot). Something seems to be
telling
kmemleak that there is a valid memory block at 0xc00000007f000000.
Yeah, it happens after a while with a booted system so that's the
first kmemleak scan.
I've bisected to this commit: d4c54919ed86302094c0ca7d48a8cbd4ee753e9=
2
quoted
quoted
quoted
"mm: add !pte_present() check on existing hugetlb_entry callbacks".
Reverting the commit fixes the issue
I can't figure how this causes the problem but I have more questions.
Is
0xc00000007f000000 address always the same in all crashes? If yes, you
could comment out start_scan_thread() in kmemleak_late_init() to avoid
the scanning thread starting. Once booted, you can run:
echo dump=3D0xc00000007f000000 > /sys/kernel/debug/kmemleak
and check the dmesg for what kmemleak knows about that address, when i=
t
quoted
quoted
was allocated and whether it should be mapped or not.
OK, so that's the DART table allocated via alloc_dart_table(). Is
dart_tablebase removed from the kernel linear mapping after allocation?
If that's the case, we need to tell kmemleak to ignore this block (see
patch below, untested). But I still can't explain how commit
d4c54919ed863020 causes this issue.
(also cc'ing the powerpc list and maintainers)
Ok, your path fixes the oops.
Ben, can you shed some light on this issue?
Thanks!
quoted hunk
---------------8<--------------------------
From 09a7f1c97166c7bdca7ca4e8a4ff2774f3706ea3 Mon Sep 17 00:00:00 2001
From: Catalin Marinas <catalin.marinas@arm.com>
Date: Fri, 13 Jun 2014 09:44:21 +0100
Subject: [PATCH] powerpc/kmemleak: Do not scan the DART table
The DART table allocation is registered to kmemleak via the
memblock_alloc_base() call. However, the DART table is later unmapped
and dart_tablebase VA no longer accessible. This patch tells kmemleak
not to scan this block and avoid an unhandled paging request.
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Cc: Paul Mackerras <redacted>
---
arch/powerpc/sysdev/dart_iommu.c | 5 +++++
1 file changed, 5 insertions(+)
diff --git a/arch/powerpc/sysdev/dart_iommu.c
b/arch/powerpc/sysdev/dart_iommu.c
index 62c47bb76517..9e5353ff6d1b 100644
From: Benjamin Herrenschmidt <benh@kernel.crashing.org> Date: 2014-06-13 21:44:55
On Fri, 2014-06-13 at 09:56 +0100, Catalin Marinas wrote:
OK, so that's the DART table allocated via alloc_dart_table(). Is
dart_tablebase removed from the kernel linear mapping after allocation?
Yes.
If that's the case, we need to tell kmemleak to ignore this block (see
patch below, untested). But I still can't explain how commit
d4c54919ed863020 causes this issue.
(also cc'ing the powerpc list and maintainers)
We remove the DART from the linear mapping because it has to be mapped
non-cachable and having it in the linear mapping would cause cache
paradoxes. We also can't just change the caching attributes in the
linear mapping because we use 16M pages for it and 970 CPUs don't
support cache-inhibited 16M pages :-( And due to the MMU segmentation
model, we also can't mix & match page sizes in that area.
So we just unmap it, and ioremap it elsewhere.
Cheers,
Ben.
quoted hunk
---------------8<--------------------------
quoted
From 09a7f1c97166c7bdca7ca4e8a4ff2774f3706ea3 Mon Sep 17 00:00:00 2001
From: Catalin Marinas <catalin.marinas@arm.com>
Date: Fri, 13 Jun 2014 09:44:21 +0100
Subject: [PATCH] powerpc/kmemleak: Do not scan the DART table
The DART table allocation is registered to kmemleak via the
memblock_alloc_base() call. However, the DART table is later unmapped
and dart_tablebase VA no longer accessible. This patch tells kmemleak
not to scan this block and avoid an unhandled paging request.
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Cc: Paul Mackerras <redacted>
---
arch/powerpc/sysdev/dart_iommu.c | 5 +++++
1 file changed, 5 insertions(+)
On 13 Jun 2014, at 22:44, Benjamin Herrenschmidt =
[off-list ref] wrote:
On Fri, 2014-06-13 at 09:56 +0100, Catalin Marinas wrote:
=20
quoted
OK, so that's the DART table allocated via alloc_dart_table(). Is
dart_tablebase removed from the kernel linear mapping after =
allocation?
=20
Yes.
=20
quoted
If that's the case, we need to tell kmemleak to ignore this block =
(see
quoted
patch below, untested). But I still can't explain how commit
d4c54919ed863020 causes this issue.
=20
(also cc'ing the powerpc list and maintainers)
=20
We remove the DART from the linear mapping because it has to be mapped
non-cachable and having it in the linear mapping would cause cache
paradoxes. We also can't just change the caching attributes in the
linear mapping because we use 16M pages for it and 970 CPUs don't
support cache-inhibited 16M pages :-( And due to the MMU segmentation
model, we also can't mix & match page sizes in that area.
=20
So we just unmap it, and ioremap it elsewhere.
OK, thanks for the explanation. So the kmemleak annotation makes sense.
Would you please take the I patch earlier (I guess with Denis=92 tested-
by). I can send it separately if more convenient.
Thanks,
Catalin=
From: Michael Ellerman <mpe@ellerman.id.au> Date: 2014-06-16 02:40:11
On Fri, 2014-06-13 at 14:26 +0400, Denis Kirjanov wrote:
On 6/13/14, Catalin Marinas [off-list ref] wrote:
quoted
On Fri, Jun 13, 2014 at 08:12:08AM +0100, Denis Kirjanov wrote:
quoted
On 6/12/14, Catalin Marinas [off-list ref] wrote:
quoted
On Thu, Jun 12, 2014 at 01:00:57PM +0100, Denis Kirjanov wrote:
quoted
On 6/12/14, Denis Kirjanov [off-list ref] wrote:
quoted
On 6/12/14, Catalin Marinas [off-list ref] wrote:
quoted
On 11 Jun 2014, at 21:04, Denis Kirjanov [off-list ref]
wrote:
quoted
On 6/11/14, Catalin Marinas [off-list ref] wrote:
quoted
On Wed, Jun 11, 2014 at 04:13:07PM +0400, Denis Kirjanov wrote:
quoted
I got a trace while running 3.15.0-08556-gdfb9454:
[ 104.534026] Unable to handle kernel paging request for data
at
address 0xc00000007f000000
Were there any kmemleak messages prior to this, like "kmemleak
disabled"? There could be a race when kmemleak is disabled
because
of
some fatal (for kmemleak) error while the scanning is taking
place
(which needs some more thinking to fix properly).
No. I checked for the similar problem and didn't find anything
relevant.
I'll try to bisect it.
Does this happen soon after boot? I guess it’s the first scan
(scheduled at around 1min after boot). Something seems to be
telling
kmemleak that there is a valid memory block at 0xc00000007f000000.
Yeah, it happens after a while with a booted system so that's the
first kmemleak scan.
I've bisected to this commit: d4c54919ed86302094c0ca7d48a8cbd4ee753e92
"mm: add !pte_present() check on existing hugetlb_entry callbacks".
Reverting the commit fixes the issue
I can't figure how this causes the problem but I have more questions.
Is
0xc00000007f000000 address always the same in all crashes? If yes, you
could comment out start_scan_thread() in kmemleak_late_init() to avoid
the scanning thread starting. Once booted, you can run:
echo dump=0xc00000007f000000 > /sys/kernel/debug/kmemleak
and check the dmesg for what kmemleak knows about that address, when it
was allocated and whether it should be mapped or not.
OK, so that's the DART table allocated via alloc_dart_table(). Is
dart_tablebase removed from the kernel linear mapping after allocation?
If that's the case, we need to tell kmemleak to ignore this block (see
patch below, untested). But I still can't explain how commit
d4c54919ed863020 causes this issue.
(also cc'ing the powerpc list and maintainers)
Ok, your path fixes the oops.
Ben, can you shed some light on this issue?
(I'm not Ben)
Yes, the memory for dart_tablebase is removed from the linear mapping. In fact
it's never mapped, see htab_initialize().
I don't easily see how commit d4c54919ed8, could have exposed this, but I don't
know enough of the kmemleak internals to say for sure.
cheers