From: Paul Menzel <hidden> Date: 2020-07-09 14:40:37
Dear Linux folks,
Despite Linux 5.8-rc4 reporting memory leaks on the IBM POWER 8 S822LC,
the file does not contain more information.
$ dmesg
[…] > [48662.953323] perf: interrupt took too long (2570 > 2500), lowering
kernel.perf_event_max_sample_rate to 77750
[48854.810636] perf: interrupt took too long (3216 > 3212), lowering kernel.perf_event_max_sample_rate to 62000
[52300.044518] perf: interrupt took too long (4244 > 4020), lowering kernel.perf_event_max_sample_rate to 47000
[52751.373083] perf: interrupt took too long (5373 > 5305), lowering kernel.perf_event_max_sample_rate to 37000
[53354.000363] perf: interrupt took too long (6793 > 6716), lowering kernel.perf_event_max_sample_rate to 29250
[53850.215606] perf: interrupt took too long (8672 > 8491), lowering kernel.perf_event_max_sample_rate to 23000
[57542.266099] perf: interrupt took too long (10940 > 10840), lowering kernel.perf_event_max_sample_rate to 18250
[57559.645404] perf: interrupt took too long (13714 > 13675), lowering kernel.perf_event_max_sample_rate to 14500
[61608.697728] Can't find PMC that caused IRQ
[71774.463111] kmemleak: 12 new suspected memory leaks (see /sys/kernel/debug/kmemleak)
[92372.044785] process '@/usr/bin/gnatmake-5' started with executable stack
[92849.380672] FS-Cache: Loaded
[92849.417269] FS-Cache: Netfs 'nfs' registered for caching
[92849.595974] NFS: Registering the id_resolver key type
[92849.596000] Key type id_resolver registered
[92849.596000] Key type id_legacy registered
[101808.079143] kmemleak: 1 new suspected memory leaks (see /sys/kernel/debug/kmemleak)
[106904.323471] Can't find PMC that caused IRQ
[129416.391456] kmemleak: 1 new suspected memory leaks (see /sys/kernel/debug/kmemleak)
[158171.604221] kmemleak: 34 new suspected memory leaks (see /sys/kernel/debug/kmemleak)
$ sudo cat /sys/kernel/debug/kmemleak
$
On Thu, Jul 09, 2020 at 04:37:10PM +0200, Paul Menzel wrote:
Despite Linux 5.8-rc4 reporting memory leaks on the IBM POWER 8 S822LC, the
file does not contain more information.
quoted
$ dmesg
[…] > [48662.953323] perf: interrupt took too long (2570 > 2500),
lowering
kernel.perf_event_max_sample_rate to 77750
quoted
[48854.810636] perf: interrupt took too long (3216 > 3212), lowering kernel.perf_event_max_sample_rate to 62000
[52300.044518] perf: interrupt took too long (4244 > 4020), lowering kernel.perf_event_max_sample_rate to 47000
[52751.373083] perf: interrupt took too long (5373 > 5305), lowering kernel.perf_event_max_sample_rate to 37000
[53354.000363] perf: interrupt took too long (6793 > 6716), lowering kernel.perf_event_max_sample_rate to 29250
[53850.215606] perf: interrupt took too long (8672 > 8491), lowering kernel.perf_event_max_sample_rate to 23000
[57542.266099] perf: interrupt took too long (10940 > 10840), lowering kernel.perf_event_max_sample_rate to 18250
[57559.645404] perf: interrupt took too long (13714 > 13675), lowering kernel.perf_event_max_sample_rate to 14500
[61608.697728] Can't find PMC that caused IRQ
[71774.463111] kmemleak: 12 new suspected memory leaks (see /sys/kernel/debug/kmemleak)
[92372.044785] process '@/usr/bin/gnatmake-5' started with executable stack
[92849.380672] FS-Cache: Loaded
[92849.417269] FS-Cache: Netfs 'nfs' registered for caching
[92849.595974] NFS: Registering the id_resolver key type
[92849.596000] Key type id_resolver registered
[92849.596000] Key type id_legacy registered
[101808.079143] kmemleak: 1 new suspected memory leaks (see /sys/kernel/debug/kmemleak)
[106904.323471] Can't find PMC that caused IRQ
[129416.391456] kmemleak: 1 new suspected memory leaks (see /sys/kernel/debug/kmemleak)
[158171.604221] kmemleak: 34 new suspected memory leaks (see /sys/kernel/debug/kmemleak)
$ sudo cat /sys/kernel/debug/kmemleak
When they are no longer present, they are most likely false positives.
Was this triggered during boot? Or under some workload?
--
Catalin
From: Paul Menzel <hidden> Date: 2020-07-09 21:10:51
Dear Catalin,
Am 09.07.20 um 19:57 schrieb Catalin Marinas:
On Thu, Jul 09, 2020 at 04:37:10PM +0200, Paul Menzel wrote:
quoted
Despite Linux 5.8-rc4 reporting memory leaks on the IBM POWER 8 S822LC, the
file does not contain more information.
quoted
$ dmesg
[…] > [48662.953323] perf: interrupt took too long (2570 > 2500), lowering kernel.perf_event_max_sample_rate to 77750
[48854.810636] perf: interrupt took too long (3216 > 3212), lowering kernel.perf_event_max_sample_rate to 62000
[52300.044518] perf: interrupt took too long (4244 > 4020), lowering kernel.perf_event_max_sample_rate to 47000
[52751.373083] perf: interrupt took too long (5373 > 5305), lowering kernel.perf_event_max_sample_rate to 37000
[53354.000363] perf: interrupt took too long (6793 > 6716), lowering kernel.perf_event_max_sample_rate to 29250
[53850.215606] perf: interrupt took too long (8672 > 8491), lowering kernel.perf_event_max_sample_rate to 23000
[57542.266099] perf: interrupt took too long (10940 > 10840), lowering kernel.perf_event_max_sample_rate to 18250
[57559.645404] perf: interrupt took too long (13714 > 13675), lowering kernel.perf_event_max_sample_rate to 14500
[61608.697728] Can't find PMC that caused IRQ
[71774.463111] kmemleak: 12 new suspected memory leaks (see /sys/kernel/debug/kmemleak)
[92372.044785] process '@/usr/bin/gnatmake-5' started with executable stack
[92849.380672] FS-Cache: Loaded
[92849.417269] FS-Cache: Netfs 'nfs' registered for caching
[92849.595974] NFS: Registering the id_resolver key type
[92849.596000] Key type id_resolver registered
[92849.596000] Key type id_legacy registered
[101808.079143] kmemleak: 1 new suspected memory leaks (see /sys/kernel/debug/kmemleak)
[106904.323471] Can't find PMC that caused IRQ
[129416.391456] kmemleak: 1 new suspected memory leaks (see /sys/kernel/debug/kmemleak)
[158171.604221] kmemleak: 34 new suspected memory leaks (see /sys/kernel/debug/kmemleak)
$ sudo cat /sys/kernel/debug/kmemleak
When they are no longer present, they are most likely false positives.
How can this be? Shouldn’t the false positive also be logged in
`/sys/kernel/debug/kmemleak`?
Was this triggered during boot? Or under some workload?
From the timestamps it looks like under some load.
Kind regards,
Paul
On Thu, Jul 09, 2020 at 11:08:52PM +0200, Paul Menzel wrote:
Am 09.07.20 um 19:57 schrieb Catalin Marinas:
quoted
On Thu, Jul 09, 2020 at 04:37:10PM +0200, Paul Menzel wrote:
quoted
Despite Linux 5.8-rc4 reporting memory leaks on the IBM POWER 8 S822LC, the
file does not contain more information.
quoted
$ dmesg
[…] > [48662.953323] perf: interrupt took too long (2570 > 2500), lowering kernel.perf_event_max_sample_rate to 77750
[48854.810636] perf: interrupt took too long (3216 > 3212), lowering kernel.perf_event_max_sample_rate to 62000
[52300.044518] perf: interrupt took too long (4244 > 4020), lowering kernel.perf_event_max_sample_rate to 47000
[52751.373083] perf: interrupt took too long (5373 > 5305), lowering kernel.perf_event_max_sample_rate to 37000
[53354.000363] perf: interrupt took too long (6793 > 6716), lowering kernel.perf_event_max_sample_rate to 29250
[53850.215606] perf: interrupt took too long (8672 > 8491), lowering kernel.perf_event_max_sample_rate to 23000
[57542.266099] perf: interrupt took too long (10940 > 10840), lowering kernel.perf_event_max_sample_rate to 18250
[57559.645404] perf: interrupt took too long (13714 > 13675), lowering kernel.perf_event_max_sample_rate to 14500
[61608.697728] Can't find PMC that caused IRQ
[71774.463111] kmemleak: 12 new suspected memory leaks (see /sys/kernel/debug/kmemleak)
[92372.044785] process '@/usr/bin/gnatmake-5' started with executable stack
[92849.380672] FS-Cache: Loaded
[92849.417269] FS-Cache: Netfs 'nfs' registered for caching
[92849.595974] NFS: Registering the id_resolver key type
[92849.596000] Key type id_resolver registered
[92849.596000] Key type id_legacy registered
[101808.079143] kmemleak: 1 new suspected memory leaks (see /sys/kernel/debug/kmemleak)
[106904.323471] Can't find PMC that caused IRQ
[129416.391456] kmemleak: 1 new suspected memory leaks (see /sys/kernel/debug/kmemleak)
[158171.604221] kmemleak: 34 new suspected memory leaks (see /sys/kernel/debug/kmemleak)
$ sudo cat /sys/kernel/debug/kmemleak
When they are no longer present, they are most likely false positives.
How can this be? Shouldn’t the false positive also be logged in
`/sys/kernel/debug/kmemleak`?
Sorry, I wasn't clear. It can be a transient false positive. At a
subsequent scan, kmemleak found pointer referring the previously
reported objects and no longer shows them.
quoted
Was this triggered during boot? Or under some workload?
From the timestamps it looks like under some load.
Was it during boot? I put a delay of 60s to avoid this but, depending on
the platform, it can still trigger.
--
Catalin
From: Paul Menzel <hidden> Date: 2020-07-14 07:01:48
Dear Catalin,
Am 13.07.20 um 20:27 schrieb Catalin Marinas:
On Thu, Jul 09, 2020 at 11:08:52PM +0200, Paul Menzel wrote:
quoted
Am 09.07.20 um 19:57 schrieb Catalin Marinas:
quoted
On Thu, Jul 09, 2020 at 04:37:10PM +0200, Paul Menzel wrote:
quoted
Despite Linux 5.8-rc4 reporting memory leaks on the IBM POWER 8 S822LC, the
file does not contain more information.
quoted
$ dmesg
[…] > [48662.953323] perf: interrupt took too long (2570 > 2500), lowering kernel.perf_event_max_sample_rate to 77750
[48854.810636] perf: interrupt took too long (3216 > 3212), lowering kernel.perf_event_max_sample_rate to 62000
[52300.044518] perf: interrupt took too long (4244 > 4020), lowering kernel.perf_event_max_sample_rate to 47000
[52751.373083] perf: interrupt took too long (5373 > 5305), lowering kernel.perf_event_max_sample_rate to 37000
[53354.000363] perf: interrupt took too long (6793 > 6716), lowering kernel.perf_event_max_sample_rate to 29250
[53850.215606] perf: interrupt took too long (8672 > 8491), lowering kernel.perf_event_max_sample_rate to 23000
[57542.266099] perf: interrupt took too long (10940 > 10840), lowering kernel.perf_event_max_sample_rate to 18250
[57559.645404] perf: interrupt took too long (13714 > 13675), lowering kernel.perf_event_max_sample_rate to 14500
[61608.697728] Can't find PMC that caused IRQ
[71774.463111] kmemleak: 12 new suspected memory leaks (see /sys/kernel/debug/kmemleak)
[92372.044785] process '@/usr/bin/gnatmake-5' started with executable stack
[92849.380672] FS-Cache: Loaded
[92849.417269] FS-Cache: Netfs 'nfs' registered for caching
[92849.595974] NFS: Registering the id_resolver key type
[92849.596000] Key type id_resolver registered
[92849.596000] Key type id_legacy registered
[101808.079143] kmemleak: 1 new suspected memory leaks (see /sys/kernel/debug/kmemleak)
[106904.323471] Can't find PMC that caused IRQ
[129416.391456] kmemleak: 1 new suspected memory leaks (see /sys/kernel/debug/kmemleak)
[158171.604221] kmemleak: 34 new suspected memory leaks (see /sys/kernel/debug/kmemleak)
$ sudo cat /sys/kernel/debug/kmemleak
When they are no longer present, they are most likely false positives.
How can this be? Shouldn’t the false positive also be logged in
`/sys/kernel/debug/kmemleak`?
Sorry, I wasn't clear. It can be a transient false positive. At a
subsequent scan, kmemleak found pointer referring the previously
reported objects and no longer shows them.
Interesting. Is it possible to print a message in that case to avoid
confusion?
quoted
quoted
Was this triggered during boot? Or under some workload?
From the timestamps it looks like under some load.
Was it during boot? I put a delay of 60s to avoid this but, depending on
the platform, it can still trigger.
No, it happened after several hours of runtime.
Kind regards,
Paul