Oops running iptables -F OUTPUT
8 messages,
4 authors,
2018-08-29 · open the first message on its own page
I'm getting this Oops when running iptables -F OUTPUT:
[ 91.139409] Unable to handle kernel paging request for data at address 0xd0000001fff12f34
[ 91.139414] Faulting instruction address: 0xd0000000016a5718
[ 91.139419] Oops: Kernel access of bad area, sig: 11 [#1]
[ 91.139426] BE SMP NR_CPUS=2 PowerMac
[ 91.139434] Modules linked in: iptable_filter ip_tables x_tables bpfilter nfsd auth_rpcgss lockd grace nfs_acl sunrpc tun af_packet snd_aoa_codec_tas snd_aoa_fabric_layout snd_aoa snd_aoa_i2sbus snd_aoa_soundbus snd_pcm_oss snd_pcm snd_seq snd_timer snd_seq_device snd_mixer_oss snd sungem sr_mod firewire_ohci cdrom sungem_phy soundcore firewire_core pata_macio crc_itu_t sg hid_generic usbhid linear md_mod ohci_pci ohci_hcd ehci_pci ehci_hcd usbcore usb_common dm_snapshot dm_bufio dm_mirror dm_region_hash dm_log dm_mod sata_svw
[ 91.139522] CPU: 1 PID: 3620 Comm: iptables Not tainted 4.19.0-rc1 #1
[ 91.139526] NIP: d0000000016a5718 LR: d0000000016a569c CTR: c0000000006f560c
[ 91.139531] REGS: c0000001fa577670 TRAP: 0300 Not tainted (4.19.0-rc1)
[ 91.139534] MSR: 900000000200b032 <SF,HV,VEC,EE,FP,ME,IR,DR,RI> CR: 84002484 XER: 20000000
[ 91.139553] DAR: d0000001fff12f34 DSISR: 40000000 IRQMASK: 0
GPR00: d0000000016a569c c0000001fa5778f0 d0000000016b0400 0000000000000000
GPR04: 0000000000000002 0000000000000000 80000001fa46418e c0000001fa0d05c8
GPR08: d0000000016b0400 d00037fffff13000 00000001ff3e7000 d0000000016a6fb8
GPR12: c0000000006f560c c00000000ffff780 0000000000000000 0000000000000000
GPR16: 0000000011635010 00003fffa1b7aa68 0000000000000000 0000000000000000
GPR20: 0000000000000003 0000000010013918 00000000116350c0 c000000000b88990
GPR24: c000000000b88ba4 0000000000000000 d0000001fff12f34 0000000000000000
GPR28: d0000000016b8000 c0000001fa20f400 c0000001fa20f440 0000000000000000
[ 91.139627] NIP [d0000000016a5718] .alloc_counters.isra.10+0xbc/0x140 [ip_tables]
[ 91.139634] LR [d0000000016a569c] .alloc_counters.isra.10+0x40/0x140 [ip_tables]
[ 91.139638] Call Trace:
[ 91.139645] [c0000001fa5778f0] [d0000000016a569c] .alloc_counters.isra.10+0x40/0x140 [ip_tables] (unreliable)
[ 91.139655] [c0000001fa5779b0] [d0000000016a5b54] .do_ipt_get_ctl+0x110/0x2ec [ip_tables]
[ 91.139666] [c0000001fa577aa0] [c0000000006233e0] .nf_getsockopt+0x68/0x88
[ 91.139674] [c0000001fa577b40] [c000000000631608] .ip_getsockopt+0xbc/0x128
[ 91.139682] [c0000001fa577bf0] [c00000000065adf4] .raw_getsockopt+0x18/0x5c
[ 91.139690] [c0000001fa577c60] [c0000000005b5f60] .sock_common_getsockopt+0x2c/0x40
[ 91.139697] [c0000001fa577cd0] [c0000000005b3394] .__sys_getsockopt+0xa4/0xd0
[ 91.139704] [c0000001fa577d80] [c0000000005b5ab0] .__se_sys_socketcall+0x238/0x2b4
[ 91.139712] [c0000001fa577e30] [c00000000000a31c] system_call+0x5c/0x70
[ 91.139716] Instruction dump:
[ 91.139721] 39290040 7d3d4a14 7fbe4840 409cff98 81380000 2b890001 419d000c 393e0060
[ 91.139736] 48000010 7d57c82a e93e0060 7d295214 <815a0000> 794807e1 41e20010 7c210b78
[ 91.139752] ---[ end trace f5d1d5431651845d ]---
Andreas.
--
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510 2552 DF73 E780 A9DA AEC1
"And now for something completely different."
On Mon, 27 Aug 2018 19:11:01 +0200
Andreas Schwab [off-list ref] wrote:
I'm getting this Oops when running iptables -F OUTPUT:
[ 91.139409] Unable to handle kernel paging request for data at address 0xd0000001fff12f34
[ 91.139414] Faulting instruction address: 0xd0000000016a5718
[ 91.139419] Oops: Kernel access of bad area, sig: 11 [#1]
[ 91.139426] BE SMP NR_CPUS=2 PowerMac
[ 91.139434] Modules linked in: iptable_filter ip_tables x_tables bpfilter nfsd auth_rpcgss lockd grace nfs_acl sunrpc tun af_packet snd_aoa_codec_tas snd_aoa_fabric_layout snd_aoa snd_aoa_i2sbus snd_aoa_soundbus snd_pcm_oss snd_pcm snd_seq snd_timer snd_seq_device snd_mixer_oss snd sungem sr_mod firewire_ohci cdrom sungem_phy soundcore firewire_core pata_macio crc_itu_t sg hid_generic usbhid linear md_mod ohci_pci ohci_hcd ehci_pci ehci_hcd usbcore usb_common dm_snapshot dm_bufio dm_mirror dm_region_hash dm_log dm_mod sata_svw
[ 91.139522] CPU: 1 PID: 3620 Comm: iptables Not tainted 4.19.0-rc1 #1
[ 91.139526] NIP: d0000000016a5718 LR: d0000000016a569c CTR: c0000000006f560c
[ 91.139531] REGS: c0000001fa577670 TRAP: 0300 Not tainted (4.19.0-rc1)
[ 91.139534] MSR: 900000000200b032 <SF,HV,VEC,EE,FP,ME,IR,DR,RI> CR: 84002484 XER: 20000000
[ 91.139553] DAR: d0000001fff12f34 DSISR: 40000000 IRQMASK: 0
GPR00: d0000000016a569c c0000001fa5778f0 d0000000016b0400 0000000000000000
GPR04: 0000000000000002 0000000000000000 80000001fa46418e c0000001fa0d05c8
GPR08: d0000000016b0400 d00037fffff13000 00000001ff3e7000 d0000000016a6fb8
GPR12: c0000000006f560c c00000000ffff780 0000000000000000 0000000000000000
GPR16: 0000000011635010 00003fffa1b7aa68 0000000000000000 0000000000000000
GPR20: 0000000000000003 0000000010013918 00000000116350c0 c000000000b88990
GPR24: c000000000b88ba4 0000000000000000 d0000001fff12f34 0000000000000000
GPR28: d0000000016b8000 c0000001fa20f400 c0000001fa20f440 0000000000000000
[ 91.139627] NIP [d0000000016a5718] .alloc_counters.isra.10+0xbc/0x140 [ip_tables]
[ 91.139634] LR [d0000000016a569c] .alloc_counters.isra.10+0x40/0x140 [ip_tables]
[ 91.139638] Call Trace:
[ 91.139645] [c0000001fa5778f0] [d0000000016a569c] .alloc_counters.isra.10+0x40/0x140 [ip_tables] (unreliable)
[ 91.139655] [c0000001fa5779b0] [d0000000016a5b54] .do_ipt_get_ctl+0x110/0x2ec [ip_tables]
[ 91.139666] [c0000001fa577aa0] [c0000000006233e0] .nf_getsockopt+0x68/0x88
[ 91.139674] [c0000001fa577b40] [c000000000631608] .ip_getsockopt+0xbc/0x128
[ 91.139682] [c0000001fa577bf0] [c00000000065adf4] .raw_getsockopt+0x18/0x5c
[ 91.139690] [c0000001fa577c60] [c0000000005b5f60] .sock_common_getsockopt+0x2c/0x40
[ 91.139697] [c0000001fa577cd0] [c0000000005b3394] .__sys_getsockopt+0xa4/0xd0
[ 91.139704] [c0000001fa577d80] [c0000000005b5ab0] .__se_sys_socketcall+0x238/0x2b4
[ 91.139712] [c0000001fa577e30] [c00000000000a31c] system_call+0x5c/0x70
[ 91.139716] Instruction dump:
[ 91.139721] 39290040 7d3d4a14 7fbe4840 409cff98 81380000 2b890001 419d000c 393e0060
[ 91.139736] 48000010 7d57c82a e93e0060 7d295214 <815a0000> 794807e1 41e20010 7c210b78
[ 91.139752] ---[ end trace f5d1d5431651845d ]---
This is due to 7290d58095 ("module: use relative references for
__ksymtab entries"). This part of kernel/module.c -
/* Divert to percpu allocation if a percpu var. */
if (sym[i].st_shndx == info->index.pcpu)
secbase = (unsigned long)mod_percpu(mod);
else
secbase = info->sechdrs[sym[i].st_shndx].sh_addr;
sym[i].st_value += secbase;
Causes the distance to the target to exceed 32-bits on powerpc, so
it doesn't fit in a rel32 reloc. Not sure how other archs cope.
Thanks,
Nick
Hello Andreas, Nick,
On 28 August 2018 at 06:06, Nicholas Piggin [off-list ref] wro=
te: On Mon, 27 Aug 2018 19:11:01 +0200
Andreas Schwab [off-list ref] wrote:
quoted I'm getting this Oops when running iptables -F OUTPUT:
[ 91.139409] Unable to handle kernel paging request for data at addres= s 0xd0000001fff12f34 quoted [ 91.139414] Faulting instruction address: 0xd0000000016a5718
[ 91.139419] Oops: Kernel access of bad area, sig: 11 [#1]
[ 91.139426] BE SMP NR_CPUS=3D2 PowerMac
[ 91.139434] Modules linked in: iptable_filter ip_tables x_tables bpfi= lter nfsd auth_rpcgss lockd grace nfs_acl sunrpc tun af_packet snd_aoa_code=
c_tas snd_aoa_fabric_layout snd_aoa snd_aoa_i2sbus snd_aoa_soundbus snd_pcm=
_oss snd_pcm snd_seq snd_timer snd_seq_device snd_mixer_oss snd sungem sr_m=
od firewire_ohci cdrom sungem_phy soundcore firewire_core pata_macio crc_it=
u_t sg hid_generic usbhid linear md_mod ohci_pci ohci_hcd ehci_pci ehci_hcd=
usbcore usb_common dm_snapshot dm_bufio dm_mirror dm_region_hash dm_log dm=
_mod sata_svw quoted [ 91.139522] CPU: 1 PID: 3620 Comm: iptables Not tainted 4.19.0-rc1 #1
[ 91.139526] NIP: d0000000016a5718 LR: d0000000016a569c CTR: c0000000= 006f560c quoted [ 91.139531] REGS: c0000001fa577670 TRAP: 0300 Not tainted (4.19.0-= rc1) quoted [ 91.139534] MSR: 900000000200b032 <SF,HV,VEC,EE,FP,ME,IR,DR,RI> CR:= 84002484 XER: 20000000 quoted [ 91.139553] DAR: d0000001fff12f34 DSISR: 40000000 IRQMASK: 0
GPR00: d0000000016a569c c0000001fa5778f0 d0000000016b0400 00000000000000= 00 quoted GPR04: 0000000000000002 0000000000000000 80000001fa46418e c0000001fa0d05= c8 quoted GPR08: d0000000016b0400 d00037fffff13000 00000001ff3e7000 d0000000016a6f= b8 quoted GPR12: c0000000006f560c c00000000ffff780 0000000000000000 00000000000000= 00 quoted GPR16: 0000000011635010 00003fffa1b7aa68 0000000000000000 00000000000000= 00 quoted GPR20: 0000000000000003 0000000010013918 00000000116350c0 c000000000b889= 90 quoted GPR24: c000000000b88ba4 0000000000000000 d0000001fff12f34 00000000000000= 00 quoted GPR28: d0000000016b8000 c0000001fa20f400 c0000001fa20f440 00000000000000= 00 quoted [ 91.139627] NIP [d0000000016a5718] .alloc_counters.isra.10+0xbc/0x140= [ip_tables] quoted [ 91.139634] LR [d0000000016a569c] .alloc_counters.isra.10+0x40/0x140 = [ip_tables] quoted [ 91.139638] Call Trace:
[ 91.139645] [c0000001fa5778f0] [d0000000016a569c] .alloc_counters.isr= a.10+0x40/0x140 [ip_tables] (unreliable) quoted [ 91.139655] [c0000001fa5779b0] [d0000000016a5b54] .do_ipt_get_ctl+0x1= 10/0x2ec [ip_tables] quoted [ 91.139666] [c0000001fa577aa0] [c0000000006233e0] .nf_getsockopt+0x68= /0x88 quoted [ 91.139674] [c0000001fa577b40] [c000000000631608] .ip_getsockopt+0xbc= /0x128 quoted [ 91.139682] [c0000001fa577bf0] [c00000000065adf4] .raw_getsockopt+0x1= 8/0x5c quoted [ 91.139690] [c0000001fa577c60] [c0000000005b5f60] .sock_common_getsoc= kopt+0x2c/0x40 quoted [ 91.139697] [c0000001fa577cd0] [c0000000005b3394] .__sys_getsockopt+0= xa4/0xd0 quoted [ 91.139704] [c0000001fa577d80] [c0000000005b5ab0] .__se_sys_socketcal= l+0x238/0x2b4 quoted [ 91.139712] [c0000001fa577e30] [c00000000000a31c] system_call+0x5c/0x= 70 quoted [ 91.139716] Instruction dump:
[ 91.139721] 39290040 7d3d4a14 7fbe4840 409cff98 81380000 2b890001 419= d000c 393e0060 quoted [ 91.139736] 48000010 7d57c82a e93e0060 7d295214 <815a0000> 794807e1 4= 1e20010 7c210b78 quoted [ 91.139752] ---[ end trace f5d1d5431651845d ]---
This is due to 7290d58095 ("module: use relative references for
__ksymtab entries"). This part of kernel/module.c -
/* Divert to percpu allocation if a percpu var. */
if (sym[i].st_shndx =3D=3D info->index.pcpu)
secbase =3D (unsigned long)mod_percpu(mod);
else
secbase =3D info->sechdrs[sym[i].st_shndx].sh_addr;
sym[i].st_value +=3D secbase;
Causes the distance to the target to exceed 32-bits on powerpc, so
it doesn't fit in a rel32 reloc. Not sure how other archs cope.
Apologies for the breakage. It does indeed appear to affect all
architectures, and I'm a bit puzzled why you are the first one to spot
it.
I will try to find a clean way to special case the per-CPU variable
__ksymtab references in the generic module code, and if that is too
cumbersome, we can switch to 64-bit relative references (or rather,
native word size relative references) instead. Or revert the whole
thing ...
On 28 August 2018 at 15:56, Ard Biesheuvel [off-list ref] wrot=
e: Hello Andreas, Nick,
On 28 August 2018 at 06:06, Nicholas Piggin [off-list ref] w= rote: quoted On Mon, 27 Aug 2018 19:11:01 +0200
Andreas Schwab [off-list ref] wrote:
quoted I'm getting this Oops when running iptables -F OUTPUT:
[ 91.139409] Unable to handle kernel paging request for data at addre= ss 0xd0000001fff12f34 quoted quoted [ 91.139414] Faulting instruction address: 0xd0000000016a5718
[ 91.139419] Oops: Kernel access of bad area, sig: 11 [#1]
[ 91.139426] BE SMP NR_CPUS=3D2 PowerMac
[ 91.139434] Modules linked in: iptable_filter ip_tables x_tables bpf= ilter nfsd auth_rpcgss lockd grace nfs_acl sunrpc tun af_packet snd_aoa_cod=
ec_tas snd_aoa_fabric_layout snd_aoa snd_aoa_i2sbus snd_aoa_soundbus snd_pc=
m_oss snd_pcm snd_seq snd_timer snd_seq_device snd_mixer_oss snd sungem sr_=
mod firewire_ohci cdrom sungem_phy soundcore firewire_core pata_macio crc_i=
tu_t sg hid_generic usbhid linear md_mod ohci_pci ohci_hcd ehci_pci ehci_hc=
d usbcore usb_common dm_snapshot dm_bufio dm_mirror dm_region_hash dm_log d=
m_mod sata_svw quoted quoted [ 91.139522] CPU: 1 PID: 3620 Comm: iptables Not tainted 4.19.0-rc1 #= 1 quoted quoted [ 91.139526] NIP: d0000000016a5718 LR: d0000000016a569c CTR: c000000= 0006f560c quoted quoted [ 91.139531] REGS: c0000001fa577670 TRAP: 0300 Not tainted (4.19.0= -rc1) quoted quoted [ 91.139534] MSR: 900000000200b032 <SF,HV,VEC,EE,FP,ME,IR,DR,RI> CR= : 84002484 XER: 20000000 quoted quoted [ 91.139553] DAR: d0000001fff12f34 DSISR: 40000000 IRQMASK: 0
GPR00: d0000000016a569c c0000001fa5778f0 d0000000016b0400 0000000000000= 000 quoted quoted GPR04: 0000000000000002 0000000000000000 80000001fa46418e c0000001fa0d0= 5c8 quoted quoted GPR08: d0000000016b0400 d00037fffff13000 00000001ff3e7000 d0000000016a6= fb8 quoted quoted GPR12: c0000000006f560c c00000000ffff780 0000000000000000 0000000000000= 000 quoted quoted GPR16: 0000000011635010 00003fffa1b7aa68 0000000000000000 0000000000000= 000 quoted quoted GPR20: 0000000000000003 0000000010013918 00000000116350c0 c000000000b88= 990 quoted quoted GPR24: c000000000b88ba4 0000000000000000 d0000001fff12f34 0000000000000= 000 quoted quoted GPR28: d0000000016b8000 c0000001fa20f400 c0000001fa20f440 0000000000000= 000 quoted quoted [ 91.139627] NIP [d0000000016a5718] .alloc_counters.isra.10+0xbc/0x14= 0 [ip_tables] quoted quoted [ 91.139634] LR [d0000000016a569c] .alloc_counters.isra.10+0x40/0x140= [ip_tables] quoted quoted [ 91.139638] Call Trace:
[ 91.139645] [c0000001fa5778f0] [d0000000016a569c] .alloc_counters.is= ra.10+0x40/0x140 [ip_tables] (unreliable) quoted quoted [ 91.139655] [c0000001fa5779b0] [d0000000016a5b54] .do_ipt_get_ctl+0x= 110/0x2ec [ip_tables] quoted quoted [ 91.139666] [c0000001fa577aa0] [c0000000006233e0] .nf_getsockopt+0x6= 8/0x88 quoted quoted [ 91.139674] [c0000001fa577b40] [c000000000631608] .ip_getsockopt+0xb= c/0x128 quoted quoted [ 91.139682] [c0000001fa577bf0] [c00000000065adf4] .raw_getsockopt+0x= 18/0x5c quoted quoted [ 91.139690] [c0000001fa577c60] [c0000000005b5f60] .sock_common_getso= ckopt+0x2c/0x40 quoted quoted [ 91.139697] [c0000001fa577cd0] [c0000000005b3394] .__sys_getsockopt+= 0xa4/0xd0 quoted quoted [ 91.139704] [c0000001fa577d80] [c0000000005b5ab0] .__se_sys_socketca= ll+0x238/0x2b4 quoted quoted [ 91.139712] [c0000001fa577e30] [c00000000000a31c] system_call+0x5c/0= x70 quoted quoted [ 91.139716] Instruction dump:
[ 91.139721] 39290040 7d3d4a14 7fbe4840 409cff98 81380000 2b890001 41= 9d000c 393e0060 quoted quoted [ 91.139736] 48000010 7d57c82a e93e0060 7d295214 <815a0000> 794807e1 = 41e20010 7c210b78 quoted quoted [ 91.139752] ---[ end trace f5d1d5431651845d ]---
This is due to 7290d58095 ("module: use relative references for
__ksymtab entries"). This part of kernel/module.c -
/* Divert to percpu allocation if a percpu var. */
if (sym[i].st_shndx =3D=3D info->index.pcpu)
secbase =3D (unsigned long)mod_percpu(mod);
else
secbase =3D info->sechdrs[sym[i].st_shndx].sh_addr;
sym[i].st_value +=3D secbase;
Causes the distance to the target to exceed 32-bits on powerpc, so
it doesn't fit in a rel32 reloc. Not sure how other archs cope.
Apologies for the breakage. It does indeed appear to affect all
architectures, and I'm a bit puzzled why you are the first one to spot
it.
I will try to find a clean way to special case the per-CPU variable
__ksymtab references in the generic module code, and if that is too
cumbersome, we can switch to 64-bit relative references (or rather,
native word size relative references) instead. Or revert the whole
thing ...
OK, after a bit of digging, and confirming that the arm64
implementation works as expected (its module loader actually detects
overflows of the 32-bit place relative relocations, so the problem
definitely does not occur there), I think I found the explanation why
this occurs on powerpc and not on x86 or arm64.
Could you please check whether this change makes the issue go away?
(whitespace damage courtesy of Gmail)
diff --git a/arch/powerpc/kernel/setup_64.c b/arch/powerpc/kernel/setup_64.= c
index 6a501b25dd85..57d09d5ceb1a 100644 --- a/arch/powerpc/kernel/setup_64.c
+++ b/arch/powerpc/kernel/setup_64.c @@ -779,7 +779,6 @@ EXPORT_SYMBOL(__per_cpu_offset);
void __init setup_per_cpu_areas ( void )
{
- const size_t dyn_size = 3 D PERCPU_MODULE_RESERVE + PERCPU_DYNAMIC_RE = SERVE;
size_t atom_size;
unsigned long delta;
unsigned int cpu; @@ -795,7 +794,9 @@ void __init setup_per_cpu_areas(void)
else
atom_size =3D 1 << 20;
- rc =3D pcpu_embed_first_chunk(0, dyn_size, atom_size, pcpu_cpu_dist=ance,
+ rc =3D pcpu_embed_first_chunk(PERCPU_MODULE_RESERVE,
+ PERCPU_DYNAMIC_RESERVE,
+ atom_size, pcpu_cpu_distance,
pcpu_fc_alloc, pcpu_fc_free);
if (rc < 0)
panic("cannot initialize percpu area (err=3D%d)", rc);
The git log does not explain why power deviates from x86 and arm64 in
the way it initializes the percpu areas.
On Aug 28 2018, Ard Biesheuvel [off-list ref] wrote:
quoted hunk diff --git a/arch/powerpc/kernel/setup_64.c b/arch/powerpc/kernel/setup_64.c
index 6a501b25dd85..57d09d5ceb1a 100644
--- a/arch/powerpc/kernel/setup_64.c
+++ b/arch/powerpc/kernel/setup_64.c @@ -779,7 +779,6 @@ EXPORT_SYMBOL(__per_cpu_offset);
void __init setup_per_cpu_areas ( void )
{
- const size_t dyn_size = PERCPU_MODULE_RESERVE + PERCPU_DYNAMIC_RESERVE ; size_t atom_size ;
unsigned long delta ;
unsigned int cpu ; @@ -795,7 +794,9 @@ void __init setup_per_cpu_areas(void)
else
atom_size = 1 << 20 ;
- rc = pcpu_embed_first_chunk ( 0 , dyn_size , atom_size , pcpu_cpu_distance , + rc = pcpu_embed_first_chunk ( PERCPU_MODULE_RESERVE , + PERCPU_DYNAMIC_RESERVE , + atom_size , pcpu_cpu_distance , pcpu_fc_alloc , pcpu_fc_free );
if ( rc < 0 )
panic ( "cannot initialize percpu area (err=%d)" , rc );
That didn't help.
Andreas.
--
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510 2552 DF73 E780 A9DA AEC1
"And now for something completely different."
On Tue, 28 Aug 2018 14:06:32 +1000
Nicholas Piggin [off-list ref] wrote:
On Mon, 27 Aug 2018 19:11:01 +0200
Andreas Schwab [off-list ref] wrote:
quoted I'm getting this Oops when running iptables -F OUTPUT:
[ 91.139409] Unable to handle kernel paging request for data at address 0xd0000001fff12f34
[ 91.139414] Faulting instruction address: 0xd0000000016a5718
[ 91.139419] Oops: Kernel access of bad area, sig: 11 [#1]
[ 91.139426] BE SMP NR_CPUS=2 PowerMac
[ 91.139434] Modules linked in: iptable_filter ip_tables x_tables bpfilter nfsd auth_rpcgss lockd grace nfs_acl sunrpc tun af_packet snd_aoa_codec_tas snd_aoa_fabric_layout snd_aoa snd_aoa_i2sbus snd_aoa_soundbus snd_pcm_oss snd_pcm snd_seq snd_timer snd_seq_device snd_mixer_oss snd sungem sr_mod firewire_ohci cdrom sungem_phy soundcore firewire_core pata_macio crc_itu_t sg hid_generic usbhid linear md_mod ohci_pci ohci_hcd ehci_pci ehci_hcd usbcore usb_common dm_snapshot dm_bufio dm_mirror dm_region_hash dm_log dm_mod sata_svw
[ 91.139522] CPU: 1 PID: 3620 Comm: iptables Not tainted 4.19.0-rc1 #1
[ 91.139526] NIP: d0000000016a5718 LR: d0000000016a569c CTR: c0000000006f560c
[ 91.139531] REGS: c0000001fa577670 TRAP: 0300 Not tainted (4.19.0-rc1)
[ 91.139534] MSR: 900000000200b032 <SF,HV,VEC,EE,FP,ME,IR,DR,RI> CR: 84002484 XER: 20000000
[ 91.139553] DAR: d0000001fff12f34 DSISR: 40000000 IRQMASK: 0
GPR00: d0000000016a569c c0000001fa5778f0 d0000000016b0400 0000000000000000
GPR04: 0000000000000002 0000000000000000 80000001fa46418e c0000001fa0d05c8
GPR08: d0000000016b0400 d00037fffff13000 00000001ff3e7000 d0000000016a6fb8
GPR12: c0000000006f560c c00000000ffff780 0000000000000000 0000000000000000
GPR16: 0000000011635010 00003fffa1b7aa68 0000000000000000 0000000000000000
GPR20: 0000000000000003 0000000010013918 00000000116350c0 c000000000b88990
GPR24: c000000000b88ba4 0000000000000000 d0000001fff12f34 0000000000000000
GPR28: d0000000016b8000 c0000001fa20f400 c0000001fa20f440 0000000000000000
[ 91.139627] NIP [d0000000016a5718] .alloc_counters.isra.10+0xbc/0x140 [ip_tables]
[ 91.139634] LR [d0000000016a569c] .alloc_counters.isra.10+0x40/0x140 [ip_tables]
[ 91.139638] Call Trace:
[ 91.139645] [c0000001fa5778f0] [d0000000016a569c] .alloc_counters.isra.10+0x40/0x140 [ip_tables] (unreliable)
[ 91.139655] [c0000001fa5779b0] [d0000000016a5b54] .do_ipt_get_ctl+0x110/0x2ec [ip_tables]
[ 91.139666] [c0000001fa577aa0] [c0000000006233e0] .nf_getsockopt+0x68/0x88
[ 91.139674] [c0000001fa577b40] [c000000000631608] .ip_getsockopt+0xbc/0x128
[ 91.139682] [c0000001fa577bf0] [c00000000065adf4] .raw_getsockopt+0x18/0x5c
[ 91.139690] [c0000001fa577c60] [c0000000005b5f60] .sock_common_getsockopt+0x2c/0x40
[ 91.139697] [c0000001fa577cd0] [c0000000005b3394] .__sys_getsockopt+0xa4/0xd0
[ 91.139704] [c0000001fa577d80] [c0000000005b5ab0] .__se_sys_socketcall+0x238/0x2b4
[ 91.139712] [c0000001fa577e30] [c00000000000a31c] system_call+0x5c/0x70
[ 91.139716] Instruction dump:
[ 91.139721] 39290040 7d3d4a14 7fbe4840 409cff98 81380000 2b890001 419d000c 393e0060
[ 91.139736] 48000010 7d57c82a e93e0060 7d295214 <815a0000> 794807e1 41e20010 7c210b78
[ 91.139752] ---[ end trace f5d1d5431651845d ]---
This is due to 7290d58095 ("module: use relative references for
__ksymtab entries"). This part of kernel/module.c -
/* Divert to percpu allocation if a percpu var. */
if (sym[i].st_shndx == info->index.pcpu)
secbase = (unsigned long)mod_percpu(mod);
else
secbase = info->sechdrs[sym[i].st_shndx].sh_addr;
sym[i].st_value += secbase;
Causes the distance to the target to exceed 32-bits on powerpc, so
it doesn't fit in a rel32 reloc. Not sure how other archs cope.
Any progress on this one? I had a bit of a look but can't see a really
trivial fix and don't have a lot of time to work on it. Maybe use 64
bit relative offsets for per-cpu exports, or better might be apply the
per-cpu fixup when linking against the symbol rather than when writing
the module symbol table.
Until then I'd like to just remove HAVE_ARCH_PREL32_RELOCATIONS from
powerpc/Kconfig, but if other archs are going to have issues too, we
could just revert
271ca788774aa ("arch: enable relative relocations for arm64, power and x86")
arm64, x86 -- can the distance between your module percpu data link
location -> module percpu runtime allocation location exceed 31 bits?
Thanks,
Nick
On Wed, 29 Aug 2018 13:28:27 +1000
Nicholas Piggin [off-list ref] wrote:
On Tue, 28 Aug 2018 14:06:32 +1000
Nicholas Piggin [off-list ref] wrote:
quoted On Mon, 27 Aug 2018 19:11:01 +0200
Andreas Schwab [off-list ref] wrote:
quoted I'm getting this Oops when running iptables -F OUTPUT:
[ 91.139409] Unable to handle kernel paging request for data at address 0xd0000001fff12f34
[ 91.139414] Faulting instruction address: 0xd0000000016a5718
[ 91.139419] Oops: Kernel access of bad area, sig: 11 [#1]
[ 91.139426] BE SMP NR_CPUS=2 PowerMac
[ 91.139434] Modules linked in: iptable_filter ip_tables x_tables bpfilter nfsd auth_rpcgss lockd grace nfs_acl sunrpc tun af_packet snd_aoa_codec_tas snd_aoa_fabric_layout snd_aoa snd_aoa_i2sbus snd_aoa_soundbus snd_pcm_oss snd_pcm snd_seq snd_timer snd_seq_device snd_mixer_oss snd sungem sr_mod firewire_ohci cdrom sungem_phy soundcore firewire_core pata_macio crc_itu_t sg hid_generic usbhid linear md_mod ohci_pci ohci_hcd ehci_pci ehci_hcd usbcore usb_common dm_snapshot dm_bufio dm_mirror dm_region_hash dm_log dm_mod sata_svw
[ 91.139522] CPU: 1 PID: 3620 Comm: iptables Not tainted 4.19.0-rc1 #1
[ 91.139526] NIP: d0000000016a5718 LR: d0000000016a569c CTR: c0000000006f560c
[ 91.139531] REGS: c0000001fa577670 TRAP: 0300 Not tainted (4.19.0-rc1)
[ 91.139534] MSR: 900000000200b032 <SF,HV,VEC,EE,FP,ME,IR,DR,RI> CR: 84002484 XER: 20000000
[ 91.139553] DAR: d0000001fff12f34 DSISR: 40000000 IRQMASK: 0
GPR00: d0000000016a569c c0000001fa5778f0 d0000000016b0400 0000000000000000
GPR04: 0000000000000002 0000000000000000 80000001fa46418e c0000001fa0d05c8
GPR08: d0000000016b0400 d00037fffff13000 00000001ff3e7000 d0000000016a6fb8
GPR12: c0000000006f560c c00000000ffff780 0000000000000000 0000000000000000
GPR16: 0000000011635010 00003fffa1b7aa68 0000000000000000 0000000000000000
GPR20: 0000000000000003 0000000010013918 00000000116350c0 c000000000b88990
GPR24: c000000000b88ba4 0000000000000000 d0000001fff12f34 0000000000000000
GPR28: d0000000016b8000 c0000001fa20f400 c0000001fa20f440 0000000000000000
[ 91.139627] NIP [d0000000016a5718] .alloc_counters.isra.10+0xbc/0x140 [ip_tables]
[ 91.139634] LR [d0000000016a569c] .alloc_counters.isra.10+0x40/0x140 [ip_tables]
[ 91.139638] Call Trace:
[ 91.139645] [c0000001fa5778f0] [d0000000016a569c] .alloc_counters.isra.10+0x40/0x140 [ip_tables] (unreliable)
[ 91.139655] [c0000001fa5779b0] [d0000000016a5b54] .do_ipt_get_ctl+0x110/0x2ec [ip_tables]
[ 91.139666] [c0000001fa577aa0] [c0000000006233e0] .nf_getsockopt+0x68/0x88
[ 91.139674] [c0000001fa577b40] [c000000000631608] .ip_getsockopt+0xbc/0x128
[ 91.139682] [c0000001fa577bf0] [c00000000065adf4] .raw_getsockopt+0x18/0x5c
[ 91.139690] [c0000001fa577c60] [c0000000005b5f60] .sock_common_getsockopt+0x2c/0x40
[ 91.139697] [c0000001fa577cd0] [c0000000005b3394] .__sys_getsockopt+0xa4/0xd0
[ 91.139704] [c0000001fa577d80] [c0000000005b5ab0] .__se_sys_socketcall+0x238/0x2b4
[ 91.139712] [c0000001fa577e30] [c00000000000a31c] system_call+0x5c/0x70
[ 91.139716] Instruction dump:
[ 91.139721] 39290040 7d3d4a14 7fbe4840 409cff98 81380000 2b890001 419d000c 393e0060
[ 91.139736] 48000010 7d57c82a e93e0060 7d295214 <815a0000> 794807e1 41e20010 7c210b78
[ 91.139752] ---[ end trace f5d1d5431651845d ]---
This is due to 7290d58095 ("module: use relative references for
__ksymtab entries"). This part of kernel/module.c -
/* Divert to percpu allocation if a percpu var. */
if (sym[i].st_shndx == info->index.pcpu)
secbase = (unsigned long)mod_percpu(mod);
else
secbase = info->sechdrs[sym[i].st_shndx].sh_addr;
sym[i].st_value += secbase;
Causes the distance to the target to exceed 32-bits on powerpc, so
it doesn't fit in a rel32 reloc. Not sure how other archs cope.
Any progress on this one? I had a bit of a look but can't see a really
trivial fix and don't have a lot of time to work on it. Maybe use 64
bit relative offsets for per-cpu exports, or better might be apply the
per-cpu fixup when linking against the symbol rather than when writing
the module symbol table.
Until then I'd like to just remove HAVE_ARCH_PREL32_RELOCATIONS from
powerpc/Kconfig, but if other archs are going to have issues too, we
could just revert
271ca788774aa ("arch: enable relative relocations for arm64, power and x86")
arm64, x86 -- can the distance between your module percpu data link
location -> module percpu runtime allocation location exceed 31 bits?
[Sorry ignore this, I missed some mail, will reply in the thread]
Thanks,
Nick
On Tue, 28 Aug 2018 18:09:09 +0200
Ard Biesheuvel [off-list ref] wrote:
quoted hunk On 28 August 2018 at 15:56, Ard Biesheuvel [off-list ref] wrote: quoted Hello Andreas, Nick,
On 28 August 2018 at 06:06, Nicholas Piggin [off-list ref] wrote: quoted On Mon, 27 Aug 2018 19:11:01 +0200
Andreas Schwab [off-list ref] wrote:
quoted I'm getting this Oops when running iptables -F OUTPUT:
[ 91.139409] Unable to handle kernel paging request for data at address 0xd0000001fff12f34
[ 91.139414] Faulting instruction address: 0xd0000000016a5718
[ 91.139419] Oops: Kernel access of bad area, sig: 11 [#1]
[ 91.139426] BE SMP NR_CPUS=2 PowerMac
[ 91.139434] Modules linked in: iptable_filter ip_tables x_tables bpfilter nfsd auth_rpcgss lockd grace nfs_acl sunrpc tun af_packet snd_aoa_codec_tas snd_aoa_fabric_layout snd_aoa snd_aoa_i2sbus snd_aoa_soundbus snd_pcm_oss snd_pcm snd_seq snd_timer snd_seq_device snd_mixer_oss snd sungem sr_mod firewire_ohci cdrom sungem_phy soundcore firewire_core pata_macio crc_itu_t sg hid_generic usbhid linear md_mod ohci_pci ohci_hcd ehci_pci ehci_hcd usbcore usb_common dm_snapshot dm_bufio dm_mirror dm_region_hash dm_log dm_mod sata_svw
[ 91.139522] CPU: 1 PID: 3620 Comm: iptables Not tainted 4.19.0-rc1 #1
[ 91.139526] NIP: d0000000016a5718 LR: d0000000016a569c CTR: c0000000006f560c
[ 91.139531] REGS: c0000001fa577670 TRAP: 0300 Not tainted (4.19.0-rc1)
[ 91.139534] MSR: 900000000200b032 <SF,HV,VEC,EE,FP,ME,IR,DR,RI> CR: 84002484 XER: 20000000
[ 91.139553] DAR: d0000001fff12f34 DSISR: 40000000 IRQMASK: 0
GPR00: d0000000016a569c c0000001fa5778f0 d0000000016b0400 0000000000000000
GPR04: 0000000000000002 0000000000000000 80000001fa46418e c0000001fa0d05c8
GPR08: d0000000016b0400 d00037fffff13000 00000001ff3e7000 d0000000016a6fb8
GPR12: c0000000006f560c c00000000ffff780 0000000000000000 0000000000000000
GPR16: 0000000011635010 00003fffa1b7aa68 0000000000000000 0000000000000000
GPR20: 0000000000000003 0000000010013918 00000000116350c0 c000000000b88990
GPR24: c000000000b88ba4 0000000000000000 d0000001fff12f34 0000000000000000
GPR28: d0000000016b8000 c0000001fa20f400 c0000001fa20f440 0000000000000000
[ 91.139627] NIP [d0000000016a5718] .alloc_counters.isra.10+0xbc/0x140 [ip_tables]
[ 91.139634] LR [d0000000016a569c] .alloc_counters.isra.10+0x40/0x140 [ip_tables]
[ 91.139638] Call Trace:
[ 91.139645] [c0000001fa5778f0] [d0000000016a569c] .alloc_counters.isra.10+0x40/0x140 [ip_tables] (unreliable)
[ 91.139655] [c0000001fa5779b0] [d0000000016a5b54] .do_ipt_get_ctl+0x110/0x2ec [ip_tables]
[ 91.139666] [c0000001fa577aa0] [c0000000006233e0] .nf_getsockopt+0x68/0x88
[ 91.139674] [c0000001fa577b40] [c000000000631608] .ip_getsockopt+0xbc/0x128
[ 91.139682] [c0000001fa577bf0] [c00000000065adf4] .raw_getsockopt+0x18/0x5c
[ 91.139690] [c0000001fa577c60] [c0000000005b5f60] .sock_common_getsockopt+0x2c/0x40
[ 91.139697] [c0000001fa577cd0] [c0000000005b3394] .__sys_getsockopt+0xa4/0xd0
[ 91.139704] [c0000001fa577d80] [c0000000005b5ab0] .__se_sys_socketcall+0x238/0x2b4
[ 91.139712] [c0000001fa577e30] [c00000000000a31c] system_call+0x5c/0x70
[ 91.139716] Instruction dump:
[ 91.139721] 39290040 7d3d4a14 7fbe4840 409cff98 81380000 2b890001 419d000c 393e0060
[ 91.139736] 48000010 7d57c82a e93e0060 7d295214 <815a0000> 794807e1 41e20010 7c210b78
[ 91.139752] ---[ end trace f5d1d5431651845d ]---
This is due to 7290d58095 ("module: use relative references for
__ksymtab entries"). This part of kernel/module.c -
/* Divert to percpu allocation if a percpu var. */
if (sym[i].st_shndx == info->index.pcpu)
secbase = (unsigned long)mod_percpu(mod);
else
secbase = info->sechdrs[sym[i].st_shndx].sh_addr;
sym[i].st_value += secbase;
Causes the distance to the target to exceed 32-bits on powerpc, so
it doesn't fit in a rel32 reloc. Not sure how other archs cope.
Apologies for the breakage. It does indeed appear to affect all
architectures, and I'm a bit puzzled why you are the first one to spot
it.
I will try to find a clean way to special case the per-CPU variable
__ksymtab references in the generic module code, and if that is too
cumbersome, we can switch to 64-bit relative references (or rather,
native word size relative references) instead. Or revert the whole
thing ...
OK, after a bit of digging, and confirming that the arm64
implementation works as expected (its module loader actually detects
overflows of the 32-bit place relative relocations, so the problem
definitely does not occur there), I think I found the explanation why
this occurs on powerpc and not on x86 or arm64.
Could you please check whether this change makes the issue go away?
(whitespace damage courtesy of Gmail)
diff --git a/arch/powerpc/kernel/setup_64.c b/arch/powerpc/kernel/setup_64.c
index 6a501b25dd85..57d09d5ceb1a 100644
--- a/arch/powerpc/kernel/setup_64.c
+++ b/arch/powerpc/kernel/setup_64.c @@ -779,7 +779,6 @@ EXPORT_SYMBOL(__per_cpu_offset);
void __init setup_per_cpu_areas ( void )
{
- const size_t dyn_size = PERCPU_MODULE_RESERVE + PERCPU_DYNAMIC_RESERVE ; size_t atom_size ;
unsigned long delta ;
unsigned int cpu ; @@ -795,7 +794,9 @@ void __init setup_per_cpu_areas(void)
else
atom_size = 1 << 20 ;
- rc = pcpu_embed_first_chunk ( 0 , dyn_size , atom_size , pcpu_cpu_distance , + rc = pcpu_embed_first_chunk ( PERCPU_MODULE_RESERVE , + PERCPU_DYNAMIC_RESERVE , + atom_size , pcpu_cpu_distance , pcpu_fc_alloc , pcpu_fc_free );
if ( rc < 0 )
panic ( "cannot initialize percpu area (err=%d)" , rc );
The git log does not explain why power deviates from x86 and arm64 in
the way it initializes the percpu areas.
The reason for 64-bit powerpc is actually that modules are allocated in
vmalloc space which is a long way out from the linear map where the per
cpu embedded chunk is.
It does look like x86 and arm64 are probably okay because they set up a
module vmalloc area close to their kernel text in the linear map, which
should be close to per-cpu I guess.
I'm not entirely sure why pcpu setup is different on powerpc, but I
think the module vmalloc addresses bite first anyway.
Okay I'd say let's just remove powerpc for now.
Thanks,
Nick