From: Gerhard Wiesinger <hidden> Date: 2015-10-25 10:49:08
On 25.10.2015 10:46, Willy Tarreau wrote:
ipset *triggered* the problem. The whole stack dump would tell more.
OK, find the stack traces in the bug report:
https://bugzilla.redhat.com/show_bug.cgi?id=1272645
Kernel 4.1.10 triggered also a kernel dump when playing with ipset
commands and IPv6, details in the bug report ....
On Sun, Oct 25, 2015 at 11:48:54AM +0100, Gerhard Wiesinger wrote:
On 25.10.2015 10:46, Willy Tarreau wrote:
quoted
ipset *triggered* the problem. The whole stack dump would tell more.
OK, find the stack traces in the bug report:
https://bugzilla.redhat.com/show_bug.cgi?id=1272645
Kernel 4.1.10 triggered also a kernel dump when playing with ipset commands
and IPv6, details in the bug report ....
There's a reason why Greg maintains stable and LTS kernels :-)
Stable kernels don't crash but definiton. :-)
At least triggered 2 kernel panics in 5min, even with 4.1.10 and ipset
commands ...
Does this happen also with Linus's tree? I suggest you ask the
networking developers about this on netdev@vger.kernel.org, there's
nothing that I can do on my own about this, sorry.
greg k-h
From: Gerhard Wiesinger <hidden> Date: 2015-10-25 17:15:16
On 25.10.2015 17:29, Greg KH wrote:
On Sun, Oct 25, 2015 at 11:48:54AM +0100, Gerhard Wiesinger wrote:
quoted
On 25.10.2015 10:46, Willy Tarreau wrote:
quoted
ipset *triggered* the problem. The whole stack dump would tell more.
OK, find the stack traces in the bug report:
https://bugzilla.redhat.com/show_bug.cgi?id=1272645
Kernel 4.1.10 triggered also a kernel dump when playing with ipset commands
and IPv6, details in the bug report ....
There's a reason why Greg maintains stable and LTS kernels :-)
Stable kernels don't crash but definiton. :-)
At least triggered 2 kernel panics in 5min, even with 4.1.10 and ipset
commands ...
Does this happen also with Linus's tree? I suggest you ask the
networking developers about this on netdev@vger.kernel.org, there's
nothing that I can do on my own about this, sorry.
Already CCed netdev and netfilter-devel mailinglist. Need patches for
the switch driver of the banana Pi to get networking up but that patch
is stable. Maybe also some patches from the Fedora SRPMS are needed. But
I'm pretty sure that this also happens with plain vanilla kernel.
Ciao,
Gerhard
ipset *triggered* the problem. The whole stack dump would tell more.
OK, find the stack traces in the bug report:
https://bugzilla.redhat.com/show_bug.cgi?id=1272645
Kernel 4.1.10 triggered also a kernel dump when playing with ipset commands
and IPv6, details in the bug report ....
It seems to me it is an architecture-specific alignment issue. I don't
have a Cortex-A7 ARM hardware and qemu doesn't seem to support it either,
so I'm unable to reproduce it (ipset passes all my tests on my hardware,
including more complex ones than what breaks here). My first wild guess is
that the dynamic array of the element structure is not aligned properly.
Could you give a try to the next patch?
If that does not solve it, then could you help to narrow down the issue?
Does the bug still appear if your remove the counter extension of the set?
Best regards,
Jozsef
-
E-mail : kadlec@blackhole.kfki.hu, kadlecsik.jozsef@wigner.mta.hu
PGP key : http://www.kfki.hu/~kadlec/pgp_public_key.txt
Address : Wigner Research Centre for Physics, Hungarian Academy of Sciences
H-1525 Budapest 114, POB. 49, Hungary
From: Gerhard Wiesinger <hidden> Date: 2015-10-25 20:09:15
On 25.10.2015 20:46, Jozsef Kadlecsik wrote:
quoted hunk
Hi,
On Sun, 25 Oct 2015, Gerhard Wiesinger wrote:
quoted
On 25.10.2015 10:46, Willy Tarreau wrote:
quoted
ipset *triggered* the problem. The whole stack dump would tell more.
OK, find the stack traces in the bug report:
https://bugzilla.redhat.com/show_bug.cgi?id=1272645
Kernel 4.1.10 triggered also a kernel dump when playing with ipset commands
and IPv6, details in the bug report ....
It seems to me it is an architecture-specific alignment issue. I don't
have a Cortex-A7 ARM hardware and qemu doesn't seem to support it either,
so I'm unable to reproduce it (ipset passes all my tests on my hardware,
including more complex ones than what breaks here). My first wild guess is
that the dynamic array of the element structure is not aligned properly.
Could you give a try to the next patch?
If that does not solve it, then could you help to narrow down the issue?
Does the bug still appear if your remove the counter extension of the set?
Hello Jozsef,
Patch applied well, compiling ...
Interesting, that it didn't happen before. Device is in production for
more than 2 month without any issue.
Also any idea regarding the second isssue? Or do you think it has the
same root cause?
Greetings from Vienna, Austria :-)
BTW: You can get the Banana Pi R1 for example at:
http://www.aliexpress.com/item/BPI-R1-Set-1-R1-Board-Clear-Case-5dB-Antenna-Power-Adapter-Banana-PI-R1-Smart/32362127917.html
I can really recommend it as a router. Power consumption is as less as
3W. Price is also IMHO very good.
Ciao,
Gerhard
From: Gerhard Wiesinger <hidden> Date: 2015-10-25 21:26:33
On 25.10.2015 21:08, Gerhard Wiesinger wrote:
On 25.10.2015 20:46, Jozsef Kadlecsik wrote:
quoted
Hi,
On Sun, 25 Oct 2015, Gerhard Wiesinger wrote:
quoted
On 25.10.2015 10:46, Willy Tarreau wrote:
quoted
ipset *triggered* the problem. The whole stack dump would tell more.
OK, find the stack traces in the bug report:
https://bugzilla.redhat.com/show_bug.cgi?id=1272645
Kernel 4.1.10 triggered also a kernel dump when playing with ipset
commands
and IPv6, details in the bug report ....
It seems to me it is an architecture-specific alignment issue. I don't
have a Cortex-A7 ARM hardware and qemu doesn't seem to support it
either,
so I'm unable to reproduce it (ipset passes all my tests on my hardware,
including more complex ones than what breaks here). My first wild
guess is
that the dynamic array of the element structure is not aligned properly.
Could you give a try to the next patch?
@@ -1319,12 +1322,12 @@ IPSET_TOKEN(HTYPE, _create)(struct net *net,
struct ip_set *set,
#endif
set->variant = &IPSET_TOKEN(HTYPE, 4_variant);
set->dsize = ip_set_elem_len(set, tb,
- sizeof(struct IPSET_TOKEN(HTYPE, 4_elem)));
+ IP_SET_BASE_ALIGN(IPSET_TOKEN(HTYPE, 4_elem)));
#ifndef IP_SET_PROTO_UNDEF
} else {
set->variant = &IPSET_TOKEN(HTYPE, 6_variant);
set->dsize = ip_set_elem_len(set, tb,
- sizeof(struct IPSET_TOKEN(HTYPE, 6_elem)));
+ IP_SET_BASE_ALIGN(IPSET_TOKEN(HTYPE, 6_elem)));
}
#endif
if (tb[IPSET_ATTR_TIMEOUT]) {
If that does not solve it, then could you help to narrow down the issue?
Does the bug still appear if your remove the counter extension of the
set?
ipset *triggered* the problem. The whole stack dump would tell more.
OK, find the stack traces in the bug report:
https://bugzilla.redhat.com/show_bug.cgi?id=1272645
Kernel 4.1.10 triggered also a kernel dump when playing with ipset
commands
and IPv6, details in the bug report ....
It seems to me it is an architecture-specific alignment issue. I don't
have a Cortex-A7 ARM hardware and qemu doesn't seem to support it either,
so I'm unable to reproduce it (ipset passes all my tests on my hardware,
including more complex ones than what breaks here). My first wild guess is
that the dynamic array of the element structure is not aligned properly.
Could you give a try to the next patch?
@@ -1319,12 +1322,12 @@ IPSET_TOKEN(HTYPE, _create)(struct net *net,
struct ip_set *set,
#endif
set->variant = &IPSET_TOKEN(HTYPE, 4_variant);
set->dsize = ip_set_elem_len(set, tb,
- sizeof(struct IPSET_TOKEN(HTYPE, 4_elem)));
+ IP_SET_BASE_ALIGN(IPSET_TOKEN(HTYPE, 4_elem)));
#ifndef IP_SET_PROTO_UNDEF
} else {
set->variant = &IPSET_TOKEN(HTYPE, 6_variant);
set->dsize = ip_set_elem_len(set, tb,
- sizeof(struct IPSET_TOKEN(HTYPE, 6_elem)));
+ IP_SET_BASE_ALIGN(IPSET_TOKEN(HTYPE, 6_elem)));
}
#endif
if (tb[IPSET_ATTR_TIMEOUT]) {
If that does not solve it, then could you help to narrow down the issue?
Does the bug still appear if your remove the counter extension of the set?
Does it crash without counters? That could narrow down where to look for.
Best regards,
Jozsef
-
E-mail : kadlec@blackhole.kfki.hu, kadlecsik.jozsef@wigner.mta.hu
PGP key : http://www.kfki.hu/~kadlec/pgp_public_key.txt
Address : Wigner Research Centre for Physics, Hungarian Academy of Sciences
H-1525 Budapest 114, POB. 49, Hungary
From: Gerhard Wiesinger <hidden> Date: 2015-10-26 07:27:28
On 25.10.2015 22:53, Jozsef Kadlecsik wrote:
On Sun, 25 Oct 2015, Gerhard Wiesinger wrote:
quoted
Any further ideas?
Does it crash without counters? That could narrow down where to look for.
Hello Jozsef,
it doesn't crash i I don't use the counters so far. So there must be a
bug with the counters.
Any idea for the root cause?
Thnx.
Ciao,
Gerhard
ipset *triggered* the problem. The whole stack dump would tell more.
OK, find the stack traces in the bug report:
https://bugzilla.redhat.com/show_bug.cgi?id=1272645
Kernel 4.1.10 triggered also a kernel dump when playing with ipset
commands
and IPv6, details in the bug report ....
It seems to me it is an architecture-specific alignment issue. I don't
have a Cortex-A7 ARM hardware and qemu doesn't seem to support it either,
so I'm unable to reproduce it (ipset passes all my tests on my hardware,
including more complex ones than what breaks here). My first wild guess is
that the dynamic array of the element structure is not aligned properly.
Could you give a try to the next patch?
@@ -1319,12 +1322,12 @@ IPSET_TOKEN(HTYPE, _create)(struct net *net, struct
ip_set *set,
#endif
set->variant = &IPSET_TOKEN(HTYPE, 4_variant);
set->dsize = ip_set_elem_len(set, tb,
- sizeof(struct IPSET_TOKEN(HTYPE, 4_elem)));
+ IP_SET_BASE_ALIGN(IPSET_TOKEN(HTYPE,
4_elem)));
#ifndef IP_SET_PROTO_UNDEF
} else {
set->variant = &IPSET_TOKEN(HTYPE, 6_variant);
set->dsize = ip_set_elem_len(set, tb,
- sizeof(struct IPSET_TOKEN(HTYPE, 6_elem)));
+ IP_SET_BASE_ALIGN(IPSET_TOKEN(HTYPE,
6_elem)));
}
#endif
if (tb[IPSET_ATTR_TIMEOUT]) {
If that does not solve it, then could you help to narrow down the issue?
Does the bug still appear if your remove the counter extension of the set?
Patch applied well, compiling ...
Interesting, that it didn't happen before. Device is in production for
more than 2 month without any issue.
You mean the device was stable with the earlier kernels, but starting with
4.2.3 (and back to 4.1.10) you have got problems, don't you?
Also any idea regarding the second isssue? Or do you think it has the
same root cause?
Looking at your RedHat bugzilla report, the "nf_conntrack: table full,
dropping packet" and "Alignment trap: not handling instruction" are two
unrelated issues and the second one is triggered by the unaligned counter
extension acccess in ipset, I'm investigating. I can't think of any reason
how those issues could be related to each other.
Cool mini gear, indeed!
Best regards,
Jozsef
-
E-mail : kadlec@blackhole.kfki.hu, kadlecsik.jozsef@wigner.mta.hu
PGP key : http://www.kfki.hu/~kadlec/pgp_public_key.txt
Address : Wigner Research Centre for Physics, Hungarian Academy of Sciences
H-1525 Budapest 114, POB. 49, Hungary
From: Gerhard Wiesinger <hidden> Date: 2015-10-26 09:11:27
On 26.10.2015 09:58, Jozsef Kadlecsik wrote:
On Sun, 25 Oct 2015, Gerhard Wiesinger wrote:
quoted
Also any idea regarding the second isssue? Or do you think it has the
same root cause?
Looking at your RedHat bugzilla report, the "nf_conntrack: table full,
dropping packet" and "Alignment trap: not handling instruction" are two
unrelated issues and the second one is triggered by the unaligned counter
extension acccess in ipset, I'm investigating. I can't think of any reason
how those issues could be related to each other.
Yes, they are unrelated.
Issue 1: nf_conntrack: table full, dropping packet => Fixed with 4.2.4
Issue 2: Alignment trap: not handling instruction => Happens when ipset
counters are enabled
Please keep in mind it happens with IPv6 commands.
Currently 4.2.4 without ipset counters runs well.
Ciao,
Gerhard
From: Gerhard Wiesinger <hidden> Date: 2015-11-08 13:51:13
On 25.10.2015 17:29, Greg KH wrote:
On Sun, Oct 25, 2015 at 11:48:54AM +0100, Gerhard Wiesinger wrote:
quoted
On 25.10.2015 10:46, Willy Tarreau wrote:
quoted
ipset *triggered* the problem. The whole stack dump would tell more.
OK, find the stack traces in the bug report:
https://bugzilla.redhat.com/show_bug.cgi?id=1272645
Kernel 4.1.10 triggered also a kernel dump when playing with ipset commands
and IPv6, details in the bug report ....
There's a reason why Greg maintains stable and LTS kernels :-)
Stable kernels don't crash but definiton. :-)
At least triggered 2 kernel panics in 5min, even with 4.1.10 and ipset
commands ...
Does this happen also with Linus's tree? I suggest you ask the
networking developers about this on netdev@vger.kernel.org, there's
nothing that I can do on my own about this, sorry.
On Sun, Nov 08, 2015 at 02:51:01PM +0100, Gerhard Wiesinger wrote:
On 25.10.2015 17:29, Greg KH wrote:
quoted
On Sun, Oct 25, 2015 at 11:48:54AM +0100, Gerhard Wiesinger wrote:
quoted
On 25.10.2015 10:46, Willy Tarreau wrote:
quoted
ipset *triggered* the problem. The whole stack dump would tell more.
OK, find the stack traces in the bug report:
https://bugzilla.redhat.com/show_bug.cgi?id=1272645
Kernel 4.1.10 triggered also a kernel dump when playing with ipset commands
and IPv6, details in the bug report ....
There's a reason why Greg maintains stable and LTS kernels :-)
Stable kernels don't crash but definiton. :-)
At least triggered 2 kernel panics in 5min, even with 4.1.10 and ipset
commands ...
Does this happen also with Linus's tree? I suggest you ask the
networking developers about this on netdev@vger.kernel.org, there's
nothing that I can do on my own about this, sorry.
From: Gerhard Wiesinger <hidden> Date: 2015-11-09 12:35:25
On 08.11.2015 18:20, Greg KH wrote:
On Sun, Nov 08, 2015 at 02:51:01PM +0100, Gerhard Wiesinger wrote:
quoted
On 25.10.2015 17:29, Greg KH wrote:
quoted
On Sun, Oct 25, 2015 at 11:48:54AM +0100, Gerhard Wiesinger wrote:
quoted
On 25.10.2015 10:46, Willy Tarreau wrote:
quoted
ipset *triggered* the problem. The whole stack dump would tell more.
OK, find the stack traces in the bug report:
https://bugzilla.redhat.com/show_bug.cgi?id=1272645
Kernel 4.1.10 triggered also a kernel dump when playing with ipset commands
and IPv6, details in the bug report ....
There's a reason why Greg maintains stable and LTS kernels :-)
Stable kernels don't crash but definiton. :-)
At least triggered 2 kernel panics in 5min, even with 4.1.10 and ipset
commands ...
Does this happen also with Linus's tree? I suggest you ask the
networking developers about this on netdev@vger.kernel.org, there's
nothing that I can do on my own about this, sorry.