bug while receiving SNMP replies (UDP)?
From: folkert <hidden>
Date: 2022-04-07 20:33:35
Hi, I'm developing my own IP-stack (for fun, nothing serious). It also features an snmp-agent. Now something strange is happening when doing an snmpwalk. This does not happen everywhere (e.g. not all friends with Linux I asked to try it so could confirm the problem) and not always but quite often. I see it happening with at least kernel 5.16.0-6-amd64 and linux-image-4.19.0-17-amd64. Also intel or raspberry pi no difference. What is happening: 21:02:18 57d/1z root@mauer:~ 47s 148 # snmpwalk -c public -v 1 myip.vanheusden.com iso iso.3.6.1.2.1.1.1.0 = STRING: "MyIP - an IP-stack implemented in C++ running in userspace" iso.3.6.1.2.1.1.2.0 = OID: iso.3.6.1.2.1.4.57850.1 iso.3.6.1.2.1.1.3.0 = Timeticks: (938172) 2:36:21.72 Timeout: No Response from myip.vanheusden.com So you see that it times-out after 3 snmp 'getnext's. The strange thing here now is though: if I do tcpdump, I see the replies come in. If I then strace snmpwalk I see the UDP packets not come in there (pselect times out). dmesg shows no errors or warnings. 21:02:21.935486 IP 192.168.178.2.55233 > 185.243.112.68.161: GetNextRequest(21) .0.1 21:02:21.943187 IP 185.243.112.68.161 > 192.168.178.2.55233: GetResponse(92) .1.3.6.1.2.1.1.1.0="MyIP - an IP-stack implemented in C++ running in userspace" 21:02:21.943523 IP 192.168.178.2.55233 > 185.243.112.68.161: GetNextRequest(28) .1.3.6.1.2.1.1.1.0 21:02:21.949822 IP 185.243.112.68.161 > 192.168.178.2.55233: GetResponse(44) .1.3.6.1.2.1.1.2.0=.1.3.6.1.2.1.4.57850.1 21:02:21.950183 IP 192.168.178.2.55233 > 185.243.112.68.161: GetNextRequest(28) .1.3.6.1.2.1.1.2.0 21:02:21.957688 IP 185.243.112.68.161 > 192.168.178.2.55233: GetResponse(38) .1.3.6.1.2.1.1.3.0=938172 21:02:21.957956 IP 192.168.178.2.55233 > 185.243.112.68.161: GetNextRequest(28) .1.3.6.1.2.1.1.3.0 21:02:21.965300 IP 185.243.112.68.161 > 192.168.178.2.55233: GetResponse(75) .1.3.6.1.2.1.1.4.0="Folkert van Heusden [off-list ref]" 21:02:22.958318 IP 192.168.178.2.55233 > 185.243.112.68.161: GetNextRequest(28) .1.3.6.1.2.1.1.3.0 21:02:22.966724 IP 185.243.112.68.161 > 192.168.178.2.55233: GetResponse(75) .1.3.6.1.2.1.1.4.0="Folkert van Heusden [off-list ref]" 21:02:23.959428 IP 192.168.178.2.55233 > 185.243.112.68.161: GetNextRequest(28) .1.3.6.1.2.1.1.3.0 21:02:23.967058 IP 185.243.112.68.161 > 192.168.178.2.55233: GetResponse(75) .1.3.6.1.2.1.1.4.0="Folkert van Heusden [off-list ref]" 21:02:24.960165 IP 192.168.178.2.55233 > 185.243.112.68.161: GetNextRequest(28) .1.3.6.1.2.1.1.3.0 21:02:24.969778 IP 185.243.112.68.161 > 192.168.178.2.55233: GetResponse(75) .1.3.6.1.2.1.1.4.0="Folkert van Heusden [off-list ref]" 21:02:25.961346 IP 192.168.178.2.55233 > 185.243.112.68.161: GetNextRequest(28) .1.3.6.1.2.1.1.3.0 21:02:25.967545 IP 185.243.112.68.161 > 192.168.178.2.55233: GetResponse(75) .1.3.6.1.2.1.1.4.0="Folkert van Heusden [off-list ref]" 21:02:26.962047 IP 192.168.178.2.55233 > 185.243.112.68.161: GetNextRequest(28) .1.3.6.1.2.1.1.3.0 21:02:26.969202 IP 185.243.112.68.161 > 192.168.178.2.55233: GetResponse(75) .1.3.6.1.2.1.1.4.0="Folkert van Heusden [off-list ref]" I thought of a firewall-issue, but I would expect to then either get ALL data or NONE at all, not a bit. But to be sure I added: iptables -A INPUT -i $EXTERNAL_I -p udp --dport 161 -j ACCEPT iptables -A INPUT -i $EXTERNAL_I -p udp --sport 161 -j ACCEPT iptables -A FORWARD -i $EXTERNAL_I -p udp --dport 161 -j ACCEPT iptables -A FORWARD -i $EXTERNAL_I -p udp --sport 161 -j ACCEPT iptables -A FORWARD -o $EXTERNAL_I -p udp --dport 161 -j ACCEPT iptables -A FORWARD -o $EXTERNAL_I -p udp --sport 161 -j ACCEPT Things like portnumbers and snmp sequence numbers are correct. Sometimes it works (I get ~73 lines) but then on a laptop behind this firewall-system I again get only 3 lines after which it stops again (that laptop has only policy-accept firewall rules and default debian kvm/virt-manager rules). When I trace dropped packets, it looks like they're dropped in nf_hook_slow+0x8f/0xb0. Maybe NF_DROP or NF_QUEUE returning an error? Could this be a kernel-issue? And/or any hints to track this further down? regards