Re: IPv6 IPSec incompatibilities between 2.6.23 and 3.6.18/4.6.4
From: Pommnitz Jörg <hidden>
Date: 2016-07-21 11:48:12
Additionally I have now built and installed Linux-4.6.4 and run the debug script.
Setup is unchanged from the previous one, except that Node 1 is now running Linux-4.6.4.
Node 1: fd01:1b10:1000::1 is running 4.6.4
Debug log from Node 1, Receiver running 4.6.4:
==================Begin==========================
Linux node-0-001 4.6.4-node-test #2 Thu Jul 21 13:22:34 CEST 2016 i686 i686 i686 GNU/Linux
tcpdump -lni lan1 -c 5
14:21:49.737030 IP6 fd01:1b10:1000::3 > ff0e::1: ESP(spi=0x00000001,seq=0xff), length 136
14:21:50.737092 IP6 fd01:1b10:1000::3 > ff0e::1: ESP(spi=0x00000001,seq=0x100), length 136
14:21:51.737155 IP6 fd01:1b10:1000::3 > ff0e::1: ESP(spi=0x00000001,seq=0x101), length 136
14:21:52.737217 IP6 fd01:1b10:1000::3 > ff0e::1: ESP(spi=0x00000001,seq=0x102), length 136
14:21:53.737280 IP6 fd01:1b10:1000::3 > ff0e::1: ESP(spi=0x00000001,seq=0x103), length 136
cat /proc/net/xfrm_stat
XfrmInError 0
XfrmInBufferError 0
XfrmInHdrError 0
XfrmInNoStates 0
XfrmInStateProtoError 0
XfrmInStateModeError 0
XfrmInStateSeqError 0
XfrmInStateExpired 0
XfrmInStateMismatch 0
XfrmInStateInvalid 0
XfrmInTmplMismatch 0
XfrmInNoPols 0
XfrmInPolBlock 0
XfrmInPolError 0
XfrmOutError 0
XfrmOutBundleGenError 0
XfrmOutBundleCheckError 0
XfrmOutNoStates 0
XfrmOutStateProtoError 0
XfrmOutStateModeError 0
XfrmOutStateSeqError 0
XfrmOutStateExpired 0
XfrmOutPolBlock 0
XfrmOutPolDead 0
XfrmOutPolError 0
XfrmFwdHdrError 0
XfrmOutStateInvalid 0
XfrmAcquireError 0
ip -s xfrm state
src fd01:1b10:1000::1 dst ff0e::1
proto esp spi 0x00000001(1) reqid 0(0x00000000) mode tunnel
replay-window 0 seq 0x00000000 flag (0x00000000)
enc cbc(aes) 0x7bef6ecaf06d29ef55b24aca6e19964b332e02e75be676a3 (192 bits)
sel src ::/0 dst ::/0 uid 0
lifetime config:
limit: soft (INF)(bytes), hard (INF)(bytes)
limit: soft (INF)(packets), hard (INF)(packets)
expire add: soft 0(sec), hard 0(sec)
expire use: soft 0(sec), hard 0(sec)
lifetime current:
0(bytes), 0(packets)
add 2016-07-21 14:18:08 use -
stats:
replay-window 0 replay 0 failed 0
ip -s xfrm policy
src ::/0 dst ff0e::1/128 uid 0
dir out action allow index 17 priority 2147483648 share any flag (0x00000000)
lifetime config:
limit: soft (INF)(bytes), hard (INF)(bytes)
limit: soft (INF)(packets), hard (INF)(packets)
expire add: soft 0(sec), hard 0(sec)
expire use: soft 0(sec), hard 0(sec)
lifetime current:
0(bytes), 0(packets)
add 2016-07-21 14:18:08 use -
tmpl src fd01:1b10:1000::1 dst ff0e::1
proto esp spi 0x00000000(0) reqid 0(0x00000000) mode tunnel
level required share any
enc-mask 00000000 auth-mask 00000000 comp-mask 00000000
src ::/0 dst ff0e::1/128 uid 0
dir fwd action allow index 10 priority 2147483648 share any flag (0x00000000)
lifetime config:
limit: soft (INF)(bytes), hard (INF)(bytes)
limit: soft (INF)(packets), hard (INF)(packets)
expire add: soft 0(sec), hard 0(sec)
expire use: soft 0(sec), hard 0(sec)
lifetime current:
0(bytes), 0(packets)
add 2016-07-21 14:18:08 use -
tmpl src fd01:1b10:1000::1 dst ff0e::1
proto esp spi 0x00000000(0) reqid 0(0x00000000) mode tunnel
level required share any
enc-mask 00000000 auth-mask 00000000 comp-mask 00000000
src ::/0 dst ff0e::1/128 uid 0
dir in action allow index 8 priority 2147483648 share any flag (0x00000000)
lifetime config:
limit: soft (INF)(bytes), hard (INF)(bytes)
limit: soft (INF)(packets), hard (INF)(packets)
expire add: soft 0(sec), hard 0(sec)
expire use: soft 0(sec), hard 0(sec)
lifetime current:
0(bytes), 0(packets)
add 2016-07-21 14:18:08 use -
tmpl src fd01:1b10:1000::1 dst ff0e::1
proto esp spi 0x00000000(0) reqid 0(0x00000000) mode tunnel
level required share any
enc-mask 00000000 auth-mask 00000000 comp-mask 00000000
===================End===========================
Kind regards
Joerg
-----Ursprüngliche Nachricht----- Von: Pommnitz Jörg Gesendet: Donnerstag, 21. Juli 2016 11:58 An: 'Ilan Tayari'; 'Shanker Wang' Cc: netdev@vger.kernel.org Betreff: AW: IPv6 IPSec incompatibilities between 2.6.23 and 3.6.18 (and probably later) Ilan and Shanker, I did as you asked. Setup: Three nodes all connected to the same Ethernet hub. Node 1: fd01:1b10:1000::1 is running 3.18.36 Node 2: fd01:1b10:1000::2 is running 2.6.23.12 Node 3: fd01:1b10:1000::3 is running 3.18.36 and generates the traffic with the command "ping6 -I lan1 ff0e::1" All three nodes are configured with the following script (IP6ADDR adjusted for every node): ==================Begin========================== #!/bin/bash IP6ANYADDR=::/0 IP6BCAST=ff0e::1 KEY="0x7bef6ecaf06d29ef55b24aca6e19964b332e02e75be676a3" #IFNAME=radio IFNAME=lan1 IP6ADDR=fd01:1b10:1000::X PREFIX6=64 SPI=0x1 ip link set dev ${IFNAME} up ip addr add ${IP6ADDR}/${PREFIX6} dev ${IFNAME} echo "flush; spdflush;" | setkey -c echo "add ${IP6ADDR} ${IP6BCAST} esp ${SPI} -m tunnel -E aes-cbc ${KEY};" | setkey -c echo "spdadd ${IP6ANYADDR} ${IP6BCAST} any -P in ipsec esp/tunnel/${IP6ADDR}-${IP6BCAST}/require;" | setkey -c echo "spdadd ${IP6ANYADDR} ${IP6BCAST} any -P out ipsec esp/tunnel/${IP6ADDR}-${IP6BCAST}/require;" | setkey -c ===================End=========================== To capture the debug information I used the following script: ==================Begin========================== uname -a echo "tcpdump -lni lan1 -c 5" tcpdump -lni lan1 -c 5 echo "cat /proc/net/xfrm_stat" cat /proc/net/xfrm_stat echo "ip -s xfrm state" ip -s xfrm state echo "ip -s xfrm policy" ip -s xfrm policy ===================End=========================== Debug log from Node 1, Receiver running 3.6.18: ==================Begin========================== Linux node-0-001 3.18.36-node-5875 #1 Tue Jul 12 14:00:52 CEST 2016 i686 i686 i686 GNU/Linux tcpdump -lni lan1 -c 5 12:03:09.580943 IP6 fd01:1b10:1000::3 > ff0e::1: ESP(spi=0x00000001,seq=0x69), length 136 12:03:10.581006 IP6 fd01:1b10:1000::3 > ff0e::1: ESP(spi=0x00000001,seq=0x6a), length 136 12:03:11.581068 IP6 fd01:1b10:1000::3 > ff0e::1: ESP(spi=0x00000001,seq=0x6b), length 136 12:03:12.581131 IP6 fd01:1b10:1000::3 > ff0e::1: ESP(spi=0x00000001,seq=0x6c), length 136 12:03:13.581193 IP6 fd01:1b10:1000::3 > ff0e::1: ESP(spi=0x00000001,seq=0x6d), length 136 cat /proc/net/xfrm_stat XfrmInError 0 XfrmInBufferError 0 XfrmInHdrError 0 XfrmInNoStates 0 XfrmInStateProtoError 0 XfrmInStateModeError 0 XfrmInStateSeqError 0 XfrmInStateExpired 0 XfrmInStateMismatch 0 XfrmInStateInvalid 0 XfrmInTmplMismatch 0 XfrmInNoPols 0 XfrmInPolBlock 0 XfrmInPolError 0 XfrmOutError 0 XfrmOutBundleGenError 0 XfrmOutBundleCheckError 0 XfrmOutNoStates 0 XfrmOutStateProtoError 0 XfrmOutStateModeError 0 XfrmOutStateSeqError 0 XfrmOutStateExpired 0 XfrmOutPolBlock 0 XfrmOutPolDead 0 XfrmOutPolError 0 XfrmFwdHdrError 0 XfrmOutStateInvalid 0 XfrmAcquireError 0 ip -s xfrm state src fd01:1b10:1000::1 dst ff0e::1 proto esp spi 0x00000001(1) reqid 0(0x00000000) mode tunnel replay-window 0 seq 0x00000000 flag (0x00000000) enc cbc(aes) 0x7bef6ecaf06d29ef55b24aca6e19964b332e02e75be676a3 (192 bits) sel src ::/0 dst ::/0 uid 0 lifetime config: limit: soft (INF)(bytes), hard (INF)(bytes) limit: soft (INF)(packets), hard (INF)(packets) expire add: soft 0(sec), hard 0(sec) expire use: soft 0(sec), hard 0(sec) lifetime current: 0(bytes), 0(packets) add 2016-07-21 12:01:44 use - stats: replay-window 0 replay 0 failed 0 ip -s xfrm policy src ::/0 dst ff0e::1/128 uid 0 dir out action allow index 41 priority 2147483648 share any flag (0x00000000) lifetime config: limit: soft (INF)(bytes), hard (INF)(bytes) limit: soft (INF)(packets), hard (INF)(packets) expire add: soft 0(sec), hard 0(sec) expire use: soft 0(sec), hard 0(sec) lifetime current: 0(bytes), 0(packets) add 2016-07-21 12:01:44 use - tmpl src fd01:1b10:1000::1 dst ff0e::1 proto esp spi 0x00000000(0) reqid 0(0x00000000) mode tunnel level required share any enc-mask 00000000 auth-mask 00000000 comp-mask 00000000 src ::/0 dst ff0e::1/128 uid 0 dir fwd action allow index 34 priority 2147483648 share any flag (0x00000000) lifetime config: limit: soft (INF)(bytes), hard (INF)(bytes) limit: soft (INF)(packets), hard (INF)(packets) expire add: soft 0(sec), hard 0(sec) expire use: soft 0(sec), hard 0(sec) lifetime current: 0(bytes), 0(packets) add 2016-07-21 12:01:44 use - tmpl src fd01:1b10:1000::1 dst ff0e::1 proto esp spi 0x00000000(0) reqid 0(0x00000000) mode tunnel level required share any enc-mask 00000000 auth-mask 00000000 comp-mask 00000000 src ::/0 dst ff0e::1/128 uid 0 dir in action allow index 24 priority 2147483648 share any flag (0x00000000) lifetime config: limit: soft (INF)(bytes), hard (INF)(bytes) limit: soft (INF)(packets), hard (INF)(packets) expire add: soft 0(sec), hard 0(sec) expire use: soft 0(sec), hard 0(sec) lifetime current: 0(bytes), 0(packets) add 2016-07-21 12:01:44 use - tmpl src fd01:1b10:1000::1 dst ff0e::1 proto esp spi 0x00000000(0) reqid 0(0x00000000) mode tunnel level required share any enc-mask 00000000 auth-mask 00000000 comp-mask 00000000 ===================End=========================== Debug log from Node 2, Receiver running 2.6.23.12 (Note: 2.6.23 did not have /proc/net/xfrm_stat): ==================Begin========================== Linux node-0-001 2.6.23.12-node-4908 #3 Mon Dec 15 17:28:03 CET 2014 i686 GNU/Linux tcpdump -lni lan1 -c 5 11:54:06.506723 IP6 fd01:1b10:1000::3 > ff0e::1: ESP(spi=0x00000001,seq=0x75), length 136 11:54:06.506723 IP6 fd01:1b10:1000::3 > ff0e::1: ICMP6, echo request, seq 117, length 64 11:54:07.506696 IP6 fd01:1b10:1000::3 > ff0e::1: ESP(spi=0x00000001,seq=0x76), length 136 11:54:07.506696 IP6 fd01:1b10:1000::3 > ff0e::1: ICMP6, echo request, seq 118, length 64 11:54:08.506731 IP6 fd01:1b10:1000::3 > ff0e::1: ESP(spi=0x00000001,seq=0x77), length 136 cat /proc/net/xfrm_stat ip -s xfrm state src fd01:1b10:1000::2 dst ff0e::1 proto esp spi 0x00000001(1) reqid 0(0x00000000) mode tunnel replay-window 0 seq 0x00000000 flag (0x00000000) enc cbc(aes) 0x7bef6ecaf06d29ef55b24aca6e19964b332e02e75be676a3 (192 bits) sel src ::/0 dst ::/0 uid 0 lifetime config: limit: soft (INF)(bytes), hard (INF)(bytes) limit: soft (INF)(packets), hard (INF)(packets) expire add: soft 0(sec), hard 0(sec) expire use: soft 0(sec), hard 0(sec) lifetime current: 8320(bytes), 80(packets) add 2016-07-21 11:52:48 use 2016-07-21 11:52:49 stats: replay-window 0 replay 0 failed 0 ip -s xfrm policy src ::/0 dst ff0e::1/128 uid 0 dir in action allow index 24 priority 2147483648 share any flag 0x00000000 lifetime config: limit: soft (INF)(bytes), hard (INF)(bytes) limit: soft (INF)(packets), hard (INF)(packets) expire add: soft 0(sec), hard 0(sec) expire use: soft 0(sec), hard 0(sec) lifetime current: 0(bytes), 0(packets) add 2016-07-21 11:52:48 use - tmpl src fd01:1b10:1000::2 dst ff0e::1 proto esp spi 0x00000000(0) reqid 0(0x00000000) mode tunnel level required share any enc-mask ffffffff auth-mask ffffffff comp-mask ffffffff src ::/0 dst ff0e::1/128 uid 0 dir out action allow index 41 priority 2147483648 share any flag 0x00000000 lifetime config: limit: soft (INF)(bytes), hard (INF)(bytes) limit: soft (INF)(packets), hard (INF)(packets) expire add: soft 0(sec), hard 0(sec) expire use: soft 0(sec), hard 0(sec) lifetime current: 0(bytes), 0(packets) add 2016-07-21 11:52:48 use - tmpl src fd01:1b10:1000::2 dst ff0e::1 proto esp spi 0x00000000(0) reqid 0(0x00000000) mode tunnel level required share any enc-mask ffffffff auth-mask ffffffff comp-mask ffffffff src ::/0 dst ff0e::1/128 uid 0 dir fwd action allow index 34 priority 2147483648 share any flag 0x00000000 lifetime config: limit: soft (INF)(bytes), hard (INF)(bytes) limit: soft (INF)(packets), hard (INF)(packets) expire add: soft 0(sec), hard 0(sec) expire use: soft 0(sec), hard 0(sec) lifetime current: 0(bytes), 0(packets) add 2016-07-21 11:52:48 use - tmpl src fd01:1b10:1000::2 dst ff0e::1 proto esp spi 0x00000000(0) reqid 0(0x00000000) mode tunnel level required share any enc-mask ffffffff auth-mask ffffffff comp-mask ffffffff ===================End=========================== Debug log from Node 3, Sender running 3.6.18: ==================Begin========================== Linux node-0-005 3.18.36-node-5875 #1 Tue Jul 12 14:00:52 CEST 2016 i686 i686 i686 GNU/Linux tcpdump -lni lan1 -c 5 10:27:07.369198 IP6 fd01:1b10:1000::3 > ff0e::1: ESP(spi=0x00000001,seq=0x5b), length 136 10:27:08.369261 IP6 fd01:1b10:1000::3 > ff0e::1: ESP(spi=0x00000001,seq=0x5c), length 136 10:27:09.369323 IP6 fd01:1b10:1000::3 > ff0e::1: ESP(spi=0x00000001,seq=0x5d), length 136 10:27:10.369386 IP6 fd01:1b10:1000::3 > ff0e::1: ESP(spi=0x00000001,seq=0x5e), length 136 10:27:11.369448 IP6 fd01:1b10:1000::3 > ff0e::1: ESP(spi=0x00000001,seq=0x5f), length 136 cat /proc/net/xfrm_stat XfrmInError 0 XfrmInBufferError 0 XfrmInHdrError 0 XfrmInNoStates 0 XfrmInStateProtoError 0 XfrmInStateModeError 0 XfrmInStateSeqError 0 XfrmInStateExpired 0 XfrmInStateMismatch 0 XfrmInStateInvalid 0 XfrmInTmplMismatch 0 XfrmInNoPols 0 XfrmInPolBlock 0 XfrmInPolError 0 XfrmOutError 0 XfrmOutBundleGenError 0 XfrmOutBundleCheckError 0 XfrmOutNoStates 0 XfrmOutStateProtoError 0 XfrmOutStateModeError 0 XfrmOutStateSeqError 0 XfrmOutStateExpired 0 XfrmOutPolBlock 0 XfrmOutPolDead 0 XfrmOutPolError 0 XfrmFwdHdrError 0 XfrmOutStateInvalid 0 XfrmAcquireError 0 ip -s xfrm state src fd01:1b10:1000::3 dst ff0e::1 proto esp spi 0x00000001(1) reqid 0(0x00000000) mode tunnel replay-window 0 seq 0x00000000 flag (0x00000000) enc cbc(aes) 0x7bef6ecaf06d29ef55b24aca6e19964b332e02e75be676a3 (192 bits) sel src ::/0 dst ::/0 uid 0 lifetime config: limit: soft (INF)(bytes), hard (INF)(bytes) limit: soft (INF)(packets), hard (INF)(packets) expire add: soft 0(sec), hard 0(sec) expire use: soft 0(sec), hard 0(sec) lifetime current: 9880(bytes), 95(packets) add 2016-07-21 10:25:32 use 2016-07-21 10:25:37 stats: replay-window 0 replay 0 failed 0 ip -s xfrm policy src ::/0 dst ff0e::1/128 uid 0 dir out action allow index 41 priority 2147483648 share any flag (0x00000000) lifetime config: limit: soft (INF)(bytes), hard (INF)(bytes) limit: soft (INF)(packets), hard (INF)(packets) expire add: soft 0(sec), hard 0(sec) expire use: soft 0(sec), hard 0(sec) lifetime current: 0(bytes), 0(packets) add 2016-07-21 10:25:32 use 2016-07-21 10:27:11 tmpl src fd01:1b10:1000::3 dst ff0e::1 proto esp spi 0x00000000(0) reqid 0(0x00000000) mode tunnel level required share any enc-mask 00000000 auth-mask 00000000 comp-mask 00000000 src ::/0 dst ff0e::1/128 uid 0 dir fwd action allow index 34 priority 2147483648 share any flag (0x00000000) lifetime config: limit: soft (INF)(bytes), hard (INF)(bytes) limit: soft (INF)(packets), hard (INF)(packets) expire add: soft 0(sec), hard 0(sec) expire use: soft 0(sec), hard 0(sec) lifetime current: 0(bytes), 0(packets) add 2016-07-21 10:25:32 use - tmpl src fd01:1b10:1000::3 dst ff0e::1 proto esp spi 0x00000000(0) reqid 0(0x00000000) mode tunnel level required share any enc-mask 00000000 auth-mask 00000000 comp-mask 00000000 src ::/0 dst ff0e::1/128 uid 0 dir in action allow index 24 priority 2147483648 share any flag (0x00000000) lifetime config: limit: soft (INF)(bytes), hard (INF)(bytes) limit: soft (INF)(packets), hard (INF)(packets) expire add: soft 0(sec), hard 0(sec) expire use: soft 0(sec), hard 0(sec) lifetime current: 0(bytes), 0(packets) add 2016-07-21 10:25:32 use - tmpl src fd01:1b10:1000::3 dst ff0e::1 proto esp spi 0x00000000(0) reqid 0(0x00000000) mode tunnel level required share any enc-mask 00000000 auth-mask 00000000 comp-mask 00000000 ===================End=========================== A difference between 2.6.23 and 3.6.18 is the value of the Masks in the policies: 2.6.23: enc-mask ffffffff auth-mask ffffffff comp-mask ffffffff 3.6.18: enc-mask 00000000 auth-mask 00000000 comp-mask 00000000 I have no idea, what these masks mean. Is this important? Regards Joergquoted
-----Ursprüngliche Nachricht----- Von: Ilan Tayari [mailto:ilant@mellanox.com] Gesendet: Dienstag, 19. Juli 2016 17:04 An: Pommnitz Jörg Cc: netdev@vger.kernel.org Betreff: RE: IPv6 IPSec incompatibilities between 2.6.23 and 3.6.18 (and probably later) > On the receiving side (e.g. fd01:1b10:1000::1) I see the decrypted packets withquoted
the 2.6.23 kernel: but NOT with the newer kernel:Hi Joerg, First steps to debug this would be: cat /proc/net/xfrm_stat ip -s xfrm state ip -s xfrm policy First command will show some error accounting, which can point to the culprit code. Second and third command will show existing objects, and some statistics like when the last packet was used with them. Last thing - for your safety you should keep those session keys private. Ilan.
________________________________ Industrieanlagen-Betriebsgesellschaft mbH Sitz der Gesellschaft: Ottobrunn, Registergericht: Amtsgericht München, HRB 5499 Geschäftsführung: Prof. Dr.-Ing. Rudolf F. Schwarz Vorsitzender des Aufsichtsrats: RA Engelbert Kupka MdL a.D.