From: Sean Tranchetti <hidden> Date: 2021-01-06 00:24:16
From: Sean Tranchetti <redacted>
Route removal is handled by two code paths. The main removal path is via
fib6_del_route() which will handle purging any PMTU exceptions from the
cache, removing all per-cpu copies of the DST entry used by the route, and
releasing the fib6_info struct.
The second removal location is during fib6_add_rt2node() during a route
replacement operation. This path also calls fib6_purge_rt() to handle
cleaning up the per-cpu copies of the DST entries and releasing the
fib6_info associated with the older route, but it does not flush any PMTU
exceptions that the older route had. Since the older route is removed from
the tree during the replacement, we lose any way of accessing it again.
As these lingering DSTs and the fib6_info struct are holding references to
the underlying netdevice struct as well, unregistering that device from the
kernel can never complete.
Signed-off-by: Sean Tranchetti <redacted>
---
net/ipv6/ip6_fib.c | 5 ++---
1 file changed, 2 insertions(+), 3 deletions(-)
From: Sean Tranchetti <hidden> Date: 2021-01-06 00:31:45
From: Sean Tranchetti <redacted>
Adds new 2 new tests to the PTMU script: pmtu_ipv4/6_route_change.
These tests explicitly test for a recently discovered problem in the
IPv6 routing framework where PMTU exceptions were not properly released
when replacing a route via "ip route change ...".
After creating PMTU exceptions, the route from the device A to R1 will be
replaced with a new route, then device A will be deleted. If the PMTU
exceptions were properly cleaned up by the kernel, this device deletion
will succeed. Otherwise, the unregistration of the device will stall, and
messages such as the following will be logged in dmesg:
unregister_netdevice: waiting for veth_A-R1 to become free. Usage count = 4
Signed-off-by: Sean Tranchetti <redacted>
---
tools/testing/selftests/net/pmtu.sh | 71 +++++++++++++++++++++++++++++++++++--
1 file changed, 69 insertions(+), 2 deletions(-)
@@ -162,7 +162,15 @@# - list_flush_ipv6_exception# Using the same topology as in pmtu_ipv6, create exceptions, and check# they are shown when listing exception caches, gone after flushing them-+#+# - pmtu_ipv4_route_change+# Use the same topology as in pmtu_ipv4, but issue a route replacement+# command and delete the corresponding device afterward. This tests for+# proper cleanup of the PMTU exceptions by the route replacement path.+# Device unregistration should complete successfully+#+# - pmtu_ipv6_route_change+# Same as above but with IPv6# Kselftest framework requirement - SKIP code is 4.ksft_skip=4
@@ -1782,6 +1792,63 @@ test_list_flush_ipv6_exception() {return${fail}}+test_pmtu_ipvX_route_change(){+family=${1}++setupnamespacesrouting||return2+trace"${ns_a}"veth_A-R1"${ns_r1}"veth_R1-A\+"${ns_r1}"veth_R1-B"${ns_b}"veth_B-R1\+"${ns_a}"veth_A-R2"${ns_r2}"veth_R2-A\+"${ns_r2}"veth_R2-B"${ns_b}"veth_B-R2++if[${family}-eq4];then+ping=ping+dst1="${prefix4}.${b_r1}.1"+dst2="${prefix4}.${b_r2}.1"+gw="${prefix4}.${a_r1}.2"+else+ping=${ping6}+dst1="${prefix6}:${b_r1}::1"+dst2="${prefix6}:${b_r2}::1"+gw="${prefix6}:${a_r1}::2"+fi++# Set up initial MTU values+mtu"${ns_a}"veth_A-R12000+mtu"${ns_r1}"veth_R1-A2000+mtu"${ns_r1}"veth_R1-B1400+mtu"${ns_b}"veth_B-R11400++mtu"${ns_a}"veth_A-R22000+mtu"${ns_r2}"veth_R2-A2000+mtu"${ns_r2}"veth_R2-B1500+mtu"${ns_b}"veth_B-R21500++# Create route exceptions+run_cmd${ns_a}${ping}-q-Mwant-i0.1-w1-s1800${dst1}+run_cmd${ns_a}${ping}-q-Mwant-i0.1-w1-s1800${dst2}++# Check that exceptions have been created with the correct PMTU+pmtu_1="$(route_get_dst_pmtu_from_exception"${ns_a}"${dst1})"+check_pmtu_value"1400""${pmtu_1}""exceeding MTU"||return1+pmtu_2="$(route_get_dst_pmtu_from_exception"${ns_a}"${dst2})"+check_pmtu_value"1500""${pmtu_2}""exceeding MTU"||return1++# Replace the route from A to R1+run_cmd${ns_a}iproutechangedefaultvia${gw}++# Delete the device in A+run_cmd${ns_a}iplinkdel"veth_A-R1"+}++test_pmtu_ipv4_route_change(){+test_pmtu_ipvX_route_change4+}++test_pmtu_ipv6_route_change(){+test_pmtu_ipvX_route_change6+}+ usage(){echoecho"$0 [OPTIONS] [TEST]..."
From: David Ahern <hidden> Date: 2021-01-06 16:56:08
On 1/5/21 5:22 PM, Sean Tranchetti wrote:
From: Sean Tranchetti <redacted>
Route removal is handled by two code paths. The main removal path is via
fib6_del_route() which will handle purging any PMTU exceptions from the
cache, removing all per-cpu copies of the DST entry used by the route, and
releasing the fib6_info struct.
The second removal location is during fib6_add_rt2node() during a route
replacement operation. This path also calls fib6_purge_rt() to handle
cleaning up the per-cpu copies of the DST entries and releasing the
fib6_info associated with the older route, but it does not flush any PMTU
exceptions that the older route had. Since the older route is removed from
the tree during the replacement, we lose any way of accessing it again.
As these lingering DSTs and the fib6_info struct are holding references to
the underlying netdevice struct as well, unregistering that device from the
kernel can never complete.
I think the right fixes tag is:
Fixes: 2b760fcf5cfb3 ("ipv6: hook up exception table to store dst cache")
cc'ed author of that patch.
From: David Ahern <hidden> Date: 2021-01-06 17:04:03
On 1/5/21 5:22 PM, Sean Tranchetti wrote:
quoted hunk
From: Sean Tranchetti <redacted>
Adds new 2 new tests to the PTMU script: pmtu_ipv4/6_route_change.
These tests explicitly test for a recently discovered problem in the
IPv6 routing framework where PMTU exceptions were not properly released
when replacing a route via "ip route change ...".
After creating PMTU exceptions, the route from the device A to R1 will be
replaced with a new route, then device A will be deleted. If the PMTU
exceptions were properly cleaned up by the kernel, this device deletion
will succeed. Otherwise, the unregistration of the device will stall, and
messages such as the following will be logged in dmesg:
unregister_netdevice: waiting for veth_A-R1 to become free. Usage count = 4
Signed-off-by: Sean Tranchetti <redacted>
---
tools/testing/selftests/net/pmtu.sh | 71 +++++++++++++++++++++++++++++++++++--
1 file changed, 69 insertions(+), 2 deletions(-)
@@ -162,7 +162,15 @@# - list_flush_ipv6_exception# Using the same topology as in pmtu_ipv6, create exceptions, and check# they are shown when listing exception caches, gone after flushing them-+#+# - pmtu_ipv4_route_change+# Use the same topology as in pmtu_ipv4, but issue a route replacement+# command and delete the corresponding device afterward. This tests for+# proper cleanup of the PMTU exceptions by the route replacement path.+# Device unregistration should complete successfully+#+# - pmtu_ipv6_route_change+# Same as above but with IPv6# Kselftest framework requirement - SKIP code is 4.ksft_skip=4
@@ -1782,6 +1792,63 @@ test_list_flush_ipv6_exception() {return${fail}}+test_pmtu_ipvX_route_change(){+family=${1}++setupnamespacesrouting||return2+trace"${ns_a}"veth_A-R1"${ns_r1}"veth_R1-A\+"${ns_r1}"veth_R1-B"${ns_b}"veth_B-R1\+"${ns_a}"veth_A-R2"${ns_r2}"veth_R2-A\+"${ns_r2}"veth_R2-B"${ns_b}"veth_B-R2++if[${family}-eq4];then+ping=ping+dst1="${prefix4}.${b_r1}.1"+dst2="${prefix4}.${b_r2}.1"+gw="${prefix4}.${a_r1}.2"+else+ping=${ping6}+dst1="${prefix6}:${b_r1}::1"+dst2="${prefix6}:${b_r2}::1"+gw="${prefix6}:${a_r1}::2"+fi++# Set up initial MTU values+mtu"${ns_a}"veth_A-R12000+mtu"${ns_r1}"veth_R1-A2000+mtu"${ns_r1}"veth_R1-B1400+mtu"${ns_b}"veth_B-R11400++mtu"${ns_a}"veth_A-R22000+mtu"${ns_r2}"veth_R2-A2000+mtu"${ns_r2}"veth_R2-B1500+mtu"${ns_b}"veth_B-R21500++# Create route exceptions+run_cmd${ns_a}${ping}-q-Mwant-i0.1-w1-s1800${dst1}+run_cmd${ns_a}${ping}-q-Mwant-i0.1-w1-s1800${dst2}++# Check that exceptions have been created with the correct PMTU+pmtu_1="$(route_get_dst_pmtu_from_exception"${ns_a}"${dst1})"+check_pmtu_value"1400""${pmtu_1}""exceeding MTU"||return1+pmtu_2="$(route_get_dst_pmtu_from_exception"${ns_a}"${dst2})"+check_pmtu_value"1500""${pmtu_2}""exceeding MTU"||return1++# Replace the route from A to R1+run_cmd${ns_a}iproutechangedefaultvia${gw}++# Delete the device in A+run_cmd${ns_a}iplinkdel"veth_A-R1"+}++test_pmtu_ipv4_route_change(){+test_pmtu_ipvX_route_change4+}++test_pmtu_ipv6_route_change(){+test_pmtu_ipvX_route_change6+}+ usage(){echoecho"$0 [OPTIONS] [TEST]..."
Thanks for adding the tests.
Reviewed-by: David Ahern <dsahern@kernel.org>
From: Jakub Kicinski <kuba@kernel.org> Date: 2021-01-07 20:05:21
On Wed, 6 Jan 2021 09:55:09 -0700 David Ahern wrote:
On 1/5/21 5:22 PM, Sean Tranchetti wrote:
quoted
From: Sean Tranchetti <redacted>
Route removal is handled by two code paths. The main removal path is via
fib6_del_route() which will handle purging any PMTU exceptions from the
cache, removing all per-cpu copies of the DST entry used by the route, and
releasing the fib6_info struct.
The second removal location is during fib6_add_rt2node() during a route
replacement operation. This path also calls fib6_purge_rt() to handle
cleaning up the per-cpu copies of the DST entries and releasing the
fib6_info associated with the older route, but it does not flush any PMTU
exceptions that the older route had. Since the older route is removed from
the tree during the replacement, we lose any way of accessing it again.
As these lingering DSTs and the fib6_info struct are holding references to
the underlying netdevice struct as well, unregistering that device from the
kernel can never complete.
I think the right fixes tag is:
Fixes: 2b760fcf5cfb3 ("ipv6: hook up exception table to store dst cache")
cc'ed author of that patch.