[BUG] Iproute2 batch-mode fails to bring up veth

Hey all,

Sorry if this isn't the right place to report Iproute2 bugs. It was implied by README.devel as well as a couple of entries I saw in bugzilla.

I use iproute2 batch mode to construct network namespaces. Example script:

   $ cat ~/bin/netns-test.sh
   #! /bin/bash

   gw=192.168.5.1
   ip=192.168.5.2
   ns=netns-test
   veth0=${ns}-0
   veth1=${ns}-1

   /usr/local/sbin/ip -b - << EOF
   link add $veth0 type veth peer name $veth1
   addr add $gw peer $ip dev $veth0
   link set dev $veth0 up
   netns add $ns
   link set $veth1 netns $ns
   netns exec $ns ip link set dev lo up
   netns exec $ns ip link set dev $veth1 up
   netns exec $ns ip addr add $ip/24 dev $veth1
   netns exec $ns ip addr add $ip peer $gw dev $veth1
   netns exec $ns ip route add default via $gw dev $veth1
   netns exec $ns ip route add 192.168.0.0/24 via $gw dev $veth1
   EOF


I noticed when version 5.11.0 dropped that this stops working. Batch mode fails to bring up the inner veth.

Expected usage (as produced by v5.10.0):

   $ sudo ./bin/netns-test.sh
   $ sudo ip netns exec netns-test ip route
   default via 192.168.5.1 dev netns-test-1
   192.168.0.0/24 via 192.168.5.1 dev netns-test-1
   192.168.5.0/24 dev netns-test-1 proto kernel scope link src 192.168.5.2
   192.168.5.1 dev netns-test-1 proto kernel scope link src 192.168.5.2

Actual behaviour:

   $ sudo ./bin/netns-test.sh
   $ sudo ip netns exec netns-test ip route  # Notice the empty output
   $ sudo ip netns exec netns-test ip link
   1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000
       link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
   39: netns-test-1@if40: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000
       link/ether 1a:96:4e:4f:84:31 brd ff:ff:ff:ff:ff:ff link-netnsid 0

System info:

* Distro: Void Linux
* Kernel version: 5.10.23
* CPU: AMD Ryzen 7 1800X and Ryzen 5 2600. (Reproduced on both.)

Git bisect pinpoints this commit: https://github.com/shemminger/iproute2/commit/1d9a81b8c9f30f9f4abeb875998262f61bf10577

That commit was focused on refactoring batch mode. This is consistent with my experience that only batch mode is affected. Everything works as expected when running the commands manually.

~ Tim

+ Petr

On Tue, Mar 16, 2021 at 06:22:08AM +1100, Tim Rice wrote:

Hey all,

Sorry if this isn't the right place to report Iproute2 bugs. It was implied by README.devel as well as a couple of entries I saw in bugzilla.

I use iproute2 batch mode to construct network namespaces. Example script:

  $ cat ~/bin/netns-test.sh
  #! /bin/bash

  gw=192.168.5.1
  ip=192.168.5.2
  ns=netns-test
  veth0=${ns}-0
  veth1=${ns}-1

  /usr/local/sbin/ip -b - << EOF
  link add $veth0 type veth peer name $veth1
  addr add $gw peer $ip dev $veth0
  link set dev $veth0 up
  netns add $ns
  link set $veth1 netns $ns
  netns exec $ns ip link set dev lo up
  netns exec $ns ip link set dev $veth1 up
  netns exec $ns ip addr add $ip/24 dev $veth1
  netns exec $ns ip addr add $ip peer $gw dev $veth1
  netns exec $ns ip route add default via $gw dev $veth1
  netns exec $ns ip route add 192.168.0.0/24 via $gw dev $veth1
  EOF


I noticed when version 5.11.0 dropped that this stops working. Batch mode fails to bring up the inner veth.

Expected usage (as produced by v5.10.0):

  $ sudo ./bin/netns-test.sh
  $ sudo ip netns exec netns-test ip route
  default via 192.168.5.1 dev netns-test-1
  192.168.0.0/24 via 192.168.5.1 dev netns-test-1
  192.168.5.0/24 dev netns-test-1 proto kernel scope link src 192.168.5.2
  192.168.5.1 dev netns-test-1 proto kernel scope link src 192.168.5.2

Actual behaviour:

  $ sudo ./bin/netns-test.sh
  $ sudo ip netns exec netns-test ip route  # Notice the empty output
  $ sudo ip netns exec netns-test ip link
  1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000
      link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
  39: netns-test-1@if40: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000
      link/ether 1a:96:4e:4f:84:31 brd ff:ff:ff:ff:ff:ff link-netnsid 0

System info:

* Distro: Void Linux
* Kernel version: 5.10.23
* CPU: AMD Ryzen 7 1800X and Ryzen 5 2600. (Reproduced on both.)

Git bisect pinpoints this commit: https://github.com/shemminger/iproute2/commit/1d9a81b8c9f30f9f4abeb875998262f61bf10577

That commit was focused on refactoring batch mode. This is consistent with my experience that only batch mode is affected. Everything works as expected when running the commands manually.

~ Tim

Thanks for the report. Would you be able to test with the following
patch?

    https://github.com/pmachata/iproute2/commit/a12eeca9caf90b3ebe24bc121819d506c9072a34.patch

I believe it fixes the issue.

On 3/15/21 1:22 PM, Tim Rice wrote:

Hey all,

Sorry if this isn't the right place to report Iproute2 bugs. It was
implied by README.devel as well as a couple of entries I saw in bugzilla.

I use iproute2 batch mode to construct network namespaces. Example script:

  $ cat ~/bin/netns-test.sh
  #! /bin/bash

  gw=192.168.5.1
  ip=192.168.5.2
  ns=netns-test
  veth0=${ns}-0
  veth1=${ns}-1

  /usr/local/sbin/ip -b - << EOF
  link add $veth0 type veth peer name $veth1
  addr add $gw peer $ip dev $veth0
  link set dev $veth0 up
  netns add $ns
  link set $veth1 netns $ns
  netns exec $ns ip link set dev lo up
  netns exec $ns ip link set dev $veth1 up
  netns exec $ns ip addr add $ip/24 dev $veth1
  netns exec $ns ip addr add $ip peer $gw dev $veth1
  netns exec $ns ip route add default via $gw dev $veth1
  netns exec $ns ip route add 192.168.0.0/24 via $gw dev $veth1
  EOF


I noticed when version 5.11.0 dropped that this stops working. Batch
mode fails to bring up the inner veth.

Expected usage (as produced by v5.10.0):

  $ sudo ./bin/netns-test.sh
  $ sudo ip netns exec netns-test ip route
  default via 192.168.5.1 dev netns-test-1
  192.168.0.0/24 via 192.168.5.1 dev netns-test-1
  192.168.5.0/24 dev netns-test-1 proto kernel scope link src 192.168.5.2
  192.168.5.1 dev netns-test-1 proto kernel scope link src 192.168.5.2

Actual behaviour:

  $ sudo ./bin/netns-test.sh
  $ sudo ip netns exec netns-test ip route  # Notice the empty output
  $ sudo ip netns exec netns-test ip link
  1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN
mode DEFAULT group default qlen 1000
      link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
  39: netns-test-1@if40: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state
DOWN mode DEFAULT group default qlen 1000
      link/ether 1a:96:4e:4f:84:31 brd ff:ff:ff:ff:ff:ff link-netnsid 0

System info:

* Distro: Void Linux
* Kernel version: 5.10.23
* CPU: AMD Ryzen 7 1800X and Ryzen 5 2600. (Reproduced on both.)

Git bisect pinpoints this commit:
https://github.com/shemminger/iproute2/commit/1d9a81b8c9f30f9f4abeb875998262f61bf10577

Petr, can you take a look at this regression?

David Ahern [off-list ref] writes:

quoted

Git bisect pinpoints this commit:
https://github.com/shemminger/iproute2/commit/1d9a81b8c9f30f9f4abeb875998262f61bf10577

Petr, can you take a look at this regression?

Yes, see elsewhere in the thread:

    https://marc.info/?l=linux-netdev&m=161589291608081&w=2

I'm pretty sure this fixes the issue, and hopefully Tim can take it for
a spin and confirm. I'll send this formally afterwards.

Hey Petr,

Thanks for the report. Would you be able to test with the following
patch?

   https://github.com/pmachata/iproute2/commit/a12eeca9caf90b3ebe24bc121819d506c9072a34.patch

I believe it fixes the issue.

Awesome! That does fix it, thanks :)

~ Tim