Re: [RFC v3 0/3] Add VRF support for VXLAN underlay
From: Alexis Bauvin <hidden>
Date: 2018-11-22 00:05:08
Le 20 nov. 2018 à 22:45, David Ahern [off-list ref] a écrit :
On 11/20/18 7:23 AM, Alexis Bauvin wrote:quoted
We are trying to isolate the VXLAN traffic from different VMs with VRF as shown in the schemas below: +-------------------------+ +----------------------------+ | +----------+ | | +------------+ | | | | | | | | | | | tap-red | | | | tap-blue | | | | | | | | | | | +----+-----+ | | +-----+------+ | | | | | | | | | | | | | | +----+---+ | | +----+----+ | | | | | | | | | | | br-red | | | | br-blue | | | | | | | | | | | +----+---+ | | +----+----+ | | | | | | | | | | | | | | | | | | | | +----+--------+ | | +--------------+ | | | | | | | | | | | vxlan-red | | | | vxlan-blue | | | | | | | | | | | +------+------+ | | +-------+------+ | | | | | | | | | VRF | | | VRF | | | red | | | blue | +-------------------------+ +----------------------------+Roopa and I were discussing this setup and are puzzled by the VRF association here. Does br-red and br-blue have an address? The commands below do not show it and from our perspective seems odd for this scenario. If it does not have an address, then there is no reason for the VRF labeling.
Yes, br-red and br-blue have an address. Addresses (both MAC and IP)
are anycast addresses, to make an anycast gateway for the VMs behind
the taps. This serves for a classical evpn symmetric distributed
routing, both between vxlans and towards external networks. I am using
FRR as a control plane to exchange bgp-evpn information between
hypervisors.
The schematic could have been more complete, including more bridges
and more taps in the vrf, with an added bridge+vxlan for the l3vni.
I did not include them as I wanted to focus on the underlay itself
and not on what vxlan was used for.
Here is a more complete example:
+---------+
| |
| vrf1000 |
| |
+--+-+-+--+
| | |
+---------------------+ | +---------------+
| | |
+------+------+ +------+------+ +----+---+
| | | | | |
| br0 | | br1 | | br1000 |
| 10.0.0.1/24 | | 10.0.1.1/24 | | |
| | | | +----+---+
+----+-+-+----+ +-----+-+-----+ |
| | | | | |
+-------+ | +-------+ +--+ +---+ |
| | | | | |
+--+---+ +---+--+ +----+---+ +---+--+ +---+----+ +-----+-----+
| | | | | | | | | | | |
| tap0 | | tap1 | | vxlan0 | | tap2 | | vxlan1 | | vxlan1000 |
| | | | | | | | | | | |
+------+ +------+ +--------+ +------+ +--------+ +-----------+
. . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
.
. +---------+ +--------------+
. | | | |
. | vrf1001 | | vrf-underlay |
. | | | |
. +---+-+---+ +-------+------+
. | | |
. +---------+ +----------+ |
. | | |
. +------+------+ +----+---+ |
. | | | | |
. | br2 | | br1001 | |
. | 10.0.2.1/24 | | | |
. | | +----+---+ |
. +----+-+-+----+ | |
. | | | | |
. +-----+ | +-------+ | |
. | | | | +-------+-------+
. +---+--+ +--+---+ +---+----+ +-----+-----+ | |
. | | | | | | | | | eth0.2030 |
. | tap3 | | tap4 | | vxlan2 | | vxlan1001 | | 172.16.0.2/24 |
. | | | | | | | | | |
. +------+ +------+ +--------+ +-----------+ +---------------+
. . . ^ .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
(vxlan lower device is eth0.2030) v
+----------------+
| |
| eth0 |
| 192.168.1.2/24 |
| |
+----------------+
In this example, the underlay is rather simple. A more complex case
could be with several uplinks in the underlay vrf, and a dummy/loopback
with a /32 ip, used as a vxlan lower device and to bind routing daemons
(e.g. bgpd).
+--------------+
| |
| vrf-underlay |
| |
+-----+-+-+----+
| | |
+---------------+ | +---------------+
| | |
+-------+-------+ +-------+-------+ +-------+--------+
| | | | | |
| eth1 | | eth2 | | dummy0 |
| 172.16.0.2/31 | | 172.16.0.4/31 | | 192.168.0.1/32 |
| | | | | |
+---------------+ +---------------+ +----------------+
.
. . . . . . . . . . . . . . . . . . . . . . .
(vxlan lower device is dummy0)
As for default vrf, it would contain for example management traffic. We
don’t want to mix the underlay routing tables to other traffics.
Also, it would be good to have a unit test this case. Can you create a shell script that creates the setup and runs a few tests verifying connectivity? You can use network namespaces and veth pairs in place of the VM with a tap device. From there the functionality is the same. Tests can be initial VRF association for the vxlan lower device, changing the VRF to another device, and then changing again back to default VRF - checking proper connectivity for each.
Sure! I’ve just finished writing it, but I am not sure of the best way so send it. I guess I will have to add it to a v4 of the patches, in tools/testing/selftests/net?
Thanks