Re: [net-next 02/10] ixgbevf: Fix broken trunk vlan

On Mon, Sep 26, 2011 at 4:24 PM, Rose, Gregory V
[off-list ref] wrote:

quoted

-----Original Message-----
From: Jesse Gross [mailto:jesse@nicira.com]
Sent: Monday, September 26, 2011 4:09 PM
To: Rose, Gregory V
Cc: Kirsher, Jeffrey T; davem@davemloft.net; netdev@vger.kernel.org;
gospo@redhat.com; Jiri Pirko
Subject: Re: [net-next 02/10] ixgbevf: Fix broken trunk vlan

On Mon, Sep 26, 2011 at 8:57 AM, Rose, Gregory V
[off-list ref] wrote:

quoted

quoted

-----Original Message-----
From: Jesse Gross [mailto:jesse@nicira.com]
Sent: Saturday, September 24, 2011 9:33 AM
To: Kirsher, Jeffrey T
Cc: davem@davemloft.net; Rose, Gregory V; netdev@vger.kernel.org;
gospo@redhat.com; Jiri Pirko
Subject: Re: [net-next 02/10] ixgbevf: Fix broken trunk vlan

What happens if you run tcpdump without configuring vlan devices?
Shouldn't you see tagged packets for the vlans that are being trunked
to you?  I think this will strip tags in that case.  The apparent
behavior of vlan filters here is also surprising to me because on one
hand if they're truly filtering this test shouldn't be needed and on
the other hand they don't seem to be disabled in promiscuous mode.

I think you're not quite understanding the action the HW is taking here.

 In the physical function driver we have put the VF on a VLAN without it
knowing that it's on a VLAN.  Once that is done by the PF, the VF is not
allowed to configure its own VLANs anymore.  However, the descriptor still
includes a bit for the VLAN tag indicating it was a packet that arrived on
a VLAN.  The HW is inserting and stripping the VLAN tag though without any
awareness of that by the VF driver.

quoted

It's a security measure to allow an administrator to put a VF on a VLAN

to provide another level of isolation for the VF.

quoted

Intel VFs don't support promiscuous mode.  If you ran tcpdump you

wouldn't see the VLAN tags because they've been stripped by the HW.  The
VF has no choice in this.

I understand that VFs are limited in what they can do.  What I'm
concerned about are differences in how the driver behaves when a vlan
device is configured on it vs. not.  There are two cases that I think
you will see different behavior:

* The VF driver has it's own vlan filters.  I'm assuming (correct me
if I'm wrong) that this will further restrict the vlans that the guest
sees from the set that have been configured on the PF driver.  If that
is correct, then putting the VF interface in promiscuous mode should
disable the filters and allow packets from all vlans that the PF
allows.  This situation is the same as if an upstream physical switch
is trunking only a subset of vlans to an end host and the NIC is put
in promiscuous mode.  However, it doesn't look like the VF driver does
anything in response to being in promiscuous mode.  This shouldn't
require any hardware support because it is just clearing the filter
table.

The VF does not have its own VLAN filters.  It must request them through the PF via the mailbox messaging feature in the NIC.  When the PF puts the VF in trunk VLAN mode via the 'ip link set <dev> vf <n> vlan <vlanid>' command then the PF also flags that VF and will not take VLAN filter requests from it.  In effect, the VF is forced to a VLAN and has no choice.  So it cannot further restrict what VLANs the guest sees.  And the VF can only be on a single trunk VLAN, it does not support multiple trunk VLANs.

And again, the VF doesn’t support promiscuous mode.  The VF device HW doesn't have that feature.

quoted

* It looks like the VF driver is receiving not just an indication that
a tag was present but the actual tag and although it was already
stripped by the NIC it can be passed up to the network stack just as
if it was stripped in a non-SR-IOV NIC.  After this patch, if you run
tcpdump on the VF interface you will see packets with tags if a vlan
device is configured but packets without tags if there is no vlan
device.

No.  In trunk VLAN mode the receive descriptor provided by the HW has a VLAN tag present bit and that is set but unless someone has used vconfig to configure a VLAN and a bit is set for that VLAN in the active_vlans bit table then the packet is sent up the stack without the tag.  Upper layers don't have access to the driver's internal descriptor rings and have no idea that the VLAN tag present bit was set in the descriptor.  Upper layer SW in the network stack won't be looking for VLAN tags since it never requested any VLANs.  And since the VLAN tag was stripped, and the upper layers haven't ever set a VLAN filter, then there is no expectation of a VLAN tag.

quoted

Does the VF know whether it is assigned to a particular vlan or is
allowed to send/receive tagged packets (essentially the difference
between an access port or a trunk port)?

Yes, it knows if it has requested a VLAN filter via the active_vlans bit table.  If no VLANs are set in the active_vlans table then it knows to ignore the VLAN tag present bit set in the descriptor since it must be in trunk VLAN mode.

quoted

 Or is that why the presence
of vlan device is used as an indication?

Correct, that is the case.

OK, maybe due to hardware limitations what I'm looking for just really
isn't possible.  However, what I'm trying to emphasize is that vconfig
is not the only way that vlans can be consumed by the network stack
and active_vlans is just an indication of whether a vlan filter was
set, nothing more (perhaps I should have picked a better name when I
originally designed this stuff).  In particular, it is not intended to
determine whether a tag should be stripped off or not because
non-vconfig users don't necessarily know which vlans they care about
(think tcpdump or trunking over a bridge).  A major goal of the
existing vlan infrastructure is to avoid having drivers make
assumptions about the consumer of the tag and instead just hand all
information over to the network stack so it can behave in a consistent
manner.  That's why I was looking for alternate ways to get this
information without depending on active_vlans as this driver behaves
quite a bit differently from others, include the ixgbe PF driver.