Re: [PATCH] bonding: If IP route look-up to send an ARP fails, mark in bonding structure as no ARP sent.

rama nichanamatlu [off-list ref] wrote:

On 11/21/2013 1:12 PM, Jay Vosburgh wrote:

quoted

rama nichanamatlu [off-list ref] wrote:

quoted

On 11/21/2013 3:10 AM, Veaceslav Falico wrote:

quoted

On Wed, Nov 20, 2013 at 04:53:20PM -0800, rama nichanamatlu wrote:

quoted

During the creation of VLAN's atop bonding the underlying interfaces
are made part of VLAN's, and at the same bonding driver gets aware
that VLAN's exists above it and hence would consult IP routing for
every ARP to  be sent to determine the route which tells bonding
driver the correct VLAN tag to attach to the outgoing ARP packet. But,
during the VLAN creation when vlan driver puts the underlying
interface into default vlan and then actual vlan, in-between this if
bonding driver consults the IP for a route, IP fails to provide a
correct route and upon which bonding driver drops the ARP packet. ARP
monitor when it
comes around next time, sees no ARP response and fails-over to the
next available slave. Consulting for a IP route,
ip_route_output(),happens in bond_arp_send_all().

bonding works as expected - nothing to fix here. And even as a
workaround/hack - I'm not sure we need that to suppress one failover *only*
when vlan is added on top.

quoted

Thank U.
With *out* this change our systems failed system testing, to
consistently be on designated primary interface on *every* single
reboot. With this change the behavior was as expected even after a few
thousand reboots & System testing could move to next level catching an
another bug in sr-iov :). And Without, the outcome was less predictable
after a reboot and bonding was on a different slave each time.
-Rama

	By "designated primary" you mean the bonding primary option,
correct?  

Yes correct. Bonding primary param is set.
ex: primary=eth1 and primary_reselect=2.
Hence it is expected to be on primary on every reboot.

	If I set up a basic bonding configuration like:

[ eth3, eth4 ] -> bond0 -> bond0.66, with primary=eth3 primary_reselect=2

	Then look at dmesg, I see this sequence:

	The bond is set up first, with an arp_ip_target on a VLAN
destination.  The slaves are added to the bond.

	The VLAN interface is configured above the bond, and brought up.

	The slaves become link up after autonegotiation, the ARP monitor
commences, and eth3 is made the active slave.  Even if eth4 is set by
the bond to be "link status up," eth3 becomes the active slave when it
becomes "link status up."

	What network device are you using for the slaves?  Are they
virtualized devices of some kind?  My suspicion is that Ethernet
autonegotiation either does not take place or occurs so quickly that the
slaves are carrier up before the VLAN is even added.

	Can you check your dmesg output for the sequence of events?  In
my test, I do not see the slaves go "NIC Link is Up 1000 Mbps Full
Duplex" until about 3 seconds after the VLAN interface has been
configured.


	-J

---
	-Jay Vosburgh, IBM Linux Technology Center, fubar@us.ibm.com