Re: [PATCH V2 09/12] net/eipoib: Add main driver functionality

On Mon, Sep 03, 2012 at 11:53:56PM +0300, Or Gerlitz wrote:

Michael S. Tsirkin [off-list ref] wrote:

quoted

[...] so it seems that a sane solution would involve an extra level of
indirection, with guest addresses being translated to host IB addresses.
As long as you do this, maybe using an ethernet frame format makes sense.

quoted

So far the things that make sense. Here are some that don't, to me:

quoted

- Is a pdf presentation all you have in terms of documentation?
  We are talking communication protocols here - I would expect a
  proper spec, and some effort to standardize, otherwise where's the
  guarantee it won't change in an incompatible way?
  Other things that I would expect to be addressed in such a spec is
  interaction with other IPoIB features, such as connected
  mode, checksum offloading etc, and IB features such as multipath etc.

- The way you encode LID/QPN in the MAC seems questionable. IIRC there's
  more to IB addressing than just the LID.  Since everyone on the subnet
  need access to this translation, I think it makes sense to store it in
  the SM. I think this would also obviate some IPv4 specific hacks in kernel.

quoted

- IGMP/MAC snooping in a driver is just too hairy.
  As you point out, bridge currently needs the uplink in promisc mode.
  I don't think a driver should work around that limitation.
  For some setups, it might be interesting to remove the promisc
 mode requirement, failing that, I think you could use macvtap passthrough.

- Currently migration works without host kernel help, would be
  preferable to keep it that way.

Hi Michael,

If we rewind to this point, basically, you had few concerns

I think some other people gave feedback too, you need to address it in
the patch (as opposed to by mail - even if it's in documentation or
comments) don't just focus on what I wrote.

0. not enough documentation

1. the sender VM MAC isn't preserved when the packet is received

2. the IGMP snooping we planned to do within netdevice - isn't good practice

3. mangling of ARPs within netdevice - isn't good practice as well.

For 0,1,2 we have a way to address  (see below)

So we are remained with #3 - the ARPs -- thinking on this a little
further, FWIW there --are-- components in the kernel which
mangle/generate ARPs and are exposing netdevice, such as openvswitch,
anyway:

does it make sense to forward ARPs received into / sent over the
eIPoIB netdevice (e.g using some sort of rule) to some outer entity
such as user-space
daemon  for interception and later re-injection into eIPoIB?

Or.

Well if this is all you want to do, you can bind a packet socket to the
interface, and drop them at the nic.  It is harder to do for incoming
ARP requests though.

I would do something else: send ARPs out to some defined IB address.
This could be local host or queries from some SA property.  Said remote
side could send you the responses in ethernet format so you do not need
to mangle responses at all.  Similarly for incoming ARP requests.

The rule to do this can also just redirect non IP packets -
this is IPoIB after all.

Documentation we will fix,

And just to stress the point, document the limitations as well.

Preserving remote VM mac at the receiver we have few directions for
solution, e.g either along your suggestion with SA records and/or with
using "alias GUIDs" (details TBD when the submission resumes).

Multicast we accept the direction you suggested - implement  support
for multicast non promiscuous in the elements "above" eIPoIB (bridge,
macvtap, etc).