On 3/24/21 10:51 PM, Maxime Coquelin wrote:

On 3/24/21 10:39 PM, Ilya Maximets wrote:

quoted

On 3/24/21 9:56 PM, Maxime Coquelin wrote:

quoted

Hi Ilya,

On 3/19/21 5:45 PM, Ilya Maximets wrote:

quoted

On 3/19/21 5:11 PM, Ilya Maximets wrote:

quoted

On 3/19/21 3:39 PM, Stefan Hajnoczi wrote:

quoted

Hi Ilya,
By the way, it's not clear to me why dpdkvhostuser is deprecated. If OVS
is restarted then existing vhost-user connections drop with an error but
QEMU could attempt to reconnect to the UNIX domain socket which the new
OVS instance will set up.

Why is it impossible to reconnect when OVS owns the listen socket?

Well, AFAIK, qemu reconnects client connections only:

    ``reconnect`` sets the timeout for reconnecting on non-server
    sockets when the remote end goes away. qemu will delay this many
    seconds and then attempt to reconnect. Zero disables reconnecting,
    and is the default.

I'm not sure about exact reason.  It was historically this way.
For me it doesn't make much sense.  I mean, your right that it's
just a socket, so it should not matter who listens and who connects.
If reconnection is possible in one direction, it should be possible
in the opposite direction too.

Sorry, my thought slipped. :)  Yes, QEMU supports re-connection
for client sockets.  So, in theory, dpdkvhostuser ports should work
after re-connection.  And that would be nice.  I don't remember
right now why this doesn't work...  Maybe vhost-user parts in QEMU
doesn't handle this case.  Need to dig some more into that and refresh
my memory.  It was so long ago...

Maxime, do you remember?

Sorry for the delay. I didn't remember, so I wanted to have a try.

I can confirm reconnect works with QEMU as client and with Vhost PMD as
server with:


    <interface type='vhostuser'>
      <mac address='56:48:4f:53:54:01'/>
      <source type='unix' path='/tmp/vhost-user1' mode='client'>
        <reconnect enabled='yes' timeout='1'/>
      </source>
      <model type='virtio'/>
      <driver name='vhost' rx_queue_size='256'/>
      <address type='pci' domain='0x0000' bus='0x07' slot='0x00'
function='0x0'/>
    </interface>

Cool, thanks for checking. :)
If it works with vhost PMD, it should probably work with OVS too.

There is still a couple of problems:

1. OpenStack Nova doesn't support configuration of a 'reconnect'
   in libvirt xml (it only adds the 'source'):
   https://github.com/openstack/nova/blob/master/nova/virt/libvirt/config.py#L1834

2. 'reconnect' configuration supported only starting from libvirt 4.1.0.
   It's released in 2018, but still some systems are using older versions.
   e.g. Ubuntu 18.04 which will be supported until 2023 uses libvirt 4.0.0.

Is it really a problem? We can keep using OVS as client for OSP,

Sure, no problem here.  I think, we scared enough users so they
mostly switched to this configuration and it's a good thing.

and use OVS as server for containers.

There is a problem here that OVS emits a deprecation warning each
time you're trying to use dpdkvhostuser ports.  There are no problems
functionally, though, except that deprecation also implies that
we're consciously not adding new features to these ports.

Another thing is that current state of the art of k8s CNI requires
mounting of the whole /var/run/openvswitch or managing directories
by hands.  So, IMO, Broker idea is still valid is some form in terms
of hassle reduction for users.

quoted

quoted

quoted

quoted

dpdkvhostuser was deprecated just to scare users and force them to
migrate to dpdkvhostuserclient and avoid constant bug reports like:

  "OVS service restarted and network is lost now".

BTW, virtio-user ports in DPDK doesn't support re-connection in client
mode too.

This is still true, though.  virtio-user in client mode doesn't reconnect.

That could be added, and it is maybe not as important for containers as
it is for VM to support it, given the ephemeral nature of containers?

Well, restart of OVS should not require restarting of all the containers
on the host even though they are "stateless".

BTW, some infrastructure changes that I made in this series might be
reused to implement client-side reconnection for virtio-user.

Great, I'll look at the series when we'll work on adding reconnect for
clients.

Please, take a look at the patch #1 in this set sooner.
It's a bug fix. :)

Thanks,
Maxime

quoted

quoted

Regards,
Maxime

quoted

quoted

BTW2, with SocketPair Broker it might be cheaper to implement server
reconnection in QEMU because all connections in these case are client
connections, i.e. both ends will connect() to a broker.

Bets regards, Ilya Maximets.