On Thu, Mar 25, 2021 at 12:00:11PM +0100, Ilya Maximets wrote:

On 3/25/21 10:35 AM, Stefan Hajnoczi wrote:

quoted

On Wed, Mar 24, 2021 at 02:11:31PM +0100, Ilya Maximets wrote:

quoted

On 3/24/21 1:05 PM, Stefan Hajnoczi wrote:

quoted

On Tue, Mar 23, 2021 at 04:54:57PM -0400, Billy McFall wrote:

quoted

On Tue, Mar 23, 2021 at 3:52 PM Ilya Maximets [off-list ref] wrote:

quoted

On 3/23/21 6:57 PM, Adrian Moreno wrote:

quoted

On 3/19/21 6:21 PM, Stefan Hajnoczi wrote:

quoted

On Fri, Mar 19, 2021 at 04:29:21PM +0100, Ilya Maximets wrote:

quoted

On 3/19/21 3:05 PM, Stefan Hajnoczi wrote:

quoted

On Thu, Mar 18, 2021 at 08:47:12PM +0100, Ilya Maximets wrote:

quoted

On 3/18/21 6:52 PM, Stefan Hajnoczi wrote:

quoted

On Wed, Mar 17, 2021 at 09:25:26PM +0100, Ilya Maximets wrote:

- How to get this fd again after the OVS restart?  CNI will not be invoked
  at this point to pass a new fd.

- If application will close the connection for any reason (restart, some
  reconfiguration internal to the application) and OVS will be re-started
  at the same time, abstract socket will be gone.  Need a persistent daemon
  to hold it.

I remembered that these two points can be solved by sd_notify(3)
FDSTORE=1. This requires that OVS runs as a systemd service. Not sure if
this is the case (at least in the CNI use case)?

https://www.freedesktop.org/software/systemd/man/sd_notify.html

IIUC, these file descriptors only passed on the restart of the service,
so port-del + port-add scenario is not covered (and this is a very
common usecase, users are implementing some configuration changes this
way and also this is internally possible scenario, e.g. this sequence
will be triggered internally to change the OpenFlow port number).
port-del will release all the resources including the listening socket.
Keeping the fd for later use is not an option, because OVS will not know
if this port will be added back or not and fds is a limited resource.

If users of the CNI plugin are reasonably expected to do this then it
sounds like a blocker for the sd_notify(3) approach. Maybe it could be
fixed by introducing an atomic port-rename (?) operation, but this is
starting to sound too invasive.

It's also unclear how to map these file descriptors to particular ports
they belong to after restart.

The first fd would be a memfd containing a description of the remaining
fds plus any other crash recovery state that OVS wants.

OVS could run as a system pod or as a systemd service.  It differs from
one setup to another.  So it might not be controlled by systemd.

Does the CNI plugin allow both configurations?

It's impossible to come up with one approach that works for everyone in
the general case (beyond the CNI plugin, beyond Kubernetes). I think we
need to enumerate use cases and decide which ones are currently not
addressed satisfactorily.

Also, it behaves as an old-style daemon, so it closes all the file
descriptors, forkes and so on.  This might be adjusted, though, with
some rework of the deamonization procedure.

Doesn't sound like fun but may be doable.

On the side note, it maybe interesting to allow user application to
create a socket and pass a pollable file descriptor directly to
rte_vhost_driver_register() instead of a socket path.  This way
the user application may choose to use an abstract socket or a file
socket or any other future type of socket connections.  This will
also allow user application to store these sockets somewhere, or
receive them from systemd/init/other management software.

Yes, sounds useful.

Stefan