On Thu, 2019-02-07 at 08:48 +0100, Jesper Dangaard Brouer wrote:

On Wed, 6 Feb 2019 00:06:33 +0000 Saeed Mahameed <saeedm@mellanox.com

quoted

wrote:

quoted

On Mon, 2019-02-04 at 19:13 -0800, David Ahern wrote:

[...]

quoted

quoted

mlx5 needs some work. As I recall it still has the bug/panic
removing xdp programs - at least I don't recall seeing a patch
for
it.  

Only when xdp_redirect to mlx5, and removing the program while
redirect is happening, this is actually due to a lack of
synchronization means between different drivers, we have some ideas
to overcome this using a standard XDP API, or just use a hack in
mlx5
driver which i don't like:

https://git.kernel.org/pub/scm/linux/kernel/git/saeed/linux.git/commit/?h=topic/xdp-redirect-fix&id=a3652d03cc35fd3ad62744986c8ccaca74c9f20c

I will be working on this towards the end of this week.

Toke and I have been discussing how to solve this.

The main idea for fixing this is to tie resource allocation to
interface
insertion into interface maps (kernel/bpf/devmap.c). As the =devmap=
already have the needed synchronisation mechanisms and steps for
safely
adding and removing =net_devices= (e.g. stopping RX side, flushing
remaining frames, waiting RCU period before freeing objects, etc.)

As described here:
 
https://github.com/xdp-project/xdp-project/blob/master/xdp-project.org#better-ndo_xdp_xmit-resource-management

--Jesper

Yes you already suggested this approach @LPC:

So 
1) on dev_map_update_elem() we will call
dev->dev->ndo_bpf() to notify the device on the intention to start/stop
redirect, and wait for it to create/destroy the HW resources
before/after actually updating the map

But:
2) this won't totally solve our problem, since sometimes the driver can
decide to recreate (change of configuration) hw resources on the fly
while redirect/devmap is already happening, so we need some kind of a
dev_map_notification or a flag with rcu synch, for when the driver want
to make the xdp redirect resources unavailable.

Thanks,
Saeed.