Re: [PATCH v2 net-next 1/5] net: switchdev: move SWITCHDEV_FDB_{ADD,DEL}_TO_DEVICE to the blocking notifier chain
flat view
From: Vlad Buslov <hidden>
Date: 2021-08-19 18:15:40
On Thu 19 Aug 2021 at 19:07, Vladimir Oltean [off-list ref] wrote:
Currently, br_switchdev_fdb_notify() uses call_switchdev_notifiers (and
br_fdb_replay() open-codes the same thing). This means that drivers
handle the SWITCHDEV_FDB_{ADD,DEL}_TO_DEVICE events on the atomic
switchdev notifier block.
Most existing switchdev drivers either talk to firmware, or to a device
over a bus where the I/O is sleepable (SPI, I2C, MDIO etc). So there
exists an (anti)pattern where drivers make a sleepable context for
offloading the given FDB entry by registering an ordered workqueue and
scheduling work items on it, and doing all the work from there.
The problem is the inherent limitation that this design imposes upon
what a switchdev driver can do with those FDB entries.
For example, a switchdev driver might want to perform FDB isolation,
i.e. associate each FDB entry with the bridge it belongs to. Maybe the
driver associates each bridge with a number, allocating that number when
the first port of the driver joins that bridge, and freeing it when the
last port leaves it.
And this is where the problem is. When user space deletes a bridge and
all the ports leave, the bridge will notify us of the deletion of all
FDB entries in atomic context, and switchdev drivers will schedule their
private work items on their private workqueue.
The FDB entry deletion notifications will succeed, the bridge will then
finish deleting itself, but the switchdev work items have not run yet.
When they will eventually get scheduled, the aforementioned association
between the bridge_dev and a number will have already been broken by the
switchdev driver. All ports are standalone now, the bridge is a foreign
interface!
One might say "why don't you cache all your associations while you're
still in the atomic context and they're still valid, pass them by value
through your switchdev_work and work with the cached values as opposed
to the current ones?"
This option smells of poor design, because instead of fixing a central
problem, we add tens of lateral workarounds to avoid it. It should be
easier to use switchdev, not harder, and we should look at the common
patterns which lead to code duplication and eliminate them.
In this case, we must notice that
(a) switchdev already has the concept of notifiers emitted from the fast
path that are still processed by drivers from blocking context. This
is accomplished through the SWITCHDEV_F_DEFER flag which is used by
e.g. SWITCHDEV_OBJ_ID_HOST_MDB.
(b) the bridge del_nbp() function already calls switchdev_deferred_process().
So if we could hook into that, we could have a chance that the
bridge simply waits for our FDB entry offloading procedure to finish
before it calls netdev_upper_dev_unlink() - which is almost
immediately afterwards, and also when switchdev drivers typically
break their stateful associations between the bridge upper and
private data.
So it is in fact possible to use switchdev's generic
switchdev_deferred_enqueue mechanism to get a sleepable callback, and
from there we can call_switchdev_blocking_notifiers().
In the case of br_fdb_replay(), the only code path is from
switchdev_bridge_port_offload(), which is already in blocking context.
So we don't need to go through switchdev_deferred_enqueue, and we can
just call the blocking notifier block directly.
To preserve the same behavior as before, all drivers need to have their
SWITCHDEV_FDB_{ADD,DEL}_TO_DEVICE handlers moved from their switchdev
atomic notifier blocks to the blocking ones. This patch attempts to make
that trivial movement. Note that now they might schedule a work item for
nothing (since they are now called from a work item themselves), but I
don't have the energy or hardware to test all of them, so this will have
to do.
Note that previously, we were under rcu_read_lock() but now we're not.
I have eyeballed the drivers that make any sort of RCU assumption and
for the most part, enclosed them between a private pair of
rcu_read_lock() and rcu_read_unlock(). The exception is
qeth_l2_switchdev_event, for which adding the rcu_read_lock and properly
calling rcu_read_unlock from all places that return would result in more
churn than what I am about to do. This function had an apparently bogus
comment "Called under rtnl_lock", but to me this is not quite possible,
since this is the handler function from register_switchdev_notifier
which is on an atomic chain. But anyway, after the rework we _are_ under
rtnl_mutex, so just drop the _rcu from the functions used by the qeth
driver.
The RCU protection can be dropped from the other drivers when they are
reworked to stop scheduling.
Signed-off-by: Vladimir Oltean <vladimir.oltean@nxp.com>
---[...]
quoted hunk ↗ jump to hunk
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/rep/bridge.c b/drivers/net/ethernet/mellanox/mlx5/core/en/rep/bridge.c index 0c38c2e319be..ea7c3f07f6fe 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en/rep/bridge.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en/rep/bridge.c@@ -276,6 +276,55 @@ mlx5_esw_bridge_port_obj_attr_set(struct net_device *dev, return err; } +static struct mlx5_bridge_switchdev_fdb_work * +mlx5_esw_bridge_init_switchdev_fdb_work(struct net_device *dev, bool add, + struct switchdev_notifier_fdb_info *fdb_info, + struct mlx5_esw_bridge_offloads *br_offloads); + +static int +mlx5_esw_bridge_fdb_event(struct net_device *dev, unsigned long event, + struct switchdev_notifier_info *info, + struct mlx5_esw_bridge_offloads *br_offloads) +{ + struct switchdev_notifier_fdb_info *fdb_info; + struct mlx5_bridge_switchdev_fdb_work *work; + struct mlx5_eswitch *esw = br_offloads->esw; + u16 vport_num, esw_owner_vhca_id; + struct net_device *upper, *rep; + + upper = netdev_master_upper_dev_get_rcu(dev); + if (!upper) + return 0; + if (!netif_is_bridge_master(upper)) + return 0; + + rep = mlx5_esw_bridge_rep_vport_num_vhca_id_get(dev, esw, + &vport_num, + &esw_owner_vhca_id); + if (!rep) + return 0; + + /* only handle the event on peers */ + if (mlx5_esw_bridge_is_local(dev, rep, esw)) + return 0;
This check is only needed for SWITCHDEV_FDB_DEL_TO_BRIDGE case. Here it breaks the offload.
quoted hunk ↗ jump to hunk
+ + fdb_info = container_of(info, struct switchdev_notifier_fdb_info, info); + + work = mlx5_esw_bridge_init_switchdev_fdb_work(dev, + event == SWITCHDEV_FDB_ADD_TO_DEVICE, + fdb_info,
Here FDB info can already be deallocated[1] since this is now executing asynchronously and races with fdb_rcu_free() that is scheduled to be called after rcu grace period by fdb_delete().
quoted hunk ↗ jump to hunk
+ br_offloads); + if (IS_ERR(work)) { + WARN_ONCE(1, "Failed to init switchdev work, err=%ld", + PTR_ERR(work)); + return PTR_ERR(work); + } + + queue_work(br_offloads->wq, &work->work); + + return 0; +} + static int mlx5_esw_bridge_event_blocking(struct notifier_block *nb, unsigned long event, void *ptr) {@@ -295,6 +344,12 @@ static int mlx5_esw_bridge_event_blocking(struct notifier_block *nb, case SWITCHDEV_PORT_ATTR_SET: err = mlx5_esw_bridge_port_obj_attr_set(dev, ptr, br_offloads); break; + case SWITCHDEV_FDB_ADD_TO_DEVICE: + case SWITCHDEV_FDB_DEL_TO_DEVICE: + rcu_read_lock(); + err = mlx5_esw_bridge_fdb_event(dev, event, ptr, br_offloads); + rcu_read_unlock(); + break; default: err = 0; }@@ -415,9 +470,7 @@ static int mlx5_esw_bridge_switchdev_event(struct notifier_block *nb, /* only handle the event on peers */ if (mlx5_esw_bridge_is_local(dev, rep, esw)) break;
I really like the idea of completely remove the driver wq from FDB
handling code, but I'm not yet too familiar with bridge internals to
easily determine whether same approach can be applied to
SWITCHDEV_FDB_{ADD|DEL}_TO_BRIDGE event after this series is accepted.
It seems that all current users already generate these events from
blocking context, so would it be a trivial change for me to do in your
opinion? That would allow me to get rid of mlx5_esw_bridge_offloads->wq
in our driver.
quoted hunk ↗ jump to hunk
- fallthrough; - case SWITCHDEV_FDB_ADD_TO_DEVICE: - case SWITCHDEV_FDB_DEL_TO_DEVICE: + fdb_info = container_of(info, struct switchdev_notifier_fdb_info, info);
[...]
[1]:
[ 579.633363] ==================================================================
[ 579.634922] BUG: KASAN: use-after-free in mlx5_esw_bridge_init_switchdev_fdb_work+0x363/0x400 [mlx5_core]
[ 579.636969] Read of size 4 at addr ffff888130175d90 by task ip/7454
[ 579.638898] CPU: 0 PID: 7454 Comm: ip Not tainted 5.14.0-rc5+ #7
[ 579.640549] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.13.0-0-gf21b5a4aeb02-prebuilt.qemu.org 04/01/2014
[ 579.643617] Call Trace:
[ 579.644444] dump_stack_lvl+0x46/0x5a
[ 579.645568] print_address_description.constprop.0+0x1f/0x140
[ 579.647195] ? mlx5_esw_bridge_init_switchdev_fdb_work+0x363/0x400 [mlx5_core]
[ 579.649365] ? mlx5_esw_bridge_init_switchdev_fdb_work+0x363/0x400 [mlx5_core]
[ 579.651203] kasan_report.cold+0x83/0xdf
[ 579.652035] ? mlx5_esw_bridge_init_switchdev_fdb_work+0x363/0x400 [mlx5_core]
[ 579.653570] mlx5_esw_bridge_init_switchdev_fdb_work+0x363/0x400 [mlx5_core]
[ 579.655005] mlx5_esw_bridge_event_blocking+0x346/0x610 [mlx5_core]
[ 579.656328] ? mlx5_esw_bridge_port_obj_attr_set+0x320/0x320 [mlx5_core]
[ 579.657708] ? rwsem_mark_wake+0x7e0/0x7e0
[ 579.658599] ? rwsem_down_read_slowpath+0x142/0xad0
[ 579.659653] blocking_notifier_call_chain+0xdb/0x130
[ 579.660724] ? switchdev_fdb_add_deferred+0x1b0/0x1b0
[ 579.661813] switchdev_fdb_del_deferred+0x10c/0x1b0
[ 579.662871] ? switchdev_fdb_add_deferred+0x1b0/0x1b0
[ 579.663964] ? _raw_spin_lock+0xd0/0xd0
[ 579.664825] ? switchdev_deferred_process+0x175/0x290
[ 579.665912] ? kfree+0xa8/0x420
[ 579.666656] switchdev_deferred_process+0x12f/0x290
[ 579.667715] del_nbp+0x35c/0xcb0 [bridge]
[ 579.668623] br_dev_delete+0x8d/0x190 [bridge]
[ 579.669609] rtnl_dellink+0x2cb/0x9b0
[ 579.670456] ? unwind_next_frame+0x11fb/0x1a40
[ 579.671431] ? rtnl_bridge_getlink+0x650/0x650
[ 579.672403] ? deref_stack_reg+0xe6/0x160
[ 579.673291] ? unwind_next_frame+0x11fb/0x1a40
[ 579.674273] ? arch_stack_walk+0x9e/0xf0
[ 579.675152] ? mutex_lock+0xa1/0xf0
[ 579.675947] ? __mutex_lock_slowpath+0x10/0x10
[ 579.676922] rtnetlink_rcv_msg+0x359/0x9a0
[ 579.677838] ? rtnl_calcit.isra.0+0x2b0/0x2b0
[ 579.678795] ? ___sys_sendmsg+0xd8/0x160
[ 579.679669] ? __sys_sendmsg+0xb7/0x140
[ 579.680532] ? do_syscall_64+0x3b/0x90
[ 579.681426] ? entry_SYSCALL_64_after_hwframe+0x44/0xae
[ 579.682540] ? kasan_save_stack+0x32/0x40
[ 579.683429] ? kasan_save_stack+0x1b/0x40
[ 579.684321] ? kasan_record_aux_stack+0xa3/0xb0
[ 579.685308] ? task_work_add+0x3a/0x130
[ 579.686168] ? fput_many.part.0+0x8c/0x110
[ 579.687071] ? path_openat+0x1e02/0x3960
[ 579.687944] ? do_filp_open+0x19e/0x3e0
[ 579.699734] ? do_sys_openat2+0x122/0x360
[ 579.700627] ? __x64_sys_openat+0x120/0x1d0
[ 579.701548] ? do_syscall_64+0x3b/0x90
[ 579.702350] netlink_rcv_skb+0x120/0x350
[ 579.703180] ? rtnl_calcit.isra.0+0x2b0/0x2b0
[ 579.704084] ? netlink_ack+0x9c0/0x9c0
[ 579.704880] ? netlink_deliver_tap+0x7f/0x8f0
[ 579.705777] ? _copy_from_iter+0x277/0xdb0
[ 579.706641] netlink_unicast+0x4c6/0x7a0
[ 579.707470] ? netlink_attachskb+0x750/0x750
[ 579.708352] ? __build_skb_around+0x1f9/0x2b0
[ 579.709250] ? __check_object_size+0x23e/0x300
[ 579.710171] netlink_sendmsg+0x70a/0xbf0
[ 579.711045] ? netlink_unicast+0x7a0/0x7a0
[ 579.711951] ? __import_iovec+0x51/0x610
[ 579.712825] ? netlink_unicast+0x7a0/0x7a0
[ 579.713736] sock_sendmsg+0xe4/0x110
[ 579.714555] ____sys_sendmsg+0x5cf/0x7d0
[ 579.715429] ? kernel_sendmsg+0x30/0x30
[ 579.716292] ? __ia32_sys_recvmmsg+0x210/0x210
[ 579.717265] ? trace_event_raw_event_mmap_lock_released+0x240/0x240
[ 579.718566] ? lru_cache_add+0x17d/0x2a0
[ 579.719440] ? wp_page_copy+0x87c/0x1370
[ 579.720315] ___sys_sendmsg+0xd8/0x160
[ 579.721156] ? sendmsg_copy_msghdr+0x110/0x110
[ 579.722142] ? do_wp_page+0x1d1/0xf50
[ 579.722970] ? __handle_mm_fault+0x1c96/0x3390
[ 579.723943] ? vm_iomap_memory+0x170/0x170
[ 579.724855] ? __fget_light+0x51/0x220
[ 579.725696] __sys_sendmsg+0xb7/0x140
[ 579.726526] ? __sys_sendmsg_sock+0x20/0x20
[ 579.727450] ? copy_page_range+0x14c0/0x2a40
[ 579.728389] do_syscall_64+0x3b/0x90
[ 579.729199] entry_SYSCALL_64_after_hwframe+0x44/0xae
[ 579.730285] RIP: 0033:0x7feb5f746c17
[ 579.731099] Code: 0c 00 f7 d8 64 89 02 48 c7 c0 ff ff ff ff eb b7 0f 1f 00 f3 0f 1e fa 64 8b 04 25 18 00 00 00 85 c0
[ 579.734799] RSP: 002b:00007fff12a9e948 EFLAGS: 00000246 ORIG_RAX: 000000000000002e
[ 579.736403] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007feb5f746c17
[ 579.737857] RDX: 0000000000000000 RSI: 00007fff12a9e9b0 RDI: 0000000000000003
[ 579.739316] RBP: 00000000611e94b8 R08: 0000000000000001 R09: 0000000000403578
[ 579.740770] R10: 00007feb5f8948b0 R11: 0000000000000246 R12: 0000000000000001
[ 579.742226] R13: 00007fff12a9f060 R14: 0000000000000000 R15: 000000000048e520
[ 579.744115] Allocated by task 0:
[ 579.744872] kasan_save_stack+0x1b/0x40
[ 579.745730] __kasan_slab_alloc+0x61/0x80
[ 579.746623] kmem_cache_alloc+0x14c/0x2f0
[ 579.747515] fdb_create+0x32/0xc30 [bridge]
[ 579.748450] br_fdb_update+0x301/0x730 [bridge]
[ 579.749444] br_handle_frame_finish+0x5f7/0x1690 [bridge]
[ 579.750611] br_handle_frame+0x55f/0x910 [bridge]
[ 579.751647] __netif_receive_skb_core+0xfc3/0x2a10
[ 579.752680] __netif_receive_skb_list_core+0x2ef/0x900
[ 579.753777] netif_receive_skb_list_internal+0x5f4/0xc60
[ 579.754933] napi_complete_done+0x188/0x5d0
[ 579.755856] mlx5e_napi_poll+0x2bc/0x1680 [mlx5_core]
[ 579.757014] __napi_poll+0xa1/0x420
[ 579.757808] net_rx_action+0x2c4/0x950
[ 579.758655] __do_softirq+0x1a0/0x57f
[ 579.759918] Freed by task 0:
[ 579.760600] kasan_save_stack+0x1b/0x40
[ 579.761464] kasan_set_track+0x1c/0x30
[ 579.762321] kasan_set_free_info+0x20/0x30
[ 579.763225] __kasan_slab_free+0xeb/0x120
[ 579.764115] kmem_cache_free+0x82/0x3f0
[ 579.764978] rcu_do_batch+0x32f/0xba0
[ 579.765802] rcu_core+0x4c4/0x910
[ 579.766560] __do_softirq+0x1a0/0x57f
[ 579.767804] Last potentially related work creation:
[ 579.768820] kasan_save_stack+0x1b/0x40
[ 579.769660] kasan_record_aux_stack+0xa3/0xb0
[ 579.770599] call_rcu+0xe3/0x1230
[ 579.771367] br_fdb_delete_by_port+0x1d7/0x270 [bridge]
[ 579.772468] br_stp_disable_port+0x150/0x180 [bridge]
[ 579.773541] del_nbp+0x11e/0xcb0 [bridge]
[ 579.774435] br_dev_delete+0x8d/0x190 [bridge]
[ 579.775391] rtnl_dellink+0x2cb/0x9b0
[ 579.776218] rtnetlink_rcv_msg+0x359/0x9a0
[ 579.777123] netlink_rcv_skb+0x120/0x350
[ 579.778035] netlink_unicast+0x4c6/0x7a0
[ 579.778904] netlink_sendmsg+0x70a/0xbf0
[ 579.779777] sock_sendmsg+0xe4/0x110
[ 579.780589] ____sys_sendmsg+0x5cf/0x7d0
[ 579.781462] ___sys_sendmsg+0xd8/0x160
[ 579.782315] __sys_sendmsg+0xb7/0x140
[ 579.783144] do_syscall_64+0x3b/0x90
[ 579.783959] entry_SYSCALL_64_after_hwframe+0x44/0xae
[ 579.785467] The buggy address belongs to the object at ffff888130175d80
which belongs to the cache bridge_fdb_cache of size 128
[ 579.788085] The buggy address is located 16 bytes inside of
128-byte region [ffff888130175d80, ffff888130175e00)
[ 579.790432] The buggy address belongs to the page:
[ 579.791461] page:0000000044cdd676 refcount:1 mapcount:0 mapping:0000000000000000 index:0xffff888130175cc0 pfn:0x130175
[ 579.795093] raw: 0017ffffc0000200 0000000000000000 dead000000000122 ffff88811ea56140
[ 579.796733] raw: ffff888130175cc0 0000000080150009 00000001ffffffff 0000000000000000
[ 579.798380] page dumped because: kasan: bad access detected
[ 579.799984] Memory state around the buggy address:
[ 579.801019] ffff888130175c80: fc fc fc fc fc fc fc fc fb fb fb fb fb fb fb fb
[ 579.802566] ffff888130175d00: fb fb fb fb fb fb fb fb fc fc fc fc fc fc fc fc
[ 579.804107] >ffff888130175d80: fa fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
[ 579.805654] ^
[ 579.806488] ffff888130175e00: fc fc fc fc fc fc fc fc fb fb fb fb fb fb fb fb
[ 579.807945] ffff888130175e80: fb fb fb fb fb fb fb fb fc fc fc fc fc fc fc fc
[ 579.809398] ==================================================================
[ 579.810865] Disabling lock debugging due to kernel taint
[ 579.811956] ==================================================================
[ 579.813429] BUG: KASAN: use-after-free in mlx5_esw_bridge_init_switchdev_fdb_work+0x339/0x400 [mlx5_core]
[ 579.815432] Read of size 2 at addr ffff888130175d94 by task ip/7454
[ 579.817174] CPU: 0 PID: 7454 Comm: ip Tainted: G B 5.14.0-rc5+ #7
[ 579.818758] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.13.0-0-gf21b5a4aeb02-prebuilt.qemu.org 04/01/2014
[ 579.821035] Call Trace:
[ 579.821649] dump_stack_lvl+0x46/0x5a
[ 579.822492] print_address_description.constprop.0+0x1f/0x140
[ 579.823706] ? mlx5_esw_bridge_init_switchdev_fdb_work+0x339/0x400 [mlx5_core]
[ 579.825317] ? mlx5_esw_bridge_init_switchdev_fdb_work+0x339/0x400 [mlx5_core]
[ 579.826845] kasan_report.cold+0x83/0xdf
[ 579.827674] ? mlx5_esw_bridge_init_switchdev_fdb_work+0x339/0x400 [mlx5_core]
[ 579.829202] mlx5_esw_bridge_init_switchdev_fdb_work+0x339/0x400 [mlx5_core]
[ 579.830638] mlx5_esw_bridge_event_blocking+0x346/0x610 [mlx5_core]
[ 579.831933] ? mlx5_esw_bridge_port_obj_attr_set+0x320/0x320 [mlx5_core]
[ 579.833308] ? rwsem_mark_wake+0x7e0/0x7e0
[ 579.834219] ? rwsem_down_read_slowpath+0x142/0xad0
[ 579.835271] blocking_notifier_call_chain+0xdb/0x130
[ 579.836345] ? switchdev_fdb_add_deferred+0x1b0/0x1b0
[ 579.837427] switchdev_fdb_del_deferred+0x10c/0x1b0
[ 579.838484] ? switchdev_fdb_add_deferred+0x1b0/0x1b0
[ 579.839577] ? _raw_spin_lock+0xd0/0xd0
[ 579.840439] ? switchdev_deferred_process+0x175/0x290
[ 579.841518] ? kfree+0xa8/0x420
[ 579.842256] switchdev_deferred_process+0x12f/0x290
[ 579.843317] del_nbp+0x35c/0xcb0 [bridge]
[ 579.844228] br_dev_delete+0x8d/0x190 [bridge]
[ 579.845212] rtnl_dellink+0x2cb/0x9b0
[ 579.846045] ? unwind_next_frame+0x11fb/0x1a40
[ 579.847023] ? rtnl_bridge_getlink+0x650/0x650
[ 579.847994] ? deref_stack_reg+0xe6/0x160
[ 579.848879] ? unwind_next_frame+0x11fb/0x1a40
[ 579.849850] ? arch_stack_walk+0x9e/0xf0
[ 579.850731] ? mutex_lock+0xa1/0xf0
[ 579.851530] ? __mutex_lock_slowpath+0x10/0x10
[ 579.852499] rtnetlink_rcv_msg+0x359/0x9a0
[ 579.853406] ? rtnl_calcit.isra.0+0x2b0/0x2b0
[ 579.854399] ? ___sys_sendmsg+0xd8/0x160
[ 579.855275] ? __sys_sendmsg+0xb7/0x140
[ 579.856135] ? do_syscall_64+0x3b/0x90
[ 579.856984] ? entry_SYSCALL_64_after_hwframe+0x44/0xae
[ 579.858101] ? kasan_save_stack+0x32/0x40
[ 579.858995] ? kasan_save_stack+0x1b/0x40
[ 579.859887] ? kasan_record_aux_stack+0xa3/0xb0
[ 579.860878] ? task_work_add+0x3a/0x130
[ 579.861751] ? fput_many.part.0+0x8c/0x110
[ 579.862677] ? path_openat+0x1e02/0x3960
[ 579.863551] ? do_filp_open+0x19e/0x3e0
[ 579.864413] ? do_sys_openat2+0x122/0x360
[ 579.865303] ? __x64_sys_openat+0x120/0x1d0
[ 579.877159] ? do_syscall_64+0x3b/0x90
[ 579.878007] netlink_rcv_skb+0x120/0x350
[ 579.878834] ? rtnl_calcit.isra.0+0x2b0/0x2b0
[ 579.879733] ? netlink_ack+0x9c0/0x9c0
[ 579.880534] ? netlink_deliver_tap+0x7f/0x8f0
[ 579.881429] ? _copy_from_iter+0x277/0xdb0
[ 579.882291] netlink_unicast+0x4c6/0x7a0
[ 579.883122] ? netlink_attachskb+0x750/0x750
[ 579.884010] ? __build_skb_around+0x1f9/0x2b0
[ 579.884906] ? __check_object_size+0x23e/0x300
[ 579.885819] netlink_sendmsg+0x70a/0xbf0
[ 579.886654] ? netlink_unicast+0x7a0/0x7a0
[ 579.887565] ? __import_iovec+0x51/0x610
[ 579.888440] ? netlink_unicast+0x7a0/0x7a0
[ 579.889344] sock_sendmsg+0xe4/0x110
[ 579.890163] ____sys_sendmsg+0x5cf/0x7d0
[ 579.891047] ? kernel_sendmsg+0x30/0x30
[ 579.891908] ? __ia32_sys_recvmmsg+0x210/0x210
[ 579.892884] ? trace_event_raw_event_mmap_lock_released+0x240/0x240
[ 579.894198] ? lru_cache_add+0x17d/0x2a0
[ 579.895084] ? wp_page_copy+0x87c/0x1370
[ 579.895960] ___sys_sendmsg+0xd8/0x160
[ 579.896803] ? sendmsg_copy_msghdr+0x110/0x110
[ 579.897744] ? do_wp_page+0x1d1/0xf50
[ 579.898537] ? __handle_mm_fault+0x1c96/0x3390
[ 579.899450] ? vm_iomap_memory+0x170/0x170
[ 579.900313] ? __fget_light+0x51/0x220
[ 579.901114] __sys_sendmsg+0xb7/0x140
[ 579.901898] ? __sys_sendmsg_sock+0x20/0x20
[ 579.902774] ? copy_page_range+0x14c0/0x2a40
[ 579.903666] do_syscall_64+0x3b/0x90
[ 579.904440] entry_SYSCALL_64_after_hwframe+0x44/0xae
[ 579.905461] RIP: 0033:0x7feb5f746c17
[ 579.906237] Code: 0c 00 f7 d8 64 89 02 48 c7 c0 ff ff ff ff eb b7 0f 1f 00 f3 0f 1e fa 64 8b 04 25 18 00 00 00 85 c0 7
[ 579.909720] RSP: 002b:00007fff12a9e948 EFLAGS: 00000246 ORIG_RAX: 000000000000002e
[ 579.911339] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007feb5f746c17
[ 579.912803] RDX: 0000000000000000 RSI: 00007fff12a9e9b0 RDI: 0000000000000003
[ 579.914271] RBP: 00000000611e94b8 R08: 0000000000000001 R09: 0000000000403578
[ 579.915737] R10: 00007feb5f8948b0 R11: 0000000000000246 R12: 0000000000000001
[ 579.917200] R13: 00007fff12a9f060 R14: 0000000000000000 R15: 000000000048e520
[ 579.919112] Allocated by task 0:
[ 579.919861] kasan_save_stack+0x1b/0x40
[ 579.920740] __kasan_slab_alloc+0x61/0x80
[ 579.921636] kmem_cache_alloc+0x14c/0x2f0
[ 579.922537] fdb_create+0x32/0xc30 [bridge]
[ 579.923470] br_fdb_update+0x301/0x730 [bridge]
[ 579.924463] br_handle_frame_finish+0x5f7/0x1690 [bridge]
[ 579.925613] br_handle_frame+0x55f/0x910 [bridge]
[ 579.926644] __netif_receive_skb_core+0xfc3/0x2a10
[ 579.927675] __netif_receive_skb_list_core+0x2ef/0x900
[ 579.928773] netif_receive_skb_list_internal+0x5f4/0xc60
[ 579.929896] napi_complete_done+0x188/0x5d0
[ 579.930828] mlx5e_napi_poll+0x2bc/0x1680 [mlx5_core]
[ 579.931985] __napi_poll+0xa1/0x420
[ 579.932785] net_rx_action+0x2c4/0x950
[ 579.933633] __do_softirq+0x1a0/0x57f
[ 579.934896] Freed by task 0:
[ 579.935586] kasan_save_stack+0x1b/0x40
[ 579.936443] kasan_set_track+0x1c/0x30
[ 579.937285] kasan_set_free_info+0x20/0x30
[ 579.938199] __kasan_slab_free+0xeb/0x120
[ 579.939087] kmem_cache_free+0x82/0x3f0
[ 579.939945] rcu_do_batch+0x32f/0xba0
[ 579.940777] rcu_core+0x4c4/0x910
[ 579.941542] __do_softirq+0x1a0/0x57f
[ 579.942806] Last potentially related work creation:
[ 579.943855] kasan_save_stack+0x1b/0x40
[ 579.944710] kasan_record_aux_stack+0xa3/0xb0
[ 579.945664] call_rcu+0xe3/0x1230
[ 579.946430] br_fdb_delete_by_port+0x1d7/0x270 [bridge]
[ 579.947557] br_stp_disable_port+0x150/0x180 [bridge]
[ 579.948649] del_nbp+0x11e/0xcb0 [bridge]
[ 579.949552] br_dev_delete+0x8d/0x190 [bridge]
[ 579.950536] rtnl_dellink+0x2cb/0x9b0
[ 579.951365] rtnetlink_rcv_msg+0x359/0x9a0
[ 579.952267] netlink_rcv_skb+0x120/0x350
[ 579.953145] netlink_unicast+0x4c6/0x7a0
[ 579.954029] netlink_sendmsg+0x70a/0xbf0
[ 579.954897] sock_sendmsg+0xe4/0x110
[ 579.955707] ____sys_sendmsg+0x5cf/0x7d0
[ 579.956584] ___sys_sendmsg+0xd8/0x160
[ 579.957427] __sys_sendmsg+0xb7/0x140
[ 579.958254] do_syscall_64+0x3b/0x90
[ 579.959069] entry_SYSCALL_64_after_hwframe+0x44/0xae
[ 579.960584] The buggy address belongs to the object at ffff888130175d80
which belongs to the cache bridge_fdb_cache of size 128
[ 579.963194] The buggy address is located 20 bytes inside of
128-byte region [ffff888130175d80, ffff888130175e00)
[ 579.965550] The buggy address belongs to the page:
[ 579.966596] page:0000000044cdd676 refcount:1 mapcount:0 mapping:0000000000000000 index:0xffff888130175cc0 pfn:0x130175
[ 579.968774] flags: 0x17ffffc0000200(slab|node=0|zone=2|lastcpupid=0x1fffff)
[ 579.970212] raw: 0017ffffc0000200 0000000000000000 dead000000000122 ffff88811ea56140
[ 579.971860] raw: ffff888130175cc0 0000000080150009 00000001ffffffff 0000000000000000
[ 579.973495] page dumped because: kasan: bad access detected
[ 579.975112] Memory state around the buggy address:
[ 579.976140] ffff888130175c80: fc fc fc fc fc fc fc fc fb fb fb fb fb fb fb fb
[ 579.977693] ffff888130175d00: fb fb fb fb fb fb fb fb fc fc fc fc fc fc fc fc
[ 579.979241] >ffff888130175d80: fa fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
[ 579.980789] ^
[ 579.981632] ffff888130175e00: fc fc fc fc fc fc fc fc fb fb fb fb fb fb fb fb
[ 579.983183] ffff888130175e80: fb fb fb fb fb fb fb fb fc fc fc fc fc fc fc fc
[ 579.984724] ==================================================================