Re: [PATCH v3] net: track raw dev_put()/dev_hold() usage under CONFIG_NET_DEV_REFCNT_TRACKER=y

From
Eric Dumazet <hidden>
Date
2026-09-14 11:44:16
Reply to
Tetsuo Handa, 2026-09-14
On Mon, Sep 14, 2026 at 3:24 AM Tetsuo Handa
[off-list ref] wrote:
On 2026/08/29 23:58, Tetsuo Handa wrote:> syzbot is still reporting
quoted
  unregister_netdevice: waiting for DEV to become free

problems. There is netdevice_tracker which helps identifying culprit
(...snipped...)
quoted
---
To: Eric
  Please see off-list mail [off-list ref] .

Changes in v3:
  - Repost as v3, as sashiko's review comments at
    https://sashiko.dev/#/patchset/56c707e7-1fb0-43ec-b8fb-cf6f451e513e@I-love.SAKURA.ne.jp
    was addressed. I assume that no more technical problem remains, and only how to split
    this patch into several patches for getting approval from relevant subsystems remains.
Sashiko did not find problems with this version
( https://sashiko.dev/#/patchset/271af679-34c1-4f7e-875c-4761c22595eb@I-love.SAKURA.ne.jp ) .
Any questions?
I am adding Dmitry and Aleksandr to Cc, because I think the best answer
to your problem is not a netdev one. More on this below.

First, sorry for letting this series sit for three rounds. You are one
of the very few people actually chasing the "unregister_netdevice:
waiting for %s to become free" reports, that work is not fun, and
silence is not the answer you deserved.

Let me also say clearly that I do not think the approach is bogus. Once
a reference is taken without a tracker, there is no slot to store a
cookie in, so counting +1/-1 per unique stack and looking at the balance
is essentially the only thing one can do. And 10 bugs fixed is more
evidence than most debug facilities come with.

My problem is the cost/benefit for mainline. This adds a field to
struct net_device and code to __dev_hold()/__dev_put(), which are inline
and expanded in thousands of places, in order to produce information
that is strictly weaker than what ref_tracker already gives us (a stack
balance cannot tell which hold was not released when the same call site
does both). It is a workaround for an incomplete migration, and we would
be carrying it forever. Even with a perfect implementation I would be
lukewarm, and I would rather tell you that now than stay silent again.

But you should not have to wait for netdev consensus to get the data you
need, so:

Could syzbot run a dedicated instance on a branch carrying this patch?

There is precedent: KMSAN lived out of tree for years with its own
syzbot instance, and that is exactly how it accumulated the evidence
that got it merged. Your real requirement, as I understand it, is not
"this must be in net-next", it is "the mechanism must be present in a
tree that syzbot builds", because the remaining bugs have no reproducer
and #syz test is therefore unavailable. A dedicated instance on, say,
linux-next plus your patch would give you continuous coverage on exactly
the bugs you cannot reproduce, immediately, and without anyone having to
agree first.

It is not free for you: you would own the branch and the rebases. But it
also turns the discussion around. If the instance produces a stream of
diagnosed imbalances over a few months, the case for upstreaming becomes
an empirical one rather than a design argument, and I would find it much
harder to object to.

Dmitry, Aleksandr, does that sound feasible?

If you would rather still pursue mainline inclusion, here is what I
would need, roughly in order of importance:

1) The per-call list walk has to go. Today every __dev_hold() /
   __dev_put() walks a per-device list doing memcmp() of up to 160 bytes
   per node. With a few thousand distinct traces on a device, that is
   hundreds of KB of memory traffic per refcount operation, and it gets
   worse precisely on the devices you are investigating. I suspect it
   costs much more than the stack_trace_save() itself. stack_depot_save()
   gives you a handle, and a small hash of { handle -> atomic count }
   makes this O(1), removes the 6 MiB static array, the global raw
   spinlock, the sticky "exhausted" flag and the dedup pass at dump time.
   Ideally this lives in lib/ref_tracker.c as a "count only" mode, so
   other ref_tracker users benefit and net/core/dev.c gains a few lines.

2) Do not overload CONFIG_NET_DEV_REFCNT_TRACKER. People enable it today
   expecting the cost of one stackdepot entry per tracked reference. A
   separate option (default n, depends on DEBUG_KERNEL), ideally behind a
   static key, lets them opt in knowingly. Some numbers on the slowdown
   would help too.

3) The kernel/softirq.c, kernel/rcu/tree.c, kernel/workqueue.c and
   net/socket.c hunks are not needed, and this one is free for you:
   filter_irq_stacks() (include/linux/stackdepot.h), already used by
   KASAN, KFENCE and kmemleak, trims the stack at the
   .irqentry/.softirqentry boundary, which is what your open-coded
   trimming does. That removes the noinline declarations, the boot time
   timer + kallsyms dance, and the need for acks outside netdev. The
   remaining name matching ("process_one_work", "__sys_bind", ...) will
   rot silently; capping the depth should be enough.

4) The dump is called from the 10 seconds warning loop, so a device that
   never goes away prints 100+ traces every 10 seconds forever. That
   message is not rare in the field, and the flood can cause hung task
   splats and RCU stalls of its own. Please print once, or behind a
   ratelimit / an explicit sysctl.

Two minor ones: in alloc_netdev_mqs() the erase_netdev_trace_buffer()
call sits inside the #ifdef CONFIG_PCPU_DEV_REFCNT block, so entries leak
on that path when it is unset; and netdev_trace_buffer_exhausted is never
cleared, so one unlucky device permanently disables the balance line for
all others.

Last thing, on netdevice_tracker. I know your view is that nobody will
convert the remaining callers, and you are not wrong that it has been
slow. But a converted call site gives paired data, which is strictly
better than a balance, and it is a permanent improvement rather than a
debug facility. If you send me the list of remaining raw
dev_hold()/dev_put() users, I will take a share of the conversions and
review the rest, subsystem by subsystem. That is a burden you should not
be carrying alone, and it is a fair thing for you to ask of us.

One small request: an automated reviewer not finding problems is not an
argument for merging, and leading with it makes it harder, not easier,
for people to engage with the series. The objections here were never
about bugs in the code.

Thanks for sticking with this.
Thread (6 messages) 6 messages, 4 authors, last reply 2026-09-15
Keyboard shortcuts
hback out one level
jnext message in thread / next series-diff position
kprevious message in thread / previous series-diff position
ldrill in
Escclose help / fold thread tree
?toggle this help