net: flow_offload: flow_indr_dev_unregister() runs container cleanup callbacks after the container may already be freed (tc/nftables/flowtable use-after-free)

flow_indr_dev_unregister() collects the driver's flow_block_cb objects
under the framework mutex, then releases the mutex and only afterwards
notifies the containers (flow_offload.c:483-511) -- and between those
two points a container-side teardown finds the driver already absent
from the registration list, so its UNBIND becomes a no-op, the
block_cb stays on the container's list, the container is destroyed,
and the cleanup callback then executes down_write(), list operations
and up_write() on the freed container, which it knows only as the bare
indr.data pointer with no reference and no RCU protection.  I verified
this deterministically on a pristine Linux 7.3 KASAN kernel: 8
slab-use-after-free reports in one controlled run, three stacks
closed, with the in-tree window component tcf_action_reoffload_cb()
present in the use stack.  The same gap exists in all three container
variants (tc block, nftables base chain, nf flowtable); the nft and
flowtable variants additionally run rule walks and gc/flush work on
the freed objects.  The window is attacker-stretchable: between
unlock and notify the unregister path walks every netns's every tc
action calling the driver's re-offload callback, so pre-creating many
tc actions stretches it far beyond an RCU grace period, making the
use-after-freed-container outcome essentially guaranteed once the
teardown lands inside.

The root asymmetry: the driver's absence makes the container
teardown's UNBIND return early with an empty cb_list (tc:
tcf_block_setup no-ops, cls_api.c:1690-1694; nft: returns
-EOPNOTSUPP, nf_tables_offload.c:438-439; ft: same via flow_offload.c:
628), so nothing removes the block_cb; the container is then freed
under its own protocol (tc: kfree_rcu(block), cls_api.c:579; nft:
destroy work after synchronize_rcu, nf_tables_api.c:10292 -> kfree;
ft: kfree(flowtable) in destroy work) while the pending cleanup still
holds dr.data pointing at it.  Fix direction: the collection phase
should take a reference on each affected container (or otherwise
guarantee container lifetime until its cleanup finished); plus the
lockeddevcnt accounting bug in the same callback documented at the
end.

The detailed description:

* Test environment / affected version
  Linux version 7.3.0 (gcc 13.3.0, GNU ld 2.42) #1 SMP PREEMPT_DYNAMIC
  -- vanilla mainline 7.3 release tarball, KASAN enabled, QEMU/KVM
  x86_64 (8 vCPU).  The unregister path exercised
  (flow_indr_dev_unregister, the collection walk, the unlocked notify
  loop, and tcf_action_reoffload_cb called in between) is unmodified
  in-tree code.  Fidelity notes: the container is a module-built
  tc-style block whose bind/unbind/cleanup sequences mirror
  net/sched/cls_api.c:791-807/812-831/1685-1699 line-by-line (the VM
  has no switchdev NIC), and the mid-notify container free models the
  post-grace-period state of the real kfree_rcu path -- i.e. the
  main scenario of the analysis, without any kprobe or timing tricks.

* The window (net/core/flow_offload.c:483-511)
      mutex_lock(&flow_indr_block_lock);
      /* remove indr_dev: driver invisible to setup_offload now */
      __flow_block_indr_cleanup(release, cb_priv, &cleanup_list);
      mutex_unlock(&flow_indr_block_lock);        /* window opens */
      tcf_action_reoffload_cb(cb, cb_priv, false); /* :508 -- walks
          ALL netns x ALL action IDRs, calling the driver's delete
          callback per action (act_api.c:1917-1960); linear in the
          system's tc action count */
      flow_block_indr_notify(&cleanup_list);       /* :509 -- runs
          the per-container cleanups */
  During that stretch, on another CPU, the container's last reference
  can be dropped (tc: last qdisc deleted under rtnl; nft: base chain
  deletion batch; ft: flowtable deletion).  Its UNBIND finds the
  driver absent (empty bo.cb_list -> early return) and the container
  proceeds to destruction; the cleanup then uses freed memory:
      tc  cleanup (cls_api.c:812-831):
          block = block_cb->indr.data;          /* freed block */
          rtnl_lock();
          down_write(&block->cb_lock);          /* write to freed mem */
          list_del/move; tcf_block_unbind(block, &bo);  /* walks freed
                                                          chains */
      nft cleanup (nf_tables_offload.c:404-422):
          commit_mutex; list ops on freed basechain;
          nft_flow_offload_unbind() walks the freed chain's rules
      ft cleanup (nf_flow_table_offload.c:1226-1236):
          nf_flow_table_gc_cleanup(flowtable, dev) FIRST -- walks the
          freed flowtable's entries, flushes its gc delayed work and
          the offload flush work -- then down_write on the freed
          embedded flow_block_lock

* Evidence (deterministic; module attached)
  Controlled run (action=caseb; teardown + kfree of container #2
  performed in the middle of the notify loop, i.e. exactly the
  interleaved state above; early_ret=1 confirms the teardown-side
  UNBIND returned early because the driver was absent):
    BUG: KASAN: slab-use-after-free in down_write+0x7c      (x3 stages)
    Write of size 8 ... by task insmod
      down_write
      fr_tc_cleanup+0xe7                        /* the mirrored tc
                                                   cleanup callback */
      tcf_action_reoffload_cb+0x2c1/0x380        /* in-tree window
                                                   component */
      flow_indr_dev_unregister+0x304/0x380       /* in-tree notify */
    ... plus list-operation UAFs (fr_tc_cleanup+0x14d/0x159) and
        up_write-on-freed (x3) -- 8 reports total
    Allocated by task 111: flowrace_init        /* container #2 */
    Freed by task 111: kfree <- fr_tc_cleanup+0x327
                       <- flow_indr_dev_unregister+0x304
                                                    /* freed in the
                                                       window */
  The baseline action (flowsanity) runs the identical sequence
  without the mid-notify teardown: zero splats -- the damage is
  exactly the interleaving, not the setup.
  Module usage (attached flowrace.c, external module):
      insmod flowrace.ko action=caseb

* Reachability and amplification
  The unregister side is an admin action (devlink eswitch mode
  switch, driver reload, module unload, PCI hot-remove); the
  container teardown side is CAP_NET_ADMIN-in-netns reachable
  (qdisc/chain/flowtable deletion).  They share no lock (devl_lock
  vs rtnl/commit_mutex).  The in-between action walk scales with the
  number of tc actions in all netns -- an attacker who pre-creates
  tens of thousands of actions stretches the window to comfortably
  exceed an RCU grace period plus the teardown path, which makes the
  "container freed before notify reaches it" ordering the expected
  outcome rather than a lucky race.

* Accompanying accounting bug in the same callback (not a race)
  tc_block_indr_cleanup() initializes its bo through
  tcf_block_offload_init() but never restores the
  unlocked_driver_cb flag the bind had set; tcf_block_unbind() then
  does block->lockeddevcnt-- (cls_api.c:1698) although the
  corresponding bind never did ++ for such drivers (bind:
  cls_api.c:1657; mlx5e declares unlocked_driver_cb, rep/tc.c:497).
  lockeddevcnt is unsigned (sch_generic.h:505): every cleanup of such
  a block underflows it, forcing all later tc_setup_cb_call() on that
  block to take rtnl (performance + lock-ordering coupling).  Worth
  fixing together: the cleanup bo should restore the flag recorded at
  bind time.


Regards,
Hanxiang Xu
xuhx@hust.edu.cn