On Wed, Oct 26, 2022 at 09:40:11PM +0200, Michael Lilja wrote:

Hi,

I just quickly tried following the flow_offload_teardown() path instead of clearing IPS_OFFLOAD in flow_offload_del() and it does have some side effects. The flow is added again before the HW has actually reported it to be NF_FLOW_HW_DEAD. 

The sequence with my patch is:
  : Retire -> Remove from hw tables -> Remove from sw tables -> kfree(flow) -> flow_offload_add()

But if flow_offload_teardown() is called on expire I see:
  : Retire -> Remove from hw tables -> flow_offload_add() -> Remove from sw tables -> kfree(flow)
 
I need to investigate why this happens, maybe the IPS_OFFLOAD flag is cleared too early and should not be cleared until the flow is actually removed, like I do? Maybe the issue is not seen before because on timeout or flow_is_dying() no packet arrive to create the flow again prematurely?

Hm, IPS_OFFLOAD should be cleared from flow_offload_del() then, it is
cleared too early.

I'll post a fix for nf.git first then I propose to follow up on this
flowtable feature. I'll keep you on Cc.