Re: [RFC] NETDEV_UNREGISTER_BATCH seems unused nowaday ?

On Fri, Aug 10, 2012 at 07:45:48AM -0700, Eric W. Biederman wrote:

Eric Dumazet [off-list ref] writes:

quoted

On Fri, 2012-08-10 at 03:42 -0700, David Miller wrote:

quoted

From: Eric Dumazet <redacted>
Date: Fri, 10 Aug 2012 11:27:04 +0200

quoted

NETDEV_UNREGISTER_BATCH seems unused we can probably remove it.

Indeed, the routing cache was the final real user.

quoted

I am tracking a device refcount issue, delaying net device dismantle by
1 second in netdev_wait_allrefs()

I guess we need to add a notifier called _after_ the final
synchronize_net() in rollback_registered_many()

It's essentially caused by DST_GC_INC, right?

No, we in fact need a rcu_barrier(), then another call to
dst_dev_event().

rcu_barrier() is needed so that in-flight call_rcu() of routes (from
rt_free()) are completed. Or else we miss these dst in the
dst_dev_event().

quoted

I have a working patch, adding the rcu_barrier() and one additional
NETDEV_UNREGISTER_FINAL event.

Can someone help bring me up to speed.  What has changed in the
dst ref counting that has invalidated our previous solutions?

As for the idea of putting an rcu_barrier inside of the rtnl_lock.  I
really don't like it. You are trading off a 1000ms singled threaded wait
without locks for extending the hold times of rtnl lock by 12ms or so.

We already have an rcu_barrier on that path in netdev_run_todo,
so we can reorganize things to use that barrier I would be much
happer.  Furthermore I talked to Paul McKenney a while ago
about creating an rcu_barrier expedited and he really did not
like the idea.

For whatever it is worth, I do have rcu_barrier_expedited() on my list
of things that at least one person has expressed interest in, but that
I do not yet have a good solution for.  Obstacles include the following:

1.	If a given CPU has lots of callbacks, but is running a real-time
	process, what do you do?  (a) Hammer the real-time process?
	(b) Make rcu_barrier_expedited() wait (current likely choice)?
	(c) Handle via a set priority for callback processing (which
	might be the case for BOOST_RCU builds)?  (d) Force migration
	of the callbacks (mmmaybe...)?  (e) Force migration of the
	real-time process (ouch!)?

2.	Ditto, but huge numbers of callbacks and non-realtime processes.
	Similar solution space.

3.	There will be some real-time disruption from any reasonable
	implementation of rcu_barrier_expedited().  Maybe the RT
	guys choose to map it to rcu_barrier()?

On the other hand, in the same email thread back in May you were also
looking for a kmem_cache_free_rcu().  At the time I didn't have a
good solution for this, but I do believe that I have one now.  ;-)

							Thanx, Paul

Reading through the code we really should get dst_rcu_free
out of the header and make it non-line.  dst_rcu_free can't
possibly be called from a location where it can be inlined.

Trying to understand your analysis I have stared at the code for
a while and I am definitely not seeing any rcu callbacks that
result in calling rt_free.  So one of us is missing something.

All I am seeing from your trace is one call of rtnetlink_dev_notifier
the refcount is at 7 and the next call of rtnetlink_dev_notifier the
refcnt has dropped to 1.

Eric


--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html