On Sun, Oct 16, 2016 at 04:45:32AM +0300, Michael S. Tsirkin wrote:

On Fri, Oct 14, 2016 at 02:20:50AM -0700, Paul E. McKenney wrote:

quoted

On Thu, Oct 13, 2016 at 07:25:56PM +0300, Michael S. Tsirkin wrote:

quoted

On Wed, Oct 12, 2016 at 01:32:23PM -0700, Paul E. McKenney wrote:

quoted

On Wed, Oct 12, 2016 at 12:15:53PM -0500, Julia Cartwright wrote:

quoted

On Wed, Oct 12, 2016 at 12:49:56PM -0400, Luiz Capitulino wrote:

quoted

On Wed, 12 Oct 2016 11:21:14 -0500
Julia Cartwright [off-list ref] wrote:

quoted

On Wed, Oct 12, 2016 at 11:12:51AM -0400, Luiz Capitulino wrote:

quoted

Hi,

We have the following patch applied to the RT tree:

  commit a9d3cc781a3306bfa332fa7cb6134b70696058d5
  Author: Josh Cartwright [off-list ref]
  Date:   Tue Oct 27 07:31:53 2015 -0500
  
      net: Make synchronize_rcu_expedited() conditional on !RT_FULL

However, as noted by Michael, making rcu_normal=1 default in the
RT kernel should have the same effect (ie.  not calling
synchronize_sched_expedited()).

So, honest question, is there a reason not to:

 1. Drop the patch above
 2. Make rcu_normal=1 default?  

I think this is probably a cleaner way to fix the problems which
motivated this patch in the first place.  In my defense, the above patch
predates rcu_normal :).

No need to defend yourself! We debugged this very spike in one of
our kernels that don't have rcu_normal. We decided to do exactly
what you're doing before looking at upstream. Your patch helped
us confirm that we were in the right track.

Great!  Glad I could help in some way!

quoted

quoted

Something like this, perhaps?

Looks good, but (honest question) what does it buy us using
rcu_normal_after_boot vs rcu_normal? Is the boot process
improved someway?

That's the idea, although I don't have data to show much it actually
buys us.

It means that grace periods can be expedited during boot.  If you really
care about boot speed, you can also set rcu_expedited=1 and also
rcu_normal_after_boot=1, which will expedite all grace periods during
the boot process, but stop doing so just before spawning init.
After that point, any attempt to do an expedited grace period gets you
a normal grace period instead.

So you get fast boot and then clean realtime.

quoted

quoted

As long as we're rcu_normal=1 before launching user-space,
this should be fine.

Agreed.

Yes, you can also set them manually instead of at boot, if you wish.

							Thanx, Paul

FWIW

Acked-by: Michael S. Tsirkin <mst@redhat.com>

But I have a question - here's the commit that started
it all:


commit be3fc413da9eb17cce0991f214ab019d16c88c41
Author: Eric Dumazet [off-list ref]
Date:   Mon May 23 23:07:32 2011 +0000

    net: use synchronize_rcu_expedited()
    
    synchronize_rcu() is very slow in various situations (HZ=100,
    CONFIG_NO_HZ=y, CONFIG_PREEMPT=n)
    
    Extract from my (mostly idle) 8 core machine :
    
     synchronize_rcu() in 99985 us
     synchronize_rcu() in 79982 us
     synchronize_rcu() in 87612 us
     synchronize_rcu() in 79827 us
     synchronize_rcu() in 109860 us
     synchronize_rcu() in 98039 us
     synchronize_rcu() in 89841 us
     synchronize_rcu() in 79842 us
     synchronize_rcu() in 80151 us
     synchronize_rcu() in 119833 us
     synchronize_rcu() in 99858 us
     synchronize_rcu() in 73999 us
     synchronize_rcu() in 79855 us
     synchronize_rcu() in 79853 us
    
    When we hold RTNL mutex, we would like to spend some cpu cycles but not
    block too long other processes waiting for this mutex.
    
    We also want to setup/dismantle network features as fast as possible at
    boot/shutdown time.


To make sure this does not regress for RT,
how about clearing this flag on shutdown as well?

By that, you mean having some way to force all grace periods to be
expedited during shutdown?  Or am I missing your point?

							Thanx, Paul

Exactly. And maybe kexec.

If the relevant maintainers are OK with that, I am OK with it as long
as it is non-default (at least to begin with) and does not introduce
additional Kconfig questions.  My guess is that a boot parameter would
work best, but something to discuss.

							Thanx, Paul