On Sat, 2008-01-26 at 09:37 +0530, Srivatsa Vaddagiri wrote:

Ben,
        I presume you had CONFIG_FAIR_USER_SCHED turned on too?

Yes. It seems to be automatically turned on whenever FAIR_GROUP is
turned on. Considering how bad the behaviour is for a standard desktop
configuration, I'd be tempted to say to change it to default n.

Also were the
dd process and the niced processes running under different user ids? If
so, that is expected behavior, that we divide CPU equally among
users first and then among the processes within each user.

They were different users and that behaviour seems to be a very stupid
default behaviour for a desktop machine. Take this situation:

 - X running as root
 - User apps running as "user"
 - Background crap (indexing daemons etc...) running as their own user
or nobody

Unless you can get some kind of grouping based on user sessions
including suid binaries, X etc... I think this shouldn't default y in
Kconfig.

Not that it seems that Michel reported far worse behaviour than what I
saw, including pretty hickup'ish X behaviour even without the fair group
scheduler compared to 2.6.23. It might be because he's running X niced
to -1 (I leave X at 0 and let the scheduler deal with it in general)
though.

When CONFIG_FAIR_GROUP_SCHED (and CONFIG_FAIR_USER_SCHED) is not
enabled, X will be given higher priority for running on CPU when compared to 
other niced tasks. When the above options are turned on, X (running
under root uid) would be given lesser priority to run when compared to other
niced tasks running user different uids. Hence I expect some drop in
interactivity-experience with FAIR_GROUP_SCHED on.

Can you pls let me know if any of these makes a difference:

1. Run niced tasks as root. This would bring X and niced tasks in the
same "scheduler group" domain, which would give X much more CPU power
when compared to niced tasks.

I'll dbl check. My tests where indeed done with different users.

2. Keep the niced tasks running under a non-root uid, but increase root users 
   cpu share.
        # echo 8192 > /sys/kernel/uids/0/cpu_share

   This should bump up root user's priority for running on CPU and also 
   give a better desktop experience.

Allright, that's something that might need to be set by default by the
kernel ... as it will take some time to have knowledge of those knobs to
percolate to distros. Too bad you can't do the opposite by default for
"nobody" as there's no standard uid for it.

The group scheduler's SMP-load balance in 2.6.24 is not the best it
could be. sched-devel has a better load balancer, which I am presuming
will go into 2.6.25 soon.

In this case, I suspect that's not the issue.  If X and the niced processes are 
running under different uids, this (niced processes getting more cpu power) is 
on expected lines. Will wait for Ben to confirm this. 

I would suggest turning the fair group scheduler to default n in stable
for now.

Cheers,
Ben.