Thomas Gleixner wrote:

On Thu, 27 Aug 2009, Christoph Lameter wrote:

quoted

On Thu, 27 Aug 2009, Chris Friesen wrote:

quoted

I just went and read the docs.  One of the things I noticed is that it
says that the offlined cpu cannot run userspace tasks.  For our
situation that's a showstopper, unfortunately.

It needs to be implemented the right way. Essentially this is a variation
on the isolcpu kernel boot option. We probably need some syscall to move
a user space process to a bare metal cpu since the cpu cannot be
considered online in the regular sense.

It can. It needs to be flagged as reserved for special tasks and you
need a separate mechanism to move and pin a task to such a CPU.

quoted

An isolated cpu can then only execute one process at a time. A process
would do all initialization and lock itsresources in memory before going
to the isolated processor. Any attempt to use OS facilities need to cause
the process to be moved back to a cpu with OS services.

You are creating a "one special case" operation mode which is not
justified in my opinion. Let's look at the problem you want to solve:

  Run exactly one thread on a dedicated CPU w/o any disturbance by the
  scheduler tick.

You can move away anything else than the scheduler tick from a CPU
today already w/o a single line of code change.

But you want to impose restrictions like resource locking and moving
back to another CPU in case of a syscall. What's the purpose of this ?
It does not buy anything except additional complexity.

That's just the wrong approach. All you need is a way to tell the
kernel that CPUx can switch off the scheduler tick when only one
thread is running and that very thread is running in user space. Once
another thread arrives on that CPU or the single thread enters the
kernel for a blocking syscall the scheduler tick has to be
restarted.

It's not rocket science to fix the well known issues of stopping and
eventually restarting the scheduler tick, the CPU time accounting and
some other small details. Such a modification would be of general use
contrary to your proposed solution which is just a hack to solve your
particular special case of operation.

I wonder if it makes sense to do something along the lines of the
sched-class...

IOW: What if we adopted one of the following models:

1) Create a new class that is higher prio than FIFO/RR and, when
selected, disables the tick.

2) Modify FIFO so that it disables tick by default...update accounting
info at next reschedule event.

3) Variation of 2..leave FIFO+tick as is by default, but have some kind
of parameter to optionally disable tick if desired.

In a way, we should probably consider (2) independent of this particular
thread.  FIFO doesn't need a tick anyway afaict...only a RESCHED+IPI
truly ever matter here....or am I missing something obvious (probably
w.r.t accounting)?

You could then couple this solution with cpusets (possibly with a little
work to get rid of any pesky per-cpy kthreads) to achieve the desired
effect of interference-free operation.  You wouldn't even have to have
funky rules eluded to above w.r.t. making sure only one userspace thread
is running on the core.

Thoughts?
-Greg