Arjan van de Ven wrote:

On Mon, 3 May 2010 12:34:26 +0200
Andi Kleen [off-list ref] wrote:

quoted

quoted

quoted

Maybe its low cost, (apparently, it is, since I can reach ~900.000
ipis on my 16 cores machine) but multiply this by 16 or 32 or 64
cpus, and clockevents_notify() cost appears to be a killer, all
cpus compete on a single lock.

Maybe this notifier could use RCU ?

could this be an artifact of the local apic stopping in deeper C
states? (which is finally fixed in the Westmere generation)

Yes it is I think.

But I suspect Eric wants a solution for Nehalem.

sure ;-)


so the hard problem is that on going idle, the local timers need to be
funneled to the external HPET. Afaik right now we use one channel of
the hpet, with the result that we have one global lock for this.

Does the HPET only need to be programmed when going idle?
That could mean that this isn't a big performance issue.
cares if you spin for a while when you're about to sleep for
at least 60usec?

HPETs have more than one channel (2 or 3 historically, newer chipsets
iirc have a few more), so in principle we can split this lock at least
a little bit... if we can get to one hpet channel per level 3 cache
domain we'd already make huge progress in terms of cost of the
contention....

Another possible approach: if a core needs the HPET and finds it
locked, it could queue up its request to a backlog which the
locking core will service.