Hi Pásztor,

Not sure if it would help you trace this down further but have you tried
running your load with the Latency Tracer configured in? The Latency
Tracer will generate its own over head latency but might track the
problem down further.


On Mon, 2007-06-25 at 09:55 +0200, Pásztor Szilárd wrote:

Hi,

  I'm using the RT patch for a project where I have to communicate over
ethernet and 8 serial ports (via a pci multi-interface card). I must gain CPU
time for a while in every millisecond which is my periodic cycle time.

Is the thread that needs to gain CPU time dependent on the accessing of
the serial and ethernet ports?

  Most things seem ok, but for some strange reason, 2-3 ms latencies show up
rarely about once out of 200000 cycle runs or so when the machine is under
load, mostly of interrupts.

Can you post the code to your program somewhere, and also show us
exactly where you see the latency?

  It took a while for me to track it down but finally found that this latency
is not related to a slower part of my cycle, but is caused indirectly by an
ioctl() call in the cycle with TIOCMGET read command on the serial ports that
translates to a driver-dependent get_mctrl call inside the kernel. If the call
is in, the delay shows up elsewhere every time, spread across the cycle (found
that with extensive timestamping) so I'm thinking of some interrupt kicking in
a while after the ioctl() call. Commenting the call out eliminates the problem
completely, but it unfortunately eliminates the functionality of my code as
well since things depend on the line states so I cannot spare the read of the
modem status register.

The only real thing that happens in get_mctrl that I see is a wake_up.
So if something is sleeping on an ioctl TIOCMWAIT, it will wake up and
run.  Do you have something that does this?  Your latency that you are
seeing, might just be another thread taking up the CPU.

I'm currently using 2.6.19-rt15 but this problem shows up with every kernel
and patch version I've tried (starting from 2.6.16 to 2.6.20) and I'm not
really able to test newer versions currently because the server is under heavy

Your personal server or are you talking about the server to download the
patch?

use. However, I suspect that this problem is not an easy sight to come by and
probably still persists. Of course my task is running with the highest
real-time priority. The lower level serial driver that is in use is
drivers/serial/8250.c.

As I said, check your application. I think there's another thread you
may have that is competing with the thread you are timing.

  I'm afraid I don't have a deep enough insight to track it down any further,
so I thought I'd try if this description rings a bell with more knowledgeable
persons.

Any help would be greatly appreciated.

-- Steve