Performance issue - VoIP application on Linux 2.4.22

Hello everybody,
I have a multithreaded VoIP application in user space and a kernel driver
module on MPC 8248 based board.
During media data transfer( Video Call) I have observed one strange behavior
that at specific interval, cpu utilization is reaching to 100 % and then
rolls back to 0 %.  This behavior repeats at regular intervals of time
during the call.
During this time, data rate and memory utilization is constant throughout
the Call. Also number of interrupts received and processed in the kernel
driver is constant.  We have checked the code many times, but couldn't find
any of the threads/driver hogging the processor time.
We are using Linux kernel 2.4.22  with kernel preemption enabled and low
latency patch added. Please let me know anybody have seen same behavior and
how to sort this out.
Thanks and Regards
Boby