Stan Hoeppner <stan <at> hardwarefreak.com> writes:

~$ cat /sys/block/sda/queue/scheduler
[CFQ] noop deadline
Wes, yours will show CFQ probably as the default on RHEL/CentOS.  You'll
want deadline for best seek and all around performance.  So:
~$ echo deadline > /sys/block/sda/queue/scheduler
Add that to an init script or cron entry so it sets on every boot.
Barriers are not an issue with this test.

Thank you all. The issue is now closed.
RHEL5 was not doing cache flush right. It was only corrected in 2.6.32+
After removing O_SYNC from seekmark the results are now comparable.
Actually it is hard to find a linux raw device random R/W benchmark tool and
seekmark being the most popular fails when comparing pre and post 2.6.32
systems (unless you remove O_SYNC).