One comment about direct I/O: Applications performing direct I/O usually know
what they're doing...And if they don't, well, it's their problem: that's what
direct I/O is all about. Therefore, perhaps the correct approach would be to
allow these apps to manipulate the memory as they see fit (with the standard
memory manipulation routines - mmap, mlock, whatever).  I don't see a logical
reason for the direct I/O stuff to be intricately tied in with memory management
(i.e. limiting atomic I/O size, etc.). Direct I/O should receive a nicely aligned
physical buffer (or a sg list of such buffers) which has already been locked in
memory by the application, and perform the direct I/O. That's it - very simple.
Note that in this case, the memory in question could be a user buffer, or it may
even be a mapping to PCI memory sitting on another device, thus allowing adapter
to adapter data transfers. Note that in NT, when performing unbuffered I/O, it is
up to the user to lock the memory in question to sector-sized boundaries. This is
the correct way to go...

Mark

Gerard Roudier wrote:

On Fri, 19 May 2000, Brian Pomerantz wrote:

quoted

On Fri, May 19, 2000 at 08:11:10PM +0200, Gerard Roudier wrote:

quoted

Low-level drivers have limits on number of scatter entries. They can do
large transfers if scatter entries point to large data area. Rather than
hacking low-level drivers that are very critical piece of code that
require specific knowledge and documentation about the hardware, I
recommend you to hack the peripheral driver used for the Ciprico and let
it use large contiguous buffers (If obviously you want to spend your time
for this device that should go to compost, IMO).

Wanting to provide best support for shitty designed hardware does not
encourage hardware vendors to provide us with well designed hardware. In
others words, the more we want to support crap, the more we will have to
support crap.

I really don't want to get into a pissing match but it is obvious to
me that you haven't had any experience with high performance external
RAID solutions.  You will see this sort of performance characteristic
with a lot of these devices.  They are used to put together very large
storage systems (we are looking at building petabtye systems within
two years).

There is no way I'm going to hack on the proprietary RAID controller
in this system and even if I wanted to or could, I'm quite certain
there is a reason for needing the large transaction size.  When you
take into account the maximum transaction unit on each drive (usually
64KB), the fact that there are 8 data drives, then you have parity
calculation, latency of the transfer, and various points at which data
is cached and queued up before there is a complete transaction, then
you come up with the magic number.  These things were designed for
large streaming data and they do it VERY well.

If you have a hardware RAID 3 or RAID 5 solution that will give me
this kind of performance for the price point and size that these
Ciprico units have, then I would LOVE to hear it because I'm in the
market for buying several of them.  If you can find me a way of
getting >= 150MB/s streaming I/O on a single I/O server for my cluster
and fill an order for 2 I/O servers for under $100K, then I may
consider something other than the Ciprico 7000.  They deliver this
performance for a very attractive price.  And in a year, I'll come
back and buy a fifty more.

The SCSI BUS is transaction based and shared. I donnot care of affordable
stuff that moves needless burden to other parts. If they need specific
support for their hardware they must pay for that, and in this situation
these products would probably not really be so affordable.

Note that the same pathology happens to PCI technology. Some PCI devices,
notably Video boards, IDE controllers, brigdes, Network boards, have
abused a LOT of this BUS too. These hardware are/were also probably for an
interesting price but made shit in that place too.

The 36 bit adress extension from Intel is of the same idiomania that
costed a lot for a pathetic result, it seems. It adds complexity to VM
handling on Intel 32 bit systems when 64 bit have been proven to be
feasible 10 years ago and works fine.

I am only interested in technical issues but, in my opinion, it is the
ones that induce costs that must pay for these costs and not others. A
device that requires special handling because it abuses of technologies it
uses should be discarded unless vendor want to pay for the additionnal
effort that such offending stuff requires. But, if such vendors have
interested customers that are ready to pay for the effort or to spend time
implementing specific stuff for these products, I donnot see any problems.

A low-latency BUS does not require huge transactions in order to allow to
use efficiently its bandwitch. If a device requires so, then it can only
be badly designed. Given you description of the Capricio, it seems that it
is based on some kind of stupid batch mode that looks extremally poor
design to me.

Gerard.

-
To unsubscribe from this list: send the line "unsubscribe linux-scsi" in
the body of a message to majordomo@vger.rutgers.edu

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux.eu.org/Linux-MM/