> I think it important to get rid of partition table reading in the kernel.
Why?
Let me be more precise.
I think it important to get rid of automatic partition table reading
in the kernel.
Why?
Because in some cases it is undesirable.
Because in some cases it crashes the kernel.
Because it involves guessing and heuristics.
Because policy belongs in user space.
> One argument is that our traditional DOS-type partition table will
> soon be at the end of its useful life. Yes, maybe it survives
> a few more years but our own stability requires slow changes,
> so we must start thinking a long time in advance.
That's a bad argument. It's not as if we want to have random formats for
this thing. Partitioning is damn important, and it has to be portable
across different machines and different operating systems. That all means
that there is absolutely _zero_ incentive to make up a partition format of
our own, since there are perfectly fine and existing formats.
That is a separate discussion best left for some other time.
[But every OS has its own partition table type, and the types
are not compatible. We started using the DOS-type partition table.
But it is dying. Windows replaces it with their dynamic disks.
What do we do? Follow Microsoft? Pick the Plan9 format?]
> Another argument is that it sometimes takes a *long* time, like several
> minutes, especially when this reading triggers hardware bugs.
This is only an argument for doing it on demand, not for dropping it.
Yes - that is my main point: doing it on demand. On demand only.
> Another argument is that nobody knows whether there is
> a partition table. (ZIP: "large floppy" vs "removable disk")
> Another argument is that tricky things happen with disk managers.
And none of these work any better in user space.
Well, in fact they do.
The user knows whether she treats her ZIP like a removable disk
or like a big floppy, that is, whether she should ask or refrain
from asking to read the pt.
And yes, if the partitions on the disk are to be shifted by 63 sectors
then partx can notice that and tell the kernel. But if the kernel does
these things automatically it can be difficult to remove Disk Manager.
You seem to think that kernel space somehow cannot do something that
user space can. I just don't see the overriding problems you claim.
It is the user who knows and wants to decide.
If my disk has media errors and I want to rescue what still can be read,
then I am very unhappy that the kernel starts reading the first sector
and the last sector and various sectors in the middle.
I want to have very precise control over what I/O happens.
If I insert a SmartMedia card then I know very precisely that it has
a FAT filesystem, a special one. Some cameras will refuse to read
such cards formatted by DOS. If the kernel starts probing, as it does
today, then it will read the first sector and the last sector, etc.
But my reader has a firmware bug, an off-by-one mistake in the reported
capacity, and the kernel tries to read a sector past the end of the card,
gets an error and the SCSI code starts retrying, resetting the device,
the host, the bus, finally takes the device offline. In the meantime
the USB code is entirely confused by aborts and crashes the kernel.
Of course both SCSI and USB code have to be improved, but it would
certainly be nice if I could tell from userspace: probe only for FAT.
No need at all to read this last sector.
I have seen partition tables with a loop. They would poison Linux
so that it was impossible to boot Linux on a system with such a disk.
I have seen disks with random test data causing Linux to go out and
read nonexistent sectors. There is the real possibility that no
partition table is present, and trying to find one may be a bad idea.
I have seen disks that form part of a multi-disk array.
Often the partition tables are meaningless.
Not doing things automatically gives power to the user.
In some situations this power is needed.
And once this partition reading is done on demand only, it does not
matter very much who does the reading. It may be the kernel.
It may be a user space program.
Andries
Again, Why?
You can always use the flat device as-is.
Because in some cases it crashes the kernel.
But moving it to user space would cause the kernel to crash anyway. Bugs
are bugs.
Because it involves guessing and heuristics.
The same guesses and heuristics would have to be in user space.
Because policy belongs in user space.
It's not policy. It's a fact of life that disks need to be split up into
parts, and the partitioning schemes are well-defined and shared across
multiple operating systems.
Yes - that is my main point: doing it on demand. On demand only.
But I actually _agree_ with this.
However, that has nothing to do with whether it is in user space or kernel
space. In many ways it is _easier_ to do on demand in kernel space: when
somebody opens /dev/sda1 and it isn't partitioned yet, you know it needs
to be.
The fact that partitioning right now is to some degree handled by device
drivers is a problem, but that's not a user space vs kernel space issue.
It's slowly getting moved to higher levels.
Linus
However, that has nothing to do with whether it is in user space or kernel
space. In many ways it is _easier_ to do on demand in kernel space: when
somebody opens /dev/sda1 and it isn't partitioned yet, you know it needs
to be.
Note that this actually allows you to do your own user-space partitioning
if you want to - simply by making sure that you do your partitioning
_before_ somebody tries to open a partition on the device.
And if you look at how fs/block_dev.c looks right now, you'll notice that
we already handle the "main device" vs "sub-partition" cases differently,
so it should be fairly straightforward to eventually do the partitioning
on demand.
We're not there yet, no. But doing it in the open() path of
fs/block_dev.c sure looks like it's the easiest way to maintain sanity wrt
partitioning, _and_ maintain 100% backwards compatibility.
[ Well, the "100% backwards compatibility" is not strictly true. Doing
partition handling on demand will mean that things like /proc/partitions
will obviously also end up being populated on demand, which may break
various sysadmin tools. But at least then it's fairly well localized,
and it's reasonably easy to grep for /proc/partitions in tools to see if
they may care ]
Linus
On Mon, Sep 02, 2002 at 08:55:47PM -0700, Linus Torvalds wrote:
Discussion so far:
(1) It is wrong when the kernel guesses, because it may guess wrong.
Userspace must tell the kernel what to do.
[The mount call is not "mount dev dir" but "mount -t type dev dir".
The kernel could guess, and often guess right, but some types are
close, like ext2 and ext3, or various ufs types, and some types may be
indistinguishable from the disk image, like msdos and vfat, where the
right type may depend on the intentions of the user.]
[In a similar way it is bad when the kernel unprovoked starts trying
to interpret the first few and last few sectors of the disk as an
Acorn, Amiga, Atari, BSD, DOS, EFI, IBM, Mac, Minix, LDM, OSF, SGI,
Sun, Ultrix partition table. Maybe there was no table. Maybe there
is a table of a kind the kernel did not know about, e.g. an AIX or
Plan 9 table, or a newer version of *BSD or Minix while the kernel
only knows about older versions, or ...]
(2) In all kinds of special situations attempts to read a partition
table lead to errors, even to kernel crashes. The kernel should not
unprovoked start doing I/O, guessing where the partition table might be,
and what type it might have.
quoted
Yes - that is my main point: doing it on demand. On demand only.
But I actually _agree_ with this.
However, that has nothing to do with whether it is in user space or kernel
space. In many ways it is _easier_ to do on demand in kernel space: when
somebody opens /dev/sda1 and it isn't partitioned yet, you know it needs
to be.
At first I misread this sentence ("partitioning" for me is something
done with fdisk) but now I take it to mean: If we have /dev/sda
but have not read its partition table, and somebody opens /dev/sda1,
then we decide that we must read a partition table.
If that is what you mean, I disagree.
(Compare: we have /mnt/cdrom and someone opens /mnt/cdrom/foo, should we
decide to automatically mount /dev/cdrom? An automounter in user space
may do such things. The kernel may not.)
Note that this actually allows you to do your own user-space partitioning
if you want to - simply by making sure that you do your partitioning
_before_ somebody tries to open a partition on the device.
You are inventing a can of worms. Suppose user space already told the
kernel where the partitions are, and the kernel knows about sda1, sda2, sda3.
Now somebody refers to sda4. Does the kernel start reading the device,
possibly changing the meaning of sda1 etc?
What if this disk is part of a RAID?
No, we must slowly migrate to the state where the kernel never takes the
initiative to search for a partition table. That initiative belongs to
user space.
Andries
I'm a newbie to the list (not unix, though :-), but anyhow...
Andries.Brouwer@cwi.nl wrote:
> Another argument is that nobody knows whether there is
> a partition table. (ZIP: "large floppy" vs "removable disk")
> Another argument is that tricky things happen with disk managers.
And none of these work any better in user space.
Well, in fact they do.
The user knows whether she treats her ZIP like a removable disk
or like a big floppy, that is, whether she should ask or refrain
from asking to read the pt.
"The User Knows"? My experience with the vast bulk of users is that
the only thing you can count on is that they _don't_ know. Much of
anything at all, in fact. Depending on an luser to know how her zip disk
is configured is _much_ less reliable than some minor kernel
heuristic (translate: guesswork).