On Mon, Feb 27, 2012 at 03:00:12PM -0700, Andreas Dilger wrote:

On 2012-02-27, at 10:44 AM, Ted Ts'o wrote:

quoted

On Mon, Feb 27, 2012 at 09:37:32AM -0600, Eric Sandeen wrote:

quoted

Essentially this would move allocation decisions to userspace, and I don't
think that sounds like a good idea.  If nothing else, the application shouldn't
assume that it "knows" anything at all about which regions of a filesystem may
be faster or slower...

What I *can* imagine is passing hints to the file system:

	* This file will be accessed a lot --- vs --- this file will
	  be written once and then will be mostly cold storage

	* This file won't be extended once originally written --- vs
         --- this file will be extended often (i.e., it is a log file
         or a unix mail directory file)

	* This file is mostly emphemeral --- vs --- this file will be
         sticking around for a long time.

	* This file will be read mostly sequentially --- vs --- this
         file will be read mostly via random access.

I definitely think that this is Zheng's real goal - to be able to give
application-level hints to the underlying filesystem.  While Lukas and
Eric may disagree with the _mechanism_ that Zheng proposed, I definitely
think the _goal_ is useful.

Often when working at the filesystem level the kernel has to try and
guess the intent of the application instead of being told what the
application actually wants.  A prime example is delalloc vs. fallocate(),
where the kernel is guessing (via delalloc) that the application may be
writing more data to the filesystem so it should delay flushing that
data to disk in the hope of making a better decision, while fallocate()
allows the application to specify exactly what file data will be written
and the kernel can make a good allocation decision immediately.

quoted

Obviously, these can be combined in various interesting ways; consider
for example an application journal file which is rarely read (except
in recovery circumstances, after a system crash, where speed might not
be the most important thing), and so even though the file is being
appended to regularly, contiguous block allocations might not matter
that much --- especially if the file is also being regularly fsync'ed,
so it would be more important if the blocks are located close to the
inode table.  This isn't a hypothetical situation, by the way; I once
saw a performance regression of ext4 vs. ext2 that was traced down to
the fact that ext2 would greedily allocate the block closest to the
inode table, whereas ext4 would optimize for reading the file later,
and so allocating a large contiguous block far, far away from the
inode table was what ext4 choose to do.  However, in this particular
case, optimizing for the frequent small write/fsync case would have
been a better choice.


In some cases the file system can infer some of these characteristics
(e.g. if the file was opened O_APPEND, it's probably a file that will
be extended often).

In other cases it makes sense for this sort of thing to be declared
via an fcntl or fadvise when the file is first opened.  Indeed we have
some of this already via fadvise's FADV_RANDOM vs. FADV_SEQUENTIAL,
although currently the expectation of this interface is that it's
mostly used for applications declare how they plan to read a
particular file from the perspective of enabling or disabling
readahead, and not from the perspective of influencing how the file
system should handle its allocation policy.

Yes, using FADV_* for files during write is exactly the kind of hint
that the kernel could use.  I expect that the current FADV_* flags are
not rich enough, but at least could form a starting point for this.

Hi Andreas,

I agree with you and Ted. Maybe we can provide more flags in fadvise(2)
to let the user to help the kernel to make a better decision.

I notice this RFC[1] in linux-kernel mailing list. This is an acceptable
solution for us.  Some flags can be added into fadvise(2).

e.g.
FADV_READ_HOT
FADV_READ_SEQ
FADV_READ_RANDOM
FADV_WRITE_ONCE
FADV_WRITE_APPEND
FADV_WRITE_FIX_FILELEN
...

Then file system can pick a subset of these flags to implement.

1. https://lkml.org/lkml/2012/2/9/473

Regards,
Zheng

quoted

I definitely agree that we don't want to go down the path of having
applications try to directly decide where block should be placed on
the disk.  That way lies madness.  However, having some way of
specifying the behaviour of how the file is going to be used can be
very useful indeed.

quoted

There are still some interesting policy/security questions, though.
Do you trust any application or any user id to be able to declare that
"this file is going to be used a lot"?  After, all if everyone
declares that their file is accessed a lot, and thus deserving of
being in the beginning third of the HDD (which can be significantly
faster than the rest of the disk), then the whole scheme falls apart.

In some sense, in the rare case where all applications are ill behaved
then it is no worse than not having any interface in the first place.
In general, however, I don't expect applications to abuse this any more
than they abuse fallocate() to reserve huge amounts of space that they
don't need to use.

quoted

Do we simply not care?  Do we reserve the ability to set certain file
usage declarations only to root, or via some cgroup?  The answers are
not obvious....  For some parameters it probably won't matter if we
let unprivileged users declare whether or not their file is mostly
accessed sequentially or random access.  But for others, it might
matter a lot if you have bad actors, or worse, bad application writers
who assume that their web browser or GUI file system navigator, or
chat program should have the very best and highest priority blocks for
their sqlite files.

Sure, and the users can stop using badly-written applications, but that
is no reason to deny the ability for well written applications from
helping the kernel make better decisions.

Cheers, Andreas