From: Zheng Liu <hidden> Date: 2012-03-05 12:50:29
Hi list,
Block allocation is a key component of file system. Every file systems try to
improve the performance with optimizing the block allocation of a file. But no
matter what file system does, it just guesses what the user expects. Thus, it
is not very accurate. fadvise(2) provides a method to let the user to give a
hint to file system. However, until now, only few flags are provided. So we
can provide more flags to tell file system how to allocate the blocks for a
file.
For example:
we can add these flags into fadvise(2):
FADV_ALLOC_READ_SEQ
FADV_ALLOC_READ_RANDOM
FADV_ALLOC_WRITE_ONCE
FADV_ALLOC_WRITE_APPEND
FADV_ALLOC_READ_* are not similar with FADV_SEQUENTIAL and FADV_RANDOM.
FADV_ALLOC_READ_SEQ tells file system that this file need to allocate some
sequential blocks, and FADV_ALLOC_READ_RADOM tells file system that this file
can endure the fragmentation.
FADV_ALLOC_WRITE_ONCE indicates that this file just is written once. So file
system can allocate some sequential blocks for it to improve the read
performance. FADV_ALLOC_WRITE_APPEND flag is set to point out that data will be
appended to the end of this file, and file system can reserve some blocks for it
to guarantee the sequence as much as possible.
File systems can support a subset of these flags according to its design. These
flags provide a rich interface that lets the user to control block allocation of
files. The user could precisely control the allocation of their files to
improve the performance of appliatons.
Any comments or suggestions are appreciated. Thank you.
Regards,
Zheng
Hi list,
Block allocation is a key component of file system. Every file systems try to
improve the performance with optimizing the block allocation of a file. But no
matter what file system does, it just guesses what the user expects. Thus, it
is not very accurate. fadvise(2) provides a method to let the user to give a
hint to file system. However, until now, only few flags are provided. So we
can provide more flags to tell file system how to allocate the blocks for a
file.
For example:
we can add these flags into fadvise(2):
FADV_ALLOC_READ_SEQ
FADV_ALLOC_READ_RANDOM
FADV_ALLOC_WRITE_ONCE
FADV_ALLOC_WRITE_APPEND
FADV_ALLOC_READ_* are not similar with FADV_SEQUENTIAL and FADV_RANDOM.
FADV_ALLOC_READ_SEQ tells file system that this file need to allocate some
sequential blocks, and FADV_ALLOC_READ_RADOM tells file system that this file
can endure the fragmentation.
File systems typically allocate the best layout they can for a file
at the time of write. Does _RANDOM mean do not do that. Find single
bits scattered around the disk. If so, why will people use it. I mean,
random IOs are slow. What you are proposing it is a further slowdown.
Hardly a feature that will be attractive to users.
FADV_ALLOC_WRITE_ONCE indicates that this file just is written once. So file
system can allocate some sequential blocks for it to improve the read
performance. FADV_ALLOC_WRITE_APPEND flag is set to point out that data will be
appended to the end of this file, and file system can reserve some blocks for it
to guarantee the sequence as much as possible.
Define ONCE. Is it one write(2)? I guess not. You probably mean
that once the file descriptor is closed, it will not be written
to. But we have no way of knowing how many writes there will be.
So it will be treated the same as APPEND. And file systems already
provide allocation reservation and/or delayed allocation to handle
APPEND write loads. So this flag does not offer much to the user
or the fs.
From: Zheng Liu <hidden> Date: 2012-03-06 02:35:05
On Mon, Mar 05, 2012 at 11:48:43AM -0800, Sunil Mushran wrote:
On 03/05/2012 04:50 AM, Zheng Liu wrote:
quoted
Hi list,
Block allocation is a key component of file system. Every file systems try to
improve the performance with optimizing the block allocation of a file. But no
matter what file system does, it just guesses what the user expects. Thus, it
is not very accurate. fadvise(2) provides a method to let the user to give a
hint to file system. However, until now, only few flags are provided. So we
can provide more flags to tell file system how to allocate the blocks for a
file.
For example:
we can add these flags into fadvise(2):
FADV_ALLOC_READ_SEQ
FADV_ALLOC_READ_RANDOM
FADV_ALLOC_WRITE_ONCE
FADV_ALLOC_WRITE_APPEND
FADV_ALLOC_READ_* are not similar with FADV_SEQUENTIAL and FADV_RANDOM.
FADV_ALLOC_READ_SEQ tells file system that this file need to allocate some
sequential blocks, and FADV_ALLOC_READ_RADOM tells file system that this file
can endure the fragmentation.
Hi Sunil,
Thank you for your feedback.
File systems typically allocate the best layout they can for a file
at the time of write. Does _RANDOM mean do not do that. Find single
bits scattered around the disk. If so, why will people use it. I
mean, random IOs are slow. What you are proposing it is a further
slowdown.
Hardly a feature that will be attractive to users.
No, _RANDOM means that file system doesn't need to try its best to find
a proper position to allocate some blocks for this file. Furthermore,
currently random IOs seem that they are not obviously slower than
sequential IOs in Flash/SSD device. For example, when users know a file
that is accessed infrequently, they can put this file in a corner, such
as in some discontinuously blocks. Then sequential blocks are reserved
for the file that needs to be accessed frequently and users can obtain
the better performance.
quoted
FADV_ALLOC_WRITE_ONCE indicates that this file just is written once. So file
system can allocate some sequential blocks for it to improve the read
performance. FADV_ALLOC_WRITE_APPEND flag is set to point out that data will be
appended to the end of this file, and file system can reserve some blocks for it
to guarantee the sequence as much as possible.
Define ONCE. Is it one write(2)? I guess not. You probably mean
that once the file descriptor is closed, it will not be written
to. But we have no way of knowing how many writes there will be.
So it will be treated the same as APPEND. And file systems already
provide allocation reservation and/or delayed allocation to handle
APPEND write loads. So this flag does not offer much to the user
or the fs.
Sorry, I don't express clearly. _ONCE means that the size of a file
doesn't be chagned after it has been created. Certainly, you are right.
We can use fallocate(2) to obtain the same result. ;-)
Regards,
Zheng
--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
No, _RANDOM means that file system doesn't need to try its best to find
a proper position to allocate some blocks for this file. Furthermore,
currently random IOs seem that they are not obviously slower than
sequential IOs in Flash/SSD device. For example, when users know a file
that is accessed infrequently, they can put this file in a corner, such
as in some discontinuously blocks. Then sequential blocks are reserved
for the file that needs to be accessed frequently and users can obtain
the better performance.
Then FADV_ALLOC_HOT_REGION and FADV_ALLOC_COLD_REGION are
probably better terms.
Hi list,
Block allocation is a key component of file system. Every file systems try to
improve the performance with optimizing the block allocation of a file. But no
matter what file system does, it just guesses what the user expects. Thus, it
is not very accurate. fadvise(2) provides a method to let the user to give a
hint to file system. However, until now, only few flags are provided. So we
can provide more flags to tell file system how to allocate the blocks for a
file.
For example:
we can add these flags into fadvise(2):
FADV_ALLOC_READ_SEQ
FADV_ALLOC_READ_RANDOM
FADV_ALLOC_WRITE_ONCE
FADV_ALLOC_WRITE_APPEND
FADV_ALLOC_READ_* are not similar with FADV_SEQUENTIAL and FADV_RANDOM.
FADV_ALLOC_READ_SEQ tells file system that this file need to allocate some
sequential blocks, and FADV_ALLOC_READ_RADOM tells file system that this file
can endure the fragmentation.
FADV_ALLOC_WRITE_ONCE indicates that this file just is written once. So file
system can allocate some sequential blocks for it to improve the read
performance. FADV_ALLOC_WRITE_APPEND flag is set to point out that data will be
appended to the end of this file, and file system can reserve some blocks for it
to guarantee the sequence as much as possible.
Hi Zheng,
those two flags does not make sense to me. The FADV_ALLOC_WRITE_ONCE is
actually the same as fallocate, and we certainly do not need more ways
to do fallocate, one is more than enough.
FADV_ALLOC_WRITE_APPEND seems weird. File systems already do some
preallocations for the files, so we do not fragment them as much. So
what might be more interesting is to be able to set how much space we
want to keep preallocated for the particular file, however strictly
speaking it is not something we would not achieve with fallocate, but it
would certainly be more convenient.
-Lukas
File systems can support a subset of these flags according to its design. These
flags provide a rich interface that lets the user to control block allocation of
files. The user could precisely control the allocation of their files to
improve the performance of appliatons.
Any comments or suggestions are appreciated. Thank you.
Regards,
Zheng
--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
From: Zheng Liu <hidden> Date: 2012-03-06 13:25:43
On Mon, Mar 05, 2012 at 08:26:03PM -0800, Sunil Mushran wrote:
On 3/5/2012 6:35 PM, Zheng Liu wrote:
quoted
No, _RANDOM means that file system doesn't need to try its best to find
a proper position to allocate some blocks for this file. Furthermore,
currently random IOs seem that they are not obviously slower than
sequential IOs in Flash/SSD device. For example, when users know a file
that is accessed infrequently, they can put this file in a corner, such
as in some discontinuously blocks. Then sequential blocks are reserved
for the file that needs to be accessed frequently and users can obtain
the better performance.
Then FADV_ALLOC_HOT_REGION and FADV_ALLOC_COLD_REGION are
probably better terms.
Make sense to me. Thanks a lot. ;-)
Regards,
Zheng
--
To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
From: Zheng Liu <hidden> Date: 2012-03-06 13:51:44
On Tue, Mar 06, 2012 at 09:27:16AM +0100, Lukas Czerner wrote:
On Mon, 5 Mar 2012, Zheng Liu wrote:
quoted
Hi list,
Block allocation is a key component of file system. Every file systems try to
improve the performance with optimizing the block allocation of a file. But no
matter what file system does, it just guesses what the user expects. Thus, it
is not very accurate. fadvise(2) provides a method to let the user to give a
hint to file system. However, until now, only few flags are provided. So we
can provide more flags to tell file system how to allocate the blocks for a
file.
For example:
we can add these flags into fadvise(2):
FADV_ALLOC_READ_SEQ
FADV_ALLOC_READ_RANDOM
FADV_ALLOC_WRITE_ONCE
FADV_ALLOC_WRITE_APPEND
FADV_ALLOC_READ_* are not similar with FADV_SEQUENTIAL and FADV_RANDOM.
FADV_ALLOC_READ_SEQ tells file system that this file need to allocate some
sequential blocks, and FADV_ALLOC_READ_RADOM tells file system that this file
can endure the fragmentation.
FADV_ALLOC_WRITE_ONCE indicates that this file just is written once. So file
system can allocate some sequential blocks for it to improve the read
performance. FADV_ALLOC_WRITE_APPEND flag is set to point out that data will be
appended to the end of this file, and file system can reserve some blocks for it
to guarantee the sequence as much as possible.
Hi Zheng,
those two flags does not make sense to me. The FADV_ALLOC_WRITE_ONCE is
actually the same as fallocate, and we certainly do not need more ways
to do fallocate, one is more than enough.
FADV_ALLOC_WRITE_APPEND seems weird. File systems already do some
preallocations for the files, so we do not fragment them as much. So
what might be more interesting is to be able to set how much space we
want to keep preallocated for the particular file, however strictly
speaking it is not something we would not achieve with fallocate, but it
would certainly be more convenient.
-Lukas
Hi Lukas,
I have realized that these two flags seem redundant, and we don't need
them.
As we discussed previously and Sunil's suggestions. The key issue is
that user provides a hint to file system, and file system can know
whether or not this file can be stored in a corner or be allocated in
non-sequential blocks. Then the sequential blocks are reserved for the
particular file that has a *_HOT* flag. Although fallocate(2) can
preallocate some blocks for a file, it cannot put a file at the
beginning of the disk to obtain a better performance. So maybe file
system can use these flags to optimize the layout of a file.
Regards,
Zheng
quoted
File systems can support a subset of these flags according to its design. These
flags provide a rich interface that lets the user to control block allocation of
files. The user could precisely control the allocation of their files to
improve the performance of appliatons.
Any comments or suggestions are appreciated. Thank you.
Regards,
Zheng
--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
On Tue, Mar 06, 2012 at 09:27:16AM +0100, Lukas Czerner wrote:
quoted
On Mon, 5 Mar 2012, Zheng Liu wrote:
quoted
Hi list,
Block allocation is a key component of file system. Every file systems try to
improve the performance with optimizing the block allocation of a file. But no
matter what file system does, it just guesses what the user expects. Thus, it
is not very accurate. fadvise(2) provides a method to let the user to give a
hint to file system. However, until now, only few flags are provided. So we
can provide more flags to tell file system how to allocate the blocks for a
file.
For example:
we can add these flags into fadvise(2):
FADV_ALLOC_READ_SEQ
FADV_ALLOC_READ_RANDOM
FADV_ALLOC_WRITE_ONCE
FADV_ALLOC_WRITE_APPEND
FADV_ALLOC_READ_* are not similar with FADV_SEQUENTIAL and FADV_RANDOM.
FADV_ALLOC_READ_SEQ tells file system that this file need to allocate some
sequential blocks, and FADV_ALLOC_READ_RADOM tells file system that this file
can endure the fragmentation.
FADV_ALLOC_WRITE_ONCE indicates that this file just is written once. So file
system can allocate some sequential blocks for it to improve the read
performance. FADV_ALLOC_WRITE_APPEND flag is set to point out that data will be
appended to the end of this file, and file system can reserve some blocks for it
to guarantee the sequence as much as possible.
Hi Zheng,
those two flags does not make sense to me. The FADV_ALLOC_WRITE_ONCE is
actually the same as fallocate, and we certainly do not need more ways
to do fallocate, one is more than enough.
FADV_ALLOC_WRITE_APPEND seems weird. File systems already do some
preallocations for the files, so we do not fragment them as much. So
what might be more interesting is to be able to set how much space we
want to keep preallocated for the particular file, however strictly
speaking it is not something we would not achieve with fallocate, but it
would certainly be more convenient.
-Lukas
Hi Lukas,
I have realized that these two flags seem redundant, and we don't need
them.
As we discussed previously and Sunil's suggestions. The key issue is
that user provides a hint to file system, and file system can know
whether or not this file can be stored in a corner or be allocated in
non-sequential blocks. Then the sequential blocks are reserved for the
particular file that has a *_HOT* flag. Although fallocate(2) can
preallocate some blocks for a file, it cannot put a file at the
beginning of the disk to obtain a better performance. So maybe file
system can use these flags to optimize the layout of a file.
However the file system do not have the information which part of the
device it resides on is faster. It might be the beginning of the file
system, but it might not be the case at all.
Moreover the flag which is stating that the file does not have to be
allocated sequentially is not particularly helpful, I can not imagine
people using it. Why would someone want to lower their performance ?
Well, they might think that it will increase performance of the other
files, but that is highly disputable and there are better solutions like
using faster storage for the files that actually needs it.
Additionally *_HOT* flag does not say anything about the allocation
policy. It might be accessed often ,but no in sequential manner, or it
can be written to a lot, it can be appended a lot, or it the content
might be changed without changing its size etc... *Hot* might mean so
many thing that this is just not useful for the file system. It would
certainly be better to come up with something less esoteric which would
actually address concrete user issues and help file system to deal with
them better, like, I do not know, do not fsync/force allocation on
rename maybe...(or whatever we are doing right now).
Thanks!
-Lukas
Regards,
Zheng
quoted
quoted
File systems can support a subset of these flags according to its design. These
flags provide a rich interface that lets the user to control block allocation of
files. The user could precisely control the allocation of their files to
improve the performance of appliatons.
Any comments or suggestions are appreciated. Thank you.
Regards,
Zheng
--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
However the file system do not have the information which part of the
device it resides on is faster. It might be the beginning of the file
system, but it might not be the case at all.
Think HSM and flash storage as the hot region. Remember these are
hints and not guaranteed to work in all cases.
Moreover the flag which is stating that the file does not have to be
allocated sequentially is not particularly helpful, I can not imagine
people using it. Why would someone want to lower their performance ?
Well, they might think that it will increase performance of the other
files, but that is highly disputable and there are better solutions like
using faster storage for the files that actually needs it.
Additionally *_HOT* flag does not say anything about the allocation
policy. It might be accessed often ,but no in sequential manner, or it
can be written to a lot, it can be appended a lot, or it the content
might be changed without changing its size etc... *Hot* might mean so
many thing that this is just not useful for the file system. It would
certainly be better to come up with something less esoteric which would
actually address concrete user issues and help file system to deal with
them better, like, I do not know, do not fsync/force allocation on
rename maybe...(or whatever we are doing right now).
_HOT/_COLD is descriptive for allocation policy though fadvise() is
the wrong call as it pertains to access patterns.
Sunil
From: Dave Chinner <david@fromorbit.com> Date: 2012-03-07 00:51:30
On Mon, Mar 05, 2012 at 08:50:29PM +0800, Zheng Liu wrote:
Hi list,
Block allocation is a key component of file system. Every file systems try to
improve the performance with optimizing the block allocation of a file. But no
matter what file system does, it just guesses what the user expects. Thus, it
is not very accurate. fadvise(2) provides a method to let the user to give a
hint to file system. However, until now, only few flags are provided. So we
can provide more flags to tell file system how to allocate the blocks for a
file.
For example:
we can add these flags into fadvise(2):
FADV_ALLOC_READ_SEQ
fallocate()
FADV_ALLOC_READ_RANDOM
Allocation can't be optimised as the read pattern cannot be defined.
FADV_ALLOC_WRITE_ONCE
fallocate()
FADV_ALLOC_WRITE_APPEND
chattr +a
Cheers,
Dave.
--
Dave Chinner
david@fromorbit.com
From: Andreas Dilger <hidden> Date: 2012-03-07 04:14:43
On 2012-03-07, at 8:51 AM, Dave Chinner wrote:
On Mon, Mar 05, 2012 at 08:50:29PM +0800, Zheng Liu wrote:
quoted
Block allocation is a key component of file system. Every file systems try to
improve the performance with optimizing the block allocation of a file. But no
matter what file system does, it just guesses what the user expects. Thus, it
is not very accurate. fadvise(2) provides a method to let the user to give a
hint to file system. However, until now, only few flags are provided. So we
can provide more flags to tell file system how to allocate the blocks for a
file.
For example:
we can add these flags into fadvise(2):
FADV_ALLOC_READ_SEQ
fallocate()
I think this is already the assumed default for any file IO, but is included for completeness (e.g. to be able to turn off READ_RANDOM).
quoted
FADV_ALLOC_READ_RANDOM
Allocation can't be optimised as the read pattern cannot be defined.
I think what this is intended for is to tell the filesystem "don't work very hard to find optimum allocation, it will have a random read pattern anyway".
quoted
FADV_ALLOC_WRITE_ONCE
fallocate()
quoted
FADV_ALLOC_WRITE_APPEND
chattr +a
and/or fallocate(KEEP_SIZE)
Having a consistent API definitely makes sense.
This proposal definitely needs to have some clear explanation of how the flags are intended to be used by applications, and why they will help filesystems to improve allocation. I'm not for adding gratuitous APIs, but at the same time I think that filesystems are often working in the dark and could benefit from more information being passed from the application.
Cheers, Andreas
However the file system do not have the information which part of the
device it resides on is faster. It might be the beginning of the file
system, but it might not be the case at all.
Think HSM and flash storage as the hot region. Remember these are
hints and not guaranteed to work in all cases.
Exactly, first we have to define what we actually need to achieve with
it. Not just randomly making up stupid pseudo-optimizations. Moreover
there is _no way_ file system has the information about the HSM nor the
flash regions, fast regions or whatever, it does not even know where is
the beginning of the disk. Stop constructing building from the roof!!
There just is not any interface for the file system to use to get such
information!
I also believe that regarding HSM user is in no damn position to decide
whether his file will be on flash or not. It just does not work that
way, every user's, or application's files has to be accessed faster than
others from their point of view.
quoted
Moreover the flag which is stating that the file does not have to be
allocated sequentially is not particularly helpful, I can not imagine
people using it. Why would someone want to lower their performance ?
Well, they might think that it will increase performance of the other
files, but that is highly disputable and there are better solutions like
using faster storage for the files that actually needs it.
Additionally *_HOT* flag does not say anything about the allocation
policy. It might be accessed often ,but no in sequential manner, or it
can be written to a lot, it can be appended a lot, or it the content
might be changed without changing its size etc... *Hot* might mean so
many thing that this is just not useful for the file system. It would
certainly be better to come up with something less esoteric which would
actually address concrete user issues and help file system to deal with
them better, like, I do not know, do not fsync/force allocation on
rename maybe...(or whatever we are doing right now).
_HOT/_COLD is descriptive for allocation policy though fadvise() is
the wrong call as it pertains to access patterns.
Of course _HOT/_COLD is totally stupid flags from both user and file
system POV. It could mean whatever you can imagine behind HOT/COLD. In
this case it is so damn esoteric I can not imagine even file systems
agree on the meaning of it. But when it comes to user it will be even
worse - total disaster - no one would be able to say what benefit should
it actually bring.
Just come up with concrete optimizations and give them concrete names.
If this is going to be of any use to file systems and users, both should
know exactly what workload would be applied to the file, or what user
actually intents to do with it, so that file system can take concrete
action. What you proposing is a flag which should spawns ponies all
around, it does not work
And if you can not come up with any flag like that, well then it certainly
tells you something about this feature as a whole.
-Lukas
On Wed, Mar 07, 2012 at 09:51:27AM +0100, Lukas Czerner wrote:
Exactly, first we have to define what we actually need to achieve with
it. Not just randomly making up stupid pseudo-optimizations. Moreover
there is _no way_ file system has the information about the HSM nor the
flash regions, fast regions or whatever, it does not even know where is
the beginning of the disk. Stop constructing building from the roof!!
There just is not any interface for the file system to use to get such
information!
I'm not really worried about this problem. This is something which
can easily be set by the system administrator via mkfs or tune2fs.
And just as we now have /sys/block/sda/queue/rotational so that upper
layers can make optimizations based on whether or not a disk is an
SSD, as we can prove that manual configuration of storage attributes
can make a measurable difference, it can be a spur to the standards
bodies to eventually (years and years and years later) come up with a
standardized way for the file system to get such interfaces
automatically.
In the meantime, we have flash vendors (or at least one flash vendor
who works with embedded/handset customers) interested in potentially
providing private interfaces to make additional storage attributes
available, or for file systems to provide information to the storage
devices so they can better optimize their behaivor.
So I'm not too worried about the fact that we don't have a way to
specify all of these things yet. If we can find a way to make things
faster, eventually the rest of the infrastructure can get plumbed in.
(Even standards bodies that move in geologic time scales. :-)
I also believe that regarding HSM user is in no damn position to decide
whether his file will be on flash or not. It just does not work that
way, every user's, or application's files has to be accessed faster than
others from their point of view.
Access control is going to be an interesting problem, and what the
requirements are for a file system running on an HPC system, or an
Android device, or a generic time-sharing system are quite different.
Given that many of us have grown up in an environment where there are
mutually suspicious (and untrustworthy) time sharing users, or equally
untrustworthy application writers who tend to optimize their
application without considering anything else, it's easy for us to
assume that if we can't solve the authorization problem completely,
that it's hopeless.
But the same argument can be made for real time scheduling priorities
(which is even easier for untrustworthy application authors to abuse),
but that's turned out to be extremely important in allowing Linux to
break through in various new fields --- including Naval Warships and
laser-wielding industrial robots. :-)
- Ted