On Thu, 9 Jun 2011, Amir G. wrote:

On Thu, Jun 9, 2011 at 11:13 AM,  [off-list ref] wrote:

quoted

On Thu, 9 Jun 2011, Amir G. wrote:

quoted

On Thu, Jun 9, 2011 at 9:50 AM, Lukas Czerner [off-list ref] wrote:

quoted

On Thu, 9 Jun 2011, Yongqiang Yang wrote:

quoted

On Thu, Jun 9, 2011 at 11:18 AM, Amir G.
[off-list ref] wrote:

quoted

On Thu, Jun 9, 2011 at 4:59 AM, Yongqiang Yang [off-list ref]
wrote:

You see, when it comes to the full fs snapshots I am not convinced that
it is *very* useful, yes it might have some users, but you can alway
take the safe way and do lvm snapshots (or better use the new multisnap)
for backup, without need to modify stable filesystem code.

You think like a developer. Try talking to some sys admins.
Especially ones that worked with Solaris/ZFS or NetApp.
See what they think about snapshots and about the LVM alternative...
Snapshots have addictive qualities. Ones you've used them, you can't
go back to not having them.
Imagine how people used to live before the 'Undo' button and imagine
that your employer forced you to use an editor without an Undo button.
This is the kind of feedback I got from sys admins that moved from Solaris
to Linux.

as a sysadmin, it's a _wonderful_ tool to have for any system that has
people editing/saving files on directly.

Thank you david. Finally some positive feedback from the people
for whom the feature is intended for :-)

No one is arguing about the advantages of snapshots. I think that we are
all clear on this. Snapshots are useful.

quoted

quoted

quoted

Also, I do not buy the whole argument of "not have to create separate
disk
space for snapshot". It is actually better for sysadmins, because you
have perfect control on what is going on, how much space is used for
your snapshots and how much is used by your data. You can always easily
extend the snapshot volume, or let it die silently when it is too old
and too big.

Seriously, Lukas, talk to sys admins.
Letting the snapshot die silently is the worst possible thing that a
snapshots
implementation can do (for long lived snapshots).

that depends on the site policy.

sometimes it is better to loose snapshots than to run out of disk space and
halt the system, sometimes you would rather halt the system.

the policy of what happens when you run out of space should not be a kernel
decision, the desired behavior varies far too much.

this includes being able to say things like "I want to always have 10% of my
disk allocated to snapshots, but if there's more free space, go ahead and
use it, but always keep at least 10% of the disk free so that you don't have
to halt new writes while you clear space"

or

"if you run out of space, try and keep the oldest snapshot and the newest
snapshot, delete other snapshots in between before touching either of these"

I fully agree.
AFAIK, there is no user space tool to manage snapshots to this level for Linux.
The only snapshot manager I know about is snapper:
http://en.opensuse.org/Portal:Snapper, which we are working on adding
ext4 snapshots support to.
Snapper does not have the free space based policy to the best of my knowledge,
but it could be improved to monitor free disk space.

A tool like that does not need any further kernel changes from
ext4 and btrfs to implement the policies suggested above.
However, with LVM snapshots, some of these policies (like use whatever space you
have free in the filesystem) are simply not possible.

And why is that ? With LVM you can shrink or extent volumes at will, I
do not think this is a problem at all, moreover, you can always add more
drives to resize your existing volumes to.

quoted

quoted

quoted

How does it actually work on ext4 snapshots ? When you're going to
rewrite a file, you will never know how much disk space it'll take in
advance, am I right ? Is the filesystem accounting for the snapshot size
as well ? or is it hidden ?

It's not hidden, it's accounted for as a regular file (usually owned by
root).
You need a bit of scripting to gather the disk space used by snapshots
(du).

the worst case when you re-write a file is that it will take the full amount
of space that the file currently takes (as if you wrote a new copy of the
file and some process had a filehandle open on the old copy, preventing the
space from being reclaimed, so it's far from being a new problem)

No. it's a new problem.
When you have a large db, which does random writes to an exiting db file,
it does not expect ENOSPC, when updating an existing record or index.
Only by keeping enough free disk space in the system at all times, can you
avoid this kind of problems.

You can very well avoid this kind of problems when you separate
filesystem and snapshots, that is what LVM can do easily.

quoted

see the note above about the need to be able to remove snapshots when you
are out of space.

since snapshots tend to be small compared to the filesystems they protect
(not in all cases, but if you are covering the entire system with one
snapshot that would be the way to bet), having the ability to put the
snapshot metadata off on a smaller/faster disk would be helpful.

Very easy to do with dm-multisnap.

Helpful for which workload?
For reading from snapshots? Yes, that would be faster.
For writing to the filesystem? I demonstrated that the performance
overhead is near zero.

quoted

having the ability to snapshot just specific files/directories would be a
killer feature IMHO

I agree to that, but I don't think the ext4 will be able to provide
that to the full extent.

And that is for being fs level snapshots a huge drawback.

quoted

David Lang
--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html