On Sun, Jan 15, 2017 at 04:01:30PM -0800, Viacheslav Dubeyko wrote:

On Fri, 2017-01-13 at 23:54 -0800, Darrick J. Wong wrote:

quoted

Hi,

I've been working on implementing online metadata scrubbing and
repair
in XFS.  Most of the code is self contained inside XFS, but there's a
small amount of interaction with the VFS freezer code that has to
happen
in order to shut down the filesystem to rebuild the extent backref
records.  It might be interesting to discuss the (fairly slight)
requirements upon the VFS to support repairs, and/or have a BoF to
discuss how to build an online checker if any of the other
filesystems
are interested in this.

How do you imagine a generic way to support repairs for different file
systems? From one point of view, to have generic way of the online file
system repairing could be the really great subsystem.

I don't, sadly.  There's not even a way to /check/ all fs metadata in a
"generic" manner -- we can use the standard VFS interfaces to read
all metadata, but this is fraught.  Even if we assume the fs can spot
check obviously garbage values, that's still not the appropriate place
for a full scan.

But, from another point of view, every file system has own
architecture, own set of metadata and own way to do fsck
check/recovering.

Yes, and this wouldn't change.  The particular mechanism of fixing a
piece of metadata will always be fs-dependent, but the thing that I'm
interested in discussing is how do we avoid having these kinds of things
interact badly with the VFS?

As far as I can judge, there are significant amount of research
efforts in this direction (Recon [1], [2], for example).

Yes, I remember Recon.  I appreciated the insight that while it's
impossible to block everything for a full scan, it /is/ possible to
check a single object and its relation to other metadata items.  The xfs
scrubber also takes an incremental approach to verifying a filesystem;
we'll lock each metadata object and verify that its relationships with
the other metadata make sense.  So long as we aren't bombarding the fs
with heavy metadata update workloads, of course.

On the repair side of things xfs added reverse-mapping records, which
the repair code uses to regenerate damaged primary metadata.  After we
land inode parent pointers we'll be able to do the same reconstructions
that we can now do for block allocations...

...but there are some sticky problems with repairing the reverse
mappings.  The normal locking order for that part of xfs is sb_writers
-> inode -> ag header -> rmap btree blocks, but to repair we have to
freeze the filesystem against writes so that we can scan all the inodes.

But we still haven't any real general online file system repair
subsystem in the Linux kernel.

I think the ocfs2 developers have encoded some ability to repair
metadata over the past year, though it seems limited to fixing some
parts of inodes.  btrfs stores duplicate copies and restores when
necessary, I think.  Unfortunately, fixing disk corruption is something
that's not easily genericized, which means that I don't think we'll ever
achieve a general subsystem.

But we could at least figure out what in the VFS has to change (if
anything) to support this type of usage.

Do you have some new insight? What's difference of your
vision? If we have online file system repair subsystem then how file
system driver will need to interact with the goal to make internal
repairing?

It's pretty much all private xfs userspace ioctls[1] with a driver
program[2].

--D

[1] https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git/log/?h=djwong-devel
[2] https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=djwong-devel

Thanks,
Vyacheslav Dubeyko.

[1] http://www.eecg.toronto.edu/~ashvin/publications/recon-fs-consistency-runtime.pdf
[2] https://www.researchgate.net/publication/269300836_Managing_the_file_system_from_the_kernel

--
To unsubscribe from this list: send the line "unsubscribe linux-xfs" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html