Hi Jan,

I have been reading through the code of this change. Since my
understanding of the ctree mechanics is quite limited at this point, I
was wondering whether you can give some overall perspective.

I understand that you added hooks in the code that does raw tree
changes, to record them in an in-memory structure, giving them a
time_seq stamp.

You say that using this mechanism, it is possible to get a consistent
tree. Do you mean by this -  a tree like it was before the current
transaction has begun? What is not consistent that callers of the
regular btrfs_search_slot() can get back? When one should consider
using btrfs_search_old_slot() vs regular btrfs_search_slot()? Also, is
there any relation between the time_seq stamps and the transaction
stamps? What example can I do, to actually see (and better understand)
the effect of your code?

Hope I am asking right questions.

Thanks,
Alex.


On Sat, May 26, 2012 at 1:59 PM, Jan Schmidt [off-list ref] wrote:

Chris: Please pull

       git://git.jan-o-sch.net/btrfs-unstable for-chris

for 3.5, it's a major improvement for the backref resolver and a good
base to build qgroups and "btrfs send" onto. And, besides all other, its
implications on existing code outside backref.c are very small, mostly
added lines with no effect outside the backref resolver.

This patch set is to provide reliable backref resolving on busy trees.
The previous attempts to block certain tree modifications while we're
resolving backrefs all ended up in (dead-) locking nightmares. What we
now do is we record all the changes we make to fs trees while backref
resolving is in progress into a tree modification log. During backref
resolving we then merge the current state of the tree with the recorded
modifications to get a consistent previous state of the tree.

The first three commits are plain bug fixes, but the tree mod log
cannot function without them. So they're included in this patch set.
To only these three fixes, you can pull

       git://git.jan-o-sch.net/btrfs-unstable 3.5-fixes

For the whole tree modification log, pull

       git://git.jan-o-sch.net/btrfs-unstable tree-mod-log-v2

The tree-mod-log-v2 head is a stable alias for "for-chris".

Both branches are based on the current for-linus branch from Chris'
repository. Checked with xfstests (fails 254 273 275, which i claim has
nothing to do with this patch set) and hammered on the filesystem with
fs_mark while resolving backrefs in a loop.

I removed the qgroups patch series still contained in v1 because I
realized that the fs_mark tests I was doing weren't sufficient. When
testing qgroups with fsstress only for a few seconds, the accounting
breaks. Nothing really bad happens, but quota patches that don't get
quotas right are of litte use. A new qgroup patch set might come out
soon, but it won't make it into 3.5. There's a branch in my git repo
called tree-mod-log-quota-v1+fixes holding the current qgroup state.

Test it, break it, report it :-)

-Jan

--
Changes v1->v2:
- qgroups patches removed
- ulist realloc bugfix added (reported by Alexander Block)
- btrfs_find_parent_nodes bugfix (reported by Alexander Block)
- various smaller bugs in the v1 patch set fixed (reported by Alexander
 Block, Tsutomu Itoh and myself)
--

Jan Schmidt (15):
 Btrfs: ulist realloc bugfix
 Btrfs: bugfix in btrfs_find_parent_nodes
 Btrfs: bugfix: ignore the wrong key for indirect tree block backrefs
 Btrfs: look into the extent during find_all_leafs
 Btrfs: don't set for_cow parameter for tree block functions
 Btrfs: move struct seq_list to ctree.h
 Btrfs: dummy extent buffers for tree mod log
 Btrfs: add tree mod log to fs_info
 Btrfs: add tree modification log functions
 Btrfs: put all block modifications into the tree mod log
 Btrfs: add del_ptr and insert_ptr modifications to the tree mod log
 Btrfs: add btrfs_search_old_slot
 Btrfs: use the tree modification log for backref resolving
 Btrfs: fs_info variable for join_transaction
 Btrfs: tree mod log sanity checks in join_transaction

 fs/btrfs/backref.c     |  465 +++++++++++++++++++---------
 fs/btrfs/backref.h     |    3 +-
 fs/btrfs/ctree.c       |  831 ++++++++++++++++++++++++++++++++++++++++++++++--
 fs/btrfs/ctree.h       |   27 ++-
 fs/btrfs/delayed-ref.h |    5 -
 fs/btrfs/disk-io.c     |    7 +-
 fs/btrfs/extent-tree.c |   10 +-
 fs/btrfs/extent_io.c   |   80 +++++-
 fs/btrfs/extent_io.h   |    3 +
 fs/btrfs/ioctl.c       |    2 +-
 fs/btrfs/transaction.c |   46 ++-
 fs/btrfs/ulist.c       |   23 +-
 fs/btrfs/ulist.h       |    9 +-
 13 files changed, 1286 insertions(+), 225 deletions(-)

--
1.7.3.4

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html