Re: [RFC][V8][PATCH 0/5] btrfs: allocation_hint mode

On Fri, Dec 17, 2021 at 12:40:55AM -0500, Zygo Blaxell wrote:

On Wed, Dec 15, 2021 at 07:56:32PM -0500, Josef Bacik wrote:

quoted

On Wed, Dec 15, 2021 at 07:53:40PM +0100, Goffredo Baroncelli wrote:

quoted

On 12/15/21 14:58, Josef Bacik wrote:

quoted

On Tue, Dec 14, 2021 at 09:41:21PM +0100, Goffredo Baroncelli wrote:

quoted

On 12/14/21 21:34, Josef Bacik wrote:

quoted

On Tue, Dec 14, 2021 at 03:04:32PM -0500, Zygo Blaxell wrote:

quoted

On Tue, Dec 14, 2021 at 08:03:45PM +0100, Goffredo Baroncelli wrote:

quoted

quoted

I don't have a strong preference for either sysfs or ioctl, nor am I
opposed to simply implementing both.  I'll let someone who does have
such a preference make their case.

I think echo'ing a name into sysfs is better than bits for sure.  However I want
the ability to set the device properties via a btrfs-progs command offline so I
can setup the storage and then mount the file system.  I want

1) The sysfs interface so you can change things on the fly.  This stays
     persistent of course, so the way it works is perfect.

2) The btrfs-progs command sets it on offline devices.  If you point it at a
     live mounted fs it can simply use the sysfs thing to do it live.

#2 is currently not implemented. However I think that we should do.

The problem is that we need to update both:

- the superblock		(simple)
- the dev_item item		(not so simple...)

What about using only bits from the superblock to store this property ?

I'm looking at the patches and you only are updating the dev_item, am I missing
something for the super block?

When btrfs write the superblocks (see write_all_supers() in disk-io.c), it copies
the dev_item fields (contained in fs_info->fs_devices->devices lists) in each
superblock before updating it.

Oh right.  Still, I hope we're doing this correctly in btrfs-progs, if not
that's a problem.

quoted

quoted

For offline all you would need to do is do the normal open_ctree,
btrfs_search_slot to the item and update the device item type, that's
straightforward.

For online if you use btrfs prop you can see if the fs is mounted and just find
the sysfs file to modify and do it that way.

But this also brings up another point, we're going to want a compat bit for
this.  It doesn't make the fs unusable for old kernels, so just a normal
BTRFS_FS_COMPAT_<whatever> flag is fine.  If the setting gets set you set the
compat flag.

Why we need a "compact" bit ? The new kernels know how treat the dev_item_type field.
The old kernels ignore it. The worst thing is that a filesystem may require a balance
before reaching a good shape (i.e. the metadata on ssd and the data on a spinning disk)

So you can do the validation below, tho I'm thinking I care about it less, if we
just make sure that type is correct regardless of the compat bit then that's
fine.  Thanks,

In theory if you get stuck in an impossible allocation situation (like all
your disks are data-only and you run out of metadata space) then one way
to recover from it is to mount with an old kernel which doesn't respect
the type bits.  Another way to recover would be to flip the type bits
while the filesystem is offline with btrfs-progs.  A third way would be to
have a mount option for newer kernels to ignore the allocation bits like
old kernels do (yes I know I already said I didn't like that idea).

This is the "preferred" vs "only" category.  You set "preferred" if you want to
be able to handle failures cleanly, you set "only" if you'd rather reprovision a
box than lose performance.

If we have a bit that says "old kernels don't mount this filesystem any
more" then we lose one of those recovery options, and the other options
aren't implemented yet.

While I think of it, the metadata reservation system eventually needs
to know that it can't use data-only devices for metadata, the same way
that df eventually needs to know about metadata-only devices.

Yeah, I'm willing to throw that into the power user bucket.  It really only
affects overcommit, because once we've allocated all our chunks we'll stop
overcommitting.  There may be some corners that need to be addressed, but we can
burn those bridges when we get to them.  Thanks,

Josef