On Mon, Aug 24, 2020 at 09:57:55PM +0200, David Sterba wrote:

On Fri, Aug 21, 2020 at 12:39:50AM -0700, Omar Sandoval wrote:

quoted

Protocol Updates
================

This series makes some changes to the send stream protocol beyond adding
the encoded write command/attributes and bumping the version. Namely, v1
has a 64k limit on the size of a write due to the 16-bit attribute
length. This is not enough for encoded writes, as compressed extents may
be up to 128k and cannot be split up. To address this, the
BTRFS_SEND_A_DATA is treated specially in v2: its length is implicitly
the remaining length of the command (which has a 32-bit length). This
was the last bad of the options I considered.

There are other commands that we've been wanting to add to the protocol:
fallocate and FS_IOC_SETFLAGS. This series reserves their command and
attribute numbers but does not implement kernel support for emitting
them. However, it does implement support in receive for them, so the
kernel can start emitting those whenever we get around to implementing
them.

Can you please outline the protocol changes (as a bullet list) and
eventually cross-ref with items
https://btrfs.wiki.kernel.org/index.php/Design_notes_on_Send/Receive#Send_stream_v2_draft

I'd like to know which and why you did not implement. The decision here
is between get v2 out with most desired options and rev v3 later with
the rest, or do v2 as complete as possible.

The short version is that I didn't implement the kernel side of any of
those :) the RWF_ENCODED series + this series is already big, and I
didn't want to make it even bigger. I figured updating the
protocol/receive now and doing the kernel side later was a good
compromise (rather than doing a huge code dump or constantly bumping the
protocol version). Is there some reason you don't like this approach?
I'm of course happy to go about this in whatever way you think is best.

Here's a breakdown of the list from the wiki:

* Send extent holes, send preallocated extents: both require fallocate.
  Boris implemented the receive side. I have some old patches
  implementing the send side [1], but they're a largish rework of extent
  tracking in send.
* Extent clones within one file: as far as I can tell, this is already
  possible with v1, it just sends redundant file paths.
* Send otime for inodes: the consensus when I posted patches to enable
  this [2] was that we don't want this after all.
* Send file flags (FS_IOC_GETFLAGS/FS_IOC_SETFLAGS): again, Boris
  implemented the receive side. I previously took a stab at the send
  side, but it's really annoying because of all of the interactions
  between directory inheritance, writes vs. NOCOW/append-only/immutable,
  etc. It's do-able, it would just take a lot of care.
* Optionally send owner/group as strings: this one I wasn't aware of.
* "block device is not sent over the stream": I don't know what this is
  referring to. It looks like we send block device nodes with mknod.

In my opinion, fallocate support is the most important, SETFLAGS would
be good but is a lot of effort, and the rest are nice-to-have.

Let me know how you'd like me to go about this.

1: https://github.com/osandov/linux/commits/btrfs-send-v2
2: https://lore.kernel.org/linux-btrfs/cover.1550136164.git.osandov@fb.com/ (local)