# Re: \[PATCH 3/7\] block: copy offload support infrastructure

- From: Bart Van Assche \<bvanassche@acm.org\>
- Date: 2021-08-20 21:18:34 
- List: linux-fsdevel
- Page: https://ratatoskr.run/linux-fsdevel/2021/08/212924
- Thread: https://ratatoskr.run/linux-nvme/2021/08/212564/t.md

```text
On 8/20/21 3:39 AM, Kanchan Joshi wrote:
> Bart, Mikulas
>
> On Tue, Aug 17, 2021 at 10:44 PM Bart Van Assche [ref] wrote:
>>
>> On 8/17/21 3:14 AM, SelvaKumar S wrote:
>>> Introduce REQ_OP_COPY, a no-merge copy offload operation. Create
>>> bio with control information as payload and submit to the device.
>>> Larger copy operation may be divided if necessary by looking at device
>>> limits. REQ_OP_COPY(19) is a write op and takes zone_write_lock when
>>> submitted to zoned device.
>>> Native copy offload is not supported for stacked devices.
>>
>> Using a single operation for copy-offloading instead of separate
>> operations for reading and writing is fundamentally incompatible with
>> the device mapper. I think we need a copy-offloading implementation that
>> is compatible with the device mapper.
>>
>
> While each read/write command is for a single contiguous range of
> device, with simple-copy we get to operate on multiple discontiguous
> ranges, with a single command.
> That seemed like a good opportunity to reduce control-plane traffic
> (compared to read/write operations) as well.
>
> With a separate read-and-write bio approach, each source-range will
> spawn at least one read, one write and eventually one SCC command. And
> it only gets worse as there could be many such discontiguous ranges (for
> GC use-case at least) coming from user-space in a single payload.
> Overall sequence will be
> - Receive a payload from user-space
> - Disassemble into many read-write pair bios at block-layer
> - Assemble those (somehow) in NVMe to reduce simple-copy commands
> - Send commands to device
>
> We thought payload could be a good way to reduce the
> disassembly/assembly work and traffic between block-layer to nvme.
> How do you see this tradeoff?  What seems necessary for device-mapper
> usecase, appears to be a cost when device-mapper isn't used.
> Especially for SCC (since copy is within single ns), device-mappers
> may not be too compelling anyway.
>
> Must device-mapper support be a requirement for the initial support atop SCC?
> Or do you think it will still be a progress if we finalize the
> user-space interface to cover all that is foreseeable.And for
> device-mapper compatible transport between block-layer and NVMe - we
> do it in the later stage when NVMe too comes up with better copy
> capabilities?

Hi Kanchan,

These days there might be more systems that run the device mapper on top of the NVMe driver or a SCSI driver than systems that do use the device mapper. It is common practice these days to use dm-crypt on personal workstations and laptops. LVM (dm-linear) is popular because it is more flexible than a traditional partition table. Android phones use dm-verity on top of hardware encryption. In other words, not supporting the device mapper means that a very large number of use cases is excluded. So I think supporting the device mapper from the start is important, even if that means combining individual bios at the bottom of the storage stack into simple copy commands.

Thanks,

Bart.

```