nvmf question - synchronization between target/initiator regarding partitions
From: hare@suse.de (Hannes Reinecke)
Date: 2017-08-07 14:20:35
Also in:
linux-block, linux-scsi
On 08/07/2017 03:42 PM, Guilherme G. Piccoli wrote:
We observed that it's possible to perform partition operations in both nvmf target and initiator block devices, like creating and deleting partitions. But there is no sync mechanism between target and initiator regarding the partitions operations. After creating a partition on initiator, for example, we don't see it on target side. We could format it like ext4 on initiator, and still target cannot see it. It's possible to trigger a BLKRRPART ioctl on target, which end up calling revalidate_disk() on nvme driver and then partitions are perceived. So, question: is this behavior expected/acceptable? Is it completely up to userspace to deal with the synchronization between host/target? I think answer might be yes since partitions are a higher level of abstraction than nvmf (which is purely block device aware).
Yes, this is to be expected. After all, one could argue that no partition information should ever be visible on the target seeing that the device is exported. As the initiator has exclusive access, the target has no business looking at the contents; in fact, this might lead to unexpected corruptions if things like udev jump in on the target side and try to 'correct' things with journal replay etc.
But if kernel could/should jump in, we possibly could try to issue a revalidate_disk on target whenever this operation is performed on initiator side (and vice-versa). I confess I couldn't find such sync idea in NVMe over fabrics spec, though. Also, reading NVMe spec 1.3, we do have the optional feature "reservations", but seems it doesn't mention target (only multiple hosts).
See above. Never try this :-) Cheers, Hannes -- Dr. Hannes Reinecke Teamlead Storage & Networking hare at suse.de +49 911 74053 688 SUSE LINUX GmbH, Maxfeldstr. 5, 90409 N?rnberg GF: F. Imend?rffer, J. Smithard, J. Guild, D. Upmanyu, G. Norton HRB 21284 (AG N?rnberg)