Re: [PATCHv2 3/6] io_uring: add support for kernel registered bvecs
From: Ming Lei <hidden>
Date: 2025-02-14 03:30:30
Also in:
io-uring
On Mon, Feb 10, 2025 at 04:56:43PM -0800, Keith Busch wrote:
From: Keith Busch <kbusch@kernel.org> Provide an interface for the kernel to leverage the existing pre-registered buffers that io_uring provides. User space can reference these later to achieve zero-copy IO. User space must register an empty fixed buffer table with io_uring in order for the kernel to make use of it. Signed-off-by: Keith Busch <kbusch@kernel.org> ---
...
+int io_buffer_register_bvec(struct io_ring_ctx *ctx, struct request *rq,
+ void (*release)(void *), unsigned int index)
+{
+ struct io_rsrc_data *data = &ctx->buf_table;
+ struct req_iterator rq_iter;
+ struct io_mapped_ubuf *imu;
+ struct io_rsrc_node *node;
+ struct bio_vec bv;
+ u16 nr_bvecs;
+ int i = 0;
+
+ lockdep_assert_held(&ctx->uring_lock);
+
+ if (!data->nr)
+ return -EINVAL;
+ if (index >= data->nr)
+ return -EINVAL;
+
+ node = data->nodes[index];
+ if (node)
+ return -EBUSY;
+
+ node = io_rsrc_node_alloc(IORING_RSRC_KBUFFER);
+ if (!node)
+ return -ENOMEM;
+
+ node->release = release;
+ node->priv = rq;
+
+ nr_bvecs = blk_rq_nr_phys_segments(rq);
+ imu = kvmalloc(struct_size(imu, bvec, nr_bvecs), GFP_KERNEL);
+ if (!imu) {
+ kfree(node);
+ return -ENOMEM;
+ }
+
+ imu->ubuf = 0;
+ imu->len = blk_rq_bytes(rq);
+ imu->acct_pages = 0;
+ imu->nr_bvecs = nr_bvecs;
+ refcount_set(&imu->refs, 1);
+ node->buf = imu;request buffer direction needs to be stored in `imu`, for READ, the buffer is write-only, and for WRITE, the buffer is read-only, which isn't different with user mapped buffer. Meantime in read_fixed/write_fixed side or buffer lookup abstraction helper, the buffer direction needs to be validated. Thanks, Ming