[PATCH 09/16] io_uring: don't sleep when polling for I/O
From: Christoph Hellwig <hch@lst.de>
Date: 2021-06-15 13:14:09
Also in:
linux-fsdevel, linux-nvme
Subsystem:
block layer, filesystems (vfs and infrastructure), the rest · Maintainers:
Jens Axboe, Alexander Viro, Christian Brauner, Linus Torvalds
There is no point in sleeping for the expected I/O completion timeout in the io_uring async polling model as we never poll for a specific I/O. Signed-off-by: Christoph Hellwig <hch@lst.de> Tested-by: Mark Wunderlich <redacted> --- block/blk-mq.c | 3 ++- fs/io_uring.c | 2 +- include/linux/blkdev.h | 2 ++ 3 files changed, 5 insertions(+), 2 deletions(-)
diff --git a/block/blk-mq.c b/block/blk-mq.c
index c2b2c7abd712..86e59ffcca44 100644
--- a/block/blk-mq.c
+++ b/block/blk-mq.c@@ -4002,7 +4002,8 @@ int blk_poll(struct request_queue *q, blk_qc_t cookie, unsigned int flags) if (current->plug) blk_flush_plug_list(current->plug, false); - if (q->poll_nsec != BLK_MQ_POLL_CLASSIC) { + if (!(flags & BLK_POLL_NOSLEEP) && + q->poll_nsec != BLK_MQ_POLL_CLASSIC) { if (blk_mq_poll_hybrid(q, cookie)) return 1; }
diff --git a/fs/io_uring.c b/fs/io_uring.c
index de8d39e9a154..6ce1e5b60725 100644
--- a/fs/io_uring.c
+++ b/fs/io_uring.c@@ -2290,7 +2290,7 @@ static void io_iopoll_complete(struct io_ring_ctx *ctx, unsigned int *nr_events, static int io_do_iopoll(struct io_ring_ctx *ctx, unsigned int *nr_events, long min) { - unsigned int poll_flags = 0; + unsigned int poll_flags = BLK_POLL_NOSLEEP; struct io_kiocb *req, *tmp; LIST_HEAD(done); int ret = 0;
diff --git a/include/linux/blkdev.h b/include/linux/blkdev.h
index c7061f25ee5f..821d721f78e7 100644
--- a/include/linux/blkdev.h
+++ b/include/linux/blkdev.h@@ -948,6 +948,8 @@ blk_status_t errno_to_blk_status(int errno); /* only poll the hardware once, don't continue until a completion was found */ #define BLK_POLL_ONESHOT (1 << 0) +/* do not sleep to wait for the expected completion time */ +#define BLK_POLL_NOSLEEP (1 << 1) int blk_poll(struct request_queue *q, blk_qc_t cookie, unsigned int flags); static inline struct request_queue *bdev_get_queue(struct block_device *bdev)
--
2.30.2