On Thu, Oct 18, 2018 at 04:42:07PM +0200, Christoph Hellwig wrote:

This all seems quite complicated.

I think the interface we'd want is more one that has a little
cache of a single page in the queue, and a little bitmap which
sub-page size blocks of it are used.

Something like (pseudo code minus locking):

void *blk_alloc_sector_buffer(struct block_device *bdev, gfp_t gfp)
{
	unsigned block_size = block_size(bdev);

	if (blocksize >= PAGE_SIZE)
		return (void *)__get_free_pages(gfp, get_order(blocksize));

	if (bdev->fragment_cache_page) {
		[ <find fragment in bdev->fragment_cache_page using
		  e.g. bitmap and return if found]
	}

	bdev->fragment_cache_page = (void *)__get_free_page(gfp);
	goto find_again;
}

This looks a lot like page_frag_alloc() except I think page_frag_alloc()
may be more efficient.