Re: [PATCH 0/3] readfile(2): a new syscall to make open/read/close faster

On Sun, Jul 05, 2020 at 09:25:39AM +0200, Jan Ziak wrote:

On Sun, Jul 5, 2020 at 8:32 AM Andreas Dilger [off-list ref] wrote:

quoted

On Jul 4, 2020, at 8:46 PM, Jan Ziak [off-list ref] wrote:

quoted

On Sun, Jul 5, 2020 at 4:16 AM Matthew Wilcox [off-list ref] wrote:

quoted

On Sun, Jul 05, 2020 at 04:06:22AM +0200, Jan Ziak wrote:

quoted

Hello

At first, I thought that the proposed system call is capable of
reading *multiple* small files using a single system call - which
would help increase HDD/SSD queue utilization and increase IOPS (I/O
operations per second) - but that isn't the case and the proposed
system call can read just a single file.

Without the ability to read multiple small files using a single system
call, it is impossible to increase IOPS (unless an application is
using multiple reader threads or somehow instructs the kernel to
prefetch multiple files into memory).

What API would you use for this?

ssize_t readfiles(int dfd, char **files, void **bufs, size_t *lens);

I pretty much hate this interface, so I hope you have something better
in mind.

I am proposing the following:

struct readfile_t {
 int dirfd;
 const char *pathname;
 void *buf;
 size_t count;
 int flags;
 ssize_t retval; // set by kernel
 int reserved; // not used by kernel
};

If you are going to pass a struct from userspace to the kernel, it
should not mix int and pointer types (which may be 64-bit values,
so that there are not structure packing issues, like:

struct readfile {
        int     dirfd;
        int     flags;
        const char *pathname;
        void    *buf;
        size_t  count;
        ssize_t retval;
};

It would be better if "retval" was returned in "count", so that
the structure fits nicely into 32 bytes on a 64-bit system, instead
of being 40 bytes per entry, which adds up over many entries, like.

I know what you mean and it is a valid point, but in my opinion it
shouldn't (in most cases) be left to the programmer to decide what the
binary layout of a data structure is - instead it should be left to an
optimizing compiler to decide it.

We don't get that luxury when creating user/kernel apis in C, sorry.

I suggest using the pahole tool if you are interested in seeing the
"best" way a structure can be layed out, it can perform that
optimization for you so that you know how to fix your code.

thanks,

greg k-h