Re: [PATCH v2 0/4] open/accept directly into io_uring fixed file table

On 8/14/21 9:42 PM, Josh Triplett wrote:

On Sat, Aug 14, 2021 at 05:03:44PM -0600, Jens Axboe wrote:

quoted

What's the plan in terms of limiting the amount of direct descriptors
(for lack of a better word)? That seems like an important aspect that
should get sorted out upfront.

[...]

quoted

Maybe we have a way to size the direct table, which will consume entries
from the same pool that the regular file table does? That would then
work both ways, and could potentially just be done dynamically similarly
to how we expand the regular file table when we exceed its current size.

I think we'll want a way to size the direct table regardless, so that
it's pre-allocated and doesn't need to be resized when an index is used.

But how do you size it then? I can see this being used into the hundreds
of thousands of fds easily, and right now the table is just an array
(though split into segments, avoiding huge allocs).

Then, we could do one of two equally easy things, depending on what
policy we want to set:

- Deduct the full size of the fixed-file table from the allowed number
  of files the process can have open. So, if RLIMIT_NOFILE is 1048576,
  and you pre-allocate 1000000 entries in the fixed-file table, you can
  have no more than 48576 file descriptors open. Stricter, but
  potentially problematic: a program *might* expect that it can
  dup2(some_fd, nofile - 1) successfully.

- Use RLIMIT_NOFILE as the maximum size of the fixed-file table. There's
  precedent for this: we already use RLIMIT_NOFILE as the maximum number
  of file descriptors you can have in flight over UNIX sockets.

I personally would favor the latter; it seems simple and
straightforward.

I strongly prefer the latter too, and hopefully that's palatable since
the default limits are quite low anyway. And, as you say, it already is
done for inflight fds as well.

-- 
Jens Axboe