Re: [PATCH] compat: Add another rudimentary poll() emulation

On Thu, May 27, 2010 at 2:36 PM, Marko Kreen [off-list ref] wrote:

On 5/27/10, Erik Faye-Lund [off-list ref] wrote:

quoted

On Thu, May 27, 2010 at 1:39 PM, Marko Kreen [off-list ref] wrote:
 > On 5/27/10, Erik Faye-Lund [off-list ref] wrote:
 >> On Thu, May 27, 2010 at 12:10 PM, Jonathan Nieder [off-list ref] wrote:
 >>  > Implement the subset of poll() semantics needed by git in terms of
 >>  > select(), for use by the Interix port.  Inspired by commit 6ed807f
 >>  > (Windows: A rudimentary poll() emulation, 2007-12-01).
 >>  >
 >>
 >>
 >> A possible problem with this approach is that the maximum number of
 >>  file descriptors poll can handle limited by RLIMIT_NOFILE, whereas the
 >>  maximum number of file descriptors select can handle is limited by
 >>  FD_SETSIZE.
 >>
 >>  I don't think this is a big problem in reality, though - both values
 >>  seem to be pretty high in most implementations. And IIRC git-daemon is
 >>  the only one who needs more than 2, and it doesn't even check
 >>  RLIMIT_NOFILE.
 >>
 >>  If we decide to go this route, perhaps it'd make sense to change to
 >>  this code for Windows also? Our Windows-implementation of poll() has
 >>  some annoying limitations...
 >
 > Example of poll() compat without FD_SETSIZE limit:
 >
 >  http://github.com/markokr/plproxy-dev/blob/master/src/poll_compat.c
 >


How does this code convince FD_SET() that the buffer has increased? It
 looks to me like it depends on a specific FD_SET() implementation...
 For instance, Windows' FD_SET() implementation is like this:

 #define FD_SET(fd, set) do { \
    if (((fd_set FAR *)(set))->fd_count < FD_SETSIZE) \
        ((fd_set FAR *)(set))->fd_array[((fd_set FAR
 *)(set))->fd_count++]=(fd);\
 } while(0)

 ...so unless another set is passed in, it won't add any more fds once
 fd_count reaches FD_SETSIZE.

 Also, FD_SETSIZE is 64 on Windows. IIRC it's 1024 on Linux, so it is
 much more likely that we encounter this issue on Windows than on
 Linux, at least ;)

Hm, good catch.  Seems such compat poll() cannot be done without
OS-specific hacks.

Perhaps getrlimit() could overridden to return FD_SETSIZE for both the
soft and hard limit when asking about RLIMIT_NOFILE? In such cases,
anyone who passes nfds above FD_SETSIZE hasn't consulted
RLIMIT_NOFILE, and should be outside the standard. But your point
might have been about a limitless poll()-implementation, like the code
you linked tried to achieve. In that context, no. I doubt it's
possible to do in a robust fashion.

For git, I don't think this is necessary, though. As said, I think
git-daemon is the only call-site for poll where nfds can be above 2.
And git-daemon's default max-connection is 32, which shouldn't cause
much problems. There's the theoretical problem of someone setting
--max-connections above their platform's limit, but I don't think we
need to touch that ;)

Do you know perhaps what other OS-es have non-bitmap fd_set?

No, I don't have much low-level file-descriptor knowledge about other
OS'es than Windows, really.

-- 
Erik "kusma" Faye-Lund