select fails to verify all file descriptors are valid

Started by Matthew Wilcox on Mar 14, 2017 · 4 messages from 3 authors · Last reply Mar 24, 2017

open the first message on its own page

Message details and related content
Quoting the manpage:

       int select(int nfds, fd_set *readfds, fd_set *writefds,
                  fd_set *exceptfds, struct timeval *timeout);

       nfds  is the highest-numbered file descriptor in any of the three sets,
       plus 1.

       EBADF  An  invalid file descriptor was given in one of the sets.  (Per‐
              haps a file descriptor that was already closed, or one on  which
              an error has occurred.)

That's not quite how Linux behaves.  We only check the fd_set up to the
maximum number of fds allocated to this task:

        rcu_read_lock();
        fdt = files_fdtable(current->files);
        max_fds = fdt->max_fds;
        rcu_read_unlock();
        if (n > max_fds)
                n = max_fds;

(then we copy in up to 'n' bits worth of bitmaps).

It is pretty straightforward to demonstrate that Linux doesn't check:

int main(void)
{
	int ret;
	struct timeval tv = { };
	fd_set fds;
	FD_ZERO(&fds);
	FD_SETFD(FD_SETSIZE - 1, &fds);
	ret = select(FD_SETSIZE, &fds, NULL, NULL, &tv);
	assert(ret == -1 && errno == EBADF);
	return 0;
}

Linux has behaved this way since 2.6.12, and I can't be bothered to get

9 more lines

Message details and related content
On 03/14/2017 12:11 PM, Matthew Wilcox wrote:

Hello Matthew :-)
 
Quoting the manpage:

       int select(int nfds, fd_set *readfds, fd_set *writefds,
                  fd_set *exceptfds, struct timeval *timeout);

       nfds  is the highest-numbered file descriptor in any of the three sets,
       plus 1.
OK.
       EBADF  An  invalid file descriptor was given in one of the sets.  (Per‐
              haps a file descriptor that was already closed, or one on  which
              an error has occurred.)

That's not quite how Linux behaves.  We only check the fd_set up to the
maximum number of fds allocated to this task:

        rcu_read_lock();
        fdt = files_fdtable(current->files);
        max_fds = fdt->max_fds;
        rcu_read_unlock();
        if (n > max_fds)
                n = max_fds;

(then we copy in up to 'n' bits worth of bitmaps).
Right, and that doesn't match the POSIX requirement either.

http://pubs.opengroup.org/onlinepubs/9699919799/functions/pselect.html
~~~
The nfds argument specifies the range of descriptors to be tested. 
The first nfds descriptors shall be checked in each set; that is, 
the descriptors from zero through nfds-1 in the descriptor sets 
shall be examined.
~~~

That is to say you should "test" `nfds - 1` descriptors, potentially
returning EBADF for invalid descriptors, not some random number that
depends on how many you have open right now.
It is pretty straightforward to demonstrate that Linux doesn't check:

int main(void)
{
	int ret;
	struct timeval tv = { };
	fd_set fds;
	FD_ZERO(&fds);
	FD_SETFD(FD_SETSIZE - 1, &fds);
	ret = select(FD_SETSIZE, &fds, NULL, NULL, &tv);
	assert(ret == -1 && errno == EBADF);
	return 0;
}
And one that compiles and works...

cat >> select-verify.c <<EOF
#include <stdio.h>
#include <stdlib.h>
#include <sys/select.h>
#include <sys/time.h>
#include <sys/types.h>
#include <unistd.h>
#include <assert.h>
#include <errno.h>

int 
main(void)
{
  int ret;
  struct timeval tv = {0, 0};
  fd_set fds;
  FD_ZERO(&fds);
  FD_SET(FD_SETSIZE - 1, &fds);
  ret = select(FD_SETSIZE, &fds, NULL, NULL, &tv);
  assert(ret == -1 && errno == EBADF);
  return 0;
}
EOF
gcc -O0 -g3 -Wall -pedantic -o ./select-verify ./select-verify.c
./select-verify 
select-verify: ./select-verify.c:19: main: Assertion `ret == -1 && errno == EBADF' failed.
Aborted (core dumped)
Linux has behaved this way since 2.6.12, and I can't be bothered to get
out the historical git trees to find out what happened before 2005.

So ... if I change this behaviour by checking all the file descriptors, I
do stand a chance of breaking an application.  On the other hand, that
application could already have been broken by the shell deciding to open
a really high file descriptor (I'm looking at you, bash), which the program
then inherits.
The user application would have to:

(a) Use a sloppy limit for nfds e.g. `FD_SETSIZE - 1`.

_and_

(b) Have an fd selected that they were not interested in selecting.

Exactly what the test application above does.
Worth fixing this bug?  Worth documenting this bug, at least?
I would fix it on the grounds of making the interface as robust as
it was designed to be.

Right now it randomly depends on your task max_fds if it's going to
work or not. That's pretty terrible.

I'm including Michael Kerrisk here if he has any comments since he
touched the manual page several times ;-)

-- 
Cheers,
Carlos.
Message details and related content
Hi Matthew,

On 03/14/2017 05:11 PM, Matthew Wilcox wrote:
Quoting the manpage:

       int select(int nfds, fd_set *readfds, fd_set *writefds,
                  fd_set *exceptfds, struct timeval *timeout);

       nfds  is the highest-numbered file descriptor in any of the three sets,
       plus 1.

       EBADF  An  invalid file descriptor was given in one of the sets.  (Per‐
              haps a file descriptor that was already closed, or one on  which
              an error has occurred.)

That's not quite how Linux behaves.  We only check the fd_set up to the
maximum number of fds allocated to this task:

        rcu_read_lock();
        fdt = files_fdtable(current->files);
        max_fds = fdt->max_fds;
        rcu_read_unlock();
        if (n > max_fds)
                n = max_fds;

(then we copy in up to 'n' bits worth of bitmaps).

It is pretty straightforward to demonstrate that Linux doesn't check:

int main(void)
{
	int ret;
	struct timeval tv = { };
	fd_set fds;
	FD_ZERO(&fds);
	FD_SETFD(FD_SETSIZE - 1, &fds);
	ret = select(FD_SETSIZE, &fds, NULL, NULL, &tv);
	assert(ret == -1 && errno == EBADF);
	return 0;
}

Linux has behaved this way since 2.6.12, and I can't be bothered to get

1 more quoted lines

Looking back, this behavior seems to have been present at least as far 
back as 2.4.0:

        if (n > current->files->max_fdset)
                n = current->files->max_fdset;

My reading on the 2.2.0 source suggests this odd behavior did not 
occur there.
So ... if I change this behaviour by checking all the file descriptors, I
do stand a chance of breaking an application.  On the other hand, that
application could already have been broken by the shell deciding to open
a really high file descriptor (I'm looking at you, bash), which the program
then inherits.
I'm not sure that bash's FD 255 oddity is a problem here, since the 
close-on-exec flag is set on that FD.
Worth fixing this bug?  Worth documenting this bug, at least?
Certainly worth documenting. Fixing? I'm less sure. But, Carlos does
have a point about POSIX...

I've updated the man page to say:

       nfds  should be set to the highest-numbered file descriptor in any
       of the three sets, plus 1.  The indicated file descriptors in each
       set are checked, up to this limit (but see BUGS).

       ...

       EBADF  An  invalid  file  descriptor was given in one of the sets.
              (Perhaps a file descriptor that was already closed, or  one
              on which an error has occurred.)  However, see BUGS.
       ...

   BUGS
       According  to  POSIX,  select()  should  check  all specified file
       descriptors in the three file descriptor sets,  up  to  the  limit
       nfds-1.   However,  the  current  implementation  ignores any file
       descriptor in these sets that is greater  than  the  maximum  file
       descriptor  number that the process currently has open.  According
       to POSIX, any such file descriptor that is specified in one of the
       sets should result in the error EBADF.

Cheers,

Michael
--
To unsubscribe from this list: send the line "unsubscribe linux-api" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

-- 
Michael Kerrisk
Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
Linux/UNIX System Programming Training: http://man7.org/training/
Message details and related content
Hi Carlos,

On 03/15/2017 06:34 AM, Carlos O'Donell wrote:
On 03/14/2017 12:11 PM, Matthew Wilcox wrote:

Hello Matthew :-)
 
quoted
Quoting the manpage:

       int select(int nfds, fd_set *readfds, fd_set *writefds,
                  fd_set *exceptfds, struct timeval *timeout);

       nfds  is the highest-numbered file descriptor in any of the three sets,
       plus 1.
OK.
quoted
       EBADF  An  invalid file descriptor was given in one of the sets.  (Per‐
              haps a file descriptor that was already closed, or one on  which
              an error has occurred.)

That's not quite how Linux behaves.  We only check the fd_set up to the
maximum number of fds allocated to this task:

        rcu_read_lock();
        fdt = files_fdtable(current->files);
        max_fds = fdt->max_fds;
        rcu_read_unlock();
        if (n > max_fds)
                n = max_fds;

(then we copy in up to 'n' bits worth of bitmaps).
Right, and that doesn't match the POSIX requirement either.

http://pubs.opengroup.org/onlinepubs/9699919799/functions/pselect.html
~~~
The nfds argument specifies the range of descriptors to be tested. 
The first nfds descriptors shall be checked in each set; that is, 
the descriptors from zero through nfds-1 in the descriptor sets 
shall be examined.
~~~

3 more quoted lines

Thanks for checking POSIX!
quoted
It is pretty straightforward to demonstrate that Linux doesn't check:

int main(void)
{
	int ret;
	struct timeval tv = { };
	fd_set fds;
	FD_ZERO(&fds);
	FD_SETFD(FD_SETSIZE - 1, &fds);
	ret = select(FD_SETSIZE, &fds, NULL, NULL, &tv);
	assert(ret == -1 && errno == EBADF);
	return 0;
}
And one that compiles and works...

cat >> select-verify.c <<EOF
#include <stdio.h>
#include <stdlib.h>
#include <sys/select.h>
#include <sys/time.h>
#include <sys/types.h>
#include <unistd.h>
#include <assert.h>
#include <errno.h>

int 
main(void)
{
  int ret;
  struct timeval tv = {0, 0};
  fd_set fds;
  FD_ZERO(&fds);
  FD_SET(FD_SETSIZE - 1, &fds);
  ret = select(FD_SETSIZE, &fds, NULL, NULL, &tv);
  assert(ret == -1 && errno == EBADF);
  return 0;
}
EOF
gcc -O0 -g3 -Wall -pedantic -o ./select-verify ./select-verify.c

3 more quoted lines

And thanks for the CWE.
 
quoted
Linux has behaved this way since 2.6.12, and I can't be bothered to get
out the historical git trees to find out what happened before 2005.

So ... if I change this behaviour by checking all the file descriptors, I
do stand a chance of breaking an application.  On the other hand, that
application could already have been broken by the shell deciding to open
a really high file descriptor (I'm looking at you, bash), which the program
then inherits.
The user application would have to:

(a) Use a sloppy limit for nfds e.g. `FD_SETSIZE - 1`.

_and_

(b) Have an fd selected that they were not interested in selecting.

Exactly what the test application above does.
quoted
Worth fixing this bug?  Worth documenting this bug, at least?
I would fix it on the grounds of making the interface as robust as
it was designed to be.

Right now it randomly depends on your task max_fds if it's going to
work or not. That's pretty terrible.
It's certainly at least sloppy. Of course, fixing this might
conceivably break some (broken) code. But, the violation of POSIX
suggests we should at least try a fix and see if anything does
get broken (presumably, it's rather unlikely).
I'm including Michael Kerrisk here if he has any comments since he
touched the manual page several times ;-)
Thanks ;-).

Cheers,

Michael

-- 
Michael Kerrisk
Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
Linux/UNIX System Programming Training: http://man7.org/training/
Keyboard shortcuts
hback out one level
jnext message in thread / next series-diff position
kprevious message in thread / previous series-diff position
ldrill in
Escclose help / fold thread tree
?toggle this help