[REGRESSION] Select hang with zero sized UDP packets

Hi,

Fedora received a report[1] of a unit test failing on Ruby when using the
4.7 kernel. This was a test to send a zero sized UDP packet. With the
4.7 kernel, the test now timing out on a select instead of completing.
The reduced ruby test is

   def test_udp_recvfrom_nonblock
     u1 = UDPSocket.new
     u2 = UDPSocket.new
     u1.bind("127.0.0.1", 0)
     u2.send("", 0, u1.getsockname)
     IO.select [u1]  # test gets stuck here
   ensure
     u1.close if u1
     u2.close if u2
   end


which roughly corresponds to this in C

int main()
{
         int fd1, fd2;
         struct sockaddr_in addr1;
         unsigned int len1;
         int ret;
         fd_set rfds;

         fd1 = socket(AF_INET, SOCK_DGRAM|SOCK_CLOEXEC, IPPROTO_UDP);
         fd2 = socket(AF_INET, SOCK_DGRAM|SOCK_CLOEXEC, IPPROTO_UDP);

         if (fd1 < 0 || fd2 < 0) {
                 printf("socket fail");
                 exit(1);
         }

         len1 = sizeof(addr1);

         memset(&addr1, 0, sizeof(addr1));
         addr1.sin_family = AF_INET;
         addr1.sin_addr.s_addr = inet_addr("127.0.0.1");
         addr1.sin_port = htons(0);
         ret = bind(fd1, (struct sockaddr *)&addr1, len1);
         if (ret < 0) {
                 printf("fu %d\n", errno);
                 exit(1);
         }

         ret = getsockname(fd1, (struct sockaddr *)&addr1, &len1);
         if (ret < 0) {
                 printf("getsockname failed %d\n", errno);
                 exit(1);
         }
         ret = sendto(fd2, "", 0, 0, (struct sockaddr *)&addr1, len1);
         if (ret < 0) {
                 printf("sendto failed %d\n", errno);
                 exit(1);
         }

         FD_ZERO(&rfds);
         FD_SET(fd1, &rfds);
	// hang here
         select(fd1+1, &rfds, NULL, NULL, NULL);
}


Bisection showed

commit e6afc8ace6dd5cef5e812f26c72579da8806f5ac
Author: samanthakumar [off-list ref]
Date:   Tue Apr 5 12:41:15 2016 -0400

     udp: remove headers from UDP packets before queueing
     
     Remove UDP transport headers before queueing packets for reception.
     This change simplifies a follow-up patch to add MSG_PEEK support.
     
     Signed-off-by: Sam Kumar [off-list ref]
     Signed-off-by: Willem de Bruijn [off-list ref]
     Signed-off-by: David S. Miller [off-list ref]


As the offending commit. The issue is still reproducible on master
as of this morning and I don't see anything explicitly tagged in
net-next as fixing this.

Any ideas?

Thanks,
Laura

[1] https://bugzilla.redhat.com/show_bug.cgi?id=1365940

From: Laura Abbott <redacted>
Date: Tue, 23 Aug 2016 10:53:26 -0700

Fedora received a report[1] of a unit test failing on Ruby when using
the
4.7 kernel. This was a test to send a zero sized UDP packet. With the
4.7 kernel, the test now timing out on a select instead of completing.
The reduced ruby test is

  def test_udp_recvfrom_nonblock
    u1 = UDPSocket.new
    u2 = UDPSocket.new
    u1.bind("127.0.0.1", 0)
    u2.send("", 0, u1.getsockname)
    IO.select [u1]  # test gets stuck here
  ensure
    u1.close if u1
    u2.close if u2
  end

Well, if there is no data, should select really wake up?

I think it's valid not to.

On Tue, 2016-08-23 at 11:25 -0700, David Miller wrote:

From: Laura Abbott <redacted>
Date: Tue, 23 Aug 2016 10:53:26 -0700

quoted

Fedora received a report[1] of a unit test failing on Ruby when using
the
4.7 kernel. This was a test to send a zero sized UDP packet. With the
4.7 kernel, the test now timing out on a select instead of completing.
The reduced ruby test is

  def test_udp_recvfrom_nonblock
    u1 = UDPSocket.new
    u2 = UDPSocket.new
    u1.bind("127.0.0.1", 0)
    u2.send("", 0, u1.getsockname)
    IO.select [u1]  # test gets stuck here
  ensure
    u1.close if u1
    u2.close if u2
  end

Well, if there is no data, should select really wake up?

I think it's valid not to.

There are skb in receive queue, with skb->len = 0

This looks like a bug in first_packet_length() or poll logic.

Definitely something we can fix.

Maybe with :

diff --git a/net/ipv4/udp.c b/net/ipv4/udp.c
index e61f7cd65d08..380c05a84041 100644
--- a/net/ipv4/udp.c
+++ b/net/ipv4/udp.c

@@ -1184,11 +1184,11 @@ out:
  *	Drops all bad checksum frames, until a valid one is found.
  *	Returns the length of found skb, or 0 if none is found.
  */
-static unsigned int first_packet_length(struct sock *sk)
+static int first_packet_length(struct sock *sk)
 {
 	struct sk_buff_head list_kill, *rcvq = &sk->sk_receive_queue;
 	struct sk_buff *skb;
-	unsigned int res;
+	int res;
 
 	__skb_queue_head_init(&list_kill);
 

@@ -1203,7 +1203,7 @@ static unsigned int first_packet_length(struct sock *sk)
 		__skb_unlink(skb, rcvq);
 		__skb_queue_tail(&list_kill, skb);
 	}
-	res = skb ? skb->len : 0;
+	res = skb ? skb->len : -1;
 	spin_unlock_bh(&rcvq->lock);
 
 	if (!skb_queue_empty(&list_kill)) {

@@ -1232,7 +1232,7 @@ int udp_ioctl(struct sock *sk, int cmd, unsigned long arg)
 
 	case SIOCINQ:
 	{
-		unsigned int amount = first_packet_length(sk);
+		int amount = max(0, first_packet_length(sk));
 
 		return put_user(amount, (int __user *)arg);
 	}

@@ -2184,7 +2184,7 @@ unsigned int udp_poll(struct file *file, struct socket *sock, poll_table *wait)
 
 	/* Check for false positives due to checksum errors */
 	if ((mask & POLLRDNORM) && !(file->f_flags & O_NONBLOCK) &&
-	    !(sk->sk_shutdown & RCV_SHUTDOWN) && !first_packet_length(sk))
+	    !(sk->sk_shutdown & RCV_SHUTDOWN) && first_packet_length(sk) == -1)
 		mask &= ~(POLLIN | POLLRDNORM);
 
 	return mask;

On 08/23/2016 12:03 PM, Eric Dumazet wrote:

Fixes the test for me. You're welcome to take this as a Tested-by.

Thanks,
Laura

On Tue, 2016-08-23 at 13:06 -0700, Laura Abbott wrote:

Fixes the test for me. You're welcome to take this as a Tested-by.

Thanks Laura, I will submit an official patch immediately.

From: Eric Dumazet <edumazet@google.com>

Laura tracked poll() [and friends] regression caused by commit
e6afc8ace6dd ("udp: remove headers from UDP packets before queueing")

udp_poll() needs to know if there is a valid packet in receive queue,
even if its payload length is 0.

Change first_packet_length() to return an signed int, and use -1
as the indication of an empty queue.

Fixes: e6afc8ace6dd ("udp: remove headers from UDP packets before queueing")
Reported-by: Laura Abbott <redacted>
Signed-off-by: Eric Dumazet <edumazet@google.com>
Tested-by: Laura Abbott <redacted>
---
 net/ipv4/udp.c |   10 +++++-----
 1 file changed, 5 insertions(+), 5 deletions(-)

diff --git a/net/ipv4/udp.c b/net/ipv4/udp.c
index e61f7cd65d08..2a2ac9e0c985 100644
--- a/net/ipv4/udp.c
+++ b/net/ipv4/udp.c

@@ -1184,11 +1184,11 @@ out:
  *	Drops all bad checksum frames, until a valid one is found.
  *	Returns the length of found skb, or 0 if none is found.
  */
-static unsigned int first_packet_length(struct sock *sk)
+static int first_packet_length(struct sock *sk)
 {
 	struct sk_buff_head list_kill, *rcvq = &sk->sk_receive_queue;
 	struct sk_buff *skb;
-	unsigned int res;
+	int res;
 
 	__skb_queue_head_init(&list_kill);
 

@@ -1203,7 +1203,7 @@ static unsigned int first_packet_length(struct sock *sk)
 		__skb_unlink(skb, rcvq);
 		__skb_queue_tail(&list_kill, skb);
 	}
-	res = skb ? skb->len : 0;
+	res = skb ? skb->len : -1;
 	spin_unlock_bh(&rcvq->lock);
 
 	if (!skb_queue_empty(&list_kill)) {

@@ -1232,7 +1232,7 @@ int udp_ioctl(struct sock *sk, int cmd, unsigned long arg)
 
 	case SIOCINQ:
 	{
-		unsigned int amount = first_packet_length(sk);
+		int amount = max_t(int, 0, first_packet_length(sk));
 
 		return put_user(amount, (int __user *)arg);
 	}

@@ -2184,7 +2184,7 @@ unsigned int udp_poll(struct file *file, struct socket *sock, poll_table *wait)
 
 	/* Check for false positives due to checksum errors */
 	if ((mask & POLLRDNORM) && !(file->f_flags & O_NONBLOCK) &&
-	    !(sk->sk_shutdown & RCV_SHUTDOWN) && !first_packet_length(sk))
+	    !(sk->sk_shutdown & RCV_SHUTDOWN) && first_packet_length(sk) == -1)
 		mask &= ~(POLLIN | POLLRDNORM);
 
 	return mask;

On Tue, 2016-08-23 at 13:53 -0700, Eric Dumazet wrote:

From: Eric Dumazet <edumazet@google.com>

...

I'll send a V2, updating the comment to :

Returns the length of found skb, or -1 if none is found.

(Thanks Willem for noticing this)

From: Eric Dumazet <edumazet@google.com>

Laura tracked poll() [and friends] regression caused by commit
e6afc8ace6dd ("udp: remove headers from UDP packets before queueing")

udp_poll() needs to know if there is a valid packet in receive queue,
even if its payload length is 0.

Change first_packet_length() to return an signed int, and use -1
as the indication of an empty queue.

Fixes: e6afc8ace6dd ("udp: remove headers from UDP packets before queueing")
Reported-by: Laura Abbott <redacted>
Signed-off-by: Eric Dumazet <edumazet@google.com>
Tested-by: Laura Abbott <redacted>
---
v2: fix the comment/doc (Willem)

 net/ipv4/udp.c |   12 ++++++------
 1 file changed, 6 insertions(+), 6 deletions(-)

diff --git a/net/ipv4/udp.c b/net/ipv4/udp.c
index e61f7cd65d08..00d18c57c83c 100644
--- a/net/ipv4/udp.c
+++ b/net/ipv4/udp.c

@@ -1182,13 +1182,13 @@ out:
  *	@sk: socket
  *
  *	Drops all bad checksum frames, until a valid one is found.
- *	Returns the length of found skb, or 0 if none is found.
+ *	Returns the length of found skb, or -1 if none is found.
  */
-static unsigned int first_packet_length(struct sock *sk)
+static int first_packet_length(struct sock *sk)
 {
 	struct sk_buff_head list_kill, *rcvq = &sk->sk_receive_queue;
 	struct sk_buff *skb;
-	unsigned int res;
+	int res;
 
 	__skb_queue_head_init(&list_kill);
 

@@ -1203,7 +1203,7 @@ static unsigned int first_packet_length(struct sock *sk)
 		__skb_unlink(skb, rcvq);
 		__skb_queue_tail(&list_kill, skb);
 	}
-	res = skb ? skb->len : 0;
+	res = skb ? skb->len : -1;
 	spin_unlock_bh(&rcvq->lock);
 
 	if (!skb_queue_empty(&list_kill)) {

@@ -1232,7 +1232,7 @@ int udp_ioctl(struct sock *sk, int cmd, unsigned long arg)
 
 	case SIOCINQ:
 	{
-		unsigned int amount = first_packet_length(sk);
+		int amount = max_t(int, 0, first_packet_length(sk));
 
 		return put_user(amount, (int __user *)arg);
 	}

@@ -2184,7 +2184,7 @@ unsigned int udp_poll(struct file *file, struct socket *sock, poll_table *wait)
 
 	/* Check for false positives due to checksum errors */
 	if ((mask & POLLRDNORM) && !(file->f_flags & O_NONBLOCK) &&
-	    !(sk->sk_shutdown & RCV_SHUTDOWN) && !first_packet_length(sk))
+	    !(sk->sk_shutdown & RCV_SHUTDOWN) && first_packet_length(sk) == -1)
 		mask &= ~(POLLIN | POLLRDNORM);
 
 	return mask;

From: Eric Dumazet <redacted>
Date: Tue, 23 Aug 2016 13:59:33 -0700

From: Eric Dumazet <edumazet@google.com>

Laura tracked poll() [and friends] regression caused by commit
e6afc8ace6dd ("udp: remove headers from UDP packets before queueing")

udp_poll() needs to know if there is a valid packet in receive queue,
even if its payload length is 0.

Change first_packet_length() to return an signed int, and use -1
as the indication of an empty queue.

Fixes: e6afc8ace6dd ("udp: remove headers from UDP packets before queueing")
Reported-by: Laura Abbott <redacted>
Signed-off-by: Eric Dumazet <edumazet@google.com>
Tested-by: Laura Abbott <redacted>
---
v2: fix the comment/doc (Willem)

Applied and queued up for -stable, thanks.

When select wakes up on a UDP socket, user is expecting to get data. Getting 
0 from recvfrom() or whatever read function she uses, is a wrong attitude.
I agree with David.

The unit test that expects select to wake up is wrong and should be changed.

-----Original Message----- 
From: David Miller
Sent: Tuesday, August 23, 2016 9:25 PM
To: labbott@redhat.com
Cc: kuznet@ms2.inr.ac.ru ; jmorris@namei.org ; yoshfuji@linux-ipv6.org ; 
kaber@trash.net ; samanthakumar@google.com ; willemb@google.com ; 
netdev@vger.kernel.org ; linux-kernel@vger.kernel.org
Subject: Re: [REGRESSION] Select hang with zero sized UDP packets

From: Laura Abbott <redacted>
Date: Tue, 23 Aug 2016 10:53:26 -0700

Fedora received a report[1] of a unit test failing on Ruby when using
the
4.7 kernel. This was a test to send a zero sized UDP packet. With the
4.7 kernel, the test now timing out on a select instead of completing.
The reduced ruby test is

  def test_udp_recvfrom_nonblock
    u1 = UDPSocket.new
    u2 = UDPSocket.new
    u1.bind("127.0.0.1", 0)
    u2.send("", 0, u1.getsockname)
    IO.select [u1]  # test gets stuck here
  ensure
    u1.close if u1
    u2.close if u2
  end

Well, if there is no data, should select really wake up?

I think it's valid not to. 

On Wed, 2016-08-24 at 11:22 +0300, Dan Akunis wrote:

When select wakes up on a UDP socket, user is expecting to get data. Getting 
0 from recvfrom() or whatever read function she uses, is a wrong attitude.
I agree with David.

The unit test that expects select to wake up is wrong and should be changed.

Please do not top post on netdev mailing list.

Program is fine and wont be changed to work around a kernel bug.

We definitely can send and receive UDP messages with 0 payload.

So select() should unblock when one such frame is received, otherwise
you could fill up the receive queue with a lot of frames like that and
when SO_RCVBUF limit is reached, block future messages.

UDP is a datagram protocol.

Bug fix is merged :
https://git.kernel.org/cgit/linux/kernel/git/davem/net.git/commit/?id=e83c6744e81abc93a20d0eb3b7f504a176a6126a

On Wed, 24 Aug 2016 11:22:09 +0300
"Dan Akunis" [off-list ref] wrote:

When select wakes up on a UDP socket, user is expecting to get data. Getting 
0 from recvfrom() or whatever read function she uses, is a wrong attitude.
I agree with David.

The unit test that expects select to wake up is wrong and should be changed.

The unit test is correct.

The behaviour of a 0 byte frame is actually well established and a 0 byte
data frame is a meaningful message is several protocols. It is distinct
from no pending data because that would report EWOULDBLOCK. It's more fun
with regard to EOF but UDP has no EOF semantics. If you want to
understand how to handle zero length datagrams in a connection oriented
protocol the old DECnet documentation covers it in all its pain 8)

Alan