From: Eric Dumazet <redacted>
Date: Thu, 27 Sep 2012 23:17:04 +0200

Yes it seems the problem. On the host I tried :

# ip ro get 8.8.8.8 from 192.168.200.1 iif tap1
8.8.8.8 from 192.168.200.1 via 172.30.42.1 dev eth0 
    cache  iif *

So if the guest tries to send a frame to 8.8.8.8 we are going to forward
the packet to eth0

But if the guest tries to send to 255.255.255.255, we try to deliver the
packet to the host itself, instead of broadcasting to eth0

# ip ro get 255.255.255.255 from 192.168.200.1 iif tap1
broadcast 255.255.255.255 from 192.168.200.1 dev lo 
    cache <local,brd>  iif *

David, maybe you'll have an idea ?

Perhaps this was introduced by:

commit 7bd86cc282a458b66c41e3f6676de6656c99b8db
Author: Yan, Zheng [off-list ref]
Date:   Sun Aug 12 20:09:59 2012 +0000

    ipv4: Cache local output routes
    
    Commit caacf05e5ad1abf causes big drop of UDP loop back performance.
    The cause of the regression is that we do not cache the local output
    routes. Each time we send a datagram from unconnected UDP socket,
    the kernel allocates a dst_entry and adds it to the rt_uncached_list.
    It creates lock contention on the rt_uncached_lock.
    
    Reported-by: Alex Shi [off-list ref]
    Signed-off-by: Yan, Zheng [off-list ref]
    Signed-off-by: David S. Miller [off-list ref]

diff --git a/net/ipv4/route.c b/net/ipv4/route.c
index e4ba974..fd9ecb5 100644
--- a/net/ipv4/route.c
+++ b/net/ipv4/route.c

@@ -2028,7 +2028,6 @@ struct rtable *__ip_route_output_key(struct net *net, struct flowi4 *fl4)
 		}
 		dev_out = net->loopback_dev;
 		fl4->flowi4_oif = dev_out->ifindex;
-		res.fi = NULL;
 		flags |= RTCF_LOCAL;
 		goto make_route;
 	}