Re: TCP and reordering

On Wed, 2012-11-28 at 09:16 -0800, Eric Dumazet wrote:

On Wed, 2012-11-28 at 16:41 +0000, David Woodhouse wrote:

quoted

Another fun issue with tunnelling protocols and BQL... packets tend to
*grow* as they get encapsulated. So you might end up calling
netdev_sent_queue() with a given size, then netdev_completed_queue()
with a bigger packet later...

Its the driver responsibility to maintain the coherent 'bytes' value for
each transmitted/completed packet.

If a driver calls an external entity, it cannot possibly use BQL, unless
doing an approximation (bytes becomes a fixed value)

BQL was really something to control/limit queueing on ethernet links,
not for stacked devices, as stacked devices normally have no queue.

Oh, of *course*... this is why my kernel would panic if I attempted to
add BQL to BR2684. The ATM low-level driver was pushing a header onto
the skb, and I ended up calling netdev_completed_queue() with a larger
'bytes' value than the one I'd called netdev_sent_queue() with. Which
leads to a BUG(), which immediately results in a panic. A moderately
suboptimal failure mode, when a nasty warning and disabling BQL on this
interface might have been nicer.

However, *perhaps* it isn't so hard to get a consistent 'bytes' value.
This version appears to work... can we use something along these lines
in the general case? What if skb_is_nonlinear()?

This probably doesn't work for L2TP where it's going to be passed down
the whole stack and get a new network header and everything. But for
BR2684 is this at least a salvageable approach?

--- net/atm/br2684.c~	2012-12-01 16:35:49.000000000 +0000
+++ net/atm/br2684.c	2012-12-02 01:18:35.216607088 +0000

@@ -180,6 +180,11 @@ static struct notifier_block atm_dev_not
 	.notifier_call = atm_dev_event,
 };
 
+static unsigned int skb_acct_len(struct sk_buff *skb)
+{
+	return skb_tail_pointer(skb) - skb_network_header(skb);
+}
+
 /* chained vcc->pop function.  Check if we should wake the netif_queue */
 static void br2684_pop(struct atm_vcc *vcc, struct sk_buff *skb)
 {

@@ -191,6 +196,8 @@ static void br2684_pop(struct atm_vcc *v
 	/* If the queue space just went up from zero, wake */
 	if (atomic_inc_return(&brvcc->qspace) == 1)
 		netif_wake_queue(brvcc->device);
+
+	netdev_completed_queue(brvcc->device, 1, skb_acct_len(skb));
 }
 
 /*

@@ -265,6 +272,7 @@ static int br2684_xmit_vcc(struct sk_buf
 			netif_wake_queue(brvcc->device);
 	}
 
+	netdev_sent_queue(brvcc->device, skb_acct_len(skb));
 	/* If this fails immediately, the skb will be freed and br2684_pop()
 	   will wake the queue if appropriate. Just return an error so that
 	   the stats are updated correctly */

@@ -710,6 +718,7 @@ static int br2684_create(void __user *ar
 		return err;
 	}
 
+	netdev_reset_queue(netdev);
 	write_lock_irq(&devs_lock);
 
 	brdev->payload = payload;

-- 
dwmw2