Target net-next

Also please have the subject line summarize the functional change
(only). Something like "tun: replace tail drop with queue pause
when full."

Simon Schippers wrote:

This patch is a result of our paper [1] and deals with the tun_net_xmit
function which drops SKB's with the reason SKB_DROP_REASON_FULL_RING
whenever the tx_ring (TUN queue) is full. This behavior results in reduced
TCP performance and packet loss for VPNs and VMs. In addition this patch
also allows qdiscs to work properly (see [2]) and to reduce buffer bloat
when reducing the TUN queue.

TUN benchmarks:
+-----------------------------------------------------------------+
| Lab setup of our paper [1]:                                     |
| TCP throughput of VPN solutions at varying RTT (values in Mbps) |
+-----------+---------------+---------------+----------+----------+
| RTT [ms]  | wireguard-go  | wireguard-go  | OpenVPN  | OpenVPN  |
|           |               | patched       |          | patched  |
+-----------+---------------+---------------+----------+----------+
| 10        | 787.3         | 679.0         | 402.4    | 416.9    |
+-----------+---------------+---------------+----------+----------+
| 20        | 765.1         | 718.8         | 401.6    | 393.18   |
+-----------+---------------+---------------+----------+----------+
| 40        | 441.5         | 529.4         | 96.9     | 411.8    |
+-----------+---------------+---------------+----------+----------+
| 80        | 218.7         | 265.7         | 57.9     | 262.7    |
+-----------+---------------+---------------+----------+----------+
| 120       | 145.4         | 181.7         | 52.8     | 178.0    |
+-----------+---------------+---------------+----------+----------+

+--------------------------------------------------------------------+
| Real-world setup of our paper [1]:                                 |
| TCP throughput of VPN solutions without and with the patch         |
| at a RTT of ~120 ms (values in Mbps)                               |
+------------------+--------------+--------------+---------+---------+
| TUN queue        | wireguard-go | wireguard-go | OpenVPN | OpenVPN |
| length [packets] |              | patched      |         | patched |
+------------------+--------------+--------------+---------+---------+
| 5000             | 185.8        | 185.6        | 184.7   | 184.8   |
+------------------+--------------+--------------+---------+---------+
| 1000             | 185.1        | 184.9        | 177.1   | 183.0   |
+------------------+--------------+--------------+---------+---------+
| 500 (default)    | 137.5        | 184.9        | 117.4   | 184.6   |
+------------------+--------------+--------------+---------+---------+
| 100              | 99.8         | 185.3        | 66.4    | 183.5   |
+------------------+--------------+--------------+---------+---------+
| 50               | 59.4         | 185.7        | 21.6    | 184.7   |
+------------------+--------------+--------------+---------+---------+
| 10               | 1.7          | 185.4        | 1.6     | 183.6   |
+------------------+--------------+--------------+---------+---------+

TAP benchmarks:
+------------------------------------------------------------------+
| Lab Setup [3]:                                                   |
| TCP throughput from host to Debian VM using TAP (values in Mbps) |
+----------------------------+------------------+------------------+
| TUN queue                  | Default          | Patched          |
| length [packets]           |                  |                  |
+----------------------------+------------------+------------------+
| 1000 (default)             | 2194.3           | 2185.0           |
+----------------------------+------------------+------------------+
| 100                        | 1986.4           | 2268.5           |
+----------------------------+------------------+------------------+
| 10                         | 625.0            | 1988.9           |
+----------------------------+------------------+------------------+
| 1                          | 2.2              | 1112.7           |
+----------------------------+------------------+------------------+
|                                                                  |
+------------------------------------------------------------------+
| Measurement with 1000 packets queue and emulated delay           |
+----------------------------+------------------+------------------+
| RTT [ms]                   | Default          | Patched          |
+----------------------------+------------------+------------------+
| 60                         | 171.8            | 341.2            |
+----------------------------+------------------+------------------+
| 120                        | 98.3             | 255.0            |
+----------------------------+------------------+------------------+

TAP+vhost_net benchmarks:
+----------------------------------------------------------------------+
| Lab Setup [3]:                                                       |
| TCP throughput from host to Debian VM using TAP+vhost_net            |
| (values in Mbps)                                                     |
+-----------------------------+--------------------+-------------------+
| TUN queue                   | Default            | Patched           |
| length [packets]            |                    |                   |
+-----------------------------+--------------------+-------------------+
| 1000 (default)              | 23403.9            | 23858.8           |
+-----------------------------+--------------------+-------------------+
| 100                         | 23372.5            | 23889.9           |
+-----------------------------+--------------------+-------------------+
| 10                          | 25837.5            | 23730.2           |
+-----------------------------+--------------------+-------------------+
| 1                           | 0.7                | 19244.8           |
+-----------------------------+--------------------+-------------------+
| Note: Default suffers from many retransmits, while patched does not. |
+----------------------------------------------------------------------+
|                                                                      |
+----------------------------------------------------------------------+
| Measurement with 1000 packets queue and emulated delay               |
+-----------------------------+--------------------+-------------------+
| RTT [ms]                    | Default            | Patched           |
+-----------------------------+--------------------+-------------------+
| 60                          | 397.1              | 397.8             |
+-----------------------------+--------------------+-------------------+
| 120                         | 200.7              | 199.9             |
+-----------------------------+--------------------+-------------------+

Implementation details:
- The netdev queue start/stop flow control is utilized.
- Compatible with multi-queue by only stopping/waking the specific
netdevice subqueue.

In the tun_net_xmit function:
- Stopping the subqueue is done when the tx_ring gets full after inserting
the SKB into the tx_ring.
- In the unlikely case when the insertion with ptr_ring_produce fails, the
old dropping behavior is used for this SKB.

In the tun_ring_recv function:
- Waking the subqueue is done after consuming a SKB from the tx_ring when
the tx_ring is empty.
- When the tx_ring is configured to be small (for example to hold 1 SKB),

That's an exaggerated case that hopefully we do not have to support.
Can this be configured? Maybe we should round_up user input to a sane
lower bound instead.

queuing might be stopped in the tun_net_xmit function while at the same
time, ptr_ring_consume is not able to grab a SKB. This prevents
tun_net_xmit from being called again and causes tun_ring_recv to wait
indefinitely for a SKB in the blocking wait queue. Therefore, the netdev
queue is woken in the wait queue.

In the tap_do_read function:
- Same behavior as in tun_ring_recv: Waking the subqueue when the tx_ring
is empty & waking the subqueue in the blocking wait queue.
- Here the netdev txq is obtained with a rcu read lock instead.

In the vhost_net_buf_produce function:
- Same behavior as in tun_ring_recv: Waking the subqueue when the tx_ring
is empty.
- Here the netdev_queue is saved in the vhost_net_virtqueue at init with
new helpers.

We are open to suggestions regarding the implementation :)
Thank you for your work!

Similarly, in the commit message, lead with the technical explanation.
Brief benchmark results are great, but this is not an academic paper.
Best concise and below the main take-away. Or in the cover letter if a
multi patch series. ..

[1] Link:
https://cni.etit.tu-dortmund.de/storages/cni-etit/r/Research/Publications/2025/Gebauer_2025_VTCFall/Gebauer_VTCFall2025_AuthorsVersion.pdf
[2] Link:
https://unix.stackexchange.com/questions/762935/traffic-shaping-ineffective-on-tun-device
[3] Link: https://github.com/tudo-cni/nodrop

Co-developed-by: Tim Gebauer <redacted>
Signed-off-by: Tim Gebauer <redacted>
Signed-off-by: Simon Schippers <redacted>
---
V2 -> V3: Added support for TAP and TAP+vhost_net.

.. please split into a series, with separate patches for TUN, TAP and
vhost-net.

Or, start with one and once that is merged after revisions, repeat
for the others. That is likely less work.

This wakes the queue only once entirely empty? That seems aggressive.

Where is the matching netif_tx_stop_queue. I had expected that
arund the ptr_ring_produce calls in tap_handle_frame.

 		/* Nothing to read, let's sleep */
 		schedule();
 	}
 	if (!noblock)
 		finish_wait(sk_sleep(&q->sk), &wait);
 
+	if (ptr_ring_empty(&q->ring)) {
+		rcu_read_lock();
+		dev = rcu_dereference(q->tap)->dev;
+		txq = netdev_get_tx_queue(dev, q->queue_index);
+		netif_tx_wake_queue(txq);
+		rcu_read_unlock();
+	}
+

Why the second test for the same condition: ring empty?

If the dev is only safe to be accessed inside an RCU readside critical
section, is it safe to use txq outside of it?

again, stop the queue before dropping is needed. Which is what the
new ptr_ring_full code below does I guess. If so, when is this reached?