Re: [patch] pktgen: bug when calling ndelay in x86 architectures

Le mercredi 19 octobre 2011 à 11:33 +0200, Daniel Turull a écrit :

Hi,
then if we want to use the spin more often.
maybe we can increase the constant from 100000 (0.1ms) to 1000000 (1ms)?
How was the current value chosen?

Based on user needs ;)

I did some measurements of the inter-arrival time between packets
and with bigger values the maximal is reduced in the rates between
2kpps and 20kpps.

ndelay()/udelay() have some inaccuracies, for 'long' values, because of
rounding errors.

If we spin, just call ktime_now() in a loop until spin_until is
reached...

That way you get max possible resolution, given kernel time service
constraints.

Untested patch :

diff --git a/net/core/pktgen.c b/net/core/pktgen.c
index 38d6577..5c7e900 100644
--- a/net/core/pktgen.c
+++ b/net/core/pktgen.c

@@ -2145,9 +2145,11 @@ static void spin(struct pktgen_dev *pkt_dev, ktime_t spin_until)
 	}
 
 	start_time = ktime_now();
-	if (remaining < 100000)
-		ndelay(remaining);	/* really small just spin */
-	else {
+	if (remaining < 100000) {
+		do {
+			end_time = ktime_now();
+		} while (ktime_lt(end_time, spin_until));
+	} else {
 		/* see do_nanosleep */
 		hrtimer_init_sleeper(&t, current);
 		do {

@@ -2162,8 +2164,8 @@ static void spin(struct pktgen_dev *pkt_dev, ktime_t spin_until)
 			hrtimer_cancel(&t.timer);
 		} while (t.task && pkt_dev->running && !signal_pending(current));
 		__set_current_state(TASK_RUNNING);
+		end_time = ktime_now();
 	}
-	end_time = ktime_now();
 
 	pkt_dev->idle_acc += ktime_to_ns(ktime_sub(end_time, start_time));
 	pkt_dev->next_tx = ktime_add_ns(spin_until, pkt_dev->delay);

Hi,

I tested the patch and it works well.


On 10/19/2011 12:13 PM, Eric Dumazet wrote:

Le mercredi 19 octobre 2011 à 11:33 +0200, Daniel Turull a écrit :

quoted

Hi,
then if we want to use the spin more often.
maybe we can increase the constant from 100000 (0.1ms) to 1000000 (1ms)?
How was the current value chosen?

Based on user needs ;)

I think if we increase the constant to 1ms, we will reduce the jitter if we have
a rate between 1kpps and 10 kpps, but I guess is not a big deal.

I've plot this new graph with this patch:
http://tslab.ssvl.kth.se/pktgen/img/inter_eric1.eps

Daniel

Le jeudi 20 octobre 2011 à 15:22 +0200, Daniel Turull a écrit :

Hi,

I tested the patch and it works well.

Thanks !

I think if we increase the constant to 1ms, we will reduce the jitter if we have
a rate between 1kpps and 10 kpps, but I guess is not a big deal.

I've plot this new graph with this patch:
http://tslab.ssvl.kth.se/pktgen/img/inter_eric1.eps

Unfortunately, the sender cpu might be preempted by timer irq or other
expensive irq, so the Min/Max values are not very different I guess.

I dont understand your Min values.

At 100 pps, how is it possible to have a Min value of ~5000 ns ?

On 10/20/2011 03:44 PM, Eric Dumazet wrote:

Le jeudi 20 octobre 2011 à 15:22 +0200, Daniel Turull a écrit :

quoted

Hi,

I tested the patch and it works well.

Thanks !

quoted

I think if we increase the constant to 1ms, we will reduce the jitter if we have
a rate between 1kpps and 10 kpps, but I guess is not a big deal.

quoted

I've plot this new graph with this patch:
http://tslab.ssvl.kth.se/pktgen/img/inter_eric1.eps

Unfortunately, the sender cpu might be preempted by timer irq or other
expensive irq, so the Min/Max values are not very different I guess.

I dont understand your Min values.

At 100 pps, how is it possible to have a Min value of ~5000 ns ?

My assumption is that for low rate, the min value is caused in the 
beginning of the test. When we start the transmission in pktgen_run(),
we set the pkt_dev->next_tx to the current time but the are
more operation to do, so the first transmission is a bit delayed. 
Even more if the cpu is preempted.
For the second packet, we are taking the pkt_dev->next_tx as a reference
and add the delay, in order to decide when to send.
So, my guess is that the first packet is delayed
and then we send the second packet only after a short time, in order
to keep the average rate in the transmission.

Daniel