From: Piotr Sawuk <hidden> Date: 2012-07-14 07:56:31
On Sa, 14.07.2012, 03:31, valdis.kletnieks@vt.edu wrote:
On Fri, 13 Jul 2012 16:55:44 -0700, Stephen Hemminger said:
quoted
quoted
+ /* Course retransmit inefficiency- this packet has been received
twice. */
quoted
+ tp->dup_pkts_recv++;
I don't understand that comment, could you use a better sentence please?
I think what was intended was:
/* Curse you, retransmit inefficiency! This packet has been received at
least twice */
LOL, no. I think "course retransmit" is short for "course-grained timeout
caused retransmit" but I can't be sure since I'm not the author of these
lines. I'll replace that comment with the non-shorthand version though.
however, I think the real comment here should be:
/*A perceived shortcoming of the standard TCP implementation: A
TCP receiver can get duplicate packets from the sender because it cannot
acknowledge packets that arrive out of order. These duplicates would happen
when the sender mistakenly thinks some packets have been lost by the network
because it does not receive acks for them but in reality they were
successfully received out of order. Since the receiver has no way of letting
the sender know about the receipt of these packets, they could potentially
be re-sent and re-received at the receiver. Not only do duplicate packets
waste precious Internet bandwidth but they hurt performance because the
sender mistakenly detects congestion from packet losses. The SACK TCP
extension speci
cally addresses this issue. A large number of duplicate
packets received would indicate a signi
cant bene
t to the wide adoption of
SACK. The duplicatepacketsreceived metric is computed at the
receiver and counts these packets on a per-connection basis.*/
as copied from his thesis at [1]. also in the thesis he writes:
In our limited experiment, the results indicated no duplicate packets were
received on any connection in the 18 hour run. This leads us to several
conclusions. Since duplicate ACKs were seen on many connections we know that
some packets were lost or reordered, but unACKed reordered packets never
caused a /coursegrainedtimeouts/ on our connections. Only these timeouts
will cause duplicate packets to be received since less severe out-of-order
conditions will be resolved with fast retransmits. The lack of course
timeouts
may be due to the quality of UCSD's ActiveWeb network or the paucity of
large gaps between received packet groups. It should be noted that Linux 2.2
implements fast retransmits for up to two packet gaps, thus reducing the
need for course grained timeouts due to the lack of SACK.
[1] https://sacerdoti.org/tcphealth/tcphealth-paper.pdf
From: Eric Dumazet <hidden> Date: 2012-07-14 08:27:12
On Sat, 2012-07-14 at 09:56 +0200, Piotr Sawuk wrote:
On Sa, 14.07.2012, 03:31, valdis.kletnieks@vt.edu wrote:
quoted
On Fri, 13 Jul 2012 16:55:44 -0700, Stephen Hemminger said:
quoted
quoted
+ /* Course retransmit inefficiency- this packet has been received
twice. */
quoted
+ tp->dup_pkts_recv++;
I don't understand that comment, could you use a better sentence please?
I think what was intended was:
/* Curse you, retransmit inefficiency! This packet has been received at
least twice */
quoted
LOL, no. I think "course retransmit" is short for "course-grained timeout
caused retransmit" but I can't be sure since I'm not the author of these
lines. I'll replace that comment with the non-shorthand version though.
however, I think the real comment here should be:
/*A perceived shortcoming of the standard TCP implementation: A
TCP receiver can get duplicate packets from the sender because it cannot
acknowledge packets that arrive out of order. These duplicates would happen
when the sender mistakenly thinks some packets have been lost by the network
because it does not receive acks for them but in reality they were
successfully received out of order. Since the receiver has no way of letting
the sender know about the receipt of these packets, they could potentially
be re-sent and re-received at the receiver. Not only do duplicate packets
waste precious Internet bandwidth but they hurt performance because the
sender mistakenly detects congestion from packet losses. The SACK TCP
extension speci
cally addresses this issue. A large number of duplicate
packets received would indicate a signi
cant bene
t to the wide adoption of
SACK. The duplicatepacketsreceived metric is computed at the
receiver and counts these packets on a per-connection basis.*/
as copied from his thesis at [1]. also in the thesis he writes:
In our limited experiment, the results indicated no duplicate packets were
received on any connection in the 18 hour run. This leads us to several
conclusions. Since duplicate ACKs were seen on many connections we know that
some packets were lost or reordered, but unACKed reordered packets never
caused a /coursegrainedtimeouts/ on our connections. Only these timeouts
will cause duplicate packets to be received since less severe out-of-order
conditions will be resolved with fast retransmits. The lack of course
timeouts
may be due to the quality of UCSD's ActiveWeb network or the paucity of
large gaps between received packet groups. It should be noted that Linux 2.2
implements fast retransmits for up to two packet gaps, thus reducing the
need for course grained timeouts due to the lack of SACK.
[1] https://sacerdoti.org/tcphealth/tcphealth-paper.pdf
Not sure how pertinent is this paper today in 2012
I would prefer you add global counters, instead of per tcp counters that
most applications wont use at all.
Example of a more useful patch : add a counter of packets queued in Out
Of Order queue ( in tcp_data_queue_ofo() )
"netstat -s" will display the total count, without any changes in
userland tools/applications.
From: Ben Hutchings <hidden> Date: 2012-07-16 13:32:44
On Sat, 2012-07-14 at 09:56 +0200, Piotr Sawuk wrote:
On Sa, 14.07.2012, 03:31, valdis.kletnieks@vt.edu wrote:
quoted
On Fri, 13 Jul 2012 16:55:44 -0700, Stephen Hemminger said:
quoted
quoted
+ /* Course retransmit inefficiency- this packet has been received
twice. */
quoted
+ tp->dup_pkts_recv++;
I don't understand that comment, could you use a better sentence please?
I think what was intended was:
/* Curse you, retransmit inefficiency! This packet has been received at
least twice */
quoted
LOL, no. I think "course retransmit" is short for "course-grained timeout
caused retransmit" but I can't be sure since I'm not the author of these
lines. I'll replace that comment with the non-shorthand version though.
however, I think the real comment here should be:
[...]
The word you are looking for is 'coarse' not 'course' (they are
generally pronounced the same, to confuse you).
Ben.
--
Ben Hutchings, Staff Engineer, Solarflare
Not speaking for my employer; that's the marketing department's job.
They asked us to note that Solarflare product names are trademarked.
From: Piotr Sawuk <hidden> Date: 2012-07-16 15:12:32
On Mo, 16.07.2012, 15:32, Ben Hutchings wrote:
On Sat, 2012-07-14 at 09:56 +0200, Piotr Sawuk wrote:
quoted
On Sa, 14.07.2012, 03:31, valdis.kletnieks@vt.edu wrote:
quoted
On Fri, 13 Jul 2012 16:55:44 -0700, Stephen Hemminger said:
quoted
quoted
+ /* Course retransmit inefficiency- this packet has been received
twice. */
quoted
+ tp->dup_pkts_recv++;
I don't understand that comment, could you use a better sentence
please?
quoted
I think what was intended was:
/* Curse you, retransmit inefficiency! This packet has been received at
least twice */
quoted
LOL, no. I think "course retransmit" is short for "course-grained timeout
caused retransmit" but I can't be sure since I'm not the author of these
lines. I'll replace that comment with the non-shorthand version though.
however, I think the real comment here should be:
[...]
The word you are looking for is 'coarse' not 'course' (they are
generally pronounced the same, to confuse you).
that was my first thought too.
but then I noticed the word "course" in the kernel's comments.
judging by context it describes the events of a round-trip.
so I guess Course-Grained means RTT-grained.
especially since this misspelling was consistent in the author's paper.
anyway, new patch, made some mistakes in my previous version.
also I added the jiffies_to_msecs noone dared to mention
comments and suggestions as always welcome:
diff -rub A/include/linux/tcp.h B/include/linux/tcp.h
@@ -492,6 +492,17 @@*containsrelatedtcp_cookie_transactionsfields.*/structtcp_cookie_values*cookie_values;++#ifdef CONFIG_TCPHEALTH+/*+*TCPhealthmonitoringcounters.+*/+__u32dup_acks_sent;+__u32dup_pkts_recv;+__u32acks_sent;+__u32pkts_recv;+__u32last_ack_sent;/* Sequence number of the last ack we sent. */+#endif};staticinlinestructtcp_sock*tcp_sk(conststructsock*sk)
@@ -4824,6 +4829,12 @@return;}+#ifdef CONFIG_TCPHEALTH+/* A packet is a "duplicate" if it contains bytes we have already
received. */
+ if (before(TCP_SKB_CB(skb)->seq, tp->rcv_nxt))
+ tp->dup_pkts_recv++;
+#endif
+
if (!after(TCP_SKB_CB(skb)->end_seq, tp->rcv_nxt)) {
/* A retransmit, 2nd most common case. Force an immediate ack. */
NET_INC_STATS_BH(sock_net(sk), LINUX_MIB_DELAYEDACKLOST);
@@ -5535,6 +5546,12 @@ tp->rx_opt.saw_tstamp = 0;+#ifdef CONFIG_TCPHEALTH+ /*+ * total per-connection packet arrivals.+ */+ tp->pkts_recv++;+#endif /* pred_flags is 0xS?10 << 16 + snd_wnd * if header_prediction is to be made * 'S' will always be tp->tcp_header_len >> 2
@@ -2772,8 +2772,19 @@skb_reserve(buff,MAX_TCP_HEADER);tcp_init_nondata_skb(buff,tcp_acceptable_seq(sk),TCPHDR_ACK);+#ifdef CONFIG_TCPHEALTH+/* If the rcv_nxt has not advanced since sending our last ACK, this is a
duplicate. */
+ if (tcp_sk(sk)->rcv_nxt == tcp_sk(sk)->last_ack_sent)
+ tcp_sk(sk)->dup_acks_sent++;
+ /* Record the total number of acks sent on this connection. */
+ tcp_sk(sk)->acks_sent++;
+#endif
+
/* Send it off, this clears delayed acks for us. */
TCP_SKB_CB(buff)->when = tcp_time_stamp;
+#ifdef CONFIG_TCPHEALTH
+ tcp_sk(sk)->last_ack_sent = tcp_sk(sk)->rcv_nxt;
+#endif
tcp_transmit_skb(sk, buff, 0, GFP_ATOMIC);
}
This still doesn't gives you the correct RTT.
srtt is in jiffies * 8.
You should do jiffies_to_msecs(tp->srtt) >> 3.
The RTT is already exposed by tcp_info anyway... (see tcp_get_info() - where
you also see the bitshift)
Christoph
--
IP Networking Lab --- http://inl.info.ucl.ac.be
MultiPath TCP in the Linux Kernel --- http://mptcp.info.ucl.ac.be
Université Catholique de Louvain
--
From: Piotr Sawuk <hidden> Date: 2012-07-19 10:37:27
On Mo, 16.07.2012, 17:24, Christoph Paasch wrote:
You should do jiffies_to_msecs(tp->srtt) >> 3.
The RTT is already exposed by tcp_info anyway... (see tcp_get_info() - where
you also see the bitshift)
thanks a lot. rtt is output for completion's sake, it helps in diagnosis.
here my hopefully final version. it comes with tcp_info interface too:
diff -rub A/include/linux/tcp.h B/include/linux/tcp.h
@@ -183,6 +183,12 @@__u32tcpi_rcv_space;__u32tcpi_total_retrans;++/* TCP Health */+__u32tcpi_dup_acks;+__u32tcpi_dup_pkts;+__u32tcpi_acks;+__u32tcpi_pkts;};/* for TCP_MD5SIG socket option */
@@ -492,6 +498,17 @@*containsrelatedtcp_cookie_transactionsfields.*/structtcp_cookie_values*cookie_values;++#ifdef CONFIG_TCPHEALTH+/*+*TCPhealthmonitoringcounters.+*/+__u32dup_acks_sent;+__u32dup_pkts_recv;+__u32acks_sent;+__u32pkts_recv;+__u32last_ack_sent;/* Sequence number of the last ack we sent. */+#endif};staticinlinestructtcp_sock*tcp_sk(conststructsock*sk)
@@ -4824,6 +4829,12 @@return;}+#ifdef CONFIG_TCPHEALTH+/* A packet is a "duplicate" if it contains bytes we have already
received. */
+ if (before(TCP_SKB_CB(skb)->seq, tp->rcv_nxt))
+ tp->dup_pkts_recv++;
+#endif
+
if (!after(TCP_SKB_CB(skb)->end_seq, tp->rcv_nxt)) {
/* A retransmit, 2nd most common case. Force an immediate ack. */
NET_INC_STATS_BH(sock_net(sk), LINUX_MIB_DELAYEDACKLOST);
@@ -5535,6 +5546,12 @@ tp->rx_opt.saw_tstamp = 0;+#ifdef CONFIG_TCPHEALTH+ /*+ * total per-connection packet arrivals.+ */+ tp->pkts_recv++;+#endif /* pred_flags is 0xS?10 << 16 + snd_wnd * if header_prediction is to be made * 'S' will always be tp->tcp_header_len >> 2
@@ -2772,8 +2772,19 @@skb_reserve(buff,MAX_TCP_HEADER);tcp_init_nondata_skb(buff,tcp_acceptable_seq(sk),TCPHDR_ACK);+#ifdef CONFIG_TCPHEALTH+/* If the rcv_nxt has not advanced since sending our last ACK, this is a
duplicate. */
+ if (tcp_sk(sk)->rcv_nxt == tcp_sk(sk)->last_ack_sent)
+ tcp_sk(sk)->dup_acks_sent++;
+ /* Record the total number of acks sent on this connection. */
+ tcp_sk(sk)->acks_sent++;
+#endif
+
/* Send it off, this clears delayed acks for us. */
TCP_SKB_CB(buff)->when = tcp_time_stamp;
+#ifdef CONFIG_TCPHEALTH
+ tcp_sk(sk)->last_ack_sent = tcp_sk(sk)->rcv_nxt;
+#endif
tcp_transmit_skb(sk, buff, 0, GFP_ATOMIC);
}