Restricting payload size in do_tcp_sendpages.

Hi,

I would like to perform some experiments where I want TCP to send
packets of size less than the MSS even more than an MSS worth of data
is available in the TCP buffers.  I am performing these experiments to
submit a patch that can avoid synchronization of TCP flows. I have
modified the following code in the function do_tcp_sendpages in the
file net/ipv4/tcp.c.  For testing purposes I have currently set the
mss_now to half of the value that would have been typically used. I
have even updated the size_goal which controls the maximum size that
can be sent when offloading is enabled. To avoid coalescing smaller
skbs I have even set the can_coalesce to 0. Despite these changes the
frames sent over the Ethernet have the length of 1514 bytes. I would
like to know which function is responsible for coalescing these
packets despite forcing a small segment size.

*** linux-3.2.6/net/ipv4/tcp.c	

--- linux-3.2.6-modified/net/ipv4/tcp.c

*************** static ssize_t do_tcp_sendpages(struct s
*** 774,779 ****

--- 774,783 ----
  	clear_bit(SOCK_ASYNC_NOSPACE, &sk->sk_socket->flags);

  	mss_now = tcp_send_mss(sk, &size_goal, flags);
+     /* hack begin */
+     mss_now = mss_now >> 1;
+     size_goal = mss_now;
+     /* hack end */
  	copied = 0;

  	err = -EPIPE;

*************** new_segment:
*** 805,810 ****

--- 809,817 ----

  		i = skb_shinfo(skb)->nr_frags;
  		can_coalesce = skb_can_coalesce(skb, i, page, offset);
+             /* hack begin */
+             can_coalesce = 0;
+             /* hack end */
  		if (!can_coalesce && i >= MAX_SKB_FRAGS) {
  			tcp_mark_push(tp, skb);
  			goto new_segment;

*************** wait_for_memory:
*** 857,862 ****

--- 864,873 ----
  			goto do_error;

  		mss_now = tcp_send_mss(sk, &size_goal, flags);
+             /* hack begin */
+             mss_now = mss_now >> 1;
+             size_goal = mss_now;
+             /* hack end */
  	}

Regards,
Ashwin

Le jeudi 01 mars 2012 à 16:16 +0100, Ashwin Rao a écrit :

You dont need to hack kernel

man 7 tcp

       TCP_MAXSEG
              The  maximum  segment  size  for  outgoing TCP packets.  If this
              option is set before connection establishment, it  also  changes
              the  MSS value announced to the other end in the initial packet.
              Values greater than the (eventual) interface MTU have no effect.
              TCP  will  also  impose  its minimum and maximum bounds over the
              value provided.

On Thu, Mar 1, 2012 at 4:58 PM, Eric Dumazet [off-list ref] wrote:

Le jeudi 01 mars 2012 à 16:16 +0100, Ashwin Rao a écrit :

quoted

Hi,

I would like to perform some experiments where I want TCP to send
packets of size less than the MSS even more than an MSS worth of data
is available in the TCP buffers.  I am performing these experiments to
submit a patch that can avoid synchronization of TCP flows. I have
modified the following code in the function do_tcp_sendpages in the
file net/ipv4/tcp.c.  For testing purposes I have currently set the
mss_now to half of the value that would have been typically used. I
have even updated the size_goal which controls the maximum size that
can be sent when offloading is enabled. To avoid coalescing smaller
skbs I have even set the can_coalesce to 0. Despite these changes the
frames sent over the Ethernet have the length of 1514 bytes. I would
like to know which function is responsible for coalescing these
packets despite forcing a small segment size.

*** linux-3.2.6/net/ipv4/tcp.c

--- linux-3.2.6-modified/net/ipv4/tcp.c

*************** static ssize_t do_tcp_sendpages(struct s
*** 774,779 ****

--- 774,783 ----

      clear_bit(SOCK_ASYNC_NOSPACE, &sk->sk_socket->flags);

      mss_now = tcp_send_mss(sk, &size_goal, flags);
+     /* hack begin */
+     mss_now = mss_now >> 1;
+     size_goal = mss_now;
+     /* hack end */
      copied = 0;

      err = -EPIPE;
*************** new_segment:
*** 805,810 ****

--- 809,817 ----

              i = skb_shinfo(skb)->nr_frags;
              can_coalesce = skb_can_coalesce(skb, i, page, offset);
+             /* hack begin */
+             can_coalesce = 0;
+             /* hack end */
              if (!can_coalesce && i >= MAX_SKB_FRAGS) {
                      tcp_mark_push(tp, skb);
                      goto new_segment;
*************** wait_for_memory:
*** 857,862 ****

--- 864,873 ----

                      goto do_error;

              mss_now = tcp_send_mss(sk, &size_goal, flags);
+             /* hack begin */
+             mss_now = mss_now >> 1;
+             size_goal = mss_now;
+             /* hack end */
      }

You dont need to hack kernel

man 7 tcp

      TCP_MAXSEG
             The  maximum  segment  size  for  outgoing TCP packets.  If this
             option is set before connection establishment, it  also  changes
             the  MSS value announced to the other end in the initial packet.
             Values greater than the (eventual) interface MTU have no effect.
             TCP  will  also  impose  its minimum and maximum bounds over the
             value provided.

My objective is to send a train of packets each of which has a
different packet size. The maximum packet size should be restricted by
the MSS however the packet sizes should not be the same. Before
randomizing the packet sizes, I wanted to check if I set the packet
size to a fixed value will it be reflected on the packets being sent
over the wire.

Regards,
Ashwin

From: Ashwin Rao <redacted>
Date: Thu, 1 Mar 2012 16:16:40 +0100

I would like to perform some experiments where I want TCP to send
packets of size less than the MSS even more than an MSS worth of data
is available in the TCP buffers.

I think if you are trying to make experimental changes to TCP you
should be at least skilled enough to implement your ideas fully on
your own rather than having to ask for a crash course on how the stack
works.

Frankly, if you aren't skilled enough to figure these things out on
your own I really don't think you should be even suggesting changes
to a complicated protocol that effects the entire world.

On Thu, Mar 1, 2012 at 10:02 PM, David Miller [off-list ref] wrote:

From: Ashwin Rao <redacted>
Date: Thu, 1 Mar 2012 16:16:40 +0100

quoted

I would like to perform some experiments where I want TCP to send
packets of size less than the MSS even more than an MSS worth of data
is available in the TCP buffers.

I think if you are trying to make experimental changes to TCP you
should be at least skilled enough to implement your ideas fully on
your own rather than having to ask for a crash course on how the stack
works.

I am not asking for a crash course on how the stack works. I am trying
to figure out what is wrong with my understanding of the code. I admit
I am not as smart as you guys but then the reason for doing
experiments is because I observed a problem based on the measurement
study that I performed on clusters and home networks. I then worked on
developing a small model to capture the behavior I observed. The
problem I observed has its roots in the synchronization of TCP flows.
This can be addressed by using queuing disciplines like RED but then
it is still a problem in networks that will use off the shelf devices
that use drop-tail queues. The numerical resolutions from the model
predict the behavior that I observed. I am now looking a possible
solutions that can address this issue and I am performing experiments
to see whether my proposed solutions are practically useful or not.
This is because the model cannot capture everything.  What I want to
do is to ensure that the packets arriving in the network do not have
the same size or at least the last packet in a window of packets has a
size smaller than the MSS. This according to the model is enough to
break the synchronization that I observed in the clusters and home
networks. To test this I forcefully set the segment size (and size
goal which controls the size in case of offloading) in the function
do_tcp_sendpages. According to my understanding this is the last point
before which the tcp packet is created before being sent to the lower
layer.

Frankly, if you aren't skilled enough to figure these things out on
your own I really don't think you should be even suggesting changes
to a complicated protocol that effects the entire world.

I may not be skilled enough like you guys but then I am not afraid to
admit I do not know the entire picture. But if there is a problem that
I see and a solution in which I believe in then I will not feel shy of
validating my solution.

Ashwin