Disable tcp MSS option in three way handshake?

Hi,

this sounds crazy, we have a weird situation that an unknown tcp
implementation not putting tcp MSS option in the SYN/ACK which caused
us some issue. I am tasked to mimic the unknown tcp immplementation on
not sending MSS in tcp SYN/ACK, I am wondering if I can achieve that
by modifying linux kernel tcp code,  there is socket option
TCP_MAXSEG, but that seems only affecting the size of MSS, not
removing the MSS option. do you have any pointer on how to do that in
kernel tcp code?

Thanks

Vincent

On Thu, 2012-10-25 at 15:27 -0700, Vincent Li wrote:

Hi,

this sounds crazy, we have a weird situation that an unknown tcp
implementation not putting tcp MSS option in the SYN/ACK which caused
us some issue. I am tasked to mimic the unknown tcp immplementation on
not sending MSS in tcp SYN/ACK, I am wondering if I can achieve that
by modifying linux kernel tcp code,  there is socket option
TCP_MAXSEG, but that seems only affecting the size of MSS, not
removing the MSS option. do you have any pointer on how to do that in
kernel tcp code?

You'll have to patch the code.

Or else, you could add a new feature to net/netfilter/xt_TCPMSS.c

(We already have 

#define XT_TCPMSS_CLAMP_PMTU 0xffff

You could add

#define XT_TCPMSS_REMOVE 0xfffe

and replace MSS option by NOP

On 10/25/2012 03:27 PM, Vincent Li wrote:

Hi,

this sounds crazy, we have a weird situation that an unknown tcp
implementation not putting tcp MSS option in the SYN/ACK which caused
us some issue.

All that means is you/your TCP stack are to assume an MSS of 536 bytes.  In that sense at least, there is nothing (supposed to be) weird about it.

rick jones

On Thu, Oct 25, 2012 at 3:42 PM, Eric Dumazet [off-list ref] wrote:

On Thu, 2012-10-25 at 15:27 -0700, Vincent Li wrote:

quoted

Hi,

this sounds crazy, we have a weird situation that an unknown tcp
implementation not putting tcp MSS option in the SYN/ACK which caused
us some issue. I am tasked to mimic the unknown tcp immplementation on
not sending MSS in tcp SYN/ACK, I am wondering if I can achieve that
by modifying linux kernel tcp code,  there is socket option
TCP_MAXSEG, but that seems only affecting the size of MSS, not
removing the MSS option. do you have any pointer on how to do that in
kernel tcp code?

You'll have to patch the code.

Or else, you could add a new feature to net/netfilter/xt_TCPMSS.c

(We already have

#define XT_TCPMSS_CLAMP_PMTU 0xffff

You could add

#define XT_TCPMSS_REMOVE 0xfffe

and replace MSS option by NOP

thanks for the reply, I did a quick look at the code, as a quick dirty
hack, can I change the

 699 /* Set up TCP options for SYN-ACKs. */
 700 static unsigned int tcp_synack_options(struct sock *sk,
 701                                    struct request_sock *req,
 702                                    unsigned int mss, struct sk_buff *skb,
 703                                    struct tcp_out_options *opts,
 704                                    struct tcp_md5sig_key **md5,
 705                                    struct tcp_extend_values *xvp)
 706 {

..................
 730         /* We always send an MSS option. */
 731         opts->mss = mss; <---------here set opts->mss = 0 ?

would that work?

thanks for the reply, I did a quick look at the code, as a quick dirty
hack, can I change the

 699 /* Set up TCP options for SYN-ACKs. */
 700 static unsigned int tcp_synack_options(struct sock *sk,
 701                                    struct request_sock *req,
 702                                    unsigned int mss, struct sk_buff *skb,
 703                                    struct tcp_out_options *opts,
 704                                    struct tcp_md5sig_key **md5,
 705                                    struct tcp_extend_values *xvp)
 706 {

..................
 730         /* We always send an MSS option. */
 731         opts->mss = mss; <---------here set opts->mss = 0 ?

would that work?

tcp_make_synack() calls tcp_options_write() has this:

        if (unlikely(opts->mss)) {
                *ptr++ = htonl((TCPOPT_MSS << 24) |
                               (TCPOLEN_MSS << 16) |
                               opts->mss);
        }

It will not add the option if opts->mss is zero, so this should work too.

Vijay

On Thu, Oct 25, 2012 at 3:42 PM, Eric Dumazet [off-list ref] wrote:

On Thu, 2012-10-25 at 15:27 -0700, Vincent Li wrote:

quoted

Hi,

this sounds crazy, we have a weird situation that an unknown tcp
implementation not putting tcp MSS option in the SYN/ACK which caused
us some issue. I am tasked to mimic the unknown tcp immplementation on
not sending MSS in tcp SYN/ACK, I am wondering if I can achieve that
by modifying linux kernel tcp code,  there is socket option
TCP_MAXSEG, but that seems only affecting the size of MSS, not
removing the MSS option. do you have any pointer on how to do that in
kernel tcp code?

You'll have to patch the code.

Or else, you could add a new feature to net/netfilter/xt_TCPMSS.c

(We already have

#define XT_TCPMSS_CLAMP_PMTU 0xffff

You could add

#define XT_TCPMSS_REMOVE 0xfffe

and replace MSS option by NOP

for the sake of complete information, net/netfilter/xt_TCPOPTSTRIP.c
already have this feature to strip tcp option

Vincent

On Friday 2012-10-26 00:42, Eric Dumazet wrote:

On Thu, 2012-10-25 at 15:27 -0700, Vincent Li wrote:

quoted

this sounds crazy, we have a weird situation that an unknown tcp
implementation not putting tcp MSS option in the SYN/ACK which caused
us some issue. I am tasked to mimic the unknown tcp immplementation on
not sending MSS in tcp SYN/ACK, I am wondering if I can achieve that
by modifying linux kernel tcp code,  there is socket option
TCP_MAXSEG, but that seems only affecting the size of MSS, not
removing the MSS option. do you have any pointer on how to do that in
kernel tcp code?

You'll have to patch the code.

Or else, you could add a new feature to net/netfilter/xt_TCPMSS.c

Or you could just use -j TCPOPTSTRIP.