[PATCH net-next] bpf: add skb_postpush_rcsum and fix dev_forward_skb occasions

Add a small helper skb_postpush_rcsum() and fix up redirect locations
that need CHECKSUM_COMPLETE fixups on ingress. dev_forward_skb() expects
a proper csum that covers also Ethernet header, f.e. since 2c26d34bbcc0
("net/core: Handle csum for CHECKSUM_COMPLETE VXLAN forwarding"), we
also do skb_postpull_rcsum() after pulling Ethernet header off via
eth_type_trans().

When using eBPF in a netns setup f.e. with vxlan in collect metadata mode,
I can trigger the following csum issue with an IPv6 setup:

  [  505.144065] dummy1: hw csum failure
  [...]
  [  505.144108] Call Trace:
  [  505.144112]  <IRQ>  [<ffffffff81372f08>] dump_stack+0x44/0x5c
  [  505.144134]  [<ffffffff81607cea>] netdev_rx_csum_fault+0x3a/0x40
  [  505.144142]  [<ffffffff815fee3f>] __skb_checksum_complete+0xcf/0xe0
  [  505.144149]  [<ffffffff816f0902>] nf_ip6_checksum+0xb2/0x120
  [  505.144161]  [<ffffffffa08c0e0e>] icmpv6_error+0x17e/0x328 [nf_conntrack_ipv6]
  [  505.144170]  [<ffffffffa0898eca>] ? ip6t_do_table+0x2fa/0x645 [ip6_tables]
  [  505.144177]  [<ffffffffa08c0725>] ? ipv6_get_l4proto+0x65/0xd0 [nf_conntrack_ipv6]
  [  505.144189]  [<ffffffffa06c9a12>] nf_conntrack_in+0xc2/0x5a0 [nf_conntrack]
  [  505.144196]  [<ffffffffa08c039c>] ipv6_conntrack_in+0x1c/0x20 [nf_conntrack_ipv6]
  [  505.144204]  [<ffffffff8164385d>] nf_iterate+0x5d/0x70
  [  505.144210]  [<ffffffff816438d6>] nf_hook_slow+0x66/0xc0
  [  505.144218]  [<ffffffff816bd302>] ipv6_rcv+0x3f2/0x4f0
  [  505.144225]  [<ffffffff816bca40>] ? ip6_make_skb+0x1b0/0x1b0
  [  505.144232]  [<ffffffff8160b77b>] __netif_receive_skb_core+0x36b/0x9a0
  [  505.144239]  [<ffffffff8160bdc8>] ? __netif_receive_skb+0x18/0x60
  [  505.144245]  [<ffffffff8160bdc8>] __netif_receive_skb+0x18/0x60
  [  505.144252]  [<ffffffff8160ccff>] process_backlog+0x9f/0x140
  [  505.144259]  [<ffffffff8160c4a5>] net_rx_action+0x145/0x320
  [...]

What happens is that on ingress, we push Ethernet header back in, either
from cls_bpf or right before skb_do_redirect(), but without updating csum.
The "hw csum failure" can be fixed by using the new skb_postpush_rcsum()
helper for the dev_forward_skb() case to correct the csum diff again.

Fixes: 3896d655f4d4 ("bpf: introduce bpf_clone_redirect() helper")
Fixes: 27b29f63058d ("bpf: add bpf_redirect() helper")
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
---
 ( Given -net pull req is already out and nobody seemed to have triggered
   this so far, net-next seems just fine imho. )

 include/linux/skbuff.h |  7 +++++++
 net/core/filter.c      | 17 +++++++++++++----
 2 files changed, 20 insertions(+), 4 deletions(-)

diff --git a/include/linux/skbuff.h b/include/linux/skbuff.h
index 6b6bd42..f8fad71 100644
--- a/include/linux/skbuff.h
+++ b/include/linux/skbuff.h

@@ -2805,6 +2805,13 @@ static inline void skb_postpull_rcsum(struct sk_buff *skb,
 
 unsigned char *skb_pull_rcsum(struct sk_buff *skb, unsigned int len);
 
+static inline void skb_postpush_rcsum(struct sk_buff *skb,
+				      const void *start, unsigned int len)
+{
+	if (skb->ip_summed == CHECKSUM_COMPLETE)
+		skb->csum = csum_add(skb->csum, csum_partial(start, len, 0));
+}
+
 /**
  *	pskb_trim_rcsum - trim received skb and update checksum
  *	@skb: buffer to trim

diff --git a/net/core/filter.c b/net/core/filter.c
index 35e6fed..c6160ce 100644
--- a/net/core/filter.c
+++ b/net/core/filter.c

@@ -1368,8 +1368,9 @@ static u64 bpf_skb_store_bytes(u64 r1, u64 r2, u64 r3, u64 r4, u64 flags)
 		/* skb_store_bits cannot return -EFAULT here */
 		skb_store_bits(skb, offset, ptr, len);
 
-	if (BPF_RECOMPUTE_CSUM(flags) && skb->ip_summed == CHECKSUM_COMPLETE)
-		skb->csum = csum_add(skb->csum, csum_partial(ptr, len, 0));
+	if (BPF_RECOMPUTE_CSUM(flags))
+		skb_postpush_rcsum(skb, ptr, len);
+
 	return 0;
 }
 

@@ -1525,8 +1526,12 @@ static u64 bpf_clone_redirect(u64 r1, u64 ifindex, u64 flags, u64 r4, u64 r5)
 	if (unlikely(!skb2))
 		return -ENOMEM;
 
-	if (BPF_IS_REDIRECT_INGRESS(flags))
+	if (BPF_IS_REDIRECT_INGRESS(flags)) {
+		if (G_TC_AT(skb2->tc_verd) & AT_INGRESS)
+			skb_postpush_rcsum(skb2, skb_mac_header(skb2),
+					   skb2->mac_len);
 		return dev_forward_skb(dev, skb2);
+	}
 
 	skb2->dev = dev;
 	skb_sender_cpu_clear(skb2);

@@ -1569,8 +1574,12 @@ int skb_do_redirect(struct sk_buff *skb)
 		return -EINVAL;
 	}
 
-	if (BPF_IS_REDIRECT_INGRESS(ri->flags))
+	if (BPF_IS_REDIRECT_INGRESS(ri->flags)) {
+		if (G_TC_AT(skb->tc_verd) & AT_INGRESS)
+			skb_postpush_rcsum(skb, skb_mac_header(skb),
+					   skb->mac_len);
 		return dev_forward_skb(dev, skb);
+	}
 
 	skb->dev = dev;
 	skb_sender_cpu_clear(skb);

-- 
1.9.3

On 07.01.2016 02:01, Daniel Borkmann wrote:

+static inline void skb_postpush_rcsum(struct sk_buff *skb,
+				      const void *start, unsigned int len)
+{
+	if (skb->ip_summed == CHECKSUM_COMPLETE)
+		skb->csum = csum_add(skb->csum, csum_partial(start, len, 0));

skb->csum = csum_partial(start, len, skb->csum);

should work without calling carry-add twice, no?

Bye,
Hannes

Hi Daniel,

[auto build test ERROR on net-next/master]

url:    https://github.com/0day-ci/linux/commits/Daniel-Borkmann/bpf-add-skb_postpush_rcsum-and-fix-dev_forward_skb-occasions/20160107-090423
config: x86_64-lkp (attached as .config)
reproduce:
        # save the attached .config to linux build tree
        make ARCH=x86_64 

All errors (new ones prefixed by >>):

   In file included from include/net/sch_generic.h:8:0,
                    from include/linux/filter.h:16,
                    from include/net/sock.h:64,
                    from include/net/inet_sock.h:27,
                    from include/net/ip.h:30,
                    from net/core/filter.c:34:
   net/core/filter.c: In function 'bpf_clone_redirect':

quoted

net/core/filter.c:1530:19: error: 'struct sk_buff' has no member named 'tc_verd'

      if (G_TC_AT(skb2->tc_verd) & AT_INGRESS)
                      ^
   include/uapi/linux/pkt_cls.h:11:25: note: in definition of macro '_TC_MAKE32'
    #define _TC_MAKE32(x) ((x))
                            ^
   include/uapi/linux/pkt_cls.h:54:26: note: in expansion of macro '_TC_GETVALUE'
    #define G_TC_AT(x)       _TC_GETVALUE(x,S_TC_AT,M_TC_AT)
                             ^
   net/core/filter.c:1530:7: note: in expansion of macro 'G_TC_AT'
      if (G_TC_AT(skb2->tc_verd) & AT_INGRESS)
          ^
   net/core/filter.c: In function 'skb_do_redirect':
   net/core/filter.c:1578:18: error: 'struct sk_buff' has no member named 'tc_verd'
      if (G_TC_AT(skb->tc_verd) & AT_INGRESS)
                     ^
   include/uapi/linux/pkt_cls.h:11:25: note: in definition of macro '_TC_MAKE32'
    #define _TC_MAKE32(x) ((x))
                            ^
   include/uapi/linux/pkt_cls.h:54:26: note: in expansion of macro '_TC_GETVALUE'
    #define G_TC_AT(x)       _TC_GETVALUE(x,S_TC_AT,M_TC_AT)
                             ^
   net/core/filter.c:1578:7: note: in expansion of macro 'G_TC_AT'
      if (G_TC_AT(skb->tc_verd) & AT_INGRESS)
          ^

vim +1530 net/core/filter.c

  1524	
  1525		skb2 = skb_clone(skb, GFP_ATOMIC);
  1526		if (unlikely(!skb2))
  1527			return -ENOMEM;
  1528	
  1529		if (BPF_IS_REDIRECT_INGRESS(flags)) {

1530			if (G_TC_AT(skb2->tc_verd) & AT_INGRESS)

  1531				skb_postpush_rcsum(skb2, skb_mac_header(skb2),
  1532						   skb2->mac_len);
  1533			return dev_forward_skb(dev, skb2);

---
0-DAY kernel test infrastructure                Open Source Technology Center
https://lists.01.org/pipermail/kbuild-all                   Intel Corporation

On 01/07/2016 04:22 AM, kbuild test robot wrote:

Hi Daniel,

[auto build test ERROR on net-next/master]

url:    https://github.com/0day-ci/linux/commits/Daniel-Borkmann/bpf-add-skb_postpush_rcsum-and-fix-dev_forward_skb-occasions/20160107-090423
config: x86_64-lkp (attached as .config)
reproduce:
         # save the attached .config to linux build tree
         make ARCH=x86_64

All errors (new ones prefixed by >>):

    In file included from include/net/sch_generic.h:8:0,
                     from include/linux/filter.h:16,
                     from include/net/sock.h:64,
                     from include/net/inet_sock.h:27,
                     from include/net/ip.h:30,
                     from net/core/filter.c:34:
    net/core/filter.c: In function 'bpf_clone_redirect':

quoted

quoted

net/core/filter.c:1530:19: error: 'struct sk_buff' has no member named 'tc_verd'

       if (G_TC_AT(skb2->tc_verd) & AT_INGRESS)

Hmm, right. I'll add an extra helper function then for v2 as we also
have such ifdef/else in other places for accessing tc_verd.

Thanks,
Daniel

On 01/07/2016 02:53 AM, Hannes Frederic Sowa wrote:

On 07.01.2016 02:01, Daniel Borkmann wrote:

quoted

+static inline void skb_postpush_rcsum(struct sk_buff *skb,
+                      const void *start, unsigned int len)
+{
+    if (skb->ip_summed == CHECKSUM_COMPLETE)
+        skb->csum = csum_add(skb->csum, csum_partial(start, len, 0));

skb->csum = csum_partial(start, len, skb->csum);

should work without calling carry-add twice, no?

Hmm, indeed, seems you're right. Since the csum_partial() only covers
this particular fragment with feeding initial csum as 0, the result
from the do_csum() stays as is for the first part, and the csum_add()
does effectively the same as if we've fed skb->csum already into
csum_partial().

There are btw couple of other places where this generic helper could be
used as well.