Re: [PATCH v2] cgroup/bpf: fast path for not loaded skb BPF filtering

On 12/14/21 07:27, Martin KaFai Lau wrote:

On Sat, Dec 11, 2021 at 07:17:49PM +0000, Pavel Begunkov wrote:

quoted

cgroup_bpf_enabled_key static key guards from overhead in cases where
no cgroup bpf program of a specific type is loaded in any cgroup. Turn
out that's not always good enough, e.g. when there are many cgroups but
ones that we're interesting in are without bpf. It's seen in server
environments, but the problem seems to be even wider as apparently
systemd loads some BPF affecting my laptop.

Profiles for small packet or zerocopy transmissions over fast network
show __cgroup_bpf_run_filter_skb() taking 2-3%, 1% of which is from
migrate_disable/enable(), and similarly on the receiving side. Also
got +4-5% of t-put for local testing.

What is t-put?  throughput?

yes

Local testing means sending to lo/dummy?

yes, it was dummy specifically

[ ... ]

quoted

diff --git a/include/linux/bpf-cgroup.h b/include/linux/bpf-cgroup.h
index 11820a430d6c..793e4f65ccb5 100644
--- a/include/linux/bpf-cgroup.h
+++ b/include/linux/bpf-cgroup.h

@@ -219,11 +219,28 @@ int bpf_percpu_cgroup_storage_copy(struct bpf_map *map, void *key, void *value);
  int bpf_percpu_cgroup_storage_update(struct bpf_map *map, void *key,
  				     void *value, u64 flags);
  
+static inline bool
+__cgroup_bpf_prog_array_is_empty(struct cgroup_bpf *cgrp_bpf,
+				 enum cgroup_bpf_attach_type type)

Lets remove this.

quoted

+{
+	struct bpf_prog_array *array = rcu_access_pointer(cgrp_bpf->effective[type]);
+
+	return array == &empty_prog_array.hdr;
+}
+
+#define CGROUP_BPF_TYPE_ENABLED(sk, atype)				       \

and change cgroup.c to directly use this instead, so
everywhere holding a fullsock sk will use this instead
of having two helpers for empty check.

Why? CGROUP_BPF_TYPE_ENABLED can't be a function atm because of header
dependency hell, and so it'd kill some of typization, which doesn't add
clarity. And also it imposes some extra overhead to *sockopt using
the first helper directly.

I think it's better with two of them. I could inline the second
one, but it wouldn't have been pretty.

[ ... ]

quoted

diff --git a/kernel/bpf/core.c b/kernel/bpf/core.c
index 2405e39d800f..fedc7b44a1a9 100644
--- a/kernel/bpf/core.c
+++ b/kernel/bpf/core.c

@@ -1967,18 +1967,10 @@ static struct bpf_prog_dummy {
  	},
  };
  
-/* to avoid allocating empty bpf_prog_array for cgroups that
- * don't have bpf program attached use one global 'empty_prog_array'
- * It will not be modified the caller of bpf_prog_array_alloc()
- * (since caller requested prog_cnt == 0)
- * that pointer should be 'freed' by bpf_prog_array_free()
- */
-static struct {
-	struct bpf_prog_array hdr;
-	struct bpf_prog *null_prog;
-} empty_prog_array = {
+struct bpf_empty_prog_array empty_prog_array = {
  	.null_prog = NULL,
  };
+EXPORT_SYMBOL(empty_prog_array);

nit. Since it is exported, may be prefix it with 'bpf_'.

yeah, sure


-- 
Pavel Begunkov