Re: [PATCH bpf-next v3] bpf/docs: Document kfunc lifecycle / stability expectations

On Sun, Feb 05, 2023 at 12:42:03PM -0800, Cong Wang wrote:

On Fri, Feb 03, 2023 at 09:57:27AM -0600, David Vernet wrote:

quoted

BPF kernel <-> kernel API stability has been discussed at length over
the last several weeks and months. Now that we've largely aligned over
kfuncs being the way forward, and BPF helpers being considered
functionally frozen, it's time to document the expectations for kfunc
lifecycles and stability so that everyone (BPF users, kfunc developers,
and maintainers) are all aligned, and have a crystal-clear understanding
of the expectations surrounding kfuncs.

To do that, this patch adds that documentation to the main kfuncs
documentation page via a new 'kfunc lifecycle expectations' section. The
patch describes how decisions are made in the kernel regarding whether
to include, keep, deprecate, or change / remove a kfunc. As described
very overtly in the patch itself, but likely worth highlighting here:

"kfunc stability" does not mean, nor ever will mean, "BPF APIs may block
development elsewhere in the kernel".

Rather, the intention and expectation is for kfuncs to be treated like
EXPORT_SYMBOL_GPL symbols in the kernel. The goal is for kfuncs to be a
safe and valuable option for maintainers and kfunc developers to extend
the kernel, without tying anyone's hands, or imposing any kind of
restrictions on maintainers in the same way that UAPI changes do.

I think they are still different, kernel modules are still considered as
a part of kernel development, while eBPF code is not that supposed to be
kernel development, at least much further. Treating them alike is
misleading, IMHO.

I'm not following. How is a BPF program not kernel development?

quoted

In addition to the 'kfunc lifecycle expectations' section, this patch
also adds documentation for a new KF_DEPRECATED kfunc flag which kfunc
authors or maintainers can choose to add to kfuncs if and when they
decide to deprecate them. Note that as described in the patch itself, a
kfunc need not be deprecated before being changed or removed -- this
flag is simply provided as an available deprecation mechanism for those
that want to provide a deprecation story / timeline to their users.
When necessary, kfuncs may be changed or removed to accommodate changes
elsewhere in the kernel without any deprecation at all.

This fundamentally contradicts with Compile-Once-Run-Everywhere
https://facebookmicrosites.github.io/bpf/blog/2020/02/19/bpf-portability-and-co-re.html

Sorry, but again, I'm not following your point. What exactly about this
"fundamentally contradicts" with CO-RE? Please elaborate if you're going
to claim that something is a fundamental contradiction.

Could you add some clarification for this too? Especically how we could
respect CO-RE meanwhile deprecating kfuncs?

I don't know what you mean by "respecting CO-RE". You can compile a BPF
program that calls a kfunc and invoke it on differents, as long as
whatever kernel you're running on provides that kfunc with the same BTF
encoding. This is no different than e.g. accessing a struct element on
two kernel versions.

Also, CO-RE doesn't provide any ironclad guarantees either. If you
access a struct element in a BPF program, and then a kernel version
removes that element from the struct, that BPF program will fail to load
on that kernel.

BTW, not related to compatibility, but still kfuncs related confusion,
it also contradicts with Documentation/bpf/bpf_design_QA.rst:

"
Q: Can BPF functionality such as new program or map types, new
helpers, etc be added out of kernel module code?

A: NO.

Agreed, we should improve the QA to mention that you can load kfuncs
from a module -- thanks for pointing that out!

"

The conntrack kfuncs like bpf_skb_ct_alloc() reside in a kernel module.

Thanks!