Re: [PATCH bpf-next v3] bpf/docs: Document kfunc lifecycle / stability expectations

Cong Wang [off-list ref] writes:

On Fri, Feb 03, 2023 at 09:57:27AM -0600, David Vernet wrote:

quoted

BPF kernel <-> kernel API stability has been discussed at length over
the last several weeks and months. Now that we've largely aligned over
kfuncs being the way forward, and BPF helpers being considered
functionally frozen, it's time to document the expectations for kfunc
lifecycles and stability so that everyone (BPF users, kfunc developers,
and maintainers) are all aligned, and have a crystal-clear understanding
of the expectations surrounding kfuncs.

To do that, this patch adds that documentation to the main kfuncs
documentation page via a new 'kfunc lifecycle expectations' section. The
patch describes how decisions are made in the kernel regarding whether
to include, keep, deprecate, or change / remove a kfunc. As described
very overtly in the patch itself, but likely worth highlighting here:

"kfunc stability" does not mean, nor ever will mean, "BPF APIs may block
development elsewhere in the kernel".

Rather, the intention and expectation is for kfuncs to be treated like
EXPORT_SYMBOL_GPL symbols in the kernel. The goal is for kfuncs to be a
safe and valuable option for maintainers and kfunc developers to extend
the kernel, without tying anyone's hands, or imposing any kind of
restrictions on maintainers in the same way that UAPI changes do.

I think they are still different, kernel modules are still considered as
a part of kernel development, while eBPF code is not that supposed to be
kernel development, at least much further. Treating them alike is
misleading, IMHO.

If you read the actual documentation text added to kfuncs.rst this
difference is indeed called out. But you're right that "treated like" in
the commit message is probably a bit strong.

quoted

In addition to the 'kfunc lifecycle expectations' section, this patch
also adds documentation for a new KF_DEPRECATED kfunc flag which kfunc
authors or maintainers can choose to add to kfuncs if and when they
decide to deprecate them. Note that as described in the patch itself, a
kfunc need not be deprecated before being changed or removed -- this
flag is simply provided as an available deprecation mechanism for those
that want to provide a deprecation story / timeline to their users.
When necessary, kfuncs may be changed or removed to accommodate changes
elsewhere in the kernel without any deprecation at all.

This fundamentally contradicts with Compile-Once-Run-Everywhere
https://facebookmicrosites.github.io/bpf/blog/2020/02/19/bpf-portability-and-co-re.html
Could you add some clarification for this too? Especically how we could
respect CO-RE meanwhile deprecating kfuncs?

Well, CO-RE doesn't work for kfuncs, currently, so... :)

What do you mean "respect CO-RE", though? CO-RE is a tool to make BPF
programs more portable, so not sure how one would "respect" that?

BTW, not related to compatibility, but still kfuncs related confusion,
it also contradicts with Documentation/bpf/bpf_design_QA.rst:

"
Q: Can BPF functionality such as new program or map types, new
helpers, etc be added out of kernel module code?

A: NO.
"

The conntrack kfuncs like bpf_skb_ct_alloc() reside in a kernel
module.

Yup, good point, we should update that. I'll send a patch...

-Toke