On Fri, May 31, 2019 at 03:25:25PM +0000, Chris Mason wrote:

I'm being pretty liberal with chopping down quoted material to help 
emphasize a particular opinion about how to bootstrap existing 
out-of-tree projects into the kernel.  My goal here is to talk more 
about the process and less about the technical details, so please 
forgive me if I've ignored or changed the technical meaning of anything 
below.

On 30 May 2019, at 12:15, Kris Van Hees wrote:

quoted

On Thu, May 23, 2019 at 01:28:44PM -0700, Alexei Starovoitov wrote:

... I believe that the discussion that has been going on in other
emails has shown that while introducing a program type that provides a
generic (abstracted) context is a different approach from what has 
been done
so far, it is a new use case that provides for additional ways in 
which BPF
can be used.

[ ... ]

quoted

Yes and no.  It depends on what you are trying to do with the BPF 
program that
is attached to the different events.  From a tracing perspective, 
providing a
single BPF program with an abstract context would ...

[ ... ]

quoted

In this model kprobe/ksys_write and 
tracepoint/syscalls/sys_enter_write are
equivalent for most tracing purposes ...

[ ... ]

quoted

I agree with what you are saying but I am presenting an additional use 
case

[ ... ]

quoted

quoted

All that aside the kernel support for shared libraries is an awesome
feature to have and a bunch of folks want to see it happen, but
it's not a blocker for 'dtrace to bpf' user space work.
libbpf can be taught to do this 'pseudo shared library' feature
while 'dtrace to bpf' side doesn't need to do anything special.

[ ... ]

This thread intermixes some abstract conceptual changes with smaller 
technical improvements, and in general it follows a familiar pattern 
other out-of-tree projects have hit while trying to adapt the kernel to 
their existing code.  Just from this one email, I quoted the abstract 
models with use cases etc, and this is often where the discussions side 
track into less productive areas.

quoted

So you are basically saying that I should redesign DTrace?

In your place, I would have removed features and adapted dtrace as much 
as possible to require the absolute minimum of kernel patches, or even 
better, no patches at all.  I'd document all of the features that worked 
as expected, and underline anything either missing or suboptimal that 
needed additional kernel changes.  Then I'd focus on expanding the 
community of people using dtrace against the mainline kernel, and work 
through the series features and improvements one by one upstream over 
time.

Well, that is actually what I am doing in the sense that the proposed patches
are quite minimal and lie at the core of the style of tracing that we need to
support.  So I definitely agree with your statement.  The code I posted
implements a minimal set of features (hardly any at all), although as Peter
pointed out, some more can be stripped from it and I have done that already
in a revision of the patchset I was preparing.

Your current approach relies on an all-or-nothing landing of patches 
upstream, and this consistently leads to conflict every time a project 
tries it.  A more incremental approach will require bigger changes on 
the dtrace application side, but over time it'll be much easier to 
justify your kernel changes.  You won't have to talk in abstract models, 
and you'll have many more concrete examples of people asking for dtrace 
features against mainline.  Most importantly, you'll make dtrace 
available on more kernels than just the absolute latest mainline, and 
removing dependencies makes the project much easier for new users to 
try.

I am not sure where I gave the impression that my approach relies on an
all-or-nothing landing of patches.  My intent (and the content of the patches
reflects that I think) was to work from a minimal base and build on that,
adding things as needed.  Granted, it depends on a rather crucial feature in
the design that apparently should be avoided for now as well, and I can
definitely work on avoiding that for now.  But I hope that it is clear from
the patch set I posted that an incremental approach is indeed what I intend
to do.

Thank you for putting it in clear terms and explaining patfalls that have
be observed in the past with projects.  I will proceed with an even more
minimalist approach.

To that end, could you advice on who patches should be Cc'd to to have the
first minimal code submitted to a tools/dtrace directory in the kernel tree?

	Kris