On Fri, Feb 14, 2020 at 11:36:34AM +0000, Bruce Richardson wrote:

On Thu, Feb 13, 2020 at 09:40:40PM -0500, Neil Horman wrote:

quoted

On Thu, Feb 13, 2020 at 05:40:43PM +0000, Ray Kinsella wrote:

quoted

On 05/02/2020 11:32, Neil Horman wrote:

quoted

On Wed, Feb 05, 2020 at 11:04:29AM +0100, Luca Boccassi wrote:

quoted

On Tue, 2020-02-04 at 07:02 -0500, Neil Horman wrote:

quoted

quoted

But if we can do the versioning in the master, LTS can backport it
and can have mature version of that API in LTS without breaking
the existing users.

But why bother?  The only thing you've changed is the version
tagging.  Its ok to leave that alone in LTS, you just cant change
it.

Thats part of the burden of an LTS release, it will have some drift
from the upstream head, because you have to keep things stable.  So
you stabilize the upstream ABI version for this API and just never
backport it to the current LTS release.

Hi,

A customer (OVS) explicitly and specifically requested backporting
the symbol change to 19.11, as they don't want to enable
experimental APIs in their releases. I replied that it would only be
acceptable with aliasing to keep compatibility, and Ferruh very
kindly did the work to implement that.

but, thats part of the agreement, no?  You can't always have new
features and stability at the same time.

I get that this is an odd corner case, because strictly speaking you
could waive the ABI change that libabigail is reporting, and most
application users (like OVS) could get away with it, because their
application does all the right things to make it ok, but I don't
think you can make a decsion like this for all users based on the
request of a single user.

It seems like the right thing is for OVS to augment their build time
configuration to allow developers to select the ability to use
experimental apis at compile time, and then the decision is placed in
the hands of end users.

So this is not isolated case right ... it is common from API's to
graduate from experimental to mature, we _want_ that to happen, we
_want_ APIs to mature. 

Sure, I can absolutely agree with that, though I would suggest that the
maturity of the ABI is orthogonal to its labeling as such (more on that
below)

quoted

I am worried what you are proposing will encourage a behavior whereby
maintainers to wait until the declaration of the next major ABI version
to make these symbol changes, causing these kinds of changes to queue
up for that release. 

I think you're probably right about that, and would make the agrument
that thats perfectly fine (again I'll clarify below)

quoted

And you would have to ask why?  In the case of a reasonably mature API,
there will be no functional or signature change - its mature!  So what
is the harm of providing an alias back to Experimental until the next
ABI version is declared?

From a philosophical standpoint, there is absoluely no harm, and I don't
think anyone would disagree, the harm comes from the details of the
implementation, as you've noted.

quoted

So while yes, you are 100% right - experimental mean no guarantees.
But if the API is baked, it is not going to change, so I don't see the
harm.

We don't want the unnecessary triumph of policy over pragmatism. 

I would make the converse argument here.  While I agree that when an API
is mature, theres no point in calling it experimental anymore, I would
also suggest that, if an API is mature, and not expected to change,
theres no harm in leaving its version tag as experimental for an LTS
release.  This is what I was referring to above.  Once an application
developer has done the work to integrate an API from DPDK into its
application, that work is done.  If the API remains stable, then I
honestly don't know that they care that the version label is EXPERIMENTAL
versus 20.11 or 20.05 or whatever.  They care about the API being stable
more so than its version label.  As such, it seems....reasonable to me to
have developers queue their migration of experimental APIs to official
versioned APIs at the next major release deliniation point.

I'd welcome counter arguments, but it seems pretty natural to me to make
these sorts of changes at the major release mark.  People using
experimantal apis have already implicity agreed to manage changes to
them, so if we just hold it stable in an LTS release (and don't update
the version label), its just gravy for them.

The counter argument that I see is that while the experimental tag remains
in place the user has no way to know that an API is stable or not, and so
in many projects cannot use the API. If for example an API that is
experimental in 19.11 is unchanged through 20.05 at which point we decide
to promote it to stable. Once the change to the exp tag it is made, any
users of 19.11 can then see that it is an unchanged and now-stable ABI and
can start using it in their software, if they wish, without having to wait
for the 20.11 release. Changing the tag early allows ABIs to be potentially
used immediately.

I would agree with this, however, when using an LTS release, in my mind at
least, part of the agreement there is you get stability in the fuctions that
were declared stable at the time of release.  I'm not sure there should be an
expectation of additional stabilization within the lifetime of the release
(thats really what the 12 month LTS release cycle is for, no)?  You never have
to wait more than a year for a new set of stable features.  If you want faster
feature integration than that, you can choose to enable the experimental API's
and accept the benefits and drawbacks thereof.

That said, if (as I understand it) the goal is to provide a mechanism to allow
experimental features to be promoted to stable status, perhaps we can find a
happy middle ground.  What if we were to create a construct such as this:

#pragma push_macro("ALLOW_EXPERIMENTAL_APIS")
#undef ALLOW_EXPERIMENTAL_APIS
void __rte_experimental func(...);
#pragma pop_macro("ALLOW_EXPERIMENTAL_APIS")

Such a consruct would allow the maintainer of an API to pseudo-promote an API
from a experimental to stable status, at least so far as compilation would be
concerned.  Its messy and clunky, and it wouldn't change the function version at
all, but the end result would be that:
a) such a wraped experimental function would no longer issue a warning when used
during compilation/linking
and
b) provide a semi-easy grepable pattern for application writers to look for when
considering the use of an API that was previously experimental

such a construct would have to be used very judiciously, in that once its
implemented, the API has to be treated as stable, even though its 'excused' from
the normal checking, but it could provide something of the more rapid promotion
path being sought.

Thoughts?
Neil

Regards,
/Bruce