Discussion:
[dpdk-dev] [dpdk-announce] important design choices - statistics - ABI
Thomas Monjalon
2015-06-16 23:29:47 UTC
Permalink
Hi all,

Sometimes there are some important discussions about architecture or design
which require opinions from several developers. Unfortunately, we cannot
read every threads. Maybe that using the announce mailing list will help
to bring more audience to these discussions.
Please note that
- the announce@ ML is moderated to keep a low traffic,
- every announce email is forwarded to dev@ ML.
In case you want to reply to this email, please use ***@dpdk.org address.

There were some debates about software statistics disabling.
Should they be always on or possibly disabled when compiled?
We need to take a decision shortly and discuss (or agree) this proposal:
http://dpdk.org/ml/archives/dev/2015-June/019461.html

During the development of the release 2.0, there was an agreement to keep
ABI compatibility or to bring new ABI while keeping old one during one release.
In case it's not possible to have this transition, the (exceptional) break
should be acknowledged by several developers.
http://dpdk.org/doc/guides-2.0/rel_notes/abi.html
There were some interesting discussions but not a lot of participants:
http://thread.gmane.org/gmane.comp.networking.dpdk.devel/8367/focus=8461

During the current development cycle for the release 2.1, the ABI question
arises many times in different threads.
To add the hash key size field, it is proposed to use a struct padding gap:
http://dpdk.org/ml/archives/dev/2015-June/019386.html
To support the flow director for VF, there is no proposal yet:
http://dpdk.org/ml/archives/dev/2015-June/019343.html
To add the speed capability, it is proposed to break ABI in the release 2.2:
http://dpdk.org/ml/archives/dev/2015-June/019225.html
To support vhost-user multiqueues, it is proposed to break ABI in 2.2:
http://dpdk.org/ml/archives/dev/2015-June/019443.html
To add the interrupt mode, it is proposed to add a build-time option
CONFIG_RTE_EAL_RX_INTR to switch between compatible and ABI breaking binary:
http://dpdk.org/ml/archives/dev/2015-June/018947.html
To add the packet type, there is a proposal to add a build-time option
CONFIG_RTE_NEXT_ABI common to every ABI breaking features:
http://dpdk.org/ml/archives/dev/2015-June/019172.html
We must also better document how to remove a deprecated ABI:
http://dpdk.org/ml/archives/dev/2015-June/019465.html
The ABI compatibility is a new constraint and we need to better understand
what it means and how to proceed. Even the macros are not yet well documented:
http://dpdk.org/ml/archives/dev/2015-June/019357.html

Thanks for your attention and your participation in these important choices.
Matthew Hall
2015-06-17 04:36:54 UTC
Permalink
Post by Thomas Monjalon
There were some debates about software statistics disabling.
Should they be always on or possibly disabled when compiled?
http://dpdk.org/ml/archives/dev/2015-June/019461.html
This goes against the idea I have seen before that we should be moving toward
a distro-friendly approach where one copy of DPDK can be used by multiple apps
without having to rebuild it. It seems like it is also a bit ABI hostile
according to the below goals / discussions.

Jemalloc is also very high-performance code and still manages to allow
enabling and disabling statistics at runtime. Are we sure it's impossible for
DPDK or just theorizing?
Post by Thomas Monjalon
During the development of the release 2.0, there was an agreement to keep
ABI compatibility or to bring new ABI while keeping old one during one release.
In case it's not possible to have this transition, the (exceptional) break
should be acknowledged by several developers.
Personally to me it seems more important to preserve the ABI on patch
releases, like 2.X.Y going to 2.X.Z. But maybe I missed something?
Post by Thomas Monjalon
During the current development cycle for the release 2.1, the ABI question
arises many times in different threads.
Most but not all of these examples point to a different issue which sometimes
happens in libraries... often seen as "old-style" versus "new-style" C library
interface. For example, in old-style like libpcap there are a lot of structs,
both opaque and non-opaque, which the caller must allocate in order to run libpcap.

However new-style libraries such as libcurl usually just have init functions
which initialize all the secret structs based on some defaults and some user
parameters and hide the actual structs from the user. If you want to adjust
some you call an adjuster function that modifies the actual secret struct
contents, with some enum saying what field to adjust, and the new value you
want it to have.

If you want to keep a stable ABI for a non-stable library like DPDK, there's a
good chance you must begin hiding all these weird device specific structs all
over the DPDK from the user needing to directly allocate and modify them.
Otherwise the ABI breaks everytime you have to add adjustments, extensions,
modifications to all these obscure special features.

Matthew.
Stephen Hemminger
2015-06-17 05:28:16 UTC
Permalink
Post by Matthew Hall
Post by Thomas Monjalon
There were some debates about software statistics disabling.
Should they be always on or possibly disabled when compiled?
http://dpdk.org/ml/archives/dev/2015-June/019461.html
This goes against the idea I have seen before that we should be moving toward
a distro-friendly approach where one copy of DPDK can be used by multiple apps
without having to rebuild it. It seems like it is also a bit ABI hostile
according to the below goals / discussions.
Jemalloc is also very high-performance code and still manages to allow
enabling and disabling statistics at runtime. Are we sure it's impossible for
DPDK or just theorizing?
Post by Thomas Monjalon
During the development of the release 2.0, there was an agreement to keep
ABI compatibility or to bring new ABI while keeping old one during one
release.
Post by Thomas Monjalon
In case it's not possible to have this transition, the (exceptional)
break
Post by Thomas Monjalon
should be acknowledged by several developers.
Personally to me it seems more important to preserve the ABI on patch
releases, like 2.X.Y going to 2.X.Z. But maybe I missed something?
Post by Thomas Monjalon
During the current development cycle for the release 2.1, the ABI
question
Post by Thomas Monjalon
arises many times in different threads.
Most but not all of these examples point to a different issue which sometimes
happens in libraries... often seen as "old-style" versus "new-style" C library
interface. For example, in old-style like libpcap there are a lot of structs,
both opaque and non-opaque, which the caller must allocate in order to run libpcap.
However new-style libraries such as libcurl usually just have init functions
which initialize all the secret structs based on some defaults and some user
parameters and hide the actual structs from the user. If you want to adjust
some you call an adjuster function that modifies the actual secret struct
contents, with some enum saying what field to adjust, and the new value you
want it to have.
If you want to keep a stable ABI for a non-stable library like DPDK, there's a
good chance you must begin hiding all these weird device specific structs all
over the DPDK from the user needing to directly allocate and modify them.
Otherwise the ABI breaks everytime you have to add adjustments, extensions,
modifications to all these obscure special features.
Matthew.
The DPDK makes extensive use of inline functions which prevents data hiding
necessary for ABI stablility. This a fundamental tradeoff, and since the
whole
reason for DPDK is performance; the ABI is going to be a moving target.

It would make more sense to provide a higher level API which was abstracted,
slower, but stable for applications. But in doing so it would mean giving
up things
like inline lockless rings. Just don't go as far as the Open (not)
dataplane API;
which is just an excuse for closed source.
Marc Sune
2015-06-17 08:23:55 UTC
Permalink
Post by Stephen Hemminger
Post by Matthew Hall
Post by Thomas Monjalon
There were some debates about software statistics disabling.
Should they be always on or possibly disabled when compiled?
http://dpdk.org/ml/archives/dev/2015-June/019461.html
This goes against the idea I have seen before that we should be moving toward
a distro-friendly approach where one copy of DPDK can be used by multiple apps
without having to rebuild it. It seems like it is also a bit ABI hostile
according to the below goals / discussions.
Jemalloc is also very high-performance code and still manages to allow
enabling and disabling statistics at runtime. Are we sure it's impossible for
DPDK or just theorizing?
Post by Thomas Monjalon
During the development of the release 2.0, there was an agreement to keep
ABI compatibility or to bring new ABI while keeping old one during one
release.
Post by Thomas Monjalon
In case it's not possible to have this transition, the (exceptional)
break
Post by Thomas Monjalon
should be acknowledged by several developers.
Personally to me it seems more important to preserve the ABI on patch
releases, like 2.X.Y going to 2.X.Z. But maybe I missed something?
Post by Thomas Monjalon
During the current development cycle for the release 2.1, the ABI
question
Post by Thomas Monjalon
arises many times in different threads.
Most but not all of these examples point to a different issue which sometimes
happens in libraries... often seen as "old-style" versus "new-style" C library
interface. For example, in old-style like libpcap there are a lot of structs,
both opaque and non-opaque, which the caller must allocate in order to run libpcap.
However new-style libraries such as libcurl usually just have init functions
which initialize all the secret structs based on some defaults and some user
parameters and hide the actual structs from the user. If you want to adjust
some you call an adjuster function that modifies the actual secret struct
contents, with some enum saying what field to adjust, and the new value you
want it to have.
If you want to keep a stable ABI for a non-stable library like DPDK, there's a
good chance you must begin hiding all these weird device specific structs all
over the DPDK from the user needing to directly allocate and modify them.
Otherwise the ABI breaks everytime you have to add adjustments, extensions,
modifications to all these obscure special features.
Matthew.
The DPDK makes extensive use of inline functions which prevents data hiding
necessary for ABI stablility. This a fundamental tradeoff, and since the
whole
reason for DPDK is performance; the ABI is going to be a moving target.
It would make more sense to provide a higher level API which was abstracted,
slower, but stable for applications. But in doing so it would mean giving
up things
like inline lockless rings. Just don't go as far as the Open (not)
dataplane API;
which is just an excuse for closed source.
+1 to what Stephen says. I don't think though a higher level API is
really worth.

I unfortunately could not participate in the ABI discussion (there are
times in which you cannot be following each and every single thread in
the mailing list, and then it got lost in my inbox, my bad).

To me, ABI compatibility is a must if we'd have long term support
releases (1.8.0, 1.8.1...), which we don't (I proposed to have them a
couple of times).

On the other side, if the argument for ABI compatibility is shared
libraries and to make it easy to package for distros, I don't see
anything better than using the MAJOR.MIN version to identify the ABI
(1.8, 1.9). Having LTS support would also help distros be able to stay
in an ABI compatible version of DPDK and get recent bug fixes, and to
better identify ABi changes (MAJ.MIN).

Moreover, I doubt that many people use shared libraries in DPDK. DPDK is
built for performance and has a lot of code inlining which makes anyway
your binary to be recompiled, as Stephen says.

DPDK is still growing, and the strict ABI policy is forcing patches to
be delayed, to add hacky work-arounds to not break the ABI policy ( Re:
[dpdk-dev] [PATCH v6 01/18] mbuf: redefine packet_type in rte_mbuf as an
example) that are announced that have to be removed afterwards. I see
the entire ABI policy as a blocker and inducing to have botches all over
the code, than anything else.

Marc
Thomas Monjalon
2015-06-17 08:23:23 UTC
Permalink
Post by Stephen Hemminger
Post by Matthew Hall
Post by Thomas Monjalon
There were some debates about software statistics disabling.
Should they be always on or possibly disabled when compiled?
http://dpdk.org/ml/archives/dev/2015-June/019461.html
This goes against the idea I have seen before that we should be moving
toward a distro-friendly approach where one copy of DPDK can be used by
multiple apps without having to rebuild it. It seems like it is also a
bit ABI hostile according to the below goals / discussions.
Jemalloc is also very high-performance code and still manages to allow
enabling and disabling statistics at runtime. Are we sure it's impossible
for DPDK or just theorizing?
Please Matthew, it is better to comment in the thread dedicated to statistics.

[...]
Post by Stephen Hemminger
Post by Matthew Hall
Personally to me it seems more important to preserve the ABI on patch
releases, like 2.X.Y going to 2.X.Z. But maybe I missed something?
The goal of the ABI deprecation process was to provide a smooth integration
of the release 2.X+1.0. There are 4 months between releases 2.X.0 and 2.X+1.0.

[...]
Post by Stephen Hemminger
Post by Matthew Hall
However new-style libraries such as libcurl usually just have init
functions which initialize all the secret structs based on some defaults
and some user parameters and hide the actual structs from the user.
If you want to adjust some you call an adjuster function that modifies
the actual secret struct contents, with some enum saying what field to
adjust, and the new value you want it to have.
If you want to keep a stable ABI for a non-stable library like DPDK,
there's a good chance you must begin hiding all these weird device
specific structs all over the DPDK from the user needing to directly
allocate and modify them.
Otherwise the ABI breaks everytime you have to add adjustments,
extensions, modifications to all these obscure special features.
The DPDK makes extensive use of inline functions which prevents data hiding
necessary for ABI stablility. This a fundamental tradeoff, and since the
whole reason for DPDK is performance; the ABI is going to be a moving target.
It would make more sense to provide a higher level API which was abstracted,
slower, but stable for applications. But in doing so it would mean giving
up things like inline lockless rings. Just don't go as far as the Open (not)
dataplane API;
which is just an excuse for closed source.
I don't understand what you mean.
Bruce Richardson
2015-06-17 11:17:09 UTC
Permalink
Post by Matthew Hall
Post by Thomas Monjalon
There were some debates about software statistics disabling.
Should they be always on or possibly disabled when compiled?
http://dpdk.org/ml/archives/dev/2015-June/019461.html
This goes against the idea I have seen before that we should be moving toward
a distro-friendly approach where one copy of DPDK can be used by multiple apps
without having to rebuild it. It seems like it is also a bit ABI hostile
according to the below goals / discussions.
Jemalloc is also very high-performance code and still manages to allow
enabling and disabling statistics at runtime. Are we sure it's impossible for
DPDK or just theorizing?
+1 to this. I think that any compile-time option to disable stats should only
be used when we have a proven performance issue with just disabling them at runtime.
I would assume that apps do not switch on or off stats multiple times per second,
so any code branches to track stats or not would be entirely predictable in the
code - since they always go one way. Therefore, when disabledi, we should be looking
at a very minimal overhead per stat. If there are lots of checks for the same
value in the one path, i.e. lots of stats in a hot path, hopefully the compiler
will be smart enough to make the check just once. If not, we can always do
that in the C code by duplicating the hotpath code for with or without stats cases -
again selectable at runtime.

Also, there is also the case where the stats tracking itself is such low overhead
that its not worth disabling. In that case, neither runtime nor compile-time disabling
should need to be provided. For example, any library that is keeping a track of
bursts of packets should not need to have that stat disable option - one increment
or addition per burst of (32) packets is not going to seriously affect any app. :-)

Regards,
/Bruce
Dumitrescu, Cristian
2015-06-18 16:32:24 UTC
Permalink
Post by Morten Brørup
-----Original Message-----
Sent: Wednesday, June 17, 2015 12:17 PM
To: Matthew Hall
Subject: Re: [dpdk-dev] [dpdk-announce] important design choices -
statistics - ABI
Post by Matthew Hall
Post by Thomas Monjalon
There were some debates about software statistics disabling.
Should they be always on or possibly disabled when compiled?
http://dpdk.org/ml/archives/dev/2015-June/019461.html
This goes against the idea I have seen before that we should be moving
toward
Post by Matthew Hall
a distro-friendly approach where one copy of DPDK can be used by multiple
apps
Post by Matthew Hall
without having to rebuild it. It seems like it is also a bit ABI hostile
according to the below goals / discussions.
Jemalloc is also very high-performance code and still manages to allow
enabling and disabling statistics at runtime. Are we sure it's impossible for
DPDK or just theorizing?
+1 to this. I think that any compile-time option to disable stats should only
be used when we have a proven performance issue with just disabling them at runtime.
I would assume that apps do not switch on or off stats multiple times per second,
so any code branches to track stats or not would be entirely predictable in the
code - since they always go one way. Therefore, when disabledi, we should be looking
at a very minimal overhead per stat. If there are lots of checks for the same
value in the one path, i.e. lots of stats in a hot path, hopefully the compiler
will be smart enough to make the check just once. If not, we can always do
that in the C code by duplicating the hotpath code for with or without stats cases -
again selectable at runtime.
I see where you're coming from, but reality is you cannot guarantee that the few conditional branches in the library are going to be predicted correctly simply because the application and the other libraries used by the app have an unknown number of conditional branches themselves. For complex apps, the total number of conditional branches is large and the more there are, the lesser probability of such a branch being predicted correctly is. I agree that for test apps like l3fwd with just a few branches the probability to have library branches being predicted correctly is high, but for me is an incorrect proof point. The cost of ~14 cycles per branch misprediction is important for DPDK packet budgets.

Since we don't control the application, I am feeling very uncomfortable with generic statements about how application is likely to execute and the impact of library conditional branches over the application should be. To me, they sound like we are willing to take chances, and to me this is not the right decision. In my opinion, the right decision for a significant framework like DPDK is to keep all options open for the apps: keep counters always enabled, keep counters always disabled, keep counters enabled first and disabled later. I suggest we should move the focus from WHY arguments like: "I don't think anybody will ever need this this" to the HOW part of making sure that technical solution we pick is correct and keeps all options open.

I think stats are not always equivalent to incrementing a counter. In the guideline proposal, I am providing several examples of more complex statistics logic that is more than just inc one counter. As an extreme case, think about the case where the metric to compute requires complex math like prediction of next packet arrival time based on recent history, etc. When defining a policy, we should consider a broad spectrum of metrics, not just n_pkts_in.
Post by Morten Brørup
Also, there is also the case where the stats tracking itself is such low overhead
that its not worth disabling. In that case, neither runtime nor compile-time disabling
should need to be provided. For example, any library that is keeping a track of
bursts of packets should not need to have that stat disable option - one increment
or addition per burst of (32) packets is not going to seriously affect any app. :-
)
Regards,
/Bruce
Dumitrescu, Cristian
2015-06-18 13:25:33 UTC
Permalink
Post by Morten Brørup
-----Original Message-----
Sent: Wednesday, June 17, 2015 5:37 AM
Subject: Re: [dpdk-dev] [dpdk-announce] important design choices -
statistics - ABI
Post by Thomas Monjalon
There were some debates about software statistics disabling.
Should they be always on or possibly disabled when compiled?
http://dpdk.org/ml/archives/dev/2015-June/019461.html
This goes against the idea I have seen before that we should be moving toward
a distro-friendly approach where one copy of DPDK can be used by multiple apps
without having to rebuild it. It seems like it is also a bit ABI hostile
according to the below goals / discussions.
Matthew, thanks for your input. As Thomas also mentions, it would be good to first read the proposal on the guideline (http://dpdk.org/ml/archives/dev/2015-June/019461.html).

In the guideline proposal, we are addressing the topic of preventing the ABI changes due to library statistics support, it would be good to get your opinion on that, I am not sure you saw it. Given DPDK paramount focus on performance, I think probably provides the best solution for ABI compatibility in this case.
Post by Morten Brørup
Jemalloc is also very high-performance code and still manages to allow
enabling and disabling statistics at runtime. Are we sure it's impossible for
DPDK or just theorizing?
A performance improvement for one environment might still be an overkill for DPDK run-time environment.

Stats cannot be enabled/disabled at run-time without performance impact, as it typically requires testing a persistent flag for incrementing the stats counters, which requires a conditional branch instruction. Sometimes the branches are predicted correctly, sometimes not, and in this case the CPU pipeline needs to be flushed; typical branch misprediction cost is ~14 cycles, which is important given a packet budget of ~200 cycles.

When only a small number of branches are present in the (application + library) code, they are predicted correctly, so I am sure that for a simple test application like l3fw the cost of run-time enabled stats is not visible. The problem is that real applications are much more complex and they typically have a lot of conditional branches, out of which the library stats test is just one of them, so they cannot be all predicted correctly, hence the cost of stats run-time enablement becomes very much visible.
Post by Morten Brørup
Post by Thomas Monjalon
During the development of the release 2.0, there was an agreement to
keep
Post by Thomas Monjalon
ABI compatibility or to bring new ABI while keeping old one during one
release.
Post by Thomas Monjalon
In case it's not possible to have this transition, the (exceptional) break
should be acknowledged by several developers.
Personally to me it seems more important to preserve the ABI on patch
releases, like 2.X.Y going to 2.X.Z. But maybe I missed something?
Post by Thomas Monjalon
During the current development cycle for the release 2.1, the ABI question
arises many times in different threads.
Most but not all of these examples point to a different issue which sometimes
happens in libraries... often seen as "old-style" versus "new-style" C library
interface. For example, in old-style like libpcap there are a lot of structs,
both opaque and non-opaque, which the caller must allocate in order to run libpcap.
However new-style libraries such as libcurl usually just have init functions
which initialize all the secret structs based on some defaults and some user
parameters and hide the actual structs from the user. If you want to adjust
some you call an adjuster function that modifies the actual secret struct
contents, with some enum saying what field to adjust, and the new value you
want it to have.
If you want to keep a stable ABI for a non-stable library like DPDK, there's a
good chance you must begin hiding all these weird device specific structs all
over the DPDK from the user needing to directly allocate and modify them.
Otherwise the ABI breaks everytime you have to add adjustments, extensions,
modifications to all these obscure special features.
Matthew.
Morten Brørup
2015-06-17 09:54:25 UTC
Permalink
Dear Thomas,

I don't have time to follow the DPDK Developers mailing list, but since you call for feedback, I would like to share my thoughts regarding these design choices.


Regarding the statistics discussion:

1. The suggested solution assumes that, when statistics is disabled, the cost of allocating and maintaining zero-value statistics is negligible. If statistics counters are only available through accessor functions, this is probably true.

However, if statistics counters are directly accessible, e.g. as elements in the fast path data structures of a library, maintaining zero-value statistics may a have memory and/or performance impact.

Since the compile time flag CONFIG_RTE_<LIBRARY_NAME>_STATS_COLLECT already tells the application if the statistics are present or not, the application should simply use this flag to determine if statistics are accessible or not.

2. The suggested solution with only one single flag per library prevents implementing statistics with varying granularity for different purposes. E.g. a library could have one set of statistics counters for ordinary SNMP purposes, and another set of statistics counters for debugging/optimization purposes.

Multiple flags per library should be possible. A hierarchy of flags per library is probably not required.


Regarding the PHY speed ABI:

1. The Ethernet PHY ABI for speed, duplex, etc. should be common throughout the entire DPDK. It might be confusing if some structures/functions use a bitmask to indicate PHY speed/duplex/personality/etc. and other structures/functions use a combination of an unsigned integer, duplex flag, personality enumeration etc. (By personality enumeration, I am referring to PHYs with multiple electrical interfaces. E.g. a dual personality PHY might have both an RJ45 copper interface and an SFP module interface, whereof only one can be active at any time.)

2. The auto-negotiation standard allows the PHY to announce (to its link partner) any subset of its capabilities to its link partner. E.g. a standard 10/100/100 Ethernet PHY (which can handle both 10 and 100 Mbit/s in both half and full duplex and 1 Gbit/s full duplex) can be configured to announce 10 Mbit/s half duplex and 100 Mbit/s full duplex capabilities to its link partner. (Of course, more useful combinations are normally announced, but the purpose of the example is to show that any combination is possible.)

The ABI for auto-negotiation should include options to select the list of capabilities to announce to the link partner. The Linux PHY ABI only allows forcing a selected speed and duplex (thereby disabling auto-negotiation) or enabling auto-negotiation (thereby announcing all possible speeds and duplex combinations the PHY is capable of). Don't make the same mistake in DPDK.

PS: While working for Vitesse Semiconductors (an Ethernet chip company) a long time ago, I actually wrote the API for their line of Ethernet PHYs. So I have hands on experience in this area.


Regarding the discussion about backwards/forwards compatibility in the ABI:

1. Sometimes, ABI breakage is required. That is the cost the users pay for getting the benefits from upgrading to the latest and greatest version of any library. The current solution of requiring acknowledgement from several qualified developers is fine - these developers will consider the cost/benefit on behalf of all the DPDK users and make a qualified decision.

2. It is my general experience that documentation is not always updated to reflect the fine details of the source code, and this also applies to release notes. For open source software, the primary point of documentation is usually the source code itself.

2a. It should be clearly visible directly in the DPDK source code (including makefiles etc.) which ABI (i.e. functions, macros, type definitions etc.) is the current, the deprecated, and the future.

2b. When a developer migrates a project using DPDK from a previous version of the DPDK, it should be easy for the developer to identify all DPDK ABI modifications and variants, e.g. by using a common indicator in the DPDK source code, such as LIBAPIVER, that developer can simply search for.

3. Adding special feature flags, e.g. CONFIG_RTE_EAL_RX_INTR, to indicate a breakage of the ABI, should only be done if it is the intention to keep both the current and the new variants of the feature in the DPDK in the future. Otherwise, such a flag should be combined with the standard ABI version indication, so it is clear that this feature belongs to certain versions (i.e. deprecated, current or future).


Med venlig hilsen / kind regards

Morten Brørup
CTO



SmartShare Systems A/S
Tonsbakken 16-18
DK-2740 Skovlunde
Denmark

Office      +45 70 20 00 93
Direct      +45 89 93 50 22
Mobile      +45 25 40 82 12

***@smartsharesystems.com
www.smartsharesystems.com
-----Original Message-----
From: announce [mailto:announce-***@dpdk.org] On Behalf Of Thomas Monjalon
Sent: 17. juni 2015 01:30
To: ***@dpdk.org
Subject: [dpdk-announce] important design choices - statistics - ABI

Hi all,

Sometimes there are some important discussions about architecture or design which require opinions from several developers. Unfortunately, we cannot read every threads. Maybe that using the announce mailing list will help to bring more audience to these discussions.
Please note that
- the announce@ ML is moderated to keep a low traffic,
- every announce email is forwarded to dev@ ML.
In case you want to reply to this email, please use ***@dpdk.org address.

There were some debates about software statistics disabling.
Should they be always on or possibly disabled when compiled?
We need to take a decision shortly and discuss (or agree) this proposal:
http://dpdk.org/ml/archives/dev/2015-June/019461.html

During the development of the release 2.0, there was an agreement to keep ABI compatibility or to bring new ABI while keeping old one during one release.
In case it's not possible to have this transition, the (exceptional) break should be acknowledged by several developers.
http://dpdk.org/doc/guides-2.0/rel_notes/abi.html
There were some interesting discussions but not a lot of participants:
http://thread.gmane.org/gmane.comp.networking.dpdk.devel/8367/focus=8461

During the current development cycle for the release 2.1, the ABI question arises many times in different threads.
To add the hash key size field, it is proposed to use a struct padding gap:
http://dpdk.org/ml/archives/dev/2015-June/019386.html
To support the flow director for VF, there is no proposal yet:
http://dpdk.org/ml/archives/dev/2015-June/019343.html
To add the speed capability, it is proposed to break ABI in the release 2.2:
http://dpdk.org/ml/archives/dev/2015-June/019225.html
To support vhost-user multiqueues, it is proposed to break ABI in 2.2:
http://dpdk.org/ml/archives/dev/2015-June/019443.html
To add the interrupt mode, it is proposed to add a build-time option CONFIG_RTE_EAL_RX_INTR to switch between compatible and ABI breaking binary:
http://dpdk.org/ml/archives/dev/2015-June/018947.html
To add the packet type, there is a proposal to add a build-time option CONFIG_RTE_NEXT_ABI common to every ABI breaking features:
http://dpdk.org/ml/archives/dev/2015-June/019172.html
We must also better document how to remove a deprecated ABI:
http://dpdk.org/ml/archives/dev/2015-June/019465.html
The ABI compatibility is a new constraint and we need to better understand what it means and how to proceed. Even the macros are not yet well documented:
http://dpdk.org/ml/archives/dev/2015-June/019357.html

Thanks for your attention and your participation in these important choices.
Dumitrescu, Cristian
2015-06-18 13:00:29 UTC
Permalink
Post by Morten Brørup
-----Original Message-----
Sent: Wednesday, June 17, 2015 10:54 AM
To: Thomas Monjalon
Subject: Re: [dpdk-dev] [dpdk-announce] important design choices -
statistics - ABI
Dear Thomas,
I don't have time to follow the DPDK Developers mailing list, but since you call
for feedback, I would like to share my thoughts regarding these design
choices.
1. The suggested solution assumes that, when statistics is disabled, the cost
of allocating and maintaining zero-value statistics is negligible. If statistics
counters are only available through accessor functions, this is probably true.
However, if statistics counters are directly accessible, e.g. as elements in the
fast path data structures of a library, maintaining zero-value statistics may a
have memory and/or performance impact.
Counters are only accessible through API functions.
Post by Morten Brørup
Since the compile time flag
CONFIG_RTE_<LIBRARY_NAME>_STATS_COLLECT already tells the
application if the statistics are present or not, the application should simply
use this flag to determine if statistics are accessible or not.
2. The suggested solution with only one single flag per library prevents
implementing statistics with varying granularity for different purposes. E.g. a
library could have one set of statistics counters for ordinary SNMP purposes,
and another set of statistics counters for debugging/optimization purposes.
Multiple flags per library should be possible. A hierarchy of flags per library is
probably not required.
Morten, thank you for your input. It would be good if you could add your contribution to the of the guidelines documentation patch by replying to the thread that Thomas indicated: http://dpdk.org/ml/archives/dev/2015-June/019461.html.

Our initial stats patch submission had a much finer granularity of stats configuration: per object type instead of per library, but a lot of people on this mailing list are against this, so we are now looking for one configuration flag per library.
Post by Morten Brørup
1. The Ethernet PHY ABI for speed, duplex, etc. should be common
throughout the entire DPDK. It might be confusing if some
structures/functions use a bitmask to indicate PHY
speed/duplex/personality/etc. and other structures/functions use a
combination of an unsigned integer, duplex flag, personality enumeration
etc. (By personality enumeration, I am referring to PHYs with multiple
electrical interfaces. E.g. a dual personality PHY might have both an RJ45
copper interface and an SFP module interface, whereof only one can be
active at any time.)
2. The auto-negotiation standard allows the PHY to announce (to its link
partner) any subset of its capabilities to its link partner. E.g. a standard
10/100/100 Ethernet PHY (which can handle both 10 and 100 Mbit/s in both
half and full duplex and 1 Gbit/s full duplex) can be configured to announce
10 Mbit/s half duplex and 100 Mbit/s full duplex capabilities to its link partner.
(Of course, more useful combinations are normally announced, but the
purpose of the example is to show that any combination is possible.)
The ABI for auto-negotiation should include options to select the list of
capabilities to announce to the link partner. The Linux PHY ABI only allows
forcing a selected speed and duplex (thereby disabling auto-negotiation) or
enabling auto-negotiation (thereby announcing all possible speeds and
duplex combinations the PHY is capable of). Don't make the same mistake in
DPDK.
PS: While working for Vitesse Semiconductors (an Ethernet chip company) a
long time ago, I actually wrote the API for their line of Ethernet PHYs. So I
have hands on experience in this area.
1. Sometimes, ABI breakage is required. That is the cost the users pay for
getting the benefits from upgrading to the latest and greatest version of any
library. The current solution of requiring acknowledgement from several
qualified developers is fine - these developers will consider the cost/benefit
on behalf of all the DPDK users and make a qualified decision.
2. It is my general experience that documentation is not always updated to
reflect the fine details of the source code, and this also applies to release
notes. For open source software, the primary point of documentation is
usually the source code itself.
2a. It should be clearly visible directly in the DPDK source code (including
makefiles etc.) which ABI (i.e. functions, macros, type definitions etc.) is the
current, the deprecated, and the future.
2b. When a developer migrates a project using DPDK from a previous version
of the DPDK, it should be easy for the developer to identify all DPDK ABI
modifications and variants, e.g. by using a common indicator in the DPDK
source code, such as LIBAPIVER, that developer can simply search for.
3. Adding special feature flags, e.g. CONFIG_RTE_EAL_RX_INTR, to indicate a
breakage of the ABI, should only be done if it is the intention to keep both
the current and the new variants of the feature in the DPDK in the future.
Otherwise, such a flag should be combined with the standard ABI version
indication, so it is clear that this feature belongs to certain versions (i.e.
deprecated, current or future).
Med venlig hilsen / kind regards
Morten Brørup
CTO
SmartShare Systems A/S
Tonsbakken 16-18
DK-2740 Skovlunde
Denmark
Office      +45 70 20 00 93
Direct      +45 89 93 50 22
Mobile      +45 25 40 82 12
www.smartsharesystems.com
-----Original Message-----
Thomas Monjalon
Sent: 17. juni 2015 01:30
Subject: [dpdk-announce] important design choices - statistics - ABI
Hi all,
Sometimes there are some important discussions about architecture or
design which require opinions from several developers. Unfortunately, we
cannot read every threads. Maybe that using the announce mailing list will
help to bring more audience to these discussions.
Please note that
There were some debates about software statistics disabling.
Should they be always on or possibly disabled when compiled?
http://dpdk.org/ml/archives/dev/2015-June/019461.html
During the development of the release 2.0, there was an agreement to keep
ABI compatibility or to bring new ABI while keeping old one during one release.
In case it's not possible to have this transition, the (exceptional) break should
be acknowledged by several developers.
http://dpdk.org/doc/guides-2.0/rel_notes/abi.html
http://thread.gmane.org/gmane.comp.networking.dpdk.devel/8367
/focus=8461
During the current development cycle for the release 2.1, the ABI question
arises many times in different threads.
http://dpdk.org/ml/archives/dev/2015-June/019386.html
http://dpdk.org/ml/archives/dev/2015-June/019343.html
http://dpdk.org/ml/archives/dev/2015-June/019225.html
http://dpdk.org/ml/archives/dev/2015-June/019443.html
To add the interrupt mode, it is proposed to add a build-time option
http://dpdk.org/ml/archives/dev/2015-June/018947.html
To add the packet type, there is a proposal to add a build-time option
http://dpdk.org/ml/archives/dev/2015-June/019172.html
http://dpdk.org/ml/archives/dev/2015-June/019465.html
The ABI compatibility is a new constraint and we need to better understand
http://dpdk.org/ml/archives/dev/2015-June/019357.html
Thanks for your attention and your participation in these important choices.
Neil Horman
2015-06-17 10:35:21 UTC
Permalink
Post by Thomas Monjalon
Hi all,
Sometimes there are some important discussions about architecture or design
which require opinions from several developers. Unfortunately, we cannot
read every threads. Maybe that using the announce mailing list will help
to bring more audience to these discussions.
Please note that
There were some debates about software statistics disabling.
Should they be always on or possibly disabled when compiled?
http://dpdk.org/ml/archives/dev/2015-June/019461.html
During the development of the release 2.0, there was an agreement to keep
ABI compatibility or to bring new ABI while keeping old one during one release.
In case it's not possible to have this transition, the (exceptional) break
should be acknowledged by several developers.
http://dpdk.org/doc/guides-2.0/rel_notes/abi.html
http://thread.gmane.org/gmane.comp.networking.dpdk.devel/8367/focus=8461
During the current development cycle for the release 2.1, the ABI question
arises many times in different threads.
http://dpdk.org/ml/archives/dev/2015-June/019386.html
http://dpdk.org/ml/archives/dev/2015-June/019343.html
http://dpdk.org/ml/archives/dev/2015-June/019225.html
http://dpdk.org/ml/archives/dev/2015-June/019443.html
To add the interrupt mode, it is proposed to add a build-time option
http://dpdk.org/ml/archives/dev/2015-June/018947.html
To add the packet type, there is a proposal to add a build-time option
http://dpdk.org/ml/archives/dev/2015-June/019172.html
http://dpdk.org/ml/archives/dev/2015-June/019465.html
The ABI compatibility is a new constraint and we need to better understand
http://dpdk.org/ml/archives/dev/2015-June/019357.html
Thanks for your attention and your participation in these important choices.
Thomas-
Just to re-iterate what you said earlier, and what was discussed in the
previous ABI discussions

1) ABI stability was introduced to promote DPDK's ability to be included with
various linux and BSD distributions. Distributions, by and large, favor
building libraries as DSO's, favoring security and updatability in favor of all
out performance.

2) The desire was to put DPDK developers in a mindset whereby ABI stability was
something they needed to think about during development, as the DPDK exposes
many data structures and instances that cannot be changed without breaking ABI

3) The versioning mechanism was introduced to allow for backward compatibility
during periods in which we needed to support both an old an new ABI

4) As Stephan and others point out, its not expected that we will always be able
to maintain ABI, and as such an easy library versioning mechanism was introduced
to prevent the loading of an incompatible library with an older application

5) The ABI policy was introduced to create a method by which new ABI facets
could be scheduled while allowing distros to prepare their downstream users for
the upcomming changes.


It seems to me, looking back over these last few months, that we're falling down
a bit on our use of (3). I've seen several people take advantage of the ABI
scheduled updates, but no one has tried the versioning interface, and as a
result patches are getting delayed, which was never my intent. Not sure whats
to be done about that, but we should probably address it. Is use of the
versionnig interface just too hard or convoluted?

Neil
Richardson, Bruce
2015-06-17 11:06:42 UTC
Permalink
Post by Morten Brørup
-----Original Message-----
Sent: Wednesday, June 17, 2015 11:35 AM
To: Thomas Monjalon
Subject: Re: [dpdk-dev] [dpdk-announce] important design choices -
statistics - ABI
Post by Thomas Monjalon
Hi all,
Sometimes there are some important discussions about architecture or
design which require opinions from several developers. Unfortunately,
we cannot read every threads. Maybe that using the announce mailing
list will help to bring more audience to these discussions.
Please note that
address.
Post by Thomas Monjalon
There were some debates about software statistics disabling.
Should they be always on or possibly disabled when compiled?
http://dpdk.org/ml/archives/dev/2015-June/019461.html
During the development of the release 2.0, there was an agreement to
keep ABI compatibility or to bring new ABI while keeping old one during
one release.
Post by Thomas Monjalon
In case it's not possible to have this transition, the (exceptional)
break should be acknowledged by several developers.
http://dpdk.org/doc/guides-2.0/rel_notes/abi.html
http://thread.gmane.org/gmane.comp.networking.dpdk.devel/8367/focus=84
61
During the current development cycle for the release 2.1, the ABI
question arises many times in different threads.
To add the hash key size field, it is proposed to use a struct padding
http://dpdk.org/ml/archives/dev/2015-June/019386.html
http://dpdk.org/ml/archives/dev/2015-June/019343.html
To add the speed capability, it is proposed to break ABI in the release
http://dpdk.org/ml/archives/dev/2015-June/019225.html
http://dpdk.org/ml/archives/dev/2015-June/019443.html
To add the interrupt mode, it is proposed to add a build-time option
CONFIG_RTE_EAL_RX_INTR to switch between compatible and ABI breaking
http://dpdk.org/ml/archives/dev/2015-June/018947.html
To add the packet type, there is a proposal to add a build-time option
http://dpdk.org/ml/archives/dev/2015-June/019172.html
http://dpdk.org/ml/archives/dev/2015-June/019465.html
The ABI compatibility is a new constraint and we need to better
understand what it means and how to proceed. Even the macros are not yet
http://dpdk.org/ml/archives/dev/2015-June/019357.html
Thanks for your attention and your participation in these important
choices.
Thomas-
Just to re-iterate what you said earlier, and what was discussed in
the previous ABI discussions
1) ABI stability was introduced to promote DPDK's ability to be included
with various linux and BSD distributions. Distributions, by and large,
favor building libraries as DSO's, favoring security and updatability in
favor of all out performance.
2) The desire was to put DPDK developers in a mindset whereby ABI
stability was something they needed to think about during development, as
the DPDK exposes many data structures and instances that cannot be changed
without breaking ABI
3) The versioning mechanism was introduced to allow for backward
compatibility during periods in which we needed to support both an old an
new ABI
4) As Stephan and others point out, its not expected that we will always
be able to maintain ABI, and as such an easy library versioning mechanism
was introduced to prevent the loading of an incompatible library with an
older application
5) The ABI policy was introduced to create a method by which new ABI
facets could be scheduled while allowing distros to prepare their
downstream users for the upcomming changes.
It seems to me, looking back over these last few months, that we're
falling down a bit on our use of (3). I've seen several people take
advantage of the ABI scheduled updates, but no one has tried the
versioning interface, and as a result patches are getting delayed, which
was never my intent. Not sure whats to be done about that, but we should
probably address it. Is use of the versionnig interface just too hard or
convoluted?
Neil
Hi Neil,

on my end, some suggestions:

1. the documentation on changing an API function provided in rte_compat.h
is really good, but I don't think this is present in our documentation in
the docs folder or on website is it (apologies if it is and I've missed
it)? This needs to go into programmers guide or some other doc (perhaps
the new doc that the coding style went into).

2. The documentation also needs an example of: this is how you add a new
function and update the map file, and this is how you a) mark a function
as deprecated and b) remove it completely. That way we could have one
guide covering API versioning, how to add, modify and remove functions.

3. This doc should also cover how to use the API checker tool, something I
haven't had the chance to look at yet, but should do in the near future!
:-)

Regards,

/Bruce
Mcnamara, John
2015-06-19 11:08:04 UTC
Permalink
Post by Morten Brørup
-----Original Message-----
Sent: Wednesday, June 17, 2015 12:07 PM
To: Neil Horman; Thomas Monjalon
Subject: Re: [dpdk-dev] [dpdk-announce] important design choices -
statistics - ABI
Hi Neil,
1. the documentation on changing an API function provided in rte_compat.h
is really good, but I don't think this is present in our documentation in
the docs folder or on website is it (apologies if it is and I've missed
it)? This needs to go into programmers guide or some other doc (perhaps
the new doc that the coding style went into).
2. The documentation also needs an example of: this is how you add a new
function and update the map file, and this is how you a) mark a function
as deprecated and b) remove it completely. That way we could have one
guide covering API versioning, how to add, modify and remove functions.
3. This doc should also cover how to use the API checker tool, something I
haven't had the chance to look at yet, but should do in the near future!
:-)
+1 on all three. We need better documentation on how to work with the ABI in DPDK. A new document in doc/guides/guidelines/ would be good.

John
Panu Matilainen
2015-06-17 12:14:01 UTC
Permalink
(initially accidentally sent to announce, resending to dev)
Post by Neil Horman
Post by Thomas Monjalon
Hi all,
Sometimes there are some important discussions about architecture or design
which require opinions from several developers. Unfortunately, we cannot
read every threads. Maybe that using the announce mailing list will help
to bring more audience to these discussions.
Please note that
There were some debates about software statistics disabling.
Should they be always on or possibly disabled when compiled?
http://dpdk.org/ml/archives/dev/2015-June/019461.html
During the development of the release 2.0, there was an agreement to keep
ABI compatibility or to bring new ABI while keeping old one during one release.
In case it's not possible to have this transition, the (exceptional) break
should be acknowledged by several developers.
http://dpdk.org/doc/guides-2.0/rel_notes/abi.html
http://thread.gmane.org/gmane.comp.networking.dpdk.devel/8367/focus=8461
During the current development cycle for the release 2.1, the ABI question
arises many times in different threads.
http://dpdk.org/ml/archives/dev/2015-June/019386.html
http://dpdk.org/ml/archives/dev/2015-June/019343.html
http://dpdk.org/ml/archives/dev/2015-June/019225.html
http://dpdk.org/ml/archives/dev/2015-June/019443.html
To add the interrupt mode, it is proposed to add a build-time option
http://dpdk.org/ml/archives/dev/2015-June/018947.html
To add the packet type, there is a proposal to add a build-time option
http://dpdk.org/ml/archives/dev/2015-June/019172.html
http://dpdk.org/ml/archives/dev/2015-June/019465.html
The ABI compatibility is a new constraint and we need to better understand
http://dpdk.org/ml/archives/dev/2015-June/019357.html
Thanks for your attention and your participation in these important choices.
Thomas-
Just to re-iterate what you said earlier, and what was discussed in the
previous ABI discussions
1) ABI stability was introduced to promote DPDK's ability to be included with
various linux and BSD distributions. Distributions, by and large, favor
building libraries as DSO's, favoring security and updatability in favor of all
out performance.
2) The desire was to put DPDK developers in a mindset whereby ABI stability was
something they needed to think about during development, as the DPDK exposes
many data structures and instances that cannot be changed without breaking ABI
3) The versioning mechanism was introduced to allow for backward compatibility
during periods in which we needed to support both an old an new ABI
4) As Stephan and others point out, its not expected that we will always be able
to maintain ABI, and as such an easy library versioning mechanism was introduced
to prevent the loading of an incompatible library with an older application
5) The ABI policy was introduced to create a method by which new ABI facets
could be scheduled while allowing distros to prepare their downstream users for
the upcomming changes.
It seems to me, looking back over these last few months, that we're falling down
a bit on our use of (3). I've seen several people take advantage of the ABI
scheduled updates, but no one has tried the versioning interface, and as a
result patches are getting delayed, which was never my intent. Not sure whats
to be done about that, but we should probably address it. Is use of the
versionnig interface just too hard or convoluted?
To me it seems that by far the biggest problem with ABI stability in
DPDK is features requiring changes to public structs (often directly
allocated and accessed by apps), which is something the symbol
versioning doesn't directly help with, you'd need to version the structs
too.

One only needs to glance at the glibc documentation on how to accomplish
it [1] to see it gets rather involved. Glibc promises backwards
compatibility for life, so the effort is justified. However in where
DPDK we're talking about extending compatibility for a few months by
minimum requirement, people are unlikely to bother.

[1] https://sourceware.org/glibc/wiki/Development/Versioning_A_Structure

- Panu -
Vincent JARDIN
2015-06-17 13:21:28 UTC
Permalink
Post by Panu Matilainen
(initially accidentally sent to announce, resending to dev)
Post by Neil Horman
Post by Thomas Monjalon
Hi all,
Sometimes there are some important discussions about architecture or design
which require opinions from several developers. Unfortunately, we cannot
read every threads. Maybe that using the announce mailing list will help
to bring more audience to these discussions.
Please note that
There were some debates about software statistics disabling.
Should they be always on or possibly disabled when compiled?
http://dpdk.org/ml/archives/dev/2015-June/019461.html
During the development of the release 2.0, there was an agreement to keep
ABI compatibility or to bring new ABI while keeping old one during one release.
In case it's not possible to have this transition, the (exceptional) break
should be acknowledged by several developers.
http://dpdk.org/doc/guides-2.0/rel_notes/abi.html
http://thread.gmane.org/gmane.comp.networking.dpdk.devel/8367/focus=8461
During the current development cycle for the release 2.1, the ABI question
arises many times in different threads.
http://dpdk.org/ml/archives/dev/2015-June/019386.html
http://dpdk.org/ml/archives/dev/2015-June/019343.html
http://dpdk.org/ml/archives/dev/2015-June/019225.html
http://dpdk.org/ml/archives/dev/2015-June/019443.html
To add the interrupt mode, it is proposed to add a build-time option
http://dpdk.org/ml/archives/dev/2015-June/018947.html
To add the packet type, there is a proposal to add a build-time option
http://dpdk.org/ml/archives/dev/2015-June/019172.html
http://dpdk.org/ml/archives/dev/2015-June/019465.html
The ABI compatibility is a new constraint and we need to better understand
http://dpdk.org/ml/archives/dev/2015-June/019357.html
Thanks for your attention and your participation in these important choices.
Thomas-
Just to re-iterate what you said earlier, and what was discussed in the
previous ABI discussions
1) ABI stability was introduced to promote DPDK's ability to be included with
various linux and BSD distributions. Distributions, by and large, favor
building libraries as DSO's, favoring security and updatability in favor of all
out performance.
2) The desire was to put DPDK developers in a mindset whereby ABI stability was
something they needed to think about during development, as the DPDK exposes
many data structures and instances that cannot be changed without breaking ABI
3) The versioning mechanism was introduced to allow for backward compatibility
during periods in which we needed to support both an old an new ABI
4) As Stephan and others point out, its not expected that we will always be able
to maintain ABI, and as such an easy library versioning mechanism was introduced
to prevent the loading of an incompatible library with an older application
5) The ABI policy was introduced to create a method by which new ABI facets
could be scheduled while allowing distros to prepare their downstream users for
the upcomming changes.
It seems to me, looking back over these last few months, that we're falling down
a bit on our use of (3). I've seen several people take advantage of the ABI
scheduled updates, but no one has tried the versioning interface, and as a
result patches are getting delayed, which was never my intent. Not sure whats
to be done about that, but we should probably address it. Is use of the
versionnig interface just too hard or convoluted?
To me it seems that by far the biggest problem with ABI stability in
DPDK is features requiring changes to public structs (often directly
allocated and accessed by apps), which is something the symbol
versioning doesn't directly help with, you'd need to version the structs
too.
One only needs to glance at the glibc documentation on how to accomplish
it [1] to see it gets rather involved. Glibc promises backwards
compatibility for life, so the effort is justified. However in where
DPDK we're talking about extending compatibility for a few months by
minimum requirement, people are unlikely to bother.
[1] https://sourceware.org/glibc/wiki/Development/Versioning_A_Structure
Does it means that it requires a specific engineering so we are back to
the needs of using two layers:
a- the direct calls to "non ABI stable layers" (current librte*) for
those we can/are fine to use it,
b- the calls thru such layers that guarantee the ABI stabilities (a
librte_compat?) for those who needs it.

Could both be exposed by the distributions so they can be consumed?

Thank you,
Vincent
Zhang, Helin
2015-06-18 08:36:13 UTC
Permalink
Hi Neil
Post by Morten Brørup
-----Original Message-----
Sent: Wednesday, June 17, 2015 6:35 PM
To: Thomas Monjalon
Subject: Re: [dpdk-dev] [dpdk-announce] important design choices - statistics -
ABI
Post by Thomas Monjalon
Hi all,
Sometimes there are some important discussions about architecture or
design which require opinions from several developers. Unfortunately,
we cannot read every threads. Maybe that using the announce mailing
list will help to bring more audience to these discussions.
Please note that
There were some debates about software statistics disabling.
Should they be always on or possibly disabled when compiled?
http://dpdk.org/ml/archives/dev/2015-June/019461.html
During the development of the release 2.0, there was an agreement to
keep ABI compatibility or to bring new ABI while keeping old one during one
release.
Post by Thomas Monjalon
In case it's not possible to have this transition, the (exceptional)
break should be acknowledged by several developers.
http://dpdk.org/doc/guides-2.0/rel_notes/abi.html
http://thread.gmane.org/gmane.comp.networking.dpdk.devel/8367/focus=84
61
During the current development cycle for the release 2.1, the ABI
question arises many times in different threads.
http://dpdk.org/ml/archives/dev/2015-June/019386.html
http://dpdk.org/ml/archives/dev/2015-June/019343.html
http://dpdk.org/ml/archives/dev/2015-June/019225.html
http://dpdk.org/ml/archives/dev/2015-June/019443.html
To add the interrupt mode, it is proposed to add a build-time option
CONFIG_RTE_EAL_RX_INTR to switch between compatible and ABI breaking
http://dpdk.org/ml/archives/dev/2015-June/018947.html
To add the packet type, there is a proposal to add a build-time option
http://dpdk.org/ml/archives/dev/2015-June/019172.html
http://dpdk.org/ml/archives/dev/2015-June/019465.html
The ABI compatibility is a new constraint and we need to better
understand what it means and how to proceed. Even the macros are not yet
http://dpdk.org/ml/archives/dev/2015-June/019357.html
Thanks for your attention and your participation in these important choices.
Thomas-
Just to re-iterate what you said earlier, and what was discussed in the
previous ABI discussions
1) ABI stability was introduced to promote DPDK's ability to be included with
various linux and BSD distributions. Distributions, by and large, favor building
libraries as DSO's, favoring security and updatability in favor of all out
performance.
2) The desire was to put DPDK developers in a mindset whereby ABI stability was
something they needed to think about during development, as the DPDK
exposes many data structures and instances that cannot be changed without
breaking ABI
3) The versioning mechanism was introduced to allow for backward compatibility
during periods in which we needed to support both an old an new ABI
4) As Stephan and others point out, its not expected that we will always be able
to maintain ABI, and as such an easy library versioning mechanism was
introduced to prevent the loading of an incompatible library with an older
application
5) The ABI policy was introduced to create a method by which new ABI facets
could be scheduled while allowing distros to prepare their downstream users for
the upcomming changes.
It seems to me, looking back over these last few months, that we're falling down
a bit on our use of (3). I've seen several people take advantage of the ABI
scheduled updates, but no one has tried the versioning interface, and as a result
patches are getting delayed, which was never my intent. Not sure whats to be
done about that, but we should probably address it. Is use of the versionnig
interface just too hard or convoluted?
That means we should try to use versioning mechanism as more as possible?
Are there any rules/instructions to judge what type of changes we should try
versioning mechanism, and what else we shouldn't try?
Is there any good examples of using versioning mechanism for reference?
Even not in DPDK project.

Thanks,
Helin
Post by Morten Brørup
Neil
O'Driscoll, Tim
2015-06-18 16:55:45 UTC
Permalink
Post by Morten Brørup
-----Original Message-----
Monjalon
Sent: Wednesday, June 17, 2015 12:30 AM
Subject: [dpdk-announce] important design choices - statistics - ABI
Hi all,
During the development of the release 2.0, there was an agreement to keep
ABI compatibility or to bring new ABI while keeping old one during one release.
In case it's not possible to have this transition, the (exceptional) break
should be acknowledged by several developers.
http://dpdk.org/doc/guides-2.0/rel_notes/abi.html
http://thread.gmane.org/gmane.comp.networking.dpdk.devel/8367/focus
=8461
During the current development cycle for the release 2.1, the ABI question
arises many times in different threads.
http://dpdk.org/ml/archives/dev/2015-June/019386.html
http://dpdk.org/ml/archives/dev/2015-June/019343.html
http://dpdk.org/ml/archives/dev/2015-June/019225.html
http://dpdk.org/ml/archives/dev/2015-June/019443.html
To add the interrupt mode, it is proposed to add a build-time option
http://dpdk.org/ml/archives/dev/2015-June/018947.html
To add the packet type, there is a proposal to add a build-time option
http://dpdk.org/ml/archives/dev/2015-June/019172.html
http://dpdk.org/ml/archives/dev/2015-June/019465.html
The ABI compatibility is a new constraint and we need to better understand
http://dpdk.org/ml/archives/dev/2015-June/019357.html
Thanks for your attention and your participation in these important choices.
There's been some good discussion on the ABI policy in various responses to this email. I think we now need to reach a conclusion on how we're going to proceed for the 2.1 release. Then, we can have further discussion on the use of versioning or other methods for avoiding the problem in future.

For the 2.1 release, I think we should agree to make patches that change the ABI controllable via a compile-time option. I like Olivier's proposal on using a single option (CONFIG_RTE_NEXT_ABI) to control all of these changes instead of a separate option per patch set (see http://dpdk.org/ml/archives/dev/2015-June/019147.html), so I think we should rework the affected patch sets to use that approach for 2.1.


Tim
Vincent JARDIN
2015-06-18 21:13:56 UTC
Permalink
I like Olivier's proposal on using a single option (CONFIG_RTE_NEXT_ABI) to control all of these changes instead of a separate option per patch set (seehttp://dpdk.org/ml/archives/dev/2015-June/019147.html), so I think we should rework the affected patch sets to use that approach for 2.1.
Do we have any other options to meet the short deadlines of 2.1?
Neil Horman
2015-06-19 10:26:54 UTC
Permalink
Post by O'Driscoll, Tim
Post by Morten Brørup
-----Original Message-----
Monjalon
Sent: Wednesday, June 17, 2015 12:30 AM
Subject: [dpdk-announce] important design choices - statistics - ABI
Hi all,
During the development of the release 2.0, there was an agreement to keep
ABI compatibility or to bring new ABI while keeping old one during one release.
In case it's not possible to have this transition, the (exceptional) break
should be acknowledged by several developers.
http://dpdk.org/doc/guides-2.0/rel_notes/abi.html
http://thread.gmane.org/gmane.comp.networking.dpdk.devel/8367/focus
=8461
During the current development cycle for the release 2.1, the ABI question
arises many times in different threads.
http://dpdk.org/ml/archives/dev/2015-June/019386.html
http://dpdk.org/ml/archives/dev/2015-June/019343.html
http://dpdk.org/ml/archives/dev/2015-June/019225.html
http://dpdk.org/ml/archives/dev/2015-June/019443.html
To add the interrupt mode, it is proposed to add a build-time option
http://dpdk.org/ml/archives/dev/2015-June/018947.html
To add the packet type, there is a proposal to add a build-time option
http://dpdk.org/ml/archives/dev/2015-June/019172.html
http://dpdk.org/ml/archives/dev/2015-June/019465.html
The ABI compatibility is a new constraint and we need to better understand
http://dpdk.org/ml/archives/dev/2015-June/019357.html
Thanks for your attention and your participation in these important choices.
There's been some good discussion on the ABI policy in various responses to this email. I think we now need to reach a conclusion on how we're going to proceed for the 2.1 release. Then, we can have further discussion on the use of versioning or other methods for avoiding the problem in future.
For the 2.1 release, I think we should agree to make patches that change the ABI controllable via a compile-time option. I like Olivier's proposal on using a single option (CONFIG_RTE_NEXT_ABI) to control all of these changes instead of a separate option per patch set (see http://dpdk.org/ml/archives/dev/2015-June/019147.html), so I think we should rework the affected patch sets to use that approach for 2.1.
This is a bad idea. Making ABI dependent on compile time options isn't a
maintainable solution. It breaks the notion of how LIBABIVER is supposed to
work (that is to say you make it impossible to really tell what ABI version you
are building). If you have two compile time options that modify the ABI, you
have to burn through 4 possible LIBABIVER version values to accomodate all
possible combinations, and then you need to remember that when you make them
statically applicable.

Neil
Post by O'Driscoll, Tim
Tim
Thomas Monjalon
2015-06-19 12:32:33 UTC
Permalink
Post by Neil Horman
Post by O'Driscoll, Tim
For the 2.1 release, I think we should agree to make patches that change
the ABI controllable via a compile-time option. I like Olivier's proposal
on using a single option (CONFIG_RTE_NEXT_ABI) to control all of these
changes instead of a separate option per patch set (see
http://dpdk.org/ml/archives/dev/2015-June/019147.html), so I think we
should rework the affected patch sets to use that approach for 2.1.
This is a bad idea. Making ABI dependent on compile time options isn't a
maintainable solution. It breaks the notion of how LIBABIVER is supposed to
work (that is to say you make it impossible to really tell what ABI version you
are building).
The idea was to make LIBABIVER increment dependent of CONFIG_RTE_NEXT_ABI.
So one ABI version number refers always to the same ABI.
Post by Neil Horman
If you have two compile time options that modify the ABI, you
have to burn through 4 possible LIBABIVER version values to accomodate all
possible combinations, and then you need to remember that when you make them
statically applicable.
The idea is to have only 1 compile-time option: CONFIG_RTE_NEXT_ABI.

Your intent when introducing ABI policy was to allow smooth porting of
applications from a DPDK version to another. Right?
The adopted solution was to provide backward compatibility during 1 release.
But there are cases where it's not possible. So the policy was to notice
the future change and wait one release cycle to break the ABI (failing
compatibility goals).
The compile-time option may provide an alternative DPDK packaging when the
ABI backward compatibility cannot be provided (case of mbuf changes).
In such case, it's still possible to upgrade DPDK by providing 2 versions of
DPDK libs. So the existing apps continue to link with the previous ABI and
have the possibility of migrating to the new one.
Another advantage of this approach is that we don't have to wait 1 release
to integrate the changes.
The last advantage is to benefit early of these changes with static libraries.
Neil Horman
2015-06-19 13:02:55 UTC
Permalink
Post by Thomas Monjalon
Post by Neil Horman
Post by O'Driscoll, Tim
For the 2.1 release, I think we should agree to make patches that change
the ABI controllable via a compile-time option. I like Olivier's proposal
on using a single option (CONFIG_RTE_NEXT_ABI) to control all of these
changes instead of a separate option per patch set (see
http://dpdk.org/ml/archives/dev/2015-June/019147.html), so I think we
should rework the affected patch sets to use that approach for 2.1.
This is a bad idea. Making ABI dependent on compile time options isn't a
maintainable solution. It breaks the notion of how LIBABIVER is supposed to
work (that is to say you make it impossible to really tell what ABI version you
are building).
The idea was to make LIBABIVER increment dependent of CONFIG_RTE_NEXT_ABI.
So one ABI version number refers always to the same ABI.
Post by Neil Horman
If you have two compile time options that modify the ABI, you
have to burn through 4 possible LIBABIVER version values to accomodate all
possible combinations, and then you need to remember that when you make them
statically applicable.
The idea is to have only 1 compile-time option: CONFIG_RTE_NEXT_ABI.
Your intent when introducing ABI policy was to allow smooth porting of
applications from a DPDK version to another. Right?
The adopted solution was to provide backward compatibility during 1 release.
But there are cases where it's not possible. So the policy was to notice
the future change and wait one release cycle to break the ABI (failing
compatibility goals).
The compile-time option may provide an alternative DPDK packaging when the
ABI backward compatibility cannot be provided (case of mbuf changes).
In such case, it's still possible to upgrade DPDK by providing 2 versions of
DPDK libs. So the existing apps continue to link with the previous ABI and
have the possibility of migrating to the new one.
Another advantage of this approach is that we don't have to wait 1 release
to integrate the changes.
The last advantage is to benefit early of these changes with static libraries.
Hm, ok, thats a bit more reasonable, but it still seems shaky to me.
Implementing an ABI preview option like this implies the notion that, after a
release, you have to remove all the ifdefs that you inserted to create the new
ABI. That seems like an easy task, but it becomes a pain when the ABI delta is
large, and is predicated on the centralization of work effort (that is to say
you need to identify someone to submit the 'remove the NEXT_ABI config ifdefs
from the build' patch every release.

What might be better would be a dpdk-next branch (or even a dpdk-next tree, of
the sort that Thomas Herbert proposed a few weeks ago). Patches that aren't ABI
stable can be put on the next-branch/tree in thier final format. You can
delcare the branch unstable (thereby reserving your right to rebase it). People
can use that to preview the next ABI version (complete with the update LIBABIVER
bump), and when you release dpdk-X, the new ABI for dpdk-X+1 is achieved by
simply merging.

Neil
Thomas Monjalon
2015-06-19 13:16:53 UTC
Permalink
Post by Neil Horman
Post by Thomas Monjalon
Post by Neil Horman
Post by O'Driscoll, Tim
For the 2.1 release, I think we should agree to make patches that change
the ABI controllable via a compile-time option. I like Olivier's proposal
on using a single option (CONFIG_RTE_NEXT_ABI) to control all of these
changes instead of a separate option per patch set (see
http://dpdk.org/ml/archives/dev/2015-June/019147.html), so I think we
should rework the affected patch sets to use that approach for 2.1.
This is a bad idea. Making ABI dependent on compile time options isn't a
maintainable solution. It breaks the notion of how LIBABIVER is supposed to
work (that is to say you make it impossible to really tell what ABI version you
are building).
The idea was to make LIBABIVER increment dependent of CONFIG_RTE_NEXT_ABI.
So one ABI version number refers always to the same ABI.
Post by Neil Horman
If you have two compile time options that modify the ABI, you
have to burn through 4 possible LIBABIVER version values to accomodate all
possible combinations, and then you need to remember that when you make them
statically applicable.
The idea is to have only 1 compile-time option: CONFIG_RTE_NEXT_ABI.
Your intent when introducing ABI policy was to allow smooth porting of
applications from a DPDK version to another. Right?
The adopted solution was to provide backward compatibility during 1 release.
But there are cases where it's not possible. So the policy was to notice
the future change and wait one release cycle to break the ABI (failing
compatibility goals).
The compile-time option may provide an alternative DPDK packaging when the
ABI backward compatibility cannot be provided (case of mbuf changes).
In such case, it's still possible to upgrade DPDK by providing 2 versions of
DPDK libs. So the existing apps continue to link with the previous ABI and
have the possibility of migrating to the new one.
Another advantage of this approach is that we don't have to wait 1 release
to integrate the changes.
The last advantage is to benefit early of these changes with static libraries.
Hm, ok, thats a bit more reasonable, but it still seems shaky to me.
Implementing an ABI preview option like this implies the notion that, after a
release, you have to remove all the ifdefs that you inserted to create the new
ABI. That seems like an easy task, but it becomes a pain when the ABI delta is
large, and is predicated on the centralization of work effort (that is to say
you need to identify someone to submit the 'remove the NEXT_ABI config ifdefs
from the build' patch every release.
It won't be so huge if we reserve the NEXT_ABI solution to changes which cannot
have easy backward compatibility with the compat macros you introduced.
I feel I can do the job of removing the ifdefs NEXT_ABI after each release.
At the same time, the deprecated API, using the compat macros, will be removed.
Post by Neil Horman
What might be better would be a dpdk-next branch (or even a dpdk-next tree, of
the sort that Thomas Herbert proposed a few weeks ago).
This tree was created after Thomas' request:
http://dpdk.org/browse/next/dpdk-next/
Post by Neil Horman
Patches that aren't ABI stable can be put on the next-branch/tree in thier
final format. You can delcare the branch unstable (thereby reserving your
right to rebase it). People can use that to preview the next ABI version
(complete with the update LIBABIVER bump), and when you release dpdk-X,
the new ABI for dpdk-X+1 is achieved by simply merging.
Having this tree living would be a nice improvement but it won't provide any
stable (and enough validated) releases to rely on.
Neil Horman
2015-06-19 15:27:30 UTC
Permalink
Post by Thomas Monjalon
Post by Neil Horman
Post by Thomas Monjalon
Post by Neil Horman
Post by O'Driscoll, Tim
For the 2.1 release, I think we should agree to make patches that change
the ABI controllable via a compile-time option. I like Olivier's proposal
on using a single option (CONFIG_RTE_NEXT_ABI) to control all of these
changes instead of a separate option per patch set (see
http://dpdk.org/ml/archives/dev/2015-June/019147.html), so I think we
should rework the affected patch sets to use that approach for 2.1.
This is a bad idea. Making ABI dependent on compile time options isn't a
maintainable solution. It breaks the notion of how LIBABIVER is supposed to
work (that is to say you make it impossible to really tell what ABI version you
are building).
The idea was to make LIBABIVER increment dependent of CONFIG_RTE_NEXT_ABI.
So one ABI version number refers always to the same ABI.
Post by Neil Horman
If you have two compile time options that modify the ABI, you
have to burn through 4 possible LIBABIVER version values to accomodate all
possible combinations, and then you need to remember that when you make them
statically applicable.
The idea is to have only 1 compile-time option: CONFIG_RTE_NEXT_ABI.
Your intent when introducing ABI policy was to allow smooth porting of
applications from a DPDK version to another. Right?
The adopted solution was to provide backward compatibility during 1 release.
But there are cases where it's not possible. So the policy was to notice
the future change and wait one release cycle to break the ABI (failing
compatibility goals).
The compile-time option may provide an alternative DPDK packaging when the
ABI backward compatibility cannot be provided (case of mbuf changes).
In such case, it's still possible to upgrade DPDK by providing 2 versions of
DPDK libs. So the existing apps continue to link with the previous ABI and
have the possibility of migrating to the new one.
Another advantage of this approach is that we don't have to wait 1 release
to integrate the changes.
The last advantage is to benefit early of these changes with static libraries.
Hm, ok, thats a bit more reasonable, but it still seems shaky to me.
Implementing an ABI preview option like this implies the notion that, after a
release, you have to remove all the ifdefs that you inserted to create the new
ABI. That seems like an easy task, but it becomes a pain when the ABI delta is
large, and is predicated on the centralization of work effort (that is to say
you need to identify someone to submit the 'remove the NEXT_ABI config ifdefs
from the build' patch every release.
It won't be so huge if we reserve the NEXT_ABI solution to changes which cannot
have easy backward compatibility with the compat macros you introduced.
I feel I can do the job of removing the ifdefs NEXT_ABI after each release.
At the same time, the deprecated API, using the compat macros, will be removed.
I think that is something you can't really predict, as its not an issue of how
stringent we are with its use, but rather a function of how much change
developers want in a given release. That is to say, if you only reserve it for
the most important/urgently needed changes, thats fine, but if you have a
release in which 50 developers want to make urgent and important changes that
breaks ABI, you still have quite a job on your hands to back out the config
changes.

Not to mention the fact that backing those changes out is a manual process.
Post by Thomas Monjalon
Post by Neil Horman
What might be better would be a dpdk-next branch (or even a dpdk-next tree, of
the sort that Thomas Herbert proposed a few weeks ago).
http://dpdk.org/browse/next/dpdk-next/
Awesome, Though I'm not sure thats entirely the right place either. IIRC that
location was intended to be an early integration site that took unreviewed
patches. I think this really calls for a branch from the mainline tree that
exclusively accepts reviewed ABI changing patches, that can then be merged after
the next release
Post by Thomas Monjalon
Post by Neil Horman
Patches that aren't ABI stable can be put on the next-branch/tree in thier
final format. You can delcare the branch unstable (thereby reserving your
right to rebase it). People can use that to preview the next ABI version
(complete with the update LIBABIVER bump), and when you release dpdk-X,
the new ABI for dpdk-X+1 is achieved by simply merging.
Having this tree living would be a nice improvement but it won't provide any
stable (and enough validated) releases to rely on.
I'm not sure I follow you entirely here. If the goal is to find a place to
accept patches that are ABI altering ahead of the main release, why do you need
to provide stable/validated releases? Just base it off the HEAD of the git tree
during the DPDK release X merge window, any testing done in the base branch
should roughly apply, save for functional changes made by the ABI patches you
add in on the branch.

Neil
Thomas Monjalon
2015-06-19 15:51:07 UTC
Permalink
Post by Neil Horman
Post by Thomas Monjalon
Post by Neil Horman
Post by Thomas Monjalon
Post by Neil Horman
Post by O'Driscoll, Tim
For the 2.1 release, I think we should agree to make patches that change
the ABI controllable via a compile-time option. I like Olivier's proposal
on using a single option (CONFIG_RTE_NEXT_ABI) to control all of these
changes instead of a separate option per patch set (see
http://dpdk.org/ml/archives/dev/2015-June/019147.html), so I think we
should rework the affected patch sets to use that approach for 2.1.
This is a bad idea. Making ABI dependent on compile time options isn't a
maintainable solution. It breaks the notion of how LIBABIVER is supposed to
work (that is to say you make it impossible to really tell what ABI version you
are building).
The idea was to make LIBABIVER increment dependent of CONFIG_RTE_NEXT_ABI.
So one ABI version number refers always to the same ABI.
Post by Neil Horman
If you have two compile time options that modify the ABI, you
have to burn through 4 possible LIBABIVER version values to accomodate all
possible combinations, and then you need to remember that when you make them
statically applicable.
The idea is to have only 1 compile-time option: CONFIG_RTE_NEXT_ABI.
Your intent when introducing ABI policy was to allow smooth porting of
applications from a DPDK version to another. Right?
The adopted solution was to provide backward compatibility during 1 release.
But there are cases where it's not possible. So the policy was to notice
the future change and wait one release cycle to break the ABI (failing
compatibility goals).
The compile-time option may provide an alternative DPDK packaging when the
ABI backward compatibility cannot be provided (case of mbuf changes).
In such case, it's still possible to upgrade DPDK by providing 2 versions of
DPDK libs. So the existing apps continue to link with the previous ABI and
have the possibility of migrating to the new one.
Another advantage of this approach is that we don't have to wait 1 release
to integrate the changes.
The last advantage is to benefit early of these changes with static libraries.
Hm, ok, thats a bit more reasonable, but it still seems shaky to me.
Implementing an ABI preview option like this implies the notion that, after a
release, you have to remove all the ifdefs that you inserted to create the new
ABI. That seems like an easy task, but it becomes a pain when the ABI delta is
large, and is predicated on the centralization of work effort (that is to say
you need to identify someone to submit the 'remove the NEXT_ABI config ifdefs
from the build' patch every release.
It won't be so huge if we reserve the NEXT_ABI solution to changes which cannot
have easy backward compatibility with the compat macros you introduced.
I feel I can do the job of removing the ifdefs NEXT_ABI after each release.
At the same time, the deprecated API, using the compat macros, will be removed.
I think that is something you can't really predict, as its not an issue of how
stringent we are with its use, but rather a function of how much change
developers want in a given release. That is to say, if you only reserve it for
the most important/urgently needed changes, thats fine, but if you have a
release in which 50 developers want to make urgent and important changes that
breaks ABI, you still have quite a job on your hands to back out the config
changes.
Not to mention the fact that backing those changes out is a manual process.
Post by Thomas Monjalon
Post by Neil Horman
What might be better would be a dpdk-next branch (or even a dpdk-next tree, of
the sort that Thomas Herbert proposed a few weeks ago).
http://dpdk.org/browse/next/dpdk-next/
Awesome, Though I'm not sure thats entirely the right place either. IIRC that
location was intended to be an early integration site that took unreviewed
patches. I think this really calls for a branch from the mainline tree that
exclusively accepts reviewed ABI changing patches, that can then be merged after
the next release
Post by Thomas Monjalon
Post by Neil Horman
Patches that aren't ABI stable can be put on the next-branch/tree in thier
final format. You can delcare the branch unstable (thereby reserving your
right to rebase it). People can use that to preview the next ABI version
(complete with the update LIBABIVER bump), and when you release dpdk-X,
the new ABI for dpdk-X+1 is achieved by simply merging.
Having this tree living would be a nice improvement but it won't provide any
stable (and enough validated) releases to rely on.
I'm not sure I follow you entirely here. If the goal is to find a place to
accept patches that are ABI altering ahead of the main release, why do you need
to provide stable/validated releases? Just base it off the HEAD of the git tree
during the DPDK release X merge window, any testing done in the base branch
should roughly apply, save for functional changes made by the ABI patches you
add in on the branch.
OK, I didn't get you. So you are saying that the changes for the next release
may be prepared in a branch. Yes, it's possible.
But before stating on practical method to execute the policy, we need to agree
on the policy.

This is my proposal:
- The ABI policy must be better explained: what happens to LIBABIVER and .map
when adding a field, adding or removing a function.
- The file doc/guides/rel_notes/abi.rst must contain the ABI notices but the
policy must be moved to doc/guides/guidelines/compat.rst
- The case "backward compatibility broken" must be replaced by the usage of
CONFIG_RTE_NEXT_ABI while describing cases where it can apply.
- The .map files must be generated in order to make it simpler and allows
the use of CONFIG_RTE_NEXT_ABI.
Thomas F Herbert
2015-06-19 16:13:49 UTC
Permalink
Post by Thomas Monjalon
Post by Neil Horman
Post by Thomas Monjalon
Post by Neil Horman
Post by O'Driscoll, Tim
For the 2.1 release, I think we should agree to make patches that change
the ABI controllable via a compile-time option. I like Olivier's proposal
on using a single option (CONFIG_RTE_NEXT_ABI) to control all of these
changes instead of a separate option per patch set (see
http://dpdk.org/ml/archives/dev/2015-June/019147.html), so I think we
should rework the affected patch sets to use that approach for 2.1.
This is a bad idea. Making ABI dependent on compile time options isn't a
maintainable solution. It breaks the notion of how LIBABIVER is supposed to
work (that is to say you make it impossible to really tell what ABI version you
are building).
The idea was to make LIBABIVER increment dependent of CONFIG_RTE_NEXT_ABI.
So one ABI version number refers always to the same ABI.
Post by Neil Horman
If you have two compile time options that modify the ABI, you
have to burn through 4 possible LIBABIVER version values to accomodate all
possible combinations, and then you need to remember that when you make them
statically applicable.
The idea is to have only 1 compile-time option: CONFIG_RTE_NEXT_ABI.
Your intent when introducing ABI policy was to allow smooth porting of
applications from a DPDK version to another. Right?
The adopted solution was to provide backward compatibility during 1 release.
But there are cases where it's not possible. So the policy was to notice
the future change and wait one release cycle to break the ABI (failing
compatibility goals).
The compile-time option may provide an alternative DPDK packaging when the
ABI backward compatibility cannot be provided (case of mbuf changes).
In such case, it's still possible to upgrade DPDK by providing 2 versions of
DPDK libs. So the existing apps continue to link with the previous ABI and
have the possibility of migrating to the new one.
Another advantage of this approach is that we don't have to wait 1 release
to integrate the changes.
The last advantage is to benefit early of these changes with static libraries.
Hm, ok, thats a bit more reasonable, but it still seems shaky to me.
Implementing an ABI preview option like this implies the notion that, after a
release, you have to remove all the ifdefs that you inserted to create the new
ABI. That seems like an easy task, but it becomes a pain when the ABI delta is
large, and is predicated on the centralization of work effort (that is to say
you need to identify someone to submit the 'remove the NEXT_ABI config ifdefs
from the build' patch every release.
It won't be so huge if we reserve the NEXT_ABI solution to changes which cannot
have easy backward compatibility with the compat macros you introduced.
I feel I can do the job of removing the ifdefs NEXT_ABI after each release.
At the same time, the deprecated API, using the compat macros, will be removed.
Post by Neil Horman
What might be better would be a dpdk-next branch (or even a dpdk-next tree, of
the sort that Thomas Herbert proposed a few weeks ago).
http://dpdk.org/browse/next/dpdk-next/
Thomas, I am sorry if I went quiet for awhile but I was on personal
travel with inconsistent access so I almost missed most of this
discussion about ABI changes.

My understanding of the purpose of the dpdk-next tree is to validate
patches by applying and compiling against a "pull" from the main dpdk
tree. I think a good way to handle ABI change while effectively using
the dpdk-next might be to do as follows:

Create a specific branch for the new ABI such as 2.X in the main dpdk
tree. Once that 2.X branch is created, dpdk-next would mirror the 2.X
branch along with master.

Since, dpdk-next would also have the 2.X branch that is in the main dpdk
tree, submitted patches could be applied to either the main branch or
the new-ABI 2.X branch. Providing that patch submitters make it clear
whether a submitted patch is for the new ABI or the old ABI, dpdk-next
could continue to validate the patches for either the main branch or the
new ABI 2.X branch.
Post by Thomas Monjalon
Post by Neil Horman
Patches that aren't ABI stable can be put on the next-branch/tree in thier
final format. You can delcare the branch unstable (thereby reserving your
right to rebase it). People can use that to preview the next ABI version
(complete with the update LIBABIVER bump), and when you release dpdk-X,
the new ABI for dpdk-X+1 is achieved by simply merging.
Having this tree living would be a nice improvement but it won't provide any
stable (and enough validated) releases to rely on.
Thomas Monjalon
2015-06-19 17:02:20 UTC
Permalink
Post by Thomas F Herbert
Post by Thomas Monjalon
Post by Neil Horman
Post by Thomas Monjalon
Post by Neil Horman
Post by O'Driscoll, Tim
For the 2.1 release, I think we should agree to make patches that change
the ABI controllable via a compile-time option. I like Olivier's proposal
on using a single option (CONFIG_RTE_NEXT_ABI) to control all of these
changes instead of a separate option per patch set (see
http://dpdk.org/ml/archives/dev/2015-June/019147.html), so I think we
should rework the affected patch sets to use that approach for 2.1.
This is a bad idea. Making ABI dependent on compile time options isn't a
maintainable solution. It breaks the notion of how LIBABIVER is supposed to
work (that is to say you make it impossible to really tell what ABI version you
are building).
The idea was to make LIBABIVER increment dependent of CONFIG_RTE_NEXT_ABI.
So one ABI version number refers always to the same ABI.
Post by Neil Horman
If you have two compile time options that modify the ABI, you
have to burn through 4 possible LIBABIVER version values to accomodate all
possible combinations, and then you need to remember that when you make them
statically applicable.
The idea is to have only 1 compile-time option: CONFIG_RTE_NEXT_ABI.
Your intent when introducing ABI policy was to allow smooth porting of
applications from a DPDK version to another. Right?
The adopted solution was to provide backward compatibility during 1 release.
But there are cases where it's not possible. So the policy was to notice
the future change and wait one release cycle to break the ABI (failing
compatibility goals).
The compile-time option may provide an alternative DPDK packaging when the
ABI backward compatibility cannot be provided (case of mbuf changes).
In such case, it's still possible to upgrade DPDK by providing 2 versions of
DPDK libs. So the existing apps continue to link with the previous ABI and
have the possibility of migrating to the new one.
Another advantage of this approach is that we don't have to wait 1 release
to integrate the changes.
The last advantage is to benefit early of these changes with static libraries.
Hm, ok, thats a bit more reasonable, but it still seems shaky to me.
Implementing an ABI preview option like this implies the notion that, after a
release, you have to remove all the ifdefs that you inserted to create the new
ABI. That seems like an easy task, but it becomes a pain when the ABI delta is
large, and is predicated on the centralization of work effort (that is to say
you need to identify someone to submit the 'remove the NEXT_ABI config ifdefs
from the build' patch every release.
It won't be so huge if we reserve the NEXT_ABI solution to changes which cannot
have easy backward compatibility with the compat macros you introduced.
I feel I can do the job of removing the ifdefs NEXT_ABI after each release.
At the same time, the deprecated API, using the compat macros, will be removed.
Post by Neil Horman
What might be better would be a dpdk-next branch (or even a dpdk-next tree, of
the sort that Thomas Herbert proposed a few weeks ago).
http://dpdk.org/browse/next/dpdk-next/
Thomas, I am sorry if I went quiet for awhile but I was on personal
travel with inconsistent access so I almost missed most of this
discussion about ABI changes.
My understanding of the purpose of the dpdk-next tree is to validate
patches by applying and compiling against a "pull" from the main dpdk
tree. I think a good way to handle ABI change while effectively using
Create a specific branch for the new ABI such as 2.X in the main dpdk
tree. Once that 2.X branch is created, dpdk-next would mirror the 2.X
branch along with master.
Since, dpdk-next would also have the 2.X branch that is in the main dpdk
tree, submitted patches could be applied to either the main branch or
the new-ABI 2.X branch. Providing that patch submitters make it clear
whether a submitted patch is for the new ABI or the old ABI, dpdk-next
could continue to validate the patches for either the main branch or the
new ABI 2.X branch.
What is the benefit of a new-ABI branch in the -next tree?

The goal of this discussion is to find a consensus on ABI policy to
smoothly integrate new features without forcing users of shared libraries
to re-build their application when upgrading DPDK, and let them do the
transition before the next upgrade.
Thomas F Herbert
2015-06-19 17:57:35 UTC
Permalink
Post by Thomas Monjalon
Post by Thomas F Herbert
Post by Thomas Monjalon
Post by Neil Horman
Post by Thomas Monjalon
Post by Neil Horman
Post by O'Driscoll, Tim
For the 2.1 release, I think we should agree to make patches that change
the ABI controllable via a compile-time option. I like Olivier's proposal
on using a single option (CONFIG_RTE_NEXT_ABI) to control all of these
changes instead of a separate option per patch set (see
http://dpdk.org/ml/archives/dev/2015-June/019147.html), so I think we
should rework the affected patch sets to use that approach for 2.1.
This is a bad idea. Making ABI dependent on compile time options isn't a
maintainable solution. It breaks the notion of how LIBABIVER is supposed to
work (that is to say you make it impossible to really tell what ABI version you
are building).
The idea was to make LIBABIVER increment dependent of CONFIG_RTE_NEXT_ABI.
So one ABI version number refers always to the same ABI.
Post by Neil Horman
If you have two compile time options that modify the ABI, you
have to burn through 4 possible LIBABIVER version values to accomodate all
possible combinations, and then you need to remember that when you make them
statically applicable.
The idea is to have only 1 compile-time option: CONFIG_RTE_NEXT_ABI.
Your intent when introducing ABI policy was to allow smooth porting of
applications from a DPDK version to another. Right?
The adopted solution was to provide backward compatibility during 1 release.
But there are cases where it's not possible. So the policy was to notice
the future change and wait one release cycle to break the ABI (failing
compatibility goals).
The compile-time option may provide an alternative DPDK packaging when the
ABI backward compatibility cannot be provided (case of mbuf changes).
In such case, it's still possible to upgrade DPDK by providing 2 versions of
DPDK libs. So the existing apps continue to link with the previous ABI and
have the possibility of migrating to the new one.
Another advantage of this approach is that we don't have to wait 1 release
to integrate the changes.
The last advantage is to benefit early of these changes with static libraries.
Hm, ok, thats a bit more reasonable, but it still seems shaky to me.
Implementing an ABI preview option like this implies the notion that, after a
release, you have to remove all the ifdefs that you inserted to create the new
ABI. That seems like an easy task, but it becomes a pain when the ABI delta is
large, and is predicated on the centralization of work effort (that is to say
you need to identify someone to submit the 'remove the NEXT_ABI config ifdefs
from the build' patch every release.
It won't be so huge if we reserve the NEXT_ABI solution to changes which cannot
have easy backward compatibility with the compat macros you introduced.
I feel I can do the job of removing the ifdefs NEXT_ABI after each release.
At the same time, the deprecated API, using the compat macros, will be removed.
Post by Neil Horman
What might be better would be a dpdk-next branch (or even a dpdk-next tree, of
the sort that Thomas Herbert proposed a few weeks ago).
http://dpdk.org/browse/next/dpdk-next/
Thomas, I am sorry if I went quiet for awhile but I was on personal
travel with inconsistent access so I almost missed most of this
discussion about ABI changes.
My understanding of the purpose of the dpdk-next tree is to validate
patches by applying and compiling against a "pull" from the main dpdk
tree. I think a good way to handle ABI change while effectively using
Create a specific branch for the new ABI such as 2.X in the main dpdk
tree. Once that 2.X branch is created, dpdk-next would mirror the 2.X
branch along with master.
Since, dpdk-next would also have the 2.X branch that is in the main dpdk
tree, submitted patches could be applied to either the main branch or
the new-ABI 2.X branch. Providing that patch submitters make it clear
whether a submitted patch is for the new ABI or the old ABI, dpdk-next
could continue to validate the patches for either the main branch or the
new ABI 2.X branch.
What is the benefit of a new-ABI branch in the -next tree?
I don't think that there is any specific benefit to an new-ABI branch in
the dpdk-next tree. I was responding to the suggestion above and perhaps
I missread it. It sounded like what was being proposed was to use the
dpdk-next tree specifically for pre-integration of new-ABI. I don't
think this is of any benefit either.

However if it should be decided to integrate new-ABI patches in a branch
of dpdk rather then in a separate new-ABI tree, then net-next can
"mirror" that branch along with the master branch so patches can be
smoke tested whether they are submitted to the master or to the new-ABI
branch.
Post by Thomas Monjalon
The goal of this discussion is to find a consensus on ABI policy to
smoothly integrate new features without forcing users of shared libraries
to re-build their application when upgrading DPDK, and let them do the
transition before the next upgrade.
I understand this and I think it is a good suggestion to have a
mechanism to ease the transition.
Loading...