Reintroducing FFmpeg to Debian

[Andreas Cadhalpun]

Hi all,

some of you may have noticed a weird ffmpeg package in the NEW queue[1].
Let me explain:

In 2011 Libav[2] was forked from FFmpeg[3]. It was a time of great
uncertainty, the fork happened with much drama that didn't help making a
technical cut, and at that peculiar time Debian switched to Libav.

Since then the two projects evolved differently, and we have now a
clearer view.

Some short answers to questions you might have now:

* Why is FFmpeg needed in Debian?
- It has features our users are asking for (mostly support for more
codecs, formats and filters)[4-6].
- Some applications break when built against Libav on Debian,
because they are developed using FFmpeg[7-10].
- There are issues[11-12] in Libav's command line tools, that can't
be reproduced with FFmpeg's tools.
- It has a big and active user and developer community. Those of
them who want to use Debian currently need to install FFmpeg from
third parties or compile their own version from source.

* Do you intend to replace Libav by FFmpeg in Debian?
No, there is no need to replace anything as long as it is maintained.
Currently the main goal is to give multimedia maintainers a choice
between the two sets of libraries to link against, and our users the
choice to use the 'ffmpeg' utility. That is possible, because the
packages are co-installable. (Only the *-dev packages conflict.)

* But I thought they were forks, why don't you just create conflicting
packages?
- Because it can't really work! If we do that, packages built with
FFmpeg won't be installable next to packages built with Libav
unless we have full binary compatibility.
- Binary compatibility can only be achieved with some tradeoffs:
a) Not all soversions of the libraries match, so we would have
to patch that away.
b) FFmpeg would have to be compiled with the configure option
--enable-incompatible-libav-abi, resulting in less tested
code paths.
c) FFmpeg and Libav would need to be updated at the same time.
d) The biggest problem is that this would allow applications only
to use the minimal set of the ABI supported by both.

* I do not believe you, explain that voodoo to me: How is it that it
won't break all of Debian and make kittens cry?
(aka: How is FFmpeg packaged for Debian?)
- We built the packages in a way that avoids filename conflicts.
The sonames of the FFmpeg libraries are suffixed with '-ffmpeg',
e.g. libavcodec-ffmpeg.so.55 instead of libavcodec.so.55.
This also means that only if a package uses pkg-config to detect
the correct linker flags, you can simply install e.g. the
libavcodec-ffmpeg-dev package and it will work transparently.
(About half the packages build with no further change, see
stats below.)
- From a user point of view, the tools have different names anyway,
e.g. ffmpeg and avconv, except for qt-faststart, which is
therefore packaged in a separate binary package that diverts
the Libav version to qt-faststart.libav.
Yes, you can have /usr/bin/ffmpeg and /usr/bin/avconv at the same
time without conflicts.
- The development packages have to conflict, because they provide
header files (and pkg-config files) with identical names in
identical locations.
- To avoid potential problems when a program is linked against
FFmpeg libraries and other libraries, which in turn are linked
against Libav, the symbol versions are changed to e.g.
LIBAVCODEC_FFMPEG_55 instead of LIBAVCODEC_55.

* Ok, let's say I'm a multimedia maintainer and want to try out
building my package against your ffmpeg, what should I do?
- If your package uses pkg-config, which is commonly the case, all
you have to do is to replace all lib{av,swscale,postproc}*-dev
build-dependencies by lib{av,swscale,postproc}*-ffmpeg-dev.
You can keep the Libav build-dependencies as alternatives.
- In the (odd) case your upstream doesn't use pkg-config
(52 packages), it's probably a good idea to start using it, so
that the program can be easily built with both.
The patches are usually quite simple as you can see in this
example:

--- squeezelite-1.6.orig/Makefile
+++ squeezelite-1.6/Makefile
@@ -26,7 +26,7 @@ LINK_ALSA = -lasound
LINK_PORTAUDIO = -lportaudio

LINKALL = -lFLAC -lmad -lvorbisfile -lfaad -lmpg123
-LINKALL_FF = -lavcodec -lavformat -lavutil
+LINKALL_FF = $(shell pkg-config --libs libavcodec libavformat
libavutil)
LINKALL_RESAMPLE = -lsoxr

DEPS = squeezelite.h slimproto.h

Patches for packages using Autoconf or Cmake are similarly
straight-forward.
- Sometimes other minor adjustments are needed. (13 packages)
- There are only 2 packages (opencv and ffms2) that would trigger a
needed transition, but that would be quite manageable as only 4
packages (3 on opencv and 1 on ffms2) depend on both libav*-dev
and the transition-needed library (libopencv-highgui-dev and
libffms2-dev).
Note that the dependencies of libopencv-highgui-dev on
libavcodec-dev, libavformat-dev and libswcale-dev seem
to be unnecessary[13].

* Does it make sense for me to switch my package?
The rule of thumb is, if your upstream uses FFmpeg for development
you probably want to switch to using it, too.

Of the 108 current reverse-build-dependencies of src:libav,
50 only need build-dependency adjustments,
50 can be patched to work with FFmpeg already,
5 can only be patched once FFmpeg is through NEW,
2 FTBFS since a long time and are not in testing [14-15] and
1 currently FTBFS for unrelated reasons [16].

Attached results.txt contains a per package list.

Any maintainer interested in switching should get in touch, and we'll
gladly help preparing the transition. When we're done dealing with those
early birds we intend to file wishlist bugs with patches on other
packages to help maintainers transition if they want to. (We already
have patches for the affected packages.)

The FFmpeg version currently in NEW has been there for more than
2 months and is thus outdated. If you want to test the current
packages, you can build them from the repository on Alioth [17]
(e.g. using git-buildpackage).

Furthermore, we'd like to move the FFmpeg packaging under the umbrella
of the pkg-multimedia team, because this would facilitate future FFmpeg
transitions.

Best regards,
Andreas (on behalf of the FFmpeg team)


1: https://ftp-master.debian.org/new/ffmpeg_7:2.2.1-1.html
2: https://libav.org/
3: https://ffmpeg.org/
4: http://lucy.pkh.me/diff/
5: https://bugs.debian.org/707476
6: https://bugs.debian.org/694616
7: https://bugs.debian.org/732159
8: https://bugs.debian.org/741170
9: https://bugs.debian.org/742896
10: https://bugs.debian.org/745060
11: https://bugs.debian.org/692876
12: https://bugs.debian.org/753923
13: https://bugs.debian.org/745934
14: https://bugs.debian.org/723099
15: https://bugs.debian.org/720796
16: https://bugs.debian.org/747536
17: https://anonscm.debian.org/gitweb/?p=collab-maint/ffmpeg.git;a=summary

[Reinhard Tartler]

On Sun, Jul 27, 2014 at 7:20 PM, Andreas Cadhalpun

Post by Andreas Cadhalpun
* Does it make sense for me to switch my package?
The rule of thumb is, if your upstream uses FFmpeg for development
you probably want to switch to using it, too.

In [1], Moritz from the security team clearly stated that he is more
than uncomfortable with having more than one copy of libavcodec in
debian/testing. In consequence this means that any package that builds
against the ffmpeg packages currently in NEW won't make it into
testing either. I am therefore surprised about the given answer to the
question above.

I think it would be best if ftp-master left the ffmpeg package in NEW
until an answer to this problem has been found.

[1] https://lists.debian.org/debian-devel/2014/02/msg00668.html

Post by Andreas Cadhalpun
The FFmpeg version currently in NEW has been there for more than
2 months and is thus outdated. If you want to test the current
packages, you can build them from the repository on Alioth [17]
(e.g. using git-buildpackage).
Furthermore, we'd like to move the FFmpeg packaging under the umbrella
of the pkg-multimedia team, because this would facilitate future FFmpeg
transitions.

I am curious why this is your first email about this matter to
pkg-multimedia, and why do you write this email only now?

Moreover, I am curious why I haven't seen you working on libavcodec
bugs in Debian before, and why do you believe you can do a better job
with the ffmpeg package currently on NEW?

[Marco d'Itri]

Post by Reinhard Tartler
Moreover, I am curious why I haven't seen you working on libavcodec
bugs in Debian before, and why do you believe you can do a better job
with the ffmpeg package currently on NEW?

Why should he work on libavcodec when he (along with many other people)
wants ffmpeg instead?

[Norbert Preining]

Hi,

Post by Reinhard Tartler
In [1], Moritz from the security team clearly stated that he is more
than uncomfortable with having more than one copy of libavcodec in
debian/testing. In consequence this means that any package that builds
against the ffmpeg packages currently in NEW won't make it into
testing either. I am therefore surprised about the given answer to the

"More than uncomfortable" does not mean "will not be included"

Post by Reinhard Tartler
I think it would be best if ftp-master left the ffmpeg package in NEW
until an answer to this problem has been found.

Why, only because libav gets a more powerful and efficient
competition?

Post by Reinhard Tartler
I am curious why this is your first email about this matter to
pkg-multimedia, and why do you write this email only now?
Moreover, I am curious why I haven't seen you working on libavcodec
bugs in Debian before, and why do you believe you can do a better job
with the ffmpeg package currently on NEW?

As Marco said, why should he fix bugs in av which have been fixed since
ages in ffmeg in many (most?) cases?

I am all for having a good ffmpeg set of cmd line progs and libs in
Debian. It is too long that this sad and useless split was made.

Norbert

------------------------------------------------------------------------
PREINING, Norbert http://www.preining.info
JAIST, Japan TeX Live & Debian Developer
GPG: 0x860CDC13 fp: F7D8 A928 26E3 16A1 9FA0 ACF0 6CAC A448 860C DC13
------------------------------------------------------------------------

[Henrique de Moraes Holschuh]

Post by Norbert Preining

Post by Reinhard Tartler
In [1], Moritz from the security team clearly stated that he is more
than uncomfortable with having more than one copy of libavcodec in
debian/testing. In consequence this means that any package that builds
against the ffmpeg packages currently in NEW won't make it into
testing either. I am therefore surprised about the given answer to the

"More than uncomfortable" does not mean "will not be included"

Yes, it does.

Someone will have to convince the security team somehow, likely by offering
to do the work themselves _and_ convincing them that these new members will
be around for long enough.

However:

The change in Debian-specific symbol versioning and sonames being done to
ffmpeg so that it is co-installable with libav *is* a problem.

It has to be done in coordination with the Canonical guys, so that both
Debian and Ubuntu do the same thing re. ffmpeg sonames and symbol
versioning. Otherwise, the ffmpeg packages will be of very limited use
(useless to run third-party binary-only games ;-p).

I understand perfectly that the soname and symbol versioning clash with
libav is not ffmpeg's fault, but that's water (well, sewage) under the
bridge. We have to deal with it. Here's an alternative proposal that
should be less painful [to our users] in the long run:

You need one of the two upstreams to do a *large* major soname bump (at
least one order of magnitude higher than what they're currently using), so
that both projects can keep evolving with little chance of soname clashes.

Symbol versioning will take care of the rest, since both libs carry over
their major soname into the symbol version. As it was done upstream,
cross-distro/third-party compatibility problems are not increased.

Debian will have to package this new "bumped" upstream release, and get rid
of anything built against the old one. It will be easier for Debian if it
is ffmpeg upstream that does the soname bump, otherwise we're talking about
a huge number of binNMUs.

But this is all academic if the security team is not prepared to deal with
both libav and ffmpeg at the same time. That effectively forces a choice of
either libav, or ffmpeg, and not both.

[Niv Sardi]

Post by Henrique de Moraes Holschuh
The change in Debian-specific symbol versioning and sonames being done to
ffmpeg so that it is co-installable with libav *is* a problem.
It has to be done in coordination with the Canonical guys, so that both
Debian and Ubuntu do the same thing re. ffmpeg sonames and symbol
versioning. Otherwise, the ffmpeg packages will be of very limited use
(useless to run third-party binary-only games ;-p).

Not really, ffmpeg is packaged as a secondary multimedia library, the
default one still being libav. If these packages get traction, we can think
of moving ffmpeg to be the default (and ship if we wish libav with the
soname change).

The package will be of use for the ffmpeg command line tools, and for the
maintainers that decide to make the switch.

For now, your binary third party games will have to link against libav.

Post by Henrique de Moraes Holschuh
I understand perfectly that the soname and symbol versioning clash with
libav is not ffmpeg's fault, but that's water (well, sewage) under the
bridge. We have to deal with it. Here's an alternative proposal that
You need one of the two upstreams to do a *large* major soname bump (at
least one order of magnitude higher than what they're currently using), so
that both projects can keep evolving with little chance of soname clashes.
Symbol versioning will take care of the rest, since both libs carry over
their major soname into the symbol version. As it was done upstream,
cross-distro/third-party compatibility problems are not increased.
Debian will have to package this new "bumped" upstream release, and get rid
of anything built against the old one. It will be easier for Debian if it
is ffmpeg upstream that does the soname bump, otherwise we're talking about
a huge number of binNMUs.

That's been discussed and ruled out in favour of not overstepping libav's
namespace.

Post by Henrique de Moraes Holschuh
But this is all academic if the security team is not prepared to deal with
both libav and ffmpeg at the same time. That effectively forces a choice of
either libav, or ffmpeg, and not both.

That is premature, let's deal with this issue when we have more data.

[Andreas Cadhalpun]

Post by Henrique de Moraes Holschuh

Post by Norbert Preining

Post by Reinhard Tartler
In [1], Moritz from the security team clearly stated that he is more
than uncomfortable with having more than one copy of libavcodec in
debian/testing. In consequence this means that any package that builds
against the ffmpeg packages currently in NEW won't make it into
testing either. I am therefore surprised about the given answer to the

"More than uncomfortable" does not mean "will not be included"

Yes, it does.
Someone will have to convince the security team somehow, likely by offering
to do the work themselves _and_ convincing them that these new members will
be around for long enough.

Michael Niedermayer from FFmpeg upstream volunteered "to help with any
future security issues in FFmpeg packages in debian" [1].

Post by Henrique de Moraes Holschuh
The change in Debian-specific symbol versioning and sonames being done to
ffmpeg so that it is co-installable with libav *is* a problem.
It has to be done in coordination with the Canonical guys, so that both
Debian and Ubuntu do the same thing re. ffmpeg sonames and symbol
versioning. Otherwise, the ffmpeg packages will be of very limited use
(useless to run third-party binary-only games ;-p).

I don't think coordination with Ubuntu will be a problem.
In comment #7 in the corresponding bug at launchpad [2] Dimitri John
Ledkov wrote that Ubuntu won't introduce FFmpeg on it's on, but instead:
"If you wish to see a supported ffmpeg stack in both Debian and Ubuntu,
please become a developer and start maintaining it in Debian."

Best regards,
Andreas


1: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=729203#528
2: https://bugs.launchpad.net/ubuntu/+source/libav/+bug/1263278

[Michael Niedermayer]

Post by Andreas Cadhalpun

Post by Henrique de Moraes Holschuh

Post by Norbert Preining

Post by Reinhard Tartler
In [1], Moritz from the security team clearly stated that he is more
than uncomfortable with having more than one copy of libavcodec in
debian/testing. In consequence this means that any package that builds
against the ffmpeg packages currently in NEW won't make it into
testing either. I am therefore surprised about the given answer to the

"More than uncomfortable" does not mean "will not be included"

Yes, it does.
Someone will have to convince the security team somehow, likely by offering
to do the work themselves _and_ convincing them that these new members will
be around for long enough.

Michael Niedermayer from FFmpeg upstream volunteered "to help with
any future security issues in FFmpeg packages in debian" [1].

Yes, i do!

[...]

[Dimitri John Ledkov]

On 28 July 2014 15:05, Andreas Cadhalpun

Post by Andreas Cadhalpun

Post by Henrique de Moraes Holschuh

Post by Norbert Preining

Post by Reinhard Tartler
In [1], Moritz from the security team clearly stated that he is more
than uncomfortable with having more than one copy of libavcodec in
debian/testing. In consequence this means that any package that builds
against the ffmpeg packages currently in NEW won't make it into
testing either. I am therefore surprised about the given answer to the

"More than uncomfortable" does not mean "will not be included"

Yes, it does.
Someone will have to convince the security team somehow, likely by offering
to do the work themselves _and_ convincing them that these new members will
be around for long enough.

Michael Niedermayer from FFmpeg upstream volunteered "to help with any
future security issues in FFmpeg packages in debian" [1].

Post by Henrique de Moraes Holschuh
The change in Debian-specific symbol versioning and sonames being done to
ffmpeg so that it is co-installable with libav *is* a problem.
It has to be done in coordination with the Canonical guys, so that both
Debian and Ubuntu do the same thing re. ffmpeg sonames and symbol
versioning. Otherwise, the ffmpeg packages will be of very limited use
(useless to run third-party binary-only games ;-p).

I don't think coordination with Ubuntu will be a problem.
In comment #7 in the corresponding bug at launchpad [2] Dimitri John Ledkov
"If you wish to see a supported ffmpeg stack in both Debian and Ubuntu,
please become a developer and start maintaining it in Debian."

I don't have an opinion about ffmpeg vs libav, apart from how hard the
soname transitions are, especially in ubuntu where we somehow ended up
with ex-multimedia packages around that either never were in debian,
or have been long removed from testing and/or unstable. Thankfully, we
have worked to make sure libav is in universe only and thus is not a
security maintenance burden. Nonetheless, libav10 transition is still
not complete in utopic today. I haven't checked, but now abi
compatible/incompatible the two stacks are? cause it would be a pain
if they are not drop in replacements, and it would also be a pain if
higher up packages link-in both ffmpeg & libav and some clashing
symbols are present... and people start requesting to have build
variants against both. Has a rebuild of all deps been done? How many
build failures there are? (On both debian & ubuntu, ideally) Is
hardening flags / toolchain enabled in both, or either of the two?

[Marco d'Itri]

Post by Dimitri John Ledkov
security maintenance burden. Nonetheless, libav10 transition is still
not complete in utopic today. I haven't checked, but now abi
compatible/incompatible the two stacks are? cause it would be a pain

They really are not, this was explained in detail in the first message.

Post by Dimitri John Ledkov
if they are not drop in replacements, and it would also be a pain if
higher up packages link-in both ffmpeg & libav and some clashing
symbols are present...

This is why the new ffmpeg will use different symbols. Again, read the
first message.

Post by Dimitri John Ledkov
and people start requesting to have build variants against both.

I don't think so. The harsh reality is that the only people who love
libav are the libav maintainers, so I expect that once ffmpeg will be
available most maintainers will just switch their packages to it.

Post by Dimitri John Ledkov
Has a rebuild of all deps been done? How many

Yes, the data was in the first message.

[IOhannes m zmölnig (Debian/GNU)]

Post by Marco d'Itri

Post by Dimitri John Ledkov
if they are not drop in replacements, and it would also be a
pain if

Post by Dimitri John Ledkov
higher up packages link-in both ffmpeg & libav and some
clashing symbols are present...

This is why the new ffmpeg will use different symbols. Again, read
the first message.

according to the first message, this is *not* true.
the packages will have different libraries-names / sonames, but this
does not mean that they don't have clashing symbols.
if both library foo (/usr/lib/libfoo.so.3.21) and library bar
(/usr/lib/i386-linux-gnu/libbar.so.4.1) export the symbol "knarzifax",
then how do you make sure that an application that is linked against
both libraries for different functionality always chooses the korrect
"knarzifax"?

this becomes a real world issue, as soon as plugins are involved
(which i find a common practice to access multimedia frameworks).
application "flurp" has a both "flurp-plugin-libav" and
"flurp-plugin-ffmpeg" installed.
whichever plugin is loaded first, will pull in a library that shadows
the symbol "knarzifax" for the *other* plugin.

fgamsdr
IOhannes

[Marco d'Itri]

Post by IOhannes m zmölnig (Debian/GNU)

Post by Marco d'Itri
This is why the new ffmpeg will use different symbols. Again, read
the first message.

according to the first message, this is *not* true.

It is:

- To avoid potential problems when a program is linked against
FFmpeg libraries and other libraries, which in turn are linked
against Libav, the symbol versions are changed to e.g.
LIBAVCODEC_FFMPEG_55 instead of LIBAVCODEC_55.

https://lists.debian.org/debian-devel/2014/07/msg01010.html

[Andreas Cadhalpun]

Hi Dimitri,

Post by Dimitri John Ledkov
I don't have an opinion about ffmpeg vs libav, apart from how hard the
soname transitions are, especially in ubuntu where we somehow ended up
with ex-multimedia packages around that either never were in debian,
or have been long removed from testing and/or unstable.

There are only 6 additional reverse-build-dependencies of src:libav in
utopic. Two build against lib*-ffmpeg-dev without further changes, one
needs a simple patch to use pkg-config, one needs a patch to adapt to
newer API (also needed for Libav 10), one is BD-uninstallable and one
fails for unrelated reasons, but its build-dependencies on libav*-dev
seem to be unnecessary anyway.

Per package list:

alsa-plugins-extra: OK
bombono-dvd: PATCH CodecID
dvdstyler: Unmet build dependencies: libwxsvg-dev (>= 2:1.0.9)
gstreamer-vaapi: error: unsupported GStreamer API version 1.4
kffmpegthumbnailer: OK
libdlna: PATCH pkg-config

The patches are attached to this mail.

Post by Dimitri John Ledkov
Thankfully, we
have worked to make sure libav is in universe only and thus is not a
security maintenance burden. Nonetheless, libav10 transition is still
not complete in utopic today.

Is bombono-dvd the last blocker?

Post by Dimitri John Ledkov
I haven't checked, but now abi
compatible/incompatible the two stacks are? cause it would be a pain
if they are not drop in replacements, and it would also be a pain if
higher up packages link-in both ffmpeg & libav and some clashing
symbols are present...

As Marco already wrote, FFmpeg is packaged to be ABI incompatible with
Libav, having different sonames and different symbol versions.

Post by Dimitri John Ledkov
and people start requesting to have build
variants against both.

This could theoretically be done, but I don't think anyone wants to
maintain such a thing, especially because it would need two different
source packages, as the development packages of FFmpeg and Libav have to
conflict.

Post by Dimitri John Ledkov
Has a rebuild of all deps been done? How many
build failures there are? (On both debian & ubuntu, ideally) Is
hardening flags / toolchain enabled in both, or either of the two?

I did a rebuild of all the reverse-build-dependencies in sid and the
results can be found in my original mail.
For Ubuntu, see the beginning of this mail.

I'm not sure what you want to know about hardening.
The packages are otherwise unchanged, so use hardening flags if they
already do.
If that question was about FFmpeg/Libav, then yes, FFmpeg uses
--toolchain=hardened and Libav hardening flags.

Best regards,
Andreas

[Pau Garcia i Quiles]

On Tue, Jul 29, 2014 at 6:10 PM, Andreas Cadhalpun <

Post by Dimitri John Ledkov
I don't have an opinion about ffmpeg vs libav, apart from how hard the

Post by Dimitri John Ledkov
soname transitions are, especially in ubuntu where we somehow ended up
with ex-multimedia packages around that either never were in debian,
or have been long removed from testing and/or unstable.

There are only 6 additional reverse-build-dependencies of src:libav in
utopic. Two build against lib*-ffmpeg-dev without further changes, one
needs a simple patch to use pkg-config, one needs a patch to adapt to newer
API (also needed for Libav 10), one is BD-uninstallable and one fails for
unrelated reasons, but its build-dependencies on libav*-dev seem to be
unnecessary anyway.
alsa-plugins-extra: OK
bombono-dvd: PATCH CodecID
dvdstyler: Unmet build dependencies: libwxsvg-dev (>= 2:1.0.9)
gstreamer-vaapi: error: unsupported GStreamer API version 1.4
kffmpegthumbnailer: OK
libdlna: PATCH pkg-config

In addition to this, I would like to note there is a lot of closed-source
software which uses ffmpeg instead of libav.

Not saying it doesn't exist but I don't know a single piece of
closed-source software which has moved from ffmpeg to libav.

I know, I know "non DFSG-free software, we don't care". Well, I do. E. g.
I'm having trouble with Qt right now because I'm using the commercial SDK
which indirectly uses ffmpeg to provide some codecs on Linux.

[Andreas Cadhalpun]

Hi Reinhard,

Post by Reinhard Tartler
On Sun, Jul 27, 2014 at 7:20 PM, Andreas Cadhalpun

Post by Andreas Cadhalpun
* Does it make sense for me to switch my package?
The rule of thumb is, if your upstream uses FFmpeg for development
you probably want to switch to using it, too.

In [1], Moritz from the security team clearly stated that he is more
than uncomfortable with having more than one copy of libavcodec in
debian/testing.

I discussed this with Moritz in the ITP bug. Moritz ended this
discussion [a], and as I wasn't convinced by his arguments, I continued
my work. If in the end really only one copy is allowed in the next
stable release, I think it should be FFmpeg.

Post by Reinhard Tartler
In consequence this means that any package that builds
against the ffmpeg packages currently in NEW won't make it into
testing either. I am therefore surprised about the given answer to the
question above.

It remains to be seen, what the release team prefers: frustrated users
and developers or both forks in jessie.

Post by Reinhard Tartler
I think it would be best if ftp-master left the ffmpeg package in NEW
until an answer to this problem has been found.

I fail to see how this would help anyone, it only makes testing the
package more difficult. Whether or not the package is acceptable for the
next stable release is not to be decided by the ftp-masters, but rather
by the release team.

Post by Reinhard Tartler
[1] https://lists.debian.org/debian-devel/2014/02/msg00668.html

Post by Andreas Cadhalpun
The FFmpeg version currently in NEW has been there for more than
2 months and is thus outdated. If you want to test the current
packages, you can build them from the repository on Alioth [17]
(e.g. using git-buildpackage).
Furthermore, we'd like to move the FFmpeg packaging under the umbrella
of the pkg-multimedia team, because this would facilitate future FFmpeg
transitions.

I am curious why this is your first email about this matter to
pkg-multimedia, and why do you write this email only now?

In the last discussion on debian-devel it was suggested to get the
FFmpeg packages into experimental first [b], before further discussion,
so I tried to achieve that.

As the package has been in NEW for a rather long time and the freeze is
getting closer, sending this mail now seemed appropriate.

Post by Reinhard Tartler
Moreover, I am curious why I haven't seen you working on libavcodec
bugs in Debian before,

It would be great if I could fix every bug in Debian, but unfortunately
my time is limited. Therefore, when I encounter a problem that cannot
immediately be fixed, I try to work around it. The workaround for
practically all problems I had with the Libav packages in Debian could
be solved by installing FFmpeg binaries from third parties. Therefore I
finally decided to work on a more sustainable solution, i.e. a FFmpeg
package in Debian.

Post by Reinhard Tartler
and why do you believe you can do a better job
with the ffmpeg package currently on NEW?

It is a lot more likely that I work on fixing a bug that affects me, if
there is no easy workaround.

Best regards,
Andreas


a: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=729203#568
b: https://lists.debian.org/debian-devel/2014/02/msg00714.html

[Julien Cristau]

Post by Andreas Cadhalpun
Hi Reinhard,

Post by Reinhard Tartler
On Sun, Jul 27, 2014 at 7:20 PM, Andreas Cadhalpun

Post by Andreas Cadhalpun
* Does it make sense for me to switch my package?
The rule of thumb is, if your upstream uses FFmpeg for development
you probably want to switch to using it, too.

In [1], Moritz from the security team clearly stated that he is more
than uncomfortable with having more than one copy of libavcodec in
debian/testing.

I discussed this with Moritz in the ITP bug. Moritz ended this discussion
[a], and as I wasn't convinced by his arguments, I continued my work. If in
the end really only one copy is allowed in the next stable release, I think
it should be FFmpeg.

Post by Reinhard Tartler
In consequence this means that any package that builds
against the ffmpeg packages currently in NEW won't make it into
testing either. I am therefore surprised about the given answer to the
question above.

It remains to be seen, what the release team prefers: frustrated users and
developers or both forks in jessie.

The release team is likely to let the people involved in multimedia foo
fight it out among themselves and pick a winner. We're not going to
ship both and hand that mess over to the security team.

Cheers,
Julien

[Alessio Treglia]

Ciao,

Post by Julien Cristau
The release team is likely to let the people involved in multimedia foo
fight it out among themselves and pick a winner. We're not going to
ship both and hand that mess over to the security team.

Personally I don't feel like dropping libav in favor of ffmpeg now at
this stage. It's too late for Jessie.
Rather I'd suggest to start reconsidering such switch for Jessie+1.

Cheers.

[Marco d'Itri]

Post by Alessio Treglia
Personally I don't feel like dropping libav in favor of ffmpeg now at
this stage. It's too late for Jessie.

Except that, for a lot of the depending packages, there would be an
immediate benefit in the number of bugs fixed.

Personally I feel that we have inflicted libav on our users for way more
time than it was sensible to do.

[Alessio Treglia]

On Mon, Jul 28, 2014 at 12:12 PM, "IOhannes m zmölnig (Debian/GNU)"

Post by Marco d'Itri
Except that, for a lot of the depending packages, there would be an
immediate benefit in the number of bugs fixed.

at least in theory.

Plus I would definitely appreciate to see some bug stats supporting
such a theory.

Cheers.

(IOhannes et Multimedia guys, please let's keep debian-devel in the
loop, I feel this is much more of general interest than a thing that
needs to be addressed internally in pkg-multimedia)

[Andreas Cadhalpun]

Post by Alessio Treglia
On Mon, Jul 28, 2014 at 12:12 PM, "IOhannes m zmölnig (Debian/GNU)"

Post by Marco d'Itri
Except that, for a lot of the depending packages, there would be an
immediate benefit in the number of bugs fixed.

at least in theory.

Plus I would definitely appreciate to see some bug stats supporting
such a theory.

My original mail mentioned some examples.

Once FFmpeg is in the archive, each maintainer of a multimedia package
could test build it against FFmpeg and see which, if any, of the bugs
reported against said package vanish.

Best regards,
Andreas

[IOhannes m zmölnig (Debian/GNU)]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

(resending, to keep debian-devel and the bug-report in the loop)

personally i would welcome if both libav and ffmpeg could co-exist
within Debian¹.
as i see it, libav and ffmpeg have diverged, and as such i would like
to have the choice which one to use.

Post by Marco d'Itri

Post by Alessio Treglia
Personally I don't feel like dropping libav in favor of ffmpeg
now at this stage.

+ 1
i don't think that dropping libav is appropriate at all.

Post by Marco d'Itri
Except that, for a lot of the depending packages, there would be
an immediate benefit in the number of bugs fixed.

at least in theory.

Post by Marco d'Itri
Personally I feel that we have inflicted libav on our users for
way more time than it was sensible to do.

i would appreciate it, if you (and anybody else) used a less flammable
&| touchy language.


fgmadr
IOhannes



¹ but then i'm not a member of the security team :-)

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1
Comment: Using GnuPG with Icedove - http://www.enigmail.net/

iQIcBAEBCAAGBQJT1jRXAAoJELZQGcR/ejb45GYP/2a06m3B6PRGyjV6oGS1xwDg
0if/Lssn500F8yjrYFKnexKGZg6xncKVKJ+NJncX3pIWMKu/fOXKJusC5Z5eMdvg
Ecruo7sXBojUnUxtaibGExkjdCWHv4wC/xwx/gQVUg3ijQGr5CQgZKXRPzf6dAG5
Sc4KS7w1SBtgLWaKvsOVhljSB39lye1cUk8vgkPkvSytJPiFMo1QSCDlbNz5JGbf
4c8viga5W9KCH5zMLzZTRQOkiPQpZMPsd/l220YX6ADwlBhnG/yRFBx7SBOnVDYb
BIWb4MFrsCikzC5gJrJZdVAkB96AWOWR6J8N0s8LI2Y1ZwOIM4nJB1FNeQvFRaJI
xe5p3dTI5DS7Kvc6i4LjKcO5m1EdZXeS1vV/OMDrLtgpfDC7pfhn3lImaYMPGCpA
60GNGo/PnbUMWGT3Z5JCeX/Q59X53d8DrW7gTcrQoSr6y0DN8AFEpcuDCYbd2ubt
/A+0MeocRPNKGiNB7lEfvpSD3x3e4pGlSFB1AMgnwCGmpXzHeA51LzbDJGtfdWon
x8L7OD5QD/LwRqQtAncRpf9jB56oJvktmznluSuCcJeY9ADSYH2YDPC1g3CCnuKG
SOJpSClZrPjlc2511emDcnOaMJhkyjeQ8R+I67+I05r0jBdk2FDnFASsNVVcRV5o
lzO+UTdVUs0nWsiDa+CX
=PGZV
-----END PGP SIGNATURE-----

[Andreas Cadhalpun]

Hi Julien,

Post by Julien Cristau

Post by Andreas Cadhalpun
It remains to be seen, what the release team prefers: frustrated users and
developers or both forks in jessie.

The release team is likely to let the people involved in multimedia foo
fight it out among themselves and pick a winner.

I am not interested in a "fight" and would prefer it very much if this
discussion remained purely technical.
Having a fresh memory of the last fight that took place on debian-devel,
I do not think that repeating a similar disaster is a good idea.

Post by Julien Cristau
We're not going to ship both and hand that mess over to the security team.

Could you please explain what "mess" you are talking about?

According to the changelog[1], there have been 8 security updates for
ffmpeg in squeeze. Two of them (4:0.5.6-2 and 4:0.5.6-3) do not contain
security related fixes, but rather fix build failures of the previous
security upload, so they do not really count.
That makes about 6 security fix uploads in about 3 years for squeeze,
i.e. 1 upload per 6 month.

If there were both forks in Jessie, this might double the number of
uploads to 12 in 3 years, but probably some of them could also go
through stable-updates instead of stable-security.

Is that an unbearable burden?

A lot of other software in Debian has already alternatives, like desktop
environments, web browsers, text editors and even init systems.

Why should this not be the case for a multimedia framework?

There is also one particularly similar case, as in the packages are
forks and require many security updates:
MySQL and MariaDB are currently in Debian testing.

Just for comparison, MySQL in squeeze had 3 uploads to stable-security
and 3 to oldstable(-security) [2].

As I mentioned this particular example in my discussion with Moritz, he
said that the security team will "be working with the release
team to sort this out for jessie"[3].

Now, 5 months later, he seems to have changed his mind, as I am not
aware of any such attempt, but instead Moritz seems to support both [4][5].

Thanks in advance for taking the time to answer these questions.

Best regards,
Andreas


1:
http://metadata.ftp-master.debian.org/changelogs//main/f/ffmpeg/ffmpeg_0.5.10-1_changelog

2:
http://metadata.ftp-master.debian.org/changelogs//main/m/mysql-5.1/mysql-5.1_5.1.73-1_changelog
3: https://bugs.debian.org/729203#435
4: https://bugs.debian.org/754940
5: https://bugs.debian.org/754941

[Raphael Geissert]

Post by Andreas Cadhalpun
According to the changelog[1], there have been 8 security updates for
ffmpeg in squeeze.

There would have been more but the code has evolved too much for it to be
feasible to backport the patches. Not to mention that some bugs that are being
fixed are, for example, for incomplete checks - checks that don't exist in the
0.5 branch.



Cheers,

[Andreas Cadhalpun]

Hi Raphael,

Post by Raphael Geissert

Post by Andreas Cadhalpun
According to the changelog[1], there have been 8 security updates for
ffmpeg in squeeze.

There would have been more

You're right, my calculation is slightly flawed.

Post by Raphael Geissert
but the code has evolved too much for it to be
feasible to backport the patches.

That is likely true for some, but not for others.

The real reason that there have not been more security updates for
ffmpeg in squeeze is that since 0.5.6 this is actually Libav and Libav
upstream stopped providing backports to 0.5 after 0.5.10 in February
2013 [1]. FFmpeg upstream released 0.5.14 in July 2014 [2] with some
more fixes [3].

So had both been in squeeze, there would have been four more, i.e. 16
security updates.

Post by Raphael Geissert
Not to mention that some bugs that are being
fixed are, for example, for incomplete checks - checks that don't exist in the
0.5 branch.

What do you mean here? If the affected code is not there, then that's
nice, because a backport is not needed.

Best regards,
Andreas

1: https://www.libav.org/releases/
2: https://www.ffmpeg.org/releases/
3:
https://git.videolan.org/?p=ffmpeg.git;a=shortlog;h=refs/heads/release/0.5

[Raphael Geissert]

Post by Andreas Cadhalpun

Post by Raphael Geissert

Post by Andreas Cadhalpun
According to the changelog[1], there have been 8 security updates for
ffmpeg in squeeze.

There would have been more

You're right, my calculation is slightly flawed.

That was my point, so please don't use it as an argument.

Post by Andreas Cadhalpun

Post by Raphael Geissert
Not to mention that some bugs that are being
fixed are, for example, for incomplete checks - checks that don't exist
in the 0.5 branch.

What do you mean here? If the affected code is not there, then that's
nice, because a backport is not needed.

Let me rephrase it: the fix is for an incomplete check, but in 0.5 the check
is missing - while the rest of the code is there. Which is kinda... worse.

Cheers,

[Andreas Cadhalpun]

Post by Raphael Geissert

Post by Andreas Cadhalpun

Post by Raphael Geissert

Post by Andreas Cadhalpun
According to the changelog[1], there have been 8 security updates for
ffmpeg in squeeze.

There would have been more

You're right, my calculation is slightly flawed.

That was my point, so please don't use it as an argument.

Maybe I didn't make my point clear enough, for which the actual number
of the security uploads is not important, only the order of magnitude.

Given the amount of software in Debian and thus the amount of security
fixes necessary for a stable release, I think that the additional
stable-security uploads for FFmpeg in the order of 10 per release will
be hardly noticeable.

While I understand and agree with the general idea of reducing code
duplication, I have a really hard time trying to understand why the
security team has such a strong opposition to the idea of having both
FFmpeg and Libav in Debian stable.

One argument against code duplication is the risk that security issues
get fixed in one, but not in the other. But in this particular case
FFmpeg upstream merges all security fixes from Libav, so an FFmpeg
package in a stable release won't have that problem.

What is particularly hard for me to understand is why e.g. MySQL and
MariaDB can be in testing at the same time without much resistance from
the security team, but FFmpeg and Libav can apparently not.

Post by Raphael Geissert

Post by Andreas Cadhalpun

Post by Raphael Geissert
Not to mention that some bugs that are being
fixed are, for example, for incomplete checks - checks that don't exist
in the 0.5 branch.

What do you mean here? If the affected code is not there, then that's
nice, because a backport is not needed.

Let me rephrase it: the fix is for an incomplete check, but in 0.5 the check
is missing - while the rest of the code is there. Which is kinda... worse.

Now I see, what you mean. Indeed that's worse, but if one notices
something like that, then the whole check can be backported instead of
the change in the check.
Though it probably would have been better to backport already the
incomplete check, when it was introduced in the development branch.

Best regards,
Andreas

[Russ Allbery]

Post by Andreas Cadhalpun
Given the amount of software in Debian and thus the amount of security
fixes necessary for a stable release, I think that the additional
stable-security uploads for FFmpeg in the order of 10 per release will
be hardly noticeable.

Er, 8 security updates over the course of a stable release is already very
high. Wouldn't adding another 10 make that the least secure source
package in Debian? I believe that's worse than web browsers, which have a
very large attack surface and huge numbers of active and well-funded
attackers. And this is just for a multimedia library.

I suppose it depends on how many of those could be grouped into one
update, and each Iceweasel update usually has multiple fixed CVEs, so
maybe this isn't an entirely fair comparison. But still, those are
jaw-dropping numbers.

Post by Andreas Cadhalpun
While I understand and agree with the general idea of reducing code
duplication, I have a really hard time trying to understand why the
security team has such a strong opposition to the idea of having both
FFmpeg and Libav in Debian stable.

Because the sorts of numbers that you're talking about indicate that this
code is a complete security disaster.

Post by Andreas Cadhalpun
What is particularly hard for me to understand is why e.g. MySQL and
MariaDB can be in testing at the same time without much resistance from
the security team, but FFmpeg and Libav can apparently not.

MySQL is already a security update problem due to Oracle's very unhelpful
attitude towards security patches. And we're still talking about rather
fewer security vulnerabilities than this, I believe.

[Andreas Cadhalpun]

Hi Russ,

Post by Russ Allbery

Post by Andreas Cadhalpun
Given the amount of software in Debian and thus the amount of security
fixes necessary for a stable release, I think that the additional
stable-security uploads for FFmpeg in the order of 10 per release will
be hardly noticeable.

Er, 8 security updates over the course of a stable release is already very
high. Wouldn't adding another 10 make that the least secure source
package in Debian? I believe that's worse than web browsers, which have a
very large attack surface and huge numbers of active and well-funded
attackers. And this is just for a multimedia library.

I must have failed to make my point again. :(
As far as I know there are hundreds of security updates (for all
packages together) in the lifetime of a stable release. Compared to that
10 is not large. And, as I already mentioned, I think that some of the
FFmpeg updates are minor enough to go through stable-updates.

It doesn't make a software less secure, if (even minor) security fixes
get backported even to old release branches, rather the contrary.

Webbrowsers tend to have a lot more security issues and e.g. for Firefox
15 security releases are planned in two years[2]. More importantly,
Mozilla supports one release only for one year. That is much worse than
the case of FFmpeg.
As e.g. the chromium browser uses FFmpeg[3] it is also under the
scrutiny of the very same attackers and security researchers as webbrowsers.

Post by Russ Allbery
I suppose it depends on how many of those could be grouped into one
update, and each Iceweasel update usually has multiple fixed CVEs, so
maybe this isn't an entirely fair comparison. But still, those are
jaw-dropping numbers.

For the numbers of CVEs fixed in each FFmpeg release, please have a look
at their security webpage[4]. Note how many of them get backported to
old releases and if one isn't, that's probably because the old release
didn't contain the vulnerable code.

Post by Russ Allbery

Post by Andreas Cadhalpun
While I understand and agree with the general idea of reducing code
duplication, I have a really hard time trying to understand why the
security team has such a strong opposition to the idea of having both
FFmpeg and Libav in Debian stable.

Because the sorts of numbers that you're talking about indicate that this
code is a complete security disaster.

Seen from a different point of view they show that the security support
of FFmpeg is very good.

Post by Russ Allbery

Post by Andreas Cadhalpun
What is particularly hard for me to understand is why e.g. MySQL and
MariaDB can be in testing at the same time without much resistance from
the security team, but FFmpeg and Libav can apparently not.

MySQL is already a security update problem due to Oracle's very unhelpful
attitude towards security patches. And we're still talking about rather
fewer security vulnerabilities than this, I believe.

So this gives me the impression that MySQL has a worse security support
than FFmpeg, which doesn't really help to understand why the security
team seems to be fine with having both forks of that in Debian testing.

Best regards,
Andreas


1: https://security-tracker.debian.org/tracker/source-package/libav
2: https://www.mozilla.org/en-US/firefox/organizations/faq/
3:
https://src.chromium.org/svn/trunk/deps/third_party/ffmpeg/README.chromium
4: https://ffmpeg.org/security.html

[Russ Allbery]

Post by Andreas Cadhalpun
I must have failed to make my point again. :(
As far as I know there are hundreds of security updates (for all packages
together) in the lifetime of a stable release. Compared to that 10 is not
large. And, as I already mentioned, I think that some of the FFmpeg
updates are minor enough to go through stable-updates.
It doesn't make a software less secure, if (even minor) security fixes get
backported even to old release branches, rather the contrary.

Well... backporting security fixes more of a bare minimum -- that's just
something that has to happen if we're going to support the software at
all, with a handful of exceptions where the software is, for one reason or
another, important enough that we're willing to release with it even
though security patches aren't backported properly and then terminate
support in the middle of our normal stable process.

But software should also not pose a significant security load in the first
place. That quantity of security vulnerabilities tells me that something
is deeply wrong with FFmpeg as an upstream source base. That's a sign of
code with a bad smell.

Now, that doesn't necessarily mean that it doesn't belong in Debian.
Sometimes we have to hold our nose and live with that, and it sounds like
libav isn't necessarily a lot better. But those are really painful
statistics that, to me at least, indicate the world is crying out for a
replacement code base that accomplishes the same goals but was written
with a higher level of quality in mind.

Obviously easier said than done, of course.

Is upstream aware that this is a really bad track record and trying to do
something proactive to increase the quality of the code, like
comprehensive auditing, or proactive rewrites to use more secure coding
practices such as some of the work that the LibreSSL team has been doing?

I'm sympathetic to the concerns of the security team and the release team
about supporting two code bases with this much security activity in a
stable release. Maybe it's still the right thing to do, but that's a lot
of work for them.

[Russ Allbery]

Post by Russ Allbery
Is upstream aware that this is a really bad track record and trying to
do something proactive to increase the quality of the code, like
comprehensive auditing, or proactive rewrites to use more secure coding
practices such as some of the work that the LibreSSL team has been doing?

Ah, I should have Googled my own question.

http://googleonlinesecurity.blogspot.com/2014/01/ffmpeg-and-thousand-fixes.html

Well, that's... better than nothing. It's certainly part of an overall
strategy, although that number of issues still indicates to me that there
are style and architecture issues here that could benefit from more
proactive design work. I could be wrong, having not looked at the code
personally -- maybe the problem space is just really hard. But that seems
like quite a lot of errors.

[Andreas Cadhalpun]

Post by Russ Allbery

Post by Andreas Cadhalpun
I must have failed to make my point again. :(
As far as I know there are hundreds of security updates (for all packages
together) in the lifetime of a stable release. Compared to that 10 is not
large. And, as I already mentioned, I think that some of the FFmpeg
updates are minor enough to go through stable-updates.
It doesn't make a software less secure, if (even minor) security fixes get
backported even to old release branches, rather the contrary.

Well... backporting security fixes more of a bare minimum -- that's just
something that has to happen if we're going to support the software at
all, with a handful of exceptions where the software is, for one reason or
another, important enough that we're willing to release with it even
though security patches aren't backported properly and then terminate
support in the middle of our normal stable process.

So it's good that FFmpeg upstream does that backporting.

Post by Russ Allbery
But software should also not pose a significant security load in the first
place. That quantity of security vulnerabilities tells me that something
is deeply wrong with FFmpeg as an upstream source base. That's a sign of
code with a bad smell.

In my opinion the large amount of security fixes is due to the fact that
FFmpeg is a large codebase and that most of the code has to deal with
untrusted data, a.k.a. multimedia files.

Post by Russ Allbery
Now, that doesn't necessarily mean that it doesn't belong in Debian.
Sometimes we have to hold our nose and live with that, and it sounds like
libav isn't necessarily a lot better.

On the contrary I think that Libav is worse, as it doesn't even apply
all patches for security vulnerabilities fixed in FFmpeg that also
affect Libav. Just have a look at the security tracker of Libav[1].

Post by Russ Allbery
But those are really painful
statistics that, to me at least, indicate the world is crying out for a
replacement code base that accomplishes the same goals but was written
with a higher level of quality in mind.
Obviously easier said than done, of course.

The time needed to do that would likely be spent a lot better with
trying to find and fix the remaining vulnerabilities in FFmpeg, because
any rewrite of such a large code base inevitably introduces it's own
security bugs.

Post by Russ Allbery
Is upstream aware that this is a really bad track record and trying to do
something proactive to increase the quality of the code, like
comprehensive auditing, or proactive rewrites to use more secure coding
practices such as some of the work that the LibreSSL team has been doing?
Ah, I should have Googled my own question.

http://googleonlinesecurity.blogspot.com/2014/01/ffmpeg-and-thousand-fixes.html

Post by Russ Allbery
Well, that's... better than nothing. It's certainly part of an
overall strategy, although that number of issues still indicates to
me that there are style and architecture issues here that could
benefit from more proactive design work. I could be wrong, having
not looked at the code personally -- maybe the problem space is just
really hard. But that seems like quite a lot of errors.

You could also have asked FFmpeg upstream. (I've CCed Michael
Niedermayer now.)

Post by Russ Allbery
I'm sympathetic to the concerns of the security team and the release team
about supporting two code bases with this much security activity in a
stable release. Maybe it's still the right thing to do, but that's a lot
of work for them.

Of course I'm also sympathetic with the concerns of the security and
release teams. But given that the alternatives are either to drop Libav,
which the Libav maintainers would probably be unhappy about, or not
having FFmpeg in the next stable release, which a lot of other people
would be unhappy about, having both in stable seems like the least bad
solution.

Best regards,
Andreas

1: https://security-tracker.debian.org/tracker/source-package/libav

[Michael Niedermayer]

Post by Andreas Cadhalpun

Post by Russ Allbery

Post by Andreas Cadhalpun
I must have failed to make my point again. :(
As far as I know there are hundreds of security updates (for all packages
together) in the lifetime of a stable release. Compared to that 10 is not
large. And, as I already mentioned, I think that some of the FFmpeg
updates are minor enough to go through stable-updates.
It doesn't make a software less secure, if (even minor) security fixes get
backported even to old release branches, rather the contrary.

Well... backporting security fixes more of a bare minimum -- that's just
something that has to happen if we're going to support the software at
all, with a handful of exceptions where the software is, for one reason or
another, important enough that we're willing to release with it even
though security patches aren't backported properly and then terminate
support in the middle of our normal stable process.

So it's good that FFmpeg upstream does that backporting.

Post by Russ Allbery
But software should also not pose a significant security load in the first
place. That quantity of security vulnerabilities tells me that something
is deeply wrong with FFmpeg as an upstream source base. That's a sign of
code with a bad smell.

In my opinion the large amount of security fixes is due to the fact
that FFmpeg is a large codebase and that most of the code has to
deal with untrusted data, a.k.a. multimedia files.

Post by Russ Allbery
Now, that doesn't necessarily mean that it doesn't belong in Debian.
Sometimes we have to hold our nose and live with that, and it sounds like
libav isn't necessarily a lot better.

On the contrary I think that Libav is worse, as it doesn't even
apply all patches for security vulnerabilities fixed in FFmpeg that
also affect Libav. Just have a look at the security tracker of
Libav[1].

Post by Russ Allbery
But those are really painful
statistics that, to me at least, indicate the world is crying out for a
replacement code base that accomplishes the same goals but was written
with a higher level of quality in mind.
Obviously easier said than done, of course.

The time needed to do that would likely be spent a lot better with
trying to find and fix the remaining vulnerabilities in FFmpeg,
because any rewrite of such a large code base inevitably introduces
it's own security bugs.

Post by Russ Allbery
Is upstream aware that this is a really bad track record and trying to do
something proactive to increase the quality of the code, like
comprehensive auditing, or proactive rewrites to use more secure coding
practices such as some of the work that the LibreSSL team has been doing?
Ah, I should have Googled my own question.
http://googleonlinesecurity.blogspot.com/2014/01/ffmpeg-and-thousand-fixes.html
Well, that's... better than nothing. It's certainly part of an
overall strategy, although that number of issues still indicates to
me that there are style and architecture issues here that could
benefit from more proactive design work. I could be wrong, having
not looked at the code personally -- maybe the problem space is just
really hard. But that seems like quite a lot of errors.

You could also have asked FFmpeg upstream. (I've CCed Michael
Niedermayer now.)

Yes the problem space is hard ...
The problem with multimedia in relation to security is that
* There are hundreads of different formats, requireing alot of code
to support them
* The input files and streams can generally not be trusted, which
makes most of the code security relevant and a potential target for
an exploit
* Many and especially the most important and efficient formats are
very complex
* The code must be very fast, users want to see their movies play
in realtime not waiting for each frame to be decoded. And most of
the code is speed relevant

Above applies to any implementation, and constrains architecture
options ...

What we do to combat that is
All patches going into FFmpeg are reviewed with security in mind

The codebase was repeatledly tested with fuzzed files to uncover
all kinds of anomalies, all such found anomalies where fixed.
Also independant of googles fuzzing efforts, some of our users
have done their own fuzzing. And during google code in several
students have as well manually tried to find security issues.

FFmpeg is regularly tested with static code analyzsers like coverity
and most issues found get quickly fixed

FFmpeg is tested regularly with runtime memory checkers like
valgrind, address sanitizer and others and all reproduceable issues
are fixed, iam not aware of any open and reproduceable one

Almost all of the internal APIs used in FFmpeg are designed to be
secure, always passing array sizes and checking them instead of
assuming the caller knows they are large enough, exceptions to this
are just the most speed critical parts.

Codecs which are WIP and arent up to security standards
can be and are marked as experimental and will not be selected
during autodetection unless overriden by the user.

...


If anyone has any ideas of how we can improve security further
with the available resources. Iam very interrested in that and
thats quite independant of FFmpeg being accepted in debian or not.
I do care about FFmpegs security


[...]

[Joseph Neal]

What we do to combat that is All patches going into FFmpeg are
reviewed with security in mind
The codebase was repeatledly tested with fuzzed files to uncover all
kinds of anomalies, all such found anomalies where fixed. Also
independant of googles fuzzing efforts, some of our users have done
their own fuzzing. And during google code in several students have as
well manually tried to find security issues.
FFmpeg is regularly tested with static code analyzsers like coverity
and most issues found get quickly fixed
FFmpeg is tested regularly with runtime memory checkers like
valgrind, address sanitizer and others and all reproduceable issues
are fixed, iam not aware of any open and reproduceable one
Almost all of the internal APIs used in FFmpeg are designed to be
secure, always passing array sizes and checking them instead of
assuming the caller knows they are large enough, exceptions to this
are just the most speed critical parts.
Codecs which are WIP and arent up to security standards can be and
are marked as experimental and will not be selected during
autodetection unless overriden by the user.

Something else to add to this list is that ffmpeg has a far larger base
of installed users bring the "more eyes" principle into play. I can't
speak as to how many linux distros use which fork, though I believe the
vast majority of all libav installs are debian and its derivatives.
Ffmpeg, meanwhile, has a huge install base in the Windows and to a
lessor degree Macintosh worlds as the backend to nearly every free video
player or transcoder out there. Virtually no upstreams with a choice
are adopting libav, and I expect the number of those supporting it to
dwindle as it continues drift away from ffmeg.

I don't see this as being much different from the
imagemagik/graphicsmagic situation.

Sorry if my message formatting is screwed up. I'm on windows and have
no idea what I'm doing.

[Thorsten Glaser]

Post by Andreas Cadhalpun
So it's good that FFmpeg upstream does that backporting.

As opposed to, for example, MySQL and Iceweasel, for which
there is practically a blanket permission to upload new
upstream releases unchecked into stable. (There appears to
be one for Mediawiki and OpenJDK too, which do their security
backporting themselves. This could apply to ffmpeg as well.)

bye,
//mirabilos

[Russ Allbery]

Post by Thorsten Glaser
As opposed to, for example, MySQL and Iceweasel, for which
there is practically a blanket permission to upload new
upstream releases unchecked into stable. (There appears to
be one for Mediawiki and OpenJDK too, which do their security
backporting themselves. This could apply to ffmpeg as well.)

Those are the ones I referenced where we're holding our nose and going
with a much less than ideal security policy because we don't have another
good option. Those aren't packages that we can realistically drop from
the archive.

The same may apply to FFmpeg as well, but it's not a happy situation to be
in.

[Josselin Mouette]

Post by Andreas Cadhalpun
I must have failed to make my point again. :(

No, you are the one who misunderstands the point.

Post by Andreas Cadhalpun
As far as I know there are hundreds of security updates (for all
packages together) in the lifetime of a stable release. Compared to that
10 is not large. And, as I already mentioned, I think that some of the
FFmpeg updates are minor enough to go through stable-updates.

No FFmpeg security update is “minor”.

Almost each ffmpeg security bug is a code execution one. Almost each and
every one of them is hard to backport.

Those 10 security updates might represent more work than 100 *really*
minor security updates.

Cheers,
--
.''`. Josselin Mouette
: :' :
`. `'
`-

[Pau Garcia i Quiles]

No FFmpeg security update is “minor”.
Almost each ffmpeg security bug is a code execution one. Almost each and
every one of them is hard to backport.
Those 10 security updates might represent more work than 100 *really*
minor security updates.

How is it better to have libav, which does a lot less security bugfixing,
in?

I'd rather have a library that fixes bugs than one that passes in order to
look "more secure". When in fact it's less.

[Didier 'OdyX' Raboud]

Post by Pau Garcia i Quiles
How is it better to have libav, which does a lot less security
bugfixing, in?

Our security team has to prepare the libav updates over the lifetime of
wheezy anyway. Introducing ffmpeg in jessie (with or without dropping
libav) means (at least) duplicating that work.

OdyX

[Andreas Cadhalpun]

Hi Didier,

Post by Didier 'OdyX' Raboud

Post by Pau Garcia i Quiles
How is it better to have libav, which does a lot less security
bugfixing, in?

Our security team has to prepare the libav updates over the lifetime of
wheezy anyway.

As far as I know, both the updates for Libav and FFmpeg are prepared by
their respective upstreams, then integrated into the Debian packages by
the respective maintainers and only then comes work for the security
team in reviewing the patches and writing a DSA.

Post by Didier 'OdyX' Raboud
Introducing ffmpeg in jessie (with or without dropping
libav) means (at least) duplicating that work.

Since all the updates for Libav are merged by FFmpeg, it's not really
duplicated work. At least in theory only the additional fixes of FFmpeg
would have to be reviewed additionally.

Best regards,
Andreas

[Russ Allbery]

Post by Pau Garcia i Quiles

Post by Josselin Mouette
No FFmpeg security update is “minor”.
Almost each ffmpeg security bug is a code execution one. Almost each
and every one of them is hard to backport.
Those 10 security updates might represent more work than 100 *really*
minor security updates.

How is it better to have libav, which does a lot less security
bugfixing, in?

It's not.

However, what was proposed was having *both* of them, not dropping libav
in favor of FFmpeg.

[Pau Garcia i Quiles]

Post by Pau Garcia i Quiles
How is it better to have libav, which does a lot less security

Post by Pau Garcia i Quiles
bugfixing, in?

It's not.
However, what was proposed was having *both* of them, not dropping libav
in favor of FFmpeg.

So had the proposal been "hey, let's replace libav with ffmpeg", would you
vote "yes" ? Only one library to maintain and review, and it happens to
have good upstream support And replacing libav with ffmpeg is easy and the
ffmpeg maintainer is committed to help port reverse dependencies. Looks
good to me. Maybe Andreas should have made a not-so-polite proposal?

[Russ Allbery]

Post by Pau Garcia i Quiles
So had the proposal been "hey, let's replace libav with ffmpeg", would
you vote "yes" ? Only one library to maintain and review, and it happens
to have good upstream support And replacing libav with ffmpeg is easy
and the ffmpeg maintainer is committed to help port reverse
dependencies. Looks good to me. Maybe Andreas should have made a
not-so-polite proposal?

I personally don't have enough information to know why libav was chosen
instead of FFmpeg, and the discussion on debian-devel so far has mostly
come from FFmpeg advocates. So there's probably another side to the story
that hasn't been stated here yet.

Based purely on security evaluations by others that I was able to find on
the web, FFmpeg appears to be better at the moment than libav on the
security front (although libav appears to be trying to catch up).

[Charles Plessy]

Post by Russ Allbery
Based purely on security evaluations by others that I was able to find on
the web, FFmpeg appears to be better at the moment than libav on the
security front (although libav appears to be trying to catch up).

Hello everybody

At that point, and given the impressive number of users who expressed interest
for having FFmpeg in Debian (see http://bugs.debian.org/729203), I think that
it would be fair to ask the libav maintainers to provide arguments on whether
to distribute both libraries or make a choice, even if it is against their own
interest since they may see their packages removed at the end.

Otherwise we are in that kind of frequent Debianesque situation where the
winning strategy is to stay silent as long as possible.

Have a nice day,

[Nikolaus Rath]

Post by Charles Plessy

Post by Russ Allbery
Based purely on security evaluations by others that I was able to find on
the web, FFmpeg appears to be better at the moment than libav on the
security front (although libav appears to be trying to catch up).

Hello everybody
At that point, and given the impressive number of users who expressed interest
for having FFmpeg in Debian (see http://bugs.debian.org/729203), I think that
it would be fair to ask the libav maintainers to provide arguments on whether
to distribute both libraries or make a choice, even if it is against their own
interest since they may see their packages removed at the end.
Otherwise we are in that kind of frequent Debianesque situation where the
winning strategy is to stay silent as long as possible.
From what I have read about this situation now and in the past, I would

recommend to bring this to the CTTE right away and ask them to decide
whether the ffmpeg package in Debian should provide libav or ffmpeg.


I think it is unlikely that discussion between the concerned parties
will lead to a consensus, the jessie freeze is upcoming, and almost
every recent CTTE decision took several months. In other words, time is
running out, don't waste it with a pointless discussion on debian-devel.


Best,
Nikolaus

[Ian Jackson]

Post by Charles Plessy

Post by Russ Allbery
Based purely on security evaluations by others that I was able to find on
the web, FFmpeg appears to be better at the moment than libav on the
security front (although libav appears to be trying to catch up).

Hello everybody
At that point, and given the impressive number of users who
expressed interest for having FFmpeg in Debian (see
http://bugs.debian.org/729203), I think that it would be fair to ask
the libav maintainers to provide arguments on whether to distribute
both libraries or make a choice, even if it is against their own
interest since they may see their packages removed at the end.
Otherwise we are in that kind of frequent Debianesque situation where the
winning strategy is to stay silent as long as possible.

CCing ***@packages.d.o.

Ian.

[Pau Garcia i Quiles]

Post by Russ Allbery
I personally don't have enough information to know why libav was chosen
instead of FFmpeg, and the discussion on debian-devel so far has mostly
come from FFmpeg advocates. So there's probably another side to the story
that hasn't been stated here yet.

libav was not chosen in Debian

ffmpeg had a leadership crisis a few years ago: Michael Niedermaier (who
has written here) was accused of dictatorial methods by some ffmpeg
developers. I don't know if they were right or not in their complains and
frankly, I don't care.

Those guys tried a coup d'etat, stealing the domain, name, code repository,
logo and everything and leaving Michael out. When Michael was able to
recover control of some things, and get a new hosting, source repository,
etc, they forked ffmpeg in libav. The libav guys knowingly tried and still
try to break ffmpeg by using the same library names and for a long time,
version numbers too.

The Debian maintainer of ffmpeg at the time (Reinhard, who has written here
too) was one of the guys who took the libav side instead of the ffmpeg
side. He used the ffmpeg package names to provide libav, and actively
blocked any effort that would lead to src:ffmpeg being actual ffmpeg.org.
That's why we have libav now instead of ffmpeg.

I'm all for forks but if you do a fork, at least you do it with ethics:
different library names, sonames, etc. The libav developers have tried from
minute zero to create a conflict. And what has been the outcome of that? A
library which worse than ffmpeg in features, codec support, security, and
essentially everything. As someone mentioned way back in the thread, the
only people who seem to prefer libav over ffmpeg are the libav developers.

I am not involved in libav or ffmpeg and if libav would be better
technically, in security, etc, I would be all for libav, even with all the
ill-intented methods they used. But it's not the case: they disrupt
peaceful open source communities (see this discussion, it's not the first
time in Debian and it has happened in other distributions too) with what
goal?... forcing a worse library in technical regards? OMG. Pointless.

[Steve Langasek]

Post by Pau Garcia i Quiles
different library names, sonames, etc.

I have nothing resembling an informed opinion on the libav vs. ffmpeg
upstream conflict, but this is just wrong. There is *nothing* unethical
about reusing the library names and sonames when forking. In fact, the
right to continue using such interfaces when creating a fork is a very
important principle.

There may have been a dispute over a domain name, and over which of two
upstream development communities is the "rightful" successor to the project.
But the only party that "owns" the interface names in Debian is Debian
itself. There are various /pragmatic/ reasons why we should care about how
each name is being used in the wider community, but that does not mean that
the side of the fork that keeps the domain name, that has a majority of mind
share, or that wins out in the end should be regarded as having a morally
superior claim *because* of this.

[Norbert Preining]

Post by Steve Langasek
upstream conflict, but this is just wrong. There is *nothing* unethical
about reusing the library names and sonames when forking. In fact, the

Besides, when changing APIs and breaking interaction with other
programs that rely on the original API.

You are right when the fork is API compatible (like extension of
a library that provides the same API plus something more), but
*not* when changing the API.

In this case, me too, would consider it *incorrect* (I don't want to
use "unethical" as it is difficult to define).

Norbert

------------------------------------------------------------------------
PREINING, Norbert http://www.preining.info
JAIST, Japan TeX Live & Debian Developer
GPG: 0x860CDC13 fp: F7D8 A928 26E3 16A1 9FA0 ACF0 6CAC A448 860C DC13
------------------------------------------------------------------------

[Marco d'Itri]

Post by Russ Allbery
I personally don't have enough information to know why libav was chosen
instead of FFmpeg, and the discussion on debian-devel so far has mostly
come from FFmpeg advocates. So there's probably another side to the story
that hasn't been stated here yet.

Probably because there are no libav advocates other than the libav
developers...

[Josselin Mouette]

Post by Pau Garcia i Quiles
How is it better to have libav, which does a lot less security
bugfixing, in?
I'd rather have a library that fixes bugs than one that passes in
order to look "more secure". When in fact it's less.

I have no informed opinion on whether ffmpeg or libav is better. On the
security front, it looks indeed like ffmpeg is doing better but it is
still hearsay.

What I know, though, is that it is not realistic to maintain two such
libraries in the course of a stable release.

Said otherwise: if we have a consensus that ffmpeg is better, then let’s
do the switch NOW. Not after the freeze.

If it’s too late, libav will be the sole implementation in jessie, and
the switch will have to be done for jessie+1.

Cheers,
--
.''`. Josselin Mouette
: :' :
`. `'
`-

[Moritz Mühlenhoff]

Post by Josselin Mouette

Post by Pau Garcia i Quiles
How is it better to have libav, which does a lot less security
bugfixing, in?
I'd rather have a library that fixes bugs than one that passes in
order to look "more secure". When in fact it's less.

I have no informed opinion on whether ffmpeg or libav is better. On the
security front, it looks indeed like ffmpeg is doing better but it is
still hearsay.

I think ffmpeg is doing better in terms of handling security issues; when
I contacted Michael Niedermeyer in private we has always quick to reply,
while libav-security@ seems understaffed: Several queries in the past needed
additional poking, some were left unaddressed until today. Also, the Google
fuzzer guys stated that more samples are unfixed in libav compared to ffmpeg.

Cheers,
Moritz

[Andreas Cadhalpun]

Hi Josselin,

Post by Josselin Mouette

Post by Andreas Cadhalpun
I must have failed to make my point again. :(

No, you are the one who misunderstands the point.

Thanks for sharing your opinion.

Post by Josselin Mouette

Post by Andreas Cadhalpun
As far as I know there are hundreds of security updates (for all
packages together) in the lifetime of a stable release. Compared to that
10 is not large. And, as I already mentioned, I think that some of the
FFmpeg updates are minor enough to go through stable-updates.

No FFmpeg security update is “minor”.

While it's hard to proof your statement, I agree that most of the FFmpeg
security fixes should not be considered minor.
Still not every FFmpeg update (note that the word 'security' is absent
here) fixes a severe security issue. Some contain only regression fixes.
For example in the 2.2 release series, only 2.2.4 fixed a CVE, the other
four updates did not, so could have gone through stable-updates.

Post by Josselin Mouette
Almost each ffmpeg security bug is a code execution one. Almost each and
every one of them is hard to backport.

When making such a statement it is very helpful to explain how you came
to this conclusion.
For example, the last security fix (CVE-2014-4609) could be trivially
backported even to the 0.5 branch. (I did so myself.)

Post by Josselin Mouette
Those 10 security updates might represent more work than 100 *really*
minor security updates.

Even if it required a lot of work to backport the security fixes, that
work would be done by FFmpeg upstream anyway. The security team would at
most have to review the changes.

Best regards,
Andreas

[Raphael Geissert]

Post by Andreas Cadhalpun
Given the amount of software in Debian and thus the amount of security
fixes necessary for a stable release, I think that the additional
stable-security uploads for FFmpeg in the order of 10 per release will
be hardly noticeable.

They are surely noticeable to the security team: the release process of a
security update is more than just a "throw and forget".
Tracking every single vulnerability for each copy of the code consumes time.
Every single update also consumes team's time, and that of many organisations
external to Debian.

Post by Andreas Cadhalpun
What is particularly hard for me to understand is why e.g. MySQL and
MariaDB can be in testing at the same time without much resistance from
the security team, but FFmpeg and Libav can apparently not.

There is resistance - we only want one, not two, not three (percona).

IMH (and personal) O, if you want to see ffmpeg in Jessie or later, you should
replace libav - i.e. no silly one binary + libraries that won't work for
anything else.

Cheers,

[Jonathan Dowland]

Post by Reinhard Tartler
I think it would be best if ftp-master left the ffmpeg package in NEW
until an answer to this problem has been found.

That would leave people who are eagerly awaiting ffmpeg(1) stranded in limbo
whilst the tangential library issue is fought over.

[Niv Sardi]

Post by Jonathan Dowland

Post by Reinhard Tartler
I think it would be best if ftp-master left the ffmpeg package in NEW
until an answer to this problem has been found.

That would leave people who are eagerly awaiting ffmpeg(1) stranded in limbo
whilst the tangential library issue is fought over.

Agreed,
The (great) packaging effort Andreas has gone through allows us to have
both in experimental/unstable in order to let the users test it, and allow
the multimedia packages maintainers make their mind about it.

With this package in we can also see how many bugs come in and how prompt
are Andreas and the FFmpeg developers (that have shown great interest in
this work) to resolve Debian bugs. And then with all that real data, decide
what we want to ship.

As described in Andreas thorough mail, there are only 2 packages that can't
use both (and iirc they currently FTBFS as upstream uses FFmpeg) and 5
packages that could need a transition, so the release team can just pop an
RC bug on ffmpeg until all is sorted out to ensure it doesn't get into
testing.

I believe we should get ffmpeg out of new asap in order to be able to have
some real data by freeze time.

Cheers,

[Holger Levsen]

Hi,

Post by Niv Sardi
As described in Andreas thorough mail, there are only 2 packages that can't
use both (and iirc they currently FTBFS as upstream uses FFmpeg) and 5
packages that could need a transition, so the release team can just pop an
RC bug on ffmpeg until all is sorted out to ensure it doesn't get into
testing.

note that also the ffmpeg maintainers could file this RC bug, which would also
show their willingness to cooperate, prevent potential mess and which can be
easily fixed anytime.


cheers,
Holger

[Andreas Cadhalpun]

Hi Holger,

Post by Holger Levsen

Post by Niv Sardi
As described in Andreas thorough mail, there are only 2 packages that can't
use both (and iirc they currently FTBFS as upstream uses FFmpeg) and 5
packages that could need a transition, so the release team can just pop an
RC bug on ffmpeg until all is sorted out to ensure it doesn't get into
testing.

note that also the ffmpeg maintainers could file this RC bug, which would also
show their willingness to cooperate, prevent potential mess and which can be
easily fixed anytime.

We could file such a RC bug if that is deemed the best way forward, but
at the moment it's a rather irrelevant point, because the package
targeted at experimental is still waiting in the NEW queue.

Best regards,
Andreas

[Thorsten Glaser]

Post by Andreas Cadhalpun
* Do you intend to replace Libav by FFmpeg in Debian?
No, there is no need to replace anything as long as it is maintained.
Currently the main goal is to give multimedia maintainers a choice
between the two sets of libraries to link against, and our users the
choice to use the 'ffmpeg' utility. That is possible, because the
packages are co-installable. (Only the *-dev packages conflict.)

Hm, I wonder if this will work, see the JPEG discussion.

But I'd *love* to see ffmpeg replace libav and to get
my mplayer back, which is currently on hold.

Thanks,
//mirabilos

[Andreas Cadhalpun]

Post by Thorsten Glaser

Post by Andreas Cadhalpun
* Do you intend to replace Libav by FFmpeg in Debian?
No, there is no need to replace anything as long as it is maintained.
Currently the main goal is to give multimedia maintainers a choice
between the two sets of libraries to link against, and our users the
choice to use the 'ffmpeg' utility. That is possible, because the
packages are co-installable. (Only the *-dev packages conflict.)

Hm, I wonder if this will work, see the JPEG discussion.

Well, it would work, if the security and release teams would agree with it.

Post by Thorsten Glaser
But I'd *love* to see ffmpeg replace libav and to get
my mplayer back, which is currently on hold.

Once FFmpeg is back in the archive, it'll be easy to reintroduce
MPlayer. It has been removed from sid, since it fails to build against
Libav, but it builds fine against FFmpeg.
(It uses some of the features only provided by FFmpeg.)

Best regards,
Andreas

[Matthias Urlichs]

Hi,

Once FFmpeg is back in the archive, it'll be easy to reintroduce MPlayer. It
has been removed from sid, since it fails to build against Libav, but it
builds fine against FFmpeg.
(It uses some of the features only provided by FFmpeg.)

Yet another reason why solely depending on libav is a bad idea.

That leaves the question of the "official" opinion of the libav
maintainers (pkg-multimedia-***@lists.alioth.debian.org).
Did none of them write anything in "defense" of libav, or have I simply
missed it?

IMHO the best idea at this point would be to toss out libav, and rebuild
the rdeps with ffmpeg. Now, before it's too late for jessie.

[Marco d'Itri]

Post by Matthias Urlichs
IMHO the best idea at this point would be to toss out libav, and rebuild
the rdeps with ffmpeg. Now, before it's too late for jessie.

Agreed. The interested parties should really raise this with the CTTE
ASAP.

[Reinhard Tartler]

Post by Matthias Urlichs
Hi,

Once FFmpeg is back in the archive, it'll be easy to reintroduce MPlayer. It
has been removed from sid, since it fails to build against Libav, but it
builds fine against FFmpeg.
(It uses some of the features only provided by FFmpeg.)

Yet another reason why solely depending on libav is a bad idea.
That leaves the question of the "official" opinion of the libav
Did none of them write anything in "defense" of libav, or have I simply
missed it?

I intended to come up with a more timely full response, but I just
didn't get to it so far.

For now, please refer to http://lwn.net/Articles/607662/,
http://codecs.multimedia.cx/?p=370 (rather old, but still true), and
http://codecs.multimedia.cx/?p=674 (recent update on that matter)

Regarding Marco's argument that libav had few friends, well, let me
point to https://medium.com/libre-parole/libav-is-beautiful-4c1a2c886948
for instance.

Post by Matthias Urlichs
IMHO the best idea at this point would be to toss out libav, and rebuild
the rdeps with ffmpeg. Now, before it's too late for jessie.

I think that is totally out of question: Uploading FFmpeg to our
archive will break the Debian archive so hard that it will take months
to get everything back to testing, if it works at all.

To the best of my knowledge, there are only two high-profile projects
that play hardball to require FFmpeg: Mplayer and mythtv. Neither of
those do that (again to the best of my knowledge) mainly because of
technical but rather very political reasons. The case of mplayer has
been elaborated extensively in
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=732159#79 (see the
following "discussion" with Reimar - my conclusion from that is while
technically possible, nobody wants to make mplayer work with Libav -
and that's why it was removed, not because of the FFmpeg dependency).
For Mythtv I can only say that they didn't even bother engaging any
discussion.

(All) other high-profile downstream projects, including VLC or XBMC
(now Kodi) work demonstrably just fine with Libav. Sure, FFmpeg may
provide them with extra features, but why on earth would anyone want 3
different implementations of a ProRes encoder (seriously, and FFmpeg
seems to claim to provide "security support" for each of them), or
support for fringe codecs such as Wing Command 3 (yes, you can decode
the videos from that video game).

There are a number of legitimate requested backports, such as for some
of FFmpeg's additional filters in libavfilter, and similar. All of
them are rather easy to backport, and this is done on a regular basis.
However, with the due diligence and attention such a widely used and
high-profile library requires.

Which brings me to my next point: Release frequency. FFmpeg has an
insane frequency of releases, and still seems to "support" (at least
providing updates) to all of them, including 0.5 which is currently in
oldstable (needless to say none of these patches made it to
oldstable-security, why is still a mystery to me). The effect of that
is that downstream projects have a hard time to choose what version of
FFmpeg they want to support. This brings effectively back to the
situation I was in when I took over maintenance of the package many
years ago: Back then, Michael Niedermeyer strongly recommended all
downstreams to avoid shared libraries and just link statically against
libavcodec.a to avoid problems regarding "incompatible" library
versions. I see exactly this problem arising again: Both mythtv and
mplayer upstream (btw, xbmc as well) bundle some copy of ffmpeg in
their sources and recommend everyone to just use the internal copy to
avoid problems.

Needless to say that this is not acceptable for use in Debian.

Yes, I agree that this situation is a mess. A big mess. My fear is
that switching to FFmpeg will bring us back to the mess we where 8
years ago, and I worked on for years.

Libav is far from being perfect. That's true. I don't know why exactly
FFmpeg seems to have more manpower, but it's not all black or white.
For instance, there are a number of developers that actively
contribute to both projects and are essential in keeping both projects
in good health. Also I don't really buy the security argument. True,
FFmpeg has more security updates, but backporting them to Libav turned
out to be difficult because many if not most of them turn out to be
either incomplete, invalid or require further clarification. Libav
developers prefer to go the unpleasant road of fully understanding the
issue, which takes extra time. For all these reasons, I do not see the
necessity to do any drastic and dangerous steps right now.


I now seriously wonder if the last 45 minutes in which I wrote this
email wouldn't have been better spent with preparing the next upload
for stable-security. YMMV.

[Jonas Smedegaard]

Quoting Reinhard Tartler (2014-08-08 14:29:59)

Post by Reinhard Tartler
For now, please refer to http://lwn.net/Articles/607662/,
http://codecs.multimedia.cx/?p=370 (rather old, but still true), and
http://codecs.multimedia.cx/?p=674 (recent update on that matter)
Regarding Marco's argument that libav had few friends, well, let me
point to https://medium.com/libre-parole/libav-is-beautiful-4c1a2c886948
for instance.

Post by Matthias Urlichs
IMHO the best idea at this point would be to toss out libav, and rebuild
the rdeps with ffmpeg. Now, before it's too late for jessie.

I think that is totally out of question: Uploading FFmpeg to our
archive will break the Debian archive so hard that it will take months
to get everything back to testing, if it works at all.
To the best of my knowledge, there are only two high-profile projects
that play hardball to require FFmpeg: Mplayer and mythtv. Neither of
those do that (again to the best of my knowledge) mainly because of
technical but rather very political reasons.

[snip]

Post by Reinhard Tartler
I now seriously wonder if the last 45 minutes in which I wrote this
email wouldn't have been better spent with preparing the next upload
for stable-security. YMMV.

Thanks a lot for your input, Reinhard.

Even if the loud ones in this thread may not, I doubt I am the only one
finding value in your references and arguments.


- Jonas

[Dimitri John Ledkov]

Post by Reinhard Tartler

Post by Matthias Urlichs
Hi,

Once FFmpeg is back in the archive, it'll be easy to reintroduce MPlayer. It
has been removed from sid, since it fails to build against Libav, but it
builds fine against FFmpeg.
(It uses some of the features only provided by FFmpeg.)

Yet another reason why solely depending on libav is a bad idea.
That leaves the question of the "official" opinion of the libav
Did none of them write anything in "defense" of libav, or have I simply
missed it?

I intended to come up with a more timely full response, but I just
didn't get to it so far.
For now, please refer to http://lwn.net/Articles/607662/,
http://codecs.multimedia.cx/?p=370 (rather old, but still true), and
http://codecs.multimedia.cx/?p=674 (recent update on that matter)
Regarding Marco's argument that libav had few friends, well, let me
point to https://medium.com/libre-parole/libav-is-beautiful-4c1a2c886948
for instance.

Post by Matthias Urlichs
IMHO the best idea at this point would be to toss out libav, and rebuild
the rdeps with ffmpeg. Now, before it's too late for jessie.

I think that is totally out of question: Uploading FFmpeg to our
archive will break the Debian archive so hard that it will take months
to get everything back to testing, if it works at all.
To the best of my knowledge, there are only two high-profile projects
that play hardball to require FFmpeg: Mplayer and mythtv. Neither of
those do that (again to the best of my knowledge) mainly because of
technical but rather very political reasons. The case of mplayer has
been elaborated extensively in
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=732159#79 (see the
following "discussion" with Reimar - my conclusion from that is while
technically possible, nobody wants to make mplayer work with Libav -
and that's why it was removed, not because of the FFmpeg dependency).
For Mythtv I can only say that they didn't even bother engaging any
discussion.

So in ubuntu, at the time we were doing libav9 transition we also
still had mplayer and pleora of packages inheritance from
deb-multimedia or some such repositories. I was very reluctant to
remove mplayer and all the reverse-deps, as I felt it is valuable to
ship mplayer. I believe it was based on unreleased experimental
packaging in git and other bits [1] Later Colin Watson also provided
minimal port to make it build on arm64. Unfortunately libav10
transition got entangled in too many ways and we didn't manager to
port mplayer to libav10. This was attempted based on top of mplayer
trunk snapshot / last stable tarball, patches from gentoo and own
porting efforts from myself and sil2100/robru but albeit
unsuccessfully. Under pressing wait of too many things stuck in
proposed migration (britney migration) we did remove mplayer and
reverse dependencies and pointed / filed bugs to port to mplayer2 or
just libav-tools where appropriate. I did ponder about compiling
mplayer with an embedded copy of libav9 statically linked, but
ultimately decided that it's unnecessary evil and majority of
use-cases are served by the current libav stack. Either of ffmpeg and
libav is a big security maintenance burden, simply from nature of
inherently handling complex and large streams of untrusted data. So in
trusty, I did work to unsplit the packages such that libav moved from
main to universe, and can be synced from debian unmodified to minimize
net-total maintenance burden as now updates and packaging can be fully
shared. I see moving to mplayer2 as a net positive benefit for Debian.
MythTV alone, whilst important package, I don't see as special enough
to grant use of an embedded copy or forcing an uncertain cost of
switching back to ffmpeg. If we need to port that project, then in
Debian/Ubuntu/UbuntuStudio there are enough interested people to get
it done.

[1] https://launchpad.net/ubuntu/+source/mplayer/2:1.1+dfsg1-0ubuntu1

Regards,

Dimitri.

Post by Reinhard Tartler
(All) other high-profile downstream projects, including VLC or XBMC
(now Kodi) work demonstrably just fine with Libav. Sure, FFmpeg may
provide them with extra features, but why on earth would anyone want 3
different implementations of a ProRes encoder (seriously, and FFmpeg
seems to claim to provide "security support" for each of them), or
support for fringe codecs such as Wing Command 3 (yes, you can decode
the videos from that video game).
There are a number of legitimate requested backports, such as for some
of FFmpeg's additional filters in libavfilter, and similar. All of
them are rather easy to backport, and this is done on a regular basis.
However, with the due diligence and attention such a widely used and
high-profile library requires.
Which brings me to my next point: Release frequency. FFmpeg has an
insane frequency of releases, and still seems to "support" (at least
providing updates) to all of them, including 0.5 which is currently in
oldstable (needless to say none of these patches made it to
oldstable-security, why is still a mystery to me). The effect of that
is that downstream projects have a hard time to choose what version of
FFmpeg they want to support. This brings effectively back to the
situation I was in when I took over maintenance of the package many
years ago: Back then, Michael Niedermeyer strongly recommended all
downstreams to avoid shared libraries and just link statically against
libavcodec.a to avoid problems regarding "incompatible" library
versions. I see exactly this problem arising again: Both mythtv and
mplayer upstream (btw, xbmc as well) bundle some copy of ffmpeg in
their sources and recommend everyone to just use the internal copy to
avoid problems.
Needless to say that this is not acceptable for use in Debian.
Yes, I agree that this situation is a mess. A big mess. My fear is
that switching to FFmpeg will bring us back to the mess we where 8
years ago, and I worked on for years.
Libav is far from being perfect. That's true. I don't know why exactly
FFmpeg seems to have more manpower, but it's not all black or white.
For instance, there are a number of developers that actively
contribute to both projects and are essential in keeping both projects
in good health. Also I don't really buy the security argument. True,
FFmpeg has more security updates, but backporting them to Libav turned
out to be difficult because many if not most of them turn out to be
either incomplete, invalid or require further clarification. Libav
developers prefer to go the unpleasant road of fully understanding the
issue, which takes extra time. For all these reasons, I do not see the
necessity to do any drastic and dangerous steps right now.
I now seriously wonder if the last 45 minutes in which I wrote this
email wouldn't have been better spent with preparing the next upload
for stable-security. YMMV.
--
regards,
Reinhard
--

[Wookey]

Post by Reinhard Tartler
To the best of my knowledge, there are only two high-profile projects
that play hardball to require FFmpeg: Mplayer and mythtv. Neither of
those do that (again to the best of my knowledge) mainly because of
technical but rather very political reasons. The case of mplayer has
been elaborated extensively in
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=732159#79 (see the
following "discussion" with Reimar - my conclusion from that is while
technically possible, nobody wants to make mplayer work with Libav -
and that's why it was removed, not because of the FFmpeg dependency).
For Mythtv I can only say that they didn't even bother engaging any
discussion.

My expertise here is extremely limited, but some practical experience
shows that mythtv does at least basically work fine with libav. I did
find one serious issue so far (in one version, on one piece of
hardware, so hardly extensive testing), but I have no data to know if
it is any better with ffmpeg, just a feeling that it might because
becuase that's what upstream test with. I believe the mythbuntu builds
are using libav rather than the internal ffmpeg, but I could be
wrong...? Who kows for sure?

I have just got another dev interested in helping with mythtv, so it
might still make it into jessie. And his help might give us time to delve
into this a bit further and fond out if using ffmpeg does make it
materially better or not. That testing is easier if ffmpeg is
available in debian, but obviously possible without.

Thanks for your response, which provides some useful info. I assure you
that it was worth your time.

Wookey

[Andreas Cadhalpun]

Hi Reinhard,

Post by Reinhard Tartler
I intended to come up with a more timely full response, but I just
didn't get to it so far.

Thanks for explaining your point of view here.

Post by Reinhard Tartler
For now, please refer to http://lwn.net/Articles/607662/,

Have a look at: http://lwn.net/Articles/608040/

I was not there, when the FFmpeg/Libav split happened and I don't want
to judge, whether it was a good thing or not. But it clearly caused a
lot of bad blood between the developers involved, which in my opinion
hurts the development of the multimedia framework in recent times.

Post by Reinhard Tartler
http://codecs.multimedia.cx/?p=370 (rather old, but still true), and

If these features weren't used, they wouldn't have been developed.
And many new features have been added to FFmpeg after that post.

Post by Reinhard Tartler
http://codecs.multimedia.cx/?p=674 (recent update on that matter)

Let me quote from there:
"But that’s not what kills the fun, “security holes” do.

With an advance of automatic fuzz tools it’s easy to generate millions
of damaged files that crash your decoder and yet there are no tools for
generating correct patches. Fixing those crashes is tedious, requires a
lot of thinking (should I disable it? will it affect decoding correct
files? etc.) and in other words not fun at all."

That seems as if he doesn't want to continue Libav development, because
fixing security issues is tedious work.
It has basically nothing to do with whether FFmpeg is of good quality
security wise or not, or a good or bad competitor against Libav.

Post by Reinhard Tartler
Regarding Marco's argument that libav had few friends, well, let me
point to https://medium.com/libre-parole/libav-is-beautiful-4c1a2c886948
for instance.

One person thinking that the code is 'beautiful' is no convincing
argument. The number of people expressing their interest in having
FFmpeg back in Debian is a lot more convincing.

Post by Reinhard Tartler

Post by Matthias Urlichs
IMHO the best idea at this point would be to toss out libav, and rebuild
the rdeps with ffmpeg. Now, before it's too late for jessie.

I think that is totally out of question: Uploading FFmpeg to our
archive will break the Debian archive so hard that it will take months
to get everything back to testing, if it works at all.

Let me repeat what I wrote in my initial mail in this thread:
Having FFmpeg in the Debian archive breaks absolutely *nothing*, but it
gives developers and users a choice between the two.

Even if Libav were to be removed, a transition to FFmpeg could be rather
smooth, as all the necessary patches already exist.
But you're right that the time to test the resulting packages is getting
rather short for the coming freeze.

Still it can make sense for packages that are tested with FFmpeg
upstream and have known issues with Libav.

So if you and the other Libav maintainers were to support the idea of
having both, the security and release teams could perhaps be convinced
to allow that. This has been my goal from the beginning and I hoped that
would be appreciated.

Post by Reinhard Tartler
To the best of my knowledge, there are only two high-profile projects
that play hardball to require FFmpeg: Mplayer and mythtv. Neither of
those do that (again to the best of my knowledge) mainly because of
technical but rather very political reasons. The case of mplayer has
been elaborated extensively in
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=732159#79 (see the
following "discussion" with Reimar - my conclusion from that is while
technically possible, nobody wants to make mplayer work with Libav -
and that's why it was removed, not because of the FFmpeg dependency).
For Mythtv I can only say that they didn't even bother engaging any
discussion.

That FFmpeg has more features is a rather technical argument.

Note that also many other projects are developed mainly with FFmpeg,
they just happen to not break completely when compiled against Libav.

For example, mpv prefers FFmpeg for good reasons:
"Although mpv attempts to work well with both FFmpeg and Libav, FFmpeg
is preferred in general. This is simply because FFmpeg merges from
Libav, and seems to have more features and fewer bugs than Libav." [1]

These features are actually requested by users, see e.g. [2].

Post by Reinhard Tartler
(All) other high-profile downstream projects, including VLC or XBMC
(now Kodi) work demonstrably just fine with Libav. Sure, FFmpeg may

Just fine? Did you see the bugs I mentioned in my initial mail?

And how does it come that XBMC needs the '--enable-libav-compat'
configure option, which then uses code from lib/xbmc-libav-hacks, to
build against Libav?

Post by Reinhard Tartler
provide them with extra features, but why on earth would anyone want 3
different implementations of a ProRes encoder (seriously, and FFmpeg
seems to claim to provide "security support" for each of them), or
support for fringe codecs such as Wing Command 3 (yes, you can decode
the videos from that video game).

One of those ProRes encoders comes from Libav and is provided in FFmpeg
for compatibility with Libav. Supporting both encoders provides the user
with additional choice.
Also security issues in an encoder are less likely, given that it
_encodes_ bitmaps into a bitstream and doesn't have to do complicated
parsing of attacker provided bitstreams into bitmaps like a decoder.

If FFmpeg wouldn't care so much about compatibility with Libav, a lot
less programs could be built with both.

Post by Reinhard Tartler
There are a number of legitimate requested backports, such as for some
of FFmpeg's additional filters in libavfilter, and similar. All of
them are rather easy to backport, and this is done on a regular basis.
However, with the due diligence and attention such a widely used and
high-profile library requires.

I'm not sure such ports to Libav would be easy, as I'm under the
impression that many filters in FFmpeg use features not available in Libav.

As far as I can tell, only 3 filters have been ported in the last two
years, including one submission by an external developer. Keeping in
mind that more than 100 additional filters have been added to FFmpeg
this is only a very small number.

The external submission came from a developer of a music player who uses
Debian/Ubuntu and therefore has to use Libav.
"I went with libav simply because it is what is in the Debian and Ubuntu
package managers, and one of my goals is to get this music player
backend into their package managers.
[...]
In order to have this functionality, I ported the "compand" audio filter
from ffmpeg." [3]

Post by Reinhard Tartler
Which brings me to my next point: Release frequency. FFmpeg has an
insane frequency of releases, and still seems to "support" (at least
providing updates) to all of them,

Generally FFmpeg supports all releases still widely used.

Post by Reinhard Tartler
including 0.5 which is currently in
oldstable (needless to say none of these patches made it to
oldstable-security, why is still a mystery to me).

It was a mystery to me, but I assumed you would know the reasons [4]:
ffmpeg (4:0.5.6-1) stable-security; urgency=low
[...]
* Note while the source package name reads 'ffmpeg', this is actually
the libav-0.5.6 release.

-- Reinhard Tartler <***@tauware.de> Mon, 26 Dec 2011 00:14:25 +0100

So even if the package in oldstable is called ffmpeg, it is actually
Libav and thus only got the security updates provided by Libav.

Nothing prevents you from uploading ffmpeg-0.5.14 to squeeze-lts.

Post by Reinhard Tartler
The effect of that
is that downstream projects have a hard time to choose what version of
FFmpeg they want to support.

What do you mean?
Every distribution includes the latest possible version and upstream
tracks how long they need to be supported [5].

Post by Reinhard Tartler
This brings effectively back to the
situation I was in when I took over maintenance of the package many
years ago: Back then, Michael Niedermeyer strongly recommended all
downstreams to avoid shared libraries and just link statically against
libavcodec.a to avoid problems regarding "incompatible" library
versions. I see exactly this problem arising again: Both mythtv and
mplayer upstream (btw, xbmc as well) bundle some copy of ffmpeg in
their sources and recommend everyone to just use the internal copy to
avoid problems.
Needless to say that this is not acceptable for use in Debian.

Both MPlayer and XBMC can be built against a system version of FFmpeg.

Post by Reinhard Tartler
Yes, I agree that this situation is a mess. A big mess. My fear is
that switching to FFmpeg will bring us back to the mess we where 8
years ago, and I worked on for years.

How can switching to FFmpeg undo the development of the last 8 years?
Especially since commits from Libav get merged on a daily basis.

Post by Reinhard Tartler
Libav is far from being perfect. That's true. I don't know why exactly
FFmpeg seems to have more manpower, but it's not all black or white.
For instance, there are a number of developers that actively
contribute to both projects and are essential in keeping both projects
in good health.

Indirectly all Libav contributors also contribute to FFmpeg, because
changes in Libav get merged into FFmpeg.

Post by Reinhard Tartler
Also I don't really buy the security argument. True,
FFmpeg has more security updates, but backporting them to Libav turned
out to be difficult because many if not most of them turn out to be
either incomplete, invalid or require further clarification. Libav
developers prefer to go the unpleasant road of fully understanding the
issue, which takes extra time.

If that extra time is about a year or more (CVE-2011-3946,
CVE-2013-0851, CVE-2013-0852, CVE-2013-0868) [6], I think it is clearly
far too long.

Post by Reinhard Tartler
For all these reasons, I do not see the
necessity to do any drastic and dangerous steps right now.
I now seriously wonder if the last 45 minutes in which I wrote this
email wouldn't have been better spent with preparing the next upload
for stable-security. YMMV.

It would be nice, if you could also spent some time thinking about the
possibility of having both FFmpeg and Libav in Debian.

Best regards,
Andreas


1: https://github.com/mpv-player/mpv/wiki/FFmpeg-versus-Libav
2: http://lwn.net/Articles/608012/
3: http://andrewkelley.me/post/quest-build-ultimate-music-player.html
4: https://tracker.debian.org/media/packages/f/ffmpeg/changelog-4%3A0.5.10-1
5: https://trac.ffmpeg.org/wiki/Downstreams
6: https://tracker.debian.org/media/packages/liba/libav/changelog-6%3A10.3-1

[Bálint Réczey]

Hi,

Post by Andreas Cadhalpun
Hi Reinhard,

Post by Reinhard Tartler
I intended to come up with a more timely full response, but I just
didn't get to it so far.

Thanks for explaining your point of view here.

Post by Reinhard Tartler
For now, please refer to http://lwn.net/Articles/607662/,

Have a look at: http://lwn.net/Articles/608040/
I was not there, when the FFmpeg/Libav split happened and I don't want to
judge, whether it was a good thing or not. But it clearly caused a lot of
bad blood between the developers involved, which in my opinion hurts the
development of the multimedia framework in recent times.

Post by Reinhard Tartler
http://codecs.multimedia.cx/?p=370 (rather old, but still true), and

If these features weren't used, they wouldn't have been developed.
And many new features have been added to FFmpeg after that post.

Post by Reinhard Tartler
http://codecs.multimedia.cx/?p=674 (recent update on that matter)

"But that’s not what kills the fun, “security holes” do.
With an advance of automatic fuzz tools it’s easy to generate millions of
damaged files that crash your decoder and yet there are no tools for
generating correct patches. Fixing those crashes is tedious, requires a lot
of thinking (should I disable it? will it affect decoding correct files?
etc.) and in other words not fun at all."
That seems as if he doesn't want to continue Libav development, because
fixing security issues is tedious work.
It has basically nothing to do with whether FFmpeg is of good quality
security wise or not, or a good or bad competitor against Libav.

Post by Reinhard Tartler
Regarding Marco's argument that libav had few friends, well, let me
point to https://medium.com/libre-parole/libav-is-beautiful-4c1a2c886948
for instance.

One person thinking that the code is 'beautiful' is no convincing argument.
The number of people expressing their interest in having FFmpeg back in
Debian is a lot more convincing.

Post by Reinhard Tartler

Post by Matthias Urlichs
IMHO the best idea at this point would be to toss out libav, and rebuild
the rdeps with ffmpeg. Now, before it's too late for jessie.

I think that is totally out of question: Uploading FFmpeg to our
archive will break the Debian archive so hard that it will take months
to get everything back to testing, if it works at all.

Having FFmpeg in the Debian archive breaks absolutely *nothing*, but it
gives developers and users a choice between the two.
Even if Libav were to be removed, a transition to FFmpeg could be rather
smooth, as all the necessary patches already exist.
But you're right that the time to test the resulting packages is getting
rather short for the coming freeze.
Still it can make sense for packages that are tested with FFmpeg upstream
and have known issues with Libav.
So if you and the other Libav maintainers were to support the idea of having
both, the security and release teams could perhaps be convinced to allow
that. This has been my goal from the beginning and I hoped that would be
appreciated.

Post by Reinhard Tartler
To the best of my knowledge, there are only two high-profile projects
that play hardball to require FFmpeg: Mplayer and mythtv. Neither of
those do that (again to the best of my knowledge) mainly because of
technical but rather very political reasons. The case of mplayer has
been elaborated extensively in
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=732159#79 (see the
following "discussion" with Reimar - my conclusion from that is while
technically possible, nobody wants to make mplayer work with Libav -
and that's why it was removed, not because of the FFmpeg dependency).
For Mythtv I can only say that they didn't even bother engaging any
discussion.

That FFmpeg has more features is a rather technical argument.
Note that also many other projects are developed mainly with FFmpeg, they
just happen to not break completely when compiled against Libav.
"Although mpv attempts to work well with both FFmpeg and Libav, FFmpeg is
preferred in general. This is simply because FFmpeg merges from Libav, and
seems to have more features and fewer bugs than Libav." [1]
These features are actually requested by users, see e.g. [2].

Post by Reinhard Tartler
(All) other high-profile downstream projects, including VLC or XBMC
(now Kodi) work demonstrably just fine with Libav. Sure, FFmpeg may

Just fine? Did you see the bugs I mentioned in my initial mail?
And how does it come that XBMC needs the '--enable-libav-compat' configure
option, which then uses code from lib/xbmc-libav-hacks, to build against
Libav?

Being the xbmc maintainer I feel being addressed and let me share my
POV regarding XBMC. :-)
XBMC works with Libav for most use-cases while it fails in the rest,
notably it can not use VDPAU acceleration which is being
(understandably) complained about very often (#742896). Another issue
is Libav crashing on bad input which makes XBMC/Libav unusable in PVR
configurations when signal is corrupted sometimes (#741170). Upstream
makes sure all their use-cases work well with FFmpeg and not
interested in Libav-related issues. I can't fix the problems because I
don't have any HW reproducing them, and I don't get help from Libav
upstream/maintainers either in fixing those issues.
Media requiring codecs not supported by Libav will not play with
Debian's pacakaged XBMC, but I have not seen any file encoded in such
way.

To overcome the situation I had to set up a "PPA" at
https://people.debian.org/~rbalint/ppa/xbmc-ffmpeg/ because I would
like to serve Debian users who would like to use VDPAU or PVR.
Maintaining this costs me extra time whenever I update XBMC in Debian
because I have to build and update those packages as well.

Regarding the '--enable-libav-compat' configure option and
lib/xbmc-libav-hacks they exist(ed) because XBMC's codebase is written
with FFmpeg API in mind and the compatibility with Libav 10 was
"hacked in" this way. I wrote "existed", because upstream removed the
support completely in their master branch. Most of my time spent to
XBMC packaging is spent on making sure that XBMC works with Libav and
many upstream developers oppose having XBMC packaged in Debian at all
because they want to see only fully functional XBMC builds everywhere.

I would like to have flawlessly working XBMC in Debian as well, but it
can't happen without fixing the Libav issues I mentioned above or
letting FFmpeg entering Debian.

IMO Andreas did a very good job and made re-introducing FFmpeg to
Debian as painless as possible. The package deserves to enter at least
experimental. Keeping it out of Debian just because and other
well-known fork is in Debian is totally unfair. We claim to create The
Universal Operating System not an App Store.

Cheers,
Balint

PS: I think Libav made a strategic mistake by not following Linux's
strategy of creating a "staging" area and not allowing less than
stellar code entering their codebase.

Post by Andreas Cadhalpun

Post by Reinhard Tartler
provide them with extra features, but why on earth would anyone want 3
different implementations of a ProRes encoder (seriously, and FFmpeg
seems to claim to provide "security support" for each of them), or
support for fringe codecs such as Wing Command 3 (yes, you can decode
the videos from that video game).

One of those ProRes encoders comes from Libav and is provided in FFmpeg for
compatibility with Libav. Supporting both encoders provides the user with
additional choice.
Also security issues in an encoder are less likely, given that it
_encodes_ bitmaps into a bitstream and doesn't have to do complicated
parsing of attacker provided bitstreams into bitmaps like a decoder.
If FFmpeg wouldn't care so much about compatibility with Libav, a lot less
programs could be built with both.

Post by Reinhard Tartler
There are a number of legitimate requested backports, such as for some
of FFmpeg's additional filters in libavfilter, and similar. All of
them are rather easy to backport, and this is done on a regular basis.
However, with the due diligence and attention such a widely used and
high-profile library requires.

I'm not sure such ports to Libav would be easy, as I'm under the impression
that many filters in FFmpeg use features not available in Libav.
As far as I can tell, only 3 filters have been ported in the last two years,
including one submission by an external developer. Keeping in mind that more
than 100 additional filters have been added to FFmpeg this is only a very
small number.
The external submission came from a developer of a music player who uses
Debian/Ubuntu and therefore has to use Libav.
"I went with libav simply because it is what is in the Debian and Ubuntu
package managers, and one of my goals is to get this music player backend
into their package managers.
[...]
In order to have this functionality, I ported the "compand" audio filter
from ffmpeg." [3]

Post by Reinhard Tartler
Which brings me to my next point: Release frequency. FFmpeg has an
insane frequency of releases, and still seems to "support" (at least
providing updates) to all of them,

Generally FFmpeg supports all releases still widely used.

Post by Reinhard Tartler
including 0.5 which is currently in
oldstable (needless to say none of these patches made it to
oldstable-security, why is still a mystery to me).

ffmpeg (4:0.5.6-1) stable-security; urgency=low
[...]
* Note while the source package name reads 'ffmpeg', this is actually
the libav-0.5.6 release.
So even if the package in oldstable is called ffmpeg, it is actually Libav
and thus only got the security updates provided by Libav.
Nothing prevents you from uploading ffmpeg-0.5.14 to squeeze-lts.

Post by Reinhard Tartler
The effect of that
is that downstream projects have a hard time to choose what version of
FFmpeg they want to support.

What do you mean?
Every distribution includes the latest possible version and upstream tracks
how long they need to be supported [5].

Post by Reinhard Tartler
This brings effectively back to the
situation I was in when I took over maintenance of the package many
years ago: Back then, Michael Niedermeyer strongly recommended all
downstreams to avoid shared libraries and just link statically against
libavcodec.a to avoid problems regarding "incompatible" library
versions. I see exactly this problem arising again: Both mythtv and
mplayer upstream (btw, xbmc as well) bundle some copy of ffmpeg in
their sources and recommend everyone to just use the internal copy to
avoid problems.
Needless to say that this is not acceptable for use in Debian.

Both MPlayer and XBMC can be built against a system version of FFmpeg.

Post by Reinhard Tartler
Yes, I agree that this situation is a mess. A big mess. My fear is
that switching to FFmpeg will bring us back to the mess we where 8
years ago, and I worked on for years.

How can switching to FFmpeg undo the development of the last 8 years?
Especially since commits from Libav get merged on a daily basis.

Post by Reinhard Tartler
Libav is far from being perfect. That's true. I don't know why exactly
FFmpeg seems to have more manpower, but it's not all black or white.
For instance, there are a number of developers that actively
contribute to both projects and are essential in keeping both projects
in good health.

Indirectly all Libav contributors also contribute to FFmpeg, because changes
in Libav get merged into FFmpeg.

Post by Reinhard Tartler
Also I don't really buy the security argument. True,
FFmpeg has more security updates, but backporting them to Libav turned
out to be difficult because many if not most of them turn out to be
either incomplete, invalid or require further clarification. Libav
developers prefer to go the unpleasant road of fully understanding the
issue, which takes extra time.

If that extra time is about a year or more (CVE-2011-3946, CVE-2013-0851,
CVE-2013-0852, CVE-2013-0868) [6], I think it is clearly far too long.

Post by Reinhard Tartler
For all these reasons, I do not see the
necessity to do any drastic and dangerous steps right now.
I now seriously wonder if the last 45 minutes in which I wrote this
email wouldn't have been better spent with preparing the next upload
for stable-security. YMMV.

It would be nice, if you could also spent some time thinking about the
possibility of having both FFmpeg and Libav in Debian.
Best regards,
Andreas
1: https://github.com/mpv-player/mpv/wiki/FFmpeg-versus-Libav
2: http://lwn.net/Articles/608012/
3: http://andrewkelley.me/post/quest-build-ultimate-music-player.html
4: https://tracker.debian.org/media/packages/f/ffmpeg/changelog-4%3A0.5.10-1
5: https://trac.ffmpeg.org/wiki/Downstreams
6: https://tracker.debian.org/media/packages/liba/libav/changelog-6%3A10.3-1

[Jonas Smedegaard]

Quoting Bálint Réczey (2014-08-09 11:38:54)

Post by Bálint Réczey
XBMC works with Libav for most use-cases while it fails in the rest,
notably it can not use VDPAU acceleration which is being
(understandably) complained about very often (#742896). Another issue
is Libav crashing on bad input which makes XBMC/Libav unusable in PVR
configurations when signal is corrupted sometimes (#741170).

Ok, so you know factually that some things are broken when linking XBMC
with Libav instead of XBMC-FFmpeg.

Post by Bálint Réczey
Upstream makes sure all their use-cases work well with FFmpeg and not
interested in Libav-related issues.

According to XBMC, they only make sure their code works with
XBMC-FFmpeg.

Post by Bálint Réczey
I can't fix the problems because I don't have any HW reproducing them,
and I don't get help from Libav upstream/maintainers either in fixing
those issues.

That sounds to me like you do not factually know if XBMC will be broken
also when linked against FFmeg (you only really know about XBMC-FFmpeg).

Post by Bálint Réczey
I would like to have flawlessly working XBMC in Debian as well, but it
can't happen without fixing the Libav issues I mentioned above or
letting FFmpeg entering Debian.

I do understand how it is easier for you to link XBMC against FFmpeg
than against Libav, since FFmpeg has similar/same API as XBMC-FFmpeg,
but it seems to me that you are jumping to conclusions when stating that
linking against (non-XBMC-)FFmpeg will make XBMC work "flawlessly".


- Jonas

[Bálint Réczey]

Quoting Bálint Réczey (2014-08-09 11:38:54)

Post by Bálint Réczey
XBMC works with Libav for most use-cases while it fails in the rest,
notably it can not use VDPAU acceleration which is being
(understandably) complained about very often (#742896). Another issue
is Libav crashing on bad input which makes XBMC/Libav unusable in PVR
configurations when signal is corrupted sometimes (#741170).

Ok, so you know factually that some things are broken when linking XBMC
with Libav instead of XBMC-FFmpeg.

Well, it depends on the definition of factually. I could not test the
VDPAU issues myself. :-)

Post by Bálint Réczey
Upstream makes sure all their use-cases work well with FFmpeg and not
interested in Libav-related issues.

According to XBMC, they only make sure their code works with
XBMC-FFmpeg.

Post by Bálint Réczey
I can't fix the problems because I don't have any HW reproducing them,
and I don't get help from Libav upstream/maintainers either in fixing
those issues.

That sounds to me like you do not factually know if XBMC will be broken
also when linked against FFmeg (you only really know about XBMC-FFmpeg).

Since XBMC switched to vanilla FFmpeg from their internal fork I would
be really surprised if XBMC did not work with the proposed new FFmpeg
packages. -dmo packages are built with external FFmpeg, too...
If this test is a deal-breaker for accepting FFmpeg into experimental
I can provide binaries for testing but probably most curious DD-s
having the right HW would be able to test if my theory is right.

Post by Bálint Réczey
I would like to have flawlessly working XBMC in Debian as well, but it
can't happen without fixing the Libav issues I mentioned above or
letting FFmpeg entering Debian.

I do understand how it is easier for you to link XBMC against FFmpeg
than against Libav, since FFmpeg has similar/same API as XBMC-FFmpeg,
but it seems to me that you are jumping to conclusions when stating that
linking against (non-XBMC-)FFmpeg will make XBMC work "flawlessly".

I would say it is not a mathematically correct reasoning, but a strong
expectation supported by observations.
Prove me wrong if you really think I'm missing something, I will stand
corrected. I made falsifiable statements.

By "flawlessly" I mean very close to upstream's expectations and
specifically fixing VDPAU and PVR issues I mentioned earlier.

Cheers,
Balint

[Jonas Smedegaard]

Quoting Bálint Réczey (2014-08-09 14:39:09)

Post by Bálint Réczey

Post by Jonas Smedegaard
Quoting Bálint Réczey (2014-08-09 11:38:54)

Post by Bálint Réczey
Upstream makes sure all their use-cases work well with FFmpeg and
not interested in Libav-related issues.

According to XBMC, they only make sure their code works with
XBMC-FFmpeg.

Post by Bálint Réczey
I can't fix the problems because I don't have any HW reproducing
them, and I don't get help from Libav upstream/maintainers either in
fixing those issues.

That sounds to me like you do not factually know if XBMC will be
broken also when linked against FFmeg (you only really know about
XBMC-FFmpeg).

Since XBMC switched to vanilla FFmpeg from their internal fork I would
be really surprised if XBMC did not work with the proposed new FFmpeg
packages.

Whoops - I confused XBMC and MythTV. Sorry for the noise.


- Jonas

[Alessio Treglia]

Hi,

Post by Matthias Urlichs
That leaves the question of the "official" opinion of the libav
Did none of them write anything in "defense" of libav, or have I simply
missed it?
IMHO the best idea at this point would be to toss out libav, and rebuild
the rdeps with ffmpeg. Now, before it's too late for jessie.

This is an eminently bad idea. As I said before, it's too late for
Jessie already.

Many Jessie's multimedia framework and packages have been developed
and QA'd with libav.

We've spent a lot of time over the past months talking to upstreams,
forwarding them our patches and make sure their programs and libraries
work with libav.
We've spent ***months*** in making the whole thing work, and dropping
libav in favour of FFmpeg at this point, roughly few weeks from the
freeze deadline, would be definitely insane.

Cheers.

[Matthias Urlichs]

Hi,

Post by Alessio Treglia
We've spent a lot of time over the past months talking to upstreams,
forwarding them our patches and make sure their programs and libraries
work with libav.
We've spent ***months*** in making the whole thing work, and dropping
libav in favour of FFmpeg at this point, roughly few weeks from the
freeze deadline, would be definitely insane.

Yes, that work might be "wasted". But I don't think that it's a good idea
to base the decision of whether or not X is better for Debian on the fact
that somebody did a lot of work for Y instead.

Yes, the freeze is not that far away. But frankly, how much effort would it
be to switch now? As far as I can tell from this discussion, the answer is
"not a whole lot". The bulk of ffmpeg/libav's reverse dependencies is just
a simple binNMU away, and as the libraries seem to be co-installable we
don't even need a big transition.

We'd also benefit from the fact that Upstream tends to use FFmpeg. I'd
hate to report some intractable codec bug which Upstream closes with an
"it works with FFmpeg" comment -- what would you recommend me to do in
that situation, next year -- install the ffmpeg libs from Experimental
and rebuild the offender?

[Jonas Smedegaard]

Quoting Matthias Urlichs (2014-08-08 21:22:38)

Post by Matthias Urlichs

Post by Alessio Treglia
We've spent a lot of time over the past months talking to upstreams,
forwarding them our patches and make sure their programs and
libraries work with libav.
We've spent ***months*** in making the whole thing work, and dropping
libav in favour of FFmpeg at this point, roughly few weeks from the
freeze deadline, would be definitely insane.

Yes, that work might be "wasted". But I don't think that it's a good
idea to base the decision of whether or not X is better for Debian on
the fact that somebody did a lot of work for Y instead.
Yes, the freeze is not that far away. But frankly, how much effort
would it be to switch now? As far as I can tell from this discussion,
the answer is "not a whole lot". The bulk of ffmpeg/libav's reverse
dependencies is just a simple binNMU away, and as the libraries seem
to be co-installable we don't even need a big transition.
We'd also benefit from the fact that Upstream tends to use FFmpeg. I'd
hate to report some intractable codec bug which Upstream closes with
an "it works with FFmpeg" comment -- what would you recommend me to do
in that situation, next year -- install the ffmpeg libs from
Experimental and rebuild the offender?

I would recommend you to take into account who is actually going to
maintain this in Debian. Perhaps those who've maintained libav in the
past will loose interest if switching to FFmpeg (or that it will attract
others who are far better at it or maintenance becomes much easier, but
I doubt that).

Just please distinguish between those in this thread telling what they
do themselves and those (bogusly!) dictating others to do stuff.


- Jonas

[Jean-Yves Avenard]

Hi.

Only going to reply to some of the misinformation provided in the post
from Reinhard Tartler <***@gmail.com>
(https://lists.debian.org/debian-devel/2014/08/msg00160.html)

For the background: I am the de-facto maintainer of the MythTV's FFmpeg fork.

Post by Reinhard Tartler
To the best of my knowledge, there are only two high-profile projects
that play hardball to require FFmpeg: Mplayer and mythtv. Neither of
those do that (again to the best of my knowledge) mainly because of
technical but rather very political reasons. The case of mplayer has
been elaborated extensively in
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=732159#79 (see the
following "discussion" with Reimar - my conclusion from that is while
technically possible, nobody wants to make mplayer work with Libav -
and that's why it was removed, not because of the FFmpeg dependency).
For Mythtv I can only say that they didn't even bother engaging any
discussion.

MythTV has always used its own internal fork of FFmpeg. I have been
involved in this project for close to a decade, and not once during
that time has MythTV been using a system libav*.

What political reasons are you referring to?
We were syncing against FFmpeg against before their childish fight
started, we continued syncing against FFmpeg after that. No more no
less.

One of the core developer of LibAV is Janne Grunau, who was also a
MythTV developer. He stopped being an active member of MythTV due to
lack of time.

MythTV do not work against stock FFmpeg and never will. We run a
heavily modified version of FFmpeg, you can call it a fork.
In particular, we use our own mpeg-ts demuxer, we support things like
MHEG, DVB/ATSC/Teletext subtitles that FFmpeg do not.
We have attempted for many years to get our changes merged in FFmpeg
but gave up.

You say MythTV didn't bother engaging any discussions? what
discussions are you referring to ? I certainly have never been
contacted by either teams.

What I can recall however is me submitting an issue to both FFmpeg and
LibAV trac. The problem was dismissed in LibAV and fixed within a
couple of days in FFmpeg.
LibAV had decided to totally remove some core APIs that we make use of
(in this case it was fast mpeg decoding used by our commercial
detection system).
FFmpeg re-introduced it at our demand.

FFmpeg also have kept the old VDPAU decoding API, LibAV removed it.

As such, it is easier for us to sync against FFmpeg than it is against
LibAV. I call this technical reasons. there's nothing political into
it.

FFmpeg cherry-pick or merge any changes occurring in LibAV in a matter
of days. So by syncing against FFmpeg we enjoy the best of both
worlds.

To end this message: I fail to see how debian's decision on which
version of FFmpeg or LibAV would have any impact on MythTV, we use
neither

[Matthias Urlichs]

Hi,

Post by Jean-Yves Avenard
We have attempted for many years to get our changes merged in FFmpeg
but gave up.

It might be a good idea to restart this effort.

Post by Jean-Yves Avenard
To end this message: I fail to see how debian's decision on which
version of FFmpeg or LibAV would have any impact on MythTV, we use
neither

Most forks cause additional work which, in the long term, is better spent
elsewhere. The ffmpeg/libav split is ample proof of that; in an ideal
world, you wouldn't need the mythtv fork either.

Debian's position is that we _really_ want to avoid having multiple copies
of essentially the same code in the archive; it's additional work for the
security team (if they even know about the copies), needlessly eats memory
when the two are in use simultaneously, and causes no end of trouble when a
plug-in is linked against copy A while the main code (or another plugin)
uses copy B (unless everybody is very careful WRT symbol versioning).

[Kieran Kunhya]

Post by Matthias Urlichs
Most forks cause additional work which, in the long term, is better spent
elsewhere. The ffmpeg/libav split is ample proof of that; in an ideal
world, you wouldn't need the mythtv fork either.
Debian's position is that we _really_ want to avoid having multiple copies
of essentially the same code in the archive; it's additional work for the
security team (if they even know about the copies), needlessly eats memory
when the two are in use simultaneously, and causes no end of trouble when a
plug-in is linked against copy A while the main code (or another plugin)
uses copy B (unless everybody is very careful WRT symbol versioning).

The reality is that in the current state of affairs static linking is
the *only* way you are guaranteed to have the features you expect and
avoid ABI mismatches. It's very complicated when your users complain
about bugs in an underlying library that is either too old on a system
or the wrong flavour of the fork. Then your users have to rebuild a
massive dependency chain to fix that bug or hope for the best in the
future.

We also use a fork specifically to work around very wasteful
calculations in libswscale during 10-bit chroma conversion that
involve multiplying a pixel by a 2^n value with 32-bit precision and
then shifting that value down by n back to 16-bit precision (achieving
nothing). The fix breaks other codepaths that we don't use but the
performance gain is big enough to warrant a fork.

FWIW I think debian should also enable libavresample. This code has
been proven to be more robust internally and was designed by a larger
group instead of libswresample which was basically one person (and
literally appeared in git out of nowhere).

Kieran

[Andreas Cadhalpun]

Hi Kieran,

Post by Kieran Kunhya
The reality is that in the current state of affairs static linking is
the *only* way you are guaranteed to have the features you expect and
avoid ABI mismatches. It's very complicated when your users complain
about bugs in an underlying library that is either too old on a system
or the wrong flavour of the fork. Then your users have to rebuild a
massive dependency chain to fix that bug or hope for the best in the
future.

I can understand that statically linking is easier from an upstream
point of view, but it has important disadvantages for a distribution
such as Debian and thus should be avoided if possible.
It is also the responsibility of a distribution to make sure that there
are no ABI mismatches or similar problems.

Post by Kieran Kunhya
We also use a fork specifically to work around very wasteful
calculations in libswscale during 10-bit chroma conversion that
involve multiplying a pixel by a 2^n value with 32-bit precision and
then shifting that value down by n back to 16-bit precision (achieving
nothing). The fix breaks other codepaths that we don't use but the
performance gain is big enough to warrant a fork.

Can you provide a commit/diff/link for reference?
The way you describe it makes it appear as if these calculations are
needed sometimes, but not always. If so, there should be away to teach
libswscale to only do these calculations if they are necessary.

Post by Kieran Kunhya
FWIW I think debian should also enable libavresample. This code has
been proven to be more robust internally and was designed by a larger
group instead of libswresample which was basically one person (and
literally appeared in git out of nowhere).

FWIW I already enabled libavresample in the Debian FFmpeg package for
compatibility reasons [1].
Still I would be interested in any proof of libavresample being 'more
robust internally'.

Best regards,
Andreas

1:
https://anonscm.debian.org/cgit/collab-maint/ffmpeg.git/commit/?id=e6d32135ec5ada648465aba8d4daad58b86ff8d0

[Kieran Kunhya]

On 9 August 2014 19:25, Andreas Cadhalpun

I can understand that statically linking is easier from an upstream point of
view, but it has important disadvantages for a distribution such as Debian
and thus should be avoided if possible.
It is also the responsibility of a distribution to make sure that there are
no ABI mismatches or similar problems.

I'm not saying Debian should do that - just that it is sometimes
worthwhile for people like myself and mythtv to do so.

Can you provide a commit/diff/link for reference?
The way you describe it makes it appear as if these calculations are needed
sometimes, but not always. If so, there should be away to teach libswscale
to only do these calculations if they are necessary.

I hacked it in:
https://github.com/kierank/ffmpeg-obe/commit/5a354020872bec61bea6be7604ec8809af463021

I made it a bit cleaner later but still ugly overall.

Kieran

[Michael Niedermayer]

On Sat, Aug 09, 2014 at 06:26:19PM +0100, Kieran Kunhya wrote:
[...]

... and was designed by a larger
group instead of libswresample which was basically one person (and
literally appeared in git out of nowhere).

For the record:
$ git shortlog libswresample/ | grep '^[^ ]'
Alexander Strasser (1):
Andreas Cadhalpun (1):
Andrew Wason (1):
Carl Eugen Hoyos (4):
Clément Bœsch (45):
Derek Buitenhuis (1):
Hendrik Leppkes (1):
James Almer (19):
Justin Ruggles (4):
Lou Logan (2):
Mans Rullgard (3):
Martin Storsjö (1):
Marton Balint (2):
Matt Oliver (2):
Michael Niedermayer (308):
Nicolas George (8):
Paul B Mahol (4):
Reimar Döffinger (3):
Rob Sykes (7):
Ronald S. Bultje (15):
Stefano Sabatini (6):
Timothy Gu (10):
jamal (2):

$ git shortlog libavresample/ | grep '^[^ ]'
Anton Khirnov (25):
Derek Buitenhuis (2):
Diego Biurrun (22):
Hendrik Leppkes (1):
James Almer (2):
Janne Grunau (5):
John Stebbins (2):
Justin Ruggles (71):
Luca Barbato (1):
Mans Rullgard (4):
Martin Storsjö (6):
Michael Niedermayer (97):
Peter Meerwald (1):
Reimar Döffinger (2):
Ronald S. Bultje (2):
Thilo Borgmann (1):
Tim Walker (2):


[...]

[Kieran Kunhya]

[...]

... and was designed by a larger
group instead of libswresample which was basically one person (and
literally appeared in git out of nowhere).

http://git.videolan.org/?p=ffmpeg.git;a=commitdiff;h=b5875b91113a0f3de6ad61e9ff8b74b81de94760

There was no mailing list discussion and initial bugs had to be
discussed on ffmpeg-cvs. It appeared out of nowhere and there was no
discussion about the API. Many of the contributors were cleaning the
inline asm up or various other fixes or had worked on the original
resample code.

The comparison you make is highly misleading. On the other hand
libavresample was developed by consensus and the API discussed
beforehand.

[Jean-Yves Avenard]

Post by Matthias Urlichs
Most forks cause additional work which, in the long term, is better spent
elsewhere. The ffmpeg/libav split is ample proof of that; in an ideal
world, you wouldn't need the mythtv fork either.
Debian's position is that we _really_ want to avoid having multiple copies
of essentially the same code in the archive; it's additional work for the
security team (if they even know about the copies), needlessly eats memory
when the two are in use simultaneously, and causes no end of trouble when a
plug-in is linked against copy A while the main code (or another plugin)
uses copy B (unless everybody is very careful WRT symbol versioning).

I beg to differ.

What would be a massive amount of waste, both in time and resources,
would be to have a project such as MythTV having to handle multiple
versions of a dependency.

Especially one such as libav* where the API changes often, and more
often than not in a totally incompatible manner from one API to the
next.
Including rename of constants (code enums id for example). You would
have to keep your own headers (like what Firefox/Mozilla is doing) and
have multiple code paths only adding to the difficulty of proper
coverage testing...

Keeping your own static version is the only reasonable approach.
As far as naming and conflicts, I don't see what the problem is unless
it's improperly packaged, or for example when the packagers decide
that they know better than the original authors and start to do weird
thing, modify the code as they wish. That's where the issues are most
of the time.
Sounds familiar ? :)

[Matthias Urlichs]

Hi,

Post by Jean-Yves Avenard
Including rename of constants (code enums id for example).

Another nail in libav's coffin, then.

IMHO it's reasonable to expect core APIs to be upwards-compatible and keep
deprecated interfaces around for another release or two.

Post by Jean-Yves Avenard
Keeping your own static version is the only reasonable approach.

That may be OK on Windows. However, a proper Linux distribution is supposed
to be an integrated whole and not a haphazard collection of programs, each
bringing along their own copy of core libraries and their own un- or
semi-fixed security problems.

[Andrew Kelley]

Post by Matthias Urlichs

Post by Jean-Yves Avenard
Including rename of constants (code enums id for example).

Another nail in libav's coffin, then.
IMHO it's reasonable to expect core APIs to be upwards-compatible and keep
deprecated interfaces around for another release or two.

High quality libraries must iterate on their API. Especially for a library
trying to solve such a complex problem as audio and video encoding and
decoding for every codec and format. It is unreasonable to expect no
incompatible changes. Also both ffmpeg and libav codebases have a lot of
legacy cruft. Libav is making a more concentrated effort at improving this,
and the evolving API is a side-effect of that.

[Matthias Urlichs]

Hi,

It is unreasonable to expect no incompatible changes.

When somebody renames constants, a compatibility #ifdef or two is not too
much to ask, IMHO.

Libav is making a more concentrated effort at improving this,
and the evolving API is a side-effect of that.

That begs the question whether they're still essentially the same library,
i.e. whether it's at all reasonable to expect code built using FFmpeg to
work with libav. Consequently, Debian should at least include it, even
if it really _is_ too late to switch (which I'm not conviced of).

[Jean-Yves Avenard]

Hi

Post by Matthias Urlichs
Hi,
IMHO it's reasonable to expect core APIs to be upwards-compatible and keep
deprecated interfaces around for another release or two.

Then it becomes unreasonable for a piece of software to be compatible
with multiple version of the same library, and support all of those.

When a used come to use complaining that something is broken, that a
file doesn't play or what else. It's a moot point trying to expect
them to understand that the issue is due to a 3rd party library.

MythTV aimed to be an appliance-like application. You install it and
you forget about it.

[Andrew Kelley]

Post by Jean-Yves Avenard
Then it becomes unreasonable for a piece of software to be compatible
with multiple version of the same library, and support all of those.

IMO it's not worth the effort to support multiple versions of the same
library. If you want to use an old library, use an old version of the
software.

[Jean-Yves Avenard]

Post by Andrew Kelley
IMO it's not worth the effort to support multiple versions of the same
library. If you want to use an old library, use an old version of the
software.

In our case, we have very long release cycles. Usually only once a
year at best. We have users on virtually all platforms. We can't
assume the OS/distribution it will be running on will have the library
we're hoping for.

In the perfect world, sure all platforms would all be running the same
versions of a given lib at a given time... In practice it never
happens.

[Reinhard Tartler]

Post by Matthias Urlichs
Hi,

Post by Jean-Yves Avenard
Including rename of constants (code enums id for example).

Another nail in libav's coffin, then.

That's one way to see it. To me, this makes mythtv unsuitable for

Post by Matthias Urlichs
IMHO it's reasonable to expect core APIs to be upwards-compatible and keep
deprecated interfaces around for another release or two.

This is exactly what Libav is doing: The deprecation process for
symbols, APIs, enums, etc. takes *years*, because so many software
packages in Debian and else where use them, and it is so believably
painful to change them. Just have a look at the last two Libav
transitions, and the massive amount of patches it took to get packages
fixed and eventually to get Debian to the new Libav release.

Now enter FFmpeg.

FFmpeg has a significant higher release frequency, (it seems to me
about every 3-4 months), so that you would get a deprecation cycle
that is considerably less than a year. In practice, the deprecation
cycle more or less seems to match Libav's cycle, because at least
right now, FFmpeg tracks Libav's API. If that were not the case (and
I promise you FFmpeg would stop tracking Libav as soon as it replaces
Libav in Debian), I can almost guarantee [1] you that FFmpeg would
very much prefer to resume to the deprecation cycle the project
before: None, i.e., every piece of software is expected to keep up
with FFmpeg's master branch for reasons Jean-Yves outlines.

[1] at least statements such as
https://ffmpeg.org/pipermail/ffmpeg-devel/2014-August/160876.html
strongly suggest this (at least if you have followed the
libavresample/libswresample mess).

Post by Matthias Urlichs

Post by Jean-Yves Avenard
Keeping your own static version is the only reasonable approach.

That may be OK on Windows. However, a proper Linux distribution is supposed
to be an integrated whole and not a haphazard collection of programs, each
bringing along their own copy of core libraries and their own un- or
semi-fixed security problems.

BTW Jean-Yves outlines an approach that used to be very common on the
past: Pick some particular snapshot of FFmpeg and maintain that in a
downstream project, and expect users to use that because it is too
much effort to keep up with FFmpeg's release frequency. Prominent
examples of projects that did this (and actually, still do) include
xine-lib, mplayer, xbmc, and many more. This lead exactly to the mess
I was talking about in my previous email: dozens of embedded
code-copies that were accepted into the Debian archive.

Over many years, I've spearheaded a significant effort in Debian with
packaging and in upstream with introducing a release culture in FFmpeg
(as release manager) to get to somewhat same release frequencies, so
that downstream projects at least had a chance to agree on common
versions of FFmpeg. At the time of the split, I was worried that this
work would have been in vain. Considering that most active developers
of FFmpeg at that time (which coincidentally supported my approach to
release management and frequency) joined what is now known as Libav, I
continued my work as upstream release manager in Libav, because I
consider Libav as much more suited for Debian than FFmpeg. Today, I
still firmly believe that this was the right move for Debian as a
project.

I do strongly believe that projects that require people to use FFmpeg
actually mean to use their own private fork (cf. the mythtv debacle),
and given the amount of packages in Debian, it would significantly
much more effort to "port" (or "patch" as Andreas is phrasing it) them
to some common version of FFmpeg than doing what we are doing now:
Making sure they work with the version of Libav's libavcodec.so
implementation. This thread has shown a couple of examples that
support this argument: Mythtv, but also mplayer that claims to work
with a system libavcodec.so, which is true as long as it matches the
version that is was built against. This is what makes mplayer so hard
to package, and was ultimately the reason why I requested mplayer's
removal (which is more than ironic, given that back then, I fought
with ftp-master for many years to get it included into Debian in the
first place).

On a related note: Most Libav developers are very tired of the
constant flamewars and defamation attempts that arises from FFmpeg.
Over years, Libav tries to convinced everyone by providing usable
software releases. Nevertheless, this particular debate is very
worrisome: The silence from the Libav camp seems to not to be taken as
consent. Quite the contrary is true.

How to proceed from here? TBH, I'm not sure. Ideally, both projects
would find some common ground and "just merge" (however that would
technically look like). However, this very debate within Debian shows
that this is unlikely to happen anytime soon: There is way to much
disagreement on very fundamental questions that require agreement
within a free software project, and the hostile and aggressive tone
the majority of participants in this debate exhibit does not help with
making progress on that front either.

[Peter B.]

Post by Matthias Urlichs
We'd also benefit from the fact that Upstream tends to use FFmpeg. I'd
hate to report some intractable codec bug which Upstream closes with
an "it works with FFmpeg" comment

Oh, btw, just a few days ago, that's exactly what happened on kdenlive [1].
A developer posted the following statement on an issue which is open for
more than 1.5 years now:

[quote]
Confirm this is a libav problem, use builds with ffmpeg or wait debian
(&derivatives) to bring ffmpeg back
[/quote]

Just thought this might fit here...


Regards,
Pb


== References:
[1] http://www.kdenlive.org/mantis/view.php?id=2943#c10195

[Jose Luis Rivas]

Post by Andreas Cadhalpun
Hi all,
some of you may have noticed a weird ffmpeg package in the NEW queue[1].
In 2011 Libav[2] was forked from FFmpeg[3]. It was a time of great
uncertainty, the fork happened with much drama that didn't help making a
technical cut, and at that peculiar time Debian switched to Libav.

Hi Andreas and everyone,

FWIW, my experience with this is that I had to make my own FFmpeg
package a while ago [0] because I needed it for a project I was working
on at the moment [1].

[0] https://github.com/ghostbar/FFmpeg.deb
[1] https://github.com/ghostbar/RTSP-Streaming.js

The reason for having to package my own FFmpeg is the current libav
which is taking the space of ffmpeg seemed to conflict with every other
ffmpeg package out there, including marillat's and for my project I
actually needed ffmpeg, not libav since it didn't had the functionality.
(More specifically: the ability to take still images from an rtsp
stream).

Not having FFmpeg available in the debian repositories is a nuissance,
and certainly having libav instead which seems to be a fork yet not
having the full FFmpeg functionality and using the same package name is
worst. I didn't figured this out at first because the binary said
`ffmpeg`. Of course, I'm talking about [2] since now that seems to not
be an issue yet remains the lack of functionality.

[2] https://packages.debian.org/wheezy/ffmpeg

If the issue is that this would mean having to fix security bugs twice
then it would be reasonable to stop shipping libav and instead ship
ffmpeg, since has more functionality and AFAICS from their repos bunch
of active bug-fixing.

I honestly do not understand why ffmpeg is not in the repos nor why
there seems to be an active movement to block it.

Kind regards.

[Niv Sardi]

I feel the debate is going a bit on a tangent in this thread, so I'd like
to take an opportunity to recenter it a tad.​

​We have many issues that were risen in this thread, ​
​but ​
I believe that the cut has to be made by the people that we have in special
roles for; -security for security concerns, -release for release scheduling
and required transitions and our many maintainers of multimedia related
packages to know what they want to link their packages against
​​
​, and well, as it has been named, tech-ctte for technical matters that we
can't resolve in -devel.​

​In few words​
,
​this is how I understand​
the core of Andreas' plan, and has motivated many technical decisions in
the packaging.

​Many ​
​​
sensible
​ technical decisions that lead to a polite and correct way to go.

Now, our main blocker to get onto the next steps Andreas has exposed with
great detail in his original mail is that the package has been sitting in
NEW for the last
​3​
months.

As people voiced their feelings, it seems the vast majority does not
opposes having FFmpeg in experimental
​​.

So my point here is:
​Shouldn't we first get it in the archive ? ​
how can we help speed​ing
​ ​
it out of NEW ?

​The current packaging is a 'low-conflict' one, it can be easily
transitioned into replacing libav if that's what gets decided further down
the line. Hence, I see no blocker​ in getting it in.

The licencing has an ancestor with libav, and the new files have been
following the same inclusion pattern. So my guess is that the blocker may
be packaging quality, as the sponsor, I have checked it, but maybe we can
help reconfort ftpmasters with more eyes on it ?
​Thanks,​

[Wookey]

Post by Andreas Cadhalpun
Hi all,
some of you may have noticed a weird ffmpeg package in the NEW queue[1].
* Ok, let's say I'm a multimedia maintainer and want to try out
building my package against your ffmpeg, what should I do?
Any maintainer interested in switching should get in touch, and we'll
gladly help preparing the transition.
The FFmpeg version currently in NEW has been there for more than
2 months and is thus outdated. If you want to test the current
packages, you can build them from the repository on Alioth [17]
(e.g. using git-buildpackage).

Thanks for doing this work. I too am a bystander in the ffmpeg/libav
debate but I have an interest due to having fixed up the ubuntu and
deb-multimedia mythtv packages for Debian:

https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=570611
http://wookware.org/software/repo/pool/main/m/mythtv/

Those packages work and and been patched to use libav. However I have
found a few bugs in real usage (like crashing after 40mins). Because
upstream deveop exclusively with ffmpeg it would be very useful to
rebuild tose packages against ffmpeg and see if they work the
same/better/worse.

Thanks for providing info on how to do this. Personally I'd like to
see ffmpeg in debian so I could trivially build against it and (quite
probably) use it by default if available, simply because it's inline
with upstream and less likely to break.

I really don't see sufficient reasons why we shouldn't at least put it
experimental so that maintainers can easily test this stuff. Yes the
whole fork thing has worked out most unfortunately, but ISTM that
Andreas' packaging is a sensible way to let maintainers use this
package or libav and easily test with both.

Yes, in theory I could fix up any bugs due to divergence between the
two libraries but that's not going to happen in practice as I know
nothing of multimedia libraries and codecs. I just want a mythtv
package in Debian (as opposed to debian-multimedia and ubuntu where
they have been for years), and am happy to do packaging-level work to
make that happen.

(I've also been a tad busy with arm64 and cross toolchains and
bootstrapping to make anything other than very slow progress on this
project...)

But overall, I vote for letting it through new.

Wookey

[Michael Gilbert]

Post by Wookey
I really don't see sufficient reasons why we shouldn't at least put it
experimental so that maintainers can easily test this stuff.

The problem is an undermanned ftpmaster team [0], so help there is
probably appreciated and the obvious way to bring this thread to
conclusion.

Best wishes,
Mike

[0] https://ftp-master.debian.org/stat.html

[Thorsten Alteholz]

Post by Michael Gilbert
The problem is an undermanned ftpmaster team [0], so help there is

I have to object here. If I read something like:
<XXX> please reject YYY
<XXX> forgot to add the full MPL 2.0 text..
I wonder why this hadn't been checked before the upload.
If all maintainers would do their homework, the situation of NEW might be
much better. I estimate that 80% of all packages in NEW, that are older
than two weeks, do have a problem like above.
So dear fellow maintainers, although it is quite boring, please pay more
attention to the Debian Policy.

Thorsten

[Charles Plessy]

Post by Thorsten Alteholz

Post by Michael Gilbert
The problem is an undermanned ftpmaster team [0], so help there is

<XXX> please reject YYY
<XXX> forgot to add the full MPL 2.0 text..
I wonder why this hadn't been checked before the upload.
If all maintainers would do their homework, the situation of NEW
might be much better. I estimate that 80% of all packages in NEW,
that are older than two weeks, do have a problem like above.
So dear fellow maintainers, although it is quite boring, please pay
more attention to the Debian Policy.

Hi Thorsten,

I fully agree.

A few years ago, I made a proposal for peer-reviewing copyright files in the
NEW queue.

https://wiki.debian.org/CopyrightReview

The goal is not to substitute for the inspection by the FTP Master team, but to
correct defects before their review, therefore saving their time.

I have done a few dozens of these reviews and share Thorsten's impression in
general (althouth in my opinion 80 % is quite an upper-range estimate…).

I encourage everybody who uploads to the NEW queue to review some packages in
exchange. To help people reviewing your package, please make sure that a
copy is accessible (source packages in the NEW queue are not accessible outside
the FTP Master team).

(Regarding the requests for removal from the NEW queue, perhaps somebody can
write a patch to DAK so that it can be dealt with a DAK command file ?)

Have a nice day,
--
Charles Plessy
Debian Med packaging team,
http://www.debian.org/devel/debian-med
Tsurumi, Kanagawa, Japan

[Andreas Cadhalpun]

user debian-***@lists.debian.org
usertags 729203 copyright-review-requested
thanks

Hi Charles,

Post by Charles Plessy
A few years ago, I made a proposal for peer-reviewing copyright files in the
NEW queue.
https://wiki.debian.org/CopyrightReview
The goal is not to substitute for the inspection by the FTP Master team, but to
correct defects before their review, therefore saving their time.

This looks like a good idea, but unfortunately it seems not to be an
often used process.

Post by Charles Plessy
I have done a few dozens of these reviews and share Thorsten's impression in
general (althouth in my opinion 80 % is quite an upper-range estimate…).

I have no accurate numbers, but I just reviewed three packages [1-3] and
found problems in all of them. It's a rather small sample size, but still...

Post by Charles Plessy
I encourage everybody who uploads to the NEW queue to review some packages in
exchange. To help people reviewing your package, please make sure that a
copy is accessible (source packages in the NEW queue are not accessible outside
the FTP Master team).

Now, could anyone review the debian/copyright file of ffmpeg?
The sources are available in this repository:
https://anonscm.debian.org/cgit/collab-maint/ffmpeg.git

Best regards,
Andreas

1: https://bugs.debian.org/686447
2: https://bugs.debian.org/735884
3: https://bugs.debian.org/683746

[Charles Plessy]

user debian-***@lists.debian.org
usertags 729203 one-copyright-review
thanks

Post by Andreas Cadhalpun
Now, could anyone review the debian/copyright file of ffmpeg?
https://anonscm.debian.org/cgit/collab-maint/ffmpeg.git

Hi Andreas,

I searched for license information missing from your debian/copyright and could
find only one case, libavutil/x86/x86inc.asm, which is under the ISC license.

The debian/copyright file of your package looks comprehensive to me.

Have a nice week-end,

[Andreas Cadhalpun]

Hi Charles,

Post by Charles Plessy
I searched for license information missing from your debian/copyright and could
find only one case, libavutil/x86/x86inc.asm, which is under the ISC license.
The debian/copyright file of your package looks comprehensive to me.

Many thanks for the copyright review. (I know it is a lot of work.)

I added the missing information you found (and also uppercased some
license names to match the copyright format specification) [1].

It's probably not necessary to make a new upload to the NEW queue for
this change. In the repository is a new upstream version anyway and it
will be uploaded, once the current version gets accepted.

Best regards,
Andreas


1:
https://anonscm.debian.org/cgit/collab-maint/ffmpeg.git/commit/?id=d5f7788c60951684851ac8ef7fbb468bd385cdeb

[Michael Gilbert]

It's probably not necessary to make a new upload to the NEW queue for this
change. In the repository is a new upstream version anyway and it will be
uploaded, once the current version gets accepted.

Based on the license review, you should anticipate a REJECT of the
first upload, so you'll need to upload the second one anyway. You
might as well do the second one now and save everyone time.

Best wishes,
Mike

[Joe Neal]

On both servers and desktops, I've been a Debian user since Sarge. I
use Debian not only because of its strong technical merits, but because
of the strong sense of ethics the project has always had.

A fork that tries to forcibly steal the name and infrastructure from
the original project while ousting the original maintainer and a good
number of developers is not ethical. In fact, it's pure slime. The
fact that up until this point the Debian Project has been so accepting
of a fork that runs so contrary to the free software spirit in its
ethics has darkened my view of the entire project.