[Libav-user] Would anyone find an OS X / Cocoa / Swift wrapper for Libav useful?

[Bradley O'Hearne]

All,

I’ve been on the Libav-user mailing list 2-3 years now, having worked on FFmpeg integration for various clients. While I develop on Windows and Linux platforms, my primary development has been spent over the past 6 years has been on OS X with respect to desktop (my core business is mobile development: iOS, Android, and newly Windows Phone). Over the past two years, my work with FFmpeg has been primarily on the OS X platform.

During that time, I have run into a number of issues about which I’ve appealed to this mailing list, and one of the things which various list members (some of which were, as I understand it, Libav devs) raised was that most (if not all) of the Libav devs did not either use or have access to OS X to be able to speak to or support problems which manifested on OS X, and I strongly suspect platform issues could have influenced some of the problems experienced. In one case, someone even encouraged me to just simply “change platforms”, which of course cannot always be done, especially if you are serving clients who have hired you to make things work on that specific platform.

This is not the best situation, and it is kind of a dubious label for Libav to be declared as working and/or supported on OS X if there are no devs using OS X or supporting it. I have generally found answers to most of the issues I have had to work through, but finding those answers has been slow-going, and a tedious process. My purpose here is to poll the list members to ask if anyone would find any value at all if someone created an OS X / Cocoa / Swift (and possibly iOS) wrapper for Libav?

Perhaps I’m the only one on the planet using Libav on Apple platforms, though I’m betting I’m not (actually I know I’m not, as a few have contacted me off-list). Also, it might take some of the headache away from the Libav devs who don’t use Apple platforms to answer some support questions. For us who like apples, a nice, clean Swift API might be very nice, and save a lot of time and headaches. I might be able to produce such a thing, first somewhat limited and rudimentary, and then mature it over time.

Would anyone be interested in such an API?

Thank you in advance for your replies.

Cheers,

Brad

Brad O'Hearne
Founder/Lead Developer
Big Hill Software LLC

[David T.]

>>My purpose here is to poll the list members to ask if anyone would find any value at all if someone created an OS X / Cocoa / Swift (and possibly iOS) wrapper for Libav?

Hi! Brad,

My company use FFmpeg on OS X. We normally just compile it for darwin and call the functions from Objective-C syntax. I'm sure that you have done all this before.

So far we have delivered several projects on iOS. But we have not found any deficiency (which you didn't describe)  on using FFmpeg on OS X. Perhaps you could tell us exactly what you expect a new API should do compared to current conventions, what would a new API solve?

[Bradley O'Hearne]

> On Jan 22, 2015, at 9:10 PM, David T. <***@gmail.com> wrote:
>
>>> My purpose here is to poll the list members to ask if anyone would find any value at all if someone created an OS X / Cocoa / Swift (and possibly iOS) wrapper for Libav?
>
> So far we have delivered several projects on iOS. But we have not found any deficiency (which you didn't describe) on using FFmpeg on OS X. Perhaps you could tell us exactly what you expect a new API should do compared to current conventions, what would a new API solve?

Hey David — thanks so much for your reply — I was wondering if this even piqued anyone’s interest, so I’m really happy to be able to dialog with someone on this. My purpose here isn’t to recite a laundry list of problems encountered, or debate design issues, or poke holes in FFmpeg in general. Suffice to say that I think the degree to which issues are encountered depends heavily on your use-case. Those that have use-cases which are a bit off the beaten path (of FFmpeg samples, for example) probably have run into various issues…I’ve had people periodically contact me off-list for issues similar to the ones I’ve encountered, so I know I’m not the only one running into them. But again, my purpose is not to debate FFmpeg use cases or problems.

What I believe would be more universally agreeable (and safer turf for discussion, which is my desire) is the fact that FFmpeg is very scantily documented and very confusing to get started with. Options are extremely limited if you run into a problem and generally lead to hours upon hours of reading source code and debugging, with little other recourse. I don’t think it would be anything controversial to say that FFmpeg doesn’t offer what I’d term a “consumer’s API” — that is, an API that is built for intuitiveness and easy use by third parties, which hides its nasty internal implementation details and implications. I would characterize FFmpeg the same way I’d characterize many technical books that are out there — created for those with the foreknowledge of the authors, rather than those coming in cold.

More to the specific case of OS X and iOS — those developing on those platforms will likely be using one or more of the following frameworks: QTKit, AVFoundation, AudioToolbox, VideoToolbox. It would be very nice to be able to deal in terms of:

a) Data structures native to those frameworks, and

b) Swift.

I believe with such an API, the learning curve would be much smoother, some of the nastiness of the FFmpeg API’s could be hidden, and a lot of common mistakes could be eliminated. I also think that some very common use cases could be encapsulated into simple, well-documented API calls. While it might not express the entire richness or capability of FFmpeg, it might handle some of the common stuff well.

I hope that communicates with some clarity the general idea. Let me know if you are interested.

On a side note — I’m curious about how your company is using FFmpeg in iOS, and if your use-case with video is static (operating on a known, existing video asset) or real-time (video being captured/created on the fly). Feel free to contact me off-list if you care to share.

Cheers,

Brad

[Robin Stevens]

Hi,

I'm not especially interested in OS X support, however I am interested in
C# interoperability.

I've written my own wrapper and imposed a classist approach on top of the
API. It deals with the subset I need, so is incomplete.

I would have benefited tremendously from better documentation; in
particular higher level stuff than just the api calls, something that talks
about what the various components (AVCodec, AVFrame, etc) are, how they are
used, the order in which they should be initialised etc.

My class library makes this somewhat discoverable: object B requires object
A ref in constructor, implying object A must be created first.

I've put my work on GitHub, though it's out of date, for anyone else who's
interested.

I'd be willing to contribute to a documentation project: but I'm far from
an expert and such a project would really benefit from the support of a
core dev; perhaps just on a discussion/interview basis.

Cheers
Rob

On Friday, January 23, 2015, Bradley O'Hearne <***@bighillsoftware.com>
wrote:

>
> > On Jan 22, 2015, at 9:10 PM, David T. <***@gmail.com <javascript:;>>
> wrote:
> >
> >>> My purpose here is to poll the list members to ask if anyone would
> find any value at all if someone created an OS X / Cocoa / Swift (and
> possibly iOS) wrapper for Libav?
> >
> > So far we have delivered several projects on iOS. But we have not found
> any deficiency (which you didn't describe) on using FFmpeg on OS X.
> Perhaps you could tell us exactly what you expect a new API should do
> compared to current conventions, what would a new API solve?
>
> Hey David — thanks so much for your reply — I was wondering if this even
> piqued anyone’s interest, so I’m really happy to be able to dialog with
> someone on this. My purpose here isn’t to recite a laundry list of problems
> encountered, or debate design issues, or poke holes in FFmpeg in general.
> Suffice to say that I think the degree to which issues are encountered
> depends heavily on your use-case. Those that have use-cases which are a bit
> off the beaten path (of FFmpeg samples, for example) probably have run into
> various issues
I’ve had people periodically contact me off-list for issues
> similar to the ones I’ve encountered, so I know I’m not the only one
> running into them. But again, my purpose is not to debate FFmpeg use cases
> or problems.
>
> What I believe would be more universally agreeable (and safer turf for
> discussion, which is my desire) is the fact that FFmpeg is very scantily
> documented and very confusing to get started with. Options are extremely
> limited if you run into a problem and generally lead to hours upon hours of
> reading source code and debugging, with little other recourse. I don’t
> think it would be anything controversial to say that FFmpeg doesn’t offer
> what I’d term a “consumer’s API” — that is, an API that is built for
> intuitiveness and easy use by third parties, which hides its nasty internal
> implementation details and implications. I would characterize FFmpeg the
> same way I’d characterize many technical books that are out there — created
> for those with the foreknowledge of the authors, rather than those coming
> in cold.
>
> More to the specific case of OS X and iOS — those developing on those
> platforms will likely be using one or more of the following frameworks:
> QTKit, AVFoundation, AudioToolbox, VideoToolbox. It would be very nice to
> be able to deal in terms of:
>
> a) Data structures native to those frameworks, and
>
> b) Swift.
>
> I believe with such an API, the learning curve would be much smoother,
> some of the nastiness of the FFmpeg API’s could be hidden, and a lot of
> common mistakes could be eliminated. I also think that some very common use
> cases could be encapsulated into simple, well-documented API calls. While
> it might not express the entire richness or capability of FFmpeg, it might
> handle some of the common stuff well.
>
> I hope that communicates with some clarity the general idea. Let me know
> if you are interested.
>
> On a side note — I’m curious about how your company is using FFmpeg in
> iOS, and if your use-case with video is static (operating on a known,
> existing video asset) or real-time (video being captured/created on the
> fly). Feel free to contact me off-list if you care to share.
>
> Cheers,
>
> Brad
>
>
> _______________________________________________
> Libav-user mailing list
> Libav-***@ffmpeg.org <javascript:;>
> http://ffmpeg.org/mailman/listinfo/libav-user
>

[Joshua Kordani]

On 1/23/15 3:38 AM, Robin Stevens wrote:
...
> I would have benefited tremendously from better documentation; in particular higher level stuff than just the api calls, something that talks about what the various components (AVCodec, AVFrame, etc) are, how they are used, the order in which they should be initialised etc.
>
..

I too would have benefited greatly from this. It was all trial and error for me.


Joshua Kordani
LSA Autonomy

[Carl Eugen Hoyos]

Bradley O'Hearne <***@...> writes:

> My purpose here isn’t to recite a laundry list
> of problems encountered, or debate design
> issues, or poke holes in FFmpeg in general.

But this is what this mailing list is here for.
Or to say it differently: It is the only thing
we - the FFmpeg developers - expect from you.

> What I believe would be more universally
> agreeable (and safer turf for discussion, which
> is my desire) is the fact that FFmpeg is very
> scantily documented

So what you suggest - iiuc - is that instead of
improving the documentation of current FFmpeg you
want to invest your time in developing another
wrapper for FFmpeg?

> More to the specific case of OS X and iOS —
> those developing on those platforms will
> likely be using one or more of the following
> frameworks: QTKit, AVFoundation, AudioToolbox,
> VideoToolbox.

I am sorry if I misunderstand this:
You do realize that three of these
frameworks are supported by FFmpeg?
And that patches improving this support are
(of course) welcome!

Carl Eugen

[Robin Stevens]

On Fri, Jan 23, 2015 at 7:57 PM, Carl Eugen Hoyos <***@ag.or.at> wrote:
>
> So what you suggest - iiuc - is that instead of
> improving the documentation of current FFmpeg you
> want to invest your time in developing another
> wrapper for FFmpeg?
>

I would personally prefer to improve the current documentation.

I have my own need for a particular type of wrapper
(https://github.com/rdstevens/ffmpeg.net - sadly out of date) and in
that context it is possible to make an API which is reasonably
discoverable.

I don't think that level of discoverability is true of the C
structs-and-functions API that the ffmpeg libraries currently use.

My only criticism of the current API is that it is a reasonably high
barrier to entry. That's not entirely a bad thing, and certainly I
would expect anyone wishing to contribute to the libraries to get
themselves over that barrier as an absolute minimum.

For someone wishing to just *use* the library - ie. people on this
mailing list... well frankly it's quite hard to do.

(I spent far too long once upon a time trying to figure out a bug in
my code: it turns out you need to set certain parameters *before*
calling av_codec_open... who knew? Probably the old hands, but the
docs didn't contain this handy nugget anywhere I could find it.)

Sorry to ramble. I don't mean to criticise ffmpeg: after all it's a
labour of love, generously given for free to whomever wants it. So I'm
certainly not demanding you do anything about it.

What I would like to see, and would be happy to contribute to, is some
documentation that describes the classes in the system, how they
relate to each other, how they are intended to be used, and some of
the thinking behind the design decisions.

A guide that says: if you want to know what flags a codec supports,
you need to look *here* in the code.

That sort of thing...

Cheers,
Rob

[wm4]

On Fri, 23 Jan 2015 20:26:35 +0000
Robin Stevens <***@gmail.com> wrote:

> On Fri, Jan 23, 2015 at 7:57 PM, Carl Eugen Hoyos <***@ag.or.at> wrote:
> >
> > So what you suggest - iiuc - is that instead of
> > improving the documentation of current FFmpeg you
> > want to invest your time in developing another
> > wrapper for FFmpeg?
> >
>
> I would personally prefer to improve the current documentation.

Go ahead.

> I have my own need for a particular type of wrapper
> (https://github.com/rdstevens/ffmpeg.net - sadly out of date) and in
> that context it is possible to make an API which is reasonably
> discoverable.

Improving the FFmpeg API, instead of writing the 100th wrapper, would
actually benefit everyone.

> I don't think that level of discoverability is true of the C
> structs-and-functions API that the ffmpeg libraries currently use.

Idiotic prejudices against C APIs are not helpful here. FFmpeg is
mostly not a good C API and could be better, simple as that.

Keep in mind that non-C APIs are mostly useless to other languages.

> My only criticism of the current API is that it is a reasonably high
> barrier to entry. That's not entirely a bad thing, and certainly I
> would expect anyone wishing to contribute to the libraries to get
> themselves over that barrier as an absolute minimum.

Part of the high entry barrier is that the library is so low level.

> For someone wishing to just *use* the library - ie. people on this
> mailing list... well frankly it's quite hard to do.
>
> (I spent far too long once upon a time trying to figure out a bug in
> my code: it turns out you need to set certain parameters *before*
> calling av_codec_open... who knew? Probably the old hands, but the
> docs didn't contain this handy nugget anywhere I could find it.)

So I wonder what your .net wrapper is supposed to help here. It
basically turns all public fields into trivial properties that get or
set the field. What's the use?

Anyway, yes, that's one of the problem with this "let's dump everything
into a big struct" API. A big class with hundreds of getters and
setters is better?

> Sorry to ramble. I don't mean to criticise ffmpeg: after all it's a
> labour of love, generously given for free to whomever wants it. So I'm
> certainly not demanding you do anything about it.
>
> What I would like to see, and would be happy to contribute to, is some
> documentation that describes the classes in the system, how they
> relate to each other, how they are intended to be used, and some of
> the thinking behind the design decisions.
>
> A guide that says: if you want to know what flags a codec supports,
> you need to look *here* in the code.
>
> That sort of thing...

Well, it's certainly welcome.

Now what could happen in the worst case if you send a patch, it will be
that everyone will comment how wrong your new documentation is, and it
will need a few iterations until it gets accepted. I can imagine that
this can push away contributors. But what can we do; we can't push
incorrect documentation either.

> Cheers,
> Rob

[Robin Stevens]

On Sat, Jan 24, 2015 at 11:14 PM, wm4 <***@googlemail.com> wrote:
> On Fri, 23 Jan 2015 20:26:35 +0000
> Robin Stevens <***@gmail.com> wrote:
>
>
>> I don't think that level of discoverability is true of the C
>> structs-and-functions API that the ffmpeg libraries currently use.
>
> Idiotic prejudices against C APIs are not helpful here. FFmpeg is
> mostly not a good C API and could be better, simple as that.
>

I have no such prejudice. The level of discoverability of the C
structs-and-functions API that the *ffmpeg* libraries currently use is
not great.

It's entirely possible to make discoverable APIs in C; just that I
don't think ffmpeg has one.

>
> So I wonder what your .net wrapper is supposed to help here. It
> basically turns all public fields into trivial properties that get or
> set the field. What's the use?
>

That's a fair criticism of much of the wrapper. The use is to expose
the FFmpeg api to C#, the getters and setters help with that. (The
wrapper is in managed C++/CLI, so the interop is taken care of for
me.)

There is some value in the methods (instance or static) on the types,
which makes for greater discoverability, but doesn't really help you
understand when and where a particular type is meant to be used.


> Anyway, yes, that's one of the problem with this "let's dump everything
> into a big struct" API. A big class with hundreds of getters and
> setters is better?
>

It can be, yes. Take the AVFrame struct, and look at the sample_rate
member. Is this a read-only property? Can it be written to? Does that
operation make even make sense?

Perhaps I want to change the sample rate of a frame. From this API I
might think I just need to set a value on that variable and the
library will magically change the sample rate for me.

The ffmpeg API does NOT tell me. The only way to find out is to dive
in to the code. That is a deficiency of the interface which should
specify such things. (And my preference is for language support so
that the specification is enforceable by the compiler, but that's just
my preference.)

My wrapper only has a getter for that property, so it's a syntax error
to attempt to set it. It's clear to the user that they cannot change
the sample rate by setting that property, and need to go and look
elsewhere.


C cannot (AFAIK) specify in the language whether a variable is
publicly read-only or writeable, so static checking is not available.
The solution to this, as far as the FFmpeg API is concerned, appears
to be to document in the comment whether the field is writeable.

That's perfectly reasonable, although not consistently done. Easy to
fix though, if you have the knowledge.


Another small win: basic class hierarchy.

The AVFrame structure encapsulates two very different types of frames:
audio and video; and has a bunch of variables that apply to one or the
other, but not both (and some variables that *do* apply to both). My
wrapper splits AVFrame into a base class, an AVVideoFrame and an
AVAudioFrame.

This could presumably be done in the FFmpeg API but the devs have
chosen to spend their time elsewhere.

I'll admit that this is a a tiny, marginal win (at best) over ffmpeg,
given the comment SAYS audio sample rate. Though it *is* a small win,
and lots of those add up.


None of the points I'm making amount to much on their own, but they
are I think useful illustrations of some of the problems I've had with
the FFmpeg API.

Improving and documenting the real API is the best option, but has no
realisitic prospect of success. I someone like me came along and
started suggesting (breaking) changes to the existing API (like
AVFrame, AVAudioFrame, AVVideoFrame), I would (rightly) be shot down
in flames.

Cheers,
Rob

[wm4]

On Mon, 26 Jan 2015 10:42:27 +0000
Robin Stevens <***@gmail.com> wrote:

> On Sat, Jan 24, 2015 at 11:14 PM, wm4 <***@googlemail.com> wrote:
> > On Fri, 23 Jan 2015 20:26:35 +0000
> > Robin Stevens <***@gmail.com> wrote:
> >
> >
> >> I don't think that level of discoverability is true of the C
> >> structs-and-functions API that the ffmpeg libraries currently use.
> >
> > Idiotic prejudices against C APIs are not helpful here. FFmpeg is
> > mostly not a good C API and could be better, simple as that.
> >
>
> I have no such prejudice. The level of discoverability of the C
> structs-and-functions API that the *ffmpeg* libraries currently use is
> not great.
>
> It's entirely possible to make discoverable APIs in C; just that I
> don't think ffmpeg has one.

OK, it's just a bit hard to make APIs in C without structs and
functions...

> >
> > So I wonder what your .net wrapper is supposed to help here. It
> > basically turns all public fields into trivial properties that get or
> > set the field. What's the use?
> >
>
> That's a fair criticism of much of the wrapper. The use is to expose
> the FFmpeg api to C#, the getters and setters help with that. (The
> wrapper is in managed C++/CLI, so the interop is taken care of for
> me.)
>
> There is some value in the methods (instance or static) on the types,
> which makes for greater discoverability, but doesn't really help you
> understand when and where a particular type is meant to be used.

Somewhat true, there are way too many API things without context.
Reading the headers is not fun at all.

>
> > Anyway, yes, that's one of the problem with this "let's dump everything
> > into a big struct" API. A big class with hundreds of getters and
> > setters is better?
> >
>
> It can be, yes. Take the AVFrame struct, and look at the sample_rate
> member. Is this a read-only property? Can it be written to? Does that
> operation make even make sense?
>
> Perhaps I want to change the sample rate of a frame. From this API I
> might think I just need to set a value on that variable and the
> library will magically change the sample rate for me.
>
> The ffmpeg API does NOT tell me. The only way to find out is to dive
> in to the code. That is a deficiency of the interface which should
> specify such things. (And my preference is for language support so
> that the specification is enforceable by the compiler, but that's just
> my preference.)

Well, it's kind of obvious (or is it?) that the AVFrame members
describe the format of the data it carries. Just setting the
sample_rate field can logically not make FFmpeg convert the format. In
a way, a setter would make things worse.

(But I acknowledge your points.)

> My wrapper only has a getter for that property, so it's a syntax error
> to attempt to set it. It's clear to the user that they cannot change
> the sample rate by setting that property, and need to go and look
> elsewhere.

You can change it, for example when you initialize the AVFrame and fill
it with your own data.

Ideally I would say AVFrame should be opaque. There should be a
separate struct that describes formats. You could freely modify the
format struct, and pass it to a function that creates an AVFrame. This
way AVFrame could be fully immutable once created. Trying to push such
an API change would probably fail, though.

>
> C cannot (AFAIK) specify in the language whether a variable is
> publicly read-only or writeable, so static checking is not available.
> The solution to this, as far as the FFmpeg API is concerned, appears
> to be to document in the comment whether the field is writeable.

That is not really the problem. The problem is that many fields in
FFmpeg structs are read-only, write-only, or both depending on context.
Sometimes fields are declared off-limits, but can be legally accessed
by AVOption. This is quite hysteric. I also despise structs like
AVCodecContext, which have literally dozens of fields, most with
questionable value.

> That's perfectly reasonable, although not consistently done. Easy to
> fix though, if you have the knowledge.
>
>
> Another small win: basic class hierarchy.
>
> The AVFrame structure encapsulates two very different types of frames:
> audio and video; and has a bunch of variables that apply to one or the
> other, but not both (and some variables that *do* apply to both). My
> wrapper splits AVFrame into a base class, an AVVideoFrame and an
> AVAudioFrame.

You can achieve the same thing with aggregation.

>
> This could presumably be done in the FFmpeg API but the devs have
> chosen to spend their time elsewhere.
>
> I'll admit that this is a a tiny, marginal win (at best) over ffmpeg,
> given the comment SAYS audio sample rate. Though it *is* a small win,
> and lots of those add up.

Except, in case of the sample_rate field, you can't make full use of
the API anymore.

And that's just my point: a wrapper which tries to paint over the
issues in the "original" will just suffer from the API mismatches.

>
> None of the points I'm making amount to much on their own, but they
> are I think useful illustrations of some of the problems I've had with
> the FFmpeg API.
>
> Improving and documenting the real API is the best option, but has no
> realisitic prospect of success. I someone like me came along and
> started suggesting (breaking) changes to the existing API (like
> AVFrame, AVAudioFrame, AVVideoFrame), I would (rightly) be shot down
> in flames.

Yes, making such radical API changes is hard and will take a long time.
Maybe it would have a chance of success if it's well planned and agreed
upon in advance. FFmpeg had other semi-radical API changes in the past.

[Bradley O'Hearne]

> On Jan 26, 2015, at 3:42 AM, Robin Stevens <***@gmail.com> wrote:
>
> Improving and documenting the real API is the best option, but has no
> realisitic prospect of success. I someone like me came along and
> started suggesting (breaking) changes to the existing API (like
> AVFrame, AVAudioFrame, AVVideoFrame), I would (rightly) be shot down
> in flames.

Agreed, which is why a wrapper approach has merit:

- It introduces no changes (disruptive or otherwise) to the existing API.

- It can expose FFmpeg functionality which is well-understood, hide functionality which is not, and expose additional functionality later when its use becomes understood. The entire FFmpeg API could be theoretically black-boxed within a wrapper API if necessary.

- It can refine the data structures the API user must deal with.

- It can provide higher-level value-added, aggregated functionality for common operations which are use-case driven. The interface which a mechanic deals with in your automobile engine is not the same one a driver wants to deal with on their dashboard while they are driving. One does not preclude the other; but rather one builds on and leverages the other.

- It provides an immediate path for improvement of API and documentation for end-users.

- It opens the door for immediate progress, without being mired in the politics of doing so.

There’s a reason the FFmpeg API is the way it is, and has remained that way for years: the pragmatic consensus (read: when it comes down to action) is that things are sufficient the way they are, and that there’s resistance to significant change. Both desire for improvement and resistance to it (even so far as considering improvements a threat) have been illustrated in this discussion thread. The likelihood of significant improvement and progress is very low when that is the culture.

Brad

[wm4]

On Mon, 26 Jan 2015 08:31:31 -0700
Bradley O'Hearne <***@bighillsoftware.com> wrote:

>
> > On Jan 26, 2015, at 3:42 AM, Robin Stevens <***@gmail.com> wrote:
> >
> > Improving and documenting the real API is the best option, but has no
> > realisitic prospect of success. I someone like me came along and
> > started suggesting (breaking) changes to the existing API (like
> > AVFrame, AVAudioFrame, AVVideoFrame), I would (rightly) be shot down
> > in flames.
>
> Agreed, which is why a wrapper approach has merit:
>
> - It introduces no changes (disruptive or otherwise) to the existing API.
>
> - It can expose FFmpeg functionality which is well-understood, hide functionality which is not, and expose additional functionality later when its use becomes understood. The entire FFmpeg API could be theoretically black-boxed within a wrapper API if necessary.
>
> - It can refine the data structures the API user must deal with.
>
> - It can provide higher-level value-added, aggregated functionality for common operations which are use-case driven. The interface which a mechanic deals with in your automobile engine is not the same one a driver wants to deal with on their dashboard while they are driving. One does not preclude the other; but rather one builds on and leverages the other.
>
> - It provides an immediate path for improvement of API and documentation for end-users.
>
> - It opens the door for immediate progress, without being mired in the politics of doing so.
>
> There’s a reason the FFmpeg API is the way it is, and has remained that way for years: the pragmatic consensus (read: when it comes down to action) is that things are sufficient the way they are, and that there’s resistance to significant change. Both desire for improvement and resistance to it (even so far as considering improvements a threat) have been illustrated in this discussion thread. The likelihood of significant improvement and progress is very low when that is the culture.
>

I think the problem is rather that you're not even trying to contribute.

This is an open source project. You have to get involved, not expect
that "support" will take care of you.

[Bradley O'Hearne]

> On Jan 26, 2015, at 8:19 PM, wm4 <***@googlemail.com> wrote:
>
> I think the problem is rather that you're not even trying to contribute.
>
> This is an open source project. You have to get involved, not expect
> that "support" will take care of you.

I’ve been on this mailing list over three years. Numerous times I’ve attempted to engage this list in discussions in hopes of definitively understanding FFmpeg at a deeper level, and leaving a discussion trail which others traveling the same path which I have could benefit from who I’ve inquired about documentation, design and architectural decisions. I’ve asked questions about how various X things are accomplished for the same reason. I’ve contributed a fully working real-time capture-encode-stream app whose code has sat on Github for over a year and has been linked to several times on this mailing list. This time, I once again tried to find a way I could contribute something to FFmpeg which based on the clients I serve, is exactly what FFmpeg needs — and so I appealed to the community in hopes of finding others whom it would help as well. All of this is directly aimed at contributing.

In all of the cases above, the efforts proved futile (save some of the folks I’ve met and good conversations I’ve had a result off-list as a result). The very ingrained collective personality of this mailing list is a one-trick pony — when you can’t stand toe-to-toe technically, turn the turrets on the person, and make the conversation personal. Same thing here
avoid the issue, fault the person. If you want a fail-safe formula for pushing contributors away, well you’ve got it.

As I stated before
you win. Your worries about threatening FFmpeg by bringing its API out of the realm of the propellor-heads and propagating clarity and understanding rather than keeping FFmpeg a black art are unnecessary. I’ll contribute elsewhere.

Brad

[wm4]

On Mon, 26 Jan 2015 21:39:20 -0700
Bradley O'Hearne <***@bighillsoftware.com> wrote:

>
> > On Jan 26, 2015, at 8:19 PM, wm4 <***@googlemail.com> wrote:
> >
> > I think the problem is rather that you're not even trying to contribute.
> >
> > This is an open source project. You have to get involved, not expect
> > that "support" will take care of you.
>
> I’ve been on this mailing list over three years. Numerous times I’ve attempted to engage this list in discussions in hopes of definitively understanding FFmpeg at a deeper level, and leaving a discussion trail which others traveling the same path which I have could benefit from who I’ve inquired about documentation, design and architectural decisions. I’ve asked questions about how various X things are accomplished for the same reason. I’ve contributed a fully working real-time capture-encode-stream app whose code has sat on Github for over a year and has been linked to several times on this mailing list. This time, I once again tried to find a way I could contribute something to FFmpeg which based on the clients I serve, is exactly what FFmpeg needs — and so I appealed to the community in hopes of finding others whom it would help as well. All of this is directly aimed at contributing.
>
> In all of the cases above, the efforts proved futile (save some of the folks I’ve met and good conversations I’ve had a result off-list as a result). The very ingrained collective personality of this mailing list is a one-trick pony — when you can’t stand toe-to-toe technically, turn the turrets on the person, and make the conversation personal. Same thing here…avoid the issue, fault the person. If you want a fail-safe formula for pushing contributors away, well you’ve got it.
>

No, no. As I said before, there are barely any developers on this list.
I guess people don't like giving tech support (especially for free). If
you want to get involved, get on ffmpeg-devel and on the IRC channel.

As for contributing... I haven't seen a single FFmpeg patch from you,
and not a single post from you on the ffmpeg-devel list in the last few
years (you know... the place where ffmpeg devs actually reading any
posts).

> As I stated before…you win. Your worries about threatening FFmpeg by bringing its API out of the realm of the propellor-heads and propagating clarity and understanding rather than keeping FFmpeg a black art are unnecessary. I’ll contribute elsewhere.
>

Well, you're not contributing in the first place. Your suggested wrapper
would have no value for the project either, because it'd be completely
Apple locked-in.

If you want to view yourself as a victim of the evil open source
conspiracy, you can do that, but please refrain from such stupid
allegations. It's quite funny to hear that when I've always argued for
API simplification and introducing higher level APIs.

[Lucas Soltic]

> Le 27 janv. 2015 à 07:26, wm4 <***@googlemail.com> a écrit :
>
> On Mon, 26 Jan 2015 21:39:20 -0700
> Bradley O'Hearne <***@bighillsoftware.com> wrote:
>
>>
>>> On Jan 26, 2015, at 8:19 PM, wm4 <***@googlemail.com> wrote:
>>>
>>> I think the problem is rather that you're not even trying to contribute.
>>>
>>> This is an open source project. You have to get involved, not expect
>>> that "support" will take care of you.
>>
>> I’ve been on this mailing list over three years. Numerous times I’ve attempted to engage this list in discussions in hopes of definitively understanding FFmpeg at a deeper level, and leaving a discussion trail which others traveling the same path which I have could benefit from who I’ve inquired about documentation, design and architectural decisions. I’ve asked questions about how various X things are accomplished for the same reason. I’ve contributed a fully working real-time capture-encode-stream app whose code has sat on Github for over a year and has been linked to several times on this mailing list. This time, I once again tried to find a way I could contribute something to FFmpeg which based on the clients I serve, is exactly what FFmpeg needs — and so I appealed to the community in hopes of finding others whom it would help as well. All of this is directly aimed at contributing.
>>
>> In all of the cases above, the efforts proved futile (save some of the folks I’ve met and good conversations I’ve had a result off-list as a result). The very ingrained collective personality of this mailing list is a one-trick pony — when you can’t stand toe-to-toe technically, turn the turrets on the person, and make the conversation personal. Same thing here…avoid the issue, fault the person. If you want a fail-safe formula for pushing contributors away, well you’ve got it.
>
> No, no. As I said before, there are barely any developers on this list.
> I guess people don't like giving tech support (especially for free). If
> you want to get involved, get on ffmpeg-devel and on the IRC channel.
>
> As for contributing... I haven't seen a single FFmpeg patch from you,
> and not a single post from you on the ffmpeg-devel list in the last few
> years (you know... the place where ffmpeg devs actually reading any
> posts).
>
>> As I stated before…you win. Your worries about threatening FFmpeg by bringing its API out of the realm of the propellor-heads and propagating clarity and understanding rather than keeping FFmpeg a black art are unnecessary. I’ll contribute elsewhere.
>
> Well, you're not contributing in the first place. Your suggested wrapper
> would have no value for the project either, because it'd be completely
> Apple locked-in.
>
> If you want to view yourself as a victim of the evil open source
> conspiracy, you can do that, but please refrain from such stupid
> allegations. It's quite funny to hear that when I've always argued for
> API simplification and introducing higher level APIs.
>
> _______________________________________________
> Libav-user mailing list
> Libav-***@ffmpeg.org
> http://ffmpeg.org/mailman/listinfo/libav-user

wm4, you're being awfully demoralizing and narrow minded.
This discussion is already by itself a great contribution: giving ideas about what should be improved and pointing out what are FFmpeg weakness. Instead of taking advantage of these you just throw the efforts away. Just the kind of stuff that makes sure you won't get more source code contribution.

[wm4]

On Tue, 27 Jan 2015 08:32:47 +0100
Lucas Soltic <***@orange.fr> wrote:

>
>
> > Le 27 janv. 2015 à 07:26, wm4 <***@googlemail.com> a écrit :
> >
> > On Mon, 26 Jan 2015 21:39:20 -0700
> > Bradley O'Hearne <***@bighillsoftware.com> wrote:
> >
> >>
> >>> On Jan 26, 2015, at 8:19 PM, wm4 <***@googlemail.com> wrote:
> >>>
> >>> I think the problem is rather that you're not even trying to contribute.
> >>>
> >>> This is an open source project. You have to get involved, not expect
> >>> that "support" will take care of you.
> >>
> >> I’ve been on this mailing list over three years. Numerous times I’ve attempted to engage this list in discussions in hopes of definitively understanding FFmpeg at a deeper level, and leaving a discussion trail which others traveling the same path which I have could benefit from who I’ve inquired about documentation, design and architectural decisions. I’ve asked questions about how various X things are accomplished for the same reason. I’ve contributed a fully working real-time capture-encode-stream app whose code has sat on Github for over a year and has been linked to several times on this mailing list. This time, I once again tried to find a way I could contribute something to FFmpeg which based on the clients I serve, is exactly what FFmpeg needs — and so I appealed to the community in hopes of finding others whom it would help as well. All of this is directly aimed at contributing.
> >>
> >> In all of the cases above, the efforts proved futile (save some of the folks I’ve met and good conversations I’ve had a result off-list as a result). The very ingrained collective personality of this mailing list is a one-trick pony — when you can’t stand toe-to-toe technically, turn the turrets on the person, and make the conversation personal. Same thing here…avoid the issue, fault the person. If you want a fail-safe formula for pushing contributors away, well you’ve got it.
> >
> > No, no. As I said before, there are barely any developers on this list.
> > I guess people don't like giving tech support (especially for free). If
> > you want to get involved, get on ffmpeg-devel and on the IRC channel.
> >
> > As for contributing... I haven't seen a single FFmpeg patch from you,
> > and not a single post from you on the ffmpeg-devel list in the last few
> > years (you know... the place where ffmpeg devs actually reading any
> > posts).
> >
> >> As I stated before…you win. Your worries about threatening FFmpeg by bringing its API out of the realm of the propellor-heads and propagating clarity and understanding rather than keeping FFmpeg a black art are unnecessary. I’ll contribute elsewhere.
> >
> > Well, you're not contributing in the first place. Your suggested wrapper
> > would have no value for the project either, because it'd be completely
> > Apple locked-in.
> >
> > If you want to view yourself as a victim of the evil open source
> > conspiracy, you can do that, but please refrain from such stupid
> > allegations. It's quite funny to hear that when I've always argued for
> > API simplification and introducing higher level APIs.
> >
> > _______________________________________________
> > Libav-user mailing list
> > Libav-***@ffmpeg.org
> > http://ffmpeg.org/mailman/listinfo/libav-user
>
> wm4, you're being awfully demoralizing and narrow minded.

More than "this project is shit, let's write a wrapper to compensate
for both technical and social issues without even asking the devs
first"?

(Oh yeah, and sorry for my disinterest in Apple tech.)

> This discussion is already by itself a great contribution: giving ideas about what should be improved and pointing out what are FFmpeg weakness. Instead of taking advantage of these you just throw the efforts away. Just the kind of stuff that makes sure you won't get more source code contribution.

Except that it's a storm (or flame) in a bottle, and nobody outside
cares about this discussion. It's not like I'm a FFmpeg developer
either.

[Bradley O'Hearne]

> On Jan 26, 2015, at 11:26 PM, wm4 <***@googlemail.com> wrote:
>
> No, no. As I said before, there are barely any developers on this list.

I don’t believe that — as I’ve spoken with them in the past, but if this is true, there’s a major problem right there: complete disconnect with your user-base. If you don’t know the needs of API consumers and where the API is showing to be cumbersome or in need of revision or improvement, then you are missing some very valuable input.

> I guess people don't like giving tech support (especially for free).

All the more reason FFmpeg needs to be well documented and intuitive.

> As for contributing... I haven't seen a single FFmpeg patch from you,

> Well, you're not contributing in the first place. Your suggested wrapper

If you spent a little less time trying to personally impugn people and listen to what they are actually saying, you might acknowledge the simple logic that thorough understanding of design/architecture/API function and ability to have productive discourse on such topics are pre-requisites to contributing. That’s the point — people likely aren’t contributing because they can’t — there’s no ability to easily understand the API, and more importantly its design intent, without dedicating your life to reading cryptic and undocumented source code, and even then it still isn’t clear. That’s probably the source of compounded frustration— people want to understand, they’d like to be able to contribute. But they pragmatically can’t…or to their credit and wisdom, they won’t contribute, because they hold a high standard for the code they produce, and they aren’t about to make a bad problem worse for lack of knowledge.

There is no nobility in haphazardly throwing a patch across the wire just to say you can (of course, maybe that explains some things). I have plenty of things I would like to change, (and I’d wager others do too) but I’m not about to do so until I have a thorough understanding of the implications of such not just to me, but throughout FFmpeg. That knowledge is not easily accessible or obtainable. Furthermore, each attempt I’ve made to discuss something like this results in a Holy Grail -esque taunting from the French soldiers on the wall. I’m not interested in that — I’m all about moving forward and shipping product.

This thread was an attempt to go around those barriers of contributing to the FFmpeg codebase, and immediately contribute what I was able to: something I know for a fact is a need; something which wouldn’t step on anyone’s existing turf; something which would be 100% value-add and not either take away or change the existing API. It is clear such a thing won’t be well-received, so no-go.

I’m an eternal optimist, and so I view every problem as creating an opportunity; every closed door leads to another open one. So I’ll roll my own. The strength of generalism is breadth: providing something to the widest audience — but that “thing” is rarely the best thing. The strength of specialization is depth: the best of breed for the task. Neither I nor my clients need anywhere near the full breadth of what FFmpeg provides — and I’d guess that is the case with most folks. I believe that the common needs, the 80-20, are probably relatively few compared with the entirety of what you *can* do with an API like FFmpeg. Just like what you *can* do with all the parts at a Home Depot is virtually infinite, what most people generally need boils down to some pretty common things, relatively few in number.

There’s no need to duplicate the entirety of what FFmpeg provides…just a few well-worn paths. If you build it….they will come.

Brad

[wm4]

On Tue, 27 Jan 2015 10:09:58 -0700
Bradley O'Hearne <***@bighillsoftware.com> wrote:

>
> > On Jan 26, 2015, at 11:26 PM, wm4 <***@googlemail.com> wrote:
> >
> > No, no. As I said before, there are barely any developers on this list.
>
> I don’t believe that — as I’ve spoken with them in the past,

Did you really? Again, I see no real proof of this. You maybe talked to
_some_ FFmpeg devs (and not even main FFmpeg devs), just like _some_
FFmpeg devs sometimes write here.

It's a bit regrettable that libav-user turned into a place where
confused users are parked and forgotten. But there's a reason for this.
The most common threads here can be put into one of 3 categories:

1. Users with a low level of understanding of low level multimedia
concepts, and properly replying to them would be the same effort as
holding a multimedia class.
2. Users which want to do something awfully special, and solving the
problem for them would be a lot of work (many of these could also go
in category 1 or 3).
3. Users which could easily solve their problem by actually reading the
documentation or doing a little bit research themselves (they
should go to stackoverflow instead - the gamification system makes
other people help for free, even if it's tedious as hell).

So in general, it's just tedious and unsatisfying to be active on this
list, and people better go doing things that are more productive. Nobody
here is paid to give support.

> but if this is true, there’s a major problem right there: complete
disconnect with your user-base. If you don’t know the needs of API
consumers and where the API is showing to be cumbersome or in need of
revision or improvement, then you are missing some very valuable input.

This is true, but it doesn't falsify my argument: if you actually
want to change anything, you need to put some effort into it, rather
than waiting that others do.

Also, it's not really a disconnect with the user-base. It's a
disconnect between libav-user and users who never get close to the
project. "Good" users contribute patches and do get a say in the
development process.

> > I guess people don't like giving tech support (especially for free).
>
> All the more reason FFmpeg needs to be well documented and intuitive.

Where did anyone disagree with this? Again, I haven't seen a single
patch from you, not even something as trivial as adding a small bit of
documentation. This makes the way you represent yourself look very
questionable.

> > As for contributing... I haven't seen a single FFmpeg patch from you,
>
> > Well, you're not contributing in the first place. Your suggested wrapper
>
> If you spent a little less time trying to personally impugn people and listen to what they are actually saying, y

And here is where I give up and stop wasting my own time.

[Bradley O'Hearne]

> On Jan 27, 2015, at 10:49 AM, wm4 <***@googlemail.com> wrote:
>
> Again, I haven't seen a single
> patch from you

Go back and actually read the prior post I sent, rather than skimming it and nicking off a phrase here and there to argue with. I explained very clearly the challenge.

> And here is where I give up and stop wasting my own time.

You can be proud of what you accomplished: you spent your time trying to knock someone personally and shut them down because of your religious technology views rather than adding anything productive, or encouraging useful discourse and input. You took a conversation where people were discussing their needs and potential desire to contribute and ended both — and I doubt anyone reading will conclude they want any part of conversations like this on a regular basis if they were to choose to get involved.

That isn’t how to build a community. It isn’t how to encourage involvement. It isn’t how to provide a warm and inviting place for newcomers.

Brad

[Carl Eugen Hoyos]

Bradley O'Hearne <***@...> writes:

> thorough understanding of design/architecture/API
> function and ability to have productive discourse
> on such topics are pre-requisites to contributing.

This is simply not true.
On the contrary, starting to send patches did and
will - through the review process - help
contributors to get the thorough understanding.

A few random remarks:
If you want to write a wrapper for FFmpeg, please do
so. If you believe it makes sense to tell us you
will write a wrapper (I believe it does make sense)
tell us (as you did). But please do not expect
everybody to agree and please don't try to convince
everybody what a great idea it is (I have doubts).

(This is probably the moment where I should try to
convince you how futile your attempt is - NIH,
maintainability and complexity come to mind - but
this is free software: Please do it and prove me
wrong!)

If you think the documentation needs improvements
(everybody agrees afaict), please send small (!)
patches to the ffmpeg-devel mailing list. If you
don't want to send small patches, don't mention
that the documentation needs improvements, as said
everybody knows it.

Time is the only limiting factor in FFmpeg
development. This alone imo is a sufficient reason
(not just explanation) why the core developers do
not post here. The idea was probably that people
like you who don't want to contribute but have some
understanding about the API help other users.

I'd like to repeat that OSX is definitely no
"stepchild" in FFmpeg development: On the contrary,
I believe FFmpeg developers as a whole consider it
a main platform and if you look the number of source
files specific to OSX you will find sufficient proof.
(And note that if you believe that we are rude I'd
like to inform you that claiming we don't care about
OSX could also be read as an offense given the time
some of us spend on supporting an operating system
that we don't use regularly.)

As wm4 tried to explain, I am not a core developer,
he (wm4) writes (iirc) less patches than I do but he
knows the API well so he is the best support you can
get here. If you don't like my (or his) attitude,
consider that we are doing this in our free time, if
you still don't like it, there is no easy solution
(sorry!) - please don't post your questions on the
developer mailing list, this is just rude.

Regarding the API (that I do not know well), I
suspect what was said in this thread is (too) true:
It mirrors how file formats (and codecs) are defined
and this is typically not easy to understand. Note
that the number of users who want to access more
(internal) fields (and therefore an even more basic
API) is not lower than the ones requesting a higher
level API.

And the final note (after re-reading the above;-) )
is that I suspect your emails were simply to long to
get an easy answer, this includes your note "I did
post code on github" (I fear nobody who reads this
list would download a software project to understand
a question): Simple and short questions are
typically much easier to answer.

Carl Eugen

[Brad O'Hearne]

> On Jan 27, 2015, at 2:52 PM, Carl Eugen Hoyos <***@ag.or.at> wrote:
>
> But please do not expect
> everybody to agree

I didn't post the original message looking for agreement (and it doesn't bother me a bit if anyone does) ...I was trying to gauge who might be interested and whose specific use cases it might help. Disagreement wasn't the issue. Being rude and uncourteous and hijacking a technical discussion to throw out personal barbs and air personal politics is what I object to.

> and please don't try to convince
> everybody what a great idea it is (I have doubts).

I already know it is a great idea, and that isn't because it is mine -- it is because of of spending 3 years with the FFmpeg myself; having clients who want a more supportable situation and don't want the uncertainty and risk FFmpeg inflicts on their projects; and because of those who contacted me off-list after I posted code to Github a year ago. The need and merit of the idea isn't even debatable -- perhaps you or someone else doesn't personally have a need for it, but others absolutely do.

> (This is probably the moment where I should try to
> convince you how futile your attempt is - NIH,
> maintainability and complexity come to mind - but
> this is free software: Please do it and prove me
> wrong!)

No fear...I won't be. If I move forward, I'll author a completely new API, not built on FFmpeg. Your turf remains unthreatened.

> If you think the documentation needs improvements

This isn't even debatable. It needs to be written by the authors of the code, not those trying to understand it. Posting tens of thousands of lines of code and then expecting someone else to understand and document it is absurd.

> If you don't like my (or his) attitude,
> consider that we are doing this in our free time, if
> you still don't like it, there is no easy solution

How anyone treats another has ZERO to do with your "time", it has everything to do with what kind of person you choose to be and how you choose to treat others. And there IS a very easy solution: ditch the ridiculous "I'm a dev spending time on this and therefore I get to be rude" attitude and DON'T BE RUDE. People are rude because they choose to be rude people -- simple as that.

> this includes your note "I did
> post code on github" (I fear nobody who reads this
> list would download a software project to understand
> a question)

Code was posted at the request of a dev. After it was posted, the same "too much code" gripe was aired, so I linked directly to the class and directed to the exact method which held the code in question: all accessible with a link in a web browser, no downloading needed. After posting that, then got the "devs aren't using OS X" and "devs don't want to read long emails" and "devs don't speak English very well" and "you're a troll for posting this and claiming it is doing X."

> : Simple and short questions are
> typically much easier to answer.

Unfortunately not all problems come in a pocket-sized Jared jewelry box. I posted not only the code in question, but a fully runnable app with a UI to demonstrate the issue. I did all of this to a) get an answer to the questions for my client, and b) to leave no stone unturned, and c) to leave no doubt as to whether there was support available or not. No ambiguity remained.

> The idea was probably that people
> like you who don't want to contribute but have some
> understanding about the API help other users.

Once again, an intentional mischaracterization which which completely ignores everything I've written and the entire purpose for the original post to begin with: an attempt to contribute not only what would benefit my interests, but others as well. It's the same tired playbook: frame the person as the problem, question their credibility, kill the discussion.

Brad

[wm4]

On Tue, 27 Jan 2015 16:57:21 -0700
Brad O'Hearne <***@bighillsoftware.com> wrote:

> > (This is probably the moment where I should try to
> > convince you how futile your attempt is - NIH,
> > maintainability and complexity come to mind - but
> > this is free software: Please do it and prove me
> > wrong!)
>
> No fear...I won't be. If I move forward, I'll author a completely new API, not built on FFmpeg. Your turf remains unthreatened.

For someone who complains about being antagonized, you sure antagonize
a lot.

I know I tend to be rude and rough, but holy shit you're one special
individual.

[Bradley O'Hearne]

On Jan 28, 2015, at 3:57 AM, wm4 <***@googlemail.com> wrote:
>
> On Tue, 27 Jan 2015 16:57:21 -0700
> Brad O'Hearne <***@bighillsoftware.com> wrote:
>
>> No fear...I won't be. If I move forward, I'll author a completely new API, not built on FFmpeg. Your turf remains unthreatened.
>
> For someone who complains about being antagonized, you sure antagonize
> a lot.

You’re words, not mine. From you in this thread:

> "Yes, I see such vendor-specific things as a "threat”.”

> I know I tend to be rude and rough, but holy shit you're one special
> individual.

More rudeness, once again for the purpose of personal insult. You just can’t quit, you can’t avoid taking a personal swipe. We are who we choose to be.

I think it is time to kill this thread…it left the realm of being productive a good while ago, and at this point it is distracting from the continued pursuit of maintaining the status quo, and scolding newcomers for top-posting.

I’ve got no ill-will for either you nor Carl. I wish you both the best in your pursuits, both on this project and elsewhere. I just find unfortunate the opposition to contribution that doesn’t align exactly with your needs or views, and I find even more unfortunate the continued propensity to turn the turrets on people personally when they’ve got the moxie to tell it like it is, for the sake of acknowledging the reality and attempting to better it. Nearly everything great that has ever been accomplished in any field has had to overcome this same resistance. I hope the FFmpeg project and mailing list at some point does too.

I’m out.

Brad

[wm4]

On Wed, 28 Jan 2015 08:37:33 -0700
Bradley O'Hearne <***@bighillsoftware.com> wrote:

> On Jan 28, 2015, at 3:57 AM, wm4 <***@googlemail.com> wrote:
> >
> > On Tue, 27 Jan 2015 16:57:21 -0700
> > Brad O'Hearne <***@bighillsoftware.com> wrote:
> >
> >> No fear...I won't be. If I move forward, I'll author a completely new API, not built on FFmpeg. Your turf remains unthreatened.
> >
> > For someone who complains about being antagonized, you sure antagonize
> > a lot.
>
> You’re words, not mine. From you in this thread:
>
> > "Yes, I see such vendor-specific things as a "threat”.”

Seeing vendor-lockin as a threat is much different from what you
implied.

> > I know I tend to be rude and rough, but holy shit you're one special
> > individual.
>
> More rudeness, once again for the purpose of personal insult. You just can’t quit, you can’t avoid taking a personal swipe. We are who we choose to be.

So what if I choose to be rude to someone who keeps spouting
inconsistent non-sense to defend his position. The "turf" thing is a
perfect example.

[Bradley O'Hearne]

> On Jan 23, 2015, at 12:57 PM, Carl Eugen Hoyos <***@ag.or.at> wrote:
>
> But this is what this mailing list is here for.
> Or to say it differently: It is the only thing
> we - the FFmpeg developers - expect from you.

Carl — my observation over the years is that this list isn’t particularly tolerant of anyone who tries to dive deep, particularly in discussing architecture, design decisions, or potential weaknesses or shortcomings of the API. There appears to be a pretty clear aversion to discussing these things to any length or depth, and attempts to do so seem to be met by responses turning things personal.

> So what you suggest - iiuc - is that instead of
> improving the documentation of current FFmpeg you
> want to invest your time in developing another
> wrapper for FFmpeg?

On documentation: I cannot improve the documentation of FFmpeg. I’d venture to say that there’s probably few who can — I would guess that the only people who realistically could write documentation to any depth and accuracy, while capturing the hidden nuances of behavior and configuration options are probably a select subset of FFmpeg devs who have been around long enough to have worked on a majority of the codebase, and are carrying a significant amount of that knowledge in their heads. Beyond that, the barrier to being equipped to contribute solid documentation is pretty steep.

No doubt, more documentation of the current FFmpeg API would be gladly welcomed by all. But more documentation isn’t going to change the complexity of the API. FFmpeg may be awesome and feature-rich — I’m not debating that — but it has chosen to expose its API in a way that inflicts considerable complexity on its users. For the typical consumer of the API, it isn’t intuitive, easy to understand, or easy to debug and support. Any newcomer to the code has a comparatively steep learning curve before they can begin maintaining or extending code using FFmpeg. FFmpeg doesn’t encapsulate things which it could otherwise provide, and as such I’d wager that there is significant duplication of effort amongst the users of FFmpeg to accomplish the exact same tasks.

Put simply — I believe a completely different API offering would be a extremely beneficial, whether the current FFmpeg API is further documented or not. With a new API geared toward users, I believe documentation could be produced which made things relatively easy to understand. I’m not suggesting a wrapper as an alternative to documentation. I’m suggesting a wrapper as a pathway to understandable documentation and overall better usability.

It isn’t a question of which is better, a current API or another one. The issue is one of abstraction, encapsulation, and the intended consumer. I’m sure the current FFmpeg API would be the choice of many if not most. But I’d wager that there would be some that would greatly benefit from a simpler, more abstracted API. Neither has to preclude the other — they can both coexist and be offerings that have their own constituency. There are many, many examples of this very thing all around us in the tech world…perhaps the simplest of which is the relationship between machine language and a programming language like C. It isn’t a question of “better” per se — one is the foundation of the other — it is a question of usability and the needs of the audience who has to use it.

>> frameworks: QTKit, AVFoundation, AudioToolbox,
>> VideoToolbox.
>
> I am sorry if I misunderstand this:
> You do realize that three of these
> frameworks are supported by FFmpeg?
> And that patches improving this support are
> (of course) welcome!

I’m not exactly sure what you mean by “support”, so no, I suppose I don’t realize it. What I do know is that I’ve never been able to get an answer for anything which specifically involved OS X or use of FFmpeg with these frameworks. In fact, I’ve been told outright that the reason I’m likely not getting answers is because I’m running on OS X, and if I want answers, I need to change my code to something the devs use.

I hope that clarifies things adequately.

Brad

[Info || Non-Lethal Applications]

Bradley,

> This is not the best situation, and it is kind of a dubious label for Libav to be declared as working and/or supported on OS X if there are no devs using OS X or supporting it. I have generally found answers to most of the issues I have had to work through, but finding those answers has been slow-going, and a tedious process. My purpose here is to poll the list members to ask if anyone would find any value at all if someone created an OS X / Cocoa / Swift (and possibly iOS) wrapper for Libav?
>
> Perhaps I’m the only one on the planet using Libav on Apple platforms, though I’m betting I’m not (actually I know I’m not, as a few have contacted me off-list). Also, it might take some of the headache away from the Libav devs who don’t use Apple platforms to answer some support questions. For us who like apples, a nice, clean Swift API might be very nice, and save a lot of time and headaches. I might be able to produce such a thing, first somewhat limited and rudimentary, and then mature it over time.
>
> Would anyone be interested in such an API?

I’ve been using libav for about 2 years now in my own project as well as in the company I’m working for.
We share a common FFmpegContext class with a couple hundred lines of code only to grab video frames and audio samples in a format we can use.

So, yes. I would love to see a high-level wrapper taking the implementation specific burdens off our shoulders.
Basically, all I'd want is:
- give me a frame at this index (and if there’s some inter frame coding to be done, just do it for me)
- give me audio samples starting from x and ending at y

While it’s rather straight forward to use libav when you have your code set up, it’s really tough getting started as application developers need to get deep into video coding to be able to use the platform.

There was only one FFmpeg wrapper at the time I was investigating that could deliver frames reliably at a given index (I don’t remember the name).
Unfortunately, it was unusable as it needed to index the file first. Well, telling your customer that he needs to wait 20 seconds before he can playback a large file is of course not doable.

A big +1 here.

Best,

Flo

[Anton Shekhovtsov]

2015-01-23 10:58 GMT+02:00 Info || Non-Lethal Applications <
***@non-lethal-applications.com>:

> Basically, all I'd want is:
> - give me a frame at this index (and if there’s some inter frame coding to
> be done, just do it for me)
> - give me audio samples starting from x and ending at y
>
>
I`d like to see that on PC too. But afaik this problem originates from
diverse formats which do not support these features. High level wrapper
will be unable to solve this.

[wm4]

On Thu, 22 Jan 2015 19:02:10 -0700
Bradley O'Hearne <***@bighillsoftware.com> wrote:


> Perhaps I’m the only one on the planet using Libav on Apple platforms,

Certainly not. We just use clean, portable software, instead of very
crappy proprietary Apple technology.

[Joshua Kordani]

On 1/22/15 9:02 PM, Bradley O'Hearne wrote:
> All,
>
> I’ve been on the Libav-user mailing list 2-3 years now, having worked on FFmpeg integration for various clients. While I develop on Windows and Linux platforms, my primary development has been spent over the past 6 years has been on OS X with respect to desktop (my core business is mobile development: iOS, Android, and newly Windows Phone). Over the past two years, my work with FFmpeg has been primarily on the OS X platform.
>
> During that time, I have run into a number of issues about which I’ve appealed to this mailing list, and one of the things which various list members (some of which were, as I understand it, Libav devs) raised was that most (if not all) of the Libav devs did not either use or have access to OS X to be able to speak to or support problems which manifested on OS X, and I strongly suspect platform issues could have influenced some of the problems experienced. In one case, someone even encouraged me to just simply “change platforms”, which of course cannot always be done, especially if you are serving clients who have hired you to make things work on that specific platform.
>
> This is not the best situation, and it is kind of a dubious label for Libav to be declared as working and/or supported on OS X if there are no devs using OS X or supporting it. I have generally found answers to most of the issues I have had to work through, but finding those answers has been slow-going, and a tedious process. My purpose here is to poll the list members to ask if anyone would find any value at all if someone created an OS X / Cocoa / Swift (and possibly iOS) wrapper for Libav?
>
> Perhaps I’m the only one on the planet using Libav on Apple platforms, though I’m betting I’m not (actually I know I’m not, as a few have contacted me off-list). Also, it might take some of the headache away from the Libav devs who don’t use Apple platforms to answer some support questions. For us who like apples, a nice, clean Swift API might be very nice, and save a lot of time and headaches. I might be able to produce such a thing, first somewhat limited and rudimentary, and then mature it over time.
>
> Would anyone be interested in such an API?
>
> Thank you in advance for your replies.
>
> Cheers,
>
> Brad
>
> Brad O'Hearne
> Founder/Lead Developer
> Big Hill Software LLC
>
> _______________________________________________
> Libav-user mailing list
> Libav-***@ffmpeg.org
> http://ffmpeg.org/mailman/listinfo/libav-user
>

I can't speak to the cocoa swift stuff, but I use libav on OSX. I use it with c/c++ code. Not very coherently mind you, but still.

I've been derided on IRC for trying to work through how the various classes interact, and as far as I'm concerned the sooner I can stop using this user hostile library the better. I have code that works now, but any changes I make to the project I'll probably end up doing by hand.

I still don't understand the difference between the FFmpeg libraries and the LibAV project, and my comments are mainly directed at the former as the latter community wise has been nicer. As far as I can see though, the API's are shared or similar and are equally opaque. Doxygen doesn't count here.


Joshua Kordani
LSA Autonomy

[c***@megaroms.co.uk]

Hi all,

I'm having some problems just doing a simple thing: getting the current
frame number from a decoded AVFrame. I tried a few things (starting from
the most obvious) but I couldn't find one that consistently works:

1) Look at AVFrame.display_picture_number. Sadly, this seems to be zero
for all the files I've decoded.

2) Look at AVFrame.display_picture_number. This seems to count
sequentially from zero for most files (e.g. MP4) but stays at zero for
others (e.g. LAGS). (Aside: it also makes me wonder if the h264 is
reordering frames for me.)

3) Use av_frame_get_best_effort_timestamp(frame) to get frame timestamp.
This field always appears to be set but to convert it to a frame number I
need to know the average display time of a frame. Sadly
AVStream.avg_frame_rate is often zero.

4) Look at AVFrame.pts. Seems to stay stuck at zero for many files and I
still can't convert it similar to 3.

So, is it possible to get the current frame number reliably?

(Background: I'm getting skipped frames when decoding a lagarith (LAGS)
video. I'm trying to detect when a frame has been skipped which
(hopefully) will give me a clue to why a frame is skipped.)

Thanks.

[Anton Shekhovtsov]

2015-01-23 19:10 GMT+02:00 <***@megaroms.co.uk>:

> Hi all,
>
> I'm having some problems just doing a simple thing: getting the current
> frame number from a decoded AVFrame. ...
>

Hi,
I spent some time researching this for virtualdub plugin.
Short answer: impossible.
Well actually if the format is good and the file is complete then there is
a full index there (not public interface).
Then you can search the index looking for matching pkt_pts from the decoded
frame.
Some other formats may work if they are continuous and have well-formed
framerate and timebase. Then you can divide either pkt_pts or pkt_dts
(which one is what anyway?)
That all is totally empiric, there is no flag in ffmpeg telling "you can
rely on that".

[Mihai Chindea]

________________________________________
From: libav-user-***@ffmpeg.org [libav-user-***@ffmpeg.org] on behalf of ***@megaroms.co.uk [***@megaroms.co.uk]
Sent: Friday, January 23, 2015 7:10 PM
To: libav-***@ffmpeg.org
Subject: [SPAM] [Libav-user] Reliably getting the current frame number

Hi all,

I'm having some problems just doing a simple thing: getting the current
frame number from a decoded AVFrame. I tried a few things (starting from
the most obvious) but I couldn't find one that consistently works:

1) Look at AVFrame.display_picture_number. Sadly, this seems to be zero
for all the files I've decoded.

2) Look at AVFrame.display_picture_number. This seems to count
sequentially from zero for most files (e.g. MP4) but stays at zero for
others (e.g. LAGS). (Aside: it also makes me wonder if the h264 is
reordering frames for me.)

3) Use av_frame_get_best_effort_timestamp(frame) to get frame timestamp.
This field always appears to be set but to convert it to a frame number I
need to know the average display time of a frame. Sadly
AVStream.avg_frame_rate is often zero.

4) Look at AVFrame.pts. Seems to stay stuck at zero for many files and I
still can't convert it similar to 3.

So, is it possible to get the current frame number reliably?

(Background: I'm getting skipped frames when decoding a lagarith (LAGS)
video. I'm trying to detect when a frame has been skipped which
(hopefully) will give me a clue to why a frame is skipped.)

Thanks.


_______________________________________________
Libav-user mailing list
Libav-***@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/libav-user




you should rely on the AVPacket.pts, and rescale that value from the AVStream.time_base to 1/avg_framerate.
you can compute the average framerate base on AVStream.time_base.

int64_t framerate = floor( 1.00 / ( ( double )AVStream.time_base.num / ( double )AVStream.time_base.den ) );
int64_t fnumber = av_rescale_q( AVPacket.pts, AVStream.time_base, { 1, avg_framerate } );

just watch pay attention to the time_base your are using, the stream usually has different timebase than the codec.

[Bradley O'Hearne]

> On Jan 23, 2015, at 8:35 AM, Joshua Kordani <***@lsa2.com> wrote:
>
> On 1/22/15 9:02 PM, Bradley O'Hearne wrote:
>>
>> Would anyone be interested in such an API?
>
> I've been derided on IRC for trying to work through how the various classes interact, and as far as I'm concerned the sooner I can stop using this user hostile library the better. I have code that works now, but any changes I make to the project I'll probably end up doing by hand.


Thank you to everyone who has responded thus far. I’m going to consolidate my replies to everyone who has responded into one email to save # of messages. To the comment above, I guess I can only say I feel your pain, my experience has been the same, though I didn’t realize this was the same state of affairs on IRC. Unfortunate. Part of my reference to documentation in a new API was specifically to get at this very issue you raised, which is not only API doc, but the ability to communicate a higher-level architectural understanding of logical interactions to those who have to use it. After being on the mailing list long enough, I think it is fair to say that unless there’s a clandestine effort underway, this isn’t coming for the existing libraries, and quite frankly, it seems pretty high barrier to entry to even begin this. My hope was that with a distilled API wrapper, it might hide this problem somewhat, and provide something simple and very straightforward that would be intuitive and easy to understand and document.

> On Jan 23, 2015, at 8:19 AM, wm4 <***@googlemail.com> wrote:
>
> On Thu, 22 Jan 2015 19:02:10 -0700
> Bradley O'Hearne <***@bighillsoftware.com> wrote:
>
>
>> Perhaps I’m the only one on the planet using Libav on Apple platforms,
>
> Certainly not. We just use clean, portable software, instead of very
> crappy proprietary Apple technology.

Nothing wrong with a preference. But understand that others don’t necessarily share that opinion, others don’t necessarily have a choice when having to follow management or client direction, and “crappy proprietary Apple technology” is a bit of a dubious criticism given the codebase we are referring to here. Quality criticisms aside, for all practical purposes, despite the fact that FFmpeg is open source, it is pragmatically proprietary, because the key architecture / design / nuances of the product are locked in a very few developers’ heads, not thoroughly documented and known. There is nothing “open” or “accessible” about FFmpeg at all, unless you consider getting lambasted for asking design questions on the mailing list and being relegated to reading through undocumented source code for a month “open”. Sure it is free of licensing charges, but FFmpeg is not free — unless you have got a vanilla use-case encapsulated in the canned examples, you are most likely in for a deep-sea expedition, and you’ll most definitely be investing a large amount of time trying to understand how to use it.

But perhaps more important than this, FFmpeg purports to support OS X, and so when someone comes to this project and mailing list asking questions relevant to FFmpeg on OS X, they shouldn’t be subjected to a firestorm of flak simply because of what platform they are using. They shouldn’t be asked to post source code and then after they do so, be told that no one is going to care or look at it if it isn’t submitted for a different platform — that’s ridiculous anyway, as the context of the question *is on* OS X.

If for some reason to the FFmpeg dev OS X is considered an outcast which no one wants to support, then fine — put a big neon sign on the front page of the FFmpeg web site which says: “FFmpeg does not work on OS X, it is not being developed on OS X, it is not supported on OS X, we do not recommend using OS X, and please do not post questions to the mailing list about FFmpeg on OS X.”

I understand not everyone can run every platform, but if someone approaches the list using a supported platform, they shouldn’t have to wade through a religious debate in order to even get to the point where they can talk about the actual question they have.

> On Jan 23, 2015, at 1:58 AM, Info || Non-Lethal Applications <***@non-lethal-applications.com> wrote:
>
>>
>> Would anyone be interested in such an API?
>
> So, yes. I would love to see a high-level wrapper taking the implementation specific burdens off our shoulders.
> Basically, all I'd want is:
> - give me a frame at this index (and if there’s some inter frame coding to be done, just do it for me)
> - give me audio samples starting from x and ending at y


This is really good feedback. If anyone else cares to respond, I’m interested. Thanks again to everyone who has responded thus far.

Brad

[wm4]

On Fri, 23 Jan 2015 10:19:52 -0700
Bradley O'Hearne <***@bighillsoftware.com> wrote:

> Nothing wrong with a preference. But understand that others don’t
> necessarily share that opinion, others don’t necessarily have a choice
> when having to follow management or client direction, and “crappy
> proprietary Apple technology” is a bit of a dubious criticism given the
> codebase we are referring to here. Quality criticisms aside, for all

I can't say I'm overly fascinated by Apple engineering.

> practical purposes, despite the fact that FFmpeg is open source, it is
> pragmatically proprietary, because the key architecture / design /
> nuances of the product are locked in a very few developers’ heads, not
> thoroughly documented and known. There is nothing “open” or
> “accessible” about FFmpeg at all, unless you consider getting lambasted

That's pretty ridiculous. Just because it's not documented as well as
you want it, it's proprietary?

Anyway, documentation for open source things is bound to be worse than
with (popular) closed source APIs, simply because if something is not
documented, you could just read the source code, find out how it's
supposed to work, and send a patch to improve the documentation. (Only
if you actually contribute something, instead of complaining how rude
everyone is.) There's also this effect that once people learned how
things work, they won't be as much interested in putting effort into
the docs as before. That's just natural; it would happen to you as well.

Last but not least, most people working on FFmpeg are volunteers (you
can be one too) who work for free, or are not paid to work mainly on
FFmpeg.

(Also, I've never encountered an API that really clearly defined
everything 100%, whether it was proprietary or open source. If it's
open source, at least you can find out yourself, and maybe even
contribute a patch for clarification.)

> for asking design questions on the mailing list and being relegated to
> reading through undocumented source code for a month “open”. Sure it is
> free of licensing charges, but FFmpeg is not free — unless you have got
> a vanilla use-case encapsulated in the canned examples, you are most
> likely in for a deep-sea expedition, and you’ll most definitely be
> investing a large amount of time trying to understand how to use it.

The main issue is actually that the FFmpeg API is pretty low-level,
while many users expect something higher level. (Somewhat typical: a
user wants frame N of video in a specific format. Yeah, implementing
this using the raw FFmpeg API will take some time.)

I don't know what you got "lambasted" for, but you probably either
touched a sensible subject, got into some flamewar and internal
politics, or you did something honestly stupid.

> But perhaps more important than this, FFmpeg purports to support OS X,
> and so when someone comes to this project and mailing list asking
> questions relevant to FFmpeg on OS X, they shouldn’t be subjected to a
> firestorm of flak simply because of what platform they are using. They
> shouldn’t be asked to post source code and then after they do so, be
> told that no one is going to care or look at it if it isn’t submitted
> for a different platform — that’s ridiculous anyway, as the context of
> the question *is on* OS X.

Getting asked for source code if you ask for help is quite reasonable.
Even if people are willing to help, they don't have the time to guess
what your problem is, and trying the user's code (or looking directly at
code) is much more effective than trying to get to the care of the
issue by trying to get "it" out of the user in English. Especially if
it's a bug in FFmpeg, being able to reproduce the user's problem is
extremely helpful. And yeah, for someone not on OSX, OSX code will be
useless.

>
> If for some reason to the FFmpeg dev OS X is considered an outcast
> which no one wants to support, then fine — put a big neon sign on the
> front page of the FFmpeg web site which says: “FFmpeg does not work on
> OS X, it is not being developed on OS X, it is not supported on OS X,
> we do not recommend using OS X, and please do not post questions to the
> mailing list about FFmpeg on OS X.”

No reason to get upset because some (or many) devs can't or don't want
to help with OSX specific issues.

Anyway, in this case I got upset because someone was arguing for
questionable Apple vendor-lockin, instead of doing something portable,
or - got forbid - actually improving FFmpeg. Seriously, you're talking
about how bad FFmpeg is, and you want to solve it by putting a shiny
Apple-only wrapper around it? Instead of improving FFmpeg? And you post
that on a FFmpeg mailing list? I can't help but to make fun of this.

> I understand not everyone can run every platform, but if someone
> approaches the list using a supported platform, they shouldn’t have to
> wade through a religious debate in order to even get to the point where
> they can talk about the actual question they have.

I don't think it's wrong to get political about evil companies, or to
make fun of people trapped by evil companies. Also, trolling is fun.

>
> > On Jan 23, 2015, at 1:58 AM, Info || Non-Lethal Applications <***@non-lethal-applications.com> wrote:
> >
> >>
> >> Would anyone be interested in such an API?
> >
> > So, yes. I would love to see a high-level wrapper taking the implementation specific burdens off our shoulders.
> > Basically, all I'd want is:
> > - give me a frame at this index (and if there’s some inter frame coding to be done, just do it for me)
> > - give me audio samples starting from x and ending at y
>
>
> This is really good feedback. If anyone else cares to respond, I’m interested. Thanks again to everyone who has responded thus far.
>
> Brad
> _______________________________________________
> Libav-user mailing list
> Libav-***@ffmpeg.org
> http://ffmpeg.org/mailman/listinfo/libav-user

[Info || Non-Lethal Applications]

>> for asking design questions on the mailing list and being relegated to
>> reading through undocumented source code for a month “open”. Sure it is
>> free of licensing charges, but FFmpeg is not free — unless you have got
>> a vanilla use-case encapsulated in the canned examples, you are most
>> likely in for a deep-sea expedition, and you’ll most definitely be
>> investing a large amount of time trying to understand how to use it.
>
> The main issue is actually that the FFmpeg API is pretty low-level,
> while many users expect something higher level. (Somewhat typical: a
> user wants frame N of video in a specific format. Yeah, implementing
> this using the raw FFmpeg API will take some time.)


I’d like to chime in on that one.
While I’m using FFmpeg on OS X only (at least right now), I wouldn’t insist of having an OS X only wrapper.
A C++ cross-platform one would work just as well for me.

Alternatively, it would be really cool to add a high-level API for the most common use cases to the roadmap:
Getting frame N of video in a specific format, getting audio samples from N to M in a specific format.

As one of the main issues of libavformat seems to be its seeking abilities, it would make a lot of sense from my perspective to encapsulate these issues in a higher level API.
If this is not provided, every user will need to handle seeking issues he encounters with a specific format/codec combination with a new if (case x) doY(); statement.
If it was encapsulated, seeking issues could be solved over time.

I don’t know FFmpeg good enough to know if there’s a policy / design philosophy that doesn’t allow for something like this.

Best regards,

Flo

[wm4]

On Sun, 25 Jan 2015 00:25:34 +0100
Info || Non-Lethal Applications <***@non-lethal-applications.com>
wrote:

> >> for asking design questions on the mailing list and being relegated to
> >> reading through undocumented source code for a month “open”. Sure it is
> >> free of licensing charges, but FFmpeg is not free — unless you have got
> >> a vanilla use-case encapsulated in the canned examples, you are most
> >> likely in for a deep-sea expedition, and you’ll most definitely be
> >> investing a large amount of time trying to understand how to use it.
> >
> > The main issue is actually that the FFmpeg API is pretty low-level,
> > while many users expect something higher level. (Somewhat typical: a
> > user wants frame N of video in a specific format. Yeah, implementing
> > this using the raw FFmpeg API will take some time.)
>
>
> I’d like to chime in on that one.
> While I’m using FFmpeg on OS X only (at least right now), I wouldn’t insist of having an OS X only wrapper.
> A C++ cross-platform one would work just as well for me.

Sure, that's kind of hard to do with Cocoa/Swift.

(Also, C++ APIs are useful for C++ code only.)

> Alternatively, it would be really cool to add a high-level API for the most common use cases to the roadmap:
> Getting frame N of video in a specific format, getting audio samples from N to M in a specific format.

I don't think the FFmpeg project would be opposed to adding such a
high-level (C) API. It was often talked about, but of course nobody has
time etc....

> As one of the main issues of libavformat seems to be its seeking abilities, it would make a lot of sense from my perspective to encapsulate these issues in a higher level API.
> If this is not provided, every user will need to handle seeking issues he encounters with a specific format/codec combination with a new if (case x) doY(); statement.
> If it was encapsulated, seeking issues could be solved over time.

There are two parts to this. One part is that seeking in libavformat is
terrible. Another is that the layer which handles file formats can't do
any form of frame-exact seeking, because it would require decoding some
video and audio (i.e. too low level).

Yet another is that with raw files, it will often seek by assuming some
random constant bitrate. Here it pretty much depends whether you
actually want this (in order to avoid that your video player or so has
to read _all_ data until the seek target), or if you need it more
exact. For the latter case, I think simple on-the-fly indexing could be
added, although it would also require dealing with the terrible seeking
code.

> I don’t know FFmpeg good enough to know if there’s a policy / design philosophy that doesn’t allow for something like this.
>
> Best regards,
>
> Flo
> _______________________________________________
> Libav-user mailing list
> Libav-***@ffmpeg.org
> http://ffmpeg.org/mailman/listinfo/libav-user

[Info || Non-Lethal Applications]

> On 25 Jan 2015, at 00:46, wm4 <***@googlemail.com> wrote:
>
> On Sun, 25 Jan 2015 00:25:34 +0100
> Info || Non-Lethal Applications <***@non-lethal-applications.com>
> wrote:
>
>>>> for asking design questions on the mailing list and being relegated to
>>>> reading through undocumented source code for a month “open”. Sure it is
>>>> free of licensing charges, but FFmpeg is not free — unless you have got
>>>> a vanilla use-case encapsulated in the canned examples, you are most
>>>> likely in for a deep-sea expedition, and you’ll most definitely be
>>>> investing a large amount of time trying to understand how to use it.
>>>
>>> The main issue is actually that the FFmpeg API is pretty low-level,
>>> while many users expect something higher level. (Somewhat typical: a
>>> user wants frame N of video in a specific format. Yeah, implementing
>>> this using the raw FFmpeg API will take some time.)
>>
>>
>> I’d like to chime in on that one.
>> While I’m using FFmpeg on OS X only (at least right now), I wouldn’t insist of having an OS X only wrapper.
>> A C++ cross-platform one would work just as well for me.
>
> Sure, that's kind of hard to do with Cocoa/Swift.
>
> (Also, C++ APIs are useful for C++ code only.)

Not exactly. They don’t work in Swift (there are also ways to use C++ source there) but they can be used in Objective-C (then called Objective-C++) without any action on the developer’s side.
You simply need to change the file extension from m to mm. Not a big deal.

>> Alternatively, it would be really cool to add a high-level API for the most common use cases to the roadmap:
>> Getting frame N of video in a specific format, getting audio samples from N to M in a specific format.
>
> I don't think the FFmpeg project would be opposed to adding such a
> high-level (C) API. It was often talked about, but of course nobody has
> time etc
.

Well, then the only solution is creating a high level wrapper?

[wm4]

On Sun, 25 Jan 2015 00:53:36 +0100
Info || Non-Lethal Applications <***@non-lethal-applications.com>
wrote:

>
> > On 25 Jan 2015, at 00:46, wm4 <***@googlemail.com> wrote:
> >
> > On Sun, 25 Jan 2015 00:25:34 +0100
> > Info || Non-Lethal Applications <***@non-lethal-applications.com>
> > wrote:
> >
> >>>> for asking design questions on the mailing list and being relegated to
> >>>> reading through undocumented source code for a month “open”. Sure it is
> >>>> free of licensing charges, but FFmpeg is not free — unless you have got
> >>>> a vanilla use-case encapsulated in the canned examples, you are most
> >>>> likely in for a deep-sea expedition, and you’ll most definitely be
> >>>> investing a large amount of time trying to understand how to use it.
> >>>
> >>> The main issue is actually that the FFmpeg API is pretty low-level,
> >>> while many users expect something higher level. (Somewhat typical: a
> >>> user wants frame N of video in a specific format. Yeah, implementing
> >>> this using the raw FFmpeg API will take some time.)
> >>
> >>
> >> I’d like to chime in on that one.
> >> While I’m using FFmpeg on OS X only (at least right now), I wouldn’t insist of having an OS X only wrapper.
> >> A C++ cross-platform one would work just as well for me.
> >
> > Sure, that's kind of hard to do with Cocoa/Swift.
> >
> > (Also, C++ APIs are useful for C++ code only.)
>
> Not exactly. They don’t work in Swift (there are also ways to use C++ source there) but they can be used in Objective-C (then called Objective-C++) without any action on the developer’s side.
> You simply need to change the file extension from m to mm. Not a big deal.
>
> >> Alternatively, it would be really cool to add a high-level API for the most common use cases to the roadmap:
> >> Getting frame N of video in a specific format, getting audio samples from N to M in a specific format.
> >
> > I don't think the FFmpeg project would be opposed to adding such a
> > high-level (C) API. It was often talked about, but of course nobody has
> > time etc….
>
> Well, then the only solution is creating a high level wrapper?
>
>

Well yes, it has to be a new API, because there's no API in FFmpeg yet
which can access both demuxer and decoder. Unless by wrapper you mean
an _external_ wrapper, which is something I've argued against in this
thread.

[Marcus Johnson]

I'm new and don't know too much about the API, but honestly seeking is the
biggest issue I think we can all agree upon, I can't implement certain
features because we just can't seek properly

I'm not sure how we could rectify this but there has to be something that
could be done.

[Lucas Soltic]

> Le 25 janv. 2015 à 04:43, Marcus Johnson <***@gmail.com> a écrit :
>
> I'm new and don't know too much about the API, but honestly seeking is the biggest issue I think we can all agree upon, I can't implement certain features because we just can't seek properly
>
> I'm not sure how we could rectify this but there has to be something that could be done.
> _______________________________________________
> Libav-user mailing list
> Libav-***@ffmpeg.org
> http://ffmpeg.org/mailman/listinfo/libav-user

I suppose this is not too off-topic and could help at designing the high level API: what exactly is currently wrong with seeking in FFmpeg? Are there use cases that just don't work? I'm asking because I got it working for a personal project and for all the file formats I've tested it works just fine. I admit it was a bit tricky to implement though, but after understanding how the demuxer and streams work (I mean in general, not FFmpeg specific) it just makes sense.

Lucas

[Marcus Johnson]

You can't really seek with get_bits, there is one seeking function, but
it's in AVStream I believe? it's been a few months since I've looked at the
specifics.

but the most important part is that that function was for seeking to the
next frame, not seeking back x bits which is necessary for verifying CRCs
because the header size isn't constant, and is itself embedded within the
header



On Sun, Jan 25, 2015 at 4:57 AM, Lucas Soltic <***@orange.fr>
wrote:

>
> Le 25 janv. 2015 à 04:43, Marcus Johnson <***@gmail.com> a
> écrit :
>
> I'm new and don't know too much about the API, but honestly seeking is the
> biggest issue I think we can all agree upon, I can't implement certain
> features because we just can't seek properly
>
> I'm not sure how we could rectify this but there has to be something that
> could be done.
> _______________________________________________
> Libav-user mailing list
> Libav-***@ffmpeg.org
> http://ffmpeg.org/mailman/listinfo/libav-user
>
>
> I suppose this is not too off-topic and could help at designing the high
> level API: what exactly is currently wrong with seeking in FFmpeg? Are
> there use cases that just don't work? I'm asking because I got it working
> for a personal project and for all the file formats I've tested it works
> just fine. I admit it was a bit tricky to implement though, but after
> understanding how the demuxer and streams work (I mean in general, not
> FFmpeg specific) it just makes sense.
>
> Lucas
>
> _______________________________________________
> Libav-user mailing list
> Libav-***@ffmpeg.org
> http://ffmpeg.org/mailman/listinfo/libav-user
>
>

[Lucas Soltic]

> Le 25 janv. 2015 à 11:25, Marcus Johnson <***@gmail.com> a écrit :
>
> You can't really seek with get_bits, there is one seeking function, but it's in AVStream I believe? it's been a few months since I've looked at the specifics.
>
> but the most important part is that that function was for seeking to the next frame, not seeking back x bits which is necessary for verifying CRCs because the header size isn't constant, and is itself embedded within the header

Didn't know about get_bits, it really looks too low level for a client, or are you speaking as a FFmpeg developer? As a client I only used avformat_seek_file() / av_seek_frame()

[Marcus Johnson]

I'm trying to extend a decoder in libavcodec, I'm not sure what that
technically makes me, but I'm not using it in a third party program.

[Bradley O'Hearne]

On Jan 24, 2015, at 4:03 PM, wm4 <***@googlemail.com> wrote:
>>
> That's pretty ridiculous. Just because it's not documented as well as
> you want it, it's proprietary?

Mine was a statement of pragmatism: if the design, intent, class relationships, and nuances of uses and behavior are unknown, and largely locked in the minds of its creators, then from a pragmatic point of view (that being the ability to readily understand, use, and leverage the full depth and breadth of the API), yes, it is a pragmatically proprietary body of knowledge.

> Anyway, documentation for open source things is bound to be worse than
> with (popular) closed source APIs, simply because if something is not
> documented, you could just read the source code, find out how it's
> supposed to work, and send a patch to improve the documentation.

That circular reasoning is the achilles heel of many open source projects — which turn their otherwise very valuable content into relatively less valuable assets, even to the point of relegating the use of of the API into a liability, because it is so difficult to understand, and products which use the API become built-in time-sinks to debug and support. Reading source code often doesn’t communicate design or intent, so “just read the source code” is a phrase much easier said than done. In the case of FFmpeg, what is missing in terms of documentation isn’t a method explanation or header doc here or there, it is more tantamount to a full book’s worth of information, which is why I’ve stated prior the realistic audience for being able to produce this is likely a pretty select group of folks.

> (Also, I've never encountered an API that really clearly defined
> everything 100%, whether it was proprietary or open source. If it's
> open source, at least you can find out yourself, and maybe even
> contribute a patch for clarification.)

You might try taking a look at Cocoa API doc, reference guides, and sample code from Apple you despise so much. They are by no means perfect, but it is a world different from what we’ve got to work with in FFmpeg. Again, we’re not talking a simple patch here and there. We are talking a significant writing endeavor.

> I don't know what you got "lambasted" for, but you probably either
> touched a sensible subject, got into some flamewar and internal
> politics, or you did something honestly stupid.

Your assumptions are wrong. I reported behavior, asked design questions, provided full source code and a running app for OS X. Without actually debugging or looking at the source code I provided, devs just denied what I reported, and turned things into a personal issue rather than answer the question. I’ve been on this mailing list for almost three years now, and while I applaud the questions that get answers, my observation is that this mailing list particularly does not like extended discussion of any kind, but particularly design issues and architectural decisions.

> Getting asked for source code if you ask for help is quite reasonable.

Which is why I provided complete source code (twice, a year apart) and a fully running app on Github. No luck. It did make for a few useful discussions off-list with a few others who had similar problems they couldn’t get solved on-list — and those were worthwhile.

> Anyway, in this case I got upset because someone was arguing for
> questionable Apple vendor-lockin

For a Mac app, yeah, I’d love a wrapper native to the platform I’m working on. It changes nothing that exists already in the FFmpeg API. It merely adds an option for those who can benefit. You don’t want to use it, fine — but it hurts nothing, and helps plenty.

> , instead of doing something portable,

That’s your requirement, not others — the entire point of the effort I was proposing was smoothing the skids on a particular platform. I fail to see the downside of this. Again, it changes nothing that exists already. If you don’t want a native API, don’t use it.

> or - got forbid - actually improving FFmpeg.

I couldn’t do it if I wanted to, and I’d speculate the case is the same for many. You have to know all those things we’ve been talking about which aren’t documented anywhere. The advantage of a wrapper is that you can expose only functionality which is more or less understood and known (the well-beaten paths, if such a thing exists) and can have confidence in its behavior, hide the rest, simplifying much for the end-user. When other behavior becomes known, it can then be exposed at a higher level.

> Seriously, you're talking
> about how bad FFmpeg is,

I said nothing of the sort. I specifically said "FFmpeg may be awesome and feature-rich — I’m not debating that “.

> and you want to solve it by putting a shiny
> Apple-only wrapper around it?

Absolutely. I’m not a prisoner of tech-religious views, I believe in the best tool for the job. If an Apple-wrapper improves usage on Apple platforms, and allows me to ship a more stable, more quickly implemented and more easily supported product using FFmpeg, then that’s justification for me. Same with a Windows-wrapper: if it allows the same on a Windows platform, then that’s justification for me. It affects nothing of the current FFmpeg API…it merely uses the current FFmpeg API as a building block to provides a higher-level, distilled API for users who wish to deal at that level of abstraction.

The better question is this — why would people want to oppose the providing of something which has absolutely zero effect on the FFmpeg API, but enhances the alternatives for using it?

> Instead of improving FFmpeg? And you post
> that on a FFmpeg mailing list? I can't help but to make fun of this.

I knew I wouldn’t be disappointed — the turf war surfaces: somehow using FFmpeg as a foundation for building something which can widen the potential audience which might use FFmpeg is viewed as a threat. The whole point was to expand the FFmpeg offering and usability options: entirely appropriate for the mailing list. But point taken…providing higher-level functions and greater usability equals a threat to FFmpeg and offense to the mailing list. That’s about as good an answer as I could get to my original question.

> I don't think it's wrong to get political about evil companies, or to
> make fun of people trapped by evil companies. Also, trolling is fun.

That might be fun for you, but not for people who are trying to get a job done, and could either care less about politics or who don’t have the luxury of dictating to the boss or client what platforms they market their products on.

That statement right there more or less captures what I’ve observed over the years as the active personality of the Libav-user mailing list (not everyone mind you, but some of the more dominant voices present), and that’s exactly what I’ve faced in the past with questions or attempts to understand things at a deeper level. It isn’t a way to endear newcomers who want to ask honest questions, it isn’t a way to build a tight-knit community of people who want to contribute. Quite frankly, it is one additional (and huge) reason why wrappers would be far superior to a user’s FFmpeg experience is that they might avoid this kind of thing entirely.

In truth, I appreciate your candor. It pretty much aligns with what my gut told me would likely surface prior to posting my original question about this — I just thought it worth the risk to ask if it benefitted some. Honesty, no matter whether the answer was the one hoped for or not, is valuable.

Cheers,

Brad

[Thomas Worth]

>
> > Anyway, in this case I got upset because someone was arguing for
> > questionable Apple vendor-lockin
>
> For a Mac app, yeah, I’d love a wrapper native to the platform I’m working
> on. It changes nothing that exists already in the FFmpeg API. It merely
> adds an option for those who can benefit. You don’t want to use it, fine —
> but it hurts nothing, and helps plenty.
>

Brad, I'm a bit confused. An Apple-specific audio/video API already exists:
AVFoundation. It is Objective-C based, and is capable of performing general
purpose audiovisual tasks. It is also documented fairly well, certainly
more so than ffmpeg/libav. I use it in my apps to write MOV files instead
of ffmpeg because I can be guaranteed the MOVs written will be compatible
with the rest of Apple's products. Have you given AVFoundation a try? Is
portability critical? Why stress over ffmpeg when you've got AVFoundation?

[Bradley O'Hearne]

On Jan 25, 2015, at 10:28 PM, Thomas Worth <***@rarevision.com> wrote:

> Have you given AVFoundation a try? Is portability critical? Why stress over ffmpeg when you've got AVFoundation?

Hey Thomas….thanks for the reply. To answer your question: I’m dealing with several use-cases, and across all of those use cases, there’s a number of reasons:

- In one use-case, I have to support QTKit, not AVFoundation.

- In another use-case, I have to support Snow Leopard, in which AVFoundation does not exist.

- I do not believe that AVFoundation supports per-frame encoding, it supports encoding for whole assets or inputs — I believe per-frame encoding is only provided in the VideoToolbox framework as of OS X Mavericks.

- In the other use-cases, which have the luxury of using any OS X version (and thus, AVFoundation can be used) there are codecs and video formats required which AVFoundation does not support.

- Presently Apple does not have any public framework that provides real-time network video streaming capabilities, and with certain protocols (like RTSP or RTMP).

If an app only has to be Yosemite (or possibly Mavericks) compatible with and encoder and video format supported (which my present requirements don’t yet allow), I believe AVFoundation / VideoToolbox / AudioToolbox could handle the capture and encoding needs in my use-case. But I’d still have to have the stream-publishing needs handled by a third-party library like FFmpeg or LIVE555.

Brad

[wm4]

On Sun, 25 Jan 2015 22:02:27 -0700
Bradley O'Hearne <***@bighillsoftware.com> wrote:

> On Jan 24, 2015, at 4:03 PM, wm4 <***@googlemail.com> wrote:
> >>
> > That's pretty ridiculous. Just because it's not documented as well as
> > you want it, it's proprietary?
>
> Mine was a statement of pragmatism: if the design, intent, class relationships, and nuances of uses and behavior are unknown, and largely locked in the minds of its creators, then from a pragmatic point of view (that being the ability to readily understand, use, and leverage the full depth and breadth of the API), yes, it is a pragmatically proprietary body of knowledge.

No, it's just the wrong choice of words.

> > Anyway, documentation for open source things is bound to be worse than
> > with (popular) closed source APIs, simply because if something is not
> > documented, you could just read the source code, find out how it's
> > supposed to work, and send a patch to improve the documentation.
>
> That circular reasoning is the achilles heel of many open source projects — which turn their otherwise very valuable content into relatively less valuable assets, even to the point of relegating the use of of the API into a liability, because it is so difficult to understand, and products which use the API become built-in time-sinks to debug and support. Reading source code often doesn’t communicate design or intent, so “just read the source code” is a phrase much easier said than done. In the case of FFmpeg, what is missing in terms of documentation isn’t a method explanation or header doc here or there, it is more tantamount to a full book’s worth of information, which is why I’ve stated prior the realistic audience for being able to produce this is likely a pretty select group of folks.

Oh no, a group of volunteers who do everything for free and let you use
their stuff for free demand that you put a _little_ effort into
understanding things!

> > (Also, I've never encountered an API that really clearly defined
> > everything 100%, whether it was proprietary or open source. If it's
> > open source, at least you can find out yourself, and maybe even
> > contribute a patch for clarification.)
>
> You might try taking a look at Cocoa API doc, reference guides, and sample code from Apple you despise so much. They are by no means perfect, but it is a world different from what we’ve got to work with in FFmpeg. Again, we’re not talking a simple patch here and there. We are talking a significant writing endeavor.

No, they're not perfect. And again, it's a different dynamic:
undocumented closed source is just as useful as what happens to be
known about it, while you can get more out of undocumented open source.

> > I don't know what you got "lambasted" for, but you probably either
> > touched a sensible subject, got into some flamewar and internal
> > politics, or you did something honestly stupid.
>
> Your assumptions are wrong. I reported behavior, asked design questions, provided full source code and a running app for OS X. Without actually debugging or looking at the source code I provided, devs just denied what I reported, and turned things into a personal issue rather than answer the question. I’ve been on this mailing list for almost three years now, and while I applaud the questions that get answers, my observation is that this mailing list particularly does not like extended discussion of any kind, but particularly design issues and architectural decisions.

I've searched the ffmpeg-devel mailing list, and didn't find your name.
Not even one mail! And my archive goes almost 2 years back.

> > Getting asked for source code if you ask for help is quite reasonable.
>
> Which is why I provided complete source code (twice, a year apart) and a fully running app on Github. No luck. It did make for a few useful discussions off-list with a few others who had similar problems they couldn’t get solved on-list — and those were worthwhile.

Sometimes I get the feeling those with "unsolved" problems just don't
understand enough about multimedia - which, with such a low level API,
is a prerequisite. Yes, the API docs can teach you only about the API,
not general multimedia concepts.

Now I don't want to accuse you or others of not knowing your stuff, nor
do I want to pretend that the problem is just "stupid users who don't
understand". But maybe there's sort of a mismatch of expectations that
leads to frustration. Yes, FFmpeg's API is awfully low level. Yes,
FFmpeg requires you to know a lot about multimedia formats and codecs.
No, FFmpeg doesn't actually abstract multimedia, rather it presents
you with the lowest common denominator. If there's a broken or very
unusual file, FFmpeg might be able to read it, but you get to keep both
pieces. If you have a problem with a certain file, it might actually be
broken, and not because you use underdocumented API wrong. So yes,
instead of shiny plastic, you get raw oil.

That doesn't preclude a high level API from being added, though.

(Just as a detail, don't make the mistake to think the high level API
could be to the low level one. Instead it would be a separate
sub-library alongside of the low level sub-libs, think libavhighlevel
or so.)

Somehow this discussion mixes several issues, like creating a high
level API, mistakes in the existing ffmpeg API, issues with the low
level API, and wrappers. Just because making a wrapper happens to solve
these problems for you on your platform (while not really improving the
general situation).

Also let me just add, cleaning up such a big API with so many users is
hard work. Actually one of the developers who did most for API
improvements lately (and who is on the Libav side) does Libav
development as pure hobby.

> > Anyway, in this case I got upset because someone was arguing for
> > questionable Apple vendor-lockin
>
> For a Mac app, yeah, I’d love a wrapper native to the platform I’m working on. It changes nothing that exists already in the FFmpeg API. It merely adds an option for those who can benefit. You don’t want to use it, fine — but it hurts nothing, and helps plenty.

I don't know, writing a new wrapper for each platform, and duplicating
all the effort to de-low-level ffmpeg sounds very silly to me.

> > , instead of doing something portable,
>
> That’s your requirement, not others — the entire point of the effort I was proposing was smoothing the skids on a particular platform. I fail to see the downside of this. Again, it changes nothing that exists already. If you don’t want a native API, don’t use it.
>
> > or - got forbid - actually improving FFmpeg.
>
> I couldn’t do it if I wanted to, and I’d speculate the case is the same for many. You have to know all those things we’ve been talking about which aren’t documented anywhere. The advantage of a wrapper is that you can expose only functionality which is more or less understood and known (the well-beaten paths, if such a thing exists) and can have confidence in its behavior, hide the rest, simplifying much for the end-user. When other behavior becomes known, it can then be exposed at a higher level.

Again, mixing several issues...

> > Seriously, you're talking
> > about how bad FFmpeg is,
>
> I said nothing of the sort. I specifically said "FFmpeg may be awesome and feature-rich — I’m not debating that “.
>
> > and you want to solve it by putting a shiny
> > Apple-only wrapper around it?
>
> Absolutely. I’m not a prisoner of tech-religious views, I believe in the best tool for the job. If an Apple-wrapper improves usage on Apple platforms, and allows me to ship a more stable, more quickly implemented and more easily supported product using FFmpeg, then that’s justification for me. Same with a Windows-wrapper: if it allows the same on a Windows platform, then that’s justification for me. It affects nothing of the current FFmpeg API…it merely uses the current FFmpeg API as a building block to provides a higher-level, distilled API for users who wish to deal at that level of abstraction.
>
> The better question is this — why would people want to oppose the providing of something which has absolutely zero effect on the FFmpeg API, but enhances the alternatives for using it?

Because duplicated effort that distracts from fixing issues in FFmpeg
directly. It just seems so illogical.

> > Instead of improving FFmpeg? And you post
> > that on a FFmpeg mailing list? I can't help but to make fun of this.
>
> I knew I wouldn’t be disappointed — the turf war surfaces: somehow using FFmpeg as a foundation for building something which can widen the potential audience which might use FFmpeg is viewed as a threat. The whole point was to expand the FFmpeg offering and usability options: entirely appropriate for the mailing list. But point taken…providing higher-level functions and greater usability equals a threat to FFmpeg and offense to the mailing list. That’s about as good an answer as I could get to my original question.

See previous paragraph.

Yes, I see such vendor-specific things as a "threat". I couldn't care
less about them, but they annoy me anyway. (On the other hand, plugging
FFmpeg into existing vendor-specific frameworks is a sane solution.
Though Perian ceased existing, apparently due to Apple being assholes.)

> > I don't think it's wrong to get political about evil companies, or to
> > make fun of people trapped by evil companies. Also, trolling is fun.
>
> That might be fun for you, but not for people who are trying to get a job done, and could either care less about politics or who don’t have the luxury of dictating to the boss or client what platforms they market their products on.
>
> That statement right there more or less captures what I’ve observed over the years as the active personality of the Libav-user mailing list (not everyone mind you, but some of the more dominant voices present), and that’s exactly what I’ve faced in the past with questions or attempts to understand things at a deeper level. It isn’t a way to endear newcomers who want to ask honest questions, it isn’t a way to build a tight-knit community of people who want to contribute. Quite frankly, it is one additional (and huge) reason why wrappers would be far superior to a user’s FFmpeg experience is that they might avoid this kind of thing entirely.

I don't know, actually rarely any real FFmpeg devs show up on
libav-user, so libav-user is not really representative of "the FFmpeg
developers".

>
> In truth, I appreciate your candor. It pretty much aligns with what my gut told me would likely surface prior to posting my original question about this — I just thought it worth the risk to ask if it benefitted some. Honesty, no matter whether the answer was the one hoped for or not, is valuable.
>
> Cheers,
>
> Brad
>
>
> _______________________________________________
> Libav-user mailing list
> Libav-***@ffmpeg.org
> http://ffmpeg.org/mailman/listinfo/libav-user

[Bradley O'Hearne]

> On Jan 25, 2015, at 11:04 PM, wm4 <***@googlemail.com> wrote:
>

This exchange appears to be turning personal (actually it did a message ago) and isn’t really headed in the direction of being productive, but I’ll address whatever makes sense to get the point across, and then I’ll bow out.

> Oh no, a group of volunteers who do everything for free and let you use
> their stuff for free demand that you put a _little_ effort into
> understanding things!

That was not the gripe
you know it, I know it, everyone that uses FFmpeg knows it. The gripe is requiring an FFmpeg user to take a multi-month, Nicolas Cage-ian cross-country National Treasure quest which requires one to steal the Declaration of Independence, find the hidden lever on the desk in the Oval Office, kidnap the President, find the Book of Secrets in the Library of Congress, and then uncover the secret FFmpeg functionality engraved on a document hidden in a treasure room buried within Mount Rushmore in order to understand it. Forcing a user to read source code for a week or a month isn’t a virtue — it is a major weakness. Even then, many times the information uncovered is merely best guess, and doesn’t answer a host of contingencies. Users of FFmpeg shouldn’t have to guess at this stuff.

> No, they're not perfect. And again, it's a different dynamic:
> undocumented closed source is just as useful as what happens to be
> known about it, while you can get more out of undocumented open source.

The comparison was not undocumented closed source vs. undocumented open source. It was a comparison between *documented* closed source with reference guides and extensive sample code vs. relatively undocumented open source where the gaps aren’t able to be easily filled with other resources (mailing list discussions, samples, etc.). More specifically, for the domain in question, I’d take Apple’s (or for that matter, Microsoft’s) API documentation and resources 1000 times out of a 1000 over FFmpeg’s. Whatever the flaws in what Apple/Microsoft provide, they at least have an appreciation of the fact that third parties are the consumers of their APIs, and make attempts to provide understandings of them.

> Sometimes I get the feeling those with "unsolved" problems just don't
> understand enough about multimedia

I’m sure that’s a theory that makes some feel better
.it just isn’t true in my observation over the years. The issues I brought to this list weren't a matter of “feelings”. They were reports of specific behavior observed paired with source code which could reproduce it for anyone who cared to try, and instead of being answered, resulted in my being repeatedly called a “troll” for reporting them by an FFmpeg dev
.of course no fault was ever found with the source (I don’t think it was ever even looked at) nor was any answer ever given. If devs can’t provide an answer, fine. But trying to hide that by attacking the person
that’s weak, and counterproductive.

> I don't know, writing a new wrapper for each platform, and duplicating
> all the effort to de-low-level ffmpeg sounds very silly to me.

Suit yourself. I’d rather use a toilet than have someone direct me to the plumbing aisle at the hardware store whenever I need to use the restroom.

> Yes, I see such vendor-specific things as a "threat". I couldn't care
> less about them, but they annoy me anyway. (On the other hand, plugging
> FFmpeg into existing vendor-specific frameworks is a sane solution.
> Though Perian ceased existing, apparently due to Apple being assholes.)

You’ve got every right to your opinion, and I respect that. But this is your personal bugaboo, and it isn’t where the entirety of the FFmpeg constituency lives. FFmpeg claims support on vendor-specific platforms, it embraces vendor-specific video and audio formats, and network protocols. Your gripe isn’t a technical issue — it is a political one. Your gripe with an FFmpeg wrapper is politically oriented — it is not user-oriented. Unless you specifically have a reason for desiring to work at a low level, and there certainly are legitimate reasons for wanting to do this, I’d guess that the majority of users would prefer to work at a higher level abstraction. Even if it weren’t “most”, it would certainly be substantial.

I am fine to agree to disagree. Again, I respect your right to your opinion. I’m just amazed at the fact that turf trumps value here (“threat”). My sole purpose was to:

- Increase the value of FFmpeg.
- Provide an easily understandable abstraction to an otherwise very difficult to understand FFmpeg API.
- Provide something which can be thoroughly documented.
- Provide something less intimidating which more users would be willing to use and support in their products.
- Contribute something of value where I am otherwise on weak footing to do so.

In the providing of these things, there is *no* alteration or neglect of the FFmpeg API required — it is left entirely alone, and FFmpeg is the foundational building block underneath. I’m just floored that some would oppose the community having such a thing. But again, I DO appreciate your honesty. I can leave this conversation duly informed, and I’ve already reached a conclusion to my original question.

Anyway, I think this discussion has run its course, so we probably don’t need to belabor the issue. I likely won’t be pursuing any public wrapper for FFmpeg. I have clients which need FFmpeg functionality (which I’ve already provided) but with expansive functionality and a higher-level API, and they need something they can understand, maintain, extend, and support. If I provide a wrapper (beyond the one I’ve already provided for them) I’ll keep it private so as not to offend the mailing list, and over time, I’ll consider a higher-level API something which needs to take place without FFmpeg as a building block, but either some other building block or rolled from scratch.

Thanks again all
.

Brad

[Carl Eugen Hoyos]

Joshua Kordani <***@...> writes:

> I still don't understand the difference between
> the FFmpeg libraries and the LibAV project

It is explained here:
http://blog.pkh.me/p/13-the-ffmpeg-libav-situation.html
Please remember that all avconv versions contain
several hundred known bugs that are not reproducible
with current FFmpeg, some of them security-relevant.

Carl Eugen

[Joshua Kordani]

On 1/23/15 2:49 PM, Carl Eugen Hoyos wrote:
> Joshua Kordani <***@...> writes:
>
>> I still don't understand the difference between
>> the FFmpeg libraries and the LibAV project
>
> It is explained here:
> http://blog.pkh.me/p/13-the-ffmpeg-libav-situation.html
> Please remember that all avconv versions contain
> several hundred known bugs that are not reproducible
> with current FFmpeg, some of them security-relevant.
>
> Carl Eugen
>
> _______________________________________________
> Libav-user mailing list
> Libav-***@ffmpeg.org
> http://ffmpeg.org/mailman/listinfo/libav-user
>

Thanks, I've read that before.

Joshua Kordani
LSA Autonomy

[Carl Eugen Hoyos]

Bradley O'Hearne <***@...> writes:

> one of the things which various list members
> raised was that most (if not all) of the Libav
> devs did not either use or have access to OS X

Just a short comment:
Apple OSX is regularly tested, see fate.ffmpeg.org
Many FFmpeg developers have access to OSX systems,
some use it as main development platform.

I can obviously not speak for the avconv developers
but I don't think "[don't] have access to OS X" is
true for them.

Carl Eugen

[Thomas Worth]

>
> Perhaps I’m the only one on the planet using Libav on Apple platforms,
> though I’m betting I’m not (actually I know I’m not, as a few have
> contacted me off-list). Also, it might take some of the headache away from
> the Libav devs who don’t use Apple platforms to answer some support
> questions. For us who like apples, a nice, clean Swift API might be very
> nice, and save a lot of time and headaches. I might be able to produce such
> a thing, first somewhat limited and rudimentary, and then mature it over
> time.
>
> Would anyone be interested in such an API?
>

I'll add my thoughts as someone who has spent plenty of sleepless nights
dissecting ffmpeg.

Right off the bat, I'll say the ffmpeg/libav C API works perfectly well on
Mac OS X. I don't see how switching the API to work with another, possibly
less-supported language would be advantageous. Mac OS X and iOS development
is already typically done in C/Objective-C, so the API just works as-is.
Most of the "under the hood" code I write is done in C for maximum
portability, and so I have not had any issues with ffmpeg/avcodec/avformat,
etc. on a Mac. It just... works. Period. It even works on Windows, which is
amazing. But not really. Why? C. Bravo.

That said, the ffmpeg APIs are pretty low-level. Their functionality
(especially avformat) in my view is very much governed by the file format
containers. What helped me understand the way these APIs work is to learn
the file formats in-and-out. I primarily work with MOV/MP4 files, and
learning the timings and the way the data is stored in those containers
gave me tremendous insight into the way avformat works. AVFoundation, in
many ways, works like avformat and it's because both of the APIs' behavior
is dictated by the file formats. Learn the file formats, and things become
much more clear.

There's also been a lot of complaints about the seek API. I'll say this:
frame-accurate seeking is absolutely possible with ffmpeg, even with GOP
formats. However, there's a lot of additional work you need to do for this
to be practical, and I'm not sure there's a "one size fits all" solution to
this. So, if you feel like spending a few days figuring this out, it's
doable. It just depends on how committed you are.

I sympathize with your situation, Brad. It took me a good while to get up
to speed, and there was certainly more than one instance where I would just
stare at a bunch of code on the screen and yell, "WHY THE FUCK DOESN'T THIS
WORK?!" But, after sticking with it, reading the ffmpeg headers (yep,
that's your documentation) and experimenting, the puzzle started coming
together. I've written my own C library for common things that works the
way I want it to for my coding style, and now new development goes much
more quickly. You might try doing something similar. That is, once you've
recovered from tearing all your hair out. ;-)

[Anton Shekhovtsov]

2015-01-25 14:53 GMT+02:00 Thomas Worth <***@rarevision.com>:

> There's also been a lot of complaints about the seek API. I'll say this:
> frame-accurate seeking is absolutely possible with ffmpeg, even with GOP
> formats. However, there's a lot of additional work you need to do for this
> to be practical, and I'm not sure there's a "one size fits all" solution to
> this. So, if you feel like spending a few days figuring this out, it's
> doable. It just depends on how committed you are.
>
>
A while ago I started to make and input driver for VirtualDub. I can go
deep on this subject. The problem is indexing API: it is not present. Event
with best container (mp4 fully indexed) there is no straight API to get
frame number by packet or seek to frame number or which frames I can seek
to etc. What I actually learned: index_entries often contains valid
information for such purpose. BUT it is not exposed as public API and no
guarantees are made.

[Don Moir]

>2015-01-25 14:53 GMT+02:00 Thomas Worth <***@rarevision.com>:

>There's also been a lot of complaints about the seek API. I'll say this: frame-accurate seeking
> is absolutely possible with ffmpeg, even with GOP formats. However, there's a lot of additional work
>you need to do for this to be practical, and I'm not sure there's a "one size fits all" solution to this.
>So, if you feel >like spending a few days figuring this out, it's doable. It just depends on how committed you are.

I front end ffmpeg with my own API. This API does time accurate seeking and a lot of work to do. The API does more than wrap ffmpeg
though. It attempts to address problems with ffmpeg and also deals with display output, audio analysis, audio devices, device
enumeration, special effects, scaling and more. You might say well ffmpeg can do this, but not really in the way I need it.

For me it had to be that it all worked in the best possible way correcting any potential problem and had to be available to others
that had no notion of ffmpeg.

Personally I am happy to do my own API for this and can easily customize it to meet current or future needs. ffmpeg just serves as
the low level decoder for audio and video and that is fine with me and ffmpeg is only part of the internal libraries and
capabilites.

I am not sure I would want to deal with yet another API, but ongoing improvements in the ffmpeg API are welcome.

[Bradley O'Hearne]

> On Jan 25, 2015, at 7:04 AM, Don Moir <***@comcast.net> wrote:
>
> I front end ffmpeg with my own API. This API does time accurate seeking and a lot of work to do. The API does more than wrap ffmpeg though. It attempts to address problems with ffmpeg and also deals with display output, audio analysis, audio devices, device enumeration, special effects, scaling and more. You might say well ffmpeg can do this, but not really in the way I need it.
>
> For me it had to be that it all worked in the best possible way correcting any potential problem and had to be available to others that had no notion of ffmpeg.
>
> Personally I am happy to do my own API for this and can easily customize it to meet current or future needs. ffmpeg just serves as the low level decoder for audio and video and that is fine with me and ffmpeg is only part of the internal libraries and capabilites.

Don — Thanks for your response…you’ve stated well great reasons for a higher-level abstraction, which is exactly along the lines of what I was thinking.

All — thanks to everyone who replied, regardless of the expressed sentiment or position on the matter. All feedback was helpful, and helped me reach a conclusion of where and for whom my efforts and ideas would be best contributed.

Cheers,

Brad