HW-accelerated codecs for oreo-x86

[Michael Goffioul]

Has anybody already attempted to port the HW-accelerated video decoding
layer to oreo-x86 branch?

Thanks,
Michael.

[Michael Goffioul]

That's something I'm considering. I had a quick look today, but it's not
gonna be trivial. In particular the changes required in frameworks/av/ to
support the ffmpeg codecs. I'm having trouble finding an entry point, like
a patch set to apply, or a git merge/rebase to start with. It ends up with
a large amount of changes and conflicts. How would you start the process?

Michael.

Post by Michael Goffioul
Has anybody already attempted to port the HW-accelerated video decoding
layer to oreo-x86 branch?

I think no...
I plan to do it after other issues
are resolved, probably in -rc2.
Do you want to try now?
--
Chih-Wei
Android-x86 project
http://www.android-x86.org
--
You received this message because you are subscribed to the Google Groups
"Android-x86" group.
To unsubscribe from this group and stop receiving emails from it, send an
Visit this group at https://groups.google.com/group/android-x86.
For more options, visit https://groups.google.com/d/optout.

[Michael Goffioul]

The main problem is actually not the codec layer itself. I could get the
OMX components to work with VAAPI acceleration pretty easily. The extractor
layer on the other end is a pain to port, because it's spread all over the
place in frameworks/av/. Fortunately, I only need the codec layer (using a
ExoPlayer app, which has its own extractors), so I might just live with
that.

Michael.



On Wed, Apr 11, 2018 at 11:44 PM, Michael Goffioul <

Post by Michael Goffioul
That's something I'm considering. I had a quick look today, but it's not
gonna be trivial. In particular the changes required in frameworks/av/ to
support the ffmpeg codecs. I'm having trouble finding an entry point, like
a patch set to apply, or a git merge/rebase to start with. It ends up with
a large amount of changes and conflicts. How would you start the process?
Michael.

Post by Michael Goffioul
Has anybody already attempted to port the HW-accelerated video decoding
layer to oreo-x86 branch?

I think no...
I plan to do it after other issues
are resolved, probably in -rc2.
Do you want to try now?
--
Chih-Wei
Android-x86 project
http://www.android-x86.org
--
You received this message because you are subscribed to the Google Groups
"Android-x86" group.
To unsubscribe from this group and stop receiving emails from it, send an
Visit this group at https://groups.google.com/group/android-x86.
For more options, visit https://groups.google.com/d/optout.

[Michael Goffioul]

Post by Michael Goffioul
That's something I'm considering. I had a quick look today, but it's not
gonna be trivial. In particular the changes required in frameworks/av/ to
support the ffmpeg codecs. I'm having trouble finding an entry point,

like a

Post by Michael Goffioul
patch set to apply, or a git merge/rebase to start with. It ends up with

a

Post by Michael Goffioul
large amount of changes and conflicts. How would you start the process?

Right. It's not a trivial work. That's why I defer it.
Actually I only did that in old android 4.3/4.4 (jb-x86/kk-x86).
Fortunately the CM (LineageOS) guys picked my patches
to their repo and maintained them to android 5/6/7.
In nougat-x86 I just merged their patches back to our repo.
But seems LineageOS hasn't ported (or dropped?) it for oreo
so we may need to do it ourselves.

I found a partial port in HalogenOS repos (on github) for Android/8.1 and
tried that as a starting point. But it's incomplete and more importantly
they don't seem to use ffmpeg codecs, so that didn't really serve my
purpose. The change in the build system (blueprint), the relocation of
headers, and the lack of clear patch set makes the port work quite
cumbersome. I spent hours going through half of the conflicts after merging
nougat into oreo, 99% of which are not relevant for my use case (camera,
audio effects, exotic video format support...).

[Michael Goffioul]

On Thu, Apr 12, 2018 at 10:59 PM, Michael Goffioul <

Post by Michael Goffioul

Post by Michael Goffioul
That's something I'm considering. I had a quick look today, but it's not
gonna be trivial. In particular the changes required in frameworks/av/

to

Post by Michael Goffioul
support the ffmpeg codecs. I'm having trouble finding an entry point,

like a

Post by Michael Goffioul
patch set to apply, or a git merge/rebase to start with. It ends up

with a

Post by Michael Goffioul
large amount of changes and conflicts. How would you start the process?

Right. It's not a trivial work. That's why I defer it.
Actually I only did that in old android 4.3/4.4 (jb-x86/kk-x86).
Fortunately the CM (LineageOS) guys picked my patches
to their repo and maintained them to android 5/6/7.
In nougat-x86 I just merged their patches back to our repo.
But seems LineageOS hasn't ported (or dropped?) it for oreo
so we may need to do it ourselves.

I found a partial port in HalogenOS repos (on github) for Android/8.1 and
tried that as a starting point. But it's incomplete and more importantly
they don't seem to use ffmpeg codecs, so that didn't really serve my
purpose. The change in the build system (blueprint), the relocation of
headers, and the lack of clear patch set makes the port work quite
cumbersome. I spent hours going through half of the conflicts after merging
nougat into oreo, 99% of which are not relevant for my use case (camera,
audio effects, exotic video format support...).

I took an alternate approach: instead of porting every customizations made
in branch nougat-x86, I only took the pieces that are needed to make the
extractor layer to work (so for instance, I dropped everything that is
related to libavextensions, camera, pcm/24bits...). I could get the
extractor layer to work, with va-api accelerated codecs for h264 and mpeg2.
I'll try to make it available later this week on github.

However, I still have problems with audio. I have a test video file
containing AC3 audio (48000 Hz, stereo, fltp, 384 kb/s). When I play it in
the gallery player, the sound is completely distorted and looks more like a
loud white noise. When I play it with ExoPlayer, which does not use the
extractor layer, it's fine. It seems to be related to the metadata that are
injected by FFmpegExtractor and passed to the codec, one notable difference
being the target PCM encoding used by SoftFFmpegAudio.

When the extractor layer is used, FFmpegExtractor set "pcm-encoding"
to kAudioEncodingPcmFloat, based on the content of the audio track. This is
passed down to the codec through FFMPEGSoftCodec::setRawAudioFormat and the
codec ends up building a resampling filter for "48000 Hz fltp => 48000 Hz
flt".

When the extactor layer is not used, there's not metadata available,
and FFMPEGSoftCodec::setRawAudioFormat uses the default
encoding kAudioEncodingPcm16bit. The codec ends up building a resampling
filter for "48000 Hz fltp => 48000 Hz s16".

Any hint or suggestion?

Thanks,
Michael.

[Michael Goffioul]

On Sun, Apr 15, 2018 at 6:52 PM, Michael Goffioul <

Post by Michael Goffioul
On Thu, Apr 12, 2018 at 10:59 PM, Michael Goffioul <

Post by Michael Goffioul

Post by Michael Goffioul
That's something I'm considering. I had a quick look today, but it's

not

Post by Michael Goffioul
gonna be trivial. In particular the changes required in frameworks/av/

to

Post by Michael Goffioul
support the ffmpeg codecs. I'm having trouble finding an entry point,

like a

Post by Michael Goffioul
patch set to apply, or a git merge/rebase to start with. It ends up

with a

Post by Michael Goffioul
large amount of changes and conflicts. How would you start the process?

Right. It's not a trivial work. That's why I defer it.
Actually I only did that in old android 4.3/4.4 (jb-x86/kk-x86).
Fortunately the CM (LineageOS) guys picked my patches
to their repo and maintained them to android 5/6/7.
In nougat-x86 I just merged their patches back to our repo.
But seems LineageOS hasn't ported (or dropped?) it for oreo
so we may need to do it ourselves.

I found a partial port in HalogenOS repos (on github) for Android/8.1 and
tried that as a starting point. But it's incomplete and more importantly
they don't seem to use ffmpeg codecs, so that didn't really serve my
purpose. The change in the build system (blueprint), the relocation of
headers, and the lack of clear patch set makes the port work quite
cumbersome. I spent hours going through half of the conflicts after merging
nougat into oreo, 99% of which are not relevant for my use case (camera,
audio effects, exotic video format support...).

I took an alternate approach: instead of porting every customizations made
in branch nougat-x86, I only took the pieces that are needed to make the
extractor layer to work (so for instance, I dropped everything that is
related to libavextensions, camera, pcm/24bits...). I could get the
extractor layer to work, with va-api accelerated codecs for h264 and mpeg2.
I'll try to make it available later this week on github.
However, I still have problems with audio. I have a test video file
containing AC3 audio (48000 Hz, stereo, fltp, 384 kb/s). When I play it in
the gallery player, the sound is completely distorted and looks more like a
loud white noise. When I play it with ExoPlayer, which does not use the
extractor layer, it's fine. It seems to be related to the metadata that are
injected by FFmpegExtractor and passed to the codec, one notable difference
being the target PCM encoding used by SoftFFmpegAudio.
When the extractor layer is used, FFmpegExtractor set "pcm-encoding"
to kAudioEncodingPcmFloat, based on the content of the audio track. This is
passed down to the codec through FFMPEGSoftCodec::setRawAudioFormat and
the codec ends up building a resampling filter for "48000 Hz fltp => 48000
Hz flt".
When the extactor layer is not used, there's not metadata available,
and FFMPEGSoftCodec::setRawAudioFormat uses the default encoding kAudioEncodingPcm16bit.
The codec ends up building a resampling filter for "48000 Hz fltp
=> 48000 Hz s16".
Any hint or suggestion?

It appears this is due to the lack of support for non-PCM16 playback, which
I haven't ported. So I simply disabled the feature in the extractor layer.
I've pushed the result to github:
https://github.com/goffioul/android-frameworks-av/tree/hwaccel-simple
https://github.com/goffioul/external-stagefright-plugins/tree/hwaccel-simple

Michael.

['Florian E' via Android-x86]

Thanks alot!

Post by Michael Goffioul

Post by Michael Goffioul

Post by Michael Goffioul

Post by Michael Goffioul

Post by Michael Goffioul
That's something I'm considering. I had a quick look today, but it's

not

Post by Michael Goffioul
gonna be trivial. In particular the changes required in

frameworks/av/ to

Post by Michael Goffioul
support the ffmpeg codecs. I'm having trouble finding an entry point,

like a

Post by Michael Goffioul
patch set to apply, or a git merge/rebase to start with. It ends up

with a

Post by Michael Goffioul
large amount of changes and conflicts. How would you start the

process?
Right. It's not a trivial work. That's why I defer it.
Actually I only did that in old android 4.3/4.4 (jb-x86/kk-x86).
Fortunately the CM (LineageOS) guys picked my patches
to their repo and maintained them to android 5/6/7.
In nougat-x86 I just merged their patches back to our repo.
But seems LineageOS hasn't ported (or dropped?) it for oreo
so we may need to do it ourselves.

I found a partial port in HalogenOS repos (on github) for Android/8.1
and tried that as a starting point. But it's incomplete and more
importantly they don't seem to use ffmpeg codecs, so that didn't really
serve my purpose. The change in the build system (blueprint), the
relocation of headers, and the lack of clear patch set makes the port work
quite cumbersome. I spent hours going through half of the conflicts after
merging nougat into oreo, 99% of which are not relevant for my use case
(camera, audio effects, exotic video format support...).

I took an alternate approach: instead of porting every customizations
made in branch nougat-x86, I only took the pieces that are needed to make
the extractor layer to work (so for instance, I dropped everything that is
related to libavextensions, camera, pcm/24bits...). I could get the
extractor layer to work, with va-api accelerated codecs for h264 and mpeg2.
I'll try to make it available later this week on github.
However, I still have problems with audio. I have a test video file
containing AC3 audio (48000 Hz, stereo, fltp, 384 kb/s). When I play it in
the gallery player, the sound is completely distorted and looks more like a
loud white noise. When I play it with ExoPlayer, which does not use the
extractor layer, it's fine. It seems to be related to the metadata that are
injected by FFmpegExtractor and passed to the codec, one notable difference
being the target PCM encoding used by SoftFFmpegAudio.
When the extractor layer is used, FFmpegExtractor set "pcm-encoding"
to kAudioEncodingPcmFloat, based on the content of the audio track. This is
passed down to the codec through FFMPEGSoftCodec::setRawAudioFormat and the
codec ends up building a resampling filter for "48000 Hz fltp => 48000 Hz
flt".
When the extactor layer is not used, there's not metadata available,
and FFMPEGSoftCodec::setRawAudioFormat uses the default
encoding kAudioEncodingPcm16bit. The codec ends up building a resampling
filter for "48000 Hz fltp => 48000 Hz s16".
Any hint or suggestion?

It appears this is due to the lack of support for non-PCM16 playback,
which I haven't ported. So I simply disabled the feature in the extractor
https://github.com/goffioul/android-frameworks-av/tree/hwaccel-simple
https://github.com/goffioul/external-stagefright-plugins/tree/hwaccel-simple
Michael.

[Michael Goffioul]

Note that you also need to add the FFMPEG codecs to media_codecs.xml,
similar to this:
https://osdn.net/projects/android-x86/scm/git/device-generic-common/commits/7f5c1f87c60d11a1e1fac226b2dbc4749bc6c7be#fd-media_codecs.xml

Michael.


On Mon, Apr 16, 2018 at 11:18 AM, 'Florian E' via Android-x86 <

Post by 'Florian E' via Android-x86
Thanks alot!

Post by Michael Goffioul

Post by Michael Goffioul
On Thu, Apr 12, 2018 at 10:59 PM, Michael Goffioul <

On Thu, Apr 12, 2018 at 10:20 PM, Chih-Wei Huang <

Post by Michael Goffioul

Post by Michael Goffioul
That's something I'm considering. I had a quick look today, but it's

not

Post by Michael Goffioul
gonna be trivial. In particular the changes required in

frameworks/av/ to

Post by Michael Goffioul
support the ffmpeg codecs. I'm having trouble finding an entry

point, like a

Post by Michael Goffioul
patch set to apply, or a git merge/rebase to start with. It ends up

with a

Post by Michael Goffioul
large amount of changes and conflicts. How would you start the

process?
Right. It's not a trivial work. That's why I defer it.
Actually I only did that in old android 4.3/4.4 (jb-x86/kk-x86).
Fortunately the CM (LineageOS) guys picked my patches
to their repo and maintained them to android 5/6/7.
In nougat-x86 I just merged their patches back to our repo.
But seems LineageOS hasn't ported (or dropped?) it for oreo
so we may need to do it ourselves.

I found a partial port in HalogenOS repos (on github) for Android/8.1
and tried that as a starting point. But it's incomplete and more
importantly they don't seem to use ffmpeg codecs, so that didn't really
serve my purpose. The change in the build system (blueprint), the
relocation of headers, and the lack of clear patch set makes the port work
quite cumbersome. I spent hours going through half of the conflicts after
merging nougat into oreo, 99% of which are not relevant for my use case
(camera, audio effects, exotic video format support...).

I took an alternate approach: instead of porting every customizations
made in branch nougat-x86, I only took the pieces that are needed to make
the extractor layer to work (so for instance, I dropped everything that is
related to libavextensions, camera, pcm/24bits...). I could get the
extractor layer to work, with va-api accelerated codecs for h264 and mpeg2.
I'll try to make it available later this week on github.
However, I still have problems with audio. I have a test video file
containing AC3 audio (48000 Hz, stereo, fltp, 384 kb/s). When I play it in
the gallery player, the sound is completely distorted and looks more like a
loud white noise. When I play it with ExoPlayer, which does not use the
extractor layer, it's fine. It seems to be related to the metadata that are
injected by FFmpegExtractor and passed to the codec, one notable difference
being the target PCM encoding used by SoftFFmpegAudio.
When the extractor layer is used, FFmpegExtractor set "pcm-encoding"
to kAudioEncodingPcmFloat, based on the content of the audio track. This is
passed down to the codec through FFMPEGSoftCodec::setRawAudioFormat and
the codec ends up building a resampling filter for "48000 Hz fltp => 48000
Hz flt".
When the extactor layer is not used, there's not metadata available,
and FFMPEGSoftCodec::setRawAudioFormat uses the default
encoding kAudioEncodingPcm16bit. The codec ends up building a
resampling filter for "48000 Hz fltp => 48000 Hz s16".
Any hint or suggestion?

It appears this is due to the lack of support for non-PCM16 playback,
which I haven't ported. So I simply disabled the feature in the extractor
https://github.com/goffioul/android-frameworks-av/tree/hwaccel-simple
https://github.com/goffioul/external-stagefright-plugins/tre
e/hwaccel-simple
Michael.
--

You received this message because you are subscribed to the Google Groups
"Android-x86" group.
To unsubscribe from this group and stop receiving emails from it, send an
Visit this group at https://groups.google.com/group/android-x86.
For more options, visit https://groups.google.com/d/optout.

[Michael Goffioul]

On Mon, Apr 16, 2018 at 11:30 AM, Michael Goffioul <

Post by Michael Goffioul
Note that you also need to add the FFMPEG codecs to media_codecs.xml,
https://osdn.net/projects/android-x86/scm/git/device-
generic-common/commits/7f5c1f87c60d11a1e1fac226b2dbc4
749bc6c7be#fd-media_codecs.xml
Michael.
On Mon, Apr 16, 2018 at 11:18 AM, 'Florian E' via Android-x86 <

Post by 'Florian E' via Android-x86
Thanks alot!

Post by Michael Goffioul

Post by Michael Goffioul
On Thu, Apr 12, 2018 at 10:59 PM, Michael Goffioul <

On Thu, Apr 12, 2018 at 10:20 PM, Chih-Wei Huang <

Post by Michael Goffioul

Post by Michael Goffioul
That's something I'm considering. I had a quick look today, but

it's not

Post by Michael Goffioul
gonna be trivial. In particular the changes required in

frameworks/av/ to

Post by Michael Goffioul
support the ffmpeg codecs. I'm having trouble finding an entry

point, like a

Post by Michael Goffioul
patch set to apply, or a git merge/rebase to start with. It ends up

with a

Post by Michael Goffioul
large amount of changes and conflicts. How would you start the

process?
Right. It's not a trivial work. That's why I defer it.
Actually I only did that in old android 4.3/4.4 (jb-x86/kk-x86).
Fortunately the CM (LineageOS) guys picked my patches
to their repo and maintained them to android 5/6/7.
In nougat-x86 I just merged their patches back to our repo.
But seems LineageOS hasn't ported (or dropped?) it for oreo
so we may need to do it ourselves.

I found a partial port in HalogenOS repos (on github) for Android/8.1
and tried that as a starting point. But it's incomplete and more
importantly they don't seem to use ffmpeg codecs, so that didn't really
serve my purpose. The change in the build system (blueprint), the
relocation of headers, and the lack of clear patch set makes the port work
quite cumbersome. I spent hours going through half of the conflicts after
merging nougat into oreo, 99% of which are not relevant for my use case
(camera, audio effects, exotic video format support...).

I took an alternate approach: instead of porting every customizations
made in branch nougat-x86, I only took the pieces that are needed to make
the extractor layer to work (so for instance, I dropped everything that is
related to libavextensions, camera, pcm/24bits...). I could get the
extractor layer to work, with va-api accelerated codecs for h264 and mpeg2.
I'll try to make it available later this week on github.
However, I still have problems with audio. I have a test video file
containing AC3 audio (48000 Hz, stereo, fltp, 384 kb/s). When I play it in
the gallery player, the sound is completely distorted and looks more like a
loud white noise. When I play it with ExoPlayer, which does not use the
extractor layer, it's fine. It seems to be related to the metadata that are
injected by FFmpegExtractor and passed to the codec, one notable difference
being the target PCM encoding used by SoftFFmpegAudio.
When the extractor layer is used, FFmpegExtractor set "pcm-encoding"
to kAudioEncodingPcmFloat, based on the content of the audio track. This is
passed down to the codec through FFMPEGSoftCodec::setRawAudioFormat
and the codec ends up building a resampling filter for "48000 Hz fltp
=> 48000 Hz flt".
When the extactor layer is not used, there's not metadata available,
and FFMPEGSoftCodec::setRawAudioFormat uses the default
encoding kAudioEncodingPcm16bit. The codec ends up building a
resampling filter for "48000 Hz fltp => 48000 Hz s16".
Any hint or suggestion?

It appears this is due to the lack of support for non-PCM16 playback,
which I haven't ported. So I simply disabled the feature in the extractor
https://github.com/goffioul/android-frameworks-av/tree/hwaccel-simple
https://github.com/goffioul/external-stagefright-plugins/tre
e/hwaccel-simple

I've pushed a new branch with a different approach in:
https://github.com/goffioul/android-frameworks-av/tree/hwaccel-merge

This is basically a merge of halogenos/XOS-8.1 into oreo-x86 and then
porting the changes from my other branch. The halogenos people have already
ported part of the features that were imported from CM/LineageOS, namely
the use of libavextensions, support for non-PCM16 audio in offloading mode,
and probably some others. I cannot really tests these features on my
hardware, so they might actually not work... Also I'm not sure whether
these features are actually useful for Android-x86.

One feature I know is missing, compared to nougat-x86, is the camera
extension. I know there's some port that already exists in the
LineageOS-15.1 branch, but again, I'm not sure this is of any use for
Android-x86.

Michael.

[youling 257]

06-19 16:43:55.866 5298 5298 W /system/bin/mediaextractor: Could not read
additional policy file '/vendor/etc/seccomp_policy/mediaextractor.policy'
06-19 16:43:55.867 5298 5298 W /system/bin/mediaextractor:
libminijail[5298]: allowing syscall: socketcall
06-19 16:43:55.867 5298 5298 W /system/bin/mediaextractor:
libminijail[5298]: allowing syscall: writev
06-19 16:43:55.867 5298 5298 W /system/bin/mediaextractor:
libminijail[5298]: allowing syscall: fcntl64
06-19 16:43:55.867 5298 5298 W /system/bin/mediaextractor:
libminijail[5298]: allowing syscall: clock_gettime
06-19 16:43:55.867 5298 5298 W /system/bin/mediaextractor:
libminijail[5298]: logging seccomp filter failures

I need vendor/etc/seccomp_policy/mediaextractor.policy file.

[Michael Goffioul]

Post by youling 257
06-19 16:43:55.866 5298 5298 W /system/bin/mediaextractor: Could not read
additional policy file '/vendor/etc/seccomp_policy/mediaextractor.policy'
libminijail[5298]: allowing syscall: socketcall
libminijail[5298]: allowing syscall: writev
libminijail[5298]: allowing syscall: fcntl64
libminijail[5298]: allowing syscall: clock_gettime
libminijail[5298]: logging seccomp filter failures
I need vendor/etc/seccomp_policy/mediaextractor.policy file.

No, you don't. The core policies are contained
in /system/etc/seccomp_policy/mediaextractor.policy.

Michael.

[youling 257]

I have problem

[ 94.752636] [drm] GPU HANG: ecode 7:0:0x85fffffc, in Binder:3237_3 [3289],
reason: Hang on rcs0, action: reset
[ 94.752660] [drm] GPU hangs can indicate a bug anywhere in the entire gfx
stack, including userspace.
[ 94.752672] [drm] Please file a _new_ bug report on bugs.freedesktop.org
against DRI -> DRM/Intel
[ 94.752682] [drm] drm/i915 developers can then reassign to the right
component if it's not a kernel issue.
[ 94.752693] [drm] The gpu crash dump is required to analyze gpu hangs, so
please always attach it.
[ 94.752704] [drm] GPU crash dump saved to /sys/class/drm/card0/error

[ 108.964507] init: Untracked pid 3481 exited with status 0
[ 108.978518] init: Service 'mediaextractor' (pid 1328) killed by signal 31
[ 108.978550] init: Sending signal 9 to service 'mediaextractor' (pid 1328)
process group...
[ 108.978790] init: Successfully killed process cgroup uid 1040 pid 1328 in
0ms
[ 108.979931] init: starting service 'mediaextractor'...

圚 2018幎6月19日星期二 UTC+8䞋午9:27:13Michael Goffioul写道

Post by Michael Goffioul

Post by youling 257
06-19 16:43:55.866 5298 5298 W /system/bin/mediaextractor: Could not read
additional policy file '/vendor/etc/seccomp_policy/mediaextractor.policy'
libminijail[5298]: allowing syscall: socketcall
libminijail[5298]: allowing syscall: writev
libminijail[5298]: allowing syscall: fcntl64
libminijail[5298]: allowing syscall: clock_gettime
libminijail[5298]: logging seccomp filter failures
I need vendor/etc/seccomp_policy/mediaextractor.policy file.

No, you don't. The core policies are contained
in /system/etc/seccomp_policy/mediaextractor.policy.
Michael.

[youling 257]

This problemhttps://blog.csdn.net/leonxu_sjtu/article/details/80576461