[Bug 753109] New: omxh264enc cannot stream to HLS: black screen in Safari and iOS

["GStreamer" (GNOME Bugzilla)]

https://bugzilla.gnome.org/show_bug.cgi?id=753109

Bug ID: 753109
Summary: omxh264enc cannot stream to HLS: black screen in
Safari and iOS
Classification: Platform
Product: GStreamer
Version: 1.4.5
OS: Linux
Status: NEW
Severity: normal
Priority: Normal
Component: gst-omx
Assignee: gstreamer-***@lists.freedesktop.org
Reporter: ***@sharp.fm
QA Contact: gstreamer-***@lists.freedesktop.org
GNOME version: ---

Video encoded with the omxh264enc codec results in a black screen on Safari and
iOS. No warnings or errors appear.

The video plays on VLC, but seems to "freeze" periodically. Again, no warnings
or errors appear.

Other bugs suggest that omxh264enc should be capable of producing streaming
video, but I cannot find a combination of gstreamer versions that work at all.

The Apple mediastreamvalidator tool passes the stream as "OK" with no warnings
or errors. A separate bug has been raised with Apple to fix
mediastreamvalidator and stop it passing broken streams.

The Adobe HLS validator claims SPS/PPS headers are missing and this is why the
stream won't play.

Getting the omxh264enc encoder up and running on the debugger shows that
OMX_IndexParamBrcmVideoAVCInlineHeaderEnable and
OMX_IndexConfigVideoAVCIntraPeriod are being set, but neither of these options
are documented nor do they have any measurable effect.

The RPi has the required mpeg2 license.

The command line used is this:

gst-launch-1.0 -v udpsrc port=5004 multicast-group=239.255.1.30
caps="application/x-rtp,media=(string)video,clock-rate=(int)90000" ! rtpbin !
rtpmp2tdepay ! tsdemux name=demux ! mpegvideoparse ! omxmpeg2videodec ! queue !
videoconvert ! deinterlace ! videoscale ! video/x-raw,width=960,height=540 !
omxh264enc target-bitrate=5000000 control-rate=variable inline-header=true
periodicty-idr=1 interval-intraframes=1 ! queue ! mpegtsmux name=mux ! hlssink
location=/var/www/stream/segment2%05d.ts
playlist-location=/var/www/stream/output2.m3u8
playlist-root=http://192.168.225.3/stream/ target-duration=2

["GStreamer" (GNOME Bugzilla)]

https://bugzilla.gnome.org/show_bug.cgi?id=753109

Tim-Philipp Müller <***@zen.co.uk> changed:

What |Removed |Added
----------------------------------------------------------------------------
Status|NEW |NEEDINFO
CC| |***@zen.co.uk

--- Comment #1 from Tim-Philipp Müller <***@zen.co.uk> ---
The stream is not necessarily broken, it's just that Apple is very particular
in what it handles. For example, Apple requires AU delimiters when H264 is put
in MPEG-TS. Check if the encoder is creating those NALS, it might not be doing
that by default. If you attach the first 500kB or so of such a stream, we can
check (without firing up an rpi ourselves).

["GStreamer" (GNOME Bugzilla)]

https://bugzilla.gnome.org/show_bug.cgi?id=753109

--- Comment #2 from ***@sharp.fm ---
Created attachment 308582
--> https://bugzilla.gnome.org/attachment.cgi?id=308582&action=edit
Test stream

I've been digging to find a canonical solution to HLS streaming and it seems
I'm pioneering this. The inconsistencies in mediastreamvalidator make this very
very difficult so I'm doing this in the dark. I've attached the first 500k of a
ts as requested.

["GStreamer" (GNOME Bugzilla)]

https://bugzilla.gnome.org/show_bug.cgi?id=753109

--- Comment #3 from ***@sharp.fm ---
I have just found https://bugzilla.gnome.org/show_bug.cgi?id=736211 which gave
a possible patch to provide AU delimiters, however this idea was rejected in
favour of adding this to h264parse.

Unfortunately the addition of h264parse breaks hlssink - instead of creating ts
fragments (and an m3u8 file) one ts file is created without bound.

["GStreamer" (GNOME Bugzilla)]

https://bugzilla.gnome.org/show_bug.cgi?id=753109

Randeep <***@tataelxsi.co.in> changed:

What |Removed |Added
----------------------------------------------------------------------------
CC| |***@tataelxsi.co.in

--- Comment #4 from Randeep <***@tataelxsi.co.in> ---
Hi,

I need to use 'omxh264enc' gstreamer plugin to encode a video stream. When I
gst-inspect, it is showing the plugin details correctly (libgstomx.so), but
when i'm running it using a gstreamer pipeline i'm getting error as 'Could not
initialize supporting library' and additional debug info as:
videoencoder.c (1428): gst_video_encoder_change_state (): Failed to open
encoder.

Is there any dependent library that we need to provide for this plugin to
work ? Can anyone please help me with this issue.


gst-launch-1.0 -v videotestsrc do-timestamp=true num-buffers=300 ! \
video/x-raw,width=720,height=480,format=NV12,framerate=30/1 ! \
videoconvert ! video/x-raw,format=NV12 ! \
omxh264enc target-bitrate=3000000 ! \
video/x-h264,profile=\(string\)baseline,level=\(string\)3.1 ! \
h264parse ! queue ! \
avimux name=mux ! filesink location=./testalsavideo2.avi \
alsasrc do-timestamp=true num-buffers=468 blocksize=4096 ! \
audio/x-raw,format=S16LE,rate=48000,channels=2 ! \
queue ! mux.audio_0

["GStreamer" (GNOME Bugzilla)]

https://bugzilla.gnome.org/show_bug.cgi?id=753109

Robert Mast <***@zonnet.nl> changed:

What |Removed |Added
----------------------------------------------------------------------------
CC| |***@zonnet.nl

--- Comment #5 from Robert Mast <***@zonnet.nl> ---
On my RPi3 this does not give any output in mystream.mp4:

gst-launch-1.0 souphttpsrc
location="http://127.0.0.1:9981/stream/channel/41ceb7b6ef96f869d7d6c78b6b3a27a6?ticket=24A946694272F14A9994F5F2280E466AB495F745?profile=pass"
! tsdemux name=demux demux. ! mpegvideoparse ! omxmpeg2videodec ! videoconvert
! omxh264enc target-bitrate=2097152 control-rate=variable !
video/x-h264,stream-format=byte-stream,profile=high,width=720,height=576,framerate=25/1
! h264parse ! filesink
location="/media/6ee0734e-108c-4f31-b19a-ec7a66c5ce09_/mystream.mp4"

This one, only missing h264parse, does
gst-launch-1.0 souphttpsrc
location="http://127.0.0.1:9981/stream/channel/41ceb7b6ef96f869d7d6c78b6b3a27a6?ticket=24A946694272F14A9994F5F2280E466AB495F745?profile=pass"
! tsdemux name=demux demux. ! mpegvideoparse ! omxmpeg2videodec ! videoconvert
! omxh264enc target-bitrate=2097152 control-rate=variable !
video/x-h264,stream-format=byte-stream,profile=high,width=720,height=576,framerate=25/1
! filesink location="/media/6ee0734e-108c-4f31-b19a-ec7a66c5ce09_/mystream.mp4"

So, whatever is added to h264parse, it removes all h264 content without any
error.

Versions 1.10.3, except for libvpx 1.5.0
patched withattachments of bugs 767801 and 775962 to get the pipeline running
at all

["GStreamer" (GNOME Bugzilla)]

https://bugzilla.gnome.org/show_bug.cgi?id=753109

Vivia Nikolaidou <***@ahiru.eu> changed:

What |Removed |Added
----------------------------------------------------------------------------
CC| |***@ahiru.eu

--- Comment #6 from Vivia Nikolaidou <***@ahiru.eu> ---
Is this still an issue with latest GStreamer?

["GStreamer" (GNOME Bugzilla)]

https://bugzilla.gnome.org/show_bug.cgi?id=753109

GStreamer system administrator <***@gstreamer.net> changed:

What |Removed |Added
----------------------------------------------------------------------------
Status|NEEDINFO |RESOLVED
Resolution|--- |OBSOLETE

--- Comment #7 from GStreamer system administrator <***@gstreamer.net> ---
-- GitLab Migration Automatic Message --

This bug has been migrated to freedesktop.org's GitLab instance and has been
closed from further activity.

You can subscribe and participate further through the new bug through this link
to our GitLab instance:
https://gitlab.freedesktop.org/gstreamer/gst-omx/issues/7.