On Tue, Nov 30, 2021 at 1:28 PM Tim Harvey [off-list ref] wrote:

On Tue, Nov 30, 2021 at 6:00 AM Ezequiel Garcia
[off-list ref] wrote:

quoted

Hi Tim,

On Mon, 29 Nov 2021 at 16:36, Tim Harvey [off-list ref] wrote:

quoted

On Mon, Nov 29, 2021 at 10:59 AM Adam Ford [off-list ref] wrote:

..

quoted

quoted

Adam,

What deps did you install in order to get v4l2codecs building? I
installed libgudev-1.0-dev based on Nicolas' suggestion and rebuilt
(not sure if I needed to re-configure somehow) but there is still
nothing in build/subprojects/gst-plugins-bad/sys/v4l2codecs/. A 'meson
configure' tells me that v4l2codecs is set to 'auto' but I'm not sure
how to find out what dependencies are needed or what may be missing.

At least in my case (Centps-derivative), this is what I've done:

...
gst-plugins-bad| Run-time dependency gudev-1.0 found: NO (tried
pkgconfig and cmake)

Installed gudev ... and then:

...
gst-plugins-bad| Dependency gudev-1.0 found: YES 232 (cached)
...
gst-plugins-bad 1.19.3.1

    Plugins               : accurip, adpcmdec, adpcmenc, aiff, asfmux,
audiobuffersplit, audiofxbad, audiomixmatrix, audiolatency,
audiovisualizers, autoconvert, bayer,
                            camerabin, codecalpha, coloreffects,
debugutilsbad, dvbsubenc, dvbsuboverlay, dvdspu, faceoverlay,
festival, fieldanalysis, freeverb, frei0r,
                            gaudieffects, gdp, geometrictransform,
id3tag, inter, interlace, ivfparse, ivtc, jp2kdecimator, jpegformat,
rfbsrc, midi, mpegpsdemux,
                            mpegpsmux, mpegtsdemux, mpegtsmux, mxf,
netsim, rtponvif, pcapparse, pnm, proxy, legacyrawparse,
removesilence, rist, rtmp2, rtpmanagerbad,
                            sdpelem, segmentclip, siren, smooth,
speed, subenc, switchbin, timecode, transcode, videofiltersbad,
videoframe_audiolevel, videoparsersbad,
                            videosignal, vmnc, y4mdec, decklink, dvb,
fbdevsink, ipcpipeline, nvcodec, shm, v4l2codecs, hls, sctp

GStreamer current master build fails. It's a known issue which will be
fixed today:

[...]
[8/9] Compiling C object
subprojects/gst-plugins-bad/sys/v4l2codecs/libgstv4l2codecs.so.p/gstv4l2codecvp9dec.c.o
FAILED: subprojects/gst-plugins-bad/sys/v4l2codecs/libgstv4l2codecs.so.p/gstv4l2codecvp9dec.c.o
cc -Isubprojects/gst-plugins-bad/sys/v4l2codecs/libgstv4l2codecs.so.p
-Isubprojects/gst-plugins-bad/sys/v4l2codecs
-I../subprojects/gst-plugins-bad/sys/v4l2codecs
-Isubprojects/gst-plugins-bad -I../subprojects/gst-plugins-bad
-Isubprojects/gstreamer/libs -I../subprojects/gstreamer/libs
-Isubprojects/gstreamer -I../subprojects/gstreamer
-Isubprojects/gst-plugins-bad/gst-libs
-I../subprojects/gst-plugins-bad/gst-libs
-Isubprojects/gst-plugins-base/gst-libs
-I../subprojects/gst-plugins-base/gst-libs -Isubprojects/orc
-I../subprojects/orc -Isubprojects/gstreamer/gst
-Isubprojects/gst-plugins-base/gst-libs/gst/video
-Isubprojects/gst-plugins-base/gst-libs/gst/pbutils
-Isubprojects/gst-plugins-base/gst-libs/gst/audio
-Isubprojects/gst-plugins-base/gst-libs/gst/tag
-I/usr/include/glib-2.0 -I/usr/lib64/glib-2.0/include
-I/usr/include/gudev-1.0 -fdiagnostics-color=always
-D_FILE_OFFSET_BITS=64 -Wall -Winvalid-pch -O2 -g -fvisibility=hidden
-fno-strict-aliasing -DG_DISABLE_DEPRECATED -Wmissing-prototypes
-Wdeclaration-after-statement -Wold-style-definition
-Wmissing-declarations -Wredundant-decls -Wwrite-strings -Wformat
-Wformat-security -Winit-self -Wmissing-include-dirs -Waddress
-Wno-multichar -Wvla -Wpointer-arith -fPIC -pthread -DHAVE_CONFIG_H
-MD -MQ subprojects/gst-plugins-bad/sys/v4l2codecs/libgstv4l2codecs.so.p/gstv4l2codecvp9dec.c.o
-MF subprojects/gst-plugins-bad/sys/v4l2codecs/libgstv4l2codecs.so.p/gstv4l2codecvp9dec.c.o.d
-o subprojects/gst-plugins-bad/sys/v4l2codecs/libgstv4l2codecs.so.p/gstv4l2codecvp9dec.c.o
-c ../subprojects/gst-plugins-bad/sys/v4l2codecs/gstv4l2codecvp9dec.c
../subprojects/gst-plugins-bad/sys/v4l2codecs/gstv4l2codecvp9dec.c:92:3:
error: unknown type name ‘grefcount’
   grefcount ref_count;
   ^~~~~~~~~
../subprojects/gst-plugins-bad/sys/v4l2codecs/gstv4l2codecvp9dec.c: In
function ‘gst_v4l2_codec_vp9_dec_picture_data_new’:
../subprojects/gst-plugins-bad/sys/v4l2codecs/gstv4l2codecvp9dec.c:106:3:
warning: implicit declaration of function ‘g_ref_count_init’; did you
mean ‘g_cond_init’? [-Wimplicit-function-declaration]
   g_ref_count_init (&pic_data->ref_count);
   ^~~~~~~~~~~~~~~~
   g_cond_init
../subprojects/gst-plugins-bad/sys/v4l2codecs/gstv4l2codecvp9dec.c: In
function ‘gst_v4l2_codec_vp9_dec_picture_data_ref’:
../subprojects/gst-plugins-bad/sys/v4l2codecs/gstv4l2codecvp9dec.c:118:3:
warning: implicit declaration of function ‘g_ref_count_inc’; did you
mean ‘g_strv_contains’? [-Wimplicit-function-declaration]
   g_ref_count_inc (&data->ref_count);
   ^~~~~~~~~~~~~~~
   g_strv_contains
../subprojects/gst-plugins-bad/sys/v4l2codecs/gstv4l2codecvp9dec.c: In
function ‘gst_v4l2_codec_vp9_dec_picture_data_unref’:
../subprojects/gst-plugins-bad/sys/v4l2codecs/gstv4l2codecvp9dec.c:125:7:
warning: implicit declaration of function ‘g_ref_count_dec’
[-Wimplicit-function-declaration]
   if (g_ref_count_dec (&data->ref_count)) {
       ^~~~~~~~~~~~~~~
ninja: build stopped: subcommand failed.

Hope this helps get you started!
Ezequiel

Ezequiel and Nicolas,

Thanks - I did manage to get gstreamer 1.19.3 built successfully with
v4l2codecs finally by getting the correct dependencies. I've attempted
to software encode from another system and decode/display on the IMX8M
Mini but thus far have not been successful.

I will post a V2 last today with the Mini's post-processing removed.
Someone, I apologize that I forget who, mentioned it was fused out of
the Mini, so the testing I've been doing was with that removed and I
removed the H1 encoder since the Mini doesn't support JPEG encoding.

I don't understand the YAML very well, and I'm struggling with the
dt-bindings which is the main reason I havne't submitted a formal
patch yet.

To date, I have only tested with v4l2-compliance and with the fluster
app that was recommended.

I see that v4l2codecs plugin v4l2slh264dec/v4l2slmpeg2dec/v4l2slvp8dec
and these all can output video/x-raw NV12/YUY2 which kmssink should
accept so I'm attempting the following :

# vp8 encode from x86
gst-launch-1.0 -v videotestsrc ! video/x-raw,width=800,height=480 !
vp8enc ! rtpvp8pay ! udpsink host=172.24.33.15 port=9001
# vp8 decode on imx8mm@172.24.33.15 which has a 800x480 display
[gst-main] root@focal-venice:~/gstreamer/build# gst-launch-1.0 -v
udpsrc port=9001 caps = "application/x-rtp, media=(string)video,
clock-rate=(int)90000, encoding-name=(string)VP8, payload=(int)96,
ssrc=(uint)2745262155, timestamp-offset=(uint)2515032683,
seqnum-offset=(uint)19579, a-framerate=(string)30" ! rtpvp8depay !
v4l2slvp8dec ! kmssink
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
/GstPipeline:pipeline0/GstKMSSink:kmssink0: display-width = 800
/GstPipeline:pipeline0/GstKMSSink:kmssink0: display-height = 480
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
/GstPipeline:pipeline0/GstUDPSrc:udpsrc0.GstPad:src: caps =
application/x-rtp, media=(string)video, clock-rate=(int)90000,
encoding-name=(string)VP8, payload=(int)96, ssrc=(uint)2745262155,
timestamp-offset=(uint)2515032683, seqnum-offset=(uint)19579,
a-framerate=(string)30
New clock: GstSystemClock
/GstPipeline:pipeline0/GstRtpVP8Depay:rtpvp8depay0.GstPad:sink: caps =
application/x-rtp, media=(string)video, clock-rate=(int)90000,
encoding-name=(string)VP8, payload=(int)96, ssrc=(uint)2745262155,
timestamp-offset=(uint)2515032683, seqnum-offset=(uint)19579,
a-framerate=(string)30
/GstPipeline:pipeline0/GstRtpVP8Depay:rtpvp8depay0.GstPad:src: caps =
video/x-vp8, framerate=(fraction)0/1, height=(int)480, width=(int)800,
profile=(string)0
ERROR: from element /GstPipeline:pipeline0/GstUDPSrc:udpsrc0: Internal
data stream error.
Additional debug info:
../subprojects/gstreamer/libs/gst/base/gstbasesrc.c(3127):
gst_base_src_loop (): /GstPipeline:pipeline0/GstUDPSrc:udpsrc0:
streaming stopped, reason not-negotiated (-4)
Execution ended after 0:00:02.076839644
Setting pipeline to NULL ...
Freeing pipeline ...

I'm getting the same thing when trying to use h264.

I've never quite been able to grasp how to debug GStreamer's
negotiation issues. If I end with fakesink it appears to decode so it
must be the v4l2slvp8dec to kmssink. I tried forcing the pixel format
using 'v4l2slvp8dec ! "video/x-raw,format=(string)NV12" ! kmssink' but
I still get the negotiation error.

What interrupts should I be seeing in /proc/interrupts? I don't see
anything vpu/hantro related there.

I don't have my Mini with me right now, but with my current patch set
that I mentioned above, I am able to see the interrupts for the
video-codec@38300000 increase each time I run a test with fluster.

I also want to make sure I have a basic understanding of the vpu
drivers and usersapce on the IMX8M Mini. The IMX6Q/DL that I'm more
familiar with has a vpu that is supported by the GStreamer video4linux
plugin which shows the following (on GStreamer 1.16.2):
  v4l2jpegenc: V4L2 JPEG Encoder
  v4l2jpegdec: V4L2 JPEG Decoder
  v4l2h264enc: V4L2 H.264 Encoder
  v4l2mpeg4enc: V4L2 MPEG4 Encoder
  v4l2mpeg4dec: V4L2 MPEG4 Decoder
  v4l2mpeg2dec: V4L2 MPEG2 Decoder
  v4l2h264dec: V4L2 H264 Decoder
The IMX6Q/DL also has an IPU that has an M2M driver that provides the
following for scaling/colorspace conversion:
  v4l2convert: V4L2 Video Converter

I believe what I'm reading is that the IMX8M Mini Hantro codecs are
'stateful' where more software is required to drive them and is
supported by the newer v4l2codecs plugin. I haven't been able to
understand what kernel version/requirements the v4l2codecs plugin
users/requires.

I'm also trying to understand how we can get scaling/colorspace
conversion on the IMX8M Mini. The IMX8M lacks an IPU... is there some
way to utilize scaling/colorspace conversion from the 2D GPU bound to
the etnaviv driver?

Best regards,

Tim