On Thu, Dec 2, 2021 at 1:00 PM Adam Ford [off-list ref] wrote:

On Thu, Dec 2, 2021 at 12:54 PM Tim Harvey [off-list ref] wrote:

quoted

On Thu, Dec 2, 2021 at 4:29 AM Adam Ford [off-list ref] wrote:

quoted

On Wed, Dec 1, 2021 at 10:17 PM Adam Ford [off-list ref] wrote:

quoted

The i.MX8M has two Hantro video decoders, called G1 and G2 which appear
to be related to the video decoders used on the i.MX8MQ, but because of
how the Mini handles the power domains, the VPU driver does not need to
handle all the functions, so a new compatible flag is required.

V3 is rebased from git://linuxtv.org/hverkuil/media_tree.git for-v5.17c
This branch has support for VP9.

I set cma=512M, but this may not be enough memory as some tests appeard to run out of memory

V3 of this series has several changes:

Update imx8m_vpu_hw to add missing 'reg' reference names for G2 and include references to VP9
Update device tree to remove IMX8MQ_VPU_RESET, remove some duplicate vpu clock parenting
Fix missing reg-names from vpu_g2 node.
Apply patch [1] to manage the power domains powering down.
[1] - https://lore.kernel.org/linux-arm-kernel/20211016210547.171717-1-marex@denx.de/t/ (local)

With the above, the following Fluster scores are produced:

G1:
./fluster.py run -dGStreamer-H.264-V4L2SL-Gst1.0
Ran 90/135 tests successfully               in 74.406 secs

./fluster.py run -d GStreamer-VP8-V4L2SL-Gst1.0
Ran 55/61 tests successfully               in 8.080 secs

G2:
./fluster.py run -d GStreamer-VP9-V4L2SL-Gst1.0
Ran 127/303 tests successfully               in 203.873 secs

Fluster and G-Streamer were both built from their respective git repos using their respective master/main branches.

I should note, that both interrupts appear to be triggering.

# cat /proc/interrupts |grep codec
 57:      13442          0          0          0     GICv3  39 Level
  38300000.video-codec
 58:       7815          0          0          0     GICv3  40 Level
  38310000.video-codec

Adam,

On another thread you had let me know that you also removed the reset
from the pgc_vpumix power domain which does appear to resolve the
hang:

diff --git a/arch/arm64/boot/dts/freescale/imx8mm.dtsi

b/arch/arm64/boot/dts/freescale/imx8mm.dtsi
index eb9dcd9d1a31..31710af544dc 100644

--- a/arch/arm64/boot/dts/freescale/imx8mm.dtsi
+++ b/arch/arm64/boot/dts/freescale/imx8mm.dtsi

@@ -681,7 +681,6 @@
                                                clocks = <&clk

IMX8MM_CLK_VPU_DEC_ROOT>;
                                                assigned-clocks =
<&clk IMX8MM_CLK_VPU_BUS>;
                                                assigned-clock-parents
= <&clk IMX8MM_SYS_PLL1_800M>;
-                                               resets = <&src
IMX8MQ_RESET_VPU_RESET>;
                                        };

                                        pgc_vpu_g1: power-domain@7 {

That would make such a patch have a 'Fixes commit d39d4bb15310
("arm64: dts: imx8mm: add GPC node")' but of course that vpu domain
isn't active until your series so I'm not sure if we should send this
separate or squash it with "arm64: dts: imx8mm: Enable VPU-G1 and
VPU-G2". I'm also not clear if removing the reset requires some
further discussion with Lucas.

Unless there is objection from Lucas, I'll likely make it the first
patch in the series marking it with a fixes tag so it gets backported,
then the rest of the series would be adding the bindings, update the
driver and adding the G1 and G2 nodes.

Adam,

I've also gotten decode+display working for vp8/h264 using this series
and gstreamer-1.19.3 (although I have to use software colorspace
conversion)

# source: vp8 software encode on x86
gst-launch-1.0 -v videotestsrc ! vp8enc ! rtpvp8pay ! udpsink
host=172.24.33.15 port=9001
# sink: vp8 hardware decode on imx8mm
gst-launch-1.0 udpsrc port=9001 caps = 'application/x-rtp,
media=(string)video, clock-rate=(int)90000, encoding-name=(string)VP8,
payload=(int)96, ssrc=(uint)3363940374,
timestamp-offset=(uint)3739685909, seqnum-offset=(uint)28161,
a-framerate=(string)30' ! rtpvp8depay ! v4l2slvp8dec ! videoconvert !
kmssink

# source: h264 software encode on x86
gst-launch-1.0 -v videotestsrc ! video/x-raw,width=800,height=480 !
x264enc ! rtph264pay ! udpsink host=172.24.33.15 port=9001
# sink: h264 hardware decode on imx8mm
gst-launch-1.0 udpsrc port=9001 caps = 'application/x-rtp,
media=(string)video, clock-rate=(int)90000,
encoding-name=(string)H264, packetization-mode=(string)1,
profile-level-id=(string)64001f,
sprop-parameter-sets=(string)"Z2QAH6zZQMg9sBagwCC0oAAAAwAgAAAHkeMGMsA\=\,aOvssiw\=",
payload=(int)96, ssrc=(uint)2753453329,
timestamp-offset=(uint)3593065282, seqnum-offset=(uint)12297,
a-framerate=(string)30' ! rtph264depay ! v4l2slh264dec ! videoconvert
! kmssink

# source: vp9 software encode on x86
gst-launch-1.0 -v videotestsrc ! video/x-raw,width=800,height=480 !
vp9enc ! rtpvp9pay ! udpsink host=172.24.33.15 port=9001
# sink: vp9 hardware decode on imx8mm
gst-launch-1.0 udpsrc port=9001 caps = 'application/x-rtp,
media=(string)video, clock-rate=(int)90000, encoding-name=(string)VP9,
payload=(int)96, ssrc=(uint)2246741422,
timestamp-offset=(uint)3441735424, seqnum-offset=(uint)30250,
a-framerate=(string)30' ! rtpvp9depay ! v4l2slvp9dec ! fakesink
^^^ this fails with no-negotiated

Note I have to use videoconvert because v4l2slvp8dev src is
NV12/YUY2/NV12_32L32 and from testing only BGRx appears compatible
with kmssink (even though gst-inspect kmssink says it can sink
NV12/YUY2). With the 800x480 resolution of my display the CPU overhead
of software colorspcae conversion with videoconvert only about 9%

I haven't yet gotten vp9 decode+display working yet as 'rtpvp9depay !
v4l2slvp9dec ! fakesink' does not negotiate and it might be because my
vp9enc source is on an old gstreamer 1.16.

When you post the next series please add:
Tested-By: Tim Harvey <tharvey@gateworks.com>