On Mon, Dec 6, 2021 at 4:30 PM Adam Ford [off-list ref] wrote:

On Mon, Dec 6, 2021 at 2:51 PM Nicolas Dufresne [off-list ref] wrote:

quoted

Le dimanche 05 décembre 2021 à 13:51 -0600, Adam Ford a écrit :

quoted

On Sun, Dec 5, 2021 at 12:16 PM Adam Ford [off-list ref] wrote:

quoted

Currently, the VPU in the i.MQ8MQ is appearing as one codec, but in reality, it's
two IP blocks called G1 and G2.  There is initialization code in VPU code to
pull some clocks, resets and other features which has been integrated into
the vpu-blk-ctrl for the i.MX8M Mini and a similar method can be used to make
the VPU codec's operate as stand-alone cores without having to know the details
of each other or the quirks unique to the i.MX8MQ, so the remaining code can be
left more generic.

This series was started by Lucas Stach, and picked up by me so some patches have
his s-o-b and mine where I might have changed a few minor items.  It's in an RFC state
because the G2 VP9 operations appear to hang, but the parent code from which I started doesn't
appear to show VP9 support, and it looks like it should.

since the g-streamer and media trees are in a constant state of change, this series is based on
git://linuxtv.org/hverkuil/media_tree.git for-v5.17e

I forgot to post Fluster results.

Before the patches to this branch:

2gst-main] root@localhost:~/gstreamer/fluster# ./fluster.py list -c |grep -i v4l

    GStreamer-H.264-V4L2-Gst1.0: GStreamer H.264 V4L2 decoder for
GStreamer 1.0... ❌
    GStreamer-H.264-V4L2SL-Gst1.0: GStreamer H.264 V4L2SL decoder for
GStreamer 1.0... ✔️
    GStreamer-H.265-V4L2-Gst1.0: GStreamer H.265 V4L2 decoder for
GStreamer 1.0... ❌
    GStreamer-H.265-V4L2SL-Gst1.0: GStreamer H.265 V4L2SL decoder for
GStreamer 1.0... ❌
    GStreamer-VP8-V4L2-Gst1.0: GStreamer VP8 V4L2 decoder for GStreamer 1.0... ❌
    GStreamer-VP8-V4L2SL-Gst1.0: GStreamer VP8 V4L2SL decoder for
GStreamer 1.0... ✔️
    GStreamer-VP9-V4L2SL-Gst1.0: GStreamer VP9 V4L2SL decoder for
GStreamer 1.0... ❌
    GStreamer-AV1-V4L2SL-Gst1.0: GStreamer AV1 V4L2SL decoder for
GStreamer 1.0... ❌
[gst-main] root@localhost:~/gstreamer/fluster#

There was no VP9 support.

./fluster.py run -dGStreamer-H.264-V4L2SL-Gst1.0
GStreamer-H.264-V4L2SL-Gst1.0
Ran 90/135 tests successfully               in 58.787 secs

AFTER this series, two decoders appear:

[   15.919137] hantro-vpu 38300000.video-codec: registered
nxp,imx8mq-vpu-g1-dec as /dev/video0
[   15.983579] hantro-vpu 38310000.video-codec: registered
nxp,imx8mq-vpu-g2-dec as /dev/video1

VP9 was listed:

    GStreamer-H.264-V4L2-Gst1.0: GStreamer H.264 V4L2 decoder for
GStreamer 1.0... ❌
    GStreamer-H.264-V4L2SL-Gst1.0: GStreamer H.264 V4L2SL decoder for
GStreamer 1.0... ✔️
    GStreamer-H.265-V4L2-Gst1.0: GStreamer H.265 V4L2 decoder for
GStreamer 1.0... ❌
    GStreamer-H.265-V4L2SL-Gst1.0: GStreamer H.265 V4L2SL decoder for
GStreamer 1.0... ❌
    GStreamer-VP8-V4L2-Gst1.0: GStreamer VP8 V4L2 decoder for GStreamer 1.0... ❌
    GStreamer-VP8-V4L2SL-Gst1.0: GStreamer VP8 V4L2SL decoder for
GStreamer 1.0... ✔️
    GStreamer-VP9-V4L2SL-Gst1.0: GStreamer VP9 V4L2SL decoder for
GStreamer 1.0... ✔️
    GStreamer-AV1-V4L2SL-Gst1.0: GStreamer AV1 V4L2SL decoder for
GStreamer 1.0... ❌

./fluster.py run -d GStreamer-VP8-V4L2SL-Gst1.0
Ran 55/61 tests successfully               in 8.565 secs


./fluster.py run -dGStreamer-H.264-V4L2SL-Gst1.0
Ran 90/135 tests successfully               in 60.269 secs

Same results for H.264

VP9 Hangs, where it didn't even appear as available before:

./fluster.py run -d GStreamer-VP9-V4L2SL-Gst1.0
****************************************************************************************************
Running test suite VP9-TEST-VECTORS with decoder GStreamer-VP9-V4L2SL-Gst1.0
Using 4 parallel job(s)
****************************************************************************************************

[TEST SUITE      ] (DECODER                    ) TEST VECTOR
                          ... RESULT
----------------------------------------------------------------------
[VP9-TEST-VECTORS] (GStreamer-VP9-V4L2SL-Gst1.0)
vp90-2-00-quantizer-00.webm                     ... Success
[VP9-TEST-VECTORS] (GStreamer-VP9-V4L2SL-Gst1.0)
vp90-2-00-quantizer-01.webm                     ... Success
[VP9-TEST-VECTORS] (GStreamer-VP9-V4L2SL-Gst1.0)
vp90-2-00-quantizer-02.webm                     ... Succes

<hang > - and yes, 'Success' didnt' finish writing to the serial port.

Looks like hope to me ! Do you get further with -j 1 (one concurrent decode) ?

Anything after the first attempt appears to fail.  I haven't figured
out if it's the G2 code or the power domain stuff.  Since VP9 didn't
work I migrated it to use the vpu-blk-ctrl, I can't really compare to
know if it was a regression or an improvement.

:-)

I managed to get the vpu-blk-ctrl updated to address the G2 hanging.

./fluster.py run -d GStreamer-VP9-V4L2SL-Gst1.0
Ran 144/303 tests successfully               in 224.847 secs

Since I couldn't get it to work before the split, I am not sure if
this is good or not.  I also wasn't sure if I needed to allocate more
CMA for this.

If/when people can give me some feedback on the RFC's, I'll submit an
updated formal patch for both the i.MQ8MQ and the i.MQ8M Mini as one
series based on  "for-v5.17e" from
git://linuxtv.org/hverkuil/media_tree.git

I wasn't sure if I needed to break up the migration from a combined
VPU to split G1 and G2.  I might need some help/suggestions on the
YAML, because I struggle with that part a little, but I'd like the 8MM
to use the 8MQ dt-binding yaml file since they are so similar.

I hope to have some more time this week because my weekends are
completely booked for the rest of the year, and leave for London, on
23 Dec 2021 and I won't return until 5 Jan 2022.

adam

quoted

quoted

quoted

Adam Ford (2):
  media: hantro: split i.MX8MQ G1 and G2 code
  arm64: dts: imx8mq: Split i.MX8MQ G1 and G2 with vpu-blk-ctrl

Lucas Stach (3):
  dt-bindings: power: imx8mq: add defines for VPU blk-ctrl domains
  dt-bindings: soc: add binding for i.MX8MQ VPU blk-ctrl
  soc: imx: imx8m-blk-ctrl: add i.MX8MQ VPU blk-ctrl

 .../soc/imx/fsl,imx8mq-vpu-blk-ctrl.yaml      |  71 +++++++++++
 arch/arm64/boot/dts/freescale/imx8mq.dtsi     |  69 ++++++----
 drivers/soc/imx/imx8m-blk-ctrl.c              |  67 ++++++++++
 drivers/staging/media/hantro/hantro_drv.c     |   4 +-
 drivers/staging/media/hantro/hantro_hw.h      |   2 +-
 drivers/staging/media/hantro/imx8m_vpu_hw.c   | 119 +++---------------
 include/dt-bindings/power/imx8mq-power.h      |   3 +
 7 files changed, 205 insertions(+), 130 deletions(-)
 create mode 100644 Documentation/devicetree/bindings/soc/imx/fsl,imx8mq-vpu-blk-ctrl.yaml

--
2.32.0

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel