Thread (23 messages) flat view 23 messages, 2 authors, 2021-12-03

RE: [EXT] Re: [PATCH v12 00/13] amphion video decoder/encoder driver

From: Ming Qian <ming.qian@nxp.com>
Date: 2021-11-24 09:00:20
Also in: linux-devicetree, linux-media, lkml

-----Original Message-----
From: Nicolas Dufresne [mailto:nicolas@ndufresne.ca]
Sent: Wednesday, November 24, 2021 3:23 AM
To: Ming Qian <ming.qian@nxp.com>; mchehab@kernel.org;
shawnguo@kernel.org; robh+dt@kernel.org; s.hauer@pengutronix.de
Cc: hverkuil-cisco@xs4all.nl; kernel@pengutronix.de; festevam@gmail.com;
dl-linux-imx [off-list ref]; Aisheng Dong [off-list ref];
linux-media@vger.kernel.org; linux-kernel@vger.kernel.org;
devicetree@vger.kernel.org; linux-arm-kernel@lists.infradead.org
Subject: [EXT] Re: [PATCH v12 00/13] amphion video decoder/encoder driver

Caution: EXT Email

Hi Ming,

For the patchset:

Tested-By: Nicolas Dufresne <nicolas.dufresne@collabora.com>

I've validated encoding manually with GStreamer:

  gst-launch-1.0 videotestsrc num-buffers=100 ! v4l2h264enc !
video/x-h264,profile=main,level=\(string\)2 ! h264parse ! qtmux ! filesink
location=test.mov

I've also verify the that the number of frames was exactly 100, this is common
issue with V4L2 encoders. Then I have validated VP8, H.264 and H.265
decoders
using fluster [0] with this PR [1] applied. You can find full summary at the end
of this email. Markdown report didn't get generated, I will have to check with
upstream fluster if there is a regression.

$> ./fluster.py run -s -so amphion-imx8qxp-conformance.md -d
GStreamer-VP8-V4L2-Gst1.0 GStreamer-H.264-V4L2-Gst1.0
GStreamer-H.265-V4L2-Gst1.0

VP8:   Ran 59/61 tests successfully               in 131.788 secs
H.264: Ran 75/135 tests successfully               in 501.206 secs
H.265: Ran 126/147 tests successfully               in 1131.933 secs

Note that in mainline, only 1 core get fired and is kept at its lowest possible
frequency, so perhaps it may cause some of the timeout seen. The driver is
overall functional, and I would like to thank you for this extra work. Also,
note that this very first time I run Fluster over the stateful CODEC wrappers. I
will need to run this on more platforms to locate the GStreamer specific fail.

VP8 note, conformance vector vp80-03-segmentation-1425 cause a hang but it
then
recover:

[ 8264.851841] amphion-vpu-core 2d040000.vpu_core: [0] sync session
timeout
[ 8264.858634] amphion-vpu-core 2d040000.vpu_core: [0] send cmd(0x2) fail
[ 8264.867992] amphion-vpu-core 2d040000.vpu_core: [0] start fail
[ 8264.905173] amphion-vpu-core 2d040000.vpu_core: reset hang core
HI Nicolas

    There is a bug in firmware that if send a command to firmware too close after stop cmd,
The firmware may enter wfi wrong, and led to hang issue you met in vp80-03-segmentation-1425.
I'll add a workaround in driver that add a delay after send stop cmd to firmware in next version.

    Because the amphion's vpu doesn't support to output i420, so the test will convert nv12_8l128 to i420 by videoconvert, it leds to most of timeout failure.

    The FM1_BT_B.h264 can't be decoded by amphion's vpu, the vpu is keeping parse sequence header, and it led to timeout failure.

    I run the test and change the timeout to 300, then most of timeout failures are gone. Besides that, my result is almost as same as yours.

    The failures of assertion error means that the vpu's output is different from the pattern, I think it should be the vpu's limitation.
See comments about your GStreamer MR below...

[0]
https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.
com%2Ffluendo%2Ffluster&amp;data=04%7C01%7Cming.qian%40nxp.com%7
C45d33490bfe546aa8e1408d9aeb6a822%7C686ea1d3bc2b4c6fa92cd99c5c3
01635%7C0%7C0%7C637732921809373007%7CUnknown%7CTWFpbGZsb3d
8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3
D%7C3000&amp;sdata=bBqADaZ9ZN00SbCPalQE5fN740hB2bMUxHDL4QEki
mg%3D&amp;reserved=0
[1]
https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.
com%2Ffluendo%2Ffluster%2Fpull%2F98&amp;data=04%7C01%7Cming.qian
%40nxp.com%7C45d33490bfe546aa8e1408d9aeb6a822%7C686ea1d3bc2b4c
6fa92cd99c5c301635%7C0%7C0%7C637732921809382989%7CUnknown%7C
TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLC
JXVCI6Mn0%3D%7C3000&amp;sdata=90dUC%2FNzK%2BcFNSMwY7bgQ0iD8
ctBwU%2FMhyrLoBBEf4g%3D&amp;reserved=0


[...]
quoted
encoder is tested with gstreamer
decoder is tested with gstreamer, but the following patches are required:
GStreamer is now a single repo (and using main branch instead of master), I
have
migrated your 3 MRs into 2 MRs.
Thank you very much.
quoted
https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgitlab.fr
eedesktop.org%2Fgstreamer%2Fgst-plugins-base%2F-%2Fmerge_requests%2F
1252&amp;data=04%7C01%7Cming.qian%40nxp.com%7C45d33490bfe546aa
8e1408d9aeb6a822%7C686ea1d3bc2b4c6fa92cd99c5c301635%7C0%7C0%7
C637732921809382989%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAw
MDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&amp;sda
ta=G4PnqxRsnPjQtV%2FUcp4dkoz9fAGxkOv%2FzKR47RaHtu0%3D&amp;reser
ved=0
quoted
https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgitlab.fr
eedesktop.org%2Fgstreamer%2Fgst-plugins-good%2F-%2Fmerge_requests%2F
1098&amp;data=04%7C01%7Cming.qian%40nxp.com%7C45d33490bfe546aa
8e1408d9aeb6a822%7C686ea1d3bc2b4c6fa92cd99c5c301635%7C0%7C0%7
C637732921809382989%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAw
MDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&amp;sda
ta=h6nEISAaiG54NPz9Xw0MiSNDx2QxWsNWyG7FLSAaLgI%3D&amp;reserved
=0

https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgitlab.fr
eedesktop.org%2Fgstreamer%2Fgstreamer%2F-%2Fmerge_requests%2F1379&
amp;data=04%7C01%7Cming.qian%40nxp.com%7C45d33490bfe546aa8e140
8d9aeb6a822%7C686ea1d3bc2b4c6fa92cd99c5c301635%7C0%7C0%7C6377
32921809382989%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiL
CJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&amp;sdata=PFe
Db4v%2BmiksgvJ08KgncRjZEt7GdMO7gV8hV0DD0uY%3D&amp;reserved=0
quoted
https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgitlab.fr
eedesktop.org%2Fgstreamer%2Fgst-plugins-good%2F-%2Fmerge_requests%2F
1099&amp;data=04%7C01%7Cming.qian%40nxp.com%7C45d33490bfe546aa
8e1408d9aeb6a822%7C686ea1d3bc2b4c6fa92cd99c5c301635%7C0%7C0%7
C637732921809382989%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAw
MDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&amp;sda
ta=e6hSDpiP2scLN5C0f8Fb%2FoMbt8k4zC45qWPrQpG%2Fe5A%3D&amp;rese
rved=0

https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgitlab.fr
eedesktop.org%2Fgstreamer%2Fgstreamer%2F-%2Fmerge_requests%2F1381&
amp;data=04%7C01%7Cming.qian%40nxp.com%7C45d33490bfe546aa8e140
8d9aeb6a822%7C686ea1d3bc2b4c6fa92cd99c5c301635%7C0%7C0%7C6377
32921809382989%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiL
CJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&amp;sdata=xkX
WjCsgVaLb0wZgfaXDMiBchtbJBNnwtZNhKe42k5o%3D&amp;reserved=0


regards,
Nicolas

[JVT-AVC_V1] (GStreamer-H.264-V4L2-Gst1.0) FM1_BT_B                 ...
Timeout
The vpu can't decode this stream.
[JCT-VC-HEVC_V1] (GStreamer-H.265-V4L2-Gst1.0)
AMP_D_Hisilicon_3               ... Timeout
Performance issue of videoconvert
[JCT-VC-HEVC_V1] (GStreamer-H.265-V4L2-Gst1.0)
AMP_E_Hisilicon_3               ... Timeout
Performance issue of videoconvert
[JCT-VC-HEVC_V1] (GStreamer-H.265-V4L2-Gst1.0)
AMP_F_Hisilicon_3               ... Timeout
Performance issue of videoconvert
[JCT-VC-HEVC_V1] (GStreamer-H.265-V4L2-Gst1.0)
BUMPING_A_ericsson_1            ... Fail
[JCT-VC-HEVC_V1] (GStreamer-H.265-V4L2-Gst1.0)
CONFWIN_A_Sony_1                ... Fail
[JCT-VC-HEVC_V1] (GStreamer-H.265-V4L2-Gst1.0)
DELTAQP_A_BRCM_4                ... Timeout
Performance issue of videoconvert
[JCT-VC-HEVC_V1] (GStreamer-H.265-V4L2-Gst1.0)
MVDL1ZERO_A_docomo_4            ... Timeout
Performance issue of videoconvert
[JCT-VC-HEVC_V1] (GStreamer-H.265-V4L2-Gst1.0)
NoOutPrior_A_Qualcomm_1         ... Fail
[JCT-VC-HEVC_V1] (GStreamer-H.265-V4L2-Gst1.0)
NoOutPrior_B_Qualcomm_1         ... Fail
[JCT-VC-HEVC_V1] (GStreamer-H.265-V4L2-Gst1.0)
NUT_A_ericsson_5                ... Fail
[JCT-VC-HEVC_V1] (GStreamer-H.265-V4L2-Gst1.0)
PICSIZE_A_Bossen_1              ... Error
The size is 1056x8440, vpu doesn't support it
[JCT-VC-HEVC_V1] (GStreamer-H.265-V4L2-Gst1.0)
PICSIZE_B_Bossen_1              ... Error
The size is 8440x1056, vpu doesn't support it
[JCT-VC-HEVC_V1] (GStreamer-H.265-V4L2-Gst1.0)
PICSIZE_C_Bossen_1              ... Fail
[JCT-VC-HEVC_V1] (GStreamer-H.265-V4L2-Gst1.0)
PICSIZE_D_Bossen_1              ... Fail
[JCT-VC-HEVC_V1] (GStreamer-H.265-V4L2-Gst1.0)
RAP_B_Bossen_2                  ... Fail
[JCT-VC-HEVC_V1] (GStreamer-H.265-V4L2-Gst1.0)
SAODBLK_A_MainConcept_4         ... Timeout
Performance issue of videoconvert
[JCT-VC-HEVC_V1] (GStreamer-H.265-V4L2-Gst1.0)
SAODBLK_B_MainConcept_4         ... Timeout
Performance issue of videoconvert
[JCT-VC-HEVC_V1] (GStreamer-H.265-V4L2-Gst1.0)
TILES_A_Cisco_2                 ... Timeout
Performance issue of videoconvert
[JCT-VC-HEVC_V1] (GStreamer-H.265-V4L2-Gst1.0)
TILES_B_Cisco_1                 ... Timeout
Performance issue of videoconvert, but assert fail
[JCT-VC-HEVC_V1] (GStreamer-H.265-V4L2-Gst1.0)
TSUNEQBD_A_MAIN10_Technicolor_2 ... Fail
[JCT-VC-HEVC_V1] (GStreamer-H.265-V4L2-Gst1.0)
VPSSPSPPS_A_MainConcept_1       ... Error
Gstream report error:
ERROR Error from element /GstPlayBin:playbin/GstURIDecodeBin:uridecodebin0/GstDecodeBin:decodebin0/v4l2h265dec:v4l2h265dec0: Could not read from resource.
Could not read from resource.
../sys/v4l2/gstv4l2bufferpool.c(1155): gst_v4l2_buffer_pool_poll (): /GstPlayBin:playbin/GstURIDecodeBin:uridecodebin0/GstDecodeBin:decodebin0/v4l2h265dec:v4l2h265dec0:
poll error 1: Success (0) for /mnt/fluster/resources/JCT-VC-HEVC_V1/VPSSPSPPS_A_MainConcept_1/VPSSPSPPS_A_MainConcept_1.bin
 
[VP8-TEST-VECTORS] (GStreamer-VP8-V4L2-Gst1.0)
vp80-03-segmentation-1425 ... Timeout
Firmware bug, after apply driver workaround, assert fail
[VP8-TEST-VECTORS] (GStreamer-VP8-V4L2-Gst1.0)
vp80-03-segmentation-1436 ... Fail

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
Keyboard shortcuts
hback out one level
jnext message in thread
kprevious message in thread
ldrill in
Escclose help / fold thread tree
?toggle this help