Hi Ming and Nicolas

On Mon, 4 Jul 2022 at 16:53, Nicolas Dufresne [off-list ref] wrote:

Le mardi 28 juin 2022 à 10:19 +0800, Ming Qian a écrit :

quoted

By setting the V4L2_BUF_FLAG_CODECCONFIG flag,
user-space should be able to hint decoder
the vb2 only contains codec config header,
but does not contain any frame data.
It's only used for parsing header, and can't be decoded.

This is copied from OMX specification. I think we we import this, we should at
least refer to the original.

quoted

Current, it's usually used by android.

Signed-off-by: Ming Qian <ming.qian@nxp.com>
---
 Documentation/userspace-api/media/v4l/buffer.rst | 9 +++++++++
 include/uapi/linux/videodev2.h                   | 2 ++
 2 files changed, 11 insertions(+)

diff --git a/Documentation/userspace-api/media/v4l/buffer.rst b/Documentation/userspace-api/media/v4l/buffer.rst
index 4638ec64db00..acdc4556f4f4 100644
--- a/Documentation/userspace-api/media/v4l/buffer.rst
+++ b/Documentation/userspace-api/media/v4l/buffer.rst

@@ -607,6 +607,15 @@ Buffer Flags
      the format. Any subsequent call to the
      :ref:`VIDIOC_DQBUF <VIDIOC_QBUF>` ioctl will not block anymore,
      but return an ``EPIPE`` error code.
+    * .. _`V4L2-BUF-FLAG-CODECCONFIG`:
+
+      - ``V4L2_BUF_FLAG_CODECCONFIG``
+      - 0x00200000
+      - This flag may be set when the buffer only contains codec config
+    header, but does not contain any frame data. Usually the codec config
+    header is merged to the next idr frame, with the flag
+    ``V4L2_BUF_FLAG_KEYFRAME``, but there is still some scenes that will
+    split the header and queue it separately.

I think the documentation is clear. Now, if a driver uses this, will existing
userland (perhaps good to check GStreamer, FFMPEG and Chromium ?) will break ?
So we need existing driver to do this when flagged to, and just copy/append when
the userland didn't opt-in that feature ?

The commit text says it is for userspace feeding data into a video
decoder, so it's a userspace choice instead of driver.

For encoders there is already V4L2_CID_MPEG_VIDEO_HEADER_MODE [1]
which allows for V4L2_MPEG_VIDEO_HEADER_MODE_SEPARATE or
V4L2_MPEG_VIDEO_HEADER_MODE_JOINED_WITH_1ST_FRAME. FFmpeg selects
_SEPARATE by default [2], whilst the default is normally
_JOINED_WITH_1ST_FRAME.

It does raise the question as to whether all decoders will support
header byte only buffers, and does there need to be a capabilities
flag to denote that it is supported.
And should encoders in V4L2_MPEG_VIDEO_HEADER_MODE_SEPARATE mode set
it on the headers only buffers?

A number of undefined elements of how this should be implemented/used :-(

  Dave

[1] https://www.kernel.org/doc/html/latest/userspace-api/media/v4l/ext-ctrls-codec.html
[2] https://github.com/FFmpeg/FFmpeg/blob/master/libavcodec/v4l2_m2m_enc.c#L196

quoted

     * .. _`V4L2-BUF-FLAG-REQUEST-FD`:

       - ``V4L2_BUF_FLAG_REQUEST_FD``

diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
index 5311ac4fde35..8708ef257710 100644
--- a/include/uapi/linux/videodev2.h
+++ b/include/uapi/linux/videodev2.h

@@ -1131,6 +1131,8 @@ static inline __u64 v4l2_timeval_to_ns(const struct timeval *tv)
 #define V4L2_BUF_FLAG_TSTAMP_SRC_SOE         0x00010000
 /* mem2mem encoder/decoder */
 #define V4L2_BUF_FLAG_LAST                   0x00100000
+/* Buffer only contains codec header */
+#define V4L2_BUF_FLAG_CODECCONFIG            0x00200000
 /* request_fd is valid */
 #define V4L2_BUF_FLAG_REQUEST_FD             0x00800000

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel