Re: [PATCH v1 04/10] media: mediatek: isp: Add V4L2 flow support for ImgSys driver

On Sat, 2025-05-24 at 19:49 +0800, Olivia Wen wrote:

The ImgSys driver is implemented as a series of patches, with this patch
focusing on the V4L2 flow. The MediaTek's Image System (ImgSys), also
known as ISP Pass2. ImgSys is a memory-to-memory hardware device
designed for advanced image processing tasks. It is composed of multiple
hardware components, including TRAW, DIP, PQDIP, ME, and WPE.

TRAW (Tile RAW):
- Provides multiple downscaled resizers to support multi-scale noise
  reduction.
- Supports RAW/RGB format conversion.

DIP (Digital Image Processing) and PQDIP (Picture Quality DIP):
- Supports such as temporal and spatial noise reduction (TNR/SNR),
  edge enhancement (EE), and sharpness (SHP).
- PQDIP also supports image scaling and rotation.

ME (Motion Estimation):
- Supports motion estimation between two consecutive frames.

WPE (Warping Engine):
- Handles image warping and de-warping processes.

The ImgSys driver, primarily utilized for Google Chromebook products,
uses Multi-Frame Combined Noise Reduction technology to enhance image
quality.

Below is a simplified architecture of the ImgSys driver:
User -> V4L2 Framework -> ImgSys Driver
     -> SCP (It packages the hardware settings into commands)
     -> ImgSys Driver
     -> CMDQ (The packaged commands are sent to the hardware via GCE)
     -> Hardware

You break this driver into multiple patches by these function block,
but this would make reviewer (include me) hard to review.
The better way is

1. Add imgsys driver with basic function (include v4l2 framework, scp, cmdq. The basic function may be downscale and format convert only)
2. Add DIP function
3. Add ME function
4. Add WPE function

Let the first patch as simple as possible but it should work.

Signed-off-by: Olivia Wen <redacted>
---
 drivers/media/platform/mediatek/Kconfig       |    1 +
 drivers/media/platform/mediatek/Makefile      |    1 +
 drivers/media/platform/mediatek/isp/Kconfig   |   23 +
 .../platform/mediatek/isp/isp_7x/Makefile     |    6 +
 .../mediatek/isp/isp_7x/imgsys/Makefile       |   11 +
 .../imgsys/modules/mtk_dip_v4l2_vnode.h       |  594 ++++++
 .../isp_7x/imgsys/modules/mtk_me_v4l2_vnode.h |  386 ++++
 .../imgsys/modules/mtk_pqdip_v4l2_vnode.h     |  149 ++
 .../imgsys/modules/mtk_traw_v4l2_vnode.h      |  371 ++++
 .../imgsys/modules/mtk_wpe_v4l2_vnode.h       |  317 ++++
 .../isp/isp_7x/imgsys/mtk_header_desc.h       |   28 +
 .../isp/isp_7x/imgsys/mtk_imgsys-dev.c        |  238 +++
 .../isp/isp_7x/imgsys/mtk_imgsys-dev.h        |  427 +++++
 .../isp/isp_7x/imgsys/mtk_imgsys-formats.c    |  139 ++
 .../isp/isp_7x/imgsys/mtk_imgsys-formats.h    |   72 +
 .../isp/isp_7x/imgsys/mtk_imgsys-module_ops.h |   27 +
 .../isp/isp_7x/imgsys/mtk_imgsys-of.c         |   39 +
 .../isp/isp_7x/imgsys/mtk_imgsys-of.h         |   55 +
 .../isp/isp_7x/imgsys/mtk_imgsys-sys.c        |   27 +
 .../isp/isp_7x/imgsys/mtk_imgsys-sys.h        |   50 +
 .../isp/isp_7x/imgsys/mtk_imgsys-v4l2.c       | 1684 +++++++++++++++++
 .../isp/isp_7x/imgsys/mtk_imgsys-vnode_id.h   |  100 +
 .../isp/isp_7x/imgsys/mtk_imgsys_v4l2_vnode.h |  135 ++
 23 files changed, 4880 insertions(+)
 create mode 100644 drivers/media/platform/mediatek/isp/Kconfig
 create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/Makefile
 create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/Makefile
 create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_dip_v4l2_vnode.h
 create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_me_v4l2_vnode.h
 create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_pqdip_v4l2_vnode.h
 create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_traw_v4l2_vnode.h
 create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_wpe_v4l2_vnode.h
 create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_header_desc.h
 create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-dev.c
 create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-dev.h
 create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-formats.c
 create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-formats.h
 create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-module_ops.h
 create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-of.c
 create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-of.h
 create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-sys.c
 create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-sys.h
 create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-v4l2.c
 create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-vnode_id.h
 create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys_v4l2_vnode.h

Place these files in drivers/media/platform/mediatek/imgsys.
This is the first SoC which support imgsys driver so it's not necessary to use so deep folder structure.

Regards,
CK