On Fri, Aug 25, 2023 at 10:15 PM Hans Verkuil [off-list ref] wrote:

On 25/08/2023 15:54, Takashi Iwai wrote:

quoted

On Fri, 25 Aug 2023 05:46:43 +0200,
Shengjiu Wang wrote:

quoted

On Fri, Aug 25, 2023 at 4:21 AM Mark Brown [off-list ref] wrote:

quoted

On Thu, Aug 24, 2023 at 07:03:09PM +0200, Takashi Iwai wrote:

quoted

Shengjiu Wang wrote:

quoted

quoted

But there are several issues:
1. Need to create sound cards.  ASRC module support multi instances, then
need to create multi sound cards for each instance.

quoted

Hm, why can't it be multiple PCM instances instead?

I'm having a hard time following this one too.

quoted

quoted

2. The ASRC is an entirety but with DPCM we need to separate input port and
output port to playback substream and capture stream. Synchronous between
playback substream and capture substream is a problem.
How to start them and stop them at the same time.

quoted

This could be done by enforcing the full duplex and linking the both
PCM streams, I suppose.

quoted

quoted

So shall we make the decision that we can go to the V4L2 solution?

quoted

Honestly speaking, I don't mind much whether it's implemented in V2L4
or not -- at least for the kernel part, we can reorganize / refactor
things internally.  But, the biggest remaining question to me is
whether this user-space interface is the most suitable one.  Is it
well defined, usable and maintained for the audio applications?  Or
is it meant to be a stop-gap for a specific use case?

I'm having a really hard time summoning much enthusiasm for using v4l
here, it feels like this is heading down the same bodge route as DPCM
but directly as ABI so even harder to fix properly.  That said all the
ALSA APIs are really intended to be used in real time and this sounds
like a non real time application?  I don't fully understand what the
actual use case is here.

Thanks for your reply.

This asrc memory to memory (memory ->asrc->memory) case is a non
real time use case.

User fills the input buffer to the asrc module,  after conversion, then asrc
sends back the output buffer to user. So it is not a traditional ALSA playback
and capture case. I think it is not good to create sound card for it,  it is
not a sound card actually.

It is a specific use case,  there is no reference in current kernel.
v4l2 memory to memory is the closed implementation,  v4l2 current
support video, image, radio, tuner, touch devices, so it is not
complicated to add support for this specific audio case.

Maybe you can go through these patches first.  Because we
had implemented the "memory -> asrc ->i2s device-> codec"
use case in ALSA.  Now the "memory->asrc->memory" needs
to reuse the code in asrc driver, so the first 3 patches is for refining
the code to make it can be shared by the "memory->asrc->memory"
driver.

The main change is in the v4l2 side, A /dev/vl42-audio will be created,
user applications only use the ioctl of v4l2 framework.

Ah, now I'm slowly understanding.  So, what you want is to have an
interface to perform the batch conversion of a data stream from an
input to an output?  And ALSA PCM interface doesn't fit fully for that
purpose because the data handling is batched and it's not like a
normal PCM streaming?

Basically the whole M2M arguments are rather subtle.  Those are
implementation details that can be resolved in several different ways
in the kernel side.  But the design of the operation is the crucial
point.

Maybe we can consider implementing a similar feature in ALSA API, too.
But it's too far-stretched for now.

So, if v4l2 interface provides the requested feature (the batched
audio stream conversion), it's OK to ride on it.

The V4L2 M2M interface is simple: you open a video device and then you can
pass data to the hardware, it processes it and you get the processed data back.

The hardware just processes the data as fast as it can. Each time you open
the video device a new instance is created, and each instance can pass jobs
to the hardware.

Currently it is used for video scalers, deinterlacers, colorspace converters and
codecs, but in the end it is just data in, data out with some job scheduling (fifo)
towards the hardware. So supporting audio using the same core m2m framework wouldn't
be a big deal. We'd probably make a /dev/v4l-audio device for that.

It doesn't come for free: it is a new API, so besides adding support for it, it
also needs to be documented, we would need compliance tests, and very likely I
would want a new virtual driver for this (vim2m.c would be a good template).

Thanks all.

I will try to pass the compliance test.  Should the virtual driver be added now?

Best regards
Wang Shengiu