From: Shengjiu Wang <hidden> Date: 2024-08-06 10:47:41
This function is base on the accelerator implementation
for compress API:
https://patchwork.kernel.org/project/alsa-devel/patch/20240731083843.59911-1-perex@perex.cz/
Audio signal processing also has the requirement for memory to
memory similar as Video.
This asrc memory to memory (memory ->asrc->memory) case is a non
real time use case.
User fills the input buffer to the asrc module, after conversion, then asrc
sends back the output buffer to user. So it is not a traditional ALSA playback
and capture case.
Because we had implemented the "memory -> asrc ->i2s device-> codec"
use case in ALSA. Now the "memory->asrc->memory" needs
to reuse the code in asrc driver, so the patch 1 and patch 2 is for refining
the code to make it can be shared by the "memory->asrc->memory"
driver.
Other change is to add memory to memory support for two kinds of i.MX ASRC
modules.
Shengjiu Wang (6):
ALSA: compress: add Sample Rate Converter codec support
ASoC: fsl_asrc: define functions for memory to memory usage
ASoC: fsl_easrc: define functions for memory to memory usage
ASoC: fsl_asrc_m2m: Add memory to memory function
ASoC: fsl_asrc: register m2m platform device
ASoC: fsl_easrc: register m2m platform device
include/uapi/sound/compress_offload.h | 2 +
include/uapi/sound/compress_params.h | 9 +-
sound/soc/fsl/Kconfig | 1 +
sound/soc/fsl/Makefile | 2 +-
sound/soc/fsl/fsl_asrc.c | 176 +++++-
sound/soc/fsl/fsl_asrc.h | 2 +
sound/soc/fsl/fsl_asrc_common.h | 68 +++
sound/soc/fsl/fsl_asrc_m2m.c | 769 ++++++++++++++++++++++++++
sound/soc/fsl/fsl_easrc.c | 259 ++++++++-
sound/soc/fsl/fsl_easrc.h | 4 +
10 files changed, 1284 insertions(+), 8 deletions(-)
create mode 100644 sound/soc/fsl/fsl_asrc_m2m.c
--
2.34.1
this sounds wrong to me. The sample rate converter is not an "encoder",
and the properties for padding/delay are totally specific to an encoder
function.
The other point is that I am not sure how standard this ratio_mod
parameter is. This could be totally specific to a specific
implementation, and another ASRC might have different parameters.
Again I am not sure how standard those parameters are, and even if they
were if their representation is reusable.
And the compressed API has a good split between encoders and decoders, I
am not sure how an SRC can be classified as either.
quoted hunk
union snd_codec_options {
struct snd_enc_wma wma;
struct snd_enc_vorbis vorbis;
From: Jeff Brower <hidden> Date: 2024-08-06 11:45:47
All-
"The sample rate converter is not an encoder ..."
Indeed, an encoder creates a compressed bitstream from audio data
(typically linear PCM samples), normally for transmission of some
type. A sample rate converter generates audio data from audio data,
and is normally applied prior to an encoder because it can only accept
a limited range of sample rates.
-Jeff
Quoting Pierre-Louis Bossart [off-list ref]:
On 8/6/24 12:26, Shengjiu Wang wrote:
quoted
Add Sample Rate Converter(SRC) codec support, define the output
format and rate for SRC.
Signed-off-by: Shengjiu Wang <redacted>
---
include/uapi/sound/compress_offload.h | 2 ++
include/uapi/sound/compress_params.h | 9 ++++++++-
2 files changed, 10 insertions(+), 1 deletion(-)
encoder at the
* beginning of the track
+ * @SNDRV_COMPRESS_SRC_RATIO_MOD: Resampling Ratio Modifier for
sample rate converter
*/
enum sndrv_compress_encoder {
SNDRV_COMPRESS_ENCODER_PADDING = 1,
SNDRV_COMPRESS_ENCODER_DELAY = 2,
+ SNDRV_COMPRESS_SRC_RATIO_MOD = 3,
};
this sounds wrong to me. The sample rate converter is not an "encoder",
and the properties for padding/delay are totally specific to an encoder
function.
The other point is that I am not sure how standard this ratio_mod
parameter is. This could be totally specific to a specific
implementation, and another ASRC might have different parameters.
quoted
/**
diff --git a/include/uapi/sound/compress_params.h
b/include/uapi/sound/compress_params.h
index ddc77322d571..0843773ea6b4 100644
Again I am not sure how standard those parameters are, and even if they
were if their representation is reusable.
And the compressed API has a good split between encoders and decoders, I
am not sure how an SRC can be classified as either.
quoted
union snd_codec_options {
struct snd_enc_wma wma;
struct snd_enc_vorbis vorbis;
this sounds wrong to me. The sample rate converter is not an "encoder",
and the properties for padding/delay are totally specific to an encoder
function.
There is only decoder and encoder definition for compress, I know
it is difficult to add SRC to encoder or decoder classification,
SRC is a Post Processing. I hope you can have a recommandation.
Thanks.
Best regards
Shengjiu Wang
The other point is that I am not sure how standard this ratio_mod
parameter is. This could be totally specific to a specific
implementation, and another ASRC might have different parameters.
Again I am not sure how standard those parameters are, and even if they
were if their representation is reusable.
And the compressed API has a good split between encoders and decoders, I
am not sure how an SRC can be classified as either.
quoted
union snd_codec_options {
struct snd_enc_wma wma;
struct snd_enc_vorbis vorbis;
this sounds wrong to me. The sample rate converter is not an "encoder",
and the properties for padding/delay are totally specific to an encoder
function.
There is only decoder and encoder definition for compress, I know
it is difficult to add SRC to encoder or decoder classification,
SRC is a Post Processing. I hope you can have a recommandation
I don't. I think we're blurring layers in a really odd way.
The main reason why the compress API was added is to remove the
byte-to-time conversions. But that's clearly not useful for a
post-processing of PCM data, where the bitrate is constant. It really
feels like we're adding this memory-to-memory API to the compress
framework because we don't have anything else available, not because it
makes sense to do so.
Then there's the issue of parameters, we chose to only add parameters
for standard encoders/decoders. Post-processing is highly specific and
the parameter definitions varies from one implementation to another -
and usually parameters are handled in an opaque way with binary
controls. This is best handled with a UUID that needs to be known only
to applications and low-level firmware/hardware, the kernel code should
not have to be modified for each and every processing and to add new
parameters. It just does not scale and it's unmaintainable.
At the very least if you really want to use this compress API, extend it
to use a non-descript "UUID-defined" type and an opaque set of
parameters with this UUID passed in a header.
this sounds wrong to me. The sample rate converter is not an "encoder",
and the properties for padding/delay are totally specific to an encoder
function.
There is only decoder and encoder definition for compress, I know
it is difficult to add SRC to encoder or decoder classification,
SRC is a Post Processing. I hope you can have a recommandation
I don't. I think we're blurring layers in a really odd way.
The main reason why the compress API was added is to remove the
byte-to-time conversions. But that's clearly not useful for a
post-processing of PCM data, where the bitrate is constant. It really
feels like we're adding this memory-to-memory API to the compress
framework because we don't have anything else available, not because it
makes sense to do so.
It makes sense to offload decoder/encoder tasks as batch processing for
standard compress stream and return back result (PCM stream or encoded stream)
to user space. So it makes sense to use the compress interface (parameters
handshake) for it. Let's talk about the proper SRC extension.
For SRC and dynamic rate modification. I would just create an ALSA control for
this. We are already using the "PCM Rate Shift 100000" control in the
sound/drivers/aloop.c for this purpose (something like pitch in MIDI) for
example. There is no requirement to add this function through metadata ioctls.
As bonus, this control can be monitored with multiple tasks.
Then there's the issue of parameters, we chose to only add parameters
for standard encoders/decoders. Post-processing is highly specific and
the parameter definitions varies from one implementation to another -
and usually parameters are handled in an opaque way with binary
controls. This is best handled with a UUID that needs to be known only
to applications and low-level firmware/hardware, the kernel code should
not have to be modified for each and every processing and to add new
parameters. It just does not scale and it's unmaintainable.
At the very least if you really want to use this compress API, extend it
to use a non-descript "UUID-defined" type and an opaque set of
parameters with this UUID passed in a header.
We don't need to use UUID-defined scheme for simple (A)SRC implementation. As
I noted, the specific runtime controls may use existing ALSA control API.
Jaroslav
--
Jaroslav Kysela [off-list ref]
Linux Sound Maintainer; ALSA Project; Red Hat, Inc.
* end of the track
* @SNDRV_COMPRESS_ENCODER_DELAY: no of samples inserted by the
encoder at the
* beginning of the track
+ * @SNDRV_COMPRESS_SRC_RATIO_MOD: Resampling Ratio Modifier for
sample rate converter
*/
enum sndrv_compress_encoder {
SNDRV_COMPRESS_ENCODER_PADDING = 1,
SNDRV_COMPRESS_ENCODER_DELAY = 2,
+ SNDRV_COMPRESS_SRC_RATIO_MOD = 3,
};
this sounds wrong to me. The sample rate converter is not an "encoder",
and the properties for padding/delay are totally specific to an encoder
function.
There is only decoder and encoder definition for compress, I know
it is difficult to add SRC to encoder or decoder classification,
SRC is a Post Processing. I hope you can have a recommandation
I don't. I think we're blurring layers in a really odd way.
The main reason why the compress API was added is to remove the
byte-to-time conversions. But that's clearly not useful for a
post-processing of PCM data, where the bitrate is constant. It really
feels like we're adding this memory-to-memory API to the compress
framework because we don't have anything else available, not because it
makes sense to do so.
It makes sense to offload decoder/encoder tasks as batch processing for
standard compress stream and return back result (PCM stream or encoded
stream) to user space. So it makes sense to use the compress interface
(parameters handshake) for it. Let's talk about the proper SRC extension.
For SRC and dynamic rate modification. I would just create an ALSA
control for this. We are already using the "PCM Rate Shift 100000"
control in the sound/drivers/aloop.c for this purpose (something like
pitch in MIDI) for example. There is no requirement to add this function
through metadata ioctls. As bonus, this control can be monitored with
multiple tasks.
this wouldn't work when the rate is estimated in firmware/hardware,
which is precisely what the 'asynchronous' part of ASRC does.
quoted
Then there's the issue of parameters, we chose to only add parameters
for standard encoders/decoders. Post-processing is highly specific and
the parameter definitions varies from one implementation to another -
and usually parameters are handled in an opaque way with binary
controls. This is best handled with a UUID that needs to be known only
to applications and low-level firmware/hardware, the kernel code should
not have to be modified for each and every processing and to add new
parameters. It just does not scale and it's unmaintainable.
At the very least if you really want to use this compress API, extend it
to use a non-descript "UUID-defined" type and an opaque set of
parameters with this UUID passed in a header.
We don't need to use UUID-defined scheme for simple (A)SRC
implementation. As I noted, the specific runtime controls may use
existing ALSA control API.
"Simple (A)SRC" is an oxymoron. There are multiple ways to define the
performance, and how the drift estimator is handled. There's nothing
simple if you look under the hood. The SOF implementation has for
example those parameters:
uint32_t source_rate; /**< Define fixed source rate or */
/**< use 0 to indicate need to get */
/**< the rate from stream */
uint32_t sink_rate; /**< Define fixed sink rate or */
/**< use 0 to indicate need to get */
/**< the rate from stream */
uint32_t asynchronous_mode; /**< synchronous 0, asynchronous 1 */
/**< When 1 the ASRC tracks and */
/**< compensates for drift. */
uint32_t operation_mode; /**< push 0, pull 1, In push mode the */
/**< ASRC consumes a defined number */
/**< of frames at input, with varying */
/**< number of frames at output. */
/**< In pull mode the ASRC outputs */
/**< a defined number of frames while */
/**< number of input frames varies. */
They are clearly different from what is suggested above with a 'ratio-mod'.
Same if you have a 'simple EQ'. there are dozens of ways to implement
the functionality with FIR, IIR or a combination of the two, and
multiple bands.
The point is that you have to think upfront about a generic way to pass
parameters. We didn't have to do it for encoders/decoders because we
only catered to well-documented standard solutions only. By choosing to
support PCM processing, a new can of worms is now open.
I repeat: please do not make the mistake of listing all processing with
an enum and a new structure for parameters every time someone needs a
specific transform in their pipeline. We made that mistake with SOF and
had to backtrack rather quickly. The only way to scale is an identifier
that is NOT included in the kernel code but is known to higher and
lower-levels only.
encoder at the
* beginning of the track
+ * @SNDRV_COMPRESS_SRC_RATIO_MOD: Resampling Ratio Modifier for
sample rate converter
*/
enum sndrv_compress_encoder {
SNDRV_COMPRESS_ENCODER_PADDING = 1,
SNDRV_COMPRESS_ENCODER_DELAY = 2,
+ SNDRV_COMPRESS_SRC_RATIO_MOD = 3,
};
this sounds wrong to me. The sample rate converter is not an "encoder",
and the properties for padding/delay are totally specific to an encoder
function.
There is only decoder and encoder definition for compress, I know
it is difficult to add SRC to encoder or decoder classification,
SRC is a Post Processing. I hope you can have a recommandation
I don't. I think we're blurring layers in a really odd way.
The main reason why the compress API was added is to remove the
byte-to-time conversions. But that's clearly not useful for a
post-processing of PCM data, where the bitrate is constant. It really
feels like we're adding this memory-to-memory API to the compress
framework because we don't have anything else available, not because it
makes sense to do so.
It makes sense to offload decoder/encoder tasks as batch processing for
standard compress stream and return back result (PCM stream or encoded
stream) to user space. So it makes sense to use the compress interface
(parameters handshake) for it. Let's talk about the proper SRC extension.
For SRC and dynamic rate modification. I would just create an ALSA
control for this. We are already using the "PCM Rate Shift 100000"
control in the sound/drivers/aloop.c for this purpose (something like
pitch in MIDI) for example. There is no requirement to add this function
through metadata ioctls. As bonus, this control can be monitored with
multiple tasks.
this wouldn't work when the rate is estimated in firmware/hardware,
which is precisely what the 'asynchronous' part of ASRC does.
quoted
quoted
Then there's the issue of parameters, we chose to only add parameters
for standard encoders/decoders. Post-processing is highly specific and
the parameter definitions varies from one implementation to another -
and usually parameters are handled in an opaque way with binary
controls. This is best handled with a UUID that needs to be known only
to applications and low-level firmware/hardware, the kernel code should
not have to be modified for each and every processing and to add new
parameters. It just does not scale and it's unmaintainable.
At the very least if you really want to use this compress API, extend it
to use a non-descript "UUID-defined" type and an opaque set of
parameters with this UUID passed in a header.
We don't need to use UUID-defined scheme for simple (A)SRC
implementation. As I noted, the specific runtime controls may use
existing ALSA control API.
"Simple (A)SRC" is an oxymoron. There are multiple ways to define the
performance, and how the drift estimator is handled. There's nothing
simple if you look under the hood. The SOF implementation has for
example those parameters:
uint32_t source_rate; /**< Define fixed source rate or */
/**< use 0 to indicate need to get */
/**< the rate from stream */
uint32_t sink_rate; /**< Define fixed sink rate or */
/**< use 0 to indicate need to get */
/**< the rate from stream */
uint32_t asynchronous_mode; /**< synchronous 0, asynchronous 1 */
/**< When 1 the ASRC tracks and */
/**< compensates for drift. */
uint32_t operation_mode; /**< push 0, pull 1, In push mode the */
/**< ASRC consumes a defined number */
/**< of frames at input, with varying */
/**< number of frames at output. */
/**< In pull mode the ASRC outputs */
/**< a defined number of frames while */
/**< number of input frames varies. */
They are clearly different from what is suggested above with a 'ratio-mod'.
I don't think so. The proposed (A)SRC for compress-accel is just one case for
the above configs where the input is known and output is controlled by the
requested rate. The I/O mechanism is abstracted enough in this case and the
driver/hardware/firmware must follow it.
Same if you have a 'simple EQ'. there are dozens of ways to implement
the functionality with FIR, IIR or a combination of the two, and
multiple bands.
The point is that you have to think upfront about a generic way to pass
parameters. We didn't have to do it for encoders/decoders because we
only catered to well-documented standard solutions only. By choosing to
support PCM processing, a new can of worms is now open.
I repeat: please do not make the mistake of listing all processing with
an enum and a new structure for parameters every time someone needs a
specific transform in their pipeline. We made that mistake with SOF and
had to backtrack rather quickly. The only way to scale is an identifier
that is NOT included in the kernel code but is known to higher and
lower-levels only.
There are two ways - black box (UUID - as you suggested) - or well defined
purpose (abstraction). For your example 'simple EQ', the parameters should be
the band (frequency range) volume values. It's abstract and the real filters
(resp. implementation) used behind may depend on the hardware/driver capabilities.
From my view, the really special cases may be handled as black box, but
others like (A)SRC should follow some well-defined abstraction IMHO to not
force user space to handle all special cases.
Jaroslav
--
Jaroslav Kysela [off-list ref]
Linux Sound Maintainer; ALSA Project; Red Hat, Inc.
Then there's the issue of parameters, we chose to only add parameters
for standard encoders/decoders. Post-processing is highly specific and
the parameter definitions varies from one implementation to another -
and usually parameters are handled in an opaque way with binary
controls. This is best handled with a UUID that needs to be known only
to applications and low-level firmware/hardware, the kernel code should
not have to be modified for each and every processing and to add new
parameters. It just does not scale and it's unmaintainable.
At the very least if you really want to use this compress API,
extend it
to use a non-descript "UUID-defined" type and an opaque set of
parameters with this UUID passed in a header.
We don't need to use UUID-defined scheme for simple (A)SRC
implementation. As I noted, the specific runtime controls may use
existing ALSA control API.
"Simple (A)SRC" is an oxymoron. There are multiple ways to define the
performance, and how the drift estimator is handled. There's nothing
simple if you look under the hood. The SOF implementation has for
example those parameters:
uint32_t source_rate; /**< Define fixed source rate or */
/**< use 0 to indicate need to get */
/**< the rate from stream */
uint32_t sink_rate; /**< Define fixed sink rate or */
/**< use 0 to indicate need to get */
/**< the rate from stream */
uint32_t asynchronous_mode; /**< synchronous 0, asynchronous 1 */
/**< When 1 the ASRC tracks and */
/**< compensates for drift. */
uint32_t operation_mode; /**< push 0, pull 1, In push mode the */
/**< ASRC consumes a defined number */
/**< of frames at input, with varying */
/**< number of frames at output. */
/**< In pull mode the ASRC outputs */
/**< a defined number of frames while */
/**< number of input frames varies. */
They are clearly different from what is suggested above with a 'ratio-
mod'.
I don't think so. The proposed (A)SRC for compress-accel is just one
case for the above configs where the input is known and output is
controlled by the requested rate. The I/O mechanism is abstracted enough
in this case and the driver/hardware/firmware must follow it.
ASRC is usually added when the nominal rates are known but the clock
sources differ and the drift needs to be estimated at run-time and the
coefficients or interpolation modified dynamically
If the ratio is known exactly and there's no clock drift, then it's a
different problem where the filter coefficients are constant.
quoted
Same if you have a 'simple EQ'. there are dozens of ways to implement
the functionality with FIR, IIR or a combination of the two, and
multiple bands.
The point is that you have to think upfront about a generic way to pass
parameters. We didn't have to do it for encoders/decoders because we
only catered to well-documented standard solutions only. By choosing to
support PCM processing, a new can of worms is now open.
I repeat: please do not make the mistake of listing all processing with
an enum and a new structure for parameters every time someone needs a
specific transform in their pipeline. We made that mistake with SOF and
had to backtrack rather quickly. The only way to scale is an identifier
that is NOT included in the kernel code but is known to higher and
lower-levels only.
There are two ways - black box (UUID - as you suggested) - or well
defined purpose (abstraction). For your example 'simple EQ', the
parameters should be the band (frequency range) volume values. It's
abstract and the real filters (resp. implementation) used behind may
depend on the hardware/driver capabilities.
Indeed there is a possibility that the parameters are high-level, but
that would require firmware or hardware to be able to generate actual
coefficients from those parameters. That usually requires some advanced
math which isn't necessarily obvious to implement with fixed-point hardware.
From my view, the really special cases may be handled as black box, but
others like (A)SRC should follow some well-defined abstraction IMHO to
not force user space to handle all special cases.
I am not against the high-level abstractions, e.g. along the lines of
what Android defined:
https://developer.android.com/reference/android/media/audiofx/AudioEffect
That's not sufficient however, we also need to make sure there's an
ability to provide pre-computed coefficients in an opaque manner for
processing that doesn't fit in the well-defined cases. In practice there
are very few 3rd party IP that fits in well-defined cases, everyone has
secret-sauce parameters and options.
From: Shengjiu Wang <shengjiu.wang@gmail.com> Date: 2024-08-09 10:14:44
On Fri, Aug 9, 2024 at 3:25 PM Pierre-Louis Bossart
[off-list ref] wrote:
quoted
quoted
quoted
quoted
Then there's the issue of parameters, we chose to only add parameters
for standard encoders/decoders. Post-processing is highly specific and
the parameter definitions varies from one implementation to another -
and usually parameters are handled in an opaque way with binary
controls. This is best handled with a UUID that needs to be known only
to applications and low-level firmware/hardware, the kernel code should
not have to be modified for each and every processing and to add new
parameters. It just does not scale and it's unmaintainable.
At the very least if you really want to use this compress API,
extend it
to use a non-descript "UUID-defined" type and an opaque set of
parameters with this UUID passed in a header.
We don't need to use UUID-defined scheme for simple (A)SRC
implementation. As I noted, the specific runtime controls may use
existing ALSA control API.
"Simple (A)SRC" is an oxymoron. There are multiple ways to define the
performance, and how the drift estimator is handled. There's nothing
simple if you look under the hood. The SOF implementation has for
example those parameters:
uint32_t source_rate; /**< Define fixed source rate or */
/**< use 0 to indicate need to get */
/**< the rate from stream */
uint32_t sink_rate; /**< Define fixed sink rate or */
/**< use 0 to indicate need to get */
/**< the rate from stream */
uint32_t asynchronous_mode; /**< synchronous 0, asynchronous 1 */
/**< When 1 the ASRC tracks and */
/**< compensates for drift. */
uint32_t operation_mode; /**< push 0, pull 1, In push mode the */
/**< ASRC consumes a defined number */
/**< of frames at input, with varying */
/**< number of frames at output. */
/**< In pull mode the ASRC outputs */
/**< a defined number of frames while */
/**< number of input frames varies. */
They are clearly different from what is suggested above with a 'ratio-
mod'.
I don't think so. The proposed (A)SRC for compress-accel is just one
case for the above configs where the input is known and output is
controlled by the requested rate. The I/O mechanism is abstracted enough
in this case and the driver/hardware/firmware must follow it.
ASRC is usually added when the nominal rates are known but the clock
sources differ and the drift needs to be estimated at run-time and the
coefficients or interpolation modified dynamically
If the ratio is known exactly and there's no clock drift, then it's a
different problem where the filter coefficients are constant.
quoted
quoted
Same if you have a 'simple EQ'. there are dozens of ways to implement
the functionality with FIR, IIR or a combination of the two, and
multiple bands.
The point is that you have to think upfront about a generic way to pass
parameters. We didn't have to do it for encoders/decoders because we
only catered to well-documented standard solutions only. By choosing to
support PCM processing, a new can of worms is now open.
I repeat: please do not make the mistake of listing all processing with
an enum and a new structure for parameters every time someone needs a
specific transform in their pipeline. We made that mistake with SOF and
had to backtrack rather quickly. The only way to scale is an identifier
that is NOT included in the kernel code but is known to higher and
lower-levels only.
There are two ways - black box (UUID - as you suggested) - or well
defined purpose (abstraction). For your example 'simple EQ', the
parameters should be the band (frequency range) volume values. It's
abstract and the real filters (resp. implementation) used behind may
depend on the hardware/driver capabilities.
Indeed there is a possibility that the parameters are high-level, but
that would require firmware or hardware to be able to generate actual
coefficients from those parameters. That usually requires some advanced
math which isn't necessarily obvious to implement with fixed-point hardware.
quoted
From my view, the really special cases may be handled as black box, but
others like (A)SRC should follow some well-defined abstraction IMHO to
not force user space to handle all special cases.
I am not against the high-level abstractions, e.g. along the lines of
what Android defined:
https://developer.android.com/reference/android/media/audiofx/AudioEffect
That's not sufficient however, we also need to make sure there's an
ability to provide pre-computed coefficients in an opaque manner for
processing that doesn't fit in the well-defined cases. In practice there
are very few 3rd party IP that fits in well-defined cases, everyone has
secret-sauce parameters and options.
Appreciate the discussion.
Let me explain the reason for the change:
Why I use the metadata ioctl is because the ALSA controls are binding
to the sound card. What I want is the controls can be bound to
snd_compr_stream, because the ASRC compress sound card can
support multi instances ( the ASRC can support multi conversion in
parallel). The ALSA controls can't be used for this case, the only
choice in current compress API is metadata ioctl. And metadata
ioctl can be called many times which can meet the ratio modifier
requirement (ratio may be drift on the fly)
And compress API uses codec as the unit for capability query and
parameter setting, So I think need to define "SND_AUDIOCODEC_SRC'
and 'struct snd_dec_src', for the 'snd_dec_src' just defined output
format and output rate, channels definition just reuse the snd_codec.ch_in.
I understand your concern, but there seems no better option.
If you have, please guide me. Thanks.
Best regards
Shengjiu Wang
Why I use the metadata ioctl is because the ALSA controls are binding
to the sound card. What I want is the controls can be bound to
snd_compr_stream, because the ASRC compress sound card can
support multi instances ( the ASRC can support multi conversion in
parallel). The ALSA controls can't be used for this case, the only
choice in current compress API is metadata ioctl.
I don't know if there is really a technical limitation for this, this is
for Jaroslav to comment. I am not sure why it would be a problem to e.g.
have a volume control prior to an encoder or after a decoder.
And metadata
ioctl can be called many times which can meet the ratio modifier
requirement (ratio may be drift on the fly)
Interesting, that's yet another way of handling the drift with userspace
modifying the ratio dynamically. That's different to what I've seen before.
And compress API uses codec as the unit for capability query and
parameter setting, So I think need to define "SND_AUDIOCODEC_SRC'
and 'struct snd_dec_src', for the 'snd_dec_src' just defined output
format and output rate, channels definition just reuse the snd_codec.ch_in.
The capability query is an interesting point as well, it's not clear how
to expose to userspace what this specific implementation can do, while
at the same time *requiring* userpace to update the ratio dynamically.
For something like this to work, userspace needs to have pre-existing
information on how the SRC works.
From: Jaroslav Kysela <perex@perex.cz> Date: 2024-08-09 14:01:15
On 09. 08. 24 14:52, Pierre-Louis Bossart wrote:
quoted
And metadata
ioctl can be called many times which can meet the ratio modifier
requirement (ratio may be drift on the fly)
Interesting, that's yet another way of handling the drift with userspace
modifying the ratio dynamically. That's different to what I've seen before.
Note that the "timing" is managed by the user space with this scheme.
quoted
And compress API uses codec as the unit for capability query and
parameter setting, So I think need to define "SND_AUDIOCODEC_SRC'
and 'struct snd_dec_src', for the 'snd_dec_src' just defined output
format and output rate, channels definition just reuse the snd_codec.ch_in.
The capability query is an interesting point as well, it's not clear how
to expose to userspace what this specific implementation can do, while
at the same time *requiring* userpace to update the ratio dynamically.
For something like this to work, userspace needs to have pre-existing
information on how the SRC works.
Yes, it's about abstraction. The user space wants to push data, read data back
converted to the target rate and eventually modify the drift using a control
managing clocks using own way. We can eventually assume, that if this control
does not exist, the drift cannot be controlled. Also, nice thing is that the
control has min and max values (range), so driver can specify the drift range,
too.
And again, look to "PCM Rate Shift 100000" control implementation in
sound/drivers/aloop.c. It would be nice to have the base offset for the
shift/drift/pitch value standardized.
Jaroslav
--
Jaroslav Kysela [off-list ref]
Linux Sound Maintainer; ALSA Project; Red Hat, Inc.
And metadata
ioctl can be called many times which can meet the ratio modifier
requirement (ratio may be drift on the fly)
Interesting, that's yet another way of handling the drift with userspace
modifying the ratio dynamically. That's different to what I've seen
before.
Note that the "timing" is managed by the user space with this scheme.
quoted
quoted
And compress API uses codec as the unit for capability query and
parameter setting, So I think need to define "SND_AUDIOCODEC_SRC'
and 'struct snd_dec_src', for the 'snd_dec_src' just defined output
format and output rate, channels definition just reuse the
snd_codec.ch_in.
The capability query is an interesting point as well, it's not clear how
to expose to userspace what this specific implementation can do, while
at the same time *requiring* userpace to update the ratio dynamically.
For something like this to work, userspace needs to have pre-existing
information on how the SRC works.
Yes, it's about abstraction. The user space wants to push data, read
data back converted to the target rate and eventually modify the drift
using a control managing clocks using own way. We can eventually assume,
that if this control does not exist, the drift cannot be controlled.
Also, nice thing is that the control has min and max values (range), so
driver can specify the drift range, too.
This mode of operation would be fine, but if you add the SRC as part of
the processing allowed in a compressed stream, it might be used in a
'regular' real-time pipeline and the arguments and implementation could
be very different.
In other words, this SRC support looks like an extension of the compress
API for a very narrow usage restricted to the memory-to-memory case
only. I worry a bit about the next steps... Are there going to be other
types of PCM processing like this, and if yes, how would we know if they
are intended to be used for the 'regular' compress API or for the
memory-to-memory scheme?
From: Shengjiu Wang <shengjiu.wang@gmail.com> Date: 2024-08-12 10:24:23
On Fri, Aug 9, 2024 at 10:01 PM Jaroslav Kysela [off-list ref] wrote:
On 09. 08. 24 14:52, Pierre-Louis Bossart wrote:
quoted
quoted
And metadata
ioctl can be called many times which can meet the ratio modifier
requirement (ratio may be drift on the fly)
Interesting, that's yet another way of handling the drift with userspace
modifying the ratio dynamically. That's different to what I've seen before.
Note that the "timing" is managed by the user space with this scheme.
quoted
quoted
And compress API uses codec as the unit for capability query and
parameter setting, So I think need to define "SND_AUDIOCODEC_SRC'
and 'struct snd_dec_src', for the 'snd_dec_src' just defined output
format and output rate, channels definition just reuse the snd_codec.ch_in.
The capability query is an interesting point as well, it's not clear how
to expose to userspace what this specific implementation can do, while
at the same time *requiring* userpace to update the ratio dynamically.
For something like this to work, userspace needs to have pre-existing
information on how the SRC works.
Yes, it's about abstraction. The user space wants to push data, read data back
converted to the target rate and eventually modify the drift using a control
managing clocks using own way. We can eventually assume, that if this control
does not exist, the drift cannot be controlled. Also, nice thing is that the
control has min and max values (range), so driver can specify the drift range,
too.
And again, look to "PCM Rate Shift 100000" control implementation in
sound/drivers/aloop.c. It would be nice to have the base offset for the
shift/drift/pitch value standardized.
Thanks.
But the ASRC driver I implemented is different, I just register one sound
card, one device/subdevice. but the ASRC hardware support 4 instances
together, so user can open the card device 4 times to create 4 instances
then the controls can only bind with compress streams.
I think I can remove the 'SNDRV_COMPRESS_SRC_RATIO_MOD',
Only define a private type for driver, which means only the ASRC driver
and its user application know the type.
For the change in 'include/uapi/sound/compress_params.h", should I
keep them, is there any other suggestion for them?
Best regards
Shengjiu Wang
Jaroslav
--
Jaroslav Kysela [off-list ref]
Linux Sound Maintainer; ALSA Project; Red Hat, Inc.
From: Jaroslav Kysela <perex@perex.cz> Date: 2024-08-12 13:31:41
On 12. 08. 24 12:24, Shengjiu Wang wrote:
On Fri, Aug 9, 2024 at 10:01 PM Jaroslav Kysela [off-list ref] wrote:
quoted
On 09. 08. 24 14:52, Pierre-Louis Bossart wrote:
quoted
quoted
And metadata
ioctl can be called many times which can meet the ratio modifier
requirement (ratio may be drift on the fly)
Interesting, that's yet another way of handling the drift with userspace
modifying the ratio dynamically. That's different to what I've seen before.
Note that the "timing" is managed by the user space with this scheme.
quoted
quoted
And compress API uses codec as the unit for capability query and
parameter setting, So I think need to define "SND_AUDIOCODEC_SRC'
and 'struct snd_dec_src', for the 'snd_dec_src' just defined output
format and output rate, channels definition just reuse the snd_codec.ch_in.
The capability query is an interesting point as well, it's not clear how
to expose to userspace what this specific implementation can do, while
at the same time *requiring* userpace to update the ratio dynamically.
For something like this to work, userspace needs to have pre-existing
information on how the SRC works.
Yes, it's about abstraction. The user space wants to push data, read data back
converted to the target rate and eventually modify the drift using a control
managing clocks using own way. We can eventually assume, that if this control
does not exist, the drift cannot be controlled. Also, nice thing is that the
control has min and max values (range), so driver can specify the drift range,
too.
And again, look to "PCM Rate Shift 100000" control implementation in
sound/drivers/aloop.c. It would be nice to have the base offset for the
shift/drift/pitch value standardized.
Thanks.
But the ASRC driver I implemented is different, I just register one sound
card, one device/subdevice. but the ASRC hardware support 4 instances
together, so user can open the card device 4 times to create 4 instances
then the controls can only bind with compress streams.
It's just a reason to add the subdevice code for the compress offload layer
like we have in other APIs for overall consistency. I'll try to work on this.
I think I can remove the 'SNDRV_COMPRESS_SRC_RATIO_MOD',
Yes.
Only define a private type for driver, which means only the ASRC driver
and its user application know the type.
The control API should be used for this IMHO.
For the change in 'include/uapi/sound/compress_params.h", should I
keep them, is there any other suggestion for them?
I don't see a problem there.
Jaroslav
--
Jaroslav Kysela [off-list ref]
Linux Sound Maintainer; ALSA Project; Red Hat, Inc.
On Fri, Aug 9, 2024 at 10:01 PM Jaroslav Kysela [off-list ref] wrote:
quoted
On 09. 08. 24 14:52, Pierre-Louis Bossart wrote:
quoted
quoted
And metadata
ioctl can be called many times which can meet the ratio modifier
requirement (ratio may be drift on the fly)
Interesting, that's yet another way of handling the drift with
userspace
modifying the ratio dynamically. That's different to what I've seen
before.
Note that the "timing" is managed by the user space with this scheme.
quoted
quoted
And compress API uses codec as the unit for capability query and
parameter setting, So I think need to define "SND_AUDIOCODEC_SRC'
and 'struct snd_dec_src', for the 'snd_dec_src' just defined output
format and output rate, channels definition just reuse the
snd_codec.ch_in.
The capability query is an interesting point as well, it's not clear
how
to expose to userspace what this specific implementation can do, while
at the same time *requiring* userpace to update the ratio dynamically.
For something like this to work, userspace needs to have pre-existing
information on how the SRC works.
Yes, it's about abstraction. The user space wants to push data, read
data back
converted to the target rate and eventually modify the drift using a
control
managing clocks using own way. We can eventually assume, that if this
control
does not exist, the drift cannot be controlled. Also, nice thing is
that the
control has min and max values (range), so driver can specify the
drift range,
too.
And again, look to "PCM Rate Shift 100000" control implementation in
sound/drivers/aloop.c. It would be nice to have the base offset for the
shift/drift/pitch value standardized.
Thanks.
But the ASRC driver I implemented is different, I just register one sound
card, one device/subdevice. but the ASRC hardware support 4 instances
together, so user can open the card device 4 times to create 4 instances
then the controls can only bind with compress streams.
It's just a reason to add the subdevice code for the compress offload
layer like we have in other APIs for overall consistency. I'll try to
work on this.
I thought this was supported already? I remember there was a request to
enable more than one compressed stream for enhanced cross-fade support
with different formats? That isn't supported with the single-device +
PARTIAL_DRAIN method.
Vinod?
quoted
I think I can remove the 'SNDRV_COMPRESS_SRC_RATIO_MOD',
Yes.
quoted
Only define a private type for driver, which means only the ASRC driver
and its user application know the type.
The control API should be used for this IMHO.
Agree, this would be a 'clean' split where the compress API is used for
the data parts and the control parts used otherwise to alter the ratio
or whatever else is needed.
quoted
For the change in 'include/uapi/sound/compress_params.h", should I
keep them, is there any other suggestion for them?
You can add the SRC type but if you use a control for the parameters you
don't need to add anything for the encoder options, do you?
From: Shengjiu Wang <shengjiu.wang@gmail.com> Date: 2024-08-14 02:22:48
On Mon, Aug 12, 2024 at 9:44 PM Pierre-Louis Bossart
[off-list ref] wrote:
On 8/12/24 15:31, Jaroslav Kysela wrote:
quoted
On 12. 08. 24 12:24, Shengjiu Wang wrote:
quoted
On Fri, Aug 9, 2024 at 10:01 PM Jaroslav Kysela [off-list ref] wrote:
quoted
On 09. 08. 24 14:52, Pierre-Louis Bossart wrote:
quoted
quoted
And metadata
ioctl can be called many times which can meet the ratio modifier
requirement (ratio may be drift on the fly)
Interesting, that's yet another way of handling the drift with
userspace
modifying the ratio dynamically. That's different to what I've seen
before.
Note that the "timing" is managed by the user space with this scheme.
quoted
quoted
And compress API uses codec as the unit for capability query and
parameter setting, So I think need to define "SND_AUDIOCODEC_SRC'
and 'struct snd_dec_src', for the 'snd_dec_src' just defined output
format and output rate, channels definition just reuse the
snd_codec.ch_in.
The capability query is an interesting point as well, it's not clear
how
to expose to userspace what this specific implementation can do, while
at the same time *requiring* userpace to update the ratio dynamically.
For something like this to work, userspace needs to have pre-existing
information on how the SRC works.
Yes, it's about abstraction. The user space wants to push data, read
data back
converted to the target rate and eventually modify the drift using a
control
managing clocks using own way. We can eventually assume, that if this
control
does not exist, the drift cannot be controlled. Also, nice thing is
that the
control has min and max values (range), so driver can specify the
drift range,
too.
And again, look to "PCM Rate Shift 100000" control implementation in
sound/drivers/aloop.c. It would be nice to have the base offset for the
shift/drift/pitch value standardized.
Thanks.
But the ASRC driver I implemented is different, I just register one sound
card, one device/subdevice. but the ASRC hardware support 4 instances
together, so user can open the card device 4 times to create 4 instances
then the controls can only bind with compress streams.
It's just a reason to add the subdevice code for the compress offload
layer like we have in other APIs for overall consistency. I'll try to
work on this.
I thought this was supported already? I remember there was a request to
enable more than one compressed stream for enhanced cross-fade support
with different formats? That isn't supported with the single-device +
PARTIAL_DRAIN method.
Vinod?
quoted
quoted
I think I can remove the 'SNDRV_COMPRESS_SRC_RATIO_MOD',
Yes.
quoted
Only define a private type for driver, which means only the ASRC driver
and its user application know the type.
The control API should be used for this IMHO.
Agree, this would be a 'clean' split where the compress API is used for
the data parts and the control parts used otherwise to alter the ratio
or whatever else is needed.
quoted
quoted
For the change in 'include/uapi/sound/compress_params.h", should I
keep them, is there any other suggestion for them?
You can add the SRC type but if you use a control for the parameters you
don't need to add anything for the encoder options, do you?
Yes, to go further, I think we can use SND_AUDIOCODEC_PCM, then
the SRC type will be dropped.
But my understanding of the control means the .set_metadata() API, right?
As I said, the output rate, output format, and ratio modifier are applied to
the instances of ASRC, which is the snd_compr_stream in driver.
so only the .set_metadata() API can be used for these purposes.
Best regards
Shengjiu Wang
Yes, to go further, I think we can use SND_AUDIOCODEC_PCM, then
the SRC type will be dropped.
sounds good.
But my understanding of the control means the .set_metadata() API, right?
As I said, the output rate, output format, and ratio modifier are applied to
the instances of ASRC, which is the snd_compr_stream in driver.
so only the .set_metadata() API can be used for these purposes.
Humm, this is more controversial.
The term 'metadata' really referred to known information present in
headers or additional ID3 tags and not in the compressed file itself.
The .set_metadata was assumed to be called ONCE before decoding.
But here you have a need to update the ratio modifier on a regular basis
to compensate for the drift. This isn't what this specific callback was
designed for. We could change and allow this callback to be used
multiple times, but then this could create problems for existing
implementations which cannot deal with modified metadata on the fly.
And then there's the problem of defining a 'key' for the metadata. the
definition of the key is a u32, so there's plenty of space for different
implementations, but a collision is possible. We'd need an agreement on
how to allocate keys to different solutions without changing the header
file for every implementation.
It sounds like we'd need a 'runtime params' callback - unless there's a
better trick to tie the control and compress layers?
From: Shengjiu Wang <shengjiu.wang@gmail.com> Date: 2024-08-14 11:12:28
On Wed, Aug 14, 2024 at 5:40 PM Pierre-Louis Bossart
[off-list ref] wrote:
quoted
Yes, to go further, I think we can use SND_AUDIOCODEC_PCM, then
the SRC type will be dropped.
sounds good.
quoted
But my understanding of the control means the .set_metadata() API, right?
As I said, the output rate, output format, and ratio modifier are applied to
the instances of ASRC, which is the snd_compr_stream in driver.
so only the .set_metadata() API can be used for these purposes.
Humm, this is more controversial.
The term 'metadata' really referred to known information present in
headers or additional ID3 tags and not in the compressed file itself.
The .set_metadata was assumed to be called ONCE before decoding.
But here you have a need to update the ratio modifier on a regular basis
to compensate for the drift. This isn't what this specific callback was
designed for. We could change and allow this callback to be used
multiple times, but then this could create problems for existing
implementations which cannot deal with modified metadata on the fly.
.set_metadata can be called multi times now, no need to change currently.
And then there's the problem of defining a 'key' for the metadata. the
definition of the key is a u32, so there's plenty of space for different
implementations, but a collision is possible. We'd need an agreement on
how to allocate keys to different solutions without changing the header
file for every implementation.
Can we define a private space for each case? For example the key larger
than 0x80000000 is private, each driver can define it by themself?
It sounds like we'd need a 'runtime params' callback - unless there's a
better trick to tie the control and compress layers?
On Wed, Aug 14, 2024 at 5:40 PM Pierre-Louis Bossart
[off-list ref] wrote:
quoted
quoted
Yes, to go further, I think we can use SND_AUDIOCODEC_PCM, then
the SRC type will be dropped.
sounds good.
quoted
But my understanding of the control means the .set_metadata() API, right?
As I said, the output rate, output format, and ratio modifier are applied to
the instances of ASRC, which is the snd_compr_stream in driver.
so only the .set_metadata() API can be used for these purposes.
Humm, this is more controversial.
The term 'metadata' really referred to known information present in
headers or additional ID3 tags and not in the compressed file itself.
The .set_metadata was assumed to be called ONCE before decoding.
But here you have a need to update the ratio modifier on a regular basis
to compensate for the drift. This isn't what this specific callback was
designed for. We could change and allow this callback to be used
multiple times, but then this could create problems for existing
implementations which cannot deal with modified metadata on the fly.
.set_metadata can be called multi times now, no need to change currently.
Not really, this set_metadata() callback is used only for gapless
transitions between tracks, see fcplay.c in tinycompress.
And now I am really confused because tinycompress uses an IOCTL directly:
metadata.key = SNDRV_COMPRESS_ENCODER_PADDING;
metadata.value[0] = mdata->encoder_padding;
if (ioctl(compress->fd, SNDRV_COMPRESS_SET_METADATA, &metadata))
Whereas you want to use the ops callback directly from the control layer?
What would present a userspace program from using the ioctl directly
then? In that case, why do we need the control? I must be missing something.
quoted
And then there's the problem of defining a 'key' for the metadata. the
definition of the key is a u32, so there's plenty of space for different
implementations, but a collision is possible. We'd need an agreement on
how to allocate keys to different solutions without changing the header
file for every implementation.
Can we define a private space for each case? For example the key larger
than 0x80000000 is private, each driver can define it by themself?
that would be a possibility indeed - provided that the opens above are
straightened out.
quoted
It sounds like we'd need a 'runtime params' callback - unless there's a
better trick to tie the control and compress layers?
From: Jaroslav Kysela <perex@perex.cz> Date: 2024-08-14 14:48:53
On 14. 08. 24 13:58, Pierre-Louis Bossart wrote:
On 8/14/24 13:12, Shengjiu Wang wrote:
quoted
On Wed, Aug 14, 2024 at 5:40 PM Pierre-Louis Bossart
[off-list ref] wrote:
quoted
quoted
Yes, to go further, I think we can use SND_AUDIOCODEC_PCM, then
the SRC type will be dropped.
sounds good.
quoted
But my understanding of the control means the .set_metadata() API, right?
As I said, the output rate, output format, and ratio modifier are applied to
the instances of ASRC, which is the snd_compr_stream in driver.
so only the .set_metadata() API can be used for these purposes.
Humm, this is more controversial.
The term 'metadata' really referred to known information present in
headers or additional ID3 tags and not in the compressed file itself.
The .set_metadata was assumed to be called ONCE before decoding.
But here you have a need to update the ratio modifier on a regular basis
to compensate for the drift. This isn't what this specific callback was
designed for. We could change and allow this callback to be used
multiple times, but then this could create problems for existing
implementations which cannot deal with modified metadata on the fly.
.set_metadata can be called multi times now, no need to change currently.
Not really, this set_metadata() callback is used only for gapless
transitions between tracks, see fcplay.c in tinycompress.
And now I am really confused because tinycompress uses an IOCTL directly:
metadata.key = SNDRV_COMPRESS_ENCODER_PADDING;
metadata.value[0] = mdata->encoder_padding;
if (ioctl(compress->fd, SNDRV_COMPRESS_SET_METADATA, &metadata))
Whereas you want to use the ops callback directly from the control layer?
What would present a userspace program from using the ioctl directly
then? In that case, why do we need the control? I must be missing something.
The whole discussion is which place is more appropriate for the runtime
controls (like the frequency shift). My opinion is, if we have a layer for
this which can be used for presence of those controls and even range / type /
notifications, we should use it.
The new/updated ioctls bounded only to active file descriptor does not allow
to monitor those values outside.
quoted
quoted
And then there's the problem of defining a 'key' for the metadata. the
definition of the key is a u32, so there's plenty of space for different
implementations, but a collision is possible. We'd need an agreement on
how to allocate keys to different solutions without changing the header
file for every implementation.
Can we define a private space for each case? For example the key larger
than 0x80000000 is private, each driver can define it by themself?
that would be a possibility indeed - provided that the opens above are
straightened out.
quoted
quoted
It sounds like we'd need a 'runtime params' callback - unless there's a
better trick to tie the control and compress layers?
I don't follow. If the compress driver code uses card/device/subdevice
numbers, we can address the control properly. The problem is just that
subdevice support in missing the current compress code / API.
For me, the compress_params.h changes may also require to pay attention to the
encoding/decoding of the current compressed streams. So something like this
may be more appropriate for the first step:
diff --git a/include/uapi/sound/compress_params.h
b/include/uapi/sound/compress_params.h
index ddc77322d571..c664d15410eb 100644
This will allow to handshake the output rate between user space and kernel
driver. Eventually we can use a rate bitmap to be more precise in "struct
snd_codec_desc_src" (or combination of range/bitmap).
Jaroslav
--
Jaroslav Kysela [off-list ref]
Linux Sound Maintainer; ALSA Project; Red Hat, Inc.
From: Jaroslav Kysela <perex@perex.cz> Date: 2024-08-09 13:51:57
On 09. 08. 24 12:14, Shengjiu Wang wrote:
On Fri, Aug 9, 2024 at 3:25 PM Pierre-Louis Bossart
[off-list ref] wrote:
quoted
quoted
quoted
quoted
quoted
Then there's the issue of parameters, we chose to only add parameters
for standard encoders/decoders. Post-processing is highly specific and
the parameter definitions varies from one implementation to another -
and usually parameters are handled in an opaque way with binary
controls. This is best handled with a UUID that needs to be known only
to applications and low-level firmware/hardware, the kernel code should
not have to be modified for each and every processing and to add new
parameters. It just does not scale and it's unmaintainable.
At the very least if you really want to use this compress API,
extend it
to use a non-descript "UUID-defined" type and an opaque set of
parameters with this UUID passed in a header.
We don't need to use UUID-defined scheme for simple (A)SRC
implementation. As I noted, the specific runtime controls may use
existing ALSA control API.
"Simple (A)SRC" is an oxymoron. There are multiple ways to define the
performance, and how the drift estimator is handled. There's nothing
simple if you look under the hood. The SOF implementation has for
example those parameters:
uint32_t source_rate; /**< Define fixed source rate or */
/**< use 0 to indicate need to get */
/**< the rate from stream */
uint32_t sink_rate; /**< Define fixed sink rate or */
/**< use 0 to indicate need to get */
/**< the rate from stream */
uint32_t asynchronous_mode; /**< synchronous 0, asynchronous 1 */
/**< When 1 the ASRC tracks and */
/**< compensates for drift. */
uint32_t operation_mode; /**< push 0, pull 1, In push mode the */
/**< ASRC consumes a defined number */
/**< of frames at input, with varying */
/**< number of frames at output. */
/**< In pull mode the ASRC outputs */
/**< a defined number of frames while */
/**< number of input frames varies. */
They are clearly different from what is suggested above with a 'ratio-
mod'.
I don't think so. The proposed (A)SRC for compress-accel is just one
case for the above configs where the input is known and output is
controlled by the requested rate. The I/O mechanism is abstracted enough
in this case and the driver/hardware/firmware must follow it.
ASRC is usually added when the nominal rates are known but the clock
sources differ and the drift needs to be estimated at run-time and the
coefficients or interpolation modified dynamically
If the ratio is known exactly and there's no clock drift, then it's a
different problem where the filter coefficients are constant.
quoted
quoted
Same if you have a 'simple EQ'. there are dozens of ways to implement
the functionality with FIR, IIR or a combination of the two, and
multiple bands.
The point is that you have to think upfront about a generic way to pass
parameters. We didn't have to do it for encoders/decoders because we
only catered to well-documented standard solutions only. By choosing to
support PCM processing, a new can of worms is now open.
I repeat: please do not make the mistake of listing all processing with
an enum and a new structure for parameters every time someone needs a
specific transform in their pipeline. We made that mistake with SOF and
had to backtrack rather quickly. The only way to scale is an identifier
that is NOT included in the kernel code but is known to higher and
lower-levels only.
There are two ways - black box (UUID - as you suggested) - or well
defined purpose (abstraction). For your example 'simple EQ', the
parameters should be the band (frequency range) volume values. It's
abstract and the real filters (resp. implementation) used behind may
depend on the hardware/driver capabilities.
Indeed there is a possibility that the parameters are high-level, but
that would require firmware or hardware to be able to generate actual
coefficients from those parameters. That usually requires some advanced
math which isn't necessarily obvious to implement with fixed-point hardware.
quoted
From my view, the really special cases may be handled as black box, but
others like (A)SRC should follow some well-defined abstraction IMHO to
not force user space to handle all special cases.
I am not against the high-level abstractions, e.g. along the lines of
what Android defined:
https://developer.android.com/reference/android/media/audiofx/AudioEffect
That's not sufficient however, we also need to make sure there's an
ability to provide pre-computed coefficients in an opaque manner for
processing that doesn't fit in the well-defined cases. In practice there
are very few 3rd party IP that fits in well-defined cases, everyone has
secret-sauce parameters and options.
Appreciate the discussion.
Let me explain the reason for the change:
Why I use the metadata ioctl is because the ALSA controls are binding
to the sound card. What I want is the controls can be bound to
snd_compr_stream, because the ASRC compress sound card can
support multi instances ( the ASRC can support multi conversion in
parallel). The ALSA controls can't be used for this case, the only
choice in current compress API is metadata ioctl. And metadata
ioctl can be called many times which can meet the ratio modifier
requirement (ratio may be drift on the fly)
This argument is not valid. The controls are bound to the card, but the
element identifiers have already iface (interface), device and subdevice
numbers. We are using controls for PCM devices for example. The binding is
straight.
Just add SNDRV_CTL_ELEM_IFACE_COMPRESS define and specify the compress device
number in the 'struct snd_ctl_elem_id'.
Jaroslav
--
Jaroslav Kysela [off-list ref]
Linux Sound Maintainer; ALSA Project; Red Hat, Inc.
From: Shengjiu Wang <hidden> Date: 2024-08-06 10:47:43
ASRC can be used on memory to memory case, define several
functions for m2m usage.
m2m_prepare: prepare for the start step
m2m_start: the start step
m2m_unprepare: unprepare for stop step, optional
m2m_stop: stop step
m2m_check_format: check format is supported or not
m2m_calc_out_len: calculate output length according to input length
m2m_get_maxburst: burst size for dma
m2m_pair_suspend: suspend function of pair, optional.
m2m_pair_resume: resume function of pair
get_output_fifo_size: get remaining data size in FIFO
Signed-off-by: Shengjiu Wang <redacted>
---
sound/soc/fsl/fsl_asrc.c | 139 ++++++++++++++++++++++++++++++++
sound/soc/fsl/fsl_asrc.h | 2 +
sound/soc/fsl/fsl_asrc_common.h | 59 ++++++++++++++
3 files changed, 200 insertions(+)
@@ -1063,6 +1063,136 @@ static int fsl_asrc_get_fifo_addr(u8 dir, enum asrc_pair_index index)returnREG_ASRDx(dir,index);}+/* Get sample numbers in FIFO */+staticunsignedintfsl_asrc_get_output_fifo_size(structfsl_asrc_pair*pair)+{+structfsl_asrc*asrc=pair->asrc;+enumasrc_pair_indexindex=pair->index;+u32val;++regmap_read(asrc->regmap,REG_ASRFST(index),&val);++val&=ASRFSTi_OUTPUT_FIFO_MASK;++returnval>>ASRFSTi_OUTPUT_FIFO_SHIFT;+}++staticintfsl_asrc_m2m_prepare(structfsl_asrc_pair*pair)+{+structfsl_asrc_pair_priv*pair_priv=pair->private;+structfsl_asrc*asrc=pair->asrc;+structdevice*dev=&asrc->pdev->dev;+structasrc_configconfig;+intret;++/* fill config */+config.pair=pair->index;+config.channel_num=pair->channels;+config.input_sample_rate=pair->rate[IN];+config.output_sample_rate=pair->rate[OUT];+config.input_format=pair->sample_format[IN];+config.output_format=pair->sample_format[OUT];+config.inclk=INCLK_NONE;+config.outclk=OUTCLK_ASRCK1_CLK;++pair_priv->config=&config;+ret=fsl_asrc_config_pair(pair,true);+if(ret){+dev_err(dev,"failed to config pair: %d\n",ret);+returnret;+}++pair->first_convert=1;++return0;+}++staticintfsl_asrc_m2m_start(structfsl_asrc_pair*pair)+{+if(pair->first_convert){+fsl_asrc_start_pair(pair);+pair->first_convert=0;+}+/*+*ClearDMArequestduringthestallstateofASRC:+*DuringSTALLstate,theremainingininputfifowouldneverbe+*smallerthantheinputthresholdwhiletheoutputfifowouldnot+*bebiggerthanoutputone.ThustheDMArequestwouldbecleared.+*/+fsl_asrc_set_watermarks(pair,ASRC_FIFO_THRESHOLD_MIN,+ASRC_FIFO_THRESHOLD_MAX);++/* Update the real input threshold to raise DMA request */+fsl_asrc_set_watermarks(pair,ASRC_M2M_INPUTFIFO_WML,+ASRC_M2M_OUTPUTFIFO_WML);++return0;+}++staticintfsl_asrc_m2m_stop(structfsl_asrc_pair*pair)+{+if(!pair->first_convert){+fsl_asrc_stop_pair(pair);+pair->first_convert=1;+}++return0;+}++/* calculate capture data length according to output data length and sample rate */+staticintfsl_asrc_m2m_calc_out_len(structfsl_asrc_pair*pair,intinput_buffer_length)+{+unsignedintin_width,out_width;+unsignedintchannels=pair->channels;+unsignedintin_samples,out_samples;+unsignedintout_length;++in_width=snd_pcm_format_physical_width(pair->sample_format[IN])/8;+out_width=snd_pcm_format_physical_width(pair->sample_format[OUT])/8;++in_samples=input_buffer_length/in_width/channels;+out_samples=pair->rate[OUT]*in_samples/pair->rate[IN];+out_length=(out_samples-ASRC_OUTPUT_LAST_SAMPLE)*out_width*channels;++returnout_length;+}++staticintfsl_asrc_m2m_get_maxburst(u8dir,structfsl_asrc_pair*pair)+{+structfsl_asrc*asrc=pair->asrc;+structfsl_asrc_priv*asrc_priv=asrc->private;+intwml=(dir==IN)?ASRC_M2M_INPUTFIFO_WML:ASRC_M2M_OUTPUTFIFO_WML;++if(!asrc_priv->soc->use_edma)+returnwml*pair->channels;+else+return1;+}++staticintfsl_asrc_m2m_get_cap(structfsl_asrc_m2m_cap*cap)+{+cap->fmt_in=FSL_ASRC_FORMATS;+cap->fmt_out=FSL_ASRC_FORMATS|SNDRV_PCM_FMTBIT_S8;+cap->rate_in=SNDRV_PCM_RATE_8000_192000|SNDRV_PCM_RATE_5512;+cap->rate_out=SNDRV_PCM_RATE_8000_192000|SNDRV_PCM_RATE_5512;+cap->chan_min=1;+cap->chan_max=10;++return0;+}++staticintfsl_asrc_m2m_pair_resume(structfsl_asrc_pair*pair)+{+structfsl_asrc*asrc=pair->asrc;+inti;++for(i=0;i<pair->channels*4;i++)+regmap_write(asrc->regmap,REG_ASRDI(pair->index),0);++pair->first_convert=1;+return0;+}+staticintfsl_asrc_runtime_resume(structdevice*dev);staticintfsl_asrc_runtime_suspend(structdevice*dev);
@@ -1147,6 +1277,15 @@ static int fsl_asrc_probe(struct platform_device *pdev)asrc->get_fifo_addr=fsl_asrc_get_fifo_addr;asrc->pair_priv_size=sizeof(structfsl_asrc_pair_priv);+asrc->m2m_prepare=fsl_asrc_m2m_prepare;+asrc->m2m_start=fsl_asrc_m2m_start;+asrc->m2m_stop=fsl_asrc_m2m_stop;+asrc->get_output_fifo_size=fsl_asrc_get_output_fifo_size;+asrc->m2m_calc_out_len=fsl_asrc_m2m_calc_out_len;+asrc->m2m_get_maxburst=fsl_asrc_m2m_get_maxburst;+asrc->m2m_pair_resume=fsl_asrc_m2m_pair_resume;+asrc->m2m_get_cap=fsl_asrc_m2m_get_cap;+if(of_device_is_compatible(np,"fsl,imx35-asrc")){asrc_priv->clk_map[IN]=input_clk_map_imx35;asrc_priv->clk_map[OUT]=output_clk_map_imx35;
From: Shengjiu Wang <hidden> Date: 2024-08-06 10:47:44
ASRC can be used on memory to memory case, define several
functions for m2m usage and export them as function pointer.
Signed-off-by: Shengjiu Wang <redacted>
---
sound/soc/fsl/fsl_easrc.c | 226 ++++++++++++++++++++++++++++++++++++++
sound/soc/fsl/fsl_easrc.h | 4 +
2 files changed, 230 insertions(+)
@@ -1861,6 +1861,222 @@ static int fsl_easrc_get_fifo_addr(u8 dir, enum asrc_pair_index index)returnREG_EASRC_FIFO(dir,index);}+/* Get sample numbers in FIFO */+staticunsignedintfsl_easrc_get_output_fifo_size(structfsl_asrc_pair*pair)+{+structfsl_asrc*asrc=pair->asrc;+enumasrc_pair_indexindex=pair->index;+u32val;++regmap_read(asrc->regmap,REG_EASRC_SFS(index),&val);+val&=EASRC_SFS_NSGO_MASK;++returnval>>EASRC_SFS_NSGO_SHIFT;+}++staticintfsl_easrc_m2m_prepare(structfsl_asrc_pair*pair)+{+structfsl_easrc_ctx_priv*ctx_priv=pair->private;+structfsl_asrc*asrc=pair->asrc;+structdevice*dev=&asrc->pdev->dev;+intret;++ctx_priv->in_params.sample_rate=pair->rate[IN];+ctx_priv->in_params.sample_format=pair->sample_format[IN];+ctx_priv->out_params.sample_rate=pair->rate[OUT];+ctx_priv->out_params.sample_format=pair->sample_format[OUT];++ctx_priv->in_params.fifo_wtmk=FSL_EASRC_INPUTFIFO_WML;+ctx_priv->out_params.fifo_wtmk=FSL_EASRC_OUTPUTFIFO_WML;+/* Fill the right half of the re-sampler with zeros */+ctx_priv->rs_init_mode=0x2;+/* Zero fill the right half of the prefilter */+ctx_priv->pf_init_mode=0x2;++ret=fsl_easrc_set_ctx_format(pair,+&ctx_priv->in_params.sample_format,+&ctx_priv->out_params.sample_format);+if(ret){+dev_err(dev,"failed to set context format: %d\n",ret);+returnret;+}++ret=fsl_easrc_config_context(asrc,pair->index);+if(ret){+dev_err(dev,"failed to config context %d\n",ret);+returnret;+}++ctx_priv->in_params.iterations=1;+ctx_priv->in_params.group_len=pair->channels;+ctx_priv->in_params.access_len=pair->channels;+ctx_priv->out_params.iterations=1;+ctx_priv->out_params.group_len=pair->channels;+ctx_priv->out_params.access_len=pair->channels;++ret=fsl_easrc_set_ctx_organziation(pair);+if(ret){+dev_err(dev,"failed to set fifo organization\n");+returnret;+}++/* The context start flag */+pair->first_convert=1;+return0;+}++staticintfsl_easrc_m2m_start(structfsl_asrc_pair*pair)+{+/* start context once */+if(pair->first_convert){+fsl_easrc_start_context(pair);+pair->first_convert=0;+}++return0;+}++staticintfsl_easrc_m2m_stop(structfsl_asrc_pair*pair)+{+/* Stop pair/context */+if(!pair->first_convert){+fsl_easrc_stop_context(pair);+pair->first_convert=1;+}++return0;+}++/* calculate capture data length according to output data length and sample rate */+staticintfsl_easrc_m2m_calc_out_len(structfsl_asrc_pair*pair,intinput_buffer_length)+{+structfsl_asrc*easrc=pair->asrc;+structfsl_easrc_priv*easrc_priv=easrc->private;+structfsl_easrc_ctx_priv*ctx_priv=pair->private;+unsignedintin_rate=ctx_priv->in_params.norm_rate;+unsignedintout_rate=ctx_priv->out_params.norm_rate;+unsignedintchannels=pair->channels;+unsignedintin_samples,out_samples;+unsignedintin_width,out_width;+unsignedintout_length;+unsignedintfrac_bits;+u64val1,val2;++switch(easrc_priv->rs_num_taps){+caseEASRC_RS_32_TAPS:+/* integer bits = 5; */+frac_bits=39;+break;+caseEASRC_RS_64_TAPS:+/* integer bits = 6; */+frac_bits=38;+break;+caseEASRC_RS_128_TAPS:+/* integer bits = 7; */+frac_bits=37;+break;+default:+return-EINVAL;+}++val1=(u64)in_rate<<frac_bits;+do_div(val1,out_rate);+val1+=(s64)ctx_priv->ratio_mod<<(frac_bits-31);++in_width=snd_pcm_format_physical_width(ctx_priv->in_params.sample_format)/8;+out_width=snd_pcm_format_physical_width(ctx_priv->out_params.sample_format)/8;++ctx_priv->in_filled_len+=input_buffer_length;+if(ctx_priv->in_filled_len<=ctx_priv->in_filled_sample*in_width*channels){+out_length=0;+}else{+in_samples=ctx_priv->in_filled_len/(in_width*channels)-+ctx_priv->in_filled_sample;++/* right shift 12 bit to make ratio in 32bit space */+val2=(u64)in_samples<<(frac_bits-12);+val1=val1>>12;+do_div(val2,val1);+out_samples=val2;++out_length=out_samples*out_width*channels;+ctx_priv->in_filled_len=ctx_priv->in_filled_sample*in_width*channels;+}++returnout_length;+}++staticintfsl_easrc_m2m_get_maxburst(u8dir,structfsl_asrc_pair*pair)+{+structfsl_easrc_ctx_priv*ctx_priv=pair->private;++if(dir==IN)+returnctx_priv->in_params.fifo_wtmk*pair->channels;+else+returnctx_priv->out_params.fifo_wtmk*pair->channels;+}++staticintfsl_easrc_m2m_pair_suspend(structfsl_asrc_pair*pair)+{+fsl_easrc_stop_context(pair);++return0;+}++staticintfsl_easrc_m2m_pair_resume(structfsl_asrc_pair*pair)+{+structfsl_easrc_ctx_priv*ctx_priv=pair->private;++pair->first_convert=1;+ctx_priv->in_filled_len=0;++return0;+}++/* val is Q31 */+staticintfsl_easrc_m2m_set_ratio_mod(structfsl_asrc_pair*pair,intval)+{+structfsl_easrc_ctx_priv*ctx_priv=pair->private;+structfsl_asrc*easrc=pair->asrc;+structfsl_easrc_priv*easrc_priv=easrc->private;+unsignedintfrac_bits;++ctx_priv->ratio_mod+=val;++switch(easrc_priv->rs_num_taps){+caseEASRC_RS_32_TAPS:+/* integer bits = 5; */+frac_bits=39;+break;+caseEASRC_RS_64_TAPS:+/* integer bits = 6; */+frac_bits=38;+break;+caseEASRC_RS_128_TAPS:+/* integer bits = 7; */+frac_bits=37;+break;+default:+return-EINVAL;+}++val<<=(frac_bits-31);+regmap_write(easrc->regmap,REG_EASRC_RUC(pair->index),EASRC_RSUC_RS_RM(val));++return0;+}++staticintfsl_easrc_m2m_get_cap(structfsl_asrc_m2m_cap*cap)+{+cap->fmt_in=FSL_EASRC_FORMATS;+cap->fmt_out=FSL_EASRC_FORMATS|SNDRV_PCM_FMTBIT_IEC958_SUBFRAME_LE;+cap->rate_in=SNDRV_PCM_RATE_8000_768000;+cap->rate_out=SNDRV_PCM_RATE_8000_768000;+cap->chan_min=1;+cap->chan_max=32;+return0;+}+staticconststructof_device_idfsl_easrc_dt_ids[]={{.compatible="fsl,imx8mn-easrc",},{}
@@ -1926,6 +2142,16 @@ static int fsl_easrc_probe(struct platform_device *pdev)easrc->release_pair=fsl_easrc_release_context;easrc->get_fifo_addr=fsl_easrc_get_fifo_addr;easrc->pair_priv_size=sizeof(structfsl_easrc_ctx_priv);+easrc->m2m_prepare=fsl_easrc_m2m_prepare;+easrc->m2m_start=fsl_easrc_m2m_start;+easrc->m2m_stop=fsl_easrc_m2m_stop;+easrc->get_output_fifo_size=fsl_easrc_get_output_fifo_size;+easrc->m2m_calc_out_len=fsl_easrc_m2m_calc_out_len;+easrc->m2m_get_maxburst=fsl_easrc_m2m_get_maxburst;+easrc->m2m_pair_suspend=fsl_easrc_m2m_pair_suspend;+easrc->m2m_pair_resume=fsl_easrc_m2m_pair_resume;+easrc->m2m_set_ratio_mod=fsl_easrc_m2m_set_ratio_mod;+easrc->m2m_get_cap=fsl_easrc_m2m_get_cap;easrc_priv->rs_num_taps=EASRC_RS_32_TAPS;easrc_priv->const_coeff=0x3FF0000000000000;
From: Shengjiu Wang <hidden> Date: 2024-08-06 10:47:46
Implement the ASRC memory to memory function using
the compress framework, user can use this function with
compress ioctl interface.
This feature can be shared by ASRC and EASRC drivers
Signed-off-by: Shengjiu Wang <redacted>
---
sound/soc/fsl/Kconfig | 1 +
sound/soc/fsl/Makefile | 2 +-
sound/soc/fsl/fsl_asrc_common.h | 9 +
sound/soc/fsl/fsl_asrc_m2m.c | 769 ++++++++++++++++++++++++++++++++
4 files changed, 780 insertions(+), 1 deletion(-)
create mode 100644 sound/soc/fsl/fsl_asrc_m2m.c
@@ -0,0 +1,769 @@+// SPDX-License-Identifier: GPL-2.0+//+// Copyright (C) 2014-2016 Freescale Semiconductor, Inc.+// Copyright (C) 2019-2024 NXP+//+// Freescale ASRC Memory to Memory (M2M) driver++#include<linux/dma/imx-dma.h>+#include<linux/dma-buf.h>+#include<linux/dma-mapping.h>+#include<linux/pm_runtime.h>+#include<sound/asound.h>+#include<sound/dmaengine_pcm.h>+#include<sound/initval.h>++#include"fsl_asrc_common.h"++#define DIR_STR(dir) (dir) == IN ? "in" : "out"++#define ASRC_xPUT_DMA_CALLBACK(dir) \+(((dir)==IN)?asrc_input_dma_callback\+:asrc_output_dma_callback)++/* Maximum output and capture buffer size */+#define ASRC_M2M_BUFFER_SIZE (512 * 1024)++/* Maximum output and capture period size */+#define ASRC_M2M_PERIOD_SIZE (48 * 1024)++/* dma complete callback */+staticvoidasrc_input_dma_callback(void*data)+{+structfsl_asrc_pair*pair=(structfsl_asrc_pair*)data;++complete(&pair->complete[IN]);+}++/* dma complete callback */+staticvoidasrc_output_dma_callback(void*data)+{+structfsl_asrc_pair*pair=(structfsl_asrc_pair*)data;++complete(&pair->complete[OUT]);+}++/**+*asrc_read_last_fifo:readalltheremainingdatafromFIFO+*@pair:Structurepointeroffsl_asrc_pair+*@dma_vaddr:virtualaddressofcapturebuffer+*@length:payloadlengthofcapturebuffer+*/+staticvoidasrc_read_last_fifo(structfsl_asrc_pair*pair,void*dma_vaddr,u32*length)+{+structfsl_asrc*asrc=pair->asrc;+enumasrc_pair_indexindex=pair->index;+u32i,reg,size,t_size=0,width;+u32*reg32=NULL;+u16*reg16=NULL;+u8*reg24=NULL;++width=snd_pcm_format_physical_width(pair->sample_format[OUT]);+if(width==32)+reg32=dma_vaddr+*length;+elseif(width==16)+reg16=dma_vaddr+*length;+else+reg24=dma_vaddr+*length;+retry:+size=asrc->get_output_fifo_size(pair);+if(size+*length>ASRC_M2M_BUFFER_SIZE)+gotoend;++for(i=0;i<size*pair->channels;i++){+regmap_read(asrc->regmap,asrc->get_fifo_addr(OUT,index),®);+if(reg32){+*reg32++=reg;+}elseif(reg16){+*reg16++=(u16)reg;+}else{+*reg24++=(u8)reg;+*reg24++=(u8)(reg>>8);+*reg24++=(u8)(reg>>16);+}+}+t_size+=size;++/* In case there is data left in FIFO */+if(size)+gotoretry;+end:+/* Update payload length */+if(reg32)+*length+=t_size*pair->channels*4;+elseif(reg16)+*length+=t_size*pair->channels*2;+else+*length+=t_size*pair->channels*3;+}++/* config dma channel */+staticintasrc_dmaconfig(structfsl_asrc_pair*pair,+structdma_chan*chan,+u32dma_addr,dma_addr_tbuf_addr,u32buf_len,+intdir,intwidth)+{+structfsl_asrc*asrc=pair->asrc;+structdevice*dev=&asrc->pdev->dev;+structdma_slave_configslave_config;+enumdma_slave_buswidthbuswidth;+unsignedintsg_len,max_period_size;+structscatterlist*sg;+intret,i;++switch(width){+case8:+buswidth=DMA_SLAVE_BUSWIDTH_1_BYTE;+break;+case16:+buswidth=DMA_SLAVE_BUSWIDTH_2_BYTES;+break;+case24:+buswidth=DMA_SLAVE_BUSWIDTH_3_BYTES;+break;+case32:+buswidth=DMA_SLAVE_BUSWIDTH_4_BYTES;+break;+default:+dev_err(dev,"invalid word width\n");+return-EINVAL;+}++memset(&slave_config,0,sizeof(slave_config));+if(dir==IN){+slave_config.direction=DMA_MEM_TO_DEV;+slave_config.dst_addr=dma_addr;+slave_config.dst_addr_width=buswidth;+slave_config.dst_maxburst=asrc->m2m_get_maxburst(IN,pair);+}else{+slave_config.direction=DMA_DEV_TO_MEM;+slave_config.src_addr=dma_addr;+slave_config.src_addr_width=buswidth;+slave_config.src_maxburst=asrc->m2m_get_maxburst(OUT,pair);+}++ret=dmaengine_slave_config(chan,&slave_config);+if(ret){+dev_err(dev,"failed to config dmaengine for %s task: %d\n",+DIR_STR(dir),ret);+return-EINVAL;+}++max_period_size=rounddown(ASRC_M2M_PERIOD_SIZE,width*pair->channels/8);+/* scatter gather mode */+sg_len=buf_len/max_period_size;+if(buf_len%max_period_size)+sg_len+=1;++sg=kmalloc_array(sg_len,sizeof(*sg),GFP_KERNEL);+if(!sg)+return-ENOMEM;++sg_init_table(sg,sg_len);+for(i=0;i<(sg_len-1);i++){+sg_dma_address(&sg[i])=buf_addr+i*max_period_size;+sg_dma_len(&sg[i])=max_period_size;+}+sg_dma_address(&sg[i])=buf_addr+i*max_period_size;+sg_dma_len(&sg[i])=buf_len-i*max_period_size;++pair->desc[dir]=dmaengine_prep_slave_sg(chan,sg,sg_len,+slave_config.direction,+DMA_PREP_INTERRUPT);+kfree(sg);+if(!pair->desc[dir]){+dev_err(dev,"failed to prepare dmaengine for %s task\n",DIR_STR(dir));+return-EINVAL;+}++pair->desc[dir]->callback=ASRC_xPUT_DMA_CALLBACK(dir);+pair->desc[dir]->callback_param=pair;++return0;+}++/* main function of converter */+staticvoidasrc_m2m_device_run(structfsl_asrc_pair*pair,structsnd_compr_task_runtime*task)+{+structfsl_asrc*asrc=pair->asrc;+structdevice*dev=&asrc->pdev->dev;+enumasrc_pair_indexindex=pair->index;+structsnd_dma_buffer*src_buf,*dst_buf;+unsignedintin_buf_len;+unsignedintout_dma_len;+unsignedintwidth;+u32fifo_addr;+intret;++/* set ratio mod */+if(asrc->m2m_set_ratio_mod){+if(pair->ratio_mod_flag){+asrc->m2m_set_ratio_mod(pair,pair->ratio_mod);+pair->ratio_mod_flag=false;+}+}++src_buf=&pair->dma_buffer[IN];+dst_buf=&pair->dma_buffer[OUT];++width=snd_pcm_format_physical_width(pair->sample_format[IN]);+fifo_addr=asrc->paddr+asrc->get_fifo_addr(IN,index);++in_buf_len=task->input_size;++if(in_buf_len<width*pair->channels/8||+in_buf_len>ASRC_M2M_BUFFER_SIZE||+in_buf_len%(width*pair->channels/8)){+dev_err(dev,"out buffer size is error: [%d]\n",in_buf_len);+gotoend;+}++/* dma config for output dma channel */+ret=asrc_dmaconfig(pair,+pair->dma_chan[IN],+fifo_addr,+src_buf->addr,+in_buf_len,IN,width);+if(ret){+dev_err(dev,"out dma config error\n");+gotoend;+}++width=snd_pcm_format_physical_width(pair->sample_format[OUT]);+fifo_addr=asrc->paddr+asrc->get_fifo_addr(OUT,index);+out_dma_len=asrc->m2m_calc_out_len(pair,in_buf_len);+if(out_dma_len>0&&out_dma_len<=ASRC_M2M_BUFFER_SIZE){+/* dma config for capture dma channel */+ret=asrc_dmaconfig(pair,+pair->dma_chan[OUT],+fifo_addr,+dst_buf->addr,+out_dma_len,OUT,width);+if(ret){+dev_err(dev,"cap dma config error\n");+gotoend;+}+}elseif(out_dma_len>ASRC_M2M_BUFFER_SIZE){+dev_err(dev,"cap buffer size error\n");+gotoend;+}++reinit_completion(&pair->complete[IN]);+reinit_completion(&pair->complete[OUT]);++/* Submit DMA request */+dmaengine_submit(pair->desc[IN]);+dma_async_issue_pending(pair->desc[IN]->chan);+if(out_dma_len>0){+dmaengine_submit(pair->desc[OUT]);+dma_async_issue_pending(pair->desc[OUT]->chan);+}++asrc->m2m_start(pair);++if(!wait_for_completion_interruptible_timeout(&pair->complete[IN],10*HZ)){+dev_err(dev,"out DMA task timeout\n");+gotoend;+}++if(out_dma_len>0){+if(!wait_for_completion_interruptible_timeout(&pair->complete[OUT],10*HZ)){+dev_err(dev,"cap DMA task timeout\n");+gotoend;+}+}++/* read the last words from FIFO */+asrc_read_last_fifo(pair,dst_buf->area,&out_dma_len);+/* update payload length for capture */+task->output_size=out_dma_len;+end:+return;+}++staticintfsl_asrc_m2m_comp_open(structsnd_compr_stream*stream)+{+structfsl_asrc*asrc=stream->private_data;+structsnd_compr_runtime*runtime=stream->runtime;+structdevice*dev=&asrc->pdev->dev;+structfsl_asrc_pair*pair;+intsize,ret;++pair=kzalloc(sizeof(*pair)+asrc->pair_priv_size,GFP_KERNEL);+if(!pair)+return-ENOMEM;++pair->private=(void*)pair+sizeof(structfsl_asrc_pair);+pair->asrc=asrc;++init_completion(&pair->complete[IN]);+init_completion(&pair->complete[OUT]);++runtime->private_data=pair;++size=ASRC_M2M_BUFFER_SIZE;+ret=snd_dma_alloc_pages(SNDRV_DMA_TYPE_DEV,dev,size,&pair->dma_buffer[IN]);+if(ret)+gotoerror_alloc_in_buf;++ret=snd_dma_alloc_pages(SNDRV_DMA_TYPE_DEV,dev,size,&pair->dma_buffer[OUT]);+if(ret)+gotoerror_alloc_out_buf;++ret=pm_runtime_get_sync(dev);+if(ret<0){+dev_err(dev,"Failed to power up asrc\n");+gotoerr_pm_runtime;+}++return0;++err_pm_runtime:+snd_dma_free_pages(&pair->dma_buffer[OUT]);+error_alloc_out_buf:+snd_dma_free_pages(&pair->dma_buffer[IN]);+error_alloc_in_buf:+kfree(pair);+returnret;+}++staticintfsl_asrc_m2m_comp_release(structsnd_compr_stream*stream)+{+structfsl_asrc*asrc=stream->private_data;+structsnd_compr_runtime*runtime=stream->runtime;+structfsl_asrc_pair*pair=runtime->private_data;+structdevice*dev=&asrc->pdev->dev;++pm_runtime_put_sync(dev);++snd_dma_free_pages(&pair->dma_buffer[IN]);+snd_dma_free_pages(&pair->dma_buffer[OUT]);++kfree(runtime->private_data);++return0;+}++staticintfsl_asrc_m2m_comp_set_params(structsnd_compr_stream*stream,+structsnd_compr_params*params)+{+structfsl_asrc*asrc=stream->private_data;+structsnd_compr_runtime*runtime=stream->runtime;+structfsl_asrc_pair*pair=runtime->private_data;+structfsl_asrc_m2m_capcap;+intret;++ret=asrc->m2m_get_cap(&cap);+if(ret)+return-EINVAL;++if(pcm_format_to_bits(params->codec.format)&cap.fmt_in)+pair->sample_format[IN]=params->codec.format;+else+return-EINVAL;++if(pcm_format_to_bits(params->codec.options.src.format_out)&cap.fmt_out)+pair->sample_format[OUT]=params->codec.options.src.format_out;+else+return-EINVAL;++if(snd_pcm_rate_to_rate_bit(params->codec.sample_rate)&cap.rate_in)+pair->rate[IN]=params->codec.sample_rate;+else+return-EINVAL;+if(snd_pcm_rate_to_rate_bit(params->codec.options.src.rate_out)&cap.rate_out)+pair->rate[OUT]=params->codec.options.src.rate_out;+else+return-EINVAL;++if(params->codec.ch_in!=params->codec.ch_out||+params->codec.ch_in<cap.chan_min||+params->codec.ch_in>cap.chan_max)+return-EINVAL;++pair->channels=params->codec.ch_in;+pair->buf_len[IN]=params->buffer.fragment_size;+pair->buf_len[OUT]=params->buffer.fragment_size;++return0;+}++staticintfsl_asrc_m2m_mmap(structdma_buf*dmabuf,structvm_area_struct*vma)+{+structsnd_dma_buffer*dmab=dmabuf->priv;++returnsnd_dma_buffer_mmap(dmab,vma);+}++staticstructsg_table*fsl_asrc_m2m_map_dma_buf(structdma_buf_attachment*attachment,+enumdma_data_directiondirection)+{+structsnd_dma_buffer*dmab=attachment->dmabuf->priv;+structsg_table*sgt;++sgt=kmalloc(sizeof(*sgt),GFP_KERNEL);+if(!sgt)+returnNULL;++if(dma_get_sgtable(attachment->dev,sgt,dmab->area,dmab->addr,dmab->bytes)<0)+gotofree;++if(dma_map_sgtable(attachment->dev,sgt,direction,0))+gotofree;++returnsgt;++free:+sg_free_table(sgt);+kfree(sgt);+returnNULL;+}++staticvoidfsl_asrc_m2m_unmap_dma_buf(structdma_buf_attachment*attachment,+structsg_table*table,+enumdma_data_directiondirection)+{+dma_unmap_sgtable(attachment->dev,table,direction,0);+}++staticvoidfsl_asrc_m2m_release(structdma_buf*dmabuf)+{+/* buffer is released by fsl_asrc_m2m_comp_release() */+}++staticconststructdma_buf_opsfsl_asrc_m2m_dma_buf_ops={+.mmap=fsl_asrc_m2m_mmap,+.map_dma_buf=fsl_asrc_m2m_map_dma_buf,+.unmap_dma_buf=fsl_asrc_m2m_unmap_dma_buf,+.release=fsl_asrc_m2m_release,+};++staticintfsl_asrc_m2m_comp_task_create(structsnd_compr_stream*stream,+structsnd_compr_task_runtime*task)+{+DEFINE_DMA_BUF_EXPORT_INFO(exp_info_in);+DEFINE_DMA_BUF_EXPORT_INFO(exp_info_out);+structfsl_asrc*asrc=stream->private_data;+structsnd_compr_runtime*runtime=stream->runtime;+structfsl_asrc_pair*pair=runtime->private_data;+structdevice*dev=&asrc->pdev->dev;+intret;++exp_info_in.ops=&fsl_asrc_m2m_dma_buf_ops;+exp_info_in.size=ASRC_M2M_BUFFER_SIZE;+exp_info_in.flags=O_RDWR;+exp_info_in.priv=&pair->dma_buffer[IN];+task->input=dma_buf_export(&exp_info_in);+if(IS_ERR(task->input)){+ret=PTR_ERR(task->input);+returnret;+}++exp_info_out.ops=&fsl_asrc_m2m_dma_buf_ops;+exp_info_out.size=ASRC_M2M_BUFFER_SIZE;+exp_info_out.flags=O_RDWR;+exp_info_out.priv=&pair->dma_buffer[OUT];+task->output=dma_buf_export(&exp_info_out);+if(IS_ERR(task->output)){+ret=PTR_ERR(task->output);+returnret;+}++/* Request asrc pair/context */+ret=asrc->request_pair(pair->channels,pair);+if(ret){+dev_err(dev,"failed to request pair: %d\n",ret);+gotoerr_request_pair;+}++ret=asrc->m2m_prepare(pair);+if(ret){+dev_err(dev,"failed to start pair part one: %d\n",ret);+gotoerr_start_part_one;+}++/* Request dma channels */+pair->dma_chan[IN]=asrc->get_dma_channel(pair,IN);+if(!pair->dma_chan[IN]){+dev_err(dev,"[ctx%d] failed to get input DMA channel\n",pair->index);+ret=-EBUSY;+gotoerr_dma_channel_in;+}++pair->dma_chan[OUT]=asrc->get_dma_channel(pair,OUT);+if(!pair->dma_chan[OUT]){+dev_err(dev,"[ctx%d] failed to get output DMA channel\n",pair->index);+ret=-EBUSY;+gotoerr_dma_channel_out;+}++return0;++err_dma_channel_out:+dma_release_channel(pair->dma_chan[IN]);+err_dma_channel_in:+if(asrc->m2m_unprepare)+asrc->m2m_unprepare(pair);+err_start_part_one:+asrc->release_pair(pair);+err_request_pair:+returnret;+}++staticintfsl_asrc_m2m_comp_task_start(structsnd_compr_stream*stream,+structsnd_compr_task_runtime*task)+{+structsnd_compr_runtime*runtime=stream->runtime;+structfsl_asrc_pair*pair=runtime->private_data;++asrc_m2m_device_run(pair,task);++return0;+}++staticintfsl_asrc_m2m_comp_task_stop(structsnd_compr_stream*stream,+structsnd_compr_task_runtime*task)+{+return0;+}++staticintfsl_asrc_m2m_comp_task_free(structsnd_compr_stream*stream,+structsnd_compr_task_runtime*task)+{+structfsl_asrc*asrc=stream->private_data;+structsnd_compr_runtime*runtime=stream->runtime;+structfsl_asrc_pair*pair=runtime->private_data;++/* Stop & release pair/context */+if(asrc->m2m_stop)+asrc->m2m_stop(pair);++if(asrc->m2m_unprepare)+asrc->m2m_unprepare(pair);+asrc->release_pair(pair);++/* Release dma channel */+if(pair->dma_chan[IN])+dma_release_channel(pair->dma_chan[IN]);+if(pair->dma_chan[OUT])+dma_release_channel(pair->dma_chan[OUT]);++return0;+}++staticintfsl_asrc_m2m_get_caps(structsnd_compr_stream*cstream,+structsnd_compr_caps*caps)+{+caps->num_codecs=1;+caps->min_fragment_size=4096;+caps->max_fragment_size=4096;+caps->min_fragments=1;+caps->max_fragments=1;+caps->codecs[0]=SND_AUDIOCODEC_SRC;++return0;+}++staticintfsl_asrc_m2m_fill_codec_caps(structfsl_asrc*asrc,+structsnd_compr_codec_caps*codec)+{+structfsl_asrc_m2m_capcap;+__u32rates[MAX_NUM_BITRATES];+snd_pcm_format_tk;+inti=0,j=0;+intret;++ret=asrc->m2m_get_cap(&cap);+if(ret)+return-EINVAL;++if(cap.rate_in&SNDRV_PCM_RATE_5512)+rates[i++]=snd_pcm_rate_bit_to_rate(SNDRV_PCM_RATE_5512);+if(cap.rate_in&SNDRV_PCM_RATE_8000)+rates[i++]=snd_pcm_rate_bit_to_rate(SNDRV_PCM_RATE_8000);+if(cap.rate_in&SNDRV_PCM_RATE_11025)+rates[i++]=snd_pcm_rate_bit_to_rate(SNDRV_PCM_RATE_11025);+if(cap.rate_in&SNDRV_PCM_RATE_16000)+rates[i++]=snd_pcm_rate_bit_to_rate(SNDRV_PCM_RATE_16000);+if(cap.rate_in&SNDRV_PCM_RATE_22050)+rates[i++]=snd_pcm_rate_bit_to_rate(SNDRV_PCM_RATE_22050);+if(cap.rate_in&SNDRV_PCM_RATE_32000)+rates[i++]=snd_pcm_rate_bit_to_rate(SNDRV_PCM_RATE_32000);+if(cap.rate_in&SNDRV_PCM_RATE_44100)+rates[i++]=snd_pcm_rate_bit_to_rate(SNDRV_PCM_RATE_44100);+if(cap.rate_in&SNDRV_PCM_RATE_48000)+rates[i++]=snd_pcm_rate_bit_to_rate(SNDRV_PCM_RATE_48000);+if(cap.rate_in&SNDRV_PCM_RATE_88200)+rates[i++]=snd_pcm_rate_bit_to_rate(SNDRV_PCM_RATE_88200);+if(cap.rate_in&SNDRV_PCM_RATE_96000)+rates[i++]=snd_pcm_rate_bit_to_rate(SNDRV_PCM_RATE_96000);+if(cap.rate_in&SNDRV_PCM_RATE_176400)+rates[i++]=snd_pcm_rate_bit_to_rate(SNDRV_PCM_RATE_176400);+if(cap.rate_in&SNDRV_PCM_RATE_192000)+rates[i++]=snd_pcm_rate_bit_to_rate(SNDRV_PCM_RATE_192000);+if(cap.rate_in&SNDRV_PCM_RATE_352800)+rates[i++]=snd_pcm_rate_bit_to_rate(SNDRV_PCM_RATE_352800);+if(cap.rate_in&SNDRV_PCM_RATE_384000)+rates[i++]=snd_pcm_rate_bit_to_rate(SNDRV_PCM_RATE_384000);+if(cap.rate_in&SNDRV_PCM_RATE_705600)+rates[i++]=snd_pcm_rate_bit_to_rate(SNDRV_PCM_RATE_705600);+if(cap.rate_in&SNDRV_PCM_RATE_768000)+rates[i++]=snd_pcm_rate_bit_to_rate(SNDRV_PCM_RATE_768000);++pcm_for_each_format(k){+if(pcm_format_to_bits(k)&cap.fmt_in){+codec->descriptor[j].max_ch=cap.chan_max;+memcpy(codec->descriptor[j].sample_rates,rates,i*sizeof(__u32));+codec->descriptor[j].num_sample_rates=i;+codec->descriptor[j].formats=k;+j++;+}+}++codec->codec=SND_AUDIOCODEC_SRC;+codec->num_descriptors=j;+return0;+}++staticintfsl_asrc_m2m_get_codec_caps(structsnd_compr_stream*stream,+structsnd_compr_codec_caps*codec)+{+structfsl_asrc*asrc=stream->private_data;++returnfsl_asrc_m2m_fill_codec_caps(asrc,codec);+}++staticintfsl_asrc_m2m_comp_set_metadata(structsnd_compr_stream*stream,+structsnd_compr_metadata*metadata)+{+structsnd_compr_runtime*runtime=stream->runtime;+structfsl_asrc_pair*pair=runtime->private_data;+intret=0;++switch(metadata->key){+caseSNDRV_COMPRESS_SRC_RATIO_MOD:+if(pair->ratio_mod_flag)+return-EINVAL;+pair->ratio_mod=metadata->value[0];+pair->ratio_mod_flag=true;+break;+default:+ret=-EINVAL;+break;+}++returnret;+}++staticstructsnd_compr_opsfsl_asrc_m2m_compr_ops={+.open=fsl_asrc_m2m_comp_open,+.free=fsl_asrc_m2m_comp_release,+.set_params=fsl_asrc_m2m_comp_set_params,+.set_metadata=fsl_asrc_m2m_comp_set_metadata,+.get_caps=fsl_asrc_m2m_get_caps,+.get_codec_caps=fsl_asrc_m2m_get_codec_caps,+.task_create=fsl_asrc_m2m_comp_task_create,+.task_start=fsl_asrc_m2m_comp_task_start,+.task_stop=fsl_asrc_m2m_comp_task_stop,+.task_free=fsl_asrc_m2m_comp_task_free,+};++intfsl_asrc_m2m_suspend(structfsl_asrc*asrc)+{+structfsl_asrc_pair*pair;+inti;++for(i=0;i<PAIR_CTX_NUM;i++){+pair=asrc->pair[i];+if(!pair)+continue;+if(!completion_done(&pair->complete[IN])){+if(pair->dma_chan[IN])+dmaengine_terminate_all(pair->dma_chan[IN]);+asrc_input_dma_callback((void*)pair);+}+if(!completion_done(&pair->complete[OUT])){+if(pair->dma_chan[OUT])+dmaengine_terminate_all(pair->dma_chan[OUT]);+asrc_output_dma_callback((void*)pair);+}++if(asrc->m2m_pair_suspend)+asrc->m2m_pair_suspend(pair);+}++return0;+}+EXPORT_SYMBOL_GPL(fsl_asrc_m2m_suspend);++intfsl_asrc_m2m_resume(structfsl_asrc*asrc)+{+structfsl_asrc_pair*pair;+inti;++for(i=0;i<PAIR_CTX_NUM;i++){+pair=asrc->pair[i];+if(!pair)+continue;+if(asrc->m2m_pair_resume)+asrc->m2m_pair_resume(pair);+}++return0;+}+EXPORT_SYMBOL_GPL(fsl_asrc_m2m_resume);++intfsl_asrc_m2m_init(structfsl_asrc*asrc)+{+structdevice*dev=&asrc->pdev->dev;+structsnd_card*card;+structsnd_compr*compr;+intret;++ret=snd_card_new(dev,SNDRV_DEFAULT_IDX1,SNDRV_DEFAULT_STR1,+THIS_MODULE,0,&card);+if(ret<0)+returnret;++strscpy(card->driver,"fsl-asrc-m2m",sizeof(card->driver));+strscpy(card->shortname,"ASRC-M2M",sizeof(card->shortname));+strscpy(card->longname,"ASRC-M2M",sizeof(card->shortname));++asrc->card=card;++compr=devm_kzalloc(dev,sizeof(*compr),GFP_KERNEL);+if(!compr){+ret=-ENOMEM;+gotoerr;+}++compr->ops=&fsl_asrc_m2m_compr_ops;+compr->private_data=asrc;++ret=snd_compress_new(card,0,SND_COMPRESS_ACCEL,"ASRC M2M",compr);+if(ret<0)+gotoerr;++ret=snd_card_register(card);+if(ret<0)+gotoerr;++return0;+err:+snd_card_free(card);+returnret;+}+EXPORT_SYMBOL_GPL(fsl_asrc_m2m_init);++voidfsl_asrc_m2m_exit(structfsl_asrc*asrc)+{+structsnd_card*card=asrc->card;++snd_card_free(card);+}+EXPORT_SYMBOL_GPL(fsl_asrc_m2m_exit);++MODULE_IMPORT_NS(DMA_BUF);+MODULE_AUTHOR("Shengjiu Wang <Shengjiu.Wang@nxp.com>");+MODULE_DESCRIPTION("Freescale ASRC M2M driver");+MODULE_LICENSE("GPL");