On Thu, Aug 26, 2021 at 11:13:38AM +0200, Peter Rosin wrote:

On 2021-08-20 21:17, Liam Beguin wrote:

quoted

From: Liam Beguin <redacted>

Reduce the risk of integer overflow by doing the scale calculation on
a 64-bit integer. Since the rescaling is only performed on *val, reuse
the IIO_VAL_FRACTIONAL_LOG2 case.

While this patch certainly helps with overflow problems, it also
potentially kills precision in some cases where there currently are
no overflow issues.

E.g. this patch transforms 5/32768 scaled by 3/10000 from the exact

15 / 327680000 (0.0000000457763671875)

to the heavily truncated plain old sorry "zero".

Sure, 9/14 improves the situation, but patch 9/14 simply cannot
make this example any better than returning 2 significant digits
since the value is so small.

The 100 ppm check introduced in 09/14 is really objective and might not
be the best choice. Changing it to

	- if (abs(rem) > 10000000 && abs(div64_s64(*val, tmp)) < 100) {
	+ if (abs(rem)) {

Helps with the precision issues you brought up here, and in 09/14.
I was originally trying to keep the original scale as much as possible,
I'll continue the rest of the discussion on the 09/14 thread we already
have.

Side note, there is also the same type of risk of overflow for
IIO_VAL_INT. Why does that case not get the same treatment as
IIO_VAL_FRACTIONAL?

Being totally honest, I noticed we have the same issue with IIO_VAL_INT,
but since I didn't run into the issue on my setup I left it out to focus
on getting the rest cleaned up.

I guess it couldn't hurt to fix that too while we're at it.
I'll work on it!

But again, I see no elegant solution. The best I can think of is the
inelegant solution to provide extra info on the input range, the
exact desired scaling method, the desired output type, some mix of
all of the above or something else that helps determining the
appropriate scaling method w/o looking at the individual number.

I don't really like having to add a range parameter.
If changing the scale type dynamically isn't an issue, I think we can
get away with not adding a parameter.
If it is an issue, we might have to look into it...

Thanks,
Liam

Cheers,
Peter

quoted

Signed-off-by: Liam Beguin <redacted>
---
 drivers/iio/afe/iio-rescale.c | 5 +----
 1 file changed, 1 insertion(+), 4 deletions(-)

diff --git a/drivers/iio/afe/iio-rescale.c b/drivers/iio/afe/iio-rescale.c
index 809e966f7058..c408c4057c08 100644
--- a/drivers/iio/afe/iio-rescale.c
+++ b/drivers/iio/afe/iio-rescale.c

@@ -27,16 +27,13 @@ int rescale_process_scale(struct rescale *rescale, int scale_type,
 	u32 neg;
 
 	switch (scale_type) {
-	case IIO_VAL_FRACTIONAL:
-		*val *= rescale->numerator;
-		*val2 *= rescale->denominator;
-		return scale_type;
 	case IIO_VAL_INT:
 		*val *= rescale->numerator;
 		if (rescale->denominator == 1)
 			return scale_type;
 		*val2 = rescale->denominator;
 		return IIO_VAL_FRACTIONAL;
+	case IIO_VAL_FRACTIONAL:
 	case IIO_VAL_FRACTIONAL_LOG2:
 		tmp = (s64)*val * 1000000000LL;
 		tmp = div_s64(tmp, rescale->denominator);