Re: [PATCH v11 08/15] iio: afe: rescale: fix accuracy for small fractional scales
From: Andy Shevchenko <hidden>
Date: 2021-12-22 12:29:07
Also in:
linux-iio, lkml
From: Andy Shevchenko <hidden>
Date: 2021-12-22 12:29:07
Also in:
linux-iio, lkml
On Wed, Dec 22, 2021 at 5:47 AM Liam Beguin [off-list ref] wrote:
From: Liam Beguin <redacted> The approximation caused by integer divisions can be costly on smaller scale values since the decimal part is significant compared to the integer part. Switch to an IIO_VAL_INT_PLUS_NANO scale type in such cases to maintain accuracy.
...
- tmp = div_s64(tmp, 1000000000LL);
+
+ tmp = div_s64_rem(tmp, 1000000000LL, &rem);
*val = tmp;- return scale_type; +
It seems you may add this blank line to one of the previous patches, may you?
+ if (!rem) + return scale_type;
-- With Best Regards, Andy Shevchenko