Thread (38 messages) 38 messages, 2 authors, 2022-01-08

Re: [PATCH v11 08/15] iio: afe: rescale: fix accuracy for small fractional scales

From: Andy Shevchenko <hidden>
Date: 2021-12-22 12:29:07
Also in: linux-iio, lkml

On Wed, Dec 22, 2021 at 5:47 AM Liam Beguin [off-list ref] wrote:
From: Liam Beguin <redacted>

The approximation caused by integer divisions can be costly on smaller
scale values since the decimal part is significant compared to the
integer part. Switch to an IIO_VAL_INT_PLUS_NANO scale type in such
cases to maintain accuracy.
...
-               tmp = div_s64(tmp, 1000000000LL);
+
+               tmp = div_s64_rem(tmp, 1000000000LL, &rem);
                *val = tmp;
-               return scale_type;
+
It seems you may add this blank line to one of the previous patches, may you?
+               if (!rem)
+                       return scale_type;
-- 
With Best Regards,
Andy Shevchenko
Keyboard shortcuts
hback out one level
jnext message in thread
kprevious message in thread
ldrill in
Escclose help / fold thread tree
?toggle this help