Thread (14 messages) 14 messages, 4 authors, 2019-08-20

Re: Bisected: Kernel 4.14 + has 3 times higher write IO latency than Kernel 4.4 with raid1

From: Jinpu Wang <jinpu.wang@cloud.ionos.com>
Date: 2019-08-16 08:11:00
Also in: lkml

On Wed, Aug 7, 2019 at 2:35 PM Jinpu Wang [off-list ref] wrote:
On Wed, Aug 7, 2019 at 8:36 AM Jinpu Wang [off-list ref] wrote:
quoted
On Wed, Aug 7, 2019 at 1:40 AM NeilBrown [off-list ref] wrote:
quoted
On Tue, Aug 06 2019, Jinpu Wang wrote:
quoted
On Tue, Aug 6, 2019 at 9:54 AM Jinpu Wang [off-list ref] wrote:
quoted
On Tue, Aug 6, 2019 at 1:46 AM NeilBrown [off-list ref] wrote:
quoted
On Mon, Aug 05 2019, Jinpu Wang wrote:
quoted
Hi Neil,

For the md higher write IO latency problem, I bisected it to these commits:

4ad23a97 MD: use per-cpu counter for writes_pending
210f7cd percpu-refcount: support synchronous switch to atomic mode.

Do you maybe have an idea? How can we fix it?
Hmmm.... not sure.
Hi Neil,

Thanks for reply, detailed result in line.
Thanks for the extra testing.
...
quoted
[  105.133299] md md0 in_sync is 0, sb_flags 2, recovery 3, external
0, safemode 0, recovery_cp 524288
...

ahh - the resync was still happening.  That explains why set_in_sync()
is being called so often.  If you wait for sync to complete (or create
the array with --assume-clean) you should see more normal behaviour.
I've updated my tests accordingly, thanks for the hint.
quoted
This patch should fix it.  I think we can do better but it would be more
complex so no suitable for backports to -stable.

Once you confirm it works, I'll send it upstream with a
Reported-and-Tested-by from you.

Thanks,
NeilBrown
Thanks a lot, Neil, my quick test show, yes, it fixed the problem for me.

I will run more tests to be sure, will report back the test result.
Hi Neil,

I've run our regression tests with your patch, everything works fine
as expected.

So Reported-and-Tested-by: Jack Wang [off-list ref]

Thank you for your quick fix.

The patch should go to stable 4.12+
Hi Neil,

I hope you're doing well, just a soft ping? do you need further
testing from my side?

Please let me know how can we move the fix forward.

Thanks,
Jack Wang
Keyboard shortcuts
hback out one level
jnext message in thread
kprevious message in thread
ldrill in
Escclose help / fold thread tree
?toggle this help