Re: vhost changes (batched) in linux-next after 12/13 trigger random crashes in KVM guests after reboot

On Wed, Dec 18, 2019 at 04:59:02PM +0100, Christian Borntraeger wrote:

On 18.12.19 16:10, Michael S. Tsirkin wrote:

quoted

On Wed, Dec 18, 2019 at 03:43:43PM +0100, Christian Borntraeger wrote:

quoted

Michael,

with 
commit db7286b100b503ef80612884453bed53d74c9a16 (refs/bisect/skip-db7286b100b503ef80612884453bed53d74c9a16)
    vhost: use batched version by default
plus
commit 6bd262d5eafcdf8cdfae491e2e748e4e434dcda6 (HEAD, refs/bisect/bad)
    Revert "vhost/net: add an option to test new code"
to make things compile (your next tree is not easily bisectable, can you fix that as well?).

I'll try.

quoted

I get random crashes in my s390 KVM guests after reboot.
Reverting both patches together with commit decd9b8 "vhost: use vhost_desc instead of vhost_log" to
make it compile again) on top of linux-next-1218 makes the problem go away.

Looks like the batched version is not yet ready for prime time. Can you drop these patches until
we have fixed the issues?

Christian

Will do, thanks for letting me know.

I have confirmed with the initial reporter (internal test team) that <driver name='qemu'/> 
with a known to be broken linux next kernel also fixes the problem, so it is really the
vhost changes.

OK I'm back and trying to make it more bisectable.

I pushed a new tag "batch-v2".
It's same code but with this bisect should get more information.


I suspect one of the following:

commit 1414d7ee3d10d2ec2bc4ee652d1d90ec91da1c79
Author: Michael S. Tsirkin [off-list ref]
Date:   Mon Oct 7 06:11:18 2019 -0400

    vhost: batching fetches
    
    With this patch applied, new and old code perform identically.
    
    Lots of extra optimizations are now possible, e.g.
    we can fetch multiple heads with copy_from/to_user now.
    We can get rid of maintaining the log array.  Etc etc.
    
    Signed-off-by: Michael S. Tsirkin [off-list ref]

commit 50297a8480b439efc5f3f23088cb2d90b799acef
Author: Michael S. Tsirkin [off-list ref]
Date:   Wed Dec 11 12:19:26 2019 -0500

    vhost: use batched version by default
    
    As testing shows no performance change, switch to that now.
    
    Signed-off-by: Michael S. Tsirkin [off-list ref]


and would like to know which.

Thanks!