On Tue, Jan 07, 2020 at 12:34:50PM +0100, Christian Borntraeger wrote:

On 07.01.20 10:39, Michael S. Tsirkin wrote:

quoted

On Tue, Jan 07, 2020 at 09:59:16AM +0100, Christian Borntraeger wrote:

quoted

On 06.01.20 11:50, Michael S. Tsirkin wrote:

quoted

On Wed, Dec 18, 2019 at 04:59:02PM +0100, Christian Borntraeger wrote:

quoted

On 18.12.19 16:10, Michael S. Tsirkin wrote:

quoted

On Wed, Dec 18, 2019 at 03:43:43PM +0100, Christian Borntraeger wrote:

quoted

Michael,

with 
commit db7286b100b503ef80612884453bed53d74c9a16 (refs/bisect/skip-db7286b100b503ef80612884453bed53d74c9a16)
    vhost: use batched version by default
plus
commit 6bd262d5eafcdf8cdfae491e2e748e4e434dcda6 (HEAD, refs/bisect/bad)
    Revert "vhost/net: add an option to test new code"
to make things compile (your next tree is not easily bisectable, can you fix that as well?).

I'll try.

quoted

I get random crashes in my s390 KVM guests after reboot.
Reverting both patches together with commit decd9b8 "vhost: use vhost_desc instead of vhost_log" to
make it compile again) on top of linux-next-1218 makes the problem go away.

Looks like the batched version is not yet ready for prime time. Can you drop these patches until
we have fixed the issues?

Christian

Will do, thanks for letting me know.

I have confirmed with the initial reporter (internal test team) that <driver name='qemu'/> 
with a known to be broken linux next kernel also fixes the problem, so it is really the
vhost changes.

OK I'm back and trying to make it more bisectable.

I pushed a new tag "batch-v2".
It's same code but with this bisect should get more information.

I get the following with this tag

drivers/vhost/net.c: In function ‘vhost_net_tx_get_vq_desc’:
drivers/vhost/net.c:574:7: error: implicit declaration of function ‘vhost_get_vq_desc_batch’; did you mean ‘vhost_get_vq_desc’? [-Werror=implicit-function-declaration]
  574 |   r = vhost_get_vq_desc_batch(tvq, tvq->iov, ARRAY_SIZE(tvq->iov),
      |       ^~~~~~~~~~~~~~~~~~~~~~~
      |       vhost_get_vq_desc

Not sure why but I pushed a wrong commit. Sorry. Should be good now.

during bisect:

drivers/vhost/vhost.c: In function ‘vhost_get_vq_desc_batch’:
drivers/vhost/vhost.c:2634:8: error: ‘id’ undeclared (first use in this function); did you mean ‘i’?
 2634 |  ret = id;
      |        ^~
      |        i

I changed that to i

Hmm no that's wrong I think. Sorry about all the errors. Let me push a
fixed v3.

The last step then gave me  (on commit 50297a8480b439efc5f3f23088cb2d90b799acef vhost: use batched version by default)
net enc1: Unexpected TXQ (0) queue failure: -5
in the guest.

bisect log so far:
[cborntra@m83lp52 linux]$ git bisect log
git bisect start
# bad: [3131e79bb9e9892a5a6bd33513de9bc90b20e867] vhost: use vhost_desc instead of vhost_log
git bisect bad 3131e79bb9e9892a5a6bd33513de9bc90b20e867
# good: [d1281e3a562ec6a08f944a876481dd043ba739b9] virtio-blk: remove VIRTIO_BLK_F_SCSI support
git bisect good d1281e3a562ec6a08f944a876481dd043ba739b9
# good: [5b00aab5b6332a67e32dace1dcd3a198ab94ed56] vhost: option to fetch descriptors through an independent struct
git bisect good 5b00aab5b6332a67e32dace1dcd3a198ab94ed56
# good: [5b00aab5b6332a67e32dace1dcd3a198ab94ed56] vhost: option to fetch descriptors through an independent struct
git bisect good 5b00aab5b6332a67e32dace1dcd3a198ab94ed56
# bad: [1414d7ee3d10d2ec2bc4ee652d1d90ec91da1c79] vhost: batching fetches
git bisect bad 1414d7ee3d10d2ec2bc4ee652d1d90ec91da1c79