Re: [V9fs-developer] 9pfs hangs since 4.7

On Sat, 7 Jan 2017 17:19:10 +0000
Al Viro [off-list ref] wrote:

On Sat, Jan 07, 2017 at 04:10:45PM +0100, Greg Kurz wrote:

quoted

quoted

virtqueue_push(), but pdu freeing is delayed until v9fs_flush() gets woken
up.  In the meanwhile, another request arrives into the slot of freed by
that virtqueue_push() and we are out of pdus.
  

Indeed. Even if this doesn't seem to be the problem here, I guess this should
be fixed.  

	FWIW, there's something that looks like an off-by-one in
v9fs_device_realize_common():
    /* initialize pdu allocator */
    QLIST_INIT(&s->free_list);
    QLIST_INIT(&s->active_list);
    for (i = 0; i < (MAX_REQ - 1); i++) {
        QLIST_INSERT_HEAD(&s->free_list, &s->pdus[i], next);
        s->pdus[i].s = s;
        s->pdus[i].idx = i;
    }

Had been there since the original merge of 9p support into qemu - that code
had moved around a bit, but it had never inserted s->pdus[MAX_REQ - 1] into
free list.  So your scenario with failing pdu_alloc() is still possible.
In that log the total amount of pending requests has reached 128 for the
first time right when the requests had stopped being handled and even
though it had dropped below that shortly after, extra requests being put
into queue had not been processed at all...

Yes, this does seem to be related to this or otherwise MAX_REQ related!
- Bumping MAX_REQ up to 1024 makes the hang go away (on 4.7).
- Dropping it to 64 makes the same hang happen on kernels where it worked
  before (I tried 4.4.x).
- Doing s/(MAX_REQ - 1)/MAX_REQ/ makes the hang go away.

I tested QEMU 2.8.0 as well and the behaviour is the same there.

Here are the logs for 4.4 hanging with MAX_REQ == 64 without the loop
condition changed:

https://gist.githubusercontent.com/dezgeg/b5f0b7f8a0f3d8b6acb1566d7edcb2f0/raw/00241426890ea28d844986243c3b706881432fb4/9p-44.dmesg.log
https://gist.githubusercontent.com/dezgeg/2bffe6c0271c4c9c382ac6363ce1864b/raw/92329eaef38305f82090de5dde3c944561afa372/9p-44.qemu.log