PCI device not properly reset after VFIO

Hi Alex,

I've been playing around with VFIO and megasas (of course).
What I did now was switching between VFIO and 'normal' operation, ie 
emulated access.

megasas is happily running under VFIO, but when I do an emergency 
stop like killing the Qemu session the PCI device is not properly reset.
IE when I load 'megaraid_sas' after unbinding the vfio_pci module
the driver cannot initialize the card and waits forever for the 
firmware state to change.

I need to do a proper pci reset via
echo 1 > /sys/bus/pci/device/XXXX/reset
to get it into a working state again.

Looking at vfio_pci_disable() pci reset is called before the config 
state and BARs are restored.
Seeing that vfio_pci_enable() calls pci reset right at the start, 
too, before modifying anything I do wonder whether the pci reset is 
at the correct location for disable.

I would have expected to call pci reset in vfio_pci_disable() 
_after_ we have restored the configuration, to ensure a sane state 
after reset.
And, as experience show, we do need to call it there.

So what is the rationale for the pci reset?
Can we move it to the end of vfio_pci_disable() or do we need to 
call pci reset twice?

Cheers,

Hannes
-- 
Dr. Hannes Reinecke		      zSeries & Storage
hare@suse.de			      +49 911 74053 688
SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg
GF: J. Hawn, J. Guild, F. Imendörffer, HRB 16746 (AG Nürnberg)