Re: usb device removed from sysfs before input children devices

On Wed, 2013-01-09 at 19:27 +0000, Karl Relton wrote:

On coming out of suspend my usb bluetooth adaptor is being reset by the
system.

In linux 3.7 the usb devices are being removed from the sysfs tree
first, and then the various 'child' devices (like my bluetooth mouse &
keyboard related devices) afterwards. This is causing the udev events
for the input devices to have 'orphaned' sysfs paths in the udev events.

This in turn means the Xorg evdev driver does not recognise the events,
and so doesn't see the removal of the input devices.

This has been picked by some downstream distributions, e.g. see this
thread by Google Chrome developers:

http://code.google.com/p/chromium-os/issues/detail?id=33813

Back on linux 3.2 this was not the case. The usb adaptor was reset, but
device removal was orderly: first the input devices (will full paths in
the udev events), then the usb devices walking up the tree.


To illustrate the issue, here is the output of 'udevadm monitor' in 3.7:

udevadm monitor
monitor will print the received events for:
KERNEL - the kernel uevent
KERNEL[2203.173080] remove   /devices/pci0000:00/0000:00:1a.0/usb3/3-2/3-2:1.0/bluetooth/hci0/rfkill2 (rfkill)
KERNEL[2203.173148] remove   /devices/pci0000:00/0000:00:1a.0/usb3/3-2/3-2:1.0/bluetooth/hci0 (bluetooth)
KERNEL[2203.173420] remove   /devices/pci0000:00/0000:00:1a.0/usb3/3-2/3-2:1.0 (usb)
KERNEL[2203.173451] remove   /devices/pci0000:00/0000:00:1a.0/usb3/3-2/3-2:1.1 (usb)
KERNEL[2203.173475] remove   /devices/pci0000:00/0000:00:1a.0/usb3/3-2/3-2:1.2 (usb)
KERNEL[2203.173693] remove   /devices/pci0000:00/0000:00:1a.0/usb3/3-2 (usb)
KERNEL[2213.152339] remove   /hci0/hci0:46/input14/mouse2 (input)
KERNEL[2213.160374] remove   /hci0/hci0:46/input14/event10 (input)
KERNEL[2213.168366] remove   /hci0/hci0:46/input14 (input)
KERNEL[2213.169058] remove   /hci0/hci0:46/0005:050D:0031.0005/hidraw/hidraw0 (hidraw)
KERNEL[2213.169198] remove   /hci0/hci0:46/0005:050D:0031.0005 (hid)
KERNEL[2213.169242] remove   /hci0/hci0:46 (bluetooth)
KERNEL[2218.176527] remove   /hci0/hci0:49/input13/event11 (input)
KERNEL[2218.180403] remove   /hci0/hci0:49/input13 (input)
KERNEL[2218.180481] remove   /hci0/hci0:49/0005:05AC:0256.0004/hidraw/hidraw1 (hidraw)
KERNEL[2218.180538] remove   /hci0/hci0:49/0005:05AC:0256.0004 (hid)
KERNEL[2218.182005] remove   /hci0/hci0:49 (bluetooth)

See how the usb devices are moved first, and then the input/bluetooth related stuff
with path-heads removed (paths are now /hci0/... instead of /devices/...)


Here is the equiv sequence back in 3.2:

KERNEL[158.378301] remove   /devices/pci0000:00/0000:00:1a.0/usb3/3-2/3-2:1.0/bluetooth/hci0/hci0:49/input11/mouse2 (input)
KERNEL[158.388283] remove   /devices/pci0000:00/0000:00:1a.0/usb3/3-2/3-2:1.0/bluetooth/hci0/hci0:49/input11/event11 (input)
KERNEL[158.409885] remove   /devices/pci0000:00/0000:00:1a.0/usb3/3-2/3-2:1.0/bluetooth/hci0/hci0:49/input11 (input)
KERNEL[158.411565] remove   /devices/pci0000:00/0000:00:1a.0/usb3/3-2/3-2:1.0/bluetooth/hci0/hci0:49/0005:050D:0031.0002/hidraw/hidraw1 (hidraw)
KERNEL[158.411598] remove   /devices/pci0000:00/0000:00:1a.0/usb3/3-2/3-2:1.0/bluetooth/hci0/hci0:49/0005:050D:0031.0002 (hid)
KERNEL[158.411621] remove   /devices/pci0000:00/0000:00:1a.0/usb3/3-2/3-2:1.0/bluetooth/hci0/hci0:49 (bluetooth)
KERNEL[158.436894] remove   /devices/pci0000:00/0000:00:1a.0/usb3/3-2/3-2:1.0/bluetooth/hci0/hci0:46/input10/event10 (input)
KERNEL[158.452211] remove   /devices/pci0000:00/0000:00:1a.0/usb3/3-2/3-2:1.0/bluetooth/hci0/hci0:46/input10 (input)
KERNEL[158.452628] remove   /devices/pci0000:00/0000:00:1a.0/usb3/3-2/3-2:1.0/bluetooth/hci0/hci0:46/0005:05AC:0256.0001/hidraw/hidraw0 (hidraw)
KERNEL[158.452662] remove   /devices/pci0000:00/0000:00:1a.0/usb3/3-2/3-2:1.0/bluetooth/hci0/hci0:46/0005:05AC:0256.0001 (hid)
KERNEL[158.452752] remove   /devices/pci0000:00/0000:00:1a.0/usb3/3-2/3-2:1.0/bluetooth/hci0/hci0:46 (bluetooth)
KERNEL[158.629847] remove   /devices/pci0000:00/0000:00:1a.0/usb3/3-2/3-2:1.0/bluetooth/hci0/rfkill2 (rfkill)
KERNEL[158.629920] remove   /devices/pci0000:00/0000:00:1a.0/usb3/3-2/3-2:1.0/bluetooth/hci0 (bluetooth)
KERNEL[158.635562] remove   /devices/pci0000:00/0000:00:1a.0/usb3/3-2/3-2:1.0 (usb)
KERNEL[158.635701] remove   /devices/pci0000:00/0000:00:1a.0/usb3/3-2/3-2:1.1 (usb)
KERNEL[158.635807] remove   /devices/pci0000:00/0000:00:1a.0/usb3/3-2/3-2:1.2 (usb)
KERNEL[158.637238] remove   /devices/pci0000:00/0000:00:1a.0/usb3/3-2 (usb)


The end result (for the user) is that even when the bluetooth
mouse/keyboard is re-added, Xorg ignores it - thinking it is some hoax
duplicate device. The keyboard/mouse is then non-operational.

Instrumenting the code suggests that the issue arises in a race between:

hidp_session() in bluetooth/hidp/core.c

and

hci_unregister_dev() in bluetooth/hci_core.c


Basically hci_unregister_dev() does a hci_dev_do_close() which in turn
shuts down the hidp connection. This causes hidp_session() running in
another thread to go into its cleanup phase, which is where the input
children devices are unregistered.

HOWEVER:
1) For some reason input_unregister_device() seems to stall some 3-5
seconds on my system before removing the input device
2) In parallel hci_unregister_dev() ploughs on, and progresses to remove
its hdev (hci<n> device) regardless, without waiting for hidp_session()
to complete.

I can't figure out why there is such a delay for
input_unregister_device().


But even if it didn't have the delay, there would still be a race
between the two threads (hci_unregister_dev vs hidp_session). No attempt
seems to be made to wait for the latter, so conceivably it could still
progress on even if hidp_session wasn't unnecessarily held up?

Question: What are the symantics of the wake_...() functions? Does the
function(thread) calling wake_... not move on until all the threads
woken by the call have completed or gone back to sleep. Or can the
calling thread() start progressing in parallel on a multi-processor
system?

I ask because if the calling function has to wait for the woken threads,
that would give the appropriate 'wait' on hci_unregister_dev ... but
ONLY IF hidp_session doesn't sleep again for some reason (e.g. while
waiting on a mutex inside input_unregister_device() perhaps ...)

Karl