Re: Forcepad interface design proposal

Hi Benjamin and Peter,

I am refreshing this old thread in order to clarify some things
discussed here :)
I specifically got lost when reading about virtual IDs for effects.
Asking more inline.

On Thu, Apr 18, 2019 at 7:00 AM Benjamin Tissoires
[off-list ref] wrote:

On Fri, Apr 12, 2019 at 8:32 PM Sean O'Brien [off-list ref] wrote:

quoted

On Fri, Apr 12, 2019 at 12:42 AM Peter Hutterer [off-list ref] wrote:

quoted

Ahoi!

On Thu, Apr 11, 2019 at 11:24:30AM -0700, Sean O'Brien wrote:

quoted

Hi Benjamin and Peter,

Thanks for your detailed replies. I'm glad you found it interesting :)

On Wed, Apr 10, 2019 at 5:47 PM Peter Hutterer [off-list ref] wrote:

quoted

Hey!

I just have a few notes on top of what Benjamin said.

On Wed, Apr 10, 2019 at 06:51:45PM +0200, Benjamin Tissoires wrote:

quoted

Hi Sean,

On Wed, Apr 10, 2019 at 1:29 AM Sean O'Brien [off-list ref] wrote:

quoted

Hello,

I'm currently working on designing an interface for controlling "forcepads";
that is, touchpads with force sensors and haptic actuators. Below is my
proposal for the protocol at both the userspace and HID interfaces. I would
appreciate any feedback you might have.

Thanks a lot for starting this discussion. This is indeed an
interesting topic, and I had a lengthy chat with Peter this morning :)

First, I'd like to get a small clarification.
In the proposal, you are making reference to both the HID side (so the
hardware protocol) and the kernel side (the client protocol).
I read it that you are trying to build two things:
- the HID design proposal, aimed at hardware makers
- the kernel behavior on how to use it

395 more quoted lines

Was there some idea up then to implement virtual effect IDs? Right now
it seems that
the number of possible FF effects is limited to FF_MAX_EFFECTS and
that it is the kernel
and not the user space that assigns the ID when an effect is uploaded.
Or was it more of a suggestion for the future than a requirement
regarding the simple haptic
feedback implementation?

quoted

quoted

quoted

quoted

quoted

But chromeOS might use different ids and different meaning for them.
The first 2 ids first should be reserved for the kernel button left
emulation though.

Note that given that the effect memory is 'virtual' in the driver, we
can use any id and we are not limited to a few numbers of them.

An other thing is that we do not need to require WAVEFORM_PRESS and
WAVEFORM_RELEASE. If they are set in the haptic device, the kernel can
set them for us, but if they are not, userspace can always decide to
put one of the other waveforms in ids 0 and 1.

We should mention that not defining these two waveforms will induce a
state were no haptic feedback will be automatically loaded from the
kernel though (userspace will have to do it).

True, I guess I would say it's strongly encouraged to include WAVEFORM_PRESS and
WAVEFORM_RELEASE for that reason.

quoted

quoted

quoted

Autonomous mode
---------------

In order to facilitate an operating system which cannot handle force feedback,
the forcepad should start up in “autonomous mode”, meaning it acts as a normal
touchpad. This means it should perform the press and release haptic feedback
autonomously at predefined force thresholds, and send the appropriate BTN_*
events.

I think that if we were to follow my idea of using effect ids 0 and 1
for release/press, we should mention that the kernel will play the
effects stored as these ids.

quoted

After verifying that all of the required haptic effects are available through
the force feedback protocol, the OS can enable host-controlled mode. This could
be done by writing to a sysfs node “host_feedback_enabled”.

You don't really want a sysfs file here. Because if the input part

352 more quoted lines

Regards,
Angela

quoted

quoted

quoted

quoted

quoted

quoted

[0]: https://www.kernel.org/doc/Documentation/input/multi-touch-protocol.txt
[1]: https://www.kernel.org/doc/Documentation/input/ff.txt
[2]: https://www.usb.org/sites/default/files/hutrr63b_-_haptics_page_redline_0.pdf
[3]: Usage ID 0x30 of HID usage table 0x0D. See chapter 16:
     https://www.usb.org/sites/default/files/documents/hut1_12v2.pdf
[4]: See section 6.2.2.7 of the HID specification:
     https://www.usb.org/sites/default/files/documents/hid1_11.pdf

Hi Angela,

On Tue, Nov 30, 2021 at 02:51:48PM +0100, Angela Czubak wrote:

Hi Benjamin and Peter,

I am refreshing this old thread in order to clarify some things
discussed here :)
I specifically got lost when reading about virtual IDs for effects.
Asking more inline.

[...]

quoted

quoted

quoted

quoted

quoted

quoted

So, the solution we came to this morning, while talking to Peter, was
that the HID driver for a simple haptic HID device would allocate a
virtual device memory to store the effects and the parameters.

This way, we can:
- upload effect WAVEFORM_RELEASE with its parameters in id 0 of the
drvdata of the device
- upload effect WAVEFORM_PRESS with its parameters in id 1 of the
drvdata of the device
- ...
- upload effect WAVEFORM_VENDOR_ZZZ_ZZZ with its parameters in id N of
the drvdata of the device -> userspace will use it while scrolling for
instance
- ...

Then the kernel on BTN_LEFT press can automatically trigger the effect
with id 1 and the one with id 0 on release in the case of the
autonomous mode mentioned below.

To solve the question of knowing which effect should be loaded in
which slot, I think we should rely on a userspace helper (udev?).
We definitively not want the kernel to keep a list of devices to
effects matches, but having a udev database (hwdb and intrinsic?)
would nicely solve the issue as we do not need to update the kernel
for each new device coming in.

From the kernel driver, we can populate the WAVEFORM_PRESS and
WAVEFORM_RELEASE with some sensible parameters, but userspace should
be allowed to override them.

The advantage of having this virtual memory of device effects, is that
each userspace implementation could use its own matching for effects.
For example, libinput might want to say:
- id 0 -> BTN_LEFT released
- id 1 -> BTN_LEFT pressed
- id 0x1000 -> scrolling up
- id 0x1001 -> scrolling down
- id 0x2042 -> hard press

Was there some idea up then to implement virtual effect IDs? Right now

6 more quoted lines

I have to admit I barely remember any of this, it happend around a very busy
time and i have little recollection beyond re-reading this thread. The above
was an idea on how to handle custom waveforms, we never went past what was
suggested in this thread.

Cheers,
   Peter

On Mon, Dec 6, 2021 at 7:24 AM Peter Hutterer [off-list ref] wrote:

Hi Angela,

On Tue, Nov 30, 2021 at 02:51:48PM +0100, Angela Czubak wrote:

quoted

Hi Benjamin and Peter,

I am refreshing this old thread in order to clarify some things
discussed here :)
I specifically got lost when reading about virtual IDs for effects.
Asking more inline.

[...]

quoted

quoted

quoted

quoted

quoted

quoted

quoted

So, the solution we came to this morning, while talking to Peter, was
that the HID driver for a simple haptic HID device would allocate a
virtual device memory to store the effects and the parameters.

This way, we can:
- upload effect WAVEFORM_RELEASE with its parameters in id 0 of the
drvdata of the device
- upload effect WAVEFORM_PRESS with its parameters in id 1 of the
drvdata of the device
- ...
- upload effect WAVEFORM_VENDOR_ZZZ_ZZZ with its parameters in id N of
the drvdata of the device -> userspace will use it while scrolling for
instance
- ...

Then the kernel on BTN_LEFT press can automatically trigger the effect
with id 1 and the one with id 0 on release in the case of the
autonomous mode mentioned below.

To solve the question of knowing which effect should be loaded in
which slot, I think we should rely on a userspace helper (udev?).
We definitively not want the kernel to keep a list of devices to
effects matches, but having a udev database (hwdb and intrinsic?)
would nicely solve the issue as we do not need to update the kernel
for each new device coming in.

From the kernel driver, we can populate the WAVEFORM_PRESS and

27 more quoted lines

Just some additional pennies around this discussion. At Sony we do
want to support our Haptic triggers (marketed as Adaptive triggers) as
for example first seen on our PS5 DualSense controllers. We created a
proof-of-concept, but we need an entirely different interface through
evdev. The existing interface is too limiting due to the amount of
data needed.

The way our haptics works is essentially that per trigger (ABS_Z /
ABS_RZ) you can set different types of haptic effects. Depending on
the effect, you can have it start at a certain axis position. From
that axis position you can make the trigger provide more resistance or
it can even start rumbling. Haptic effects include a weapon like
effect, a rumble effect, resistance effect and others. Apple
implemented some in iOS recently for us.
https://developer.apple.com/documentation/gamecontroller/gcdualsenseadaptivetrigger

In any case, where I'm going with this is that there might be some
similarities and we are running into issues with FF. For our PoC we
created a new EV API (EV_HE for haptic effect). I didn't have time to
polish it in anyway and wanted to keep it generic.

However, there is reluctance in my understanding to add new evdev
APIs. Benjamin for example is very keen on leveraging eBPF. So it is
probably a bigger discussion.

Thanks,
Roderick