Re: [MAINTAINERS SUMMIT] Device Passthrough Considered Harmful?
From: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Date: 2024-07-26 13:11:31
Also in:
linux-cxl, linux-rdma
On Fri, Jul 26, 2024 at 10:02:27AM +0200, Ricardo Ribalda Delgado wrote:
On Thu, Jul 25, 2024 at 9:44 PM Laurent Pinchart wrote:quoted
On Thu, Jul 25, 2024 at 04:20:35PM +0300, Leon Romanovsky wrote:quoted
On Thu, Jul 25, 2024 at 03:02:13PM +0200, Ricardo Ribalda Delgado wrote:quoted
On Thu, Jul 25, 2024 at 2:23 PM Leon Romanovsky wrote:quoted
On Thu, Jul 25, 2024 at 11:26:38AM +0200, Ricardo Ribalda Delgado wrote:quoted
On Wed, Jul 24, 2024 at 10:02 PM Laurent Pinchart wrote:<...>quoted
It would be great to define what are the free software communities here. Distros and final users are also "free software communities" and they do not care about niche use cases covered by proprietary software.Are you certain about that?As a user, and as an open source Distro developer I have a small hint. But you could also ask users what they think about not being able to use their notebook's cameras. The last time that I could not use some basic hardware from a notebook with Linux was 20 years ago.Lucky you, I still have consumer hardware (speaker) that doesn't work with Linux, and even now, there is basic hardware in my current laptop (HP docking station) that doesn't work reliably in Linux.quoted
quoted
quoted
They only care (and should care) about normal workflows.What is a normal workflow? Does it mean that if user bought something very expensive he should not be able to use it with free software, because his usage is different from yours? ThanksIt means that we should not block the standard usage for 99% of the population just because 1% of the users cannot do something fancy with their device.Right, the problem is that in some areas the statistics slightly different. 99% population is blocked because 1% of the users don't need it and don't think that it is "normal" flow.quoted
Let me give you an example. When I buy a camera I want to be able to do Video Conferencing and take some static photos of documents. I do not care about: automatic makeup, AI generated background, unicorn filters, eyes recentering... But we need to give a way to vendors to implement those things closely, without the marketing differentiators, vendors have zero incentive to invest in Linux, and that affects all the population.I've seen these kind of examples being repeatedly given in discussions related to camera ISP support in Linux. They are very misleading. These are not the kind of features that are relevant for the device pass-through discussion these day. Those are high-level use cases implemented in userspace, and vendors can ship any closed-source binaries they want there. What I care about is the features exposed by the kernel to userspace API.The ISPs are gradually becoming programmable devices and they indeed help during all of those examples.
I'd like to see more technical information to substantiate this claim. So far what I've sometimes seen is ISPs that include programmable elements, but hiding those behind a firmware that exposes a fixed (configurable) pipeline. I've also heard of attempts to expose some of that programmability to the operating system, which were abandoned in the end due to lack usefulness.
Userspace needs to send/receive information from the ISP, and that is exactly what vendors want to keep in the close.
But that's exactly what we need to implement an open userspace ecosystem :-)
Describing how they implement those algorithms is a patent minefield and their differentiating factor.
Those are also arguments I've heard many times before. The differentiating factor for cameras today is mostly in userspace ISP control algorithms, and nobody is telling vendors they need to open all that. When it comes to patents, we all know how software patents is a minefield, and hardware is also affected. I can't have much sympathy for this argument though, those patents mostly benefit the largest players in the market, and those are the ones who currently claim they can't open anything due to patents.
quoted
quoted
quoted
This challenge seems to be solved for GPUs. I am using my AMD GPU freely and my nephew can install the amdgpu-pro proprietary user space driver to play duke nukem (or whatever kids play now) at 2000 fps. There are other other subsystems that allow vendor passthrough and their ecosystem has not collapsed.Yes, I completely agree with you on that.quoted
Can we have some general guidance of what is acceptable? Can we define together the "normal workflow" and focus on a *full* open source implementation of that?I don't think that is possible to define "normal workflow". Requirement to have open-source counterpart to everything exposed through UAPI is a valid one. I'm all for that.That's my current opinion as well, as least when it comes to the kernel areas I mostly work with.
-- Regards, Laurent Pinchart