Hi Jesse,
Discussion here in Budapest with v4l and embedded graphics folks was
extremely fruitful. A few quick things to take away - I'll try to dig
through all
the stuff I've learned more in-depth later (probably in a blog post or two):
- embedded graphics is insane. The output routing/blending/whatever
currently shipping hw can do is crazy and kms as-is is nowhere near up
to snuff to support this. We've discussed omap4 and a ti chip targeted at
video surveillance as use cases. I'll post block diagrams and explanations
some when later.
- we should immediately stop to call anything an overlay. It's a confusing
concept that has a different meaning in every subsystem and for every hw
manufacturer. More sensible names are dma fifo engines for things that slurp
in planes and make them available to the display subsystem. Blend engines
for blocks that take multiple input pipes and overlay/underlay/blend them
together. Display subsytem/controller for the aggregate thing including
encoders/resizers/outputs and especially the crazy routing network that
connects everything.
Most of the discussion centered around clearing up the confusion and
reaching a mutual understanding between desktop graphics, embedded
graphics and v4l people. Two rough ideas emerged though:
1) Splitting the crtc object into two objects: crtc with associated output mode
(pixel clock, encoders/connectors) and dma engines (possibly multiple) that
feed it. omap 4 has essentially just 4 dma engines that can be freely assigned
to the available outputs, so a distinction between normal crtcs and overlay
engines just does not make sense. There's the major open question of where
to put the various attributes to set up the output pipeline. Also some of these
attributes might need to be changed atomicly together with pageflips on
a bunch of dma engines all associated with the same crtc on the next vsync,
e.g. output position of an overlaid video buffer.
2) The above should be good enough to support halfway sane chips like
omap4. But hw with more insane routing capabilities that can also use v4l
devices as sources (even video input connectors) media controller might be
a good fit. Media controller is designed to expose multimedia pipe routing
across different subsystem. But the first version, still marked experimental,
only got merged in .39. We discussed a few ideas as how to splice
media controller into kms but nothing clear emerged. So a possible kms
integration with media
controller is rather far away.
Yours, Daniel
--
Daniel Vetter
daniel.vetter@ffwll.ch - +41 (0) 79 364 57 48 - http://blog.ffwll.ch
On Fri, 13 May 2011 18:16:30 +0200
Daniel Vetter [off-list ref] wrote:
Hi Jesse,
Discussion here in Budapest with v4l and embedded graphics folks was
extremely fruitful. A few quick things to take away - I'll try to dig
through all
the stuff I've learned more in-depth later (probably in a blog post or two):
- embedded graphics is insane. The output routing/blending/whatever
currently shipping hw can do is crazy and kms as-is is nowhere near up
to snuff to support this. We've discussed omap4 and a ti chip targeted at
video surveillance as use cases. I'll post block diagrams and explanations
some when later.
Yeah I expected that; even just TVs can have really funky restrictions
about z order and blend capability.
quoted hunk
- we should immediately stop to call anything an overlay. It's a confusing
concept that has a different meaning in every subsystem and for every hw
manufacturer. More sensible names are dma fifo engines for things that slurp
in planes and make them available to the display subsystem. Blend engines
for blocks that take multiple input pipes and overlay/underlay/blend them
together. Display subsytem/controller for the aggregate thing including
encoders/resizers/outputs and especially the crazy routing network that
connects everything.
How about just "display plane" then? Specifically in the context of
display output hardware...
1) Splitting the crtc object into two objects: crtc with associated output mode
(pixel clock, encoders/connectors) and dma engines (possibly multiple) that
feed it. omap 4 has essentially just 4 dma engines that can be freely assigned
to the available outputs, so a distinction between normal crtcs and overlay
engines just does not make sense. There's the major open question of where
to put the various attributes to set up the output pipeline. Also some of these
attributes might need to be changed atomicly together with pageflips on
a bunch of dma engines all associated with the same crtc on the next vsync,
e.g. output position of an overlaid video buffer.
Yeah, that's a good goal, and pretty much what I had in mind here.
However, breaking the existing interface is a non-starter, so either we
need a new CRTC object altogether, or we preserve the idea of a
"primary" plane (whatever that means for a given platform) that's tied
to each CRTC, which each additional plane described in a separate
structure. Z order and blend restrictions will have to be communicated
separately I think...
Thanks,
--
Jesse Barnes, Intel Open Source Technology Center
On Fri, May 13, 2011 at 8:02 PM, Jesse Barnes [off-list ref] wrote:
On Fri, 13 May 2011 18:16:30 +0200
Daniel Vetter [off-list ref] wrote:
quoted
Hi Jesse,
Discussion here in Budapest with v4l and embedded graphics folks was
extremely fruitful. A few quick things to take away - I'll try to dig
through all
the stuff I've learned more in-depth later (probably in a blog post or two):
Hi Daniel, thanks for writing this up
quoted
- embedded graphics is insane. The output routing/blending/whatever
currently shipping hw can do is crazy and kms as-is is nowhere near up
to snuff to support this. We've discussed omap4 and a ti chip targeted at
video surveillance as use cases. I'll post block diagrams and explanations
some when later.
Yeah I expected that; even just TVs can have really funky restrictions
about z order and blend capability.
quoted
- we should immediately stop to call anything an overlay. It's a confusing
concept that has a different meaning in every subsystem and for every hw
manufacturer. More sensible names are dma fifo engines for things that slurp
in planes and make them available to the display subsystem. Blend engines
for blocks that take multiple input pipes and overlay/underlay/blend them
together. Display subsytem/controller for the aggregate thing including
encoders/resizers/outputs and especially the crazy routing network that
connects everything.
How about just "display plane" then? Specifically in the context of
display output hardware...
"display plane" could be a good name.. actually in omap4 case it is a
single dma engine that is multiplexing fetches for however many
attached video pipes.. that is perhaps an implementation detail, but
it makes "display plane" sound nicer as a name
quoted
1) Splitting the crtc object into two objects: crtc with associated output mode
(pixel clock, encoders/connectors) and dma engines (possibly multiple) that
feed it. omap 4 has essentially just 4 dma engines that can be freely assigned
to the available outputs, so a distinction between normal crtcs and overlay
engines just does not make sense. There's the major open question of where
to put the various attributes to set up the output pipeline. Also some of these
attributes might need to be changed atomicly together with pageflips on
a bunch of dma engines all associated with the same crtc on the next vsync,
e.g. output position of an overlaid video buffer.
Yeah, that's a good goal, and pretty much what I had in mind here.
However, breaking the existing interface is a non-starter, so either we
need a new CRTC object altogether, or we preserve the idea of a
"primary" plane (whatever that means for a given platform) that's tied
to each CRTC, which each additional plane described in a separate
structure. Z order and blend restrictions will have to be communicated
separately I think...
In the cases I can think of, you'll always have a primary plane, so
userspace need not explicitly specify it. But I think you want the
driver to pick which display plane to be automatically hooked between
the primary fb and crtc, or at least this should be the case if some
new bit is set in driver_features to indicate the driver supports
multiple display planes per crtc.
BR,
-R
Thanks,
--
Jesse Barnes, Intel Open Source Technology Center
Hi Daniel,
On Friday 13 May 2011 18:16:30 Daniel Vetter wrote:
Hi Jesse,
Discussion here in Budapest with v4l and embedded graphics folks was
extremely fruitful. A few quick things to take away - I'll try to dig
through all
the stuff I've learned more in-depth later (probably in a blog post or
two):
- embedded graphics is insane. The output routing/blending/whatever
currently shipping hw can do is crazy and kms as-is is nowhere near up
to snuff to support this. We've discussed omap4 and a ti chip targeted at
video surveillance as use cases. I'll post block diagrams and
explanations some when later.
- we should immediately stop to call anything an overlay. It's a confusing
concept that has a different meaning in every subsystem and for every hw
manufacturer. More sensible names are dma fifo engines for things that
slurp in planes and make them available to the display subsystem. Blend
engines for blocks that take multiple input pipes and overlay/underlay/blend
them together. Display subsytem/controller for the aggregate thing including
encoders/resizers/outputs and especially the crazy routing network that
connects everything.
Most of the discussion centered around clearing up the confusion and
reaching a mutual understanding between desktop graphics, embedded
graphics and v4l people. Two rough ideas emerged though:
1) Splitting the crtc object into two objects: crtc with associated output
mode (pixel clock, encoders/connectors) and dma engines (possibly
multiple) that feed it. omap 4 has essentially just 4 dma engines that can
be freely assigned to the available outputs, so a distinction between
normal crtcs and overlay engines just does not make sense. There's the
major open question of where to put the various attributes to set up the
output pipeline. Also some of these attributes might need to be changed
atomicly together with pageflips on a bunch of dma engines all associated
with the same crtc on the next vsync, e.g. output position of an overlaid
video buffer.
I like that idea. Setting attributes atomically will likely be one of the
biggest challenge. V4L2 shares the same need, but we haven't had time to
address it yet.
2) The above should be good enough to support halfway sane chips like
omap4. But hw with more insane routing capabilities that can also use v4l
devices as sources (even video input connectors) media controller might be
a good fit. Media controller is designed to expose multimedia pipe routing
across different subsystem. But the first version, still marked
experimental, only got merged in .39. We discussed a few ideas as how to
splice media controller into kms but nothing clear emerged. So a possible
kms integration with media controller is rather far away.
You're probably right, but far away doesn't mean never. Especially when one of
the media controller developers is interested in the project and could spend
time on it :-)
I've started working on a prototype implementation that would use the media
controller API to report the CRTCs, encoders and connectors topology to
userspace. The learning curve is pretty steep as I'm not familiar with the DRM
and KMS code, but the code base turned out to be much easier to dive in than
it seemed a couple of years ago.
--
Regards,
Laurent Pinchart