From: Tomi Valkeinen <hidden> Date: 2012-12-14 14:28:21
Hi,
I have been testing Common Display Framework on OMAP, and making changes that
I've discussed in the posts I've sent in reply to the CDF series from Laurent.
While my CDF code is rather hacky and not at all ready, I wanted to post the
code for comments and also as a reference code to my posts.
So here is CDF-T (Tomi-edition =).
Changes compared to Laurent's CDF
---------------------------------
- DBI bus is removed. I don't have any DBI devices, and as I said in my posts,
I didn't see need for a real bus for DBI (or DSI). I thus also removed the
DBI panels.
- Split the display_entity into two parts: video_source, which is used to
connect the display chips and panels, and display_entity, which represent
either the panel or the whole pipeline (I'm not sure which is better).
- Added ops for DVI and DSI
- Added driver for TFP410, a DPI to DVI converter chip
- Added driver for generic DVI monitor
- Added driver for Taal, a DSI command mode panel
- Removed DISPLAY_ENTITY_STREAM_SINGLE_SHOT, and instead there's update() op.
It's perhaps possible to use the single-shot method, but I went for a
separate op to get a "update done"-callback implemented easily.
Notes about the code
--------------------
As I said, the code is rather untidy, but I think it's more or less
understandable. I've skipped adding any helper funcs, for example to call the
ops, so it's easier for me to change the API.
I also (ab)used some of the omapdss panel headers to ease my hacking for
omapdss+cdf. These should be quite clear.
The code, including hacks to omapdss to make it use CDF, can be found from:
git://gitorious.org/linux-omap-dss2/linux.git work/dss-dev-model-cdf
The DSI part is very unfinished. I am able to use it to start up the Taal
panel, and send frames to the panel, but it's really missing lots of stuff.
About display_entity & video_source
-----------------------------------
I've discussed this split in previous posts, but I'll describe it here again.
As I see it, the connections between the display blocks, and the use of the
panel (and thus the whole pipeline) are very different things, and I think they
should be separated. So in my version I have struct video_source, used to
connect the blocks, and display_entity, representing the panel or the whole
pipeline. display_entity is probably not a good name for it anymore, but I
didn't come up with a good one yet.
As an example, let's look at chip-tfp410.c and panel-dvi.c.
tfp410 uses two video_sources, one for input and one for output. The input
comes from some other display block, in my case from OMAP display subsystem.
OMAP DSS has registered a DPI video_source, and the tfp410 driver will get this
source using video_source_find().
tfp410 registers its output as a video source, using video_source_register.
panel-dvi will in turn use video_source_find() to get it.
Both drivers use video_source to configure the input bus, i.e. tfp410
configures the DPI bus between OMAP DSS and TFP410 using, for example,
set_data_lines op to configure the number of datalines on the DPI bus.
With the video_sources in place, the whole video pipeline can be used. However,
we still need to expose an API so that somebody can actually use the pipeline.
This is done with display_entity. display_entity contains higher level ops,
which are not bus specific. The display_entity is registered by the panel at
the end of the chain.
In my model the display_entity ops go to the panel driver, which then calls ops
in the input video source to do the work. Laurent has objected to this model,
and would rather have the display_entity ops go to the DSS side (if I
understood right), which would then propagate forward towards the panel. I have
still kept my model, as I don't see the downsides with my model, nor do I see
any use for propagating the ops from DSS to the panel. But I'm happy to hear
examples how it is beneficial and could be used.
About the bus model
-------------------
Lauren't version uses a linux bus for DBI. The idea here is that DBI is the
control bus fro the panel/chip, and should thus be represented by a real bus.
While I agreed to this approach when we discussed about it, I now am slightly
against it.
My reason is that DBI (or DSI or any other similar bus) is not really control
bus, it is a video bus. It _can_ be used for control, but video is the main
purpose. This has the partical issues:
- There's never use for DBI only for control. DBI is always used for either
video only, or control+video.
- If DBI is used only for video, there's no DBI bus. How to configure DBI in
this case?
- If DBI is used for control and video, we have two separate APIs for the bus.
In theory it's possible to handle this, but in practice it may be impossible,
especially for more complex busses like DSI.
So in my version I added DSI as a plain video_source, without a real linux bus.
I think this model is a lot simpler, and works better.
Tomi
Tomi Valkeinen (6):
video: add display-core
video: add generic dpi panel
video: add tfp410
video: add generic dvi monitor
video: add taal panel
video: add makefile & kconfig
drivers/video/Kconfig | 1 +
drivers/video/Makefile | 1 +
drivers/video/display/Kconfig | 26 +++
drivers/video/display/Makefile | 5 +
drivers/video/display/chip-tfp410.c | 164 +++++++++++++++
drivers/video/display/display-core.c | 207 ++++++++++++++++++
drivers/video/display/panel-dpi.c | 155 ++++++++++++++
drivers/video/display/panel-dvi.c | 164 +++++++++++++++
drivers/video/display/panel-taal.c | 383 ++++++++++++++++++++++++++++++++++
include/video/display.h | 166 +++++++++++++++
include/video/omap-panel-nokia-dsi.h | 4 +-
include/video/omap-panel-tfp410.h | 4 +
include/video/panel-dpi.h | 25 +++
include/video/panel-dvi.h | 18 ++
14 files changed, 1321 insertions(+), 2 deletions(-)
create mode 100644 drivers/video/display/Kconfig
create mode 100644 drivers/video/display/Makefile
create mode 100644 drivers/video/display/chip-tfp410.c
create mode 100644 drivers/video/display/display-core.c
create mode 100644 drivers/video/display/panel-dpi.c
create mode 100644 drivers/video/display/panel-dvi.c
create mode 100644 drivers/video/display/panel-taal.c
create mode 100644 include/video/display.h
create mode 100644 include/video/panel-dpi.h
create mode 100644 include/video/panel-dvi.h
--
1.7.10.4
@@ -0,0 +1,166 @@+/*+*DisplayCore+*+*Copyright(C)2012RenesasSolutionsCorp.+*+*Contacts:LaurentPinchart<laurent.pinchart@ideasonboard.com>+*+*Thisprogramisfreesoftware;youcanredistributeitand/ormodify+*itunderthetermsoftheGNUGeneralPublicLicenseversion2as+*publishedbytheFreeSoftwareFoundation.+*/++#ifndef __DISPLAY_H__+#define __DISPLAY_H__++#include<linux/kref.h>+#include<linux/list.h>+#include<linux/module.h>+#include<video/omapdss.h>++structdisplay_entity;+structvideo_source;+structvideomode;++/* -----------------------------------------------------------------------------+*DisplayEntity+*/++/* Hack to get the first registered display entity */+structdisplay_entity*display_entity_get_first(void);++enumdisplay_entity_state{+DISPLAY_ENTITY_STATE_OFF,+DISPLAY_ENTITY_STATE_STANDBY,+DISPLAY_ENTITY_STATE_ON,+};++structdisplay_entity_control_ops{+int(*set_state)(structdisplay_entity*ent,+enumdisplay_entity_statestate);+int(*update)(structdisplay_entity*ent,+void(*callback)(int,void*),void*data);+int(*get_modes)(structdisplay_entity*ent,+conststructvideomode**modes);+int(*get_size)(structdisplay_entity*ent,+unsignedint*width,unsignedint*height);+};++structdisplay_entity{+structlist_headlist;+structdevice*dev;+structmodule*owner;+structkrefref;++conststructdisplay_entity_control_ops*ops;++void(*release)(structdisplay_entity*ent);++enumdisplay_entity_statestate;+};++intdisplay_entity_set_state(structdisplay_entity*entity,+enumdisplay_entity_statestate);+intdisplay_entity_get_modes(structdisplay_entity*entity,+conststructvideomode**modes);+intdisplay_entity_get_size(structdisplay_entity*entity,+unsignedint*width,unsignedint*height);++structdisplay_entity*display_entity_get(structdisplay_entity*entity);+voiddisplay_entity_put(structdisplay_entity*entity);++int__must_check__display_entity_register(structdisplay_entity*entity,+structmodule*owner);+voiddisplay_entity_unregister(structdisplay_entity*entity);++#define display_entity_register(display_entity) \+__display_entity_register(display_entity,THIS_MODULE)+++/* -----------------------------------------------------------------------------+*VideoSource+*/++enumvideo_source_stream_state{+DISPLAY_ENTITY_STREAM_STOPPED,+DISPLAY_ENTITY_STREAM_CONTINUOUS,+};++structcommon_video_source_ops{+int(*set_stream)(structvideo_source*src,+enumvideo_source_stream_statestate);+};++structdpi_video_source_ops{+int(*set_videomode)(structvideo_source*src,+conststructvideomode*vm);+int(*set_data_lines)(structvideo_source*src,intlines);+};++structdsi_video_source_ops{+/* enable/disable dsi bus */+int(*enable)(structvideo_source*src);+void(*disable)(structvideo_source*src);++/* bus configuration */+int(*configure_pins)(structvideo_source*src,+conststructomap_dsi_pin_config*pins);+int(*set_clocks)(structvideo_source*src,+unsignedlongddr_clk,+unsignedlonglp_clk);++void(*set_operation_mode)(structvideo_source*src,+enumomap_dss_dsi_modemode);+void(*set_pixel_format)(structvideo_source*src,+enumomap_dss_dsi_pixel_formatfmt);+void(*set_size)(structvideo_source*src,u16w,u16h);++void(*enable_hs)(structvideo_source*src,boolenable);++/* data transfer */+int(*dcs_write)(structvideo_source*src,intchannel,+u8*data,size_tlen);+int(*dcs_read)(structvideo_source*src,intchannel,u8dcs_cmd,+u8*data,size_tlen);+int(*update)(structvideo_source*src,intchannel,+void(*callback)(int,void*),void*data);+};++structdvi_video_source_ops{+int(*set_videomode)(structvideo_source*src,+conststructvideomode*vm);+};++structvideo_source{+structlist_headlist;+structdevice*dev;+structmodule*owner;+structkrefref;++constchar*name;++conststructcommon_video_source_ops*common_ops;++union{+conststructdpi_video_source_ops*dpi;+conststructdsi_video_source_ops*dsi;+conststructdvi_video_source_ops*dvi;+}ops;++void(*release)(structvideo_source*src);+};+++#define video_source_register(video_source) \+__video_source_register(video_source,THIS_MODULE)++int__must_check__video_source_register(structvideo_source*entity,+structmodule*owner);+voidvideo_source_unregister(structvideo_source*entity);++structvideo_source*video_source_get(structvideo_source*src);+voidvideo_source_put(structvideo_source*src);++structvideo_source*video_source_find(constchar*name);++#endif /* __DISPLAY_H__ */
@@ -0,0 +1,25 @@+/*+*DPIDisplayPanel+*+*Copyright(C)2012RenesasSolutionsCorp.+*+*Contacts:LaurentPinchart<laurent.pinchart@ideasonboard.com>+*+*Thisprogramisfreesoftware;youcanredistributeitand/ormodify+*itunderthetermsoftheGNUGeneralPublicLicenseversion2as+*publishedbytheFreeSoftwareFoundation.+*/++#ifndef __PANEL_DPI_H__+#define __PANEL_DPI_H__++#include<linux/videomode.h>++structpanel_dpi_platform_data{+constchar*video_source;+unsignedlongwidth;/* Panel width in mm */+unsignedlongheight;/* Panel height in mm */+conststructvideomode*mode;+};++#endif /* __PANEL_DPI_H__ */
@@ -0,0 +1,383 @@+/*+*TaalDSIcommandmodepanel+*+*Copyright(C)2012TexasInstruments+*Author:TomiValkeinen<tomi.valkeinen@ti.com>+*+*Thisprogramisfreesoftware;youcanredistributeitand/ormodifyit+*underthetermsoftheGNUGeneralPublicLicenseversion2aspublishedby+*theFreeSoftwareFoundation.+*+*Thisprogramisdistributedinthehopethatitwillbeuseful,butWITHOUT+*ANYWARRANTY;withouteventheimpliedwarrantyofMERCHANTABILITYor+*FITNESSFORAPARTICULARPURPOSE.SeetheGNUGeneralPublicLicensefor+*moredetails.+*+*YoushouldhavereceivedacopyoftheGNUGeneralPublicLicensealongwith+*thisprogram.Ifnot,see<http://www.gnu.org/licenses/>.+*/++#define DEBUG++#include<linux/module.h>+#include<linux/delay.h>+#include<linux/jiffies.h>+#include<linux/sched.h>+#include<linux/gpio.h>+#include<linux/mutex.h>+#include<linux/platform_device.h>+#include<linux/videomode.h>++#include<video/omapdss.h>+#include<video/display.h>+#include<video/omap-panel-nokia-dsi.h>+#include<video/mipi_display.h>++/* DSI Virtual channel. Hardcoded for now. */+#define TCH 0++#define DCS_READ_NUM_ERRORS 0x05+#define DCS_BRIGHTNESS 0x51+#define DCS_CTRL_DISPLAY 0x53+#define DCS_WRITE_CABC 0x55+#define DCS_READ_CABC 0x56+#define DCS_GET_ID1 0xda+#define DCS_GET_ID2 0xdb+#define DCS_GET_ID3 0xdc++structtaal_data{+structplatform_device*pdev;+structvideo_source*src;+structdisplay_entityentity;++structmutexlock;++unsignedlonghw_guard_end;/* next value of jiffies when we can+*issuethenextsleepin/outcommand+*/+unsignedlonghw_guard_wait;/* max guard time in jiffies */++/* panel HW configuration from DT or platform data */+intreset_gpio;++/* runtime variables */+boolenabled;++boolte_enabled;++intchannel;++boolcabc_broken;+unsignedcabc_mode;++boolintro_printed;+};++staticvoidhw_guard_start(structtaal_data*td,intguard_msec)+{+td->hw_guard_wait=msecs_to_jiffies(guard_msec);+td->hw_guard_end=jiffies+td->hw_guard_wait;+}++staticvoidhw_guard_wait(structtaal_data*td)+{+unsignedlongwait=td->hw_guard_end-jiffies;++if((long)wait>0&&wait<=td->hw_guard_wait){+set_current_state(TASK_UNINTERRUPTIBLE);+schedule_timeout(wait);+}+}++staticinttaal_dcs_read_1(structtaal_data*td,u8dcs_cmd,u8*data)+{+intr;+u8buf[1];+structvideo_source*src=td->src;++r=src->ops.dsi->dcs_read(src,td->channel,dcs_cmd,buf,1);+if(r<0)+returnr;++*data=buf[0];++return0;+}++staticinttaal_dcs_write_0(structtaal_data*td,u8dcs_cmd)+{+structvideo_source*src=td->src;++returnsrc->ops.dsi->dcs_write(src,td->channel,&dcs_cmd,1);+}++staticinttaal_sleep_out(structtaal_data*td)+{+intr;++hw_guard_wait(td);++r=taal_dcs_write_0(td,MIPI_DCS_EXIT_SLEEP_MODE);+if(r)+returnr;++hw_guard_start(td,120);++msleep(5);++return0;+}++staticinttaal_get_id(structtaal_data*td,u8*id1,u8*id2,u8*id3)+{+intr;++r=taal_dcs_read_1(td,DCS_GET_ID1,id1);+if(r)+returnr;+r=taal_dcs_read_1(td,DCS_GET_ID2,id2);+if(r)+returnr;+r=taal_dcs_read_1(td,DCS_GET_ID3,id3);+if(r)+returnr;++return0;+}++staticvoidtaal_hw_reset(structtaal_data*td)+{+if(!gpio_is_valid(td->reset_gpio))+return;++gpio_set_value(td->reset_gpio,1);+udelay(10);+/* reset the panel */+gpio_set_value(td->reset_gpio,0);+/* assert reset */+udelay(10);+gpio_set_value(td->reset_gpio,1);+/* wait after releasing reset */+msleep(5);+}++#define to_panel(p) container_of(p, struct taal_data, entity)++staticinttaal_set_state(structdisplay_entity*entity,+enumdisplay_entity_statestate)+{+structtaal_data*td=to_panel(entity);+structvideo_source*src=td->src;+intr;++switch(state){+caseDISPLAY_ENTITY_STATE_OFF:+caseDISPLAY_ENTITY_STATE_STANDBY:+r=taal_dcs_write_0(td,MIPI_DCS_SET_DISPLAY_OFF);+if(r)+printk("display off failed\n");++src->ops.dsi->disable(src);++break;++caseDISPLAY_ENTITY_STATE_ON:+r=src->ops.dsi->enable(src);+if(r)+printk("failed to enable bus\n");++taal_hw_reset(td);++r=taal_sleep_out(td);+if(r)+printk("sleep out failed\n");++src->ops.dsi->enable_hs(src,true);+++r=taal_dcs_write_0(td,MIPI_DCS_SET_DISPLAY_ON);+if(r)+printk("display on failed\n");+break;+}++return0;+}++staticconststructvideomodetaal_mode={+.hactive=864,+.vactive=480,+};++staticinttaal_get_modes(structdisplay_entity*entity,+conststructvideomode**modes)+{+//struct panel_data *data = to_panel(entity);++*modes=&taal_mode;+return1;+}++staticinttaal_get_size(structdisplay_entity*entity,+unsignedint*width,unsignedint*height)+{+//struct panel_data *data = to_panel(entity);++*width=10;+*height=10;+return0;+}++staticinttaal_update(structdisplay_entity*entity,+void(*callback)(int,void*),void*data)+{+structtaal_data*td=to_panel(entity);+structvideo_source*src=td->src;++returnsrc->ops.dsi->update(src,td->channel,callback,data);+}++staticconststructdisplay_entity_control_opstaal_control_ops={+.set_state=taal_set_state,+.get_modes=taal_get_modes,+.get_size=taal_get_size,+.update=taal_update,+};++staticvoidpanel_taal_release(structdisplay_entity*entity)+{+printk("panel taal release\n");+}++staticinttaal_probe(structplatform_device*pdev)+{+conststructnokia_dsi_panel_data*pdata=pdev->dev.platform_data;+structtaal_data*td;+intr;+u8id1,id2,id3;+structvideo_source*src;++dev_dbg(&pdev->dev,"probe\n");++td=devm_kzalloc(&pdev->dev,sizeof(*td),GFP_KERNEL);+if(!td)+return-ENOMEM;++td->pdev=pdev;+++td->reset_gpio=pdata->reset_gpio;++platform_set_drvdata(pdev,td);++mutex_init(&td->lock);++if(gpio_is_valid(td->reset_gpio)){+r=devm_gpio_request_one(&pdev->dev,td->reset_gpio,+GPIOF_OUT_INIT_LOW,"taal rst");+if(r){+dev_err(&pdev->dev,"failed to request reset gpio\n");+returnr;+}+}+++/* setup input */+src=video_source_find(pdata->video_source);+if(src=NULL){+printk("failed to get video source\n");+return-EINVAL;+}++td->src=src;++r=src->ops.dsi->configure_pins(src,&pdata->pin_config);+if(r)+dev_err(&pdev->dev,"failed to configure DSI pins\n");++r=src->ops.dsi->set_clocks(src,216000000,10000000);+if(r)+dev_err(&pdev->dev,"failed to set HS and LP clocks\n");++src->ops.dsi->set_size(src,864,480);+src->ops.dsi->set_pixel_format(src,OMAP_DSS_DSI_FMT_RGB888);+src->ops.dsi->set_operation_mode(src,OMAP_DSS_DSI_CMD_MODE);++/* setup panel entity */++td->entity.dev=&pdev->dev;+td->entity.release=panel_taal_release;+td->entity.ops=&taal_control_ops;++r=display_entity_register(&td->entity);+if(r<0){+printk("failed to register display entity\n");+returnr;+}++/* show version */++r=src->ops.dsi->enable(src);+if(r)+dev_err(&pdev->dev,"failed to enable bus\n");++taal_hw_reset(td);++r=taal_get_id(td,&id1,&id2,&id3);+if(r)+returnr;++dev_info(&pdev->dev,"panel revision %02x.%02x.%02x\n",id1,id2,id3);++src->ops.dsi->disable(src);+++return0;+#if 0+r=omap_dsi_request_vc(dssdev,&td->channel);+if(r){+dev_err(&pdev->dev,"failed to get virtual channel\n");+gotoerr_req_vc;+}++r=omap_dsi_set_vc_id(dssdev,td->channel,TCH);+if(r){+dev_err(&pdev->dev,"failed to set VC_ID\n");+gotoerr_vc_id;+}+#endif+}++staticinttaal_remove(structplatform_device*pdev)+{+structtaal_data*td=platform_get_drvdata(pdev);++dev_dbg(&pdev->dev,"remove\n");++display_entity_unregister(&td->entity);++video_source_put(td->src);++/* reset, to be sure that the panel is in a valid state */+taal_hw_reset(td);++#if 0+omap_dsi_release_vc(dssdev,td->channel);+#endif+return0;+}++staticstructplatform_drivertaal_driver={+.probe=taal_probe,+.remove=taal_remove,+.driver={+.name="taal",+.owner=THIS_MODULE,+},+};++module_platform_driver(taal_driver);++MODULE_AUTHOR("Tomi Valkeinen <tomi.valkeinen@ti.com>");+MODULE_DESCRIPTION("Taal Driver");+MODULE_LICENSE("GPL");
Hi Tomi,
On Friday 14 December 2012 16:27:26 Tomi Valkeinen wrote:
Hi,
I have been testing Common Display Framework on OMAP, and making changes
that I've discussed in the posts I've sent in reply to the CDF series from
Laurent. While my CDF code is rather hacky and not at all ready, I wanted
to post the code for comments and also as a reference code to my posts.
So here is CDF-T (Tomi-edition =).
We've discussed your approach extensively face-to-face today so I won't review
the patches in detail, but I will instead summarize our discussion to make
sure we understood each other (and let other developers jump in).
For the purpose of this discussion the term "display controller driver" (or
just "display controller") refer to both the low-level driver layer that
communicates directly with the display controller hardware, and to the higher-
level driver layer that implements and exposes the userspace API (FBDEV, KMS
and/or V4L). Those layers can be implemented in multiple kernel modules (such
as in the OMAP DSS case, with omapdss for the low-level layer and omapdrm,
omapfb and omapvout for the API-level layer) or a single kernel module.
Control model
-------------
The figure at http://www.ideasonboard.org/media/cdf/cdf-panel-control-
model.png shows the CDF control model.
The panel object depicted on the figure doesn't need to be a panel in the
stricter sense but could be any chain of off-SoC (both on-board or off-board)
display entities. It however helps thinking about it as a panel and doesn't
hurt the model.
The panel is controlled through abstract control requests. Those requests are
used to retrieve panel information (such as the physical size, the supported
video modes, EDID information, ...), set the panel configuration (such as the
active video timings) or control the panel operation state (enabling/disabling
the panel, controlling panel blanking and power management, ...). They are
exposed by the panel using function pointers, and called by other kernel
components in response to userspace requests (through the FBDEV, KMS or V4L2
APIs) or in-kernel events (for instance hotplug notifications).
In response to the control requests the panel driver will communicate with the
panel through the panel control bus (I2C, SPI, DBI, DSI, GPIO, ..., not shown
on the figure) and will control the video stream it receives on its input.
The panel is connected at the hardware level to a video source (shown as a
green hashed rectangle) that provides it with a video stream. The video stream
flows from the video source to the panel and is directly controlled by its
source, as shown by the green arrow from the display controller to the video
stream. The video source exposes stream control operations as function
pointers that are used by the panel to control the video stream, as shown by
the green arrow from the panel to the video source.
The figure at http://www.ideasonboard.org/media/cdf/cdf-panel-control-
model-2.png shows the call flow across entities when the panel is a pipeline
made of more than a single entity. In this case the SoC (on the left of the
dashed line) outputs a video stream on a DSI bus connected to a DSI to LVDS
transmitter. The output of the DSI to LVDS transmitter is connected to an LVDS
panel (or, more accurately, an LVDS panel module made of an LVDS panel
controller and a panel).
The transmitter and panel module are seen by the display controller and
userspace API implementations as a single entity that exposes control request
operations and controls its input video stream. When a control request is
performed (outermost green arrow) the DSI to LVDS transmitter will propagate
it to the panel, possibly mangling the input parameters or the response. For
panel operation state control requests the last entity in the pipeline will
likely want to control the video stream it receives on its input. The video
stream control calls will be propagated from right to left as shown by the red
arrows.
Every entity in the call stack can communicate with its hardware device
through the corresponding control bus, and/or control the video stream it
receives on its input.
This model allows filtering out modes and timings supported by the panel but
unsupported by the transmitter and mangling the modes and timings according to
the transmitter limitations. It has no complexity drawback for simple devices,
as the corresponding drivers can just forward the calls directly. Similar use
cases could exist for other control operations than mode and information
retrieval.
Discovery
---------
Before being able to issue control requests, panel devices need to be
discovered and associated with the connected display controller(s).
Panels and display controllers are cross-dependent. There is no way around
that, as the display controller needs a reference to the panel to call control
requests in response to userspace API, and the panel needs a reference to the
display controller to call video stream control functions (in addition to
requiring generic resources such as clocks, GPIOs or even regulators that
could be provided by the display controller).
As we can't probe the display controller and the panel together, a probe order
needs to be defined. The decision was to consider video sources as resources
and defer panel probing until all required resources (video stream source,
clocks, GPIOs, regulators and more) are available. Display controller probing
must succeed without the panel being available. This mimicks the hotpluggable
monitor model (VGA, HDMI, DP) that doesn't prevent display controllers from
being successfully probed without a connected monitor.
Our design goal is to handle panel discovery in a similar (if not identical)
way as HDMI/DP hotplug in order to implement a single display discovery method
in display controller drivers. This might not be achievable, in which case
we'll reconsider the design requirement.
When the display controller driver probes the device it will register the
video source(s) at the output of the display controller with the CDF core.
Those sources will be identified by the display controller dev_name() and a
source integer index. A new structure, likely called display_entity_port, will
be used to represent a source or sink video port on a display entity.
Panel drivers will handle video sources as resources. They will retrieve at
probe time the video source the panel is connected to using a phandle or a
source name (depending on whether the platform uses DT). If the source isn't
available the probe function will return -EPROBE_DEFER.
In addition to the video stream control operations mentioned above, ports will
also expose a connect/disconnect operation use to notify them of
connection/disconnection events. After retrieving the connected video source
panel drivers call the connect/disconnect operation on the video source to
notify it that the panel is available.
When the panel is a pipeline made of more than a single entity, entities are
probed in video source to video sink order. Out-of-order probe will result in
probe deferral as explained above due to the video source not being available,
resulting in the source to sink probe order. Entities should not call the
connect operation of their video source at probe time in that case, but only
when their own connect operation for the video source(s) they provide to the
next entity is called by the next entity. Connect operations will thus be
called in sink to source order starting at the entity at the end of the
pipeline and going all the way back to the display controller.
This notification system is a hotplug mechanism that replaces the display
entity notifier system from my previous RFC. Alan Cox rightly objected to the
notification system, arguing that such system-wide notifications were used by
FBDEV and very subject to abuse. I agree with his argument, this new mechanism
should result in a cleaner implementation as video sources will only be
notified of connect/disconnect events for the entity they're connected to.
DBI/DSI busses
--------------
My RFC introduced a DBI bus using the Linux device and bus model. Its purpose
was multifold:
- Support (un)registration, matching and binding of devices and drivers.
- Provide power management (suspend/resume) services through the standard
Linux PM bus/device model, to make sure that DBI devices will be
suspended/resumed after/before their DBI bus controller.
- Provide bus services to access the connected devices. For DBI that took the
form of command read and data read/write functions.
A DSI bus implementation using the same model was also planned.
Tomi's patches removed the DBI bus and replaced DBI devices with platform
devices, moving the bus services implementation to the video source. DBI and
DSI busses are always either pure video or video + control busses (although
controlling a DPI panel through DSI is conceivable, nobody in his right mind,
not even a hardware engineer, would likely implement that), so there will
always be a video source to provide the DBI/DSI control operations.
(Un)registration, matching and binding of devices and drivers is provided by
the platform device bus. Bus services to access connected devices are provided
by the video source, wrapper functions will be used to handle serialization
and locking, and possibly to offer higher level services (such as DCS for
instance).
One drawback of using the platform bus is that PM relationships between the
bus master and slaves will not be taken into account during suspend/resume.
However, a similar issue exists for DPI panels, and PM relationships at the
video bus level for DBI and DSI are not handled by the DBI/DSI busses either.
As we need a generic solution to handle those (likely through early suspend
and late resume), the same solution can be used to handle DBI and DSI control
bus PM relationships without requiring a Linux DBI or DSI bus.
Even though I still like the idea of DBI and DSI busses, I agree with Tomi
that they're not strictly needed and I will drop them.
Entity model
------------
Tomi's proposal split the display entities into video sources (struct
video_source) and display entities (struct display_entity). To make generic
pipeline operations easier, we agreed to merge the video source and the
display entity back. struct display_entity thus models a display entity that
has any number of sink and/or source ports, modeled as struct
display_entity_port instances.
Video stream operations will be exposed by the display entity as function
pointers and will take a port reference as argument (this could take the form
of struct display_entity * and port index, or struct display_entity_port *).
The DVI and DSI operations model proposed by Tomi in this patch series will be
kept.
Points that we forgot to discuss
--------------------------------
- DISPLAY_ENTITY_STREAM_SINGLE_SHOT vs. update() operation
I'll look into that.
Please let me know if I've forgotten anything.
--
Regards,
Laurent Pinchart
From: Tomi Valkeinen <hidden> Date: 2012-12-19 14:53:17
On 2012-12-19 15:21, Laurent Pinchart wrote:
Hi Tomi,
On Friday 14 December 2012 16:27:26 Tomi Valkeinen wrote:
quoted
Hi,
I have been testing Common Display Framework on OMAP, and making changes
that I've discussed in the posts I've sent in reply to the CDF series from
Laurent. While my CDF code is rather hacky and not at all ready, I wanted
to post the code for comments and also as a reference code to my posts.
So here is CDF-T (Tomi-edition =).
We've discussed your approach extensively face-to-face today so I won't review
the patches in detail, but I will instead summarize our discussion to make
sure we understood each other (and let other developers jump in).
I have some comments =). But mostly it looks good.
For the purpose of this discussion the term "display controller driver" (or
just "display controller") refer to both the low-level driver layer that
communicates directly with the display controller hardware, and to the higher-
level driver layer that implements and exposes the userspace API (FBDEV, KMS
and/or V4L). Those layers can be implemented in multiple kernel modules (such
as in the OMAP DSS case, with omapdss for the low-level layer and omapdrm,
omapfb and omapvout for the API-level layer) or a single kernel module.
Control model
-------------
The figure at http://www.ideasonboard.org/media/cdf/cdf-panel-control-
model.png shows the CDF control model.
The panel object depicted on the figure doesn't need to be a panel in the
stricter sense but could be any chain of off-SoC (both on-board or off-board)
display entities. It however helps thinking about it as a panel and doesn't
hurt the model.
I don't think it needs to be off-soc. The dispc and the panel in the
image could be any two components, in- or off-soc.
The panel is controlled through abstract control requests. Those requests are
used to retrieve panel information (such as the physical size, the supported
video modes, EDID information, ...), set the panel configuration (such as the
active video timings) or control the panel operation state (enabling/disabling
the panel, controlling panel blanking and power management, ...). They are
exposed by the panel using function pointers, and called by other kernel
components in response to userspace requests (through the FBDEV, KMS or V4L2
APIs) or in-kernel events (for instance hotplug notifications).
In response to the control requests the panel driver will communicate with the
panel through the panel control bus (I2C, SPI, DBI, DSI, GPIO, ..., not shown
on the figure) and will control the video stream it receives on its input.
The panel is connected at the hardware level to a video source (shown as a
green hashed rectangle) that provides it with a video stream. The video stream
flows from the video source to the panel and is directly controlled by its
source, as shown by the green arrow from the display controller to the video
stream. The video source exposes stream control operations as function
pointers that are used by the panel to control the video stream, as shown by
the green arrow from the panel to the video source.
The figure at http://www.ideasonboard.org/media/cdf/cdf-panel-control-
model-2.png shows the call flow across entities when the panel is a pipeline
made of more than a single entity. In this case the SoC (on the left of the
dashed line) outputs a video stream on a DSI bus connected to a DSI to LVDS
transmitter. The output of the DSI to LVDS transmitter is connected to an LVDS
panel (or, more accurately, an LVDS panel module made of an LVDS panel
controller and a panel).
Here also I don't see any reason to separate in- or off-soc components.
I think the call from DISPC, which now goes to the transmitter, should
first go to the DPI/DSI block. Whether the DPI/DSI block is in- or
off-soc should be irrelevant regarding CDF.
The transmitter and panel module are seen by the display controller and
userspace API implementations as a single entity that exposes control request
operations and controls its input video stream. When a control request is
I don't like the sound of this. I think the CDF shouldn't care how the
userspace API is implemented. There's no reason in CDF level to separate
in- or out-soc entities, nor expose only one entity. If DRM requires
mapping DRM's crtc, encoder and connector to, respectively, dispc,
DPI/DSI, the-rest, it should be able to do that, but CDF shouldn't force
that model.
Of course, the implementor of the particular SoC display driver could
decide to use one display entity that covers both dispc and DPI/DSI
blocks. But at least I would like to have OMAP's DISPC as a display
entity (or actually multiple entities, one for each DISPC output), and
the various in-SoC DPI-to-something encoders as display entities.
performed (outermost green arrow) the DSI to LVDS transmitter will propagate
it to the panel, possibly mangling the input parameters or the response. For
panel operation state control requests the last entity in the pipeline will
likely want to control the video stream it receives on its input. The video
stream control calls will be propagated from right to left as shown by the red
arrows.
Every entity in the call stack can communicate with its hardware device
through the corresponding control bus, and/or control the video stream it
receives on its input.
This model allows filtering out modes and timings supported by the panel but
unsupported by the transmitter and mangling the modes and timings according to
the transmitter limitations. It has no complexity drawback for simple devices,
as the corresponding drivers can just forward the calls directly. Similar use
cases could exist for other control operations than mode and information
retrieval.
Discovery
---------
Before being able to issue control requests, panel devices need to be
discovered and associated with the connected display controller(s).
Panels and display controllers are cross-dependent. There is no way around
Perhaps semantics, but I don't think they are cross-dependent. True,
they will call ops in each other, but the dispc will get the pointer to
the panel when the panel connects the dispc and the panel. And when the
panel disconnects, dispc will lose the reference to the panel.
For me, cross-dependent would mean that dispc could have a reference to
the panel regardless of what the panel does. In our case it is not so,
and there's no harm with the dispc's reference to the panel, as the
panel can remove it (almost) at any time.
that, as the display controller needs a reference to the panel to call control
requests in response to userspace API, and the panel needs a reference to the
display controller to call video stream control functions (in addition to
requiring generic resources such as clocks, GPIOs or even regulators that
could be provided by the display controller).
As we can't probe the display controller and the panel together, a probe order
needs to be defined. The decision was to consider video sources as resources
and defer panel probing until all required resources (video stream source,
clocks, GPIOs, regulators and more) are available. Display controller probing
must succeed without the panel being available. This mimicks the hotpluggable
monitor model (VGA, HDMI, DP) that doesn't prevent display controllers from
being successfully probed without a connected monitor.
Our design goal is to handle panel discovery in a similar (if not identical)
way as HDMI/DP hotplug in order to implement a single display discovery method
in display controller drivers. This might not be achievable, in which case
we'll reconsider the design requirement.
When the display controller driver probes the device it will register the
video source(s) at the output of the display controller with the CDF core.
Those sources will be identified by the display controller dev_name() and a
source integer index. A new structure, likely called display_entity_port, will
be used to represent a source or sink video port on a display entity.
Panel drivers will handle video sources as resources. They will retrieve at
probe time the video source the panel is connected to using a phandle or a
source name (depending on whether the platform uses DT). If the source isn't
available the probe function will return -EPROBE_DEFER.
In addition to the video stream control operations mentioned above, ports will
also expose a connect/disconnect operation use to notify them of
connection/disconnection events. After retrieving the connected video source
panel drivers call the connect/disconnect operation on the video source to
notify it that the panel is available.
When the panel is a pipeline made of more than a single entity, entities are
probed in video source to video sink order. Out-of-order probe will result in
probe deferral as explained above due to the video source not being available,
resulting in the source to sink probe order. Entities should not call the
connect operation of their video source at probe time in that case, but only
when their own connect operation for the video source(s) they provide to the
next entity is called by the next entity. Connect operations will thus be
called in sink to source order starting at the entity at the end of the
pipeline and going all the way back to the display controller.
This notification system is a hotplug mechanism that replaces the display
entity notifier system from my previous RFC. Alan Cox rightly objected to the
notification system, arguing that such system-wide notifications were used by
FBDEV and very subject to abuse. I agree with his argument, this new mechanism
should result in a cleaner implementation as video sources will only be
notified of connect/disconnect events for the entity they're connected to.
DBI/DSI busses
--------------
My RFC introduced a DBI bus using the Linux device and bus model. Its purpose
was multifold:
- Support (un)registration, matching and binding of devices and drivers.
- Provide power management (suspend/resume) services through the standard
Linux PM bus/device model, to make sure that DBI devices will be
suspended/resumed after/before their DBI bus controller.
- Provide bus services to access the connected devices. For DBI that took the
form of command read and data read/write functions.
A DSI bus implementation using the same model was also planned.
Tomi's patches removed the DBI bus and replaced DBI devices with platform
devices, moving the bus services implementation to the video source. DBI and
DSI busses are always either pure video or video + control busses (although
controlling a DPI panel through DSI is conceivable, nobody in his right mind,
not even a hardware engineer, would likely implement that), so there will
always be a video source to provide the DBI/DSI control operations.
(Un)registration, matching and binding of devices and drivers is provided by
the platform device bus. Bus services to access connected devices are provided
by the video source, wrapper functions will be used to handle serialization
and locking, and possibly to offer higher level services (such as DCS for
instance).
One drawback of using the platform bus is that PM relationships between the
bus master and slaves will not be taken into account during suspend/resume.
However, a similar issue exists for DPI panels, and PM relationships at the
video bus level for DBI and DSI are not handled by the DBI/DSI busses either.
As we need a generic solution to handle those (likely through early suspend
and late resume), the same solution can be used to handle DBI and DSI control
bus PM relationships without requiring a Linux DBI or DSI bus.
Even though I still like the idea of DBI and DSI busses, I agree with Tomi
that they're not strictly needed and I will drop them.
I'd like to highlight two points I made about the bus model:
- If DBI is used only for video, there's no DBI bus. How to configure
DBI in this case?
- If DBI is used for control and video, we have two separate APIs for
the bus. In theory it's possible to handle this, but in practice it may
be impossible, especially for more complex busses like DSI.
I think both of those issues would make the bus model very difficult to
implement. I have no idea how it could be done neatly. So as I see it,
it's not only about "not strictly needed", but that the bus model
wouldn't work without complex code.
Entity model
------------
Tomi's proposal split the display entities into video sources (struct
video_source) and display entities (struct display_entity). To make generic
pipeline operations easier, we agreed to merge the video source and the
display entity back. struct display_entity thus models a display entity that
has any number of sink and/or source ports, modeled as struct
display_entity_port instances.
Video stream operations will be exposed by the display entity as function
pointers and will take a port reference as argument (this could take the form
of struct display_entity * and port index, or struct display_entity_port *).
I'd very much like to have only one parameter to pass, as there may be
lots of ops for some busses. Having two parameters to refer to the
source is just extra code that has no extra benefit when using video
source ops. Then again, having separate port index parameter could be
perhaps simpler to implement for the one handling the video source ops,
so...
The DVI and DSI operations model proposed by Tomi in this patch series will be
kept.
Points that we forgot to discuss
--------------------------------
- DISPLAY_ENTITY_STREAM_SINGLE_SHOT vs. update() operation
Ah, yes, we missed that. I think it's possible to use SINGLE_SHOT, but
then it requires some kind of system to inform about finished update. Or
we could, of course, decide that informing about the update is done in
dispc-specific code, like handling VSYNC.
Hmm, except that won't probably work, as the panel (or any DSI device)
may need to know if the DSI bus is currently used or not.
Tomi
Hi Tomi,
On Wednesday 19 December 2012 16:53:10 Tomi Valkeinen wrote:
On 2012-12-19 15:21, Laurent Pinchart wrote:
quoted
On Friday 14 December 2012 16:27:26 Tomi Valkeinen wrote:
quoted
Hi,
I have been testing Common Display Framework on OMAP, and making changes
that I've discussed in the posts I've sent in reply to the CDF series
from Laurent. While my CDF code is rather hacky and not at all ready, I
wanted to post the code for comments and also as a reference code to my
posts.
So here is CDF-T (Tomi-edition =).
We've discussed your approach extensively face-to-face today so I won't
review the patches in detail, but I will instead summarize our discussion
to make sure we understood each other (and let other developers jump in).
I have some comments =). But mostly it looks good.
Thanks :-)
quoted
For the purpose of this discussion the term "display controller driver"
(or just "display controller") refer to both the low-level driver layer
that communicates directly with the display controller hardware, and to
the higher- level driver layer that implements and exposes the userspace
API (FBDEV, KMS and/or V4L). Those layers can be implemented in multiple
kernel modules (such as in the OMAP DSS case, with omapdss for the
low-level layer and omapdrm, omapfb and omapvout for the API-level layer)
or a single kernel module.
Control model
-------------
The figure at http://www.ideasonboard.org/media/cdf/cdf-panel-control-
model.png shows the CDF control model.
The panel object depicted on the figure doesn't need to be a panel in the
stricter sense but could be any chain of off-SoC (both on-board or
off-board) display entities. It however helps thinking about it as a
panel and doesn't hurt the model.
I don't think it needs to be off-soc. The dispc and the panel in the
image could be any two components, in- or off-soc.
That's correct, whether the components are on-SoC or off-SoC doesn't matter
here.
quoted
The panel is controlled through abstract control requests. Those requests
are used to retrieve panel information (such as the physical size, the
supported video modes, EDID information, ...), set the panel
configuration (such as the active video timings) or control the panel
operation state (enabling/disabling the panel, controlling panel blanking
and power management, ...). They are exposed by the panel using function
pointers, and called by other kernel components in response to userspace
requests (through the FBDEV, KMS or V4L2 APIs) or in-kernel events (for
instance hotplug notifications).
In response to the control requests the panel driver will communicate with
the panel through the panel control bus (I2C, SPI, DBI, DSI, GPIO, ...,
not shown on the figure) and will control the video stream it receives on
its input.
The panel is connected at the hardware level to a video source (shown as a
green hashed rectangle) that provides it with a video stream. The video
stream flows from the video source to the panel and is directly
controlled by its source, as shown by the green arrow from the display
controller to the video stream. The video source exposes stream control
operations as function pointers that are used by the panel to control the
video stream, as shown by the green arrow from the panel to the video
source.
The figure at http://www.ideasonboard.org/media/cdf/cdf-panel-control-
model-2.png shows the call flow across entities when the panel is a
pipeline made of more than a single entity. In this case the SoC (on the
left of the dashed line) outputs a video stream on a DSI bus connected to
a DSI to LVDS transmitter. The output of the DSI to LVDS transmitter is
connected to an LVDS panel (or, more accurately, an LVDS panel module
made of an LVDS panel controller and a panel).
Here also I don't see any reason to separate in- or off-soc components.
I think the call from DISPC, which now goes to the transmitter, should
first go to the DPI/DSI block. Whether the DPI/DSI block is in- or
off-soc should be irrelevant regarding CDF.
Agreed.
quoted
The transmitter and panel module are seen by the display controller and
userspace API implementations as a single entity that exposes control
request operations and controls its input video stream. When a control
request is
I don't like the sound of this. I think the CDF shouldn't care how the
userspace API is implemented. There's no reason in CDF level to separate
in- or out-soc entities, nor expose only one entity. If DRM requires
mapping DRM's crtc, encoder and connector to, respectively, dispc,
DPI/DSI, the-rest, it should be able to do that, but CDF shouldn't force
that model.
Of course, the implementor of the particular SoC display driver could decide
to use one display entity that covers both dispc and DPI/DSI blocks. But at
least I would like to have OMAP's DISPC as a display entity (or actually
multiple entities, one for each DISPC output), and the various in-SoC DPI-
to-something encoders as display entities.
OK, I haven't expressed myself correctly. I tried to describe here how the
"russian doll model" of cascaded calls across entities works. I didn't want to
force any in-/off-SoC model.
quoted
performed (outermost green arrow) the DSI to LVDS transmitter will
propagate it to the panel, possibly mangling the input parameters or the
response. For panel operation state control requests the last entity in
the pipeline will likely want to control the video stream it receives on
its input. The video stream control calls will be propagated from right
to left as shown by the red arrows.
Every entity in the call stack can communicate with its hardware device
through the corresponding control bus, and/or control the video stream it
receives on its input.
This model allows filtering out modes and timings supported by the panel
but unsupported by the transmitter and mangling the modes and timings
according to the transmitter limitations. It has no complexity drawback
for simple devices, as the corresponding drivers can just forward the
calls directly. Similar use cases could exist for other control
operations than mode and information retrieval.
Discovery
---------
Before being able to issue control requests, panel devices need to be
discovered and associated with the connected display controller(s).
Panels and display controllers are cross-dependent. There is no way around
Perhaps semantics, but I don't think they are cross-dependent. True,
they will call ops in each other, but the dispc will get the pointer to
the panel when the panel connects the dispc and the panel. And when the
panel disconnects, dispc will lose the reference to the panel.
For me, cross-dependent would mean that dispc could have a reference to
the panel regardless of what the panel does. In our case it is not so,
and there's no harm with the dispc's reference to the panel, as the
panel can remove it (almost) at any time.
The panel can ask the dispc to release its reference to the panel by calling
the disconnect function, but it can't disappear all of a sudden. I will test
the model for RFCv3.
quoted
that, as the display controller needs a reference to the panel to call
control requests in response to userspace API, and the panel needs a
reference to the display controller to call video stream control
functions (in addition to requiring generic resources such as clocks,
GPIOs or even regulators that could be provided by the display
controller).
As we can't probe the display controller and the panel together, a probe
order needs to be defined. The decision was to consider video sources as
resources and defer panel probing until all required resources (video
stream source, clocks, GPIOs, regulators and more) are available. Display
controller probing must succeed without the panel being available. This
mimicks the hotpluggable monitor model (VGA, HDMI, DP) that doesn't
prevent display controllers from being successfully probed without a
connected monitor.
Our design goal is to handle panel discovery in a similar (if not
identical) way as HDMI/DP hotplug in order to implement a single display
discovery method in display controller drivers. This might not be
achievable, in which case we'll reconsider the design requirement.
When the display controller driver probes the device it will register the
video source(s) at the output of the display controller with the CDF core.
Those sources will be identified by the display controller dev_name() and
a source integer index. A new structure, likely called
display_entity_port, will be used to represent a source or sink video port
on a display entity.
Panel drivers will handle video sources as resources. They will retrieve
at probe time the video source the panel is connected to using a phandle
or a source name (depending on whether the platform uses DT). If the
source isn't available the probe function will return -EPROBE_DEFER.
In addition to the video stream control operations mentioned above, ports
will also expose a connect/disconnect operation use to notify them of
connection/disconnection events. After retrieving the connected video
source panel drivers call the connect/disconnect operation on the video
source to notify it that the panel is available.
When the panel is a pipeline made of more than a single entity, entities
are probed in video source to video sink order. Out-of-order probe will
result in probe deferral as explained above due to the video source not
being available, resulting in the source to sink probe order. Entities
should not call the connect operation of their video source at probe time
in that case, but only when their own connect operation for the video
source(s) they provide to the next entity is called by the next entity.
Connect operations will thus be called in sink to source order starting
at the entity at the end of the pipeline and going all the way back to
the display controller.
This notification system is a hotplug mechanism that replaces the display
entity notifier system from my previous RFC. Alan Cox rightly objected to
the notification system, arguing that such system-wide notifications were
used by FBDEV and very subject to abuse. I agree with his argument, this
new mechanism should result in a cleaner implementation as video sources
will only be notified of connect/disconnect events for the entity they're
connected to.
DBI/DSI busses
--------------
My RFC introduced a DBI bus using the Linux device and bus model. Its
purpose was multifold:
- Support (un)registration, matching and binding of devices and drivers.
- Provide power management (suspend/resume) services through the standard
Linux PM bus/device model, to make sure that DBI devices will be
suspended/resumed after/before their DBI bus controller.
- Provide bus services to access the connected devices. For DBI that took
the form of command read and data read/write functions.
A DSI bus implementation using the same model was also planned.
Tomi's patches removed the DBI bus and replaced DBI devices with platform
devices, moving the bus services implementation to the video source. DBI
and DSI busses are always either pure video or video + control busses
(although controlling a DPI panel through DSI is conceivable, nobody in
his right mind, not even a hardware engineer, would likely implement
that), so there will always be a video source to provide the DBI/DSI
control operations.
(Un)registration, matching and binding of devices and drivers is provided
by the platform device bus. Bus services to access connected devices are
provided by the video source, wrapper functions will be used to handle
serialization and locking, and possibly to offer higher level services
(such as DCS for instance).
One drawback of using the platform bus is that PM relationships between
the bus master and slaves will not be taken into account during
suspend/resume. However, a similar issue exists for DPI panels, and PM
relationships at the video bus level for DBI and DSI are not handled by
the DBI/DSI busses either. As we need a generic solution to handle those
(likely through early suspend and late resume), the same solution can be
used to handle DBI and DSI control bus PM relationships without requiring
a Linux DBI or DSI bus.
Even though I still like the idea of DBI and DSI busses, I agree with Tomi
that they're not strictly needed and I will drop them.
I'd like to highlight two points I made about the bus model:
- If DBI is used only for video, there's no DBI bus. How to configure
DBI in this case?
I was planning to handle DBI video configuration through video stream
operations, so we agree here.
- If DBI is used for control and video, we have two separate APIs for
the bus. In theory it's possible to handle this, but in practice it may
be impossible, especially for more complex busses like DSI.
I think both of those issues would make the bus model very difficult to
implement. I have no idea how it could be done neatly. So as I see it,
it's not only about "not strictly needed", but that the bus model
wouldn't work without complex code.
quoted
Entity model
------------
Tomi's proposal split the display entities into video sources (struct
video_source) and display entities (struct display_entity). To make
generic pipeline operations easier, we agreed to merge the video source
and the display entity back. struct display_entity thus models a display
entity that has any number of sink and/or source ports, modeled as struct
display_entity_port instances.
Video stream operations will be exposed by the display entity as function
pointers and will take a port reference as argument (this could take the
form of struct display_entity * and port index, or struct
display_entity_port *).
I'd very much like to have only one parameter to pass, as there may be lots
of ops for some busses. Having two parameters to refer to the source is just
extra code that has no extra benefit when using video source ops. Then
again, having separate port index parameter could be perhaps simpler to
implement for the one handling the video source ops, so...
quoted
The DVI and DSI operations model proposed by Tomi in this patch series
will be kept.
Points that we forgot to discuss
--------------------------------
- DISPLAY_ENTITY_STREAM_SINGLE_SHOT vs. update() operation
Ah, yes, we missed that. I think it's possible to use SINGLE_SHOT, but
then it requires some kind of system to inform about finished update. Or
we could, of course, decide that informing about the update is done in
dispc-specific code, like handling VSYNC.
Hmm, except that won't probably work, as the panel (or any DSI device) may
need to know if the DSI bus is currently used or not.
Hi Laurent,
On Wed, Dec 19, 2012 at 6:51 PM, Laurent Pinchart
[off-list ref] wrote:
Hi Tomi,
On Friday 14 December 2012 16:27:26 Tomi Valkeinen wrote:
quoted
Hi,
I have been testing Common Display Framework on OMAP, and making changes
that I've discussed in the posts I've sent in reply to the CDF series from
Laurent. While my CDF code is rather hacky and not at all ready, I wanted
to post the code for comments and also as a reference code to my posts.
So here is CDF-T (Tomi-edition =).
We've discussed your approach extensively face-to-face today so I won't review
the patches in detail, but I will instead summarize our discussion to make
sure we understood each other (and let other developers jump in).
For the purpose of this discussion the term "display controller driver" (or
just "display controller") refer to both the low-level driver layer that
communicates directly with the display controller hardware, and to the higher-
level driver layer that implements and exposes the userspace API (FBDEV, KMS
and/or V4L). Those layers can be implemented in multiple kernel modules (such
as in the OMAP DSS case, with omapdss for the low-level layer and omapdrm,
omapfb and omapvout for the API-level layer) or a single kernel module.
Control model
-------------
The figure at http://www.ideasonboard.org/media/cdf/cdf-panel-control-
model.png shows the CDF control model.
The panel object depicted on the figure doesn't need to be a panel in the
stricter sense but could be any chain of off-SoC (both on-board or off-board)
display entities. It however helps thinking about it as a panel and doesn't
hurt the model.
The panel is controlled through abstract control requests. Those requests are
used to retrieve panel information (such as the physical size, the supported
video modes, EDID information, ...), set the panel configuration (such as the
active video timings) or control the panel operation state (enabling/disabling
the panel, controlling panel blanking and power management, ...). They are
exposed by the panel using function pointers, and called by other kernel
components in response to userspace requests (through the FBDEV, KMS or V4L2
APIs) or in-kernel events (for instance hotplug notifications).
In response to the control requests the panel driver will communicate with the
panel through the panel control bus (I2C, SPI, DBI, DSI, GPIO, ..., not shown
on the figure) and will control the video stream it receives on its input.
The panel is connected at the hardware level to a video source (shown as a
green hashed rectangle) that provides it with a video stream. The video stream
flows from the video source to the panel and is directly controlled by its
source, as shown by the green arrow from the display controller to the video
stream. The video source exposes stream control operations as function
pointers that are used by the panel to control the video stream, as shown by
the green arrow from the panel to the video source.
The figure at http://www.ideasonboard.org/media/cdf/cdf-panel-control-
model-2.png shows the call flow across entities when the panel is a pipeline
made of more than a single entity. In this case the SoC (on the left of the
dashed line) outputs a video stream on a DSI bus connected to a DSI to LVDS
transmitter. The output of the DSI to LVDS transmitter is connected to an LVDS
panel (or, more accurately, an LVDS panel module made of an LVDS panel
controller and a panel).
The transmitter and panel module are seen by the display controller and
userspace API implementations as a single entity that exposes control request
operations and controls its input video stream. When a control request is
performed (outermost green arrow) the DSI to LVDS transmitter will propagate
it to the panel, possibly mangling the input parameters or the response. For
panel operation state control requests the last entity in the pipeline will
likely want to control the video stream it receives on its input. The video
stream control calls will be propagated from right to left as shown by the red
arrows.
Every entity in the call stack can communicate with its hardware device
through the corresponding control bus, and/or control the video stream it
receives on its input.
This model allows filtering out modes and timings supported by the panel but
unsupported by the transmitter and mangling the modes and timings according to
the transmitter limitations. It has no complexity drawback for simple devices,
as the corresponding drivers can just forward the calls directly. Similar use
cases could exist for other control operations than mode and information
retrieval.
Discovery
---------
Before being able to issue control requests, panel devices need to be
discovered and associated with the connected display controller(s).
Panels and display controllers are cross-dependent. There is no way around
that, as the display controller needs a reference to the panel to call control
requests in response to userspace API, and the panel needs a reference to the
display controller to call video stream control functions (in addition to
requiring generic resources such as clocks, GPIOs or even regulators that
could be provided by the display controller).
As we can't probe the display controller and the panel together, a probe order
needs to be defined. The decision was to consider video sources as resources
and defer panel probing until all required resources (video stream source,
clocks, GPIOs, regulators and more) are available. Display controller probing
must succeed without the panel being available. This mimicks the hotpluggable
monitor model (VGA, HDMI, DP) that doesn't prevent display controllers from
being successfully probed without a connected monitor.
Our design goal is to handle panel discovery in a similar (if not identical)
way as HDMI/DP hotplug in order to implement a single display discovery method
in display controller drivers. This might not be achievable, in which case
we'll reconsider the design requirement.
When the display controller driver probes the device it will register the
video source(s) at the output of the display controller with the CDF core.
Those sources will be identified by the display controller dev_name() and a
source integer index. A new structure, likely called display_entity_port, will
be used to represent a source or sink video port on a display entity.
Panel drivers will handle video sources as resources. They will retrieve at
probe time the video source the panel is connected to using a phandle or a
source name (depending on whether the platform uses DT). If the source isn't
available the probe function will return -EPROBE_DEFER.
In addition to the video stream control operations mentioned above, ports will
also expose a connect/disconnect operation use to notify them of
connection/disconnection events. After retrieving the connected video source
panel drivers call the connect/disconnect operation on the video source to
notify it that the panel is available.
When the panel is a pipeline made of more than a single entity, entities are
probed in video source to video sink order. Out-of-order probe will result in
probe deferral as explained above due to the video source not being available,
resulting in the source to sink probe order. Entities should not call the
connect operation of their video source at probe time in that case, but only
when their own connect operation for the video source(s) they provide to the
next entity is called by the next entity. Connect operations will thus be
called in sink to source order starting at the entity at the end of the
pipeline and going all the way back to the display controller.
This notification system is a hotplug mechanism that replaces the display
entity notifier system from my previous RFC. Alan Cox rightly objected to the
notification system, arguing that such system-wide notifications were used by
FBDEV and very subject to abuse. I agree with his argument, this new mechanism
should result in a cleaner implementation as video sources will only be
notified of connect/disconnect events for the entity they're connected to.
DBI/DSI busses
--------------
My RFC introduced a DBI bus using the Linux device and bus model. Its purpose
was multifold:
- Support (un)registration, matching and binding of devices and drivers.
- Provide power management (suspend/resume) services through the standard
Linux PM bus/device model, to make sure that DBI devices will be
suspended/resumed after/before their DBI bus controller.
- Provide bus services to access the connected devices. For DBI that took the
form of command read and data read/write functions.
A DSI bus implementation using the same model was also planned.
Tomi's patches removed the DBI bus and replaced DBI devices with platform
devices, moving the bus services implementation to the video source. DBI and
DSI busses are always either pure video or video + control busses (although
controlling a DPI panel through DSI is conceivable, nobody in his right mind,
not even a hardware engineer, would likely implement that), so there will
always be a video source to provide the DBI/DSI control operations.
(Un)registration, matching and binding of devices and drivers is provided by
the platform device bus. Bus services to access connected devices are provided
by the video source, wrapper functions will be used to handle serialization
and locking, and possibly to offer higher level services (such as DCS for
instance).
One drawback of using the platform bus is that PM relationships between the
bus master and slaves will not be taken into account during suspend/resume.
However, a similar issue exists for DPI panels, and PM relationships at the
video bus level for DBI and DSI are not handled by the DBI/DSI busses either.
As we need a generic solution to handle those (likely through early suspend
and late resume), the same solution can be used to handle DBI and DSI control
bus PM relationships without requiring a Linux DBI or DSI bus.
Even though I still like the idea of DBI and DSI busses, I agree with Tomi
that they're not strictly needed and I will drop them.
Entity model
------------
Tomi's proposal split the display entities into video sources (struct
video_source) and display entities (struct display_entity). To make generic
pipeline operations easier, we agreed to merge the video source and the
display entity back. struct display_entity thus models a display entity that
has any number of sink and/or source ports, modeled as struct
display_entity_port instances.
Looking at Tomi's patchset, he has considered panel as "display entity"
and MIPI DSI as "video source entity". So if we are planning to merge it back
how should we treat panel and MIPI DSI. i mean should we consider both
panel and MIPI DSI has 2 different display entities.
i.e, during the probe of each of these drivers, should we register a
display entity with CDF.
Video stream operations will be exposed by the display entity as function
pointers and will take a port reference as argument (this could take the form
of struct display_entity * and port index, or struct display_entity_port *).
The DVI and DSI operations model proposed by Tomi in this patch series will be
kept.
so you mean you will be adding these "ops" as part of "struct display
entity" rather than video source ops,
static const struct dsi_video_source_ops dsi_dsi_ops = {
.update = dsi_bus_update,
.dcs_write = dsi_bus_dcs_write,
.dcs_read = dsi_bus_dcs_read,
.configure_pins = dsi_bus_configure_pins,
.set_clocks = dsi_bus_set_clocks,
.enable = dsi_bus_enable,
.disable = dsi_bus_disable,
.set_size = dsi_bus_set_size,
.set_operation_mode = dsi_bus_set_operation_mode,
.set_pixel_format = dsi_bus_set_pixel_format,
.enable_hs = dsi_bus_enable_hs,
};
if you can post CDF v3 patches early, it will give us more clarity
w.r.t to discussions you and Tomi had.
Points that we forgot to discuss
--------------------------------
- DISPLAY_ENTITY_STREAM_SINGLE_SHOT vs. update() operation
I'll look into that.
Please let me know if I've forgotten anything.
--
Regards,
Laurent Pinchart
Regards
Vikas Sajjan
Samsung India Software Operations Pvt Ltd.
Bangalore , India.
Hi Vikas,
On Monday 24 December 2012 12:33:50 Vikas Sajjan wrote:
On Wed, Dec 19, 2012 at 6:51 PM, Laurent Pinchart wrote:
quoted
On Friday 14 December 2012 16:27:26 Tomi Valkeinen wrote:
quoted
Hi,
I have been testing Common Display Framework on OMAP, and making changes
that I've discussed in the posts I've sent in reply to the CDF series
from Laurent. While my CDF code is rather hacky and not at all ready, I
wanted to post the code for comments and also as a reference code to my
posts.
So here is CDF-T (Tomi-edition =).
We've discussed your approach extensively face-to-face today so I won't
review the patches in detail, but I will instead summarize our discussion
to make sure we understood each other (and let other developers jump in).
For the purpose of this discussion the term "display controller driver"
(or just "display controller") refer to both the low-level driver layer
that communicates directly with the display controller hardware, and to
the higher- level driver layer that implements and exposes the userspace
API (FBDEV, KMS and/or V4L). Those layers can be implemented in multiple
kernel modules (such as in the OMAP DSS case, with omapdss for the
low-level layer and omapdrm, omapfb and omapvout for the API-level layer)
or a single kernel module.
Control model
-------------
The figure at http://www.ideasonboard.org/media/cdf/cdf-panel-control-
model.png shows the CDF control model.
The panel object depicted on the figure doesn't need to be a panel in the
stricter sense but could be any chain of off-SoC (both on-board or
off-board) display entities. It however helps thinking about it as a
panel and doesn't hurt the model.
The panel is controlled through abstract control requests. Those requests
are used to retrieve panel information (such as the physical size, the
supported video modes, EDID information, ...), set the panel
configuration (such as the active video timings) or control the panel
operation state (enabling/disabling the panel, controlling panel blanking
and power management, ...). They are exposed by the panel using function
pointers, and called by other kernel components in response to userspace
requests (through the FBDEV, KMS or V4L2 APIs) or in-kernel events (for
instance hotplug notifications).
In response to the control requests the panel driver will communicate with
the panel through the panel control bus (I2C, SPI, DBI, DSI, GPIO, ...,
not shown on the figure) and will control the video stream it receives on
its input.
The panel is connected at the hardware level to a video source (shown as a
green hashed rectangle) that provides it with a video stream. The video
stream flows from the video source to the panel and is directly
controlled by its source, as shown by the green arrow from the display
controller to the video stream. The video source exposes stream control
operations as function pointers that are used by the panel to control the
video stream, as shown by the green arrow from the panel to the video
source.
The figure at http://www.ideasonboard.org/media/cdf/cdf-panel-control-
model-2.png shows the call flow across entities when the panel is a
pipeline made of more than a single entity. In this case the SoC (on the
left of the dashed line) outputs a video stream on a DSI bus connected to
a DSI to LVDS transmitter. The output of the DSI to LVDS transmitter is
connected to an LVDS panel (or, more accurately, an LVDS panel module
made of an LVDS panel controller and a panel).
The transmitter and panel module are seen by the display controller and
userspace API implementations as a single entity that exposes control
request operations and controls its input video stream. When a control
request is performed (outermost green arrow) the DSI to LVDS transmitter
will propagate it to the panel, possibly mangling the input parameters or
the response. For panel operation state control requests the last entity
in the pipeline will likely want to control the video stream it receives
on its input. The video stream control calls will be propagated from
right to left as shown by the red arrows.
Every entity in the call stack can communicate with its hardware device
through the corresponding control bus, and/or control the video stream it
receives on its input.
This model allows filtering out modes and timings supported by the panel
but unsupported by the transmitter and mangling the modes and timings
according to the transmitter limitations. It has no complexity drawback
for simple devices, as the corresponding drivers can just forward the
calls directly. Similar use cases could exist for other control
operations than mode and information retrieval.
Discovery
---------
Before being able to issue control requests, panel devices need to be
discovered and associated with the connected display controller(s).
Panels and display controllers are cross-dependent. There is no way around
that, as the display controller needs a reference to the panel to call
control requests in response to userspace API, and the panel needs a
reference to the display controller to call video stream control
functions (in addition to requiring generic resources such as clocks,
GPIOs or even regulators that could be provided by the display
controller).
As we can't probe the display controller and the panel together, a probe
order needs to be defined. The decision was to consider video sources as
resources and defer panel probing until all required resources (video
stream source, clocks, GPIOs, regulators and more) are available. Display
controller probing must succeed without the panel being available. This
mimicks the hotpluggable monitor model (VGA, HDMI, DP) that doesn't
prevent display controllers from being successfully probed without a
connected monitor.
Our design goal is to handle panel discovery in a similar (if not
identical) way as HDMI/DP hotplug in order to implement a single display
discovery method in display controller drivers. This might not be
achievable, in which case we'll reconsider the design requirement.
When the display controller driver probes the device it will register the
video source(s) at the output of the display controller with the CDF core.
Those sources will be identified by the display controller dev_name() and
a source integer index. A new structure, likely called
display_entity_port, will be used to represent a source or sink video port
on a display entity.
Panel drivers will handle video sources as resources. They will retrieve
at probe time the video source the panel is connected to using a phandle
or a source name (depending on whether the platform uses DT). If the
source isn't available the probe function will return -EPROBE_DEFER.
In addition to the video stream control operations mentioned above, ports
will also expose a connect/disconnect operation use to notify them of
connection/disconnection events. After retrieving the connected video
source panel drivers call the connect/disconnect operation on the video
source to notify it that the panel is available.
When the panel is a pipeline made of more than a single entity, entities
are probed in video source to video sink order. Out-of-order probe will
result in probe deferral as explained above due to the video source not
being available, resulting in the source to sink probe order. Entities
should not call the connect operation of their video source at probe time
in that case, but only when their own connect operation for the video
source(s) they provide to the next entity is called by the next entity.
Connect operations will thus be called in sink to source order starting
at the entity at the end of the pipeline and going all the way back to
the display controller.
This notification system is a hotplug mechanism that replaces the display
entity notifier system from my previous RFC. Alan Cox rightly objected to
the notification system, arguing that such system-wide notifications were
used by FBDEV and very subject to abuse. I agree with his argument, this
new mechanism should result in a cleaner implementation as video sources
will only be notified of connect/disconnect events for the entity they're
connected to.
DBI/DSI busses
--------------
My RFC introduced a DBI bus using the Linux device and bus model. Its
purpose was multifold:
- Support (un)registration, matching and binding of devices and drivers.
- Provide power management (suspend/resume) services through the standard
Linux PM bus/device model, to make sure that DBI devices will be
suspended/resumed after/before their DBI bus controller.
- Provide bus services to access the connected devices. For DBI that took
the form of command read and data read/write functions.
A DSI bus implementation using the same model was also planned.
Tomi's patches removed the DBI bus and replaced DBI devices with platform
devices, moving the bus services implementation to the video source. DBI
and DSI busses are always either pure video or video + control busses
(although controlling a DPI panel through DSI is conceivable, nobody in
his right mind, not even a hardware engineer, would likely implement
that), so there will always be a video source to provide the DBI/DSI
control operations.
(Un)registration, matching and binding of devices and drivers is provided
by the platform device bus. Bus services to access connected devices are
provided by the video source, wrapper functions will be used to handle
serialization and locking, and possibly to offer higher level services
(such as DCS for instance).
One drawback of using the platform bus is that PM relationships between
the bus master and slaves will not be taken into account during
suspend/resume. However, a similar issue exists for DPI panels, and PM
relationships at the video bus level for DBI and DSI are not handled by
the DBI/DSI busses either. As we need a generic solution to handle those
(likely through early suspend and late resume), the same solution can be
used to handle DBI and DSI control bus PM relationships without requiring
a Linux DBI or DSI bus.
Even though I still like the idea of DBI and DSI busses, I agree with Tomi
that they're not strictly needed and I will drop them.
Entity model
------------
Tomi's proposal split the display entities into video sources (struct
video_source) and display entities (struct display_entity). To make
generic pipeline operations easier, we agreed to merge the video source
and the display entity back. struct display_entity thus models a display
entity that has any number of sink and/or source ports, modeled as struct
display_entity_port instances.
Looking at Tomi's patchset, he has considered panel as "display entity"
and MIPI DSI as "video source entity". So if we are planning to merge it
back how should we treat panel and MIPI DSI. i mean should we consider both
panel and MIPI DSI has 2 different display entities.
i.e, during the probe of each of these drivers, should we register a
display entity with CDF.
Both the DSI encoder and the DSI panel would be modeled as display entities.
The DSI encoder would have a source port that models its DSI video source, and
the DSI panel would have a sink port.
quoted
Video stream operations will be exposed by the display entity as function
pointers and will take a port reference as argument (this could take the
form of struct display_entity * and port index, or struct
display_entity_port *). The DVI and DSI operations model proposed by Tomi
in this patch series will be kept.
so you mean you will be adding these "ops" as part of "struct display
entity" rather than video source ops,
That's correct.
static const struct dsi_video_source_ops dsi_dsi_ops = {
.update = dsi_bus_update,
.dcs_write = dsi_bus_dcs_write,
.dcs_read = dsi_bus_dcs_read,
.configure_pins = dsi_bus_configure_pins,
.set_clocks = dsi_bus_set_clocks,
.enable = dsi_bus_enable,
.disable = dsi_bus_disable,
.set_size = dsi_bus_set_size,
.set_operation_mode = dsi_bus_set_operation_mode,
.set_pixel_format = dsi_bus_set_pixel_format,
.enable_hs = dsi_bus_enable_hs,
};
if you can post CDF v3 patches early, it will give us more clarity
w.r.t to discussions you and Tomi had.
I'm working on that.
quoted
Points that we forgot to discuss
--------------------------------
- DISPLAY_ENTITY_STREAM_SINGLE_SHOT vs. update() operation
I'll look into that.
Please let me know if I've forgotten anything.
From: Tomasz Figa <hidden> Date: 2012-12-31 11:36:33
Hi Laurent,
On Wednesday 26 of December 2012 13:14:46 Laurent Pinchart wrote:
Hi Vikas,
On Monday 24 December 2012 12:33:50 Vikas Sajjan wrote:
quoted
On Wed, Dec 19, 2012 at 6:51 PM, Laurent Pinchart wrote:
quoted
On Friday 14 December 2012 16:27:26 Tomi Valkeinen wrote:
quoted
Hi,
I have been testing Common Display Framework on OMAP, and making
changes that I've discussed in the posts I've sent in reply to the
CDF series from Laurent. While my CDF code is rather hacky and not
at all ready, I wanted to post the code for comments and also as a
reference code to my posts.
So here is CDF-T (Tomi-edition =).
We've discussed your approach extensively face-to-face today so I
won't
review the patches in detail, but I will instead summarize our
discussion to make sure we understood each other (and let other
developers jump in).
For the purpose of this discussion the term "display controller
driver"
(or just "display controller") refer to both the low-level driver
layer
that communicates directly with the display controller hardware, and
to
the higher- level driver layer that implements and exposes the
userspace API (FBDEV, KMS and/or V4L). Those layers can be
implemented in multiple kernel modules (such as in the OMAP DSS
case, with omapdss for the low-level layer and omapdrm, omapfb and
omapvout for the API-level layer) or a single kernel module.
Control model
-------------
The figure at
http://www.ideasonboard.org/media/cdf/cdf-panel-control-
model.png shows the CDF control model.
The panel object depicted on the figure doesn't need to be a panel
in the stricter sense but could be any chain of off-SoC (both
on-board or off-board) display entities. It however helps thinking
about it as a panel and doesn't hurt the model.
The panel is controlled through abstract control requests. Those
requests are used to retrieve panel information (such as the
physical size, the supported video modes, EDID information, ...),
set the panel
configuration (such as the active video timings) or control the
panel
operation state (enabling/disabling the panel, controlling panel
blanking and power management, ...). They are exposed by the panel
using function pointers, and called by other kernel components in
response to userspace requests (through the FBDEV, KMS or V4L2
APIs) or in-kernel events (for instance hotplug notifications).
In response to the control requests the panel driver will
communicate with the panel through the panel control bus (I2C, SPI,
DBI, DSI, GPIO, ..., not shown on the figure) and will control the
video stream it receives on its input.
The panel is connected at the hardware level to a video source
(shown as a green hashed rectangle) that provides it with a video
stream. The video stream flows from the video source to the panel
and is directly controlled by its source, as shown by the green
arrow from the display controller to the video stream. The video
source exposes stream control operations as function pointers that
are used by the panel to control the video stream, as shown by the
green arrow from the panel to the video source.
The figure at
http://www.ideasonboard.org/media/cdf/cdf-panel-control-
model-2.png shows the call flow across entities when the panel is a
pipeline made of more than a single entity. In this case the SoC (on
the left of the dashed line) outputs a video stream on a DSI bus
connected to a DSI to LVDS transmitter. The output of the DSI to
LVDS transmitter is connected to an LVDS panel (or, more
accurately, an LVDS panel module made of an LVDS panel controller
and a panel).
The transmitter and panel module are seen by the display controller
and
userspace API implementations as a single entity that exposes
control
request operations and controls its input video stream. When a
control
request is performed (outermost green arrow) the DSI to LVDS
transmitter will propagate it to the panel, possibly mangling the
input parameters or the response. For panel operation state control
requests the last entity in the pipeline will likely want to
control the video stream it receives on its input. The video stream
control calls will be propagated from right to left as shown by the
red arrows.
Every entity in the call stack can communicate with its hardware
device
through the corresponding control bus, and/or control the video
stream it receives on its input.
This model allows filtering out modes and timings supported by the
panel but unsupported by the transmitter and mangling the modes and
timings according to the transmitter limitations. It has no
complexity drawback for simple devices, as the corresponding
drivers can just forward the calls directly. Similar use cases
could exist for other control operations than mode and information
retrieval.
Discovery
---------
Before being able to issue control requests, panel devices need to
be
discovered and associated with the connected display controller(s).
Panels and display controllers are cross-dependent. There is no way
around that, as the display controller needs a reference to the
panel to call control requests in response to userspace API, and
the panel needs a reference to the display controller to call video
stream control functions (in addition to requiring generic
resources such as clocks, GPIOs or even regulators that could be
provided by the display controller).
As we can't probe the display controller and the panel together, a
probe order needs to be defined. The decision was to consider video
sources as resources and defer panel probing until all required
resources (video stream source, clocks, GPIOs, regulators and more)
are available. Display controller probing must succeed without the
panel being available. This mimicks the hotpluggable monitor model
(VGA, HDMI, DP) that doesn't prevent display controllers from being
successfully probed without a connected monitor.
Our design goal is to handle panel discovery in a similar (if not
identical) way as HDMI/DP hotplug in order to implement a single
display discovery method in display controller drivers. This might
not be achievable, in which case we'll reconsider the design
requirement.
When the display controller driver probes the device it will
register the video source(s) at the output of the display
controller with the CDF core. Those sources will be identified by
the display controller dev_name() and a source integer index. A new
structure, likely called
display_entity_port, will be used to represent a source or sink
video port on a display entity.
Panel drivers will handle video sources as resources. They will
retrieve at probe time the video source the panel is connected to
using a phandle or a source name (depending on whether the platform
uses DT). If the source isn't available the probe function will
return -EPROBE_DEFER.
In addition to the video stream control operations mentioned above,
ports will also expose a connect/disconnect operation use to notify
them of connection/disconnection events. After retrieving the
connected video source panel drivers call the connect/disconnect
operation on the video source to notify it that the panel is
available.
When the panel is a pipeline made of more than a single entity,
entities are probed in video source to video sink order.
Out-of-order probe will result in probe deferral as explained above
due to the video source not being available, resulting in the
source to sink probe order. Entities should not call the connect
operation of their video source at probe time in that case, but
only when their own connect operation for the video source(s) they
provide to the next entity is called by the next entity. Connect
operations will thus be called in sink to source order starting at
the entity at the end of the pipeline and going all the way back to
the display controller.
This notification system is a hotplug mechanism that replaces the
display entity notifier system from my previous RFC. Alan Cox
rightly objected to the notification system, arguing that such
system-wide notifications were used by FBDEV and very subject to
abuse. I agree with his argument, this new mechanism should result
in a cleaner implementation as video sources will only be notified
of connect/disconnect events for the entity they're connected to.
DBI/DSI busses
--------------
My RFC introduced a DBI bus using the Linux device and bus model.
Its
purpose was multifold:
- Support (un)registration, matching and binding of devices and
drivers.
- Provide power management (suspend/resume) services through the
standard Linux PM bus/device model, to make sure that DBI devices
will be suspended/resumed after/before their DBI bus controller.
- Provide bus services to access the connected devices. For DBI that
took the form of command read and data read/write functions.
A DSI bus implementation using the same model was also planned.
Tomi's patches removed the DBI bus and replaced DBI devices with
platform devices, moving the bus services implementation to the
video source. DBI and DSI busses are always either pure video or
video + control busses (although controlling a DPI panel through
DSI is conceivable, nobody in his right mind, not even a hardware
engineer, would likely implement that), so there will always be a
video source to provide the DBI/DSI control operations.
(Un)registration, matching and binding of devices and drivers is
provided by the platform device bus. Bus services to access
connected devices are provided by the video source, wrapper
functions will be used to handle serialization and locking, and
possibly to offer higher level services (such as DCS for instance).
One drawback of using the platform bus is that PM relationships
between
the bus master and slaves will not be taken into account during
suspend/resume. However, a similar issue exists for DPI panels, and
PM
relationships at the video bus level for DBI and DSI are not handled
by
the DBI/DSI busses either. As we need a generic solution to handle
those (likely through early suspend and late resume), the same
solution can be used to handle DBI and DSI control bus PM
relationships without requiring a Linux DBI or DSI bus.
Even though I still like the idea of DBI and DSI busses, I agree
with Tomi that they're not strictly needed and I will drop them.
Entity model
------------
Tomi's proposal split the display entities into video sources
(struct
video_source) and display entities (struct display_entity). To make
generic pipeline operations easier, we agreed to merge the video
source
and the display entity back. struct display_entity thus models a
display entity that has any number of sink and/or source ports,
modeled as struct display_entity_port instances.
Looking at Tomi's patchset, he has considered panel as "display
entity"
and MIPI DSI as "video source entity". So if we are planning to merge
it back how should we treat panel and MIPI DSI. i mean should we
consider both panel and MIPI DSI has 2 different display entities.
i.e, during the probe of each of these drivers, should we register a
display entity with CDF.
Both the DSI encoder and the DSI panel would be modeled as display
entities. The DSI encoder would have a source port that models its DSI
video source, and the DSI panel would have a sink port.
quoted
quoted
Video stream operations will be exposed by the display entity as
function pointers and will take a port reference as argument (this
could take the form of struct display_entity * and port index, or
struct
display_entity_port *). The DVI and DSI operations model proposed by
Tomi in this patch series will be kept.
so you mean you will be adding these "ops" as part of "struct display
entity" rather than video source ops,
That's correct.
quoted
static const struct dsi_video_source_ops dsi_dsi_ops = {
.update = dsi_bus_update,
.dcs_write = dsi_bus_dcs_write,
.dcs_read = dsi_bus_dcs_read,
.configure_pins = dsi_bus_configure_pins,
.set_clocks = dsi_bus_set_clocks,
.enable = dsi_bus_enable,
.disable = dsi_bus_disable,
.set_size = dsi_bus_set_size,
.set_operation_mode = dsi_bus_set_operation_mode,
.set_pixel_format = dsi_bus_set_pixel_format,
.enable_hs = dsi_bus_enable_hs,
};
if you can post CDF v3 patches early, it will give us more clarity
w.r.t to discussions you and Tomi had.
I'm working on that.
May I ask you to add me on CC in v3?
This would allow me to track the development, without missing anything
like it happened with the discussion about bus-less design.
Best regards,
--
Tomasz Figa
Samsung Poland R&D Center
SW Solution Development, Linux Platform