[RFC] Video Mode Handling

Started by Antonino A. Daplas on Aug 9, 2004 · 12 messages from 4 authors · Last reply Aug 9, 2004

open the first message on its own page

Message details and related content
Hi,

Video mode handling in fbdev-2.6 is becoming better, but is still not in
the same league as with fbdev-2.4.  The reason is that the upper vt layer
does not have any information about the video mode itself which leaves
the lower fbdev layer to fill in or replace the fields in a
partial/incorrect fb_var_sreeninfo structure (var).  This limitation is
partly alleviated by using fb_find_mode() to pass a complete var. 

fb_find_mode() will look for entries in a database in which most entries
are not compatible with the graphics/display combination.  The driver will
still need to "massage" the contents of var to make it work with the
particular combination.  Secondly, the current system does not preserve
the entire graphics per-console settings (such as bpp, virtual resolution,
etc). Thirdly, fb_find_mode() requires that the mode database need to stay
in the memory for the entire lifetime of fbdev/fbcon, increasing the kernel's
memory footprint. Finally, this requires that all drivers are capable of
guaranteeing a working display without any user intervention.

Most fbdev developers are proposing to bring back the per-display var
structures to ease the difficulty of video mode handling.  This proposal
is countered by a few (notably James) in that it severely increases the 
memory footprint of the kernel.  

How about a compromise then?

The fb_var_screeninfo structure contains the following:

	a. settings of the graphics card
	b. settings of the attached display
	c. fields used internally by core fbdev/fbcon which has no
	   relevance in video mode setting (ie, xoffset, yoffset)

Instead of saving the entire var, we just save the settings of the
graphics card.  Settings of the attached display will be replaced by
pointers to entries in a mode database.  Internally used fields will not be
saved, as these are typically handled by fbcon for each mode switch.

One requirement is the creation of a mode database that is valid and private
to the driver. This database must also be capable of adding new entries to its

66 more lines

Message details and related content
There was a lot of discussion about mode handling at OLS. Here's some
of the lwn coverage...

http://lwn.net/Articles/94202/
http://lwn.net/Articles/94763/

I'll attach a list of points discussed. Mode setting needs to cover
multihead cards, and devices that only use the BIOS for mode setting.
There also needs to be coordinated memory management when multiple
heads are involved. Point 7 talks about a design where most of the mode
setting code is moved to user space via hotplug.

Can your design be adjusted to work in the framework outlined below?

---------------------------------------------------------------
After talking to many people at OLS this seems to be the consensus for
rearchitecting fbdev/DRM. It has been proposed that this design be a
five year goal and there is still disagreement about the exact path to
get there.

This is a first pass for lkml people. I will incorporate comments and
repost for the final pass. I'm sure everyone will let me know if I've
misinterpreted some of the design points.

First a basic definition. There are two consoles, the kernel one
(printk, kdbg, OOPs, etc) and user one (normal console logins). In
Linux these are currently implemented with the same code. There may be
advantages to splitting the implementation in a future design.

1) PCI ROMs - these would be exposed via sysfs. A quirk is needed to
track the boot video device and expose the contents of C000:0 for that
special case. See the lkml thread: Exposing ROM's though sysfs, there
are already proposed patches.

2) VGA control - there needs to be a device for coordinating this. It
would ensure that only a single VGA device gets enabled at a time. It
would also adjust PCI bus routing as needed. It needs commands for
disabling all VGA devices and then enabling a selected one. This device
may need to coordinate with VGA console. You have to use this device
even if you aren't using VGA console since it ensures that only a

164 more lines

Message details and related content
"Antonino A. Daplas" [off-list ref] wrote:
Most fbdev developers are proposing to bring back the per-display var
 structures to ease the difficulty of video mode handling.  This proposal
 is countered by a few (notably James) in that it severely increases the 
 memory footprint of the kernel.  
Surprised.

By how much does this increase the kernel's memory footprint?

And how much simpler would the code be if we were to bring back the
per-display var structures?

IOW: what's the tradeoff here?


-------------------------------------------------------
This SF.Net email is sponsored by OSTG. Have you noticed the changes on
Linux.com, ITManagersJournal and NewsForge in the past few weeks? Now,
one more big change to announce. We are now OSTG- Open Source Technology
Group. Come see the changes on the new OSTG site. www.ostg.com
Message details and related content
On Sun, 8 Aug 2004, Andrew Morton wrote:
"Antonino A. Daplas" [off-list ref] wrote:
quoted
Most fbdev developers are proposing to bring back the per-display var
 structures to ease the difficulty of video mode handling.  This proposal
 is countered by a few (notably James) in that it severely increases the
 memory footprint of the kernel.
Surprised.

By how much does this increase the kernel's memory footprint?
Indeed, you need a full mode database per driver.
And how much simpler would the code be if we were to bring back the
per-display var structures?

IOW: what's the tradeoff here?
  - per-display var:
      - increase memory footprint by
        MAX_NR_CONSOLES.*sizeof(struct var_screen_info)
      + simpler code
      + 100% compatible with 2.4

  - private modedb:
      - increase memory footprint by full mode database per driver.
      - more complex (and larger) code
      - subtle differences with 2.4, because not the whole var is saved
        (BTW, which fields do you want to save, and which not, so I can
	 evaluate the impact on some drivers?)

Gr{oetje,eeting}s,

						Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
							    -- Linus Torvalds


-------------------------------------------------------
This SF.Net email is sponsored by OSTG. Have you noticed the changes on
Linux.com, ITManagersJournal and NewsForge in the past few weeks? Now,
one more big change to announce. We are now OSTG- Open Source Technology
Group. Come see the changes on the new OSTG site. www.ostg.com
Message details and related content
On Monday 09 August 2004 11:50, Jon Smirl wrote:
There was a lot of discussion about mode handling at OLS. Here's some
of the lwn coverage...

http://lwn.net/Articles/94202/
http://lwn.net/Articles/94763/
I'll attach a list of points discussed. Mode setting needs to cover
multihead cards, and devices that only use the BIOS for mode setting.
There also needs to be coordinated memory management when multiple
heads are involved. Point 7 talks about a design where most of the mode
setting code is moved to user space via hotplug.

Can your design be adjusted to work in the framework outlined below?
Sure, if I correctly understand the pertinent parts of the discussion,
they mostly pertain to moving mode handling to userspace.  The patches
that I submitted are just that, move more of the mode handling away from
the kernel and let userspace determine the numbers to give to the kernel.
The job of the driver is to de-normalize the numbers, and write them to the
registers.

Currently, the main problem is "are the values coming from userspace safe to
use?" The fbdev drivers have a check_var() that does the verification.
However the function is severely limited because not all details are known
about the hardware and the display/s.  Drivers take different approaches,
accept everything as gospel truth, be paranoid and reject everything, or
check the numbers to varying degrees of effectivity (by using EDID, kernel
boot options, etc).

I guess most people will agree that writing to the hardware belongs in the
kernel, while choosing the correct values to write belongs to userspace.  The
unanswered question is the middle part, who does the verification?

As for hardware that depends on the BIOS for mode setting, this is basically
unsupported in the current fbdev framework (vesafb).  Either we allow BIOS
code to run in the kernel (search for vesafb-tng) or run BIOS code in
userspace (search for vm86d, vesafbd). 

Multi-head cards can be configured as multiple fb_info's sharing a single par
if each output is independent from the other (in which case each head has
its own private mode list) or a single fb_info with multiple pars if each
output are dependent on each other (in which case they share a common
mode list which are filtered so all entries will work for all heads).

As for memory management, yes, that's needed.  The i810fb for example explicitly
avoids the framebuffer address that is used by X/DRI.  It's also fortunate that
the i810 hardware has 2 ringbuffers, one is used by X/DRI, one by i810fb.  Add
another application that needs the ringbuffers, and the hardware will definitely
crash.

Tony 

7 more lines

Message details and related content
On Monday 09 August 2004 14:04, Andrew Morton wrote:
"Antonino A. Daplas" [off-list ref] wrote:
quoted
Most fbdev developers are proposing to bring back the per-display var
 structures to ease the difficulty of video mode handling.  This proposal
 is countered by a few (notably James) in that it severely increases the
 memory footprint of the kernel.
Surprised.

By how much does this increase the kernel's memory footprint?
Each var is approx 140 bytes multiplied by the number of consoles, which
is 64. James is in the embedded systems market, so I can see his point.
And how much simpler would the code be if we were to bring back the
per-display var structures?
The switching code will be much simpler.  All that needs to be done is
grab display->var and pass it to fbdev.  However, the stty/resize route still needs
to be addressed. 
IOW: what's the tradeoff here?
Absence of a per-display var or similar mechanism is simply unacceptable.
Without it, mode setting is basically a mess.  It's possible that by simply
switching consoles you can be left with a corrupt display.

With  the per-display var, console switching is a breeze.  No need for mode
verification or creation.  The downside is the increased memory footprint.

The stty/fbcon_resize path is the other problem.  This is a request coming
from the upper layer (which has no knowledge whatsoever about graphics/
video) asking fbdev to change the video mode of the hardware.  Drivers can
'guess', but not all drivers are good at that, or get the mode from a known,
working modelist (which the 1st and 2nd patch addresses).  A simple solution
is to just remove stty/fbcon_resize support. But James might disagree.

BTW:  The series of patches addresses all of the above.  It can save the
settings per display, but with only half the memory requirement of a per-display
var, and makes the stty/fbcon_resize path work correctly.  It also eliminates 
the need for a static mode database as well.  The downside is increased code
complexity.

Tony




-------------------------------------------------------
This SF.Net email is sponsored by OSTG. Have you noticed the changes on
Linux.com, ITManagersJournal and NewsForge in the past few weeks? Now,
one more big change to announce. We are now OSTG- Open Source Technology
Group. Come see the changes on the new OSTG site. www.ostg.com
Message details and related content
Geert Uytterhoeven [off-list ref] wrote:
On Sun, 8 Aug 2004, Andrew Morton wrote:
quoted
"Antonino A. Daplas" [off-list ref] wrote:
quoted
Most fbdev developers are proposing to bring back the per-display var
 structures to ease the difficulty of video mode handling.  This proposal
 is countered by a few (notably James) in that it severely increases the
 memory footprint of the kernel.
Surprised.

By how much does this increase the kernel's memory footprint?
Indeed, you need a full mode database per driver.
Sorry, that's way outside my area of knowledge ;)
quoted
And how much simpler would the code be if we were to bring back the
per-display var structures?

IOW: what's the tradeoff here?
  - per-display var:
      - increase memory footprint by
        MAX_NR_CONSOLES.*sizeof(struct var_screen_info)
I see no var_screen_info.  sizeof(struct screen_info) is 56 bytes, and
there are 63 of them.  This is a negligible amount of memory!
      + simpler code
      + 100% compatible with 2.4

  - private modedb:
      - increase memory footprint by full mode database per driver.
      - more complex (and larger) code
      - subtle differences with 2.4, because not the whole var is saved
        (BTW, which fields do you want to save, and which not, so I can
	 evaluate the impact on some drivers?)
Well gee.  If we're talking about 3k, or even 10k or 30k, let's go with the
simplest code?


-------------------------------------------------------
This SF.Net email is sponsored by OSTG. Have you noticed the changes on
Linux.com, ITManagersJournal and NewsForge in the past few weeks? Now,
one more big change to announce. We are now OSTG- Open Source Technology
Group. Come see the changes on the new OSTG site. www.ostg.com
Message details and related content
On Mon, 9 Aug 2004, Antonino A. Daplas wrote:
On Monday 09 August 2004 14:04, Andrew Morton wrote:
quoted
"Antonino A. Daplas" [off-list ref] wrote:
quoted
Most fbdev developers are proposing to bring back the per-display var
 structures to ease the difficulty of video mode handling.  This proposal
 is countered by a few (notably James) in that it severely increases the
 memory footprint of the kernel.
Surprised.

By how much does this increase the kernel's memory footprint?
Each var is approx 140 bytes multiplied by the number of consoles, which
is 64. James is in the embedded systems market, so I can see his point.
IMHO embedded is much better off with configuring MAX_NR_CONSOLES to a smaller
value, or disabling virtual consoles completely.

Even on desktop systems, most people don't need more than 8 VCs.
BTW:  The series of patches addresses all of the above.  It can save the
settings per display, but with only half the memory requirement of a per-display
But it doesn't save the full var, which may cause subtle differences.

BTW, if size matters, you can save the full var without the reserved fields.

Gr{oetje,eeting}s,

						Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
							    -- Linus Torvalds


-------------------------------------------------------
This SF.Net email is sponsored by OSTG. Have you noticed the changes on
Linux.com, ITManagersJournal and NewsForge in the past few weeks? Now,
one more big change to announce. We are now OSTG- Open Source Technology
Group. Come see the changes on the new OSTG site. www.ostg.com
Message details and related content
On Mon, 9 Aug 2004, Andrew Morton wrote:
Geert Uytterhoeven [off-list ref] wrote:
quoted
On Sun, 8 Aug 2004, Andrew Morton wrote:
quoted
"Antonino A. Daplas" [off-list ref] wrote:
quoted
Most fbdev developers are proposing to bring back the per-display var
 structures to ease the difficulty of video mode handling.  This proposal
 is countered by a few (notably James) in that it severely increases the
 memory footprint of the kernel.
Surprised.

By how much does this increase the kernel's memory footprint?
Indeed, you need a full mode database per driver.
Sorry, that's way outside my area of knowledge ;)
Each driver needs a list of supported video modes.
quoted
quoted
And how much simpler would the code be if we were to bring back the
per-display var structures?

IOW: what's the tradeoff here?
  - per-display var:
      - increase memory footprint by
        MAX_NR_CONSOLES.*sizeof(struct var_screen_info)
I see no var_screen_info.  sizeof(struct screen_info) is 56 bytes, and
there are 63 of them.  This is a negligible amount of memory!
Sorry, struct fb_var_screeninfo.
quoted
      + simpler code
      + 100% compatible with 2.4

  - private modedb:
      - increase memory footprint by full mode database per driver.
      - more complex (and larger) code
      - subtle differences with 2.4, because not the whole var is saved
        (BTW, which fields do you want to save, and which not, so I can
	 evaluate the impact on some drivers?)
Well gee.  If we're talking about 3k, or even 10k or 30k, let's go with the
simplest code?
I agree!

Gr{oetje,eeting}s,

						Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
							    -- Linus Torvalds


-------------------------------------------------------
This SF.Net email is sponsored by OSTG. Have you noticed the changes on
Linux.com, ITManagersJournal and NewsForge in the past few weeks? Now,
one more big change to announce. We are now OSTG- Open Source Technology
Group. Come see the changes on the new OSTG site. www.ostg.com
Message details and related content
On Monday 09 August 2004 15:23, Geert Uytterhoeven wrote:
On Sun, 8 Aug 2004, Andrew Morton wrote:
quoted
"Antonino A. Daplas" [off-list ref] wrote:
quoted
Most fbdev developers are proposing to bring back the per-display var
 structures to ease the difficulty of video mode handling.  This
proposal is countered by a few (notably James) in that it severely
increases the memory footprint of the kernel.
Surprised.

By how much does this increase the kernel's memory footprint?
Indeed, you need a full mode database per driver.
Not necessarily.  For drivers which cannot change the videomode, a single
entry is tops. 
 
quoted
And how much simpler would the code be if we were to bring back the
per-display var structures?

IOW: what's the tradeoff here?
  - per-display var:
      - increase memory footprint by
        MAX_NR_CONSOLES.*sizeof(struct var_screen_info)
      + simpler code
      + 100% compatible with 2.4

  - private modedb:
      - increase memory footprint by full mode database per driver.
Note: the private modedb will start with a single entry.  Entries will be added
each time the user does an fbset.  A typical user will probably use less than 10
modelines. I personally use only 3.  In contrast, saving the modeline per display 
already amounts to approximately 64 modelines. 

Also, the private modedb is needed for the resize/stty path.  Without the private
modedb, fbcon will use the default modedb.  And the modedb contain entries
from a mixed hardware.   
      - more complex (and larger) code
Yes, larger code. But we eliminate the the static mode database and instead of
saving sizeof(struct fb_var_screeninfo) per console, we save only 1/2 of that
per console.
      - subtle differences with 2.4, because not the whole var is saved
With the patch, the whole var is saved except for xoffset, yoffset and activate.
The timings will be derived from the fb_videomode which  is only a pointer in
the display structure. There should be no difference between 2.4, unless
xoffset, yoffset and activate are critical to the driver's functioning. 
 
        (BTW, which fields do you want to save, and which not, so I can
	 evaluate the impact on some drivers?)
All the fields in var are saved, one way or the other,  except for xoffset, yoffset and
activate.

Tony




-------------------------------------------------------
This SF.Net email is sponsored by OSTG. Have you noticed the changes on
Linux.com, ITManagersJournal and NewsForge in the past few weeks? Now,
one more big change to announce. We are now OSTG- Open Source Technology
Group. Come see the changes on the new OSTG site. www.ostg.com
Message details and related content
On Monday 09 August 2004 16:03, Andrew Morton wrote:
quoted
  - private modedb:
      - increase memory footprint by full mode database per driver.
      - more complex (and larger) code
      - subtle differences with 2.4, because not the whole var is saved
        (BTW, which fields do you want to save, and which not, so I can
	 evaluate the impact on some drivers?)
Well gee.  If we're talking about 3k, or even 10k or 30k, let's go with the
simplest code?
The simplest code is adding per-display var and removing stty/resize support.
If stty/resize support is needed, then we need a per-driver mode database.
Using fb_find_mode() + default modedb is no guarantee.

Best to wait for the other developers to respond.

Tony




-------------------------------------------------------
This SF.Net email is sponsored by OSTG. Have you noticed the changes on
Linux.com, ITManagersJournal and NewsForge in the past few weeks? Now,
one more big change to announce. We are now OSTG- Open Source Technology
Group. Come see the changes on the new OSTG site. www.ostg.com
Message details and related content
--- "Antonino A. Daplas" <adaplas@hotpop.com> wrote:
I guess most people will agree that writing to the hardware belongs
in the kernel, while choosing the correct values to write belongs to
userspace.  The unanswered question is the middle part, who does the
verification?
At OLS we talk about a system like this:
1) user owns graphics devices
2) user sets mode with string (or similar) format using ioctl common to
all drivers.
3) driver is locked to prevent multiple mode sets
4) common code takes this string and does a hotplug event with it.
5) hotplug event runs root context in user space 
6) mode is decoded and verified, this may involve a little process that
maintains the DDC database and reads a file of legal modes. Other
schemes are possible.
7a) mode is set using VBIOS and vm86, signal driver mode is set
7b) the register values needed to set the mode are passed into a root
priv ioctl.
8) driver is unlocked.

In this model all of the verification happens in user space. If you
want to set modes other than the ones from DDC you have to add them to
the config file. There is no need for DDC support and mode verification
in the kernel.

To give credit this is Alan Cox's design.

=====
Jon Smirl
jonsmirl@yahoo.com


		
__________________________________
Do you Yahoo!?
New and Improved Yahoo! Mail - Send 10MB messages!
http://promotions.yahoo.com/new_mail 


-------------------------------------------------------
This SF.Net email is sponsored by OSTG. Have you noticed the changes on
Linux.com, ITManagersJournal and NewsForge in the past few weeks? Now,
one more big change to announce. We are now OSTG- Open Source Technology
Group. Come see the changes on the new OSTG site. www.ostg.com
Keyboard shortcuts
hback out one level
jnext message in thread / next series-diff position
kprevious message in thread / previous series-diff position
ldrill in
Escclose help / fold thread tree
?toggle this help