Bug 28779

Summary: Advanced Screen and Output management
Product: xorg Reporter: Sergey Kondakov <virtuousfox>
Component: * OtherAssignee: Xorg Project Team <xorg-team>
Status: RESOLVED MOVED QA Contact: Xorg Project Team <xorg-team>
Severity: enhancement    
Priority: medium    
Version: unspecified   
Hardware: All   
OS: All   
Whiteboard: 2011BRB_Reviewed
i915 platform: i915 features:
Attachments:
Description Flags
example 1 - multi-head|xinerama.jpg
none
example 2 - multi-head|multi-screen.jpg
none
example 3 - multi-seat.jpg
none
example 4 - complex setup.jpg
none
kms problem - example.jpg
none
kms problem - proposal (#2).jpg
none
concept #1+#2+#3.jpg none

Description Sergey Kondakov 2010-06-27 04:49:41 UTC
in summary, i have two proposals for improvements: "easy" (as not too much pain for developers) and "hard" (as teeth-grindingly difficult with possible changes beyond just X and major protocol alterations).

an easy one will be:
1) make multi-head/multi-screen (non-xinerama screens) configuration possibility a requirement for all in-tree X drivers.

i heard some term called "zaphod mode" - i believe this is some cryptic term by which some few people call such simple thing which should be pretty much vendor/driver-independent and easy to set up/use.

we all use "virtual desktops" but those are _real_ multiple desktops !
why would you want them to always be glued together into one giant virtual desktop (xinerama) ? 0_o
(rhetorical question - i know why. this is just different use-case)

also looks like that KDE devs decided to ignore multi-screen configurations in their new DE, as it (particularly, plasma) goes apeshit when sees it (from what i remember in kde4.0-times) but can we blame them as such serious dudes as Intel devs apparently have ditched such thing completely from their drivers and not talking about better way to get it back ? o_0

and the juicy hard one will be:
1) same as previous, except make it more seamless and straight-forward as randr manages outputs per one screen (and less hackish since using several instances of same driver for same card is not a good way to do things).

On Windows(tm) its default behaviour (in XP(r) anyway) to add new outputs as screens of available window space but not create one giant desktop _with common elements_ as it tends with FreeDesktop environment (X,Xinerama, x\g\kdm, kicker\plasma\gnome-panel, kwin\metacity, etc.). with FreeDesktop it's quite unorthodox to get separate screens per same X instance:

assigning screens at "ServerLayout" section _and_ defining exact outputs for each screen is NOT enough, you _must_ also define two same "device" sections even for same device with one difference - "Screen <number>" directive so it will not tend to take everything on same screen even though you explicitly defined that in "layout" and "screen" sections.
only then you will be able to launch applications with "DISPLAY=:0.1 mplayer"-like command and manage windows there with your pointer without fear of stuff on first screen affecting it. xrandr doesn't show all of the screens in this case - you have to explicitly define it, e.g. "DISPLAY=:0.1 xrandr" will show outputs and properties for Screen 1 but never for both at a same time.
That's what i call "Multi-Head", dammit - when several _people's heads_ can make use of hardware while only one "seat" is involved in operating.

otherwise screen elements tend to fill up all available space. it, probably, semi-configurable but sucks so much :(

2) as KMS now involved and outputs configured at boot time we have little problem and nice opportunity, both having same root - VTs (kinda kernel-level "screens", in my understanding) shared per outputs and TTYs shared per card, single card.

so the problem will be - if you stick several devices with different parameters (e.g. FullHD panel + TV) to several outputs before booting kernel those devices Will have their native resolution set which is right but VT will be all squeezed to the size of smallest device (using 'clone' picture from smallest device across all of them).
maybe because there is only one kernel framebuffer per whole device ?

the opportunity will be to turn this situation around with use of KMS - instead of just cloning picture of squeezed VT, to assign 12 separate VTs to each of the detected devices dynamically while also dynamically binding TTYs to those VTs.
in which case same ttys could be binded (or not, at will. e.g. you will not want all of them to be accessible in restricted station but one) to same VT numbers as before but displayed separately (separate fb per active output ?).

in this case one X instance will be able to run with same VT number\s as concurrent one on a different output\s, also providing Multi-Seat users with ability to use non-X VTs or/and own several X instances while not disrupting anything for other X users.

if X instance is trying to use output which already taken by another X instance then next VT (8-12 ?) of that output should be used for that screen(multi-screen)/screen-part(xinerama) with separate tool (xrandr?) or hot-key (while pointer is on that screen\screen-part) should be used to switch them.

3) speaking of Multi-Seat - it should be achievable on a single card (without crutches as Xephyr !)

with KMS/non-root X it's not several instances of same driver for same card should spawn per X instance but several X instances should be able to benefit from single driver (or from kernel drm code directly ?).

so, if someone will come up with practical idea of how to achieve that (like making xf86-<driver> instance shared per X-server instances somehow or even independent of them. and with most work done with KMS in kernel it may be possible, right ?) then DE developers could add little menus in theirs DMs to select output/screen configuration (after all, those ARE _Display Managers_ and they should, you know, manage displays) before session's start (and keep last used configuration per user, or with KMS/non-root X involved - in each user's home). this would be sweeeet

of course, it would be also nice to have a non-X analogy for master/slave input assignments to create input(keyboard, mouse,etc.)/output(+bunch of independent VTs) groups and effectively getting Multi-Seat completely independent of X-server, but this is probably too much.
_ _
with all 3 implemented it will be possible to manipulate all of inputs and outputs of your machine in pretty much any way imaginable.

guts tell me (i'm not a programmer, so it guts and not brain) that direction to implement all that is to center around outputs, their properties, common properties of theirs displays and encoders, effective video memory management; making output management more generic and card/vendor/driver independent or putting it simply - expanding stuff that's going on right now with KMS and randr without hesitation and fear for some "backward compatibility" and stuff.
the scalpel is already taken by Intel devs, they say, but where are needles ?

it is nice to have network-transparent environment where it is so damn easy to mix local and network resources from separate machines as UNIX-like world presents (and X and FreeDesktop are not exceptions) but when you have bunch of people and hardware in close proximity of each other and some fast-paced input/output activity is involved (for information ;) - it's much better to use each hardware potential to full extend until you have to make use of networking.
(and no, in contradiction to my previous mentions - Windows(tm) doesn't reveal that potential any better, in fact it reveals Jack shit)

some interesting links: 
http://www.mail-archive.com/xorg@lists.freedesktop.org/msg10484.html
http://www.mail-archive.com/xorg@lists.freedesktop.org/msg10519.html
http://www.mail-archive.com/xorg@lists.freedesktop.org/msg10524.html
http://www.mail-archive.com/xorg@lists.freedesktop.org/msg10529.html
http://www.mail-archive.com/xorg@lists.freedesktop.org/msg10537.html
http://www.mail-archive.com/xorg@lists.freedesktop.org/msg10623.html
http://www.mail-archive.com/xorg@lists.freedesktop.org/msg10646.html
http://www.mail-archive.com/xorg@lists.freedesktop.org/msg10648.html
http://www.mail-archive.com/xorg@lists.freedesktop.org/msg10649.html
						
http://www.x.org/wiki/Projects/XRandR
http://www.thinkwiki.org/wiki/Xorg_RandR_1.2
http://wiki.debian.org/XStrikeForce/HowToRandR12
http://intellinuxgraphics.org/dualhead.html
						
http://www.nvnews.net/vbulletin/showthread.php?t=101547
http://wiki.archlinux.org/index.php/Xorg_multiseat
http://www.gentoo-wiki.info/HOWTO_Multiseat_X
https://help.ubuntu.com/community/MultiseatX
						
http://www.phoronix.com/vr.php?view=15018

colourful examples are coming soon...

PS: heh... a lot of words when you put it in text, i hope someone will be able to make some sense of that mind-dump.
wanted to write it all up like a year ago when first heard about KMS and multi-pointer stuff but thought this is asking too much in unclear fashion.
that until i've noticed major gpu memory management, drm and fb rework, dri2 transition, xrandr improvements being done and decided that this is just a time to post/suggest something like this until its too late and stuff is stabilized in places.

what a hell is "zaphod" anyway ? it's not an english word for sure...
Comment 1 Sergey Kondakov 2010-06-27 04:52:56 UTC
Created attachment 36542 [details]
example 1 - multi-head|xinerama.jpg

simpliest milti-head/xinerama set up, configured statically via xorg.conf or dynamically via randr.
Comment 2 Sergey Kondakov 2010-06-27 04:55:22 UTC
Created attachment 36543 [details]
example 2 - multi-head|multi-screen.jpg

simpliest multi-head/multi-screen setup, configured only statically via xorg.conf and in very non-intuitive way

known as "zaphod mode", strangely
Comment 3 Sergey Kondakov 2010-06-27 05:00:17 UTC
Created attachment 36545 [details]
example 3 - multi-seat.jpg

simpliest multi-seat setup that can be.

[!] VT Switching is DISABLED while two separate X's are active
[!] manual configuration of global x\g\kdm config is required

been tested on two nvidia cards with proprietary driver (build-in + pci-e and pci-e + pci-e) while retaining full acceleration on both but others should work, probably with limitations right now.
Comment 4 Sergey Kondakov 2010-06-27 05:08:08 UTC
Created attachment 36546 [details]
example 4 - complex setup.jpg

most sophisticated setup currently theoretically possible.

combines multi-seat with xinerama and multi-screen:
first X instance took first card and all outputs on it but spawned two instances of video driver for it - one to create first screen/xinerama desktop and the other for second, simple screen.
second X instance took second card and one more xinerama desktop, for another user, was created, dynamically or statically - no difference.

possible only theoretically since, they say, only radeon driver support multi-head/multi-screen at all and even it, probably, will cut off all the outputs which not form one screen (only xinerama or only multi-screen, not both)
Comment 5 Sergey Kondakov 2010-06-27 05:11:08 UTC
Created attachment 36547 [details]
kms problem - example.jpg

boot screen looks ugly if several outputs are active at boot-time and they have different resolutions - image sized for smallest one.

and, from what a remember, only card initialized by BIOS is able to produce image at this stage.
Comment 6 Sergey Kondakov 2010-06-27 05:24:42 UTC
Created attachment 36548 [details]
kms problem - proposal (#2).jpg

here , each output is designated with its own fb-device interface and can dynamically allocate video memory for its need according to its state and resolution/depth.

separate bunch of VTs is spawned for each one so Xs can live across several outputs and cards and without suid (not for xinerama screen though since it needs common buffer space for whole screen inside same card).

[.] same image is scaled on this picture but in reality each output should configure terminal size on it according to resolution and font used so it would not look stretched.

[?] to switch VTs, like on "TV-0" example while not using X, a way to bind only selected device to output must be created, like X does. and it should work before login is made.
but default behaviour still should be to bind all ttys to all VTs and all inputs to all outputs, just images of them renderred separately and the rest configuration is user's business.
Comment 7 Sergey Kondakov 2010-06-27 05:35:03 UTC
Created attachment 36549 [details]
concept #1+#2+#3.jpg

full concept.

Xs are non-root and can assign screens for themselves in whatever way any user desire, preferrably in all ways - statically (xorg.conf), pre-login (in x\g\kdm screen, as gdm in ubuntu let's you to choose default keymap for your session but for screen configuration) and dynamically (xrandr-like).

on picture - 3 users spawned three instances of X:
user#1 (purple) using xinerama screen on first card plus second, independent screen (multi-screen) on second;
user#2 (yellow) using one simple little screen with gnome session;
user#3 (blue) using xinerama screen before user#1 took it away (with #3's permission and manual switch on VT8) for his multi-screen. but xinerama configuration was not reseted and working in "background".
Comment 8 Adam Jackson 2018-06-12 19:10:07 UTC
Mass closure: This bug has been untouched for more than six years, and is not
obviously still valid. Please reopen this bug or file a new report if you continue to experience issues with current releases.
Comment 9 Sergey Kondakov 2018-06-12 21:04:02 UTC
(In reply to Adam Jackson from comment #8)
> Mass closure: This bug has been untouched for more than six years, and is not
> obviously still valid. Please reopen this bug or file a new report if you
> continue to experience issues with current releases.

Closing abandoned or obsolete reports is fine and all BUT this is usability concept, not an issue of some particular codebase. I guess that it can be realised now with Wayland or by cleverly using device access separation in udev & evdev but in practice, display management, especially before logging in, still sucks. The worst offender, though, is kernel's VT console. kmscon and systemd-consoled are both dead now and there is no alternatives in sight.
Comment 10 Adam Jackson 2018-12-13 20:15:06 UTC
This is really a design discussion, not a bug. Please follow up on xorg-devel@ or the xorg gitlab instance.

Use of freedesktop.org services, including Bugzilla, is subject to our Code of Conduct. How we collect and use information is described in our Privacy Policy.