Qt
KDE
News
Posted by bnilsen
 in Qt, KDE, News
 on Thursday, August 09, 2007 @ 12:33

No, this is not a wet dream — it’s all true; with Qt 4.4 you will be able to get rid of issues you have had with flickering. Just to make sure we are talking the same terminology, I’ve created a small movie demonstrating the problem:

Given the fact that we’ve passed the 90’s, this is not acceptable and up to Qt’s standard. As of today, every single widget in Qt has a native window that is managed by the window system — in this case the X server. The flicker you see on the movie is a result of multiple X windows trying to synchronize and play funny games with the X server.

But how can we improve? With Qt 4.1 we took a huge step towards resolving similar issues when introducing the backing store, but it can’t cover resize flicker as it’s fully caused by the X server. The only consequent and necessary next step is therefore to bypass the X server and let Qt do the job. In order to do that we must make our child widgets non-native — a.k.a. alien — such that X doesn’t know they exist. That sounds pretty easy, doesn’t it?

Moving away from native windows isn’t done over the night. Window handles have been in Qt from day one, and consequently our GUI kernel is heavily built around them. The main technical challenges in that respect have been core functionality such as mouse events, widget cursors, enter/leave events, drag’n'drop and so on. With native windows all widget events are related to a window handle and our job is simply to map system events into Qt events. Consequently, when we make our child widgets windowless, all events are related to the top-level window and we have to figure out ourselves which widget is the correct receiver for the given event. To make it even more interesting, all this depends on the platform your application is running on, so there’s no way it can be fixed properly in cross-platform code.

A somewhat funny scenario is drag’n'drop on Windows which is based on Object Linking and Embedding (OLE). Every drag’n'drop operation consists of at least two COM objects representing the source and the target. A widget acting as a drop target must register its window handle such that Windows can communicate between the source and the target. It’s not hard to imagine that some black magic is needed when the top-level window is the only native window we have, possibly acting as source and target at the same time.

And what happens if we suddenly create a widget that requires a window handle? Consider the scenario where a QGLWidget is the central widget of QMainWindow. Widgets with the Qt::WA_PaintOnScreen attribute set — in this case the QGLWidget — must have a native window to function properly. That’s OK, but the fun starts when we have to deal with clipping between native and non-native windows. Clipping between native windows is indeed handled by the window system, but it doesn’t help much when the only clipping we get for free is between the top-level and the GL widget.

Needless to say, but apart from reducing flickering, an application with windowless child widgets is much more light-weight and resource friendly as we don’t allocate native windows for every single widget.

I wish we could have taken this step a bit earlier, now that KDE 4 is around the corner, but there’s little point in adding hacks and workarounds now to avoid flicker with Qt 4.4 bringing the real solution soon.

I could probably tell you a lot more, but I’ll keep this blog short. Currently this work is going on in a separate research branch so it won’t show up in our snapshots, but within few weeks you should be able to try it out. I’ll blog with more details as soon as we have it integrated.

Last, but not least, here’s the obligatory movie showing how resizing with windowless child widgets looks like:

39 Responses to “Qt Invaded By Aliens — The End of All Flicker”

» Posted by Max Howell
 on Thursday, August 09, 2007 @ 12:56

I love the results, such flickr really sucks, I assume it will also help where you move several widgets, and you can see the widgets move one after another (if you are moving many anyway).

I haven’t written any code that requires child widget WinIDs, but I imagine some people have, what will they do? And perhaps I’m glad if you will decide not to care, as the results in that movie are superb.

OSX I notice neatly avoids this kind of flickr. Is that just because it buffers everything everywhere?

» Posted by Will Stokes
 on Thursday, August 09, 2007 @ 13:17

If I’m correct, this improvement is literally *huge*. Flicker is annoying yes, although as you said Qt 4.1 laregely fixed the most annoying flicker issue. What really peaked my interest was your comment about things being much lighter weight. Correct me if I’m wrong but widget construction should be cheaper with this change, right? If so you’ve really gotten me exicted. In the commercial app I’m writing we have some very complicated dialogs with a lot of widgets. Creating these dialogs is so painful we cache and reuse the dialogs after resetting them if we need to pull them up again. The pain is most particularly felt on OSX. I’m hoping the work you’re describing we’ll help reduce the widget construction penalty and speed up dialog and main window construction and make Qt based apps feel far more snappy.

» Posted by Matthias Kretz
 on Thursday, August 09, 2007 @ 13:30

I’m using QWidget::winId() for phonon-xine’s VideoWidget implementation. But in addition I’m calling setAttribute(Qt::WA_PaintOnScreen, true) so this widget should just behave the same as it did with previous Qt versions (I’d expect).

Anyway, great work!

» Posted by Ossi
 on Thursday, August 09, 2007 @ 13:32

maxH: your world is slightly too black&white. ;)
there is nothing to prevent adding a WA_NativeWindow flag - in fact, the mentioned QGLWidget would auto-set it. not sure it is actually worth adding another flag though - i’ve yet to come up with a case where WA_NativeWindow but no WA_PaintOnScreen is needed.

» Reply from bnilsen
 on Thursday, August 09, 2007 @ 13:37
bnilsen

Max: If you’re talking about moving of child widgets, the backing store optimizes that very well in most cases, but you might see improvements there as well.

Code that requires window ids will still magically work, as we’ll force creation of a native window when calling QWidget::winId.

Mac OSX has its own backing store which is a bit smarter than X is :)

» Reply from bnilsen
 on Thursday, August 09, 2007 @ 13:50
bnilsen

Will: I don’t have any numbers yet, but widget creation will definitely be much cheaper without having to create native windows.

» Posted by monkeykong
 on Thursday, August 09, 2007 @ 13:55

impressive!

» Posted by Maurizio Monge
 on Thursday, August 09, 2007 @ 14:25

I’m very happy with TT working out this problem, because i’ve been suffering for some time for not being able to put widget animation effects in my application, as flicker was making such effect actually look awful.

Anyway, how is this problem going to be solved under X? I mean, having just one toplevel window is probably not optimal because most child widget do not want all mouse movement events, but only enter/leave notifications, and getting all such events would put some load over the connection with the X server. Another idea that came up discussing about this problem was using InputOnly windows, this would solve this problem but there should still be one window per widget, and widgets would not be so ‘lightweight’.
I’m just a bit curious about which will be the final solution, maybe i’m simply overstaing the performace hit of receiving all mouse events through the connection with X.

Thanks a lot for fixing this and keep up the good work!

» Posted by nick
 on Thursday, August 09, 2007 @ 15:18

this is all great. but why don’t you just improve xorg? because its code is too ugly? :)

» Reply from bnilsen
 on Thursday, August 09, 2007 @ 15:39
bnilsen

Maurizio: We haven’t noticed any performance issues with the processing of events received from X. We could of course look into it and see if there’s something we can optimize, but I suspect that would be micro optimization :-)

» Posted by Rince
 on Thursday, August 09, 2007 @ 15:47

I wonder why nobody’s pointed out yet that you have the “flickerless” and “flickerful” videos swapped. :)

» Posted by Andrew Medico
 on Thursday, August 09, 2007 @ 15:47

Am I mistaken, or will this change (no native widgets for child widgets) make Spy++ unable to usefully inspect Qt 4.4 applications?

» Posted by Maurizio Monge
 on Thursday, August 09, 2007 @ 15:52

bnilsen: good to hear, it is always nice when in real life there are fewer problems than expected! :)

» Posted by Bernhard Rode
 on Thursday, August 09, 2007 @ 15:53

Great work,
but it maks me sad, that It won’t make it into KDE4.0… because this one is a big one for a Desktop Environment….

» Reply from bnilsen
 on Thursday, August 09, 2007 @ 15:54
bnilsen

Andrew: True, Winspector/Spy++ will only see the top-level window.

» Posted by xbix
 on Thursday, August 09, 2007 @ 16:15

So there’s no other way than simply dropping native widgets? How do flickerless native GUIs work in general then? Or is it related to the fact, that Qt paints the widgets itself?

» Posted by Ossi
 on Thursday, August 09, 2007 @ 16:16

re andrew’s comment: i guess that means another command line switch to enforce native widgets everywhere. :)

» Posted by Andreas
 on Thursday, August 09, 2007 @ 16:44

Wow, exciting! Changing the guts of QWidget after so many versions really does not sound like a cakewalk…
Now you have us all curious, and I have a number of questions:
- Are there any approximate numbers for CPU and memory efficiency gains?
- Does it have an impact on input to display latency? That is something where I have never seen Windows beaten, and OS X was even worse than X11 when I tried it.
- How does it influence performance of remote X sessions? (OK, who cares? ;) )
- When will the new code be released in an official version, i.e. 4.4?

» Posted by Jengu
 on Thursday, August 09, 2007 @ 17:02

The second video looks like an awesome improvement and is something that has always bothered me. As a commercial product, this seems like an excellent feature for QT to have. On the other hand, as part of the general Linux software stack, might this be solving the problem at the wrong level? I’m not too familiar with Xlib, but this seems like something that should be fixed at the X level. Why is all the weirdness happening between the native child windows and the server in the first place? I’d prefer a fix that helped my GTK / Xlib / your_favorite_toolkit apps as well.

Also, wasn’t XCB supposed to help with this problem somehow?

» Posted by Sébastien Laoût
 on Thursday, August 09, 2007 @ 17:28

A few questions:

1. Did you tryed to improve XOrg? Why did you deamed that possibility?

2. What about resizing the main window? I mean currently, when you resize a window, the border resizes, and the content keep its old size for a little moment. Windows would feel more “rock solid” if the window manager waited for the main window to be reflowed & repainted offscreen and then the window and the resized border be painted all at once. I think GNOME solved that with Compiz or something else (I don’t remember exactly). Of course, it’s a KWin problem (perhapse fixed in KDE 4) but this involves the same X11 protocol adaptation as your hack requieres to have flicker-free widgets, IMHO.

3. When will Qt use XBL (or was it XCB?), the replacement of XLib, but asynchronous, this time, so Qt does not wait X11 replies for every single operation? This would result in huge latency improvements! We heard about it a few years ago but it seams dead: non news lately.

» Posted by Andreas
 on Thursday, August 09, 2007 @ 17:39

As long as X11 is asynchronous (with all its benefits, by all means), resize and move operations will also be asynchronous, and flicker will appear as X11 first clears the exposed area (or leaves a trail of junk behind), then sends an expose notification, allowing the client to redraw. You can probably avoid the flicker with clever use of XFlush, but that will mean performance goes down the drain. With this solution, we don’t only remove the flicker, and remove the need to wait for X11 roundtrips, but we also free up X11 resources as Qt no longer needs the subwidgets on the server side. Which means less memory consumption :->. And on Windows, it means people can create more widgets than they could before…

» Posted by GN
 on Thursday, August 09, 2007 @ 18:25

Max Howell. I’m not a grammar nazi, I just thought it was fun that you spelled flicker as flickr :-) Strange how we are influenced by brand names. It’s simply del.ici.us

» Posted by Jengu
 on Thursday, August 09, 2007 @ 19:02

@Andreas: The problem with X11 roundtrips I believe is what XCB is supposed to solve. It returns handles to results rather than results directly, so you don’t have to block until the result is actually used (the handle is “cashed in”). Again, I am not an X hacker though :)

» Posted by Andrew Fedoniouk
 on Thursday, August 09, 2007 @ 20:18

For whom it may be interesting:

Here is another windowless toolkit: http://harmonia.terrainformatica.com/.
That is Harmonia toolkit written in D language ( www.digitalmars.com )
Live demos (windows) are here: http://harmonia.terrainformatica.com/HarmoniaDemo.zip (1.7 MB)

» Posted by Matthias
 on Thursday, August 09, 2007 @ 20:18

Sebastien, Xlib is by far not as synchronous as you imply it was. Creating an incompatible X-server was never an option, no. It wouldn´t help existing customers (this server will never make it onto commercial UNIX systems) and it wouldn´t help our Windows users. Our solution now is cross platform, not restricted to X11.

» Posted by Ossi
 on Thursday, August 09, 2007 @ 20:59

i’m confused … XCB replaces Xlib, and it is still X11-based. there is no reason why it could not be another lib in 3rdparty/ for those antiquated commercial UNIXes.
whether it would be actually such an improvement is another question …

» Posted by Max Howell
 on Friday, August 10, 2007 @ 00:30

At the end of the day it is impractical to expect Trolltech to spend the time fixing X.org. The patch may be simple, but the social requirements ridiculous. It is very hard for one project in the commercial or open source world to influence the direction of another.

Far simpler is it to implement what you need and hope the other project notices and decides to do it too.

Trolltech did the right thing. Don’t hold it against them that they didn’t try or wait to get X.org to do it, it’s bloody hard.

» Posted by dejavu
 on Friday, August 10, 2007 @ 00:32

Dejavu. Delphi does this on Windows, too - for more than one decade now. With the difference that it came with a tool to follow native window messages and its own ones as well, iirc (don’t remember the name, didn’t code in Delphi for a long while).

What about the downsides of this approach? E.g. protocols like NX base their compression on the X11 window changes, don’t they!?

» Posted by skierpage
 on Friday, August 10, 2007 @ 02:37

Xlib is implented as a layer on top of XCB in X.Org 7.2, see http://www.x.org/wiki/Releases/7.2. Using XCB directly can eliminate some synchronous requests and server round-trips, but I’m sure TT is already smart about minimizing these in Qt.

bnilsen, “tell you allot more”: “a lot” is two separate words, just like “a bit” and “a little”. (Your English is way better than most native speakers!)

Will Stokes, “really peaked my interest”: that’s “piqued” as in hooked, it’s from the French.

I’ll stop now, this fundamental re-engineering is damn hard and you don’t have time for grammar nazis.

» Posted by Martin
 on Friday, August 10, 2007 @ 03:14

How does this affect performance when running X over a slow network link? I was concerned that it might be slower in some cases, since the X server can no longer simply rearrange mostly unchanged sub-windows when the parent is resized, but rather has to ask the client to redraw the whole thing. I am not so familiar with the inner workings of X so this may be totally of the mark.

Nevertheless; did you profile this change for the communication-limited case? Thanks!

» Posted by felix
 on Friday, August 10, 2007 @ 05:57

it looks like the relationship between swing and awt of java.
but there are a z-order problem if we using awt’s heavy component and swing’s light component in the same top-level window.
the suggest from swing is that do not using awt and swing like this way.
how do you deal with this problem on QT?

» Posted by NoTiG
 on Friday, August 10, 2007 @ 06:12

can someone tell me if this knowledge applies to gnome? I seem to have jagged windows problems when i move them around and it annoys me so, despite having a powerful machine

http://www.youtube.com/watch?v=8gh7ABSs1NY

» Posted by thomas muders
 on Friday, August 10, 2007 @ 07:05

Not only Delphi does it on Windows, but I think a lot of bigger Windows applications like IE and MS Office also do it, I remember some blog entry by a MS developer that they had to do it because of performance reason and because there are in fact hard limits on the number of native widgets.

» Posted by Moma
 on Friday, August 10, 2007 @ 07:43

The XCB library is going to replace the old XLib.

Do you think that XCB will give better performance and drawing speed?

See:
http://xcb.freedesktop.org/

http://en.wikipedia.org/wiki/XCB

» Posted by Moma
 on Friday, August 10, 2007 @ 07:46

Sorry…
XCB was already discussed by others in the earlier postings.

» Posted by gogoliu
 on Friday, August 10, 2007 @ 07:47

I think widget system should base on canvas.

» Posted by lu_zero
 on Friday, August 10, 2007 @ 12:03

Hmm, what about providing links for those who don’t use flash? (yes there are some still around)

» Posted by nick
 on Friday, August 10, 2007 @ 12:43

lu_zerolu_zero: apt-get install youtube-dl

» Posted by dpeterc
 on Friday, August 10, 2007 @ 17:36

The gray old ugly bloated eighties Motif can do resize without flicker (*).
So the problem might not be in X Windows, but in the way Qt uses it.
You get rid of majority of flicker, if you do not clear window on
resize, but just paint into it. And if the Qt’s geometry management and
draw code would not be slow, even a full repaint would not be noticeable.
So why not just profile the Qt, see where time is lost, and fix that?

(*)
Check my application ArahWeave from
http://www.arahne.si/installAWeave.html
Run ArahWeave, go in Thread pattern -> Colors,
http://www.arahne.si/colorsPantone.gif
and resize this dialog.

Leave a Reply