So last time, I discovered that Ubuntu 12.04 has patched Qt to support multitouch. But since Qt doesn’t support MT on X11, I’m considering having a layer that allows me to either use Qt’s MT events or to handle XInput 2 events. Like last time… we start with Qt’s fingerpaint application and try to get it working with XInput 2.

To do this, we’re going to sub-class QApplication so that we can re-implement the virtual function QApplication::x11EventFilter().  This allows us to see, inspect, handle, intercept, or pass on any events that the X server sends us. We’ll call our class TApplication and make the function print something out when we get on of XInput’s touch events XI_TouchBegin, XI_TouchUpdate, and XI_TouchEnd:

bool TApplication::x11EventFilter(XEvent *ev)
{
    bool handled = false;
    XGenericEventCookie *cookie;

    // All XI2 events are GenericEvents
    if (ev->type != GenericEvent)
        return false;

    cookie = &ev->xcookie;

    if (cookie->extension != m_xi_opcode)
        return false;

    /**********************************************
     * BEYOND THIS POINT ARE ONLY XINPUT EVENTS
     **********************************************/

    switch (cookie->evtype)
    {
    case XI_TouchBegin:
        qDebug() << "XI_TouchBegin";
        break;
    case XI_TouchUpdate:
        qDebug() << "XI_TouchUpdate";
        break;
    case XI_TouchEnd:
        qDebug() << "XI_TouchEnd";
        break;
    default:
        qDebug() << "XI_somethingelse";
    }

    return handled;
}

Are you going to eat that cookie?

You might ask, “what’s the deal with the cookie?”  An XEvent isn’t actually a C struct, it’s a union of all the different event structs that the X sever uses.  All of them have a ‘type‘ as their first member so that you can discern what kind of struct this is.  We’re only interested in the type GenericEvent.  For these events it means that the ‘ev‘ pointer is a pointer to an XGenericEventCookie struct.  When we do ‘cookie = ev->xcookie‘, it’s equivalent to ‘cookie = (XGenericEventCookie*)ev‘ — but much more elegant.

The GenericEvent is the event type used for extensions to the X Server — things that weren’t thought of when they originally set up the specification.  While the XGenericEventCookie struct has some useful info… what you really want is the data in its ‘data‘ member.  Normally we have to call XGetEventData(), but in Ubuntu’s Qt it already fetches (and frees) it for us.  (See [2] for more details.)

Taking Responsibility

Because Ubuntu is patched with XInput 2 support… it was kind enough to set up everything for us in the background. For example, it tells the X server that we want touch events, pre-fetches our cookie, and then cleans up our cookie crumbs afterwards. However, since we can’t always depend on our mommy (Ubuntu) looking out for us… we need to be Big Boys and set up all the background stuff for ourselves.  We start by disabling multitouch from Qt in the ScribbleArea:

 ScribbleArea::ScribbleArea(QWidget *parent)
     : QWidget(parent)
 {
-    setAttribute(Qt::WA_AcceptTouchEvents);
+    /*setAttribute(Qt::WA_AcceptTouchEvents);*/
     setAttribute(Qt::WA_StaticContents);

If you re-run the application after doing this, you won’t see any X11 events. All events that you want to receive from the X server must be explicitly enabled (even mouse clicks). There is no notion of ‘cascading’ events. Every window. No exceptions. You can’t just set these flags to your top-most parent window and have all the windows get the events. Every single window has to do this for itself.[1]

So instead of setAttribute(), we call our own function:

    TApplication::sendTouchEventsTo(this);

And inside that function we declare that we want touch events for this window:

/* static */
void TApplication::sendTouchEventsTo(QWidget *w)
{
    XIEventMask mask;
    Window win;
    Display *dpy;

    dpy = QX11Info::display();
    win = w->winId();

    memset(&mask, 0, sizeof(XIEventMask));
    mask.deviceid = XIAllMasterDevices;
    mask.mask_len = XIMaskLen(XI_LASTEVENT);
    mask.mask = (unsigned char*) calloc(mask.mask_len, sizeof(char));

    XISetMask(mask.mask, XI_TouchBegin);
    XISetMask(mask.mask, XI_TouchUpdate);
    XISetMask(mask.mask, XI_TouchEnd);

    XISelectEvents(dpy, win, &mask, 1);

    free(mask.mask);
}

This must be run for every widget that you want to have touch events. Using setAttribute() more or less does the same thing.

Notice that we got the Window ID from the widget so that we could give that back to the XServer. The documentation for QWidget::winId() says:

Portable in principle, but if you use it you are probably about to do something non-portable. Be careful.

…This value may change at run-time. An event with type QEvent::WinIdChange will be sent to the widget following a change in window system identifier.

…and they’re not joking, either.  From the time that this code is run in the constructor to the time you receive your first touch event… the window ID changes twice for me.  In fact, I think this may be part of the reason why the pre-compiled fingerpaint demo will sometimes work and sometimes not.  So, obeying the docs, we handle the event:

bool ScribbleArea::event(QEvent *event)
{
    switch (event->type()) {
    case QEvent::WinIdChange:
    {
        TApplication::sendTouchEventsTo(this);
    }   break;
    /* ... */

Wrapping Up

You can download the code in the file xinput2-part1.tar.bz2 (or browse it here).  Next time we’ll talk about decoding the events and sending them to our widget.

Resources


[1] – …and every QWidget is an X11 Window (or two or three). That’s one advantage to QGraphicsView and QtQuick — they’re typically all contained in a single window.

[2] – The Qt convention seems to be that XGetEventData() is called before entering QApplication::x11EventFilter() and that XFreeEventData() is called in QApplication::x11ProcessEvent(). This is what the Ubuntu patch does, too. However, I also wrote the code to handle the non-Ubuntu-patched case. See tapplication.cpp for detailed comments in the code.

Advertisements

Qt does not support for XInput 2.2’s multitouch protocol.[1] So instead of whining, I approached it as an opportunity to learn XInput2 and possibly give a more awesome MT experience in Composite. Qt allows you to catch and handle raw X11 events using QApplication::x11EventFilter().

As a starting project, I took the Qt fingerpaint demo to make it work with XInput2. With Ubuntu 12.04:

$ sudo apt-get install xinput libxi-dev qt4-demos

N.B. I already have Qt4 tool and -dev packages and a bunch of stuff installed, so YMMV.

Next, using my Lenovo Ideapad, let’s confirm that fingerpaint is broken:

$ /usr/lib/qt4/examples/touch/fingerpaint/fingerpaint

And contrary to what I expected… multitouch works!! However, sometimes it works and sometimes it doesn’t.[2] Usually if you wait until “Fingerpaint” appears in the title bar you’re good to go.

As it happens, Ubuntu 12.04 has a patch that enables Qt’s multitouch. 🙂

So next, I compiled the Ivory application that I did for MeeGo. And:

Got touch without getting TouchBegin for id 58
Got touch without getting TouchBegin for id 58
Got touch without getting TouchBegin for id 58
Got touch without getting TouchBegin for id 58
Got touch without getting TouchBegin for id 58
Got touch without getting TouchBegin for id 58

Turns out that this is Ubuntu bug #1007847 and is easily fixed like this:

diff --git a/src/Application.cpp b/src/Application.cpp
index 05e1a92..4fc590b 100644
--- a/src/Application.cpp
+++ b/src/Application.cpp
@@ -59,7 +59,7 @@ namespace Ivory
         setFrameShape(QFrame::NoFrame);

         setScene(_scene);
-        setViewport(new QGLWidget);
+        setViewport(new QWidget);

         _scene->set_octave(0);
         QStringList ports;

…and it works!

This isn’t the blog I expected to write — so it’s been a pleasant surprise.  However, I am still interested in doing raw XInput processing… so I’ll still explore that in the next blog.


[1] – …as far as I know. Last I heard the status was that there was no support (nor plans to support) XInput 2.2 or anything else new in X11.  (…but patches welcome.)

[2] – I think I have an idea why it sometimes doesn’t work. More on that next time.

In seeking to continue Composite development, I needed a multi-touch environment.  All along, I figured that MeeGo would be that environment.  But the death of MeeGo means that this is no longer an attractive option.  I could do it, but it would be a stagnant OS, forever stuck at release 1.2.0.  I don’t want this for a development environment.

But as far as I know, MeeGo is still the only distro that gives you an out-of-the-box multi-touch experience.

Whatever distro I choose, I’ll need to fiddle with the Qt and Xorg packages on the system.  This is best done by creating new packages to replace the system ones (just in case you mess everything up, you can just revert back the package).  I chose Fedora because it tracks upstream projects very well, and also because I find RPM packaging much more enjoyable than DEB packaging.[2]

Here’s how I got a multi-touch-enabled Qt installed and working on Fedora 16.  I installed the 64-bit OS on my IdeaPad, so replace “x86_64” with “x86” if you’re using the 32-bit version.

Prerequisites

If you don’t know how to install a Linux distro, fire off commands on a command line, or boot to single-user mode and fix a couple things — then this will probably be too much for you to take on. (E.g. if you just Googled “single-user mode” — STOP NOW! :-))

After installing Fedora 16, you need to set it up for building rpms. I didn’t keep good notes during this phase, so this is probably not complete:

    $ sudo yum groups install \
        "Development Tools" \
        "X Software Development"
    $ mkdir ~/rpmbuild
    $ cd ~/rpmbuild
    $ mkdir BUILD BUILDROOT SOURCES SPECS RPMS SRPMS

Building the MTEV driver

The X11 input driver “mtev” is an interim solution to get multi-touch onto Linux devices like the N900, WeTab, N950, N9, etc. When XInput 2.2 finally hits the main-stream with a matching evdev driver, it will be obsolete.[4]

First, grab the sources and install the mtdev library:

    $ cd ~/code
    $ git clone git://gitorious.org/gabrbedd/xorg-x11-drv-mtev.git
    $ sudo yum install mtdev-devel

Note that the tarball you need is already in the repository. Also, this has been modified to (a) compile with the 1.11 xserver and (b) it includes my right-click-emulation patch. If you don’t like it… choose the version you want with git.

Build it like this:

    $ cd ~/rpmbuild
    $ cp -f ~/code/xorg-x11-drv-mtev/* SOURCES/
    $ mv -f SOURCES/xorg-x11-drv-mtev.spec SPECS/
    $ rpmbuild -ba SPECS/xorg-x11-drv-mtev.spec 2>&1 | log-50

It only takes a few seconds to build. Then you can install it like this:

    $ sudo rpm -Uvh RPMS/x86_64/xorg-x11-drv-mtev-0.1.13-10.0.x86_64.rpm

Building Qt with Multi-Touch

On my IdeaPad, Qt takes about 9 or 10 hours to compile.  If you have a fast machine with F16, I recommend doing it there.

Start by checking out my source repository and grabbing the Qt sources that you need.  (Note, if the URL’s are cut off in your browser… there are links near the end of the article.  See “Resources.”)

    $ mkdir ~/code
    $ cd ~/code
    $ git clone git://gitorious.org/gabrbedd/fedora-pkg-qt.git
    $ cd fedora-pkg-qt
    $ git checkout topic/multitouch-00
    $ wget http://get.qt.nokia.com/qt/source/qt-everywhere-opensource-src-4.8.0.tar.gz
    $ wget http://pkgs.fedoraproject.org/repo/pkgs/qt4/hi128-app-qt4-logo.png/d9f511e4b51983b4e10eb58b320416d5/hi128-app-qt4-logo.png
    $ wget http://pkgs.fedoraproject.org/repo/pkgs/qt4/hi48-app-qt4-logo.png/6dcc0672ff9e60a6b83f95c5f42bec5b/hi48-app-qt4-logo.png">hi48-app-qt4-logo.png

Now you can build it like this:

    $ cd ~/rpmbuild
    $ cp -f ~/code/fedora-pkg-qt/* SOURCES/
    $ mv -i SOURCES/qt.spec SPECS/
    $ rpmbuild -ba SPECS/qt.spec 2>&1 | tee log-00

Chances are that rpmbuild will choke on you, saying that you’re missing several required packages. You will need to install those. For example if it says that it needs “pkgconfig(foo)”, you can install it like this:

    $ sudo yum install "pkgconfig(foo)"

When done, install the packages in RPMS/. You probably want to do this:

    $ cd RPMS/x86_64/
    $ sudo rpm -Uvh qt{,-x11,-examples,-devel,-demos}-4.8.0-1.1.fc16.x86_64.rpm

To see which Qt packages you already have on your system, do:

    $ rpm -qa | grep qt

You only need to replace the ones that have a “4.8.0” version.

Verifying that it works

Reboot your computer and verify that it works. Run:

    $ /usr/lib64/qt4/examples/touch/fingerpaint/fingerpaint

If you can use 2 fingers to draw, then you are good to go! 🙂

Troubleshooting

X11 Fails to Start

If you reboot to a black screen, chances are that the MTEV driver is not kosher. Reboot to single-user mode and uninstall it.

    $ rpm -ev xorg-x11-drv-mtev

If you want to debug this… go right ahead. Chances are that its a failure in the driver’s PreInit() or something. However, I’m not going to be able to support you much with this.

No Multi-Touch

Do not be alarmed! Because this is driver is kind of a hack, you have to explicitly declare that the device should be managed by the mtev driver. xorg.conf.d files are installed for Cando, Sitronix, Hanvon, and ILI touchscreens. You might have a different one.

You can find out for sure by inspecting /var/log/Xorg.0.log. You should see something like:

[    26.125] (II) Using input driver 'mtev' for 'Cando Corporation Cando 10.1 Multi Touch Panel with Controller'

This was arranged by the /etc/X11/xorg.conf.d/60-cando-mtev.conf file, which looks like this:

Section "InputClass"
        Identifier              "Cando Multi Touch Panel"
        MatchVendor 		"Cando"
        MatchDevicePath 	"/dev/input/event*"
        Driver                  "mtev"
        Option                  "Ignore"                "off"
EndSection

Your touchscreen device will need a different Identifier and MatchVendor string. Inspect your /var/log/Xorg.0.log for clues.

Using Right-Click Emulation

Open up the terminal application. Put one finger down and hold it. Now tap the screen with a second finger. The context menu will pop up. This gesture is easier (at first) if you use two hands.

Resources

Here are links to (most of) the resources that we used above.

[1] The Xorg devs think this will end with the release of F17.  XInput 2.2 is in the review process to be added to xserver 1.12.  However, I’ve heard that Qt will not add official support for it, instead waiting for Wayland.

[2] I cut my teeth on DEB.  For MeeGo I had to learn RPM, and found that I liked it better.  Less time tracking down opaque debhelper issues and DEB styles.

[3] I don’t know where the original files were served up, these links are to a cached copy at Fedora.  You can also get it from their qt SRPM.

[4] …presuming that someone actually breaks down and adds support to Qt. Otherwise, it’s still useful for Qt dev.