Q: So, MeeGo is using mutter like GNOME3, right?

A: Um… yes and no.

When engaging people outside of MeeGo, I find a lot of confusion over what is MeeGo.  And when you try MeeGo, the confusion doesn’t actually get much clearer.  Is MeeGo a new Linux distro?  Is MeeGo a new desktop environment?  Is it a suite of apps?[MeeGo Handset UX]

The trouble is that the answer to all of these is “yes and no.”  Which doesn’t really help.

MeeGo is a new Linux distro.  MeeGo is not anything like Ubuntu or Fedora or Debian or even Damn Small Linux.  MeeGo is designed to be a really good starting point for people who want to make custom devices with custom graphical interfaces (i.e. phones, tablets, TV/set-top boxes, kiosks, etc.)

MeeGo is not a new desktop environment nor a bundle of apps.  Inside the embedded device community, the graphical “desktop” is called the “User eXperience” (“UX” for short).  MeeGo does provide several reference UX’s.  This includes the Netbook UX (which came from Moblin and uses Mutter), the IVI UX (which is a new thing based on Qt), and the (alpha-quality) Tablet UX (a new thing based on Qt).  But the thing that’s hard to grasp is that these are just proof-of-concept UX’s.  It is expected that device vendors will not use them, but replace the UX’s with their own custom UX.  And by “expected,” I mean… this is what you’re supposed to do (by design).

This UX concept is hard for users to grasp… because articles and videos show a user interface and you think, “That’s MeeGo.”  It was hard for me, personally, to grasp because the engineer in me thinks, “Why are we reinventing the wheel every time??”  However, that’s the plan — and AFAIK the only OS of its kind that takes this approach.[1]  It allows device vendors to do cool things with their device that help them stand out in the market place.  (In contrast… do users really care if they get a Motorola Android phone or an HTC Android phone??  Probably not.  They look/feel the same… so they buy based on price instead of features.)

And this is the reason why the MeeGo API (i.e. Qt) is so important… with everyone writing their own UX… having a preffered API makes it easier to write apps that will work on all the different custom UX’s that the vendors come up with.

[1] Edit: However, doing this is possible with other OS’s (like Android).  However, it’s the only one I know of that has this as a core implementation strategy.

MeeGo Conference 2011

May 30, 2011

In a Nut-shell

The MeeGo Conference was held in San Francisco on May 23-25, 2011. Summary:

“I have to admit #MeeGoConf started off flat but community got me pumped, now back @ Intel & getting more pumped. Cool MeeGo stuff on horizon” – Bob Duffy (@bobduffy)

Community, community, community…

"Sitting in the hack space..."The MeeGo Community (mostly Maemo community members, Nokia employees, and former Nokia employees) made the conference exciting. It was typically their enthusiasm and friendship that pumped energy into the conference.

When the keynote failed to deliver any news or announcements about devices shipping. Meanwhile, the developers in the hacker’s lounge discussed how they could take matters into their own hands.

The keynote was failing to register with the community. But when they brought established community member Robin Burchell (w00t) was brought on-stage for a demo, the audience became more responsive.

Most sessions, as well, were stocked full of people who knew each other well from forums, mailing lists, IRC, and the Maemo days. Tweets coming across the wire identified which topics were hot or exciting. (E.g. the AppUp encapsulator wowed several in the community).

The hacker’s lounge was hands-down the highlight of the conference. Most were the same people that I interact with day-to-day on MeeGo mailing lists and IRC, but there’s something different about meeting them face-to-face. In fact, Arjan van de Ven said as much when I met him, “That’s what these conferences are all about — names and faces.”

Call to Innovate

The most important session I attended was: The MeeGo Architecture Update with Sunil Saxena and Arjan van de Ven. They discussed the changes that happened in 1.2 and some of the changes planned for 1.3. They also discussed how some of the decisions were made (which has been discussed ad nauseum on the mailing lists). But he also made a call for innovation.

Arjan said that he was disappointed at the list of improvements in MeeGo 1.2, and that there is not enough innovation happening within MeeGo right now. Meanwhile, Android is pumping out releases and innovation at the same time. MeeGo will not compete if it does not innovate. “We need to have a mindset in MeeGo to do interesting technology,” said van de Ven. For example, the plans for Qt 5 look very compelling, and van de Ven said, “Can I get it in October??”

And this is not a call for Intel to innovate. It’s a call for innovation within the community. The Architecture board has said that in the future they will not adopt immature technologies again — which is what happened with several of the technologies dropped for 1.2 (e.g. buteo). Instead, they will wait for the technologies to be more mature before adopting them in MeeGo. This means that those in the community must establish and develop the technologies first, and then they may be included based on their merit.

Important Sessions

Another important talk was Carsten Munk’s Transparency, Inclusion, and Meritocracy in MeeGo: Theory and Practice. Munk presented several different patterns for how groups work within the MeeGo community. Each pattern was evaluated for transparency, inclusion, and effectiveness. He also gave suggestions for improvement for each type. If you missed it, be sure to go back and watch the whole thing.

Auke Kok’s talk on systemd was particularly enlightening to me as a distribution maintainer for Indamixx. Kok outlined the history of fast booting in Moblin and MeeGo, and how systemd addresses the problems with that system. He also gave handles to how the new system is configured and extended.

My own session, Writing Interactive Audio Applications with Low Latency was well received. Afterwards, I was surprised to find out how many people outside of Pro-Audio were having troubles with PulseAudio. So, I plan to look more closely at what’s happening and see how I might help things. I was also told that my talk was the most technical talk they attended… so I’m not sure if that’s a good or a bad thing. 🙂

Booths and Demos

There were several vendors with booths in the expo area. This included Intel, Qt, Digia, ICS, Nomovok, and basysKom. Most of these were offering outsourcing and training.

Of particular interest was our own Indamixx 2 demo in the Intel booth (See videos from Engadget, The Nokia Blog, and Netbook News. We showed off our version of MeeGo that’s tweaked for audio, and people were very receptive. Probably the best reception is for the combination of Ivory (virtual midi keyboard) and Yoshimi (a synthesizer). But I’m biased, of course.

Nomovok was demonstrating their Steelrat framework. When vendors want to ship their own custom tablet, they help the vendors to do this from start to finish. They also include their own “Steelrat Framework” that accelerates the development of custom UX’s. They also sell their expertise on accelerated graphics. Very impressive vendor.

Round-Up

MeeGo is currently doing well as a starting point for appliances (set-top boxes, kiosks, PIM). The design and openness makes it very easy to create custom applications and user experiences. While tablet/handset/app-store experiences aren’t here today… they are coming.

The split between Intel and “the community” is getting better, but slowly. The Intel AppUp crew made great efforts to mingle with the community. Other key Intel folks seemed… hard to find. Perhaps word of this got out, because on the 3rd day the sessions seemed littered with Intel employees wearing black Intel shirts. (A good thing.)

Last year, I expected that the future of MeeGo would start with the 1.2 release. From the conference it’s clear to me that MeeGo has a future, and it will start with the 1.3 release.

Thanks

I would like to thank the Linux Foundation for sponsoring my trip. Also thanks to Intel for featuring us in their booth. And a world of thanks to Dawn Foster for her tireless effort to build community.

Last time I described how to use LD_LIBRARY_PATH for your private libs in your package. While this will usually get you what you want, it has one extremely big drawback: poisoning 3rd party libs. It also has one minor drawback: applications start up slower.

Starting Slower

When your application starts, a program called ld.so looks to see what libraries you need. (E.g. libQtCore.so.4 and libxml2.so.0 and so forth.) Since this happens all the time, it keeps an index of these libraries and where they are located. Thus, it’s able to resolve the libraries very fast.

But when you set LD_LIBRARY_PATH, these paths have to be searched first, and this is slower because you actually have to access the file system and search directories.

But this is only a minor problem… the bigger problem is…

Poisoning 3rd Part Libraries

To be MeeGo compliant, there is a prescribed list of libraries that we can depend on. For everything else, we have to bring our own. Suppose, for instance, that MeeGo’s version of libAtion.so.1 does not have all the features that you need. So, you package your own version of libAtion.so.1 with the extra features and ship it as a private library in your package.

Meanwhile, Qt is already linked to libAtion, and you use it (indirectly) in your code. When your program is initialized, it’ll see that Qt needs libAtion.so.1 and use your private version instead of the system version that Qt shipped with. Do you see how this might cause problems?

Or, suppose you had no idea that libAtion.so.1 was in any MeeGo, anywhere. However, DeviceMan, Inc. shipped a MeeGo device where they added libAtion.so.1 as part of their default Qt theme plug-in. Now when your application loads the theme, it is poisoned with your private libAtion.

Solutions to LD_LIBRARY_PATH

So, LD_LIBRARY_PATH isn’t the best solution (and some will say it is to be avoided at all costs). What other options do you have?

  • Statically link your programs to your private libraries — While this isn’t as convenient to your opt/non-opt build workflow, it is effective.
  • Use rpath when you link — the GNU linker has an -rpath option that is like setting LD_LIBRARY_PATH at compile time. The advantage is that it only applies to your binary… and not other 3rd party libs.
  • Do lots of homework — Make sure that your version is compatible with whatever version may be on the system. If possible, prefer the system’s version of the library. However, this option is destined to fail, eventually.

Further Reading

I found the following articles informative:

So last time I convinced you that packaging for /opt is a Good Thing for add-on software (e.g. app store stuff).[1]  But now that you are convinced… this is a total re-think of how packaging works. Here are some tips on how to get it done without totally messing up your non-opt installs.

Suppose our company is indie.biz and our program is called Floobydust.  On unix-like operating systems we’re used to installing our software like this:

  • Executable programs go in /usr/bin/
  • Libraries go in /usr/lib/
  • Desktop files go in /usr/share/applications/
  • Our extra stuff goes in /usr/share/floobydust/

But now, everything must go in /opt.  This means that:

  • Our program won’t be in $PATH
  • Our libraries won’t be found by ld.so [4]
  • Our desktop files won’t be discovered
  • Our hard-coded "/usr/share/floobydust" strings are now bugs

So… how do you handle this?  In fact… how can I handle both cases without having to totally re-write my build system?

Step 1: Use PREFIX

Most build systems allow the user to define a PREFIX at build-time.  This PREFIX will determine the location of the bin, lib, and share folders that we’ll use for install.  All the paths above set PREFIX=/usr.  It typically defaults to PREFIX=/usr/local.  So, what happens when you set PREFIX=/opt/indie.biz/floobydust ?[3]  Then…

  • Executable programs go in /opt/indie.biz/floobydust/
  • Libraries go in /opt/indie.biz/floobydust/lib/
  • Desktop files go in /opt/indie.biz/floobydust/share/applications/[2]
  • Our extra stuff goes in /opt/indie.biz/floobydust/share/floobydust/

Now, it’s essentially the same thing as if we had set any other prefix.  Note also that you may choose to use PREFIX=/opt/indie.biz — as long as you make sure that all your programs cooperate!

Step 2: Use a redirect script to set up your private libraries

When installed in /usr/bin, ld.so (the dynamic linker) will search in /usr/local/lib, /usr/lib, and /lib for the shared object (.so) files that you need. However, you have installed them to /opt/indie.biz/floobydust/lib instead. To get ld.so to find your libraries, you need to modify LD_LIBRARY_PATH to include your package’s lib folder. You can do this by renaming your program to something like floobydust-1.2.3, and creating a redirect script called floobydust like this:

#!/bin/bash
LD_LIBRARY_PATH="/opt/indie.biz/floobydust/lib:$LD_LIBRARY_PATH"
export LD_LIBRARY_PATH
exec /opt/indie.biz/floobydust/bin/floobydust-1.2.3 "$@"

So, we (a) set LD_LIBRARY_PATH, (b) call our program, and (c) pass it all the command-line arguments (that’s the “$@” thing). However, this script only covers the happy case… so here’s the script that you would really use:

#!/bin/bash
PREFIX=/opt/indie.biz/floobydust
BINDIR="${PREFIX}/bin"
LIBDIR="${PREFIX}/lib"
EXEFILE=floobydust-1.2.3

LD_LIBRARY_PATH="${LIBDIR}${LD_LIBRARY_PATH:+:$LD_LIBRARY_PATH}"
export LD_LIBRARY_PATH
PATH="${BINDIR}${PATH:+:$PATH}"
export PATH

exec "${BINDIR}/${EXEFILE}" "$@"

This is something that you could reuse in several different applications. Also note the shell expansion trick for LD_LIBRARY_PATH and PATH that makes sure we don’t end up with a trailing colon ("/opt/indie.biz/floobydust/lib:").

This method will help no matter where you installed your libs. You can configure the path at compile time.

Step 3: Namespace your .desktop file

Add your organization name to your desktop file indie.biz.floobydust.desktop. Since you are using your domain name, it’s extremely unlikely that you will conflict with anyone else’s .desktop file. Then install the file using the desktop-file-install utility (which will usually install to /usr/share/applications). You can optionally have that command do the file-name mangling for you. (Read the manual page for it.)

Also, be sure to put the full path to your redirect script from Step 2 (/opt/indie.biz/floobydust/bin/floobydust), as well as any icons.  Most build systems will let you make your desktop file a configure script (typ. floobydust.desktop.in) that gets modified for your compile-time configuration options.

Step 4: Make sure you don’t hard-code paths in your application

This is actually step 1, but it’s boring so I put it last.  If some part of your program looks to /usr/share/floobydust without thinking first… you need to make your application configurable.[5]

What about $PATH??

Are you really shipping command-line apps through an app store? There isn’t really a solution to this (and still be MeeGo Compliant.) The .desktop file is the replacement for a $PATH, and the redirect script from step 2 can reset it if we need it internally. Another option is to create symlinks in /usr/bin. While this makes the app non-compliant, it’s a good compromise for a multi-distribution package.

Wrapping up

Whatever we do to install in /opt can translate back to /usr and /usr/local. Using this strategy not only helps in app-store situations, but also creates a safe practice for application-specific private libraries (.so). With a little extra run-time detection, these can also help in supporting relocatable RPM packages. While there are no good solutions for CLI apps, it seems unlikely that many CLI apps will be targeting app stores.

——
[1] Yeah, right! 🙂
[2] Not really… but we’ll get to that.
[3] Note that MeeGo compliance says you can install in /opt/<packagename>. Using your organization name is optional, but recommended.
[4] ld.so is the dynamic linker.  It’s the program that finds the .so files for your app to use.
[5] You are supposed to do this anyway… since you don’t know what PREFIX will be a priori.

The current MeeGo Compliance Spec will often give developers a start when they read this:

An application shall be installed to /opt/packagename/ and, if necessary to the /etc/opt/packagename/ and /var/opt/packagename/ directories.[1]

Most users reply, “Huh?  Why don’t we install in /usr like a normal Linux distro??”  The rationale is given as follows:

The rationale for these rules is to avoid filename clashes between application packages and with system files, by defining portions of the filesystem certain to be unique to that application.[2]

This leaves most devs a little… underwhelmed.  The first response is to appeal to the FHS/LSB… but lets start with an example.

Suppose I write a MeeGo game called “Zombie Professer Shootout.”  Because I’ve got several modules for this game, I have an internal library called libzps.so.0.  Because I don’t know any better, I install it in /usr/lib/libzps.so.0.  Put the package in the App Store.  Done.

Meanwhile, suppose that YOU, Zeek, write a hand MeeGo time management program called “Zeek’s Personal Scheduler.”  For whatever reason, you put a lot of the core functionality in a library called libzps.so.0 so that other people could reuse it in their code.  So, you install it to /usr/lib/libzps.so.0.  Put the package in the App Store.  Done.

First, they install my Zombie game.  Then, months later, they install your Personal Scheduler.  However, the app WILL NOT install because it would overwrite /usr/lib/libzps.so.0.  There’s no way around it.  You have a broken package.

And the user says, “This sucks!”

The whole point of being a MeeGo compliant app is that it’s a single, stand-alone, 3rd party app that can be installed to MeeGo… and it Just Works.  But if we all install in /usr… then we all have to be on the same page with respect to file names, package names, etc.  In a normal linux distro (where all packages are served from one repository), these get discovered and discussed among the developers.  But with MeeGo apps coming from a myriad of sources there is no way that can happen.

So, MeeGo chose to use namespaces by requiring installation to /opt.  The opt folder is defined in the FHS as follows:

/opt is reserved for the installation of add-on application software packages.

A package to be installed in /opt must locate its static files in a separate /opt/<package> or /opt/<provider> directory tree, where <package> is a name that describes the software package and <provider> is the provider’s LANANA registered name.[3]

Note that this is nearly identical to the MeeGo requirement… and serves to solve this exact problem.  It’s a system that allows uncoordinated developers the freedom to organize their package however they need without stepping on anyone else’s toes.  Everything they do is under the namespace of the company’s LANANA registered name (e.g. google.com or something).  Without this, users will be inundated with broken packages and always come to the same conclusion: “This sucks.”

“But how do I find my project’s libs?  What about $PATH?  This is no fun for the developer!”  We’ll talk about that next time.

[1] MeeGo 1.1 Compliance Specification, Section 3.3.4

[2] Ibid.

[3] http://www.pathname.com/fhs/pub/fhs-2.3.html#OPTADDONAPPLICATIONSOFTWAREPACKAGES