Planet Tanglu
March 30, 2015

And once again, it’s time for another Limba blogpost :-)limba-small

Limba is a solution to install 3rd-party software on Linux, without interfering with the distribution’s native package manager. It can be useful to try out different software versions, use newer software on a stable OS release or simply to obtain software which does not yet exist for your distribution.

Limba works distribution-independent, so software authors only need to publish their software once for all Linux distributions.

I recently released version 0.4, with which all most important features you would expect from a software manager are complete. This includes installing & removing packages, GPG-signing of packages, package repositories, package updates etc. Using Limba is still a bit rough, but most things work pretty well already.

So, it’s time for another progress report. Since a FAQ-like list is easier to digest. compared to a long blogpost, I go with this again. So, let’s address one important general question first:

How does Limba relate to the GNOME Sandboxing approach?

(If you don’t know about GNOMEs sandboxes, take a look at the GNOME Wiki – Alexander Larsson also blogged about it recently)

First of all: There is no rivalry here and no NIH syndrome involved. Limba and GNOMEs Sandboxes (XdgApp) are different concepts, which both have their place.

The main difference between both projects is the handling of runtimes. A runtime is the shared libraries and other shared ressources applications use. This includes libraries like GTK+/Qt5/SDL/libpulse etc. XdgApp applications have one big runtime they can use, built with OSTree. This runtime is static and will not change, it will only receive critical security updates. A runtime in XdgApp is provided by a vendor like GNOME as a compilation of multiple single libraries.

Limba, on the other hand, generates runtimes on the target system on-the-fly out of several subcomponents with dependency-relations between them. Each component can be updated independently, as long as the dependencies are satisfied. The individual components are intended to be provided by the respective upstream projects.

Both projects have their individual up and downsides: While the static runtime of XdgApp projects makes testing simple, it is also harder to extend and more difficult to update. If something you need is not provided by the mega-runtime, you will have to provide it by yourself (e.g. we will have some applications ship smaller shared libraries with their binaries, as they are not part of the big runtime).

Limba does not have this issue, but instead, with its dynamic runtimes, relies on upstreams behaving nice and not breaking ABIs in security updates, so existing applications continue to be working even with newer software components.

Obviously, I like the Limba approach more, since it is incredibly flexible, and even allows to mimic the behaviour of GNOMEs XdgApp by using absolute dependencies on components.

Do you have an example of a Limba-distributed application?

Yes! I recently created a set of package for Neverball – Alexander Larsson also created a XdgApp bundle for it, and due to the low amount of stuff Neverball depends on, it was a perfect test subject.

One of the main things I want to achieve with Limba is to integrate it well with continuous integration systems, so you can automatically get a Limba package built for your application and have it tested with the current set of dependencies. Also, building packages should be very easy, and as failsafe as possible.

You can find the current Neverball test in the Limba-Neverball repository on Github. All you need (after installing Limba and the build dependencies of all components) is to run the script.

Later, I also want to provide helper tools to automatically build the software in a chroot environment, and to allow building against the exact version depended on in the Limba package.

Creating a Limba package is trivial, it boils down to creating a simple “control” file describing the dependencies of the package, and to write an AppStream metadata file. If you feel adventurous, you can also add automatic build instructions as a YAML file (which uses a subset of the Travis build config schema)

This is the Neverball Limba package, built on Tanglu 3, run on Fedora 21:

Limba-installed Neverball

Which kernel do I need to run Limba?

The Limba build tools run on any Linux version, but to run applications installed with Limba, you need at least Linux 3.18 (for Limba 0.4.2). I plan to bump the minimum version requirement to Linux 4.0+ very soon, since this release contains some improvements in OverlayFS and a few other kernel features I am thinking about making use of.

Linux 3.18 is included in most Linux distributions released in 2015 (and of course any rolling release distribution and Fedora have it).

Building all these little Limba packages and keeping them up-to-date is annoying…

Yes indeed. I expect that we will see some “bigger” Limba packages bundling a few dependencies, but in general this is a pretty annoying property of Limba currently, since there are so few packages available you can reuse. But I plan to address this. Behind the scenes, I am working on a webservice, which will allow developers to upload Limba packages.

This central ressource can then be used by other developers to obtain dependencies. We can also perform some QA on the received packages, map the available software with CVE databases to see if a component is vulnerable and publish that information, etc.

All of this is currently planned, and I can’t say a lot more yet. Stay tuned! (As always: If you want to help, please contact me)

Are the Limba interfaces stable? Can I use it already?

The Limba package format should be stable by now – since Limba is still Alpha software, I will however, make breaking changes in case there is a huge flaw which makes it reasonable to break the IPK package format. I don’t think that this will happen though, as the Limba packages are designed to be easily backward- and forward compatible.

For the Limba repository format, I might make some more changes though (less invasive, but you might need to rebuilt the repository).

tl;dr: Yes! Plase use Limba and report bugs, but keep in mind that Limba is still in an early stage of development, and we need bug reports!

Will there be integration into GNOME-Software and Muon?

From the GNOME-Software side, there were positive signals about that, but some technical obstancles need to be resolved first. I did not yet get in contact with the Muon crew – they are just implementing AppStream, which is a prerequisite for having any support for Limba[1].

Since PackageKit dropped the support for plugins, every software manager needs to implement support for Limba.

So, thanks for reading this (again too long) blogpost :) There are some more exciting things coming soon, especially regarding AppStream on Debian/Ubuntu!


[1]: And I should actually help with the AppStream support, but currently I can not allocate enough time to take that additional project as well – this might change in a few weeks. Also, Muon does pretty well already!

March 23, 2015

Cutelyst the Qt/C++ web framework just got a new release, 0.7.0 brings many improvements, bugfixes and features.

Most notably of the changes is a shinny new Chained dispatcher that finally got implemented, with it Cutelyst Tutorial got finished, and a new command line tool to bootstrap new projects.

* Request::bodyData() can return a QJsonDocument is content-type is application/json
* Chained dispatcher
* Fixes on QStringListeral and QLatin1String usage
* More debug information
* Some API changes to match Catalyst’s
* Some rework on the API handlying Plugins (not the best thing yet)
* Moved to Gitlab (due to the gitorious being acquired by it)
* “cutelyst” command line tool

For the next release I’ll try to make the QML integration a reality, and try to improve the Plugins API, which is right now the part that needs more love would love to hear some advice.

Download it here.

Have fun!

February 12, 2015

Now that Cutelyst is allowing me to write web applications with the tools I like, I can use it to build the kind of web applications I need but am not fine with using the existing ones…

WordPress is a great software, but it’s written in PHP, which is not a language I like, and there are many other software like this written in all sorts of languages but C++/Qt/Cutelyst, so why not?

This first release is a very shine one, it allows for templating, static pages and posts (without comments) and even a RSS feed.

To make things more interesting CMlyst has a separate library, this library abstracts the Content Management part, allowing for different backends to be written, the current one is based on QSettings files, but it’s very easy to write one for SQL, git who knows :P

I have not measured it speed against WordPress or any other CMS, it probably won’t be fair. But if you are curious I have run it against weighttp and it gives me 4k req/s on a page show, 2,5k req/s on last posts listing and 6,5k on RSS feed delivering, this on a Core2Duo 2.4Ghz, using ~10MB of RAM.

Give it a try!

Download here.

Cutelyst, the Qt/C++ web framework just got another step into API stabilization.

Since 0.3.0 I’ve been trying to take the most request per second out of it, and because of that I decided to replace most QStrings with QByteArrays, the allocation call is indeed simpler in QByteArray but since most of Qt use QString for strings it started to create a problem rather than solving one. Grantlee didn’t play nice with QByteArray breaking ifequal and in the end some implicit conversions from UTF-8 were triggered.

So on 0.6.0 I’ve replaced all QByteArray usage in favor of QString, and from the benchmarks of hello world the difference is really negligible.

Another big change I actually took today was to change the Headers class that is used on Request and Response, the keys stored there were always in camel-case style and split by dashes, and if you want to get say the User-Agent in a Grantlee template it won’t work as dashes will break the variable name, so now all keys are lower casa and words are separated by underscores. Allowing for {{ ctx.request.headers.user_agent }} in Grantlee templates.

Some changes were also made to make it behave more close to what Catalyst does, and fixed a bunch of bugs from last release.

On my TODO list still need to:
Write a chained dispatcher to allow a more flexible way of dispatching.
Write tutorials, yeah this is boring…
Write QML integration to allow for writing the application logic in QML too!

This last one would allow for something like this:

Application {
    Controller {
        Action {
            attributes: ":Path(hello)"
            onRequest: {
                ctx.response.body = "Hello World!"

Download here

Have fun!

January 27, 2015

Yesterday I released version 0.8 of AppStream, the cross-distribution standard for software metadata, that is currently used by GNOME-Software, Muon and Apper in to display rich metadata about applications and other software components.

 What’s new?

The new release contains some tweaks on AppStreams documentation, and extends the specification with a few more tags and refinements. For example we now recommend sizes for screenshots. The recommended sizes are the ones GNOME-Software already uses today, and it is a good idea to ship those to make software-centers look great, as others SCs are planning to use them as well. Normal sizes as well as sizes for HiDPI displays are defined. This change affects only the distribution-generated data, the upstream metadata is unaffected by this (the distro-specific metadata generator will resize the screenshots anyway).

Another addition to the spec is the introduction of an optional <source_pkgname/> tag, which holds the source package name the packages defined in <pkgname/> tags are built from. This is mainly for internal use by the distributor, e.g. it can decide to use this information to link to internal resources (like bugtrackers, package-watch etc.). It may also be used by software-center applications as additional information to group software components.

Furthermore, we introduced a <bundle/> tag for future use with 3rd-party application installation solutions. The tag notifies a software-installer about the presence of a 3rd-party application bundle, and provides the necessary information on how to install it. In order to do that, the software-center needs to support the respective installation solution. Currently, the Limba project and Xdg-App bundles are supported. For software managers, it is a good idea to implement support for 3rd-party app installers, as soon as the solutions are ready. Currently, the projects are worked on heavily. The new tag is currently already used by Limba, which is the reason why it depends on the latest AppStream release.

How do I get it?

All AppStream libraries, libappstream, libappstream-qt and libappstream-glib, are supporting the 0.8 specification in their latest version – so in case you are using one of these, you don’t need to do anything. For Debian, the DEP-11 spec is being updated at time, and the changes will land in the DEP-11 tools soon.

Improve your metadata!

This call goes especilly to many KDE projects! Getting good data is partly a task for the distributor, since packaging issues can result in incorrect or broken data, screenshots need to be properly resized etc. However, flawed upstream data can also prevent software from being shown, since software with broken data or missing data will not be incorporated in the distro XML AppStream data file.

Richard Hughes of Fedora has created a nice overview of software failing to be included. You can see the failed-list here – the data can be filtered by desktop environment etc. For KDE projects, a Comment= field is often missing in their .desktop files (or a <summary/> tag needs to be added to their AppStream upstream XML file). Keep in mind that you are not only helping Fedora by fixing these issues, but also all other distributions cosuming the metadata you ship upstream.

For Debian, we will have a similar overview soon, since it is also a very helpful tool to find packaging issues.

If you want to get more information on how to improve your upstream metadata, and how new metadata should look like, take a look at the quickstart guide in the AppStream documentation.

December 02, 2014

Disclaimer: Limba is stilllimba-small in a very early stage of development. Bugs happen, and I give to guarantees on API stability yet.

Limba is a very simple cross-distro package installer, utilizing OverlayFS found in recent Linux kernels (>= 3.18).

As example I created a small Limba package for one of the Qt5 demo applications, and I would like to share the process of creating Limba packages – it’s quite simple, and I could use some feedback on how well the resulting packages work on multiple distributions.

I assume that you have compiled Limba and installed it – how that is done is described in its README file. So, let’s start.

1. Prepare your application

The cool thing about Limba is that you don’t really have to do many changes on your application. There are a few things to pay attention to, though:

  • Ensure the binaries and data are installed into the right places in the directory hierarchy. Binaries must go to $prefix/bin, for example.
  • Ensure that configuration can be found under /etc as well as under $prefix/etc

This needs to be done so your application will find its data at runtime. Additionally, you need to write an AppStream metadata file, and find out which stuff your application depends on.

2. Create package metadata & install software

1.1 Basics

Now you can create the metadata necessary to build a Limba package. Just run

cd /path/to/my/project
lipkgen make-template

This will create a “lipkg” directory, containing a “control” file and a “metainfo.xml” file, which can be a symlink to the AppStream metadata, or be new metadata.

Now, configure your application with /opt/bundle as install prefix (-DCMAKE_INSTALL_PREFIX=/opt/bundle, –prefix=/opt/bundle, etc.) and install it to the lipkg/inst_target directory.

1.2 Handling dependencies

If your software has dependencies on other packages, just get the Limba packages for these dependencies, or build new ones. Then place the resulting IPK packages in the lipkg/repo directory. Ideally, you should be able to fetch Limba packages which contain the software components directly from their upstream developers.

Then, open the lipkg/control file and adjust the “Requires” line. The names of the components you depend on match their AppStream-IDs (<id/> tag in the AppStream XML document). Any version-relation (>=, >>, <<, <=, <>) is supported, and specified in brackets after the component-id.

The resulting control-file might look like this:

Format-Version: 1.0

Requires: Qt5Core (>= 5.3), Qt5DBus (>= 5.3), libpng12

If the specified dependencies are in the repo/ subdirectory, these packages will get installed automatically, if your application package is installed. Otherwise, Limba depends on the user to install these packages manually – there is no interaction with the distribution’s package-manager (yet?).

3. Building the package

In order to build your package, make sure the content in inst_target/ is up to date, then run

lipkgen build lipkg/

This will build your package and output it in the lipkg/ directory.

4. Testing the package

You can now test your package, Just run

sudo lipa install package.ipk

Your software should install successfully. If you provided a .desktop file in $prefix/share/applications, you should find your application in your desktop’s application-menu. Otherwise, you can run a binary from the command-line, just append the version of your package to the binary name (bash-comletion helps). Alternatively, you can use the runapp command, which lets you run any binary in your bundle/package, which is quite helpful for debugging (since the environment a Limba-installed application is run is different from the one of other applications).


runapp ${component_id}-${version}:/bin/binary-name

And that’s it! :-)

I used these steps to create a Limba package for the OpenGL Qt5 demo on Tanglu 2 (Bartholomea), and tested it on Kubuntu 15.04 (Vivid) with KDE, as well as on an up-to-date Fedora 21, with GNOME and without any Qt or KDE stuff installed:


I encountered a few obstacles when building the packages, e.g. Qt5 initially didn’t find the right QPA plugin – that has been fixed by adjusting a config file in the Qt5Gui package. Also, on Fedora, a matching libpng was missing, so I included that as well.

You can find the packages at Github, currently (but I am planning to move them to a different place soon). The biggest issue with Limba is at time, that it needs Linux 3.18, or an older kernel with OverlayFS support compiled in. Apart from that and a few bugs, the experience is quite smooth. As soon as I am sure there are now hidden fundamental issues, I can think of implementing more features, like signing packages and automatically updating them.

Have fun playing around with Limba!

November 24, 2014

A bit more than one year after the initial commit, Cutelyst makes it’s 5th release.

It’s now powering 3 commercial applications, the last one recently got into production and is the most complex of them, making heavy use of Grantlee and Cutelyst capabilities.

Speaking of Grantlee if you use it on Qt5 you will get hit by QTBUG-41469 which sadly doesn’t seems to get fixed in time for 5.4, but uWSGI can constrain your application resources so your server doesn’t go out of memory (worth the leak due to it’s usefulness).

Here is an overview since 0.4.0 release:

  • Remove hardcode build setting to “Debug”, so that one can build with “Release” increasing performance up to 20% –
  • Request::uploads() API was changed to be useful in real world, filling a QMap with the form field name as a key and in the proper order sent by the client
  • Introduced a new C_ATTR macro which allows to have the same Perl attributs syntax like C_ATTR(method_name, :Path(/foo/bar) :Args)
  • Added an Action class RoleACL which allows for doing authorization on control lists, making it easy to deny access to some resources if a user doesn’t match the needed role
  • Added a RenderView class to make it easier to delegate the rendering to a view such as Grantlee
  • Request class is now QObject class so that we can use it on Grantlee as ctx.request.something
  • Make use of uWSGI ini (–init) configuration file to also configure the Cutelyst application
  • Better docs
  • As always some bugs were fixed

I’m very happy with the results, all those site performance tools like webpagetest give great scores for the apps, and I have started to work on translating Catalyst tutorial to Cutelyst, but I realize that I need Chained dispatcher working before that…

If you want to try it, I’ve made a hello-world app available today at

Download here!

November 10, 2014

As some of you already know, since the larger restructuring in PackageKit for the 1.0 release, I am rethinking Listaller, the 3rd-party application installer for Linux systems, as well.

During the past weeks, I was playing around with a lot of different ideas and code, to make installations of 3rd-party software easily possible on Linux, but also working together with the distribution package manager. I now have come up with an experimental project, which might achieve this.


Many of you know Lennart’s famous blogpost on how we put together Linux distributions. And he makes a lot of good and valid points there (in fact, I agree with his reasoning there). The proposed solution, however, is not something which I am very excited about, at least not for the use-case of installing a simple application[1]. Leaving things like the exclusive dependency on technology like Btrfs aside, the solution outlined by Lennart basically bypasses the distribution itself, instead of working together with it. This results in a duplication of installed libraries, making it harder to overview which versions of which software component are actually running on the system. There is also a risk for security holes due to libraries not being updated. The security issues are worked around by a superior sandbox, which still needs to be implemented (but will definitively come soon, maybe next year).

I wanted to explore a different approach of managing 3rd-party applications on Linux systems, which allows sharing as much code as possible between applications.

Limba – Glick2 and Listaller concepts mergedlimba-small

In order to allow easy creation of software packages, as well as the ability to share software between different 3rd-party applications, I took heavy inspiration from Alexander Larssons Glick2 project, combining it with ideas from the application-directory based Listaller.

The result is Limba (named after Limba tree, not the voodoo spirit – I needed some name starting with “li” to keep the prefix used in Listaller, and for a tool like this the name didn’t really matter ;-) ).

Limba uses OverlayFS to combine an application with its dependencies before running it, as well as mount namespaces and shared subtrees. Except for OverlayFS, which just landed in the kernel recently, all other kernel features needed by Limba are available for years now (and many distributions ship with OverlayFS on older kernels as well).

How does it work?

In order to to achieve separation of software, each software component is located in a separate container (= package). A software component can be an application, like Kate or GEdit, but also be a single shared library (openssl) or even a full runtime (KDE Frameworks 5 parts, GNOME 3).

Each of these software components can be identified via AppStream metadata, which is just a few bits of XML. A Limba package can declare a dependency on any other software component. In case that software is available in the distribution’s repositories, the version found there can be used. Otherwise, another Limba package providing the software is required.

Limba packages can be provided from software repositories (e.g. provided by the distributor), or be nested in other packages. For example, imagine the software “Kate” requires a version of the Qt5 libraries, >= 5.2. The downloadable package for “Kate” can be bundled with that dependency, by including the “Qt5 5.2″ Limba package in the “Kate” package. In case another software is installed later, which also requires the same version of Qt, the already installed version will be used.

Since the software components are located in separate directories under /opt/software, an application will not automatically find its dependencies, or be able to locate its own files. Therefore, each application has to be run by a special tool, which merges the directory trees of the main application and it’s dependencies together using OverlayFS. This has the nice sideeffect that the main application could override files from its dependencies, if necessary. The tool also sets up a new mount namespace, so if the application is compiled with a certain prefix, it does not need to be relocatable to find its data files.

At installation time, to achieve better system integration, certain files (like e.g. the .desktop file) are split out of the installed directory tree, so the newly installed application achieves almost full system integration.


Can I use Limba now?

Limba is an experiment. I like it very much, but it might happen that I find some issues with it and kill it off again. So, if you feel adventurous, you can compile the source code and use the example “Foobar” application to play around with Limba. Before it can be used in production (if at all), some more time is needed.

I will publish documentation on how to test the project soon.

Doesn’t OverlayFS have a maximum stacking depth?

Oh yes it has! The “How does it work” explanation doesn’t tell the whole truth in that regard (mainly to keep the section small). In fact, Limba will generate a “runtime” for the newly installed software, which is a directory with links to the actual individual software components the runtime consists of. The runtime is identified by an UUID. This runtime is then mounted together with the respective applications using OverlayFS. This works pretty great, and also results in no dependency-resolution to be done immediately before an application is started.

Than dependency stuff gives me a headache…

Admittedly, allowing dependencies adds a whole lot of complexity. Other approaches, like the one outlined by Lennart work around that (and there are good reasons for doing that as well).

In my opinion, the dependency-sharing and de-duplication of software components, as well as the ability to use the components which are packaged by your Linux distribution is worth the extra effort.

Can you give an overview of future plans for Limba?

Sure, so here is the stuff which currently works:

  • Creating simple packages
  • Installing packages
  • Very basic dependency resolution (no relations (like >, <, =) are respected yet)
  • Running applications
  • Initial bits of system integration (.desktop files are registered)

These features are planned for the new future:

  • Support for removing software
  • Automatic software updates of 3rd-party software
  • Atomic updates
  • Better system integration
  • Integration with the new sandboxing features
  • GPG signing of packages
  • More documentation / bugfixes

Remember that Limba is an experiment, still ;-)

XKCD 927

Technically, I am replacing one solution with another one here, so the situation does not change at all ;-). But indeed, some duplicate work is done due to more people working in this area now on similar questions.

But I think this is a good thing, because the solutions worked on are fundamentally different approaches, and by exploring multiple ways of doing things, we will come up with something great in the end. (XKCD reference)

Doesn’t the use of OverlayFS have an impact on the performance of software running with Limba?

I ran some synthetic benchmarks and didn’t notice any problems – even the startup speed of Limba applications is only a few milliseconds slower than the startup of the “raw” native application. However, I will still have to run further tests to give a definitive answer on this.

How do you solve ABI compatibility issues?

This approach requires software to keep their ABI stable. But since software can have strict dependencies on a specific version of a software (although I’d discourage that), even people who are worried about this issue can be happy. We are getting much better at tracking unwanted ABI breaks, and larger projects offer stable API/ABI during a major release cycle. For smaller dependencies, there are, as explained above, stricter dependencies.

In summary, I don’t think ABI incompatibilities will be a problem with this approach – at least not more than they have been in general. (The libuild facilities from Listaller to minimize dependencies will still be present im Limba, of course)

You are wrong because of $X!

Please leave a comment in this case! I’d love to discuss new ideas and find the limits of the Limba concept – that’s why I am writing C code afterall, since what looks great on paper might not work in reality or have issues one hasn’t thought about before. So any input is welcomed!


Last but not least I want to thank Alexander Larsson for writing Glick2, which Limba is heavily inspired from, and for his patient replies to my emails.

If Limba turns out to be a good idea, you can expect a few more blog posts about it soon.

* Answered questions nobody asked yet

[1]: Don’t get me wrong, I would like to have these ideas implemented – they offer great value. But I think for “simple” software deployment, the solution is an overkill.

November 06, 2014

… or “Why do I not see any update notifications on my brand-new Debian Jessie installation??”

This is a short explanation of the status quo, and also explains the “no update notifications” issue in a slightly more detailed way, since I am already getting bug reports for that.

As you might know, GNOME provides GNOME-Software for installation of applications via PackageKit. In order to work properly, GNOME-Software needs AppStream metadata, which is not yet available in Debian. There was a GSoC student working on the necessary code for that, but the code is not yet ready and doesn’t produce good results yet. Therefore, I postponed AppStream integration to Jessie+1, with an option to include some metadata for GNOME and KDE to use via a normal .deb package.

Then, GNOME was updated to 3.14. GNOME 3.14 moved lots of stuff into GNOME-Software, including the support for update-notifications (which have been in g-s-d before). GNOME-Software is also the only thing which can edit the application groups in GNOME-Shell, at least currently.

So obviously, there was no a much stronger motivation to support GNOME-Software in Jessie. The appstream-glib library, which GNOME-Software uses exclusively to read AppStream metadata, didn’t support the DEP-11 metadata format which Debian uses in place of the original AppSTream XML for a while, but does so in it’s current development branch. So that component had to be packaged first. Later, GNOME-Software was uploaded to the archive as well, but still lacked the required metadata. That data was provided by me as a .deb package later, locally generated using the current code by my SoC student (the data isn’t great, but better than nothing). So far with the good news.

But there are multiple issues at time. First of all, the appstream-data package didn’t pass NEW so far, due to it’s complex copyright situation (nothing we can’t resolve, since app-install-data, which appstream-data would replace, is in Debian as well). Also, GNOME-Software is exclusively using offline-updates (more information also on [1] and [2]) at time. This isn’t always working at the moment, since I haven’t had the time to test it properly – and I didn’t expect it to be used in Debian Jessie as well[3].

Furthermore, the offline-updates feature requires systemd (which isn’t an issue in itself, I am quite fine with that, but people not using it will get unexpected results, unless someone does the work to implement offline-updates with sysvinit).

Since we are in freeze at time, and obviously this stuff is not ready yet, GNOME is currently without update notifications and without a way to change the shell application groups.

So, how can we fix this? One way would of course be to patch notification support back into g-s-d, if the new layout there allows doing that. But that would not give us the other features GNOME-Software provides, like application-group-editing.

Implementing that differently and patching it to make it work would be more or at least the same amount of work like making GNOME-Software run properly. I therefore prefer getting GNOME-Software to run, at least with basic functionality. That would likely mean hiding things like the offline-update functionality, and using online-updates with GNOME-PackageKit instead.

Obviously, this approach has it’s own issues, like doing most of the work post-freeze, which kind of defeats the purpose of the freeze and would need some close coordination with the release-team.

So, this is the status quo at time. It is kind of unfortunate that GNOME moved crucial functionality into a new component which requires additional integration work by the distributors so quickly, but that’s something which isn’t worth to talk about. We need a way forward to bring update-notifications back, and there is currently work going on to do that. For all Debian users: Please be patient while we resolve the situation, and sorry for the inconvenience. For all developers: If you would like to help, please contact me or Laurent Bigonville, there are some tasks which could use some help.

As a small remark: If you are using KDE, you are lucky – Apper provides the notification support like it always did, and thanks to improvements in aptcc and PackageKit, it even is a bit faster now. For the Xfce and <other_desktop> people, you need to check if your desktop provides integration with PackageKit for update-checking. At least Xfce doesn’t, but after GNOME-PackageKit removed support for it (which was moved to gnome-settings-daemon and now to GNOME-Software) nobody stepped up to implement it yet (so if you want to do it – it’s not super-complicated, but knowledge of C and GTK+ is needed).


[3]: It looks like dpkg tries to ask a debconf question for some reason, or an external tool like apt-listchanges is interfering with the process, which must run completely unsupervised. There is some debugging needed to resolve these Debian-specific issues.

October 10, 2014

As you might know, due to invasive changes in PackageKit, I am currently rewriting the 3rd-party application installer Listaller. Since I am not the only one looking at the 3rd-party app-installation issue (there is a larger effort going on at GNOME, based on Lennarts ideas), it makes sense to redesign some concepts of Listaller.

Currently, dependencies and applications are installed into directories in /opt, and Listaller contains some logic to make applications find dependencies, and to talk to the package manager to install missing things. This has some drawbacks, like the need to install an application before using it, the need for applications to be relocatable, and application-installations being non-atomic.


There is/was another 3rd-party app installer approach on the GNOME side, by Alexander Larsson, called Glick2. Glick uses application bundles (do you remember Klik from back in the days?) mounted via FUSE. This allows some neat features, like atomic installations and software upgrades, no need for relocatable apps and no need to install the application.

However, it also has disadvantages. Quoting the introduction document for Glick2:

“Bundling isn’t perfect, there are some well known disadvantages. Increased disk footprint is one, although current storage space size makes this not such a big issues. Another problem is with security (or bugfix) updates in bundled libraries. With bundled libraries its much harder to upgrade a single library, as you need to find and upgrade each app that uses it. Better tooling and upgrader support can lessen the impact of this, but not completely eliminate it.”

This is what Listaller does better, since it was designed to do a large effort to avoid duplication of code.

Also, currently Glick doesn’t have support for updates and software-repositories, which Listaller had.

Combining Listaller and Glick ideas

So, why not combine the ideas of Listaller and Glick? In order to have Glick share resources, the system needs to know which shared resources are available. This is not possible if there is one huge Glick bundle containing all of the application’s dependencies. So I modularized Glick bundles to contain just one software component, which is e.g. GTK+ or Qt, GStreamer or could even be a larger framework (e.g. “GNOME 3.14 Platform”). These components are identified using AppStream XML metadata, which allows them to be installed from the distributor’s software repositories as well, if that is wanted.

If you now want to deploy your application, you first create a Glick bundle for it. Then, in a second step, you bundle your application bundle with it’s dependencies in one larger tarball, which can also be GPG signed and can contain additional metadata.

The resulting “metabundle” will look like this:











This doesn’t look like we share resources yet, right? The dependencies are still bundled with the application requiring them. The trick lies in the “installation” step: While the application above can be executed right away without installing it, there will also be an option to install it. For the user, this will mean that the application shows up in GNOME-Shell’s overview or KDEs Plasma launcher, gets properly registered with mimetypes and is – if installed for all users – available system-wide.

Technically, this will mean that the application’s main bundle is extracted and moved to a special location on the file system, so are the dependency-bundles. If bundles already exist, they will not be installed again, and the new application will simply use the existing software. Since the bundles contain information about their dependencies, the system is able to determine which software is needed and which can simply be deleted from the installation directories.

If the application is started now, the bundles are combined and mounted, so the application can see the libraries it depends on.

Additionally, this concept allows secure updates of applications and shared resources. The bundle metadata contains an URL which points to a bundle repository. If new versions are released, the system’s auto-updater can automatically pick these up and install them – this means e.g. the Qt bundle will receive security updates, even if the developer who shipped it with his/her app didn’t think of updating it.


So far, no productive code exists for this – I just have a proof-of-concept here. But I pretty much like the idea, and I am thinking about going further in that direction, since it allows deploying applications on the Linux desktop as well as deploying software on servers in a way which plays nice with the native package manager, and which does not duplicate much code (less risk of having not-updated libraries with security flaws around).

However, there might be issues I haven’t thought about yet. Also, it makes sense to look at GNOME to see how the whole “3rd-party app deployment” issue develops. In case I go further with Listaller-NEXT, it is highly likely that it will make use of the ideas sketched above (comments and feedback are more than welcome!).

October 07, 2014

This is yet another big step for Cutelyst, this release bring several important fixes to the last version, and stabilizes the API a bit more. So far I have successfully deployed 3 commercial applications build on top of Cutelyst + Grantlee5, and the result is great.

If you don’t know Cutelyst yet it’s a Web Framework allowing you to build web applications using Qt, for more info check (this blog and) our website/wiki which is still work in progress: or join #cutelyst on freenode

This release brings the following improvements:

  • Further speed improvements by simplifying several code paths and improving the dispatcher logic was an overall of 15% speedup
  • Added an API to enable Grantlee template caching which greatly improve speed when using Grantlee templating
  • Improved Query and Body parameters to allow for properly dealing with posts that contains the same field id multiple times

New features:

  • REST API – Since 0.1.0 I was asked about supporting REST, and since I needed it for another project that I got involved the support landed early, the behavior is the same as Catalyst::Action::REST which allows you to easily add a foo_DELETE method which will get automatically called if the request method is DELETE for example.
  • Added a Credential HTTP plugin to handle Basic HTTP (and in future Digest) authentication
  • Added support for Authenticate and ProxyAuthenticate basic parsing on the Headers class
  • Finished Context::uriFor() methods that allows for easily building an URI.
  • Added a method to do a DNS PTR lookup to get the hostname of the client
  • Added a C_PATH to more easily set the matching part of the path (thanks to Dan Vrátil)


  • Fixed a few memory leaks
  • Fixed a crash if the body wasn’t set
  • Fixed uWSGI body buffered device
  • And a lot of other misbehaviors found on post release…

For the next release I hope to be able to port the Catalyst tutorial to the Cutelyst equivalent, and finish a few other API changes.

As before the API is unstable but don’t be afraid of playing with it, most changes will simply require a rebuild of your application.

Have fun!

September 11, 2014

Listaller-Logo (with text)It is time for another report on Listaller, the cross-distro 3rd-party package installer, which is now in development for – depending how you count – 5-6 years. This will become a longer post, so you might grab some coffee or tea ;-)

The original idea

The Listaller project was initially started with the goal to make application deployment on Linux distributions as simple as possible, by providing a unified package installation format and tools which make building apps for multiple distributions easier and deployment of updates simple. The key ideas were:

  • Seamless integration of all installation steps into the system – users shouldn’t care about the origin of their application, they just handle all installed apps with the same tool and update all apps with the same interface they use for updating the system.
  • Out-of-the-boy sandboxing for all 3rd-party apps
  • Easy signing and key-validation for Listaller packages
  • Simple creation of updates for developers
  • Resource-sharing: It should always be clear which application uses which library, duplicates should be avoided. The distribution-provided software should take priority, since it is often well-maintained and receives security updates.

The current state

The current release of Listaller handles all of this with a plugin for PackageKit, the cross-distro package-management abstraction layer. It hooks into PackageKit and reads information passing through to the native distributor backend, and if it encounters Listaller software, it handles it appropriately. It can also inject update information. This results in all Listaller software being shown in any PackageKit frontends, and people can work with it just like if the packages were native packages. Listaller package installations are controlled by a machine policy, so the administrator can decide that e.g. only packages from a trusted source (= GPG signature in trusted database) can be installed. Dependencies can be pulled from the distributor’s repositories, or optionally from external sources, like the PyPI.

This sounds good on paper, but the current implementation has various problems.

The issues

The current Listaller approach has some problems. The biggest one lies in the future: Soon, there will be no PackageKit plugins anymore! PackageKit 1.0 will remove support for them, because they appear to be a major source for crashes, even the in-tree plugins cause problems. Also, the PackageKit service itself is currently being trimmed of unneeded features and less-used code. These changes in PackageKit are great and needed for the project (and I support these efforts), but they cause a pretty huge problem for Listaller: The project relies on the PackageKit plugin – if used without it, you loose the system-integration part, which is one of the key concepts of Listaller, and a primary goal.

But this issue is not the only one. There are more. One huge problem for Listaller is dependency-solving: It needs to know where to get software from in case it isn’t installed already. And that has to be done in a cross-distributional way. This is an incredibly complex task, and Listaller contains lots of workarounds for various quirks. It contains so much hacks for distro-specific stuff, that it became really hard to understand. The Listaller dependency model also became very complex, because it tried to handle many corner-cases. This is bad, of course. But the workarounds weren’t added for fun, but because it was assumed to be easier than to fixing the root cause, which would have required collaboration between distributors and some changes on the stack, which seemed unlikely to happen at the time the code was written.

The systemd effort

Also a thing which affects Listaller, is the latest push from the systemd team to allow cross-distro 3rd-party installations to happen. I definitively recommend reading the linked blogpost from Lennart, if you have some spare time! The identified problems are the same as for Listaller, but the solution they propose is completely different, and about three orders of magnitude more invasive than whatever the Listaller project had in mind (I make these numbers up, so don’t ask!). There are also a few issues I see with Lennarts approach, I will probably go into detail about that in another blogpost (e.g. it requires multiple copies of a library lying around, where one version might have a security vulnerability, and another one doesn’t – it’s hard to ensure everything is up to date and secure that way, even if you have a top-notch sandbox). I have great respect for the systemd crew and especially Lennart, and I hope them to succeed with their efforts. However, I also think Listaller can achieve a similar things with a less-invasive solution, at least for the 3rd-party app-installations (Listaller is one of the partial-fix solutions with strict focus, so not a direct competitor to the holistic systemd approach. Both solutions could happily live together.)

A step into the future

Some might have guessed it already: There are some bigger changes coming to Listaller! The most important one is that there will be no Listaller anymore, at least not in its old form.

Since the current code relies heavily on the PackageKit plugin, and contains some ugly workarounds, it doesn’t make much sense to continue working on it.

Instead, I started the Listaller.NEXT project, which is a rewrite of Listaller in C. There are a some goals for the rewrite:

  • No stupid hacks and workarounds: We will not add any workaround. If there is a problem, we will fix it at its source, even if that might be more invasive.
  • Trimmed down project: The new incarnation of Listaller will only support installations of statically linked software at the beginning. We will start with a very small, robust core, and then add more features (like dependency-solving) gradually, but only if they are useful. There will be no feature-creep like in the previous version.
  • Faster development cycle: Releases will happen much faster, not only two or three times a year
  • Integration: Since there is no PackageKit plugin anymore, but integration is still one of Listaller’s key concepts, we will integrate Listaller into downstream tools, ranging from Apper to GNOME-Software. Richard Hughes will help with the integration and user interfaces, so Listaller applications get displayed properly.
  • AppStream-first: AppStream is the ultimate tool for Listaller to detect dependencies. With the 0.6 release, the Listaller component-concept was merged into it, which makes it a very powerful and non-hackish solution for dependency-detection. We will advance the use of its metadata, and probably use it exclusively, which would restrict Listaller to only work properly on distributions which ship AppStream metadata.
  • No desktop-only focus: The previous Listaller was focused only on desktop GUI apps. The new version will be developed with a much larger target audience in mind, including server deployments (“Can I use it to deploy my server app” is one very frequently asked questions about Listaller – with the new version, the answer is yes)
  • We will continue to improve the static-linking and cross-distro development toolchain (libuild, with ligcc, lig++ and binreloc), to make building portable apps easier.

I made a last release of the 0.5.x series of Listaller, to work with PackageKit 0.9.x – the future lies in the C port.

If you are using Listaller (and I know of people who do, for example some deploy statically-linked stuff on internal test-setups with it), stay tuned. The packaging format will stay mostly compatible with the current version, so you will not see many changes there (the plan is to freeze it very soon, so no backwards-incompatible changes are made anymore). The o.5.x series will receive critical bugfixes if necessary.

Help needed!

As always, there is help needed! Writing C is not that difficult ;-) But user feedback is welcome as well, in case you have an idea. The new code will be hosted on Github in the new listaller-next branch (currently not that much to find there). Long-term, we will completely migrate away from Launchpad.

You can expect more blogposts about the Listaller concepts and progress in the next months (as soon as I am done with some AppStream-related things, which take priority).

August 25, 2014

Another mostly bugfix release to make packagers and users happy :)

Sadly I needed to change the direction of where I put most of my efforts, which means that I’m focusing more on getting some commercial products done to get bills payed (as fundraising campaigns doesn’t work well all the time). For a long time I’ve been trying to polish everything I could to have the desktop I wanted, but recently I realized that the way I was doing it would never work, first because I’d need to convince people to think like I do, second because no one in free software writes stuff for free, and this took me a lot of time to realize.

Almost everyone writes stuffs for himself, otherwise there’s no pleasure, so unlike companies you can’t tell a free software developer to work on something he/she doesn’t like, which is one explanation for why most of the projects I started received very little help, an important help (don’t get offended) but still I don’t have active developers in Apper*, print-manager, libudisks-qt, colord-kde, aptcc* (* Matthias and some Fedora dudes have added some nice features) and a few others. The KDE community has always been kind to notice my code mistakes or even fix the code by themselves but featurea are a different matter.

Don’t worry I’m not moving to OSX :P

But for a long time I’ve been building in my mind the Workspace that I want, and with Wayland I realized It would be somehow easy to achieve what I want when speaking of a desktop shell, which would basically be a shell where all widgets are independent process, where a QML compositor just properly place it’s surfaces, Aaron already covered the pros/cons of such approach however I’m stubborn …, I know it’s a huge task to start a new workspace/DE whatever and I’m not going to do that right away (tho I have played with some Wayland code already), instead I’m trying to get my commercial software to pay for it, which might take quite some time :P

So I just would like to maybe catch someone that cares for some of these stuff I maintain and give a hand, specially on KF5. I don’t yet have KF5 packages ready in my distro and as I said I’m focusing on other stuff, I’ll still maintain them and eventually port them by myself but I’m mostly in bugfix mode :P except for Cutelyst which is a project I’m actively working on as I need it for the web stuff I’ve been doing :)

A good start is porting print-manager to KF5 which should be rather easy.

And here is hopefully the last Qt4/KDE4 based Apper :P


August 16, 2014

There hasn’t been a progress-report on DEP-11 for some time, but that doesn’t mean there was no work going on on it.

DEP-11 is Debian’s implementation of AppStream, as well as an effort to enhance the metadata available about software in Debian. While initially, AppStream was only about applications, DEP-11 was designed with a larger scope, to collect data about libraries, binaries and things like Python modules. Now, since AppStream 0.6, DEP-11 and AppStream have essentially the same scope, with the difference of DEP-11 metadata being described in YAML, while official AppStream data is XML. That was due to a request by our ftpmasters team, which doesn’t like XML (which is also not used anywhere in Debian, as opposed to YAML). But this doesn’t mean that people will have to deal with the YAML file format: The libappstream library will just take DEP-11 data as another data source for it’s Xapian database, allowing anything using libappstream to access that data just like the XML stuff. Richards libappstream-glib will also receive support for the DEP-11 format soon, filling it’s in-memory data cache and enabling the use of GNOME-Software on Debian.

So, what has been done so far? The past months, my Google Summer of Code student. Abhishek Bhattacharjee, was working hard to integrate DEP-11 support into dak, the Debian Archive Kit, which maintains the whole Debian archive. The result will be an additional metadata table in our internal Postgres database, storing detailed information about the software available in a Debian package, as well as “Components-<arch>.yml.gz” files in the Debian repositories. Dak will also produce an application icon-cache and a screenshots repository. During the time of the SoC, Abhishek focused mainly on the applications part of things, and less on the other components (like extracting data about Python modules or libraries) – these things can easily be implemented later.

The remaining steps will be to polish the code and make it merge-ready for Debian’s dak (as soon as it has received enough testing, we will likely give it a try on the Tanglu Debian derivative). Following that, Apt will be extended to fetch the DEP-11 data on-demand on systems where it is useful (which is currently mostly desktop-systems) – if you want to save a little bit of space, you will be able to disable downloading this extra metadata in Apt. From there, libappstream will take the data for it’s Xapian db. This will lead to the removal of the much-hated (from ftpmasters and maintainers side) app-install-data package, which has not been updated for two years and only contains a small fraction of the metadata provided by DEP-11.

What Debian will ultimately gain from this effort is support for software-centers like GNOME-Software, and improved support for tools like Apper and Muon in displaying applications. Long-term, with more metadata being available, It would be cool to add support for it to “specialized package managers”, like Python’s pip, npm or gem, to make them fetch information about available distribution software and install that instead of their own copies from 3rd-party repositories, if possible. This should ultimatively lead to less code duplication on distributions and will likely result in fewer security issues, since the officially maintained and integrated distribution packages can easily be used, if possible. This is no attempt to make tools like pip obsolete, but an attempt to have the different tools installing software on your machine communicate better, instead of creating parallel worlds in terms of software management. Another nice sideeffect of more metadata will be options to search for tools handling mimetypes in the software repos (in case you can’t open a file), smart software centers installing missing firmware, and automatic suggestions for developers which software they need to install in order to build a specific software package. Also, the data allows us to match software across distributions, for that, I will have some news soon (not sure how soon though, as I am currently in thesis-writing-mode, and therefore have not that much spare time). Since the goal is to have these features available on all distributions supporting AppStream, it will take longer to realize – but we are on a good way.

So, if you want some more information about my student’s awesome work, you can read his blogpost about it. He will also be at Debconf’14 (Portland). (I can’t make it this time, but I surely won’t miss the next Debconf)

Sadly, I only see a very small chance to have the basic DEP-11 stuff land in-time for Jessie (lots of review work needs to be done, and some more code needs to be written), but we will definitively have it in Jessie+1.

A small example on how this data will look like can be found here – a larger, actual file is available here. Any questions and feedback are highly appreciated.

July 24, 2014

Release early, release often has never been my strength especially since I don’t do a fair scheduling of all the projects I’m involved…

So since I was in need to better polish Cutelyst I took more time on it, and the result is great, around 100% speed up and a few new features added.

Cutelyst uWSGI plugin now has support for –thread, which will create a QThread to process a request, however I strongly discourage its usage in Cutelyst, the performance is ~7% inferior and a crash in your code will break other requests, and as of now ASYNC mode is not supported in threaded mode due to a limitation in uWSGI request queue.

Thanks to valgrind I managed to make a hello world application from 5K request per second on 0.2.0 to 10K req/s on 0.3.0 on an Intel Core2Duo 2.4Ghz (or 44K req/s on an AMD Phenom II 965 x4 3.4Ghz), however if you enable Grantlee templating you get around 600 req/s so if I happen to have time I will be looking into improving its performance.

Response::body() is now a QIODevice so you can set a QFile* of something else and have Cutelyst to send it back.

Now points to a gitorious Wiki which is slowly getting populated, and API is available as

The 0.3.0 tarball can be downloaded here

Have fun :)

July 16, 2014

appstream-logoToday I am very happy to announce the release of AppStream 0.7, the second-largest release (judging by commit number) after 0.6. AppStream 0.7 brings many new features for the specification, adds lots of good stuff to libappstream, introduces a new libappstream-qt library for Qt developers and, as always, fixes some bugs.

Unfortunately we broke the API/ABI of libappstream, so please adjust your code accordingly. Apart from that, any other changes are backwards-compatible. So, here is an overview of what’s new in AppStream 0.7:

Specification changes

Distributors may now specify a new <languages/> tag in their distribution XML, providing information about the languages a component supports and the completion-percentage for the language. This allows software-centers to apply smart filtering on applications to highlight the ones which are available in the users native language.

A new addon component type was added to represent software which is designed to be used together with a specific other application (think of a Firefox addon or GNOME-Shell extension). Software-center applications can group the addons together with their main application to provide an easy way for users to install additional functionality for existing applications.

The <provides/> tag gained a new dbus item-type to expose D-Bus interface names the component provides to the outside world. This means in future it will be possible to search for components providing a specific dbus service:

$ appstream-index what-provides dbus org.freedesktop.PackageKit.desktop system

(if you are using the cli tool)

A <developer_name/> tag was added to the generic component definition to define the name of the component developer in a human-readable form. Possible values are, for example “The KDE Community”, “GNOME Developers” or even the developer’s full name. This value can be (optionally) translated and will be displayed in software-centers.

An <update_contact/> tag was added to the specification, to provide a convenient way for distributors to reach upstream to talk about changes made to their metadata or issues with the latest software update. This tag was already used by some projects before, and has now been added to the official specification.

Timestamps in <release/> tags must now be UNIX epochs, YYYYMMDD is no longer valid (fortunately, everyone is already using UNIX epochs).

Last but not least, the <pkgname/> tag is now allowed multiple times per component. We still recommend to create metapackages according to the contents the upstream metadata describes and place the file there. However, in some cases defining one component to be in multiple packages is a short way to make metadata available correctly without excessive package-tuning (which can become difficult if a <provides/> tag needs to be satisfied).

As small sidenote: The multiarch path in /usr/share/appdata is now deprecated, because we think that we can live without it (by shipping -data packages per library and using smarter AppStream metadata generators which take advantage of the ability to define multiple <pkgname/> tags)

Documentation updates

In general, the documentation of the specification has been reworked to be easier to understand and to include less duplication of information. We now use excessive crosslinking to show you the information you need in order to write metadata for your upstream project, or to implement a metadata generator for your distribution.

Because the specification needs to define the allowed tags completely and contain as much information as possible, it is not very easy to digest for upstream authors who just want some metadata shipped quickly. In order to help them, we now have “Quickstart pages” in the documentation, which are rich of examples and contain the most important subset of information you need to write a good metadata file. These quickstart guides already exist for desktop-applications and addons, more will follow in future.

We also have an explicit section dealing with the question “How do I translate upstream metadata?” now.

More changes to the docs are planned for the next point releases. You can find the full project documentation at Freedesktop.

AppStream GObject library and tools

The libappstream library also received lots of changes. The most important one: We switched from using LGPL-3+ to LGPL-2.1+. People who know me know that I love the v3 license family of GPL licenses – I like it for tivoization protection, it’s explicit compatibility with some important other licenses and cosmetic details, like entities not loosing their right to use the software forever after a license violation. However, a LGPL-3+ library does not mix well with projects licensed under other open source licenses, mainly GPL-2-only projects. I want libappstream to be used by anyone without forcing the project to change its license. For some reason, using the library from proprietary code is easier than using it from a GPL-2-only open source project. The license change was also a popular request of people wanting to use the library, so I made the switch with 0.7. If you want to know more about the LGPL-3 issues, I recommend reading this blogpost by Nikos (GnuTLS).

On the code-side, libappstream received a large pile of bugfixes and some internal restructuring. This makes the cache builder about 5% faster (depending on your system and the amount of metadata which needs to be processed) and prepares for future changes (e.g. I plan to obsolete PackageKit’s desktop-file-database in the long term).

The library also brings back support for legacy AppData files, which it can now read. However, appstream-validate will not validate these files (and kindly ask you to migrate to the new format).

The appstream-index tool received some changes, making it’s command-line interface a bit more modern. It is also possible now to place the Xapian cache at arbitrary locations, which is a nice feature for developers.

Additionally, the testsuite got improved and should now work on systems which do not have metadata installed.

Of course, libappstream also implements all features of the new 0.7 specification.

With the 0.7 release, some symbols were removed which have been deprecated for a few releases, most notably as_component_get/set_idname, as_database_find_components_by_str, as_component_get/set_homepage and the “pkgname” property of AsComponent (which is now a string array and called “pkgnames”). API level was bumped to 1.


A Qt library to access AppStream data has been added. So if you want to use AppStream metadata in your Qt application, you can easily do that now without touching any GLib/GObject based code!

Special thanks to Sune Vuorela for his nice rework of the Qt library!

And that’s it with the changes for now! Thanks to everyone who helped making 0.7 ready, being it feedback, contributions to the documentation, translation or coding. You can get the release tarballs at Freedesktop. Have fun!

July 05, 2014

Five months ago I announced the very first release of Cutelyst, a web framework powered by Qt 5. Time passes and I started shaping my real world applications written with it, I should probably release a newer version earlier but it’s not very nice to keep releasing API that changes a lot, however next version (0.3.0) will still contains API changes. What changed from 0.1.0 to 0.2.0:

  • I have setup a documentation website
  • I started a WordPress like blog written with Cutelyst this right now is a nice example app, if you like it please help :)
  • Changed the API to expose the forked behavior thus being able to setup database connection in each new process avoiding a mess
  • UWSGI got a patch to deal with workers that don’t want to work, in other words when something on post-fork fails like reaching the dababase connection limit the worker can exit and won’t be restarted.
  • A bunch of bugfixed of course
  • Make use of categorized logging (ie qCDebug)
  • Allow for UWSGI to restart the application as soon as your application .so file changes, this greatly improve development speed :)
  • Encapsulated the post body into a QIODevice subclass on UWSGI engine this means a QFile is –post-buffering limit is reached, or a custom IODevice that deals with the request memory buffer
  • A parser to get uploaded content “multipart/form-data” which is fast and can work with lots of data

For the next version I’ll try to get a properQt loop integration. Download here Have fun!

June 18, 2014

Bartholomea annulata | (c) Kevin Bryant

It is time for a new Tanglu update, which has been overdue for a long time now!

Many things happened in Tanglu development, so here is just a short overview of what was done in the past months.



The whole Tanglu distribution is now built with Debile, replacing Jenkins, which was difficult to use for package building purposes (although Jenkins is great for other things). You can see the Tanglu builders in action at

The migration to Debile took a lot of time (a lot more than expected), and blocked the Bartholomea development at the beginning, but now it is working smoothly. Many thanks to all people who have been involved with making Debile work for Tanglu, especially Jon Severinsson. And of course many thanks to the Debile developers for helping with the integration, Sylvestre Ledru and of course Paul Tagliamonte.

Archive Server Migration

Those who read the tanglu-announce mailinglist know this already: We moved the main archive server stuff at to to a new location, and to a very powerful machine. We also added some additional security measures to it, to prevent attacks.

The previous machine is now being used for the bugtracker at and for some other things, including an archive mirror and the new Tanglu User Forums. See more about that below :-)


There is huge ongoing work on package transitions. Take a look at our transition tracker and the staging migration log to get a taste of it.

Merging with Debian Unstable is also going on right now, and we are working on merging some of the Tanglu changes which are useful for Debian as well (or which just reduce the diff to Tanglu) back to their upstream packages.


Work on the Tanglu Live-Installer, although badly needed, has not yet been started (it’s a task ready for taking by anyone who likes to do it!) – however, some awesome progress has been made in making the Debian-Installer work for Tanglu, which allows us to perform minimal installations of the Tanglu base systems and allows easier support of alternative Tanglu falvours. The work on d-i also uncovered a bug which appeared with the latest version of findutils, which has been reported upstream before Debian could run into it. This awesome progress was possible thanks to the work of Philip Muškovac and Thomas Funk (in really hard debug sessions).

Tanglu ForumsTanglu Users

We finally have the long-awaited Tanglu user forums ready! As discussed in the last meeting, a popular demand on IRC and our mailing lists was a forum or Stackexchange-like service for users to communicate, since many people can work better with that than with mailinglists.

Therefore, the new English TangluUsers forum is now ready at The forum software is in an alpha version though, so we might experience some bugs which haven’t been uncovered in the testing period. We will watch how the software performs and then decide if we stick to it or maybe switch to another one. But so far, we are really happy with the Misago Forums, and our usage of it already led to the inclusion of some patches against Misago. It also is actively maintained and has an active community.

Misc Thingstanglu logo pure


We will ship with at least KDE Applications 4.13, maybe some 4.14 things as well (if we are lucky, since Tanglu will likely be in feature-freeze when this stuff is released). The other KDE parts will remain on their latest version from the 4.x series. For Tanglu 3, we might update KDE SC 4.x to KDE Frameworks 5 and use Plasma 5 though.


Due to the lack manpower on the GNOME flavor, GNOME will ship in the same version available in Debian Sid – maybe with some stuff pulled from Experimental, where it makes sense. A GNOME flavor is planned to be available.

Common infrastructure

We currently run with systemd 208, but a switch to 210 is planned. Tanglu 2 also targets the server in version 1.16. For more changes, stay tuned. The kernel release for Bartholomea is also not yet decided.


Work on the default Tanglu 2 design has started as well – any artwork submissions are most welcome!

Tanglu joins the OIN

The Tanglu project is now a proud member (licensee)  of the Open Invention Network (OIN), which build a pool of defensive patents to protect the Linux ecosystem from companies who are trying to use patents against Linux. Although the Tanglu community does not fully support the generally positive stance the OIN has about software patents, the OIN effort is very useful and we agree with it’s goal. Therefore, Tanglu joined the OIN as licensee.

And that’s the stuff for now! If you have further questions, just join us on #tanglu or #tanglu-devel on Freenode, or write to our newly created forum! – You can, as always, also subscribe to our mailinglists to get in touch.

May 26, 2014

And again, another KDE-AppStream post ;-) If you want to know more about AppStream metadata and why adding it to your project is a good idea, you might be interested in this blogpost (and several previous ones I wrote).

Originally, my plan was to directly push metadata to most KDE projects. The problem is that there is no way to reach all maintainers and have them opt-out for getting metadata pushed to their repositories. There is also no technical policy for a KDE project, since “KDE” is really only about the community right now, and there are no technical criteria a project under the KDE umbrella has to fulfill (at least to my knowledge, in theory, even GTK+ projects are perfectly fine within KDE).

Since I feel very uncomfortable in touching other people’s repositories without sending them a note first, I think the best way forward is an opt-in approach.

So, if you want your KDE project to ship metadata, follow these simple steps:

1. Check if there already is metadata for your project

That’s right – we already have some metadata available. Checkout the kde-appstream-metadata-templates repository at Github. You can take the XML file from there, if you want. Just make sure that there are no invalid tags in the description field (no <a/> nodes allowed, for example – the content is not HTML!), check if you have an SPDX-compliant <project_license/> tag, check if the provided public interfaces in the <provides/> tag match your project and also test if the URLs work.

Then you can copy the modified AppStream metadata to your project.

2. Write new metadata

How to write new metadata is described in detail at this TechBase Wiki page. Just follow the instructions.

In case you need help or want me to push the metadata to your project if you don’t have the time, you can also write me an email: matthias [{AT}] tenstral . net – or alternatively file a bug against the Github project linked above.

Don’t forget to have CMake install your shiny new metadata into /usr/share/appdata/.

All metadata you add to your project will automatically get translated by the KDE l10n scripts, no further action is required. So far, projects like Ark, K3b and Calligra are shipping metadata, and the number of AppStream-aware projects in KDE is growing constantly, which greatly improves their visibility in software centers, and will help distributions a lot in organizing installed software.

If you have further questions, please ask! .-)

May 05, 2014

This new release doesn’t bring new features but it’s an important one since it supports PackageKit 0.9.x API, also this is the first release which uses the first 100% async calls on PackageKit-Qt.

Making Pk-Qt sync-free wasn’t so complicated but since I had to rethink the API the client code of Apper also needed to be adapted, I must say that I was lazy sometimes and just created a QEventLoop in Apper to wait for the reply… and this basically means that there will be new bugs in Apper, and hopefully some will be fixed.

It’s interesting that using DBus doesn’t means the code talking to it is async, the proxy classes generated by qdbus2cpp do a blocking call everytime you ask for a property, so you have to avoid it and do a GetProperties call which returns to a callback, although this worked well for PackageKitQt I failed to do the same for libnm-qt, and that is because it’s nearly impossible to make it somehow reliable due to NetworkManager itself not using gdbus, so you can’t have a reliable snapshot of it, unless you do an introspection call which is also insane, so I prefer to wait for when they switch to gdbus :P

Apper 0.9.0 depends on Qt4, KDE4 libs and PackageKitQt 0.9.2.

I’m not giving it much attention lately since I’m slowly building Apper on Qt5 which will be using QtQuick2, as my time is short supporting this old version is mainly to make packagers happy :)

May 03, 2014

kde-appstreamIt took some time, but now it’s finally done: KDE has translations for AppStream upstream metadata!

AppStream is a Freedesktop project to extend metadata about the software projects which is available in distributions, especially regarding applications. Distributions compile a metadata file from data collected from packages, .desktop files and possibly other information sources, and create an AppStream XML file from it, which is then – directly or via a Xapian cache – read by software-center-like applications such as GNOME-Software or KDEs Apper.

Since the metadata available from current sources is not standardized and rather poor, upstream projects can ship small XML files, AppStream upstream metadata or AppData in short. These files contain additional information about a project, such as a long description and links to screenshots. They also provide hints about public interfaces a software provides, for example binaries and libraries, making it possible for distributors to give users exactly the right package name in case they are missing a software component.

So, in order to represent graphical KDE applications like they deserve it in the new software centers making use of AppStream, we need to ship AppData files, with long descriptions, screenshots and a few URLs.

But how can you create these metadata files? In case you want your graphical KDE app to ship an AppData file, there is now a help page on the Techbase Wiki which provides all information needed to get started!

For non-visual stuff or software which just wants to publish it’s provided interfaces with AppStream metadata, there is a dedicated page for that as well. Shipping metadata for non-GUI apps will help programmers to satisfy depedencies in order to compile new software, enhance bash-completion for missing binaries and provides some other neat stuff (take a look at this blogpost to get a taste of it).

And if you want to read a FAQ about the metadata stuff and get the bigger picture, just go to the Techbase Wiki page about AppStream metadata as well.

The pages are not 100% final, so if you have questions, please write me a mail and I’ll update the pages, or simply correct/refine it by yourself (it’s a wiki afterall).

And now to the best thing: As soon as you ship an AppStream upstream metadata file (*.appdata.xml* for apps or *.metainfo.xml* for other stuff), the KDE l10n-script (Scripty!) will automatically start translating it, just like we already do with .desktop files. No further actions are necessary.

I already have a large amount of metadata files here, partially auto-generated, which show that we have about 160+ applications in KDE which could get an AppData file, not counting any frameworks or other non-GUI stuff yet. Since that is a bit much to submit via Reviewboard (which I originally planned to do), I hope I can commit the changes directly to the respective repositories, where the maintainers can take a look at it and adjust it to their liking. If that idea does not receive approval, I will just publish a set of data at some place for the KDE app maintainers to take as a reference (the auto-generated stuff needs some fixup to be commit-ready (which I’d do in case I can just commit changes)). Either way, it is safe now to write and ship AppData files in KDE projects!

In order to get your stuff translated, it is necessary that you follow the AppStream 0.6 metadata specification, and not one of ther older revisions. You can easily detect 0.6 metadata by the <component> root node, instead of <application>, or by it having a metadata_license tag. We don’t support the older versions simply because it’s not necessary, as there were only two KDE projects shipping AppData before, which are now using 0.6 data as well. Since 0.6, the metadata XML format is guaranteed to be stable, and the only reason which could make me change it in an incompatible way is to prevent something as bad as the end of the world from happening (== won’t happen ;-) ). You can find the full specification (upstream and distro data) here.

All parsers are able to handle 0.6 data now, and the existing tools are almost all migrated already (might take a few months to hit the distributions though).

So, happy metadata-writing! :-)

Thanks to all people who helped with making this happen, and especially Burkhard Lück and Albert Astals Cid for their patch-review and help with integrating the necessary changes into the KDE l10n-script.

April 25, 2014

Today I released AppStream and libappstream 0.6.1, which feature mostly bugfixes, so nothing incredibly exciting to see there (but this also means no API/ABI breaks). The release clarifies some paragraphs in the spec which people found confusing, and fixes a few issues (like one example in the docs not being valid AppStream metadata). As only spec extension, we introduce a “priority” property in distro metadata to allow metadata from one repository to override data shipped by another one. This is used (although with a similar syntax) in Fedora already to have “placeholder” data for non-free stuff, which gets overridden by the real metadata if a new application was added. In general, the property tag was added to make the answer to the question “which data is preferred” much less magic.

The libappstream library got some new API to query component data in different ways, and I also brought back support for Vala (so if you missed the Vapi file: It’s back now, although you have to manually enable this feature).

The CLI tool also got some extensions to query AppStream data. Here is a brief introduction:

First of all, we need to make sure the database is up-to-date, which should be the case already (it is rebuilt automatically):

$ sudo appstream-index refresh

The database will only be rebuilt when necessary, if you want to force a rebuild anyway, use the “–force” parameter.

Now imagine we want to search for an app containing the word “media” (in description, keywords, summary, …):

$ appstream-index s media

which will return:

Identifier: gnome-media-player.desktop [desktop-app]
Name: GNOME Media Player
Summary: A simple media player for GNOME
Package: gnome-media-player
Identifier: mediaplayer-app.desktop [desktop-app]
Name: Media Player
Summary: Media Player
Package: mediaplayer-app
Identifier: kde4__plasma-mediacenter.desktop [desktop-app]
Name: Plasma Media Center
Summary: A mediacenter user interface written with the Plasma framework
Package: plasma-mediacenter

If we already know the name if a .desktop-file or the ID of a component, we can have the tool print out information about the application, including which package it was installed from:

$ appstream-index get lyx.desktop

If we want to see more details, including e.g. a screenshot URL and a longer description, we can pass “–details” to the tool:

Identifier: lyx.desktop [desktop-app]
Name: LyX
Summary: An advanced document processor with the power of LaTeX.
Package: lyx-common
Icon: lyx.png
Description: LyX is a document processor that encourages an approach to writing
 based on the structure of your documents (WYSIWYM) and not simply
 their appearance (WYSIWYG).
 LyX combines the power and flexibility of TeX/LaTeX[...]
Sample Screenshot URL:
License: GPLv2+

(I truncated the results slightly ;-) )

Okay, so far so good. But now it gets really exciting (and this is a feature added with 0.6.1): We can now query a component by the items it provides. For example, I want to know which software provides the library libfoo.2:

appstream-index what-provides lib

This also works with binaries, or Python modules:

appstream-index what-provides bin apper

This stuff works distribution-agnostic, and as long as software ships upstream metadata with a valid <provides/> field, or the distributor adds it while generating AppStream distro metadata.
This means that software can – as soon as we have sufficient metadata of this kind – declare it’s dependencies upstream in form of a simple text file, referencing the needed components to build and run it on any Linux distribution. Users can simply install missing stuff by passing that file to their package manager, which can look up the components->packaging mapping and versions and do the right thing in installing the dependencies. So basically, this allows things “pip -r” does for Python, but for any application (not only Python stuff), and based on the distributors package database.

With the provides-items, we can also scan software to detect it’s dependencies automatically (and have it in a distro-agnostic form directly). We can also easily search for missing mimetype-handlers, missing kernel-modules, missing firmware etc. to install it on-demand, making the system much smarter in handling it’s dependencies. And users don’t need to do search orgies to find the right component for a given task.

Also on my todo list for the future, based on this feature: A small tool telling upstream authors which distribution has their application in which version, using just one command (and AppStream data from multiple distros).

Also planned: A cross-distro information page showing which distros ship which library versions, Python modules and application versions (and also the support status of the distro), so developers know which library versions (or GCC versions etc.) they should at least support to make their application easily available on most distributions.

As always, you can get the releases on Freedesktop, as well es the AppStream specification.

April 19, 2014


this is just a quick bug fix release, the last one depending on PackageKit 0.8 series, it doesn’t have new features apart from having some fixed support for appstream 0.6.

For the next 0.8.3 version PackageKit 0.9 will be required.

For more information you can look at the git logs.

Have fun :)

February 12, 2014

FOSDEM is over, and it has been a great time! I talked to many people (thanks to full devrooms ;-) Graph Databases and Config management was crazy, JavaScript as well…) and there might be some cool new things to come soon, which we discussed there :-)
I also got some good and positive feedback on the projects I work on, and met many people from Debian, Kubuntu, KDE and GNOME (haven’t seen some of them for almost 3 years) – one of the best things of being at FOSDEM is, that you not only see people “of your own kind” – for example, for me as Debian developer it was great to see Fedora people and discuss things with them, something which rarely happens at Debian conferences. Also, having GNOME and KDE closely together again (litterally, their stands were next to each other…) is something I missed since the last Desktop Summit in 2011.

My talks were also good, except for the beamer-slides-technical-error at the beginning, which took quite some time (I blame KScreen ;-) ).

In case you’re interested in the slides, here they are: slides for FOSDEM’14 AppStream/Listaller talks.

The slides can likely be understood without the talk, they are way too detailed (usually I only show images on slides, but that doesn’t help people who can’t see the talk ^^)

I hope I can make it to FOSDEM’15 as well – I’ve been there only once, but it already is my favourite FOSS-conference (and I love Belgian waffles) ;-)

February 11, 2014

Three months ago Custelyst was started,

it was somehow a proof of concept to see if I could build something that could be used in the real world, but the more I progressed on getting all the pieces together the happier with the overall result I got. And the initial result is here today!

I have made a few benchmarks comparing some simple pages delivery using Django (python) and Perl Catalyst, and overall the time to first byte was around 3 times faster than both, comparing RAM usage it was like 2MB vs 50MB from the Perl version and 20MB of the Python one, the CPU time was also much smaller if compared to both which probably means it could handle more requests, but this isn’t a completely fair benchmark (as always) because there are several other things to measure…

Below I’ll try to clarify some doubts raised from my last post and what makes this new Web Framework unique:

But there is already QDjango and Tufao…

I didn’t know about these ones when I first started Cutelyst, but after taking a look at them, both misses a templating system (or integration with one), I also didn’t like the way one would register actions to be dispatched, IMO the way it’s done in Cutelyst is easier and nicer. And they doesn’t count with WSGI integration, they have their own HTTP parser/server. I also didn’t like looking at code :P

Why not DJango or any other well established framework out there?….

Apart from QDjango and Tufao there are no other (AFAIK) Qt based frameworks, there are indeed some C++ ones that I knew before but I didn’t like the way of doing things there, in fact when I started with Perl Catalyst I really liked how it was written. So in short I don’t know well enough other languages and I have no plans wasting my time learning other languages, I’d rather try to master at least one (although I know I’m far from it).

What’s in this release?

  • Glad you asked (duh!)
  • Production ready integration with uWSGI, meaning Cutelyst has a uWSGI plugin that will load your application (which will also be a plugin), and thus thanks to uWSGI it will have out of the box support for FastCGI, HTTP(s), uWSGI (protocol) and probably some other.
  • ClearSilver templates integration, this templating system is written in C and it’s amazingly fast, but it’s also incredible limited to what you can do in the view.
  • Grantlee (Django) templates integration, it’s used in KDE PIM to allow for easy templating of HTML, it’s slower than ClearSilver but offers much more fexibilities on your views, and since it’s also written in Qt the introspection comes for free.
  • Complete Path dispatcher, which means if you Port a Perl Catalyst Application that was using Path actions it will work as expected. This dispatcher matches an url to a Q_INVOKABLE method depending on it’s signature, The Catalyst Chained dispatcher is not implemented yet.
  • Session plugin, easy to use way of persisting data across requests, right now it’s done with a QSettings file, but integration with Redis and MongoDB are planned.
  • Authentication plugin, allowing you to authenticate your users using different Realms/stores, currently a StoreMinimal is implemented which can be used mostly for testing purposes (ie on the code you add your users)
  • StaticSimple plugin, allows for serving static content reducing the complexity of serving static content while developing your app.

If you are concerned about API/ABI stability I don’t promise it just right now it’s almost stable, probably 0.2 or 0.3 versions will have a promised stable API.

And yes, I have just put it on production is running it right now (but the site is user restricted).

My plans now is to setup with docs and examples, as well as writing a web blog, bug tracker and wiki based on it.

Have fun!


January 26, 2014

After years of missing this conference, I will finally make it to Brussels this time!

I will even have some talks, one about Listaller (in the lightning-talks section) and one about AppStream and Listaller in the Distributions track. The lightning-talk will explain why distributing (some) applications independent from the distribution makes sense, and how Listaller does this. I will also very briefly talk about the concepts behind Listaller and which tools it offers for application developers to create & package cross-distro apps.

The AppStream & Listaller talk will be much more detailed. It will cover the rationale for AppStream, what AppData is good for and how AppData files relate to AppStream. I will also reflect the AppData adoption in GNOME/Fedora and why GNOME-Software is the right way to go forward in developing software-centers. It will of course also include our future plans for AppStream.

On the Listaller side, I will talk about how Listaller is connected to AppStream and PackageKit and why distributions should ship a way to install cross-distro packaged apps at all. I will explain module-definitions and why they are useful. An overview of the internals of Listaller and it’s system integration is also included, as well as how it can be compared to competing installation solutions.

If you are at FOSDEM and have questions about AppStream/PackageKit/Listaller/Tanglu/Debian/etc., please ask them! ;-) See you there!


January 19, 2014

While writing this post, I recognize that we might need a news-blog for tanglu logo pureTanglu :-)

Since yesterday, we have an (installable!) Beta2 release of Tanglu!

Compared to the previous snapshot, it comes with a huge amount of improvements and bugfixes:

  • The systemd journal is now used for logging exclusively. You can install rsyslog to get the old behaviour back, and switch the journal to non-persistent mode. If you want to test the journal, type “journalctl -b” to get all messages from last boot. If we experience issues, we might revert to rsyslog for the final release.
  • KDE received a new theming for KDM/KSplash, so we can now show a face-browser in KDM, which is considered more user-friendly. User faces are now loaded from the user’s settings, instead of system directories by default. If you want to load a large user list (> 100) over network, you might not want this, but in that case you can easily revert the change.
  • Tanglu does no longer include an MTA by default. If you want one, install one and configure it. On desktop systems, having an MTA makes no sense, especially when it is not properly configured (we follow Ubuntu/Fedora here).
  • We ship with a functional live-installer now! You can install Tanglu easily to try it out. Unfortunately, the Debian installer is still not working, so you will have to boot into the live session to install Tanglu.
  • GNOME is now working. GDM and g-s-d will no longer crash, and Mutter is able to utilize llvmpipe for rendering in a virtual machine. The Shell and various other GNOME parts were updated to GNOME 3.10
  • We now ship the 1.14 version of Xorg now
  • Linux 3.12 is now default

Known issues:

  • People with ATi graphics cards might have trouble with installing the proprietary fglrx driver
  • In some very rare cases, the live-cd won’t auto-login. In that case, just reboot and try again, or use the password “live” to access the account.
  • Some packages are not installable, there is a list compiled here, with detailed info why they are broken (they will be removed from the release if they are not fixed) Some transitions are still pending (causes broken/not-coinstallable packages)
  • Users with a non-English locale will have an additional “Public” folder in their home-directory on KDE
  • We don’t support UEFI at time, please use the legacy mode to boot
  • We just noticed that the GNOME images won’t start the GNOME-Session for some reason. [UPDATE:21.01]: We published new live-cd images which have this issue fixed.

You can download the release from one of these mirrors:


Additionally, there is now a release schedule for Tanglu Aequorea, and we will do a vote on the name for the next release soon (proposed names).

As a note to Debian: systemd is working pretty well for Tanglu so far ;-)

I am pretty happy with this Beta2 release, because Tanglu is shaping out to be the distribution we imagined in the beginning.

Have fun! And as bonus, here are some images from Beta2:


tanglu-beta2-kde1 tanglu-beta2-kde2












November 04, 2013

appstream-logoWhile reading stuff posted by others about AppStream, and because of the discussion happening about AppData on kde-core-devel right now, I feel the need to clarify a few things. Especially because some are implementing AppStream in a way which is not really ideal right now. This is to some extend my fault, because I should have communicated this in a much more visible way.

To those people who don’t know it already: AppStream is a Freedesktop project aiming at providing basic building blocks to create distro- and desktop-agnostic software centers.

So, let’s answer some questions about AppStream!

Is AppStream GNOME or $distro specific?

No, not at all! It was originally created by people from at least 4 different distributions, and I took great care of it not becoming specific to any desktop or distribution. GNOME just happened to go ahead and implement the specs, which was absolutely necessary, since there was less progress in implementing AppStream for a long time.

How does AppStream actually work?

AppStream is a bunch of things, so I will only focus on what we have specified right now and what is working.

Basically, the distributor compiles a list of applications available in the repositories, and makes it available in some defined directories on the system. AppStream defines an XML specification for that, but since some peple don’t want to use it or can’t use it, there are also others ways to publish AppStream application data. For example, Debian will likely use YAML for that.

This data is taked by the AppStream software (running as a PackageKit plugin) and transformed into a Xapian database. This database is then in turn used by software-centers, in combination with PackageKit, to present applications.

This is the reason why it is bad to use the XML data directly – it might not be available on every distribution. The Xapian database is what matters. The database can be accessed using libappstream, a GLib based library (so far, there was no need for a Qt version).

Why is GNOME using the XML data directly then?

The libappsream stuff was under heavy construction, and GNOME wanted to be fast and ship the stuff with Fedora in their next release. They’ll likely switch to the Xapian db soon, or offer it as backend.

Is AppStream already used in KDE?

Yes, Apper can utilize it, see one of my previous blogposts.

What is AppData?

AppData is an initiative started by Richard Hughes in order to enhance the quality of applications descriptions shipped with AppStream. It defines a small file $appname.appdata.xml, which describes the application, sets screenshots etc. These files can be parsed at the distribution’s side in order to enhance the app metadata. They can also be translated upstream.

AppData might be merged into the XDG AppStream pages later, but that is to be discussed.

Are application authors forced to ship AppData files?

No, nobody is forced ;-) However, the GNOME Software application-center prefers applications shipping more metadata, and “punishes” the others, so shipping an AppData file makes sense for higher lising in GS. This is a policy decision by GNOME, KDE can make it’s own ones here.

Shipping AppData files makes sense, in general, because it enhances the metadata distributed with your application. It is also the file-format used by Listaller (well, a superset of it) in order to generate cross-distro app-packages, so you might want to consider adding one.

Are there any rules for the AppData files?

Yes, you can find them on the AppData specification page. However, these are more recommendations than “forced”, and it is currently aimed at GNOME apps. I later want to generalize that and create an own page with recommendations for KDE (martin had some good ideas already).

Where do software-centers get the screenshots from?

The screenshots are defined in AppData files, and the cached by the distributor. If there are no “official” screenshots, user-provided screenshots will be taken, using a screenshots-service with screenshots.d.n-like API.

Where do I find the specs?

The official AppStream spec, the thing which distributors should implement, can be found on Freedesktop.

The AppData spec can be found here. It also includes some nice hints on how to handle screenshots etc. and includes it’s own FAQ.

Where do I find libappstream?

The source code can be found on Gitorious, releases are available on fd.o software. Contributions to the code are welcome!

I want to change X in the spec, because of argument Y which brings improvement Z

Great! Please get in contact with me or Richard. The only feature we would not consider for the official standard is desktop/distro-specific stuff (which should be obvious ;-) ).

I will extend this FAQ, if I feel the need for it, so this article might change a bit.

November 01, 2013

Do you recall that “Qt everywhere” branding?

Well that’s one of the things I really like in Qt, being able to deploy everywhere with minimal to zero changes in source code. But sometimes your application needs a web interface some sort of web backend or web admin, and when that happens it breaks that slogan (AFAIK)…

I got in contact with web languages (not HTML) only when at university, ColdFusion and PHP where the first ones, and while PHP was sort of cool because it has a syntax close to C, I never really like it. I’m not a web designer nor know how to do pretty websites, but in today’s world building web apps is inevitable.

So years ago I learned about RubyOnRails, and people where talking all the good stuff about it, it’s was the framework of the hype, although I knew Ruby wasn’t the fastest scripted language my need didn’t demand that, so I got a book, read it cover to cover and happily started my web app, a week latter I got a complex problem with an apparently impossible solution. It seemed that RoR makes easy things stupidly easy and complex things impossible…

The I met Catalyst a Perl framework, got exited again, but I knew nothing about Perl so I read more 3 books and was started to love the language, I still do, but last weekend when trying to resurrect and old Catalyst application I got an error which I had no idea how to fix, on IRC I was told the problem could be that I mixed CPAN packages with distro packages, not very convinced I realized the bigger problem I was actually facing, I’m not fluent in Perl, well no matter how many books you read about a language you only get fluency in it after speaking it almost everyday, this applies to computing languages too of course.

So if 99% of the time (or less now with QML) I’m programming C++ it means that I know a lot more about how to debug it than with Perl, so Cutelyst! Writting web apps in Qt/C++ will be much more efficient to me and hopefully to you too.

I share lots of the Catalyst API and design ideas, tho the implementation is of course completely different. Also for the sake of HTML templating I plan to create a Grantlee View plugin.

You can take a look at the proof of concept:

Have fun.

October 28, 2013

Long time with no article about Tanglu! This was mainly because we were busy with the project itself, improving various aspects of the distribution.

So, here is a new summary on what has been done and what you can expect from the first release ;-)

Tanglu QA

We further improved the automatic archive QA. There is now, which constantly monitors the number of uninstallable or unbuildable packages in the Tanglu suites. It also provides tanglu-platform-smallstatus information on the metapackage generator, which helps us in finding out which packages are available on the live-cds. Furthermore, information about the staging->devel migration process is provided, to answer the question why a package does not migrate (this still needs some improvements, but it is being worked on).

We also use some code from Ubuntu to monitor package versions in Debian and upstream, which helps to see if others have released newer versions of software shipped with Tanglu.

This already resulted in many improvements: The Tanglu Aequorea suite does not contain unbuildable packages (at least not due to build-dependency changed), and all live-cds are working well.

We will soon migrate the archive to a new server, which frees some server capacities we can use for automated QA and things like automatic live-CD building.

Live-CDs, Installer, Alpha-Releases

We currently don’t do Alpha-Releases of Tanglu, but we create Live-CD snapshots of Tanglu, which are available at (or mirror1, mirror2). These snapshots still have issues and are just early previews. They also ship without and installer, we are still working on that part. Please note that “CD” more or less means DVD or USB-Stick right now (and this won’t change – the expected image size will be around 800MB).

Release Planning

I am happy to announce that we will do a release this year, most likely in December. But what can you expect from the release?

KDE 4.11kde-konqui

We will ship with KDE 4.11, which will be the only desktop we officially support so far. The reason is simply lack of manpower – we could promise to support more, but that would just be not realistic for the small team. So we focus on KDE (Plasma Shells) right now, and try to make it awesome. Also, the team consists mostly of KDE people right now, which contributed to that decision ;-).

If you want to try Tanglu, right now the KDE live-images are the best to try it out.


gnome-logoWe will also provide images with GNOME. The problem with GNOME is, that the GNOME team does not have enough manpower to maintain the whole desktop or to upgrade it to the latest version (it is essentially just me looking at it from time to time). So GNOME will be available in a “preview” state. We invite everyone with GNOME knowledge to join the project and help improving Tanglu-GNOME – GNOMErs, we want you!

systemd >= 204

We ship with systemd by default, which works nicely so far, although more testing needs to be done. The logind service will be used to replace ConsoleKit, if we manage to get everything in place and working in time (if there are issues, we might switch back to CK for one release). There are some plans to use a higher systemd version, due to some improvements made there, but if this will be done is still unclear (Debian will most likely stick to 204 for some time, because with systemd > 205, running it as pid 1 will be mandatory to use logind (and Debian is just in the process to decide which init-system we will use there)).

Systemd will run in SysVInit compatibility mode for most of the available services. This will improve in later Tanglu releases. Of course, systemd is usable, even if not every init-script has been converted to a service file. It just has an impact on startup times, so Tanglu will not be the distributions with the fastest startup times (yet ;-) ).

Debian Package Mixdebian_logo1

Tanglu consists mostly of packages from Debian Testing (Jessie), but we take full advantage of the Debian archive, so you will also find package versions from Unstable or even Experimental (where it made sense). A very small portion of packages has also been merged with Ubuntu. Although stuff has been changed, the incompatibilities with Debian are almost zero, so if you are installing Tanglu, it will currently feel like an more up-to-date Debian Testing with some extras.

Still, the differences are large enough that upgrading a Debian system to Tanglu might result in some issues.


Right now, the installer is a major field of work. Tanglu will most likely ship with the Debian-Installer, because it is the easiest thing to do right now.

For later releases, it is planned to also offer the Ubiquity installer (the thing Ubuntu uses), or a new installer with a similar UI and concept.

Other Cool Things266px-Wayland_Logo

Tanglu will ship with a fully working Qt5 (which is currently being tested and updated) and the latest version of Wayland/Weston as a preview for developers to play around with.

We also ship with Perl 5.18 and Haskell GHC 7.6, as well as with GCC 4.8 as default compiler (although the whole distribution does not yet compile with GCC 4.8). We might ship with Linux 3.12, but this also depends on other things. The Linux kernel build will be the same as in Debian. There might be more things to come :-)

Please do always keep in mind that Tanglu is a new project, so the first release might still have some teething problems – but we’re working on it. The Aequorea release will be supported one to two more months after the next release is out.

Tanglu Policy

I started to draft a Tanglu policy, which defines stuff like the procedures to become a Tanglu project member and/or developer, some basic values of the distribution, decision making processes etc. This work is at a very early stage right now, and will need lots of feedback later. But as soon as it is done, joining the project will be easier and what Tanglu is will be well-defined.

The policy will also include a Code-of-Conduct as an additional document from the start.

We need help! ;-)

First of all, thanks to everyone working on Tanglu! You are amazing! Also, many thanks to every Debian developer or contributor who helped a lot in setting the project up and contributed knowledge. And of course, thanks to everyone else who contributed by creating awesome artwork, helped with code, TangluTanglu Logo (big) archive mirrors, buildd-servers or by testing the distribution and providing feedback! (given the state the aequorea suite was in at the beginning, testing was a really frustrating activity – but people still did it!)

So, if you would like to help, just find us on #tanglu-devel on Freenode or join the tg-devel mailinglist. We also really need some people taking care of the Tanglu website and updating it with some recent information from time to time, so people can see what is going on. Before we have the Tanglu policy finished and the Tanglu members and developers directory in place (a software which allows us to track all registered developers and gives them an mailalias), the start might be a bit confusing, but we do our best to make it easy for new people to join. The best way is asking people.

Tanglu is still created by an incredibly small team, which has a large task to accomplish. Help is welcome :-)

August 25, 2013

Tanglu Logo (big)Long time since I published the last update about Tanglu. So here is a short summary about what we did meanwhile (even if you don’t hear that much, there is lots of stuff going on!)

Better QA in the archive

We now use a modified version of Debian’s Britney tool to migrate packages from the newly-created “staging” suite to our current development branch “aequorea”. This ensures that all packages are completely built on all architectures and don’t break other packages.

New uploads and syncs/merges now happen through the staging area and can be tested there as well as being blocked on demand, so our current aequorea development branch stays installable and usable for development. People who want the *very* latest stuff, can embed the staging-sources into their sources.list (but we don’t encourage you to do that).

Improved syncs from Debian

The Synchrotron toolset we use to sync packages with Debian recently gained the ability to sync packages using regular expressions. This makes it possible to sync many packages quickly which match a pattern.

Tons of infrastructure bugfixes

The infrastructure has been tweaked a lot to remove quirks, and it now works quite smoothly. Also, all Debian tools now work flawless in the Tanglu environment.

A few issues are remaining, but nothing really important is affected anymore (and some problems are merely cosmetic).

Long term we plan to replace the Jenkins build-infrastructure with the software which is running Paul Tagliamontes (only the buildd service, the archive will still be managed by dak). This requires lots of work, but will result in software not only usable by Tanglu, but also by Debian itself and everyone who wants a build-service capable of building a Debian-distro.

KDE 4.10 and GNOME 3.8

Tanglu now offers KDE 4.10 by default, a sync of GNOME 3.8 is currently in progress. The packages will be updated depending on our resources and otherwise just be synced from Debian unstable/experimental (and made working for Tanglu).

systemd 204 & libudev1

Tanglu now offers systemd 204 as default init system, and we transitioned the whole distribution to the latest version of udev. This even highlighted a few issues which could be fixed before the latest systemd reached Debian experimental. The udev transition went nicely, and hopefully Debian will fix bug#717418 too, soon, so both distributions run with the same udev version (which obviously makes things easier for Tanglu ^^)


We now have a plymouth-bootscreen and wallpapers and stuff is in progress :-)

Alpha-release & Live-CD…?

This is what we are working on – we have some issues in creating a working live-cd, since live-build does have some issues with our stack. We are currently resolving issues, but because of the lack of manpower, this is progressing slowly (all contributors also work on other FLOSS projects and of course also have their work :P)

As soon as we have working live-media, we can do a first alpha release and offer installation media.


Tanglu is a large task. And we need every help we can get, right now especially technical help from people who can build packages (Debian Developers/Maintainers, Ubuntu developers, etc.) We especially need someone to take care of the live-cd.

But also the website needs some help, and we need more artwork or improved artwork ;-) In general, if you have an idea how to make Tanglu better (of course, something which matches it’s goals :P ) and want to realize it, just come back to us! You can reach us on the mailiglists (tanglu-project for stuff related to the project, tanglu-devel for development of the distro) or in #tanglu / #tanglu-devel (on Freenode).

August 06, 2013

Some days ago I was listening the radio and a famous Brazilian comedian was saying how stupid it is to express you thoughts about things because latter you probably regret what you said, and realize how dumb you are… although I partly agree with him I decided to share this as maybe someone is facing the same problems I’m facing.

I don’t want to spread FUD, so if my tests/conclusion or everything I said is wrong please explain and I’ll update or drop the text entirely :P

Lately I’ve been working a lot on an Qt5/Quick2 application and I decided to share some thoughts about this new piece of tech. The application that I’m working on basically has to index >16k of videos and music files, and show their covers so the user can choose what to listen/see, it’s an app for jukebox machines. Xapian rocks!

First I need to say that I really like Quick2, the name is right about how fast can you write your app,  on Quick2 we have some very cool graphics effects like blur, shadows and others, and those are quite interesting for this kind of app. I can blur the background, draw shadows on the CD covers and so on. The multimedia part is also very easy to use, so how to play a video or music file is not something you worry about (tho I need to find a way to play files that I encrypt stupid recorders).

Most people also probably heard about Scene Graph which is the new rendering of QML 2 which uses OpenGL (EGL iirc), this brings many improvements on performance and quality, and expands the QML2 possibilities.

However maybe you are considering it for your own projects, the QWidgets vs QtQuick is something many developers don’t know the answer yet. To me it’s now a bit more clear on what to choose.

Every now and then I listen people saying that OSX is good because it’s software made for the hardware, and while in parts I agree with that the same is valid for Quick2. Martin recently wrote an interesting blog about “what makes a lightweight desktop lightweight“, which describes the problem of properly describing a technology.

My point is that with Quick2 we need to set a barrier between legacy and supported hardware, to me this is quite complicated because you can have some very powerful CPUs from like 8yo coupled with very slow GPU, and brand new hardware with the opposite configuration, Raspberry PI for example has a rather slow CPU with a great GPU.

So while QWidgets/Quick1 performed quite well on those old machines Quick2 doesn’t. To exemplify I have a computer to test my jukebox application that has a crap VIA graphics with a 3.2 intel D CPU. When I went to deploy/test my super cool and animated application there, I almost cried, after removing blur/shadow/transparency effects I could get a maximun of 10fps on fullscreen with 1024×768. Toggling the composite switch didn’t make any difference (nice work KWin team), and using other Window Managers also didin’t improve the poor experience.

Aaron give me a hint about llvmpipe, which is something I had heard but never truly understand what it was, basically llvmpipe is a software render for this exactly case where the GPU can’t handle the OpenGL instructions, but after some investigation it seems all new distros have it to render on these poor GPUs. Now that I knew about it I went to look at CPU usage and it was pretty high and leaking…

Before you jump on the wrong conclusion that Quick2 is bad please note that for any hardware capable of OpenGL it’s an excellent choice because it can offload the rendering stuff from your CPU to the GPU. And this is where we have a new era of “lightweight” of desktops as they use less CPU.

Something that does bother me is that Plasma 2 will need newer hardware specs, I won’t be able to run KDE SC 5 (or whatever it will be called) on such hardware, I went on thinking that it would be nice to have something like lxde-qt as our QWidget shell and still use Qt5 with our KDE stuff. On the long run I believe we won’t be able to share much of the GUI apps as Quick2 will probably become more and more common, not to mention it’s a nicer way of developing GUI stuff.

And if you wonder what I’m going to do, I not totally sure, but right now I have 2 user interfaces on the same binary, one QWidget based and another Quick2. I thought about going back to Quick1/Qt4, but then I wouldn’t have the cool Quick2 stuff. Thanks to the model based approach I’m sharing most code between the two interfaces. And yes, since QWidget on Qt5 still uses raster rendering it provides a much more responsive experience for the application.

I hope this gives you some clarity if you might be targeting hardware like that, in the future I hope to be able to deploy on Raspberry PIs but the lack of RTC doesn’t make it an option right now.

If you know some magic way to improve FPS please let me know, I’d love to have a single GUI :D

May 29, 2013

For those who doesn’t know print-manager is a project I started back in 2010 (yes it’s 3 years old now!), but only got included into KDE 4.10 (the current stable release). The reason for that is that since it was meant to replace system-config-printer-kde it needed to provide at least the most used features, and that was only possible as the logic to find a good PPD for a given printer was then exposed through DBus, so we could use that but not it’s GUI.

I’m very pleased to say that my expectation of receiving bug reports of missing features, surprised me by being just one of the few bugs it got (sadly it’s not the easiest to one to fix). The number of bugs was also quite low and it’s acceptance quite positive, which helped me managing fixing most of them then in a short time. Currently there are only 2 standing bugs, and none of them are crashes (all were fixed), one is a missing convenience feature and the other is half fixed but I failed to setup a similar environment to figure out what was happening, but this next version will include some debug info to try to figure that out.

What about new stuff?

So CUPS is not a new project that earns features everyday, actually we fear to have more features removed, that being said print-manager already covers most of it’s functionality. One CUPS feature I didn’t make use in 4.10 was the ability to reprint canceled/completed jobs, this feature also only works if the job is preserved which is something that can be disabled on the server. Of course we only enable the button if the job can do that: p-m-queue-reprint Notice the Reprint button, now there’s something even cooler than that, the last column “From Hostname” is also new, but the coolest part of it was that this was actually a feature request from a KDE 3 (kprint) bug, funny enough I fixed it on the exact same day were it completed 10 years :D (I accept cookies…) Next we have a bunch under the hood changed and fixes.

When I first presented the print-manager plasmoid it would have a feature to configure which jobs to show. Having “Active jobs”, “Completed jobs” and “All jobs” ended up becoming an issue, since if the completed job list had like 20 entries the plasmoid was taking several seconds to load, so since the “Active jobs” is always the smallest list I removed the other options to avoid bug reports. Latter Apper gained a plasmoid and I run into the same slowness very quickly, actually a list of 200 updates (which is not uncommon) was taking half a minute to load.

My previous investigations showed the Plasma DataEngines were too slow, in fact I believe it’s the mapping between the DataEngine and QML since plasmaengineexplorer is not _that_ slow. This was a no go for Apper, so I created a hybird C++/QML plasmoid, and quickly I noticed that this would also be the best thing for printer manager. It’s not only is blazing fast now, but also has the important benefit of not keeping four models implementations as the QWidgets dialogs already had them, so now I fix one model and both plasmoid/KCM/print-queue get it. p-m-plasmoid-configIf you have used the 4.10 version you will also notice an important improvement on the above, instead of a weird LineEdit to select which printers the plasmoid will display, now you have a nice checkable list.

Bonus points to Kai Uwe Broulik that added the second icon there (and did some more stuff) which is the full System Settings printers module so you don’t need to open system settings. And yes I’d like to have a different icon for the first option but I failed to find the option (if it even exists) to do that as Plasma uses the plasmoid icon for the first item. The internal library that talks to CUPS also got several improvements and fixes which made it Qt5 safe, due to QHash getting items stored randomly (yes Qt4 says we can’t rely on the order but we know we can :P). To be honest I don’t really know how well this worked with the mess it was… The plasmoid also got several improvements:

  • It’s fully usable from the keyboard now
  • The highlights/states work more reliable
It isn't empty anymore if no printer is configured

It isn’t empty anymore if no printer is configured

You get this if you filter some printers out and they get removed

It detects if CUPS has stopped and show a proper error message

It detects if CUPS has stopped and show a proper error message

And finally the plasmoid full of printers and a few jobs, with the new NIHSwitch, fully draggable and with I/O visual to avoid confusing, done my real world testing and so far nobody got confused:

print-manager plasmoid in KDE SC 4.11

print-manager plasmoid in KDE SC 4.11

Until KDE Frameworks 5 isn’t released (and probably also packaged), the development of print-manager will continue in the next SC 4.x releases, of course the list of TODOs is quite small and if you are willing to give a hand send me an email.


May 12, 2013

I was asked by some people to write a status report about the whole PK/AS/LI stuff – sorry guys that it took so much time to write it ;-).


(PackageKit is an abstraction layer for package-management systems, allowing applications to access the package-manager using simple DBus calls)Package (with list)

PackageKit is an incredibly successful project. With the 0.8.x series, it received many performance improvements, and has now the same speed on my computer than the distribution’s native tools. PackageKit is used in almost all major Linux distributions, except for Ubuntu[1]. But even Ubuntu has written some compatibility layer, so most calls to PackageKit will work.

The only major distro where PackageKit is currently not available, seems to be Gentoo (and I am not sure about the shape of the Gentoo PackageKit backend too).

Debian Wheezy includes PackageKit by default, and in Jessy we are going to replace some distribution-specific tools with PackageKit frontends (mostly the old and unmaintained update-notifier and Software-Updater – no worries, we are not going for a Synaptic replacement ;-) (currently this won’t be possible with PK anyway))

Unfortunately, some PackageKit backends are still not adjusted for the 0.8.x API and are only running on 0.7.x. This is bad, since 0.8.x is a huge step forward for PackageKit. But the situation is slowly improving, with the latest OpenSUSE release, the Zypper backend is now available on 0.8.x too.

Being able to run a PackageKit from the 0.8.x series is a requirement for both AppStream and Listaller.


(AppStream is a cross-distro effort for building Software-Center-like Software Center Logoapplications. It contains stuff like a screenshot-service, ratings&reviews etc. The most important component is a Xapian database, storing information about all available applications in the distribution’s repositories. The Xapian DB is distro-agnostik, but distributors need to provide data to fill it. AppStream offers an application-centric way to look at a package database)

The AppStream side doesn’t look incredibly great, but the situation is improving. As far as I know, OpenSUSE is shipping AppStream XML to generate the database. Ubuntu ships the desktop-files, and I am working on AppStream support in Debian’s Archive Kit. On the Fedora side, negotiations with the infrastructure-team are still going on. I haven’t heard anything from Mageia and other AppStream participants yet.

Unfortunately, at least for OpenSUSE, the AppStream efforts seem to be stalled, due to people having moved to different tasks. But efforts to add the remaining missing things exist.

On the software side, Apper (KDE PackageKit frontend) has full support for AppStream. Apper just needs to be compiled with some extra flags to make it use the AppStream database.

On the GNOME-side, GNOME-Software is being developed. The tool will make use of the AppStream database, on distributions where it is available.

Also, a Software-Center for Elementary and other GTK+-based desktops is being developed, which is based on AppStream (already quite usable!).

Using the Ubuntu Software Center on not-Ubuntu-based distributions ist still not much fun, but with the AppStream database available and a working PackageKit 0.8.x with a backend which supports parallel transactions, it is possible to use it.

On the infrastructure side: I recently landed some patches in AppStream-Core, which will improve the search function a lot. AppStream-Core contains all tools necessary to generate the AppStream database. It also contains libappstream, a GObject-based library which can be used to access the AppStream database.

Also, we discuss dropping PackageKit’s internal desktop-file-cache in favour of using the AppStream database. If we do that, we will also add software descriptions to the AppStream db, to improve search results and to speed up search for applications. Because we have to deprecate API for that, I expect this change to happen with PackageKit 0.9.x.

As soon as the Freedesktop-Wiki is alive again and my account is re-enabled, I will create compatibility-list, showing which distribution implements what of the PK/AS/LI stuff, especially focusing on components needed for AppStream.

Only a few distributions package AppStream-Core so far. Although it is beta-software, creating packages for it and shipping the required data to generate the AppStream database would be a very huge step forward.


(Listaller is a cross-distro 3rd-party software installer, which integrates into Listaller-LogoPackageKit and AppStream. It allows installing 3rd-party applications, which are not part of the distributor’s repositories, using standard tools used also for native-package handling. Everything which uses PackageKit can make use of Listaller packages too. Listaller also allows sandboxing of new applications, and uses an AppDir-like approach for installing software.)

Listaller is currently undergoing it’s last transition before a release with stable API and specifications can be made. Dependency solving will be improved a lot during the current release-cycle, making it less powerful, but working on all distributions instead. (Fedora always had an advantage in dependency-solving, due to RPM providing more package metadata for Listaller to use) This change was delayed due to discussing a possible use of ZeroInstall-feeds to provide missing dependencies with the ZeroInstall team. We did not come to a conclusion about extending the XML, so Listaller will contain an own format to define a dependency, which can reference a ZeroInstall feed. That should be a good solution for everyone.

All these changes will result in IPK1.2, a new version of the IPK spec with small changes in the pkoptions file syntax and huge changes in dependency-handling. The new code is slowly stabilizing in a separate branch, and will soon be merged into master.

The next Listaller release will be the last one of the 0.5.x series, we will start 0.6.x then. KDE currently has support for Listaller through Apper, which is enabled on a few distributions. In GNOME, optional Listaller support is being developed and will be available in one of the upcoming releases.

Currently, to my knowledge, only a few distributions package Listaller. This should improve, so it is easier for application developers to deploy IPK packages.

The upcoming changes in KDE and GNOME to build stable developer platforms will help Listaller a lot in finding matching dependencies, and for stuff which only depends on one software frameworks, installations should be a matter of seconds.

As you can see, lots of things are happening, and there is improvement in all components related to installing and presenting software on Linux machines. However, all these projects have a severe lack of manpower, especially AppStream and Listaller have the lowest number of developers working on the tools (at time, only two active developers). This is the main reason for the slow development. But I am confident that we will have something shipped in the next distribution releases. At least AppStream should be ready then.

[1]: I don’t blame Ubuntu for that – during the time they wrote an own solution, PackageKit did not have all the required features. (This situation has changed now, fortunately.)

NOTE: I might extend this post with feedback from the different distributions, as soon as I get it.

April 26, 2013


this is short news mostly for packagers, but well I want to thank José Manuel Santamaría Lema which did most (if not all) of the changes in this release, he found a few bugs and extended the debconf protocol support. I didn’t manage to release this soon as I’m way too busy. His work was made when I got arrested in Germany last year so I’m quite late here…


Next week I’ll probably do a new colord-kde release stay tunned!

April 08, 2013

Hello everyone! I am very excited to report about the awesome progress we made with Tanglu, the new Debian-based Linux-Distribution. :)

First of all, some off-topic info: I don’t feel comfortable with posting too much Tanglu stuff to Planet-KDE, as this is often not KDE-related. So, in future Planet-KDE won’t get Tanglu information, unless it is KDE-related ;-) You might want to take a look at Planet-Tanglu for (much) more information.

So, what happened during the last weeks? Because I haven’t had lectures, I nearly worked full-time on Tanglu, setting up most of the infrastructure we need. (this will change with the next week, where I have lectures again, and I also have work to do on other projects, not just Tanglu ^^) Also, we already have an awesome community of translators, designers and developers. Thanks to them, the Tanglu-Website is now translated to 6 languages, more are in the pipeline and will be merged later. Also, a new website based on the Django framework is in progress.

The Logo-Contest

We’ve run a logo-contest, to find a new and official Tanglu logo, as the previous logo draft was too close to the Debian logo (I asked the trademark people at Debian). More than 30 valid votes (you had to be subscribed to a Tanglu Mailinglist) were received for 7 logo proposals, and we now have a final logo:

Tanglu Logo (Text)

I like it very much :)

Fun with dak

I decided to use dak, the Debian Archive Kit, to handle the Tanglu archive. Choosing dak over smaller and easy-to-use solutions had multiple reasons, the main reason is that dak is way more flexible than the smaller solution (like reprepro or min-dak) and able to handle the large archive of Tanglu. Also, dak is lightning fast. And I would have been involved with dak sooner or later anyway, because I will implement the DEP-11 extension to the Debian Archive later (making the archive application-friendly).

duckWorking with dak is not exactly fun. The documentation is not that awesome, and dak contains many hardcoded stuff for Debian, e.g. it often expects the “unstable” suite to be present. Also, running dak on Debian Wheezy turned out to be a problem, as the Python module apt_pkg changed the API and dak naturally had problems with that. But with the great help of some Debian ftpmasters (many thanks to that!), dak is now working for Tanglu, managing the whole archive. There are still some quirks which need to be fixed, but the archive is in an usable state, accepting and integrating packages.

The work on dak is also great for Debian: I resolved many issues with non-Debian dak installations, and made many parts of dak Wheezy-proof. Also, I added a few functions which might also be useful for Debian itself. All patches have of course been submitted to upstream-dak. ;-)

Wanna-build and buildds

GearThis is also nearly finished :) Wanna-build, the software which manages all buildds for an archive, is a bit complicated to use. I still have some issues with it, but it does it’s job so far. (need to talk to the Debian wanna-build admins for help, e.g. wanna-build seems to be unable to handle arch:all-only packages, also, build logs are only submitted in parts)

The status of Tanglu builds can be viewed at the provisoric Buildd-Status pages.

Setting up a working buildd is also a tricky thing, it involves patching sbuild to escape bug #696841 and applying various workarounds to make the buildd work and upload packages correctly. I will write instructions how to set-up and maintain a buildd soon. At time, we have only one i386 buildd up and running, but more servers (in numbers: 3) are prepared and need to be turned into buildds.

After working on Wanna-build and dak, I fully understand why Canonical developed Launchpad and it’s Soyuz module for Ubuntu. But I think we might be able to achieve something similar, using just the tools Debian already uses (maybe a little less confortable than LP, but setting up an own LP instance would have been much more trouble).

Debian archive import

The import of packages from the Debian archive has finished. Importing the archive resulted in many issues and some odd findings (I didn’t know that there are packages in the archive which didn’t receive an upload since 2004!), but it has finally finished, and the archive is in a consistent state at time. To have a continuous package import from Debian while a distribution is in development, we need some changes to wanna-build, which will hopefully be possible.

Online package search

The Online-Package-Search is (after resolving many issues, who expected that? :P) up and running. You can search for any package there. Some issues are remaining, e.g. the file-contents-listing doesn’t work, and changelog support is broken, but the basic functionality is there.

Tanglu Bugtracker

We now also have a bugtracker which is based on the Trac software. The Tanglu-Bugtracker is automatically synced with the Tanglu archive, meaning that you find all packages in Trac to report bugs against them. The dak will automatically update new packages every day. Trac still needs a few confort-adjustments, e.g. submitting replies via email or tracking package versions.

Tanglu base system

tanglu-platform-smallThe Tanglu metapackages have been published in a first alpha version. We will support GNOME-3 and KDE4, as long as this is possible (= enough people working on the packaging). The Tanglu packages will also depend on systemd, which we will need in GNOME anyway, and which also allows some great new features in KDE.

Side-effect of using systemd is – at least for the start – that Tanglu boots a bit slow, because we haven’t done any systemd adjustments, and because systemd is very old. We will have to wait for the systemd and udev maintainers to merge the packages and release a new version first, before this will improve. (I don’t want to do this downstream in Tanglu, because I don’t know the plans for that at Debian (I only know the information Tollef Fog Heen & Co. provided at FOSDEM))

The community

The community really surprised me! We got an incredible amount of great feedback on Tanglu, and most of the people liked the idea of Tanglu. I think we are one of the less-flamed new distributions ever started ;-). Also, without the very active community, kickstarting Tanglu would not have been possible. My guess was that we might be able to have something running next year. Now, with the community help, I see a chance for a release in October :)

The only thing people complained about was the name of the distribution. And to be really honest, I am not too happy with the name. But finding a name was an incredibe difficult process (finding something all parties liked), and Tanglu was a good compromise. “Tanglu” has absolutely no meaning, it was taken because it sounded interesting. The name was created by combining the Brazilian “Tangerine” (Clementine) and the German “Iglu” (igloo). I also dont think the name matters that much, and I am more interested in the system itself than the name of it. Also, companies produce a lot of incredibly weird names, Tanglu is relatively harmless compared to that ;-).

In general, thanks to everyone participating in Tanglu! You are driving the project forward!

The first (planned) release

I hereby announce the name of the first Tanglu release, 1.1 “Aequorea Victoria“. It is Daniel’s fault that Tanglu releases will be named after jellyfishes, you can ask him why if you want ;-) I picked Aequorea, because this kind of jellyfish was particularly important for research in molecular biology. The GFP protein, a green fluorescent protein (=GFP), caused a small revolution in science and resulted in a Nobel Prize in 2008 for the researchers involved in GFP research (for the interested people: You can tag proteins with GFP and determine their position using light microscopy. GFP also made many other fancy new lab methods possible).

Because Tanglu itself is more or less experimental at time, I found the connection to research just right for the very first release. We don’t have a time yet when this version will be officially released, but I expect it to be in October, if the development speed increases a little and more developers get interested and work on it.

Project Policy

We will need to formalize the Tanglu project policy soon, both the technical and the social policies. In general, regarding free software and technical aspects, we strictly adhere to the Dbian Free Software Guidelines, the Debian Social Contract and the Debian Policy. Some extra stuff will be written later, please be patient!

Tanglu OIN membership

I was approached by the Open Invention Network, to join it as member. In general, I don’t have objections to do that, because it will benefit Tanglu. However, the OIN has a very tolerant public stance on software patents, which I don’t like that much. Debian did not join the OIN for this reason. For Tanglu, I think we could still join the OIN without someone thinking that we support the stance on software patents. Joining would simply be pragmatic: We support the OIN as a way to protect the Linux ecosystem from software patents, even if we don’t like the stance on software patents and see it differently.

Because this affects the whole Tanglu project, I don’t want to decide this alone, but get some feedback from the Tanglu community before making a decision.

Can I install Tanglu now?

Yes and no. We don’t provide installation images yet, so trying Tanglu is a difficult task (you need to install Debian and then upgrade it to Tanglu) – if you want to experiment with it, I recomment trying Tanglu in a VM.

I want to help!

Great, then please catch one of us on IRC or subscribe to the mailinglists. The best thing is not to ask for work, but suggest something you want to do, others will then tell you if that is possible and maybe help with the task.

Packages can for now only be uploaded by Debian Developers, Ubuntu Developers or Debian Maintainers who contacted me directly and whose keys have been verified. This will be changed later, but at the current state of the Tanglu archive (= less safety checks for packages), I only want people to upload stuff who definitely have the knowledge to create sane packages (you can also proove that otherwise, of course). We will later establish a new-member process.

If you want to provide a Tanglu archive mirror, we would be very happy, so that the main server doesn’t have to carry all the load.

If you have experience in creating Linux Live-CDs or have worked with the Ubiquity installer, helping with these parts would be awesome!

Unfortunately, we cannot reuse parts of Linux Mint Debian, because many of their packages don’t build from source and are repackaged binaries, which is a no-go for the Tanglu main archive.

Sneak peek

And here is a screenshot of the very first Tanglu installation (currently more Debian than Tanglu):


Something else…

I am involved in Debian for a very long time now, first as Debian Maintainer and then as Debian Developer – and I never thought much about the work the Debian system administrators do. I didn’t know how dak worked or how Wanna-build handles the buildds and what exactly the ftpmasters have to do. By not knowing, I mean I knew the very basic theory and what these people do. But this is something different than experiencing how much work setting up and maintaining the infrastructure is and what an awesome job the people do for Debian, keeping it all up and running and secure! Kudos for that, to all people maintaining Debian infrastructure! You rock! (And I will never ever complain about slow buildds or packages which stay in NEW for too long ;-) )

April 01, 2013
I am just considering some modifications to the first round of sketches, the original post is here.

Different color:

Different font:

Playing nice with other shapes (examples):

March 26, 2013
After several attempts I am satisfied with this one (for now), so below is my proposal for the Tanglu logo.
Note: the text part is just for positioning reference, used just plain Vera Sans font but think the font part needs some care.

Variations of the above one and other possible uses:

A shortened "making of" and some draft versions:

Link to the post on the tanglu-devel mailing list with more description about the logo.

Download the above images in editable SVG format:

Creative Commons License
Tanglu logo proposal by Ink Apnea is licensed under a Creative Commons Attribution-ShareAlike 3.0 Unported License.
March 21, 2013
If not, read this. This sounds like it's going to be an awesome project, and I'm going to contribute in any way I can.

To quote some highlights from the initial announcement:
Tanglu will be based on Debian Testing and follow the Debian development closely. It will have a 6-months release-cycle and it’s target audience are Linux desktop users. We will make installing and setting up the distro as easy as possible. 
Tanglu will be usable for both developers of upstream software and the average Linux user and Linux newbie. This is possible because in our opinion developers and users don’t have different needs for a desktop system. Both kinds of users like a polished desktop which “just works”. We will, hwever, not apply any kind of fancy modification on upstream software, we will basically just distribute what upstream created, so users can get an almost “pure” GNOME and KDE experience.
Tanglu is designed to be able to solve the issue that Debian is frozen for a long time and Debian Developers can’t make new upstream versions available for testing easily. During a Debian freeze, DDs can upload their software to the current Tanglu development version and later start the new Debian cycle with already tested packages from Tanglu. The delta between Tanglu and Debian should be kept as minimal as possible. However, Tanglu is not meant as experimental distribution for Debian, so please upload experimental stuff to Experimental. Only packages good enough for a release should go into Tanglu.
And the best part (to me, anyway :-) ):

Which desktop will you use? 
Everyone can add a new desktop to Tanglu, as long as the desktop-environment is present in Debian. Long term, we will have to offer Linux-newbies a default flavour, probably by setting a default download on the website. But as long as there is a community for a given desktop-environment, the desktop is considered as supported.
At the beginning, we will focus on KDE, as many people have experience with it. But adding vanilla GNOME is planned too.

Yay for another option for a Debian based, pure KDE distro. I'm running both openSUSE and Kubuntu now on different machines. I'm surely more comfortable with Debian, but Kubuntu is simply more up-to-date, and KDE is really well integrated. I'm trying openSUSE, but having used Debian based distros for going on 10 years now, I still don't have the level of comfort, so it lives on my (less critical) Laptop.

I see this as great because it will help the overall Debian ecosystem, which feeds not only Debian, but The 'buntu's, Mint CrunchBang and many others.

I just think it's awesome to have something with a pure KDE more closely aligned with Debian coming soon.

Does this all sound as exciting to you as it does to me? Because, man, I'm pumped!

If so, do like I did and sign up for the mailing list, join the Freenode channel and get ready to contribute.
March 14, 2013

Hi everyone!

Today I make an announcement I thought I would never ever make. But things changed.

Discussion about this has a long history, starting as a non-serious suggestion at DesktopSummit 2011, continued with people on IRC, but it was decided back then that it wouldn’t be worth the effort. This has changed too, and a small team has formed to work on it.

We hereby announce Tanglu, a new Debian-based-Linux distribution.

A new logo?Why do we need another one? Let me explain the concepts of that distro:

Tanglu will be based on Debian Testing and follow the Debian development closely. It will have a 6-months release-cycle and it’s target audience are Linux desktop users. We will make installing and setting up the distro as easy as possible.

Tanglu will be usable for both developers of upstream software and the average Linux user and Linux newbie. This is possible because in our opinion developers and users don’t have different needs for a desktop system. Both kinds of users like a polished desktop which “just works”. We will, hwever, not apply any kind of fancy modification on upstream software, we will basically just distribute what upstream created, so users can get an almost “pure” GNOME and KDE experience.

Tanglu is designed to be able to solve the issue that Debian is frozen for a long time and Debian Developers can’t make new upstream versions available for testing easily. During a Debian freeze, DDs can upload their software to the current Tanglu development version and later start the new Debian cycle with already tested packages from Tanglu. The delta between Tanglu and Debian should be kept as minimal as possible. However, Tanglu is not meant as experimental distribution for Debian, so please upload experimental stuff to Experimental. Only packages good enough for a release should go into Tanglu.

Ideally, Tanglu and Debian should be working well together in mixed environments, where you for example have Debian servers and multiple Tanglu desktops with the new software, targeted at desktop user. Since the differences between Tanglu and Debian should not be very high, administering both systems should be very easy (if you know Debian).

Tanglu will be an open project, driven by community. At the beginning of each cycle, people can make suggestions for release goals they want to implement (similar to Fedora, but without FESCo). These proposals are discussed in public and are rejected if there are major technical concerns. If consensus about a certain proposal is lacking, a vote is done for it. The proposal can be accepted with absolute majority. If this does not happen, the proposal is postponed for the next release, where people can vote for it again. If nobody wants that function, it is rejected. In general, decisions made by Debian are superior and have to be followed.

We don’t think we know every package and every software better than the original upstream. That’s why it makes much sense to rely on feedback from others and to have a community-based and peer-reviewed distribution, instead of secretly developing stuff and dumping it on the community. Tanglu will have a highly predictable set of features, defined at the beginning of a cycle, so you will know what you can expect from the next release as soon as possible and plan for it.

Tanglu will make it easy to deploy applications for it. It will contain a software-center, similar to what Ubuntu has. We will also try to establish a solution for a “Linux-AppCenter”, a place for Linux applications, which will be open not only for Tanglu, but can be implemented in any other distribution too. Possible income will flow back into development of the platform.

Now, let’s answer the FQA (Future Questions Asked):

Why don’t you contribute to Debian directly and create yet another distribution?

First of all, we contribute to Debian ;-) And for me, I can say that I will contribute to Debian even more. The point is that Debian can not cover all possible use-cases, and with Tanglu we want to make a distro which solves this. You might ask why we have to create a new distro for that, instead of creating improvements inside Debian? Creating a new distro allows us to do stuff we can never do in Debian. For example, we will include proprietary firmware in that distro, we will make installations of proprietary stuff possible easily (but don’t ship with it by default) and we will have a time-based release cycle. These are already things which are a no-go for Debian, and that’s fine. We don’t want Debian to support these cases, as it is already a great distribution. We want to offer a distro as close to Debian as possible, but with a few modifications for use-cases which are not covered by Debian itself. Of course we will participate in DEX.

If Debian Developers contribute to Tanglu, freezes will take even longer!

This is an often-heard concern, it comes up on every mailinglist discussion about continuing development while freeze. I would disagree here, packaging new upstream stuff is not slowing down testing and improving of packages in Testing. Also, Tanglu is an offer for Debian developers to participate (we will sync privileges for their packages) – we don’t expect anyone to work on it, but as we think DDs know their packages best, we will make it possible for them to participate without extra barriers. We hope that Tanglu can add value to Debian and that Debian cycles can start with better-tested packages.

You said you are a small team – you cannot develop a whole distribution with it!

Let’s put that to the test! ;-) All people working on this are well aware of the issue that the project can not survive without much community-involvement on the long run. But we see a chance that many people are interested in it and that there is a high demand for it.

At the beginning, we will just start with a small set of packages. We will also sync many packages from Ubuntu, to reduce workload. For example, it is planned to use the Ubuntu-Kernel and KDE packaging. By doing this, we keep the workload at the beginning low. We also reduce duplicate work with that.

We even have some possible sponsors for the new distribution. But nothing is set in stone yet, so just wait for it to happen. :)

Why not participate in Arch, OpenSUSE $other_distro?

These are not Debian ;-) . I know, it sounds odd, but if you like the Debian way of doing things, you want to use a Debian-based distribution. There is nothing wrong with OpenSUSE. And Debian has issues too. But we want to be close to Debian and use it’s tools and way of doing things.

I hate you!!! You are doing it wrong!! The project is useless!

Well, that’s fine. But there is no reason for hating us. If you dislike our idea, there are basically two options: First, you hate us but the project is successful. In that case, you have been wrong with hate, as there are definitely people who liked the project and contributed to it. Second, you hate us and we fail. In this case, there is no reason for hate, as the project will just vanish and you don’t have to worry about it. So hating it would’ve been just a big waste of energy.

Also keep in mind that forking is a way to keep development healthy and to adapt software to new use-cases which it didn’t target before. And we are not introducing incompatibilities here (like e.g. writing our own display server could). Instead, we want to stay close to Debian and reuse as much code as possible.

Which desktop will you use?

Everyone can add a new desktop to Tanglu, as long as the desktop-environment is present in Debian. Long term, we will have to offer Linux-newbies a default flavour, probably by setting a default download on the website. But as long as there is a community for a given desktop-environment, the desktop is considered as supported.

At the beginning, we will focus on KDE, as many people have experience with it. But adding vanilla GNOME is planned too.

Can you say something about the software used in Tanglu?

Yes, but this is still in flow, so I can’t promise final decisions here. On the server, side, we will try to use the Dak for repository management, as soon as we have enough server capacity. We will also use the standard Debian repository GUI and basically reuse most of the software there, to diverge less from Debian.

The distribution itself could probably use a Linux Kernel from Ubuntu and systemd as the primary init system, as well as the login manager logind. It will be based on current Debian Testing with some fresh packages from Unstable and Experimental. We might also use the Ubuntu driver packages and KDE packaging. We expect to have a very rough start with the first release, but there will be enough time to polish Tanglu.

UPDATE20140214: Just because this pops up online incredibly often: Tanglu does not and will likely not use an Ubuntu Kernel. Tanglu 1 (Aequorea Victoria) ships with Linux 3.12 derived directly from Debian.

Nice idea! How can I help?

Well, you can help with basically anything at time – from writing manuals, over designing logos and pages to administering a webserver and create packages. We are at an early stage of development at the moment, but we wanted to go public with it as soon as possible, to include the community and receive feedback so we can make that distro community-centric from the beginning. Most of the infrastructure is currently in progress too.

So, if you want to get started with Tanglu, subscribe to our mailinglist tanglu-devel and write a mail to it, intruducing yourself. We can then include you in the loop. Generally, if you want to get access to our machines, a trusted GPG-signature will help a lot.

If you want to talk to us, join #tanglu-devel or Freenode! Most discussions are currently happening there.

And that’s it! Tanglu will be awesome!

Some other projects of mine will develop a bit slower because I am now involved in Tanglu. But nothing will stop, and there is some pretty cool stuff coming for both GNOME and KDE (and I still have to implement DEP-11 for Debian). :)

March 06, 2013

Disclaimer: This post just sums up a concept for a new distribution which matches certain ideals. It is not the announcement of a new distribution. These are just abstract ideas. (However, if there is high interest in a project like this, it might of course develop into something real…)

I have been involved in Debian and Ubuntu for a long time now. When Ubuntu started, I was a Debian Testing user, and I immediately switched to Ubuntu when it started, because I liked the idea of a short-release-cycle, user-centric company-supported Debian based Linux distribution. However, I am now back to Debian for a long time, because of many reasons which nearly all had to do with Canonical policy. But this is not a post to criticise Ubuntu, so I’ll leave out most of that part. I am highly disappointed on how Ubuntu develops – not only the technical decisions are at least questionable, but also the social and community part is not that great anymore. There is a high asymetry in the relation between Canonical and other developers, Ubuntu mailinglists basically don’t create meaningful results, they sometimes even mutate to a Canonical Q/A session. The community does not seem to have a large influence on decisions about core services, and it can’t have it if there are things developed behind closed doors. (This is all, of course my subjective impression)

But really nobody can argue against the basic idea of Ubuntu and the great things Ubuntu created Also, many of the processes Ubuntu uses to develop the distribution are just great and well-working, as well as there is a highly active community around it. As you simply cannot argue with Canonical to change their policy (they are a company and have hidden plans, also they have every right to apply whatever policy they want), the natural way in any OSS project would be to fork it. But doing that blindly would just create another distribution, which would almost certainly vanish again soon, since there are already many Ubuntu derivatives which cover many use-cases using an Ubuntu base.

I discussed this stuff with Daniel some time ago, and we did some kind of brainstorming about what a perfect distribution would look like, from the perspective of a developer who wants to use a Debian-based distribution.

Here is a list of points which would define such a project:

  • Every available package complies with the DFSG and Debian policy.
  • Packages of DISTRO stay in close sync with Debian packages, changes are preferrably applied in Debian. DISTRO might work as a playground for new technology while Debian is in freeze.
  • DISTRO stays as close to upstream as possible. It applies as less patches as possible, to deploy desktop environments which look like the thing upstream intended it to look like. Changes for DISTRO are developed upstream and only applied downstream if doing that doesn’t make sense or changes are distribution-specific and can’t be abstracted.
  • All desktop environments are treated equally. There is no preferred DE.
  • DISTRO stays in sync with release cycles of KDE and GNOME, to provide developers the latest stable development environment and users a recent version of fresh upstream software.
  • DISTRO is user-centric. It tries to make the distribution work on as many hardware as possible and to fix any usability quirks which are found. But work on that should go upstream, of course.
  • At the beginning of a release cycle, packages are merged from Debian and selectively merged from Ubuntu, where it makes sense.
  • All Debian developers automatically have upload access to DISTRO, so they can upload versions of their packages to the cutting-edge DISTRO and also maintain the Debian things.
  • All DISTRO developers can upload any package, but are responsible for everything they might break. Also, changes should be discussed with the original maintainer at Debian – this is the person who knows the package best.
  • DISTRO is not a testing environment for Debian, it is developed like a short-cycle-less-stable-but-usable Debian. DISTRO recommends Debian Stable as “LTS” version.
  • No CLA is enforced on anyone. People are free to contribute, as well as companies. The project structure is meritocratic.
  • Features are discussed in the open. If there is disagreement about the direction of the project or about any technical issue, a public voting on this feature is created, where every project member can vote. The vote result is final (but can be changed in future, of course).
  • Create a well-maintained core, make it possible to install the newest applications on that distribution (by using native packages, Listaller or Glick) Think about rolling releases, where applications are constantly updated and the core stuff is refreshed once a year/every 6 months. With this users have new apps immediately, but don’t live in fear that some core API or a desktop workflow changes immediately. (Also not desired for desktops in schools, universities and companies – maintaining a moving target in these larger environments is a pain)
  • Create an Application-Center and application-ecosystem based on AppStream and Listaller around it. Encourage upstream developers to publish fresh applications there. Make it possible for every distribution to use this solution and help others to adapt it. Don’t try to lock people on that one platform by providing stuff exclusively for DISTRO or making it harder to use it.
  • Encourage using additional commercial stuff like Amazon searches, magazine stores etc., but make these things available on a separate repository and make them entirely opt-in. Never enable stuff like this by default, but add simple instructions how to use these things for people who want to use them.

Debian DevelopmentThis is basically what we would like to have in a new distribution. :) If you take a closer look, you will see that an effort like this would basically create a close-to-upstream, user-centric short-cycle Debian-based and close-to-Debian distribution, which would cover many use cases, including fixing the “use experimental for new packages during freeze” issue at Debian (DISTRO could be used as environment to run cutting-edge technology, which is generally stable enough, but not yet “Debian-Stable”-stable). Something like this does not exist at the moment. If you take a second look at the list above, you will also see that I mixed technical aspects with organizational aspects. This is intentional. This is just brainstorming, because it is good to know what you would like to have, instead of complaing about the status quo of other projects.

But maybe there will be a distribution which matches some of the above points, to create an upstream-friendly entirely community based Ubuntu.

November 29, 2012
Maecenas sed egestas erat. Curabitur suscipit massa varius libero condimentum eget consequat erat elementum. Duis velit nunc, venenatis non consequat in, viverra scelerisque ipsum. Sed eget lectus non justo pharetra ultrices. Maecenas eget mi eget magna mattis pulvinar. Class aptent taciti sociosqu ad litora torquent per conubia nostra, per inceptos himenaeos. Maecenas magna libero, facilisis at hendrerit at, placerat at ipsum. Ut interdum euismod scelerisque. Vivamus sed enim in augue facilisis aliquet. Etiam lectus leo, aliquam vel interdum at, ultricies id dui. Donec quis diam neque. Lorem ipsum dolor sit amet, consectetur adipiscing elit. Fusce aliquet faucibus nibh et pellentesque.
November 12, 2012

“Once again darkened and the sun set in the heart I could not bear, I miss my home, I prayed to God in the affliction He opened my heart was when someone touched me and heard Jesus say this: It’s just a little more, a little more, then I’ll get you there, love is real, there is a river of peace, there is a heaven that God called him home. Again dawned and the sun was born of faith in me and when I think about quitting Jesus says: It’s just a little more, a little more, then I’ll get you there, there’s real love there is a river of peace, there is a heaven that God called home …”
King’s Heralds

Love you so much my husband Daniel Nicoletti, God will get you out there and sure ara sent angels take care of you … I will move heaven and earth,’m going through, I will do everything, would give my life for you … love you so much … miss …

Elisabeth Nicoletti …

November 09, 2012

Hello! This blogpost is personal and not technology-related at all, but I would appreciate it if you would read it anyway, in this case. Generally, I don’t like sending off-topic posts to the Planets, but this is an extreme situation and I feel it is important that you know what has happened.

First of all, who is Daniel Nicoletti? He is a KDE and Freedesktop developer, the author of many great things like Apper, the PackageKit-based KDE package manager, a new KDE Print Manager, colord-kde and the Aptcc backend PackageKit uses on Debian-based systems. Also, he is a friend of mine and we work on many projects together.

Recently, he was traveling to the Linux Color Management Hackfest 2012, but he never reached Brno, instead he was arrested by Interpol at the airport in Munich.

The reason is a tragic accident about a year ago. If you are a KDE developer/user, you might know the story already. Daniel and his family wanted to move to Argentina. On the way, a car suddenly stopped on the road. To avoid a collision, Daniel tried to dodge the car, which then collided with a truck. His daughter died later in cause of the accident. The KDE release 4.6.3 was dedicated in memory of her.

After these events, the family decided to move back to Brazil for various reasons. Meanwhile – and without Daniel knowing anything about it – Argentina filed an Interpol request to get him, accusing him for murdering his daughter. (As – of course – he was no longer in Argentina)

At time, he is arrested in Munich, and Argentina asks for his extradition, but Germany does not totally agree with his guilt.

Thanks to our really awesome community (really, I can’t express how impressed and proud I am) he received some help quickly, after his wife posted a desperate request for help via his Google+ account. – At first, she didn’t even know what exactly had happened, because no communication was allowed to Daniel (and still is not), and we all assumed an accident.

Now he has a lawyer who is communicating with his wife. In the worst case, Daniel will be struck for at least another 6 months, unless his wife can come over and intercede for him.

This surreal story really came out of nowhere. The day before he left for the hackfest, I was reviewing a patch with him and everything was fine, and he was excited to go to the CM Hackfest.

At time, his family is collecting money so his wife is able to go to Munich and clear the situation, because due to these incidents, there is not much money left. So, if you want you can donate for him.
Click here to lend your support to:  URGENT flights to go to the Brazilian Consulate in Munich and make a donation at !

I really appreciate any help to get Daniel back and clear this injustice, and I wish him and his family the all the best! And I hope that this nightmare will end quickly.

Thank you for reading through this…

UPDATE: Daniel’s wife will soon travel to Germany with an one-way ticket. She will meet a member of the community at the airport who can show her around and help in communicating with the locals. (And offer her a place to stay) At time, they seek a lawyer who is specialized in this international affairs. (Brazilian citizen, accused in Argentina, currently sitting in Germany…)

Because some people asked this, you can find the whole story about the accident on Daniel’s Blog. Also, a local newspaper reported about it. (But my Portuguese or Spanish is as good as the Google Translator allows it) Also, as far as I know communication between the Brazilian side and Germany is going on.

Currently, nobody can speak to Daniel. I’ll keep you updated.

Also, for Brazilians: You can also donate here, if Pledgie does not work for you. (requested some times) – I just hope they will get the money even if goals are not reached, I don’t know the rules these platforms apply.

October 04, 2012

Many awesome new releases have been made this week so far!

PackageKit 0.8.4

The new PackageKit features lots of bugfixes and small improvements. It also contains a few new features, like showing PolicyKit dialogs on the command-line, even if no GUI is running.

If you want to try the new PackageKit and are a Debian user, you can install it from Debian Experimental. (Packages are available) The next Apper release will support this PackageKit series too.

Listaller 0.5.6

The “we-break-everything” Listaller release. Contains a finalized IPK1.1 spec which is frozen now, so if you create packages, you can be sure future Listaller releases will support them too. Also the Listaller API was refreshed so it is much easier to use.

This new release does not include the promised support for IPK-repositories/software-updates, this feature was not ready for 0.5.6, but it might come with 0.5.7 – then it will also include deltaIPKs, one of the most-requested features by game developers (and probably LibreOffice users :P).

Instead the new Listaller contains rewritten support for GPG-key handling, giving distributors and users full control of the GPG public keys Listaller trusts. Also, Listaller won’t install packages with security level “Low” anymore, by default. Of course you can change this setting, if you want.

The documentation of Listaller is currently rewritten, so we can kill the Wiki, so you will soon have excellent documentation of Listaller’s features. Also, lots of other improvements happened, check the release notes for details.

The new Listaller requires at least PackageKit 0.8.4, because it requires some changes done in this PackageKit version.

AppStream-Core 0.1

I am really excited about this one! AppStream-Core is a collection of tools I started writing during my SoC which allows creating & accessing the AppStream database. In combination with PackageKit, AppStream-Core offers a nice way to build Software-Center-like applications. AppStream-Core is also a central place where all changes for AppStream support can be implemented in a distro-agnostic way. If your distribution wants to support software-centers, it should ship this collection of tools. (of course, AppStream data still needs to be provided by distribution-specific tools)

The 0.1 release is not meant as a stable release, it is only something you can play with at time. Right after 0.1 was released, I already did some important changes in the master branch, so the whole project is moving at time, and there is no API-stability-promise yet. But of course I suggest everyone to test the project ;-)

AppStream-Core is a prerequisite to make stuff like application-display in KDE’s Apper work or to help GNOME implementing a tool like GNOME-Software. (it is also recommended to run the USC on other distros)

September 03, 2012

Nearly everyone who’s reading this blog will know about Listaller, a project started by me to make cross-distro 3rd-party software installations possible & secure.

This project has been started years ago, and lots of things have changed in the Linux desktop world and down the stack, so I think it is time to look back now and summarize what has happened – and also to finally write down my evil plan about what Listaller should achieve. People who know me and who I’m working with already know what I would like to achieve and pushed me a little to write this “vision” down.

I’m not a big fan of project “visions”, but in this case to define a final goal visible to everyone (and not just me) will help a lot.

So, if you want to read something about the state of the Linux desktop and my view on the recent “Linux desktop has failed” discussion as well as lots of history and future visions of the Listaller Project, keep on reading! (If not, go here and enjoy the day ;-))

The Past

When Listaller was started in December 2007 (I guess, I don’t have an exact date), software management on Linux distributions was in a very sad state. It was usual to manage software in distributions using a native package manager, like Synaptic or YaST. Also, extending the software repository was not really easy. Ubuntu did it’s first experiments with an application-centric software manager that time and PPAs weren’t that common. (I don’t know if Launchpad already offered this service back then – I think it was implemented later)

Regarding cross-distro 3rd-party software installing, there were a few competing projects like LOKI/MoJo, Autopackage, ZeroInstall, Klik, etc. which all weren’t that common and only a few people used that software. Also, they didn’t solve the main issue: Packages aren’t user-friendly! A “package” is a somewhat abstract concept which is hard to understand for non-technical users. These projects only focused on installing cross-distro packages, but I wanted more.

So I started the Listaller project. Listaller was designed to build a cache of applications installed on a distribution and offer uninstalling these apps, no matter how they were installed. Listaller implemented it’s own package-manager abstraction layer, which basically worked with plugins which then called native distribution tools like “dpkg”, “rpm”, “apt-get” or “yum”. The only distributions supported were Debian, Ubuntu, SUSE and we had very poor support for Fedora.

Listaller also contained a new format to install applications, which was designed to abstract package management complexity. The Listaller package format was able to carry links to existing native packages and download & install them in order. It also was able to contain any binary installer, so it could also carry Autopackage or LOKI installers and perform necessary steps to execute these binaries and make sure everything is set up correctly. Finally, it was also possible to include the application install files directly into a Listaller package (file extension *.lipkg at the beginning, later changed to *.ipk) – It was planned that this solution should later be the only possible way to install software, the other IPK-package-modes were made to ease transition. The Listaller package generator also produced a logobutton, showing all distributions this package supported, which developers were able to put on their websites, because the generated Listaller packages were sometimes still limited to some distributions. (Listaller was able to detect that to a certain degree) I found only one of these buttons on my disk. It looked like the thing on the right side.

The software manager was very similar to later tools managing applications instead of packages, which was the goal of this effort.

Here’s a screenshot of the very early Listaller application manager (and it’s crappy UI):

With the rise of PackageKit, the whole situation shifted a little: Now we finally had one abstraction layer for all package managers distributions offered, which was much more powerful than the poor thing I created. I followed the PackageKit project since version 0.1.18 (I think) and later joined the project as developer and implemented support for it in Listaller.

During that time, the Klik installer project died and Autopackage merged into Listaller (after both projects first only did collaboration), leaving only very few projects with cross-distro software management in scope.

The the AppStream project was created (unfortunately I wasn’t able to attend the meeting due to exams) and suddenly all points the Listaller Project wanted to achieve back then had been solved – except for one: Cross-distro software installations. (AppStream implements the app-centric software management in a cross-distro way, so this Listaller goal is now achieved)

So I shifted the scope of Listaller to 3rd-party software-installations only and started a new implementation of it from scratch using the new Vala language. I also extended PackageKit with support for Listaller, because having multiple UIs for package management is highly confusing for users and the new design of Listaller made it possible to extend any PackageKit GUI with the ability to manage Listaller packages.

Listaller was extended to use ZeroInstall feeds to satisfy dependencies too, so resolving dependencies would work better, and both ZeroInstall and Listaller could benefit from the availability of more feeds. The Listaller support in PackageKit support was then split out into a plugin, so distributions are able to enable/disable Listaller as they like. (and also the plugin code made PackageKit’s code much better and easier to handle!)

Many other changes happened too, which I won’t summarize, this blogpost is already too long ;-) – Unfortunately the rewrite made Listaller unusable for a very long time and some people already considered the project dead.

The Present

So, many people might now ask the following:

Why do we need cross-distro packages?

I consider cross-distro software management a key element for the success of Linux desktops.

At time we have a very old protocol how software reaches end-users: Upstream creates it, downstream the distributions package it and deliver it to their users. This unfortunately does not leave any choice for people to selectively install or update the software they want. It is also not easily possible to install software which is not in the repositories. Of course, someone could create packages – but then he has to build his software for a multitude of distributions, which is lots of work (you can’t even target all RPM-based distributions using just one spec-file without doing many distro-specific checks)

It is important that developers can get quick feedback from their users on new software, without needing to go into the details of package management and having to wait for distributors to ship new software. (Take Debian stable as an example) Also, users should be able to install software without having to worry about compromising their system. (By installing a foreign package, you’re basically giving someone else root access to your computer)

The OpenBuildService, although being a great tool, is not a solution for this – also PPAs aren’t, because any change on the native package configuration can make the system unusable, install malicious software or break upgrade paths, and they’re hard to handle for non-technical users.

Where are the app developers?

There is a big lack of commercial software for Linux. Why? Usually we hear the argument that “Linux has not a big enough market share”. I don’t think this argument counts that much, as Apple also had a very small market share and they had commercial apps too back then. The situation we have in Linux is that you – as a proprietary software vendor – not just have to support “Linux”. You need to support a multitude of different configurations and distributions, and this fragmented world is very hard to handle.

First, some clarifications about my point of view on certain “social” aspects in the Linux community, so you can understand me easier:

Regarding FLOSS vs. proprietary software

I think both is important and we should fully support both. For me, I want the OS platform as open as possible to have unlimited possibilities regarding stuff I can do with it. And there should never ever be something limiting my freedom do make any change I want on the Linux platform. BUT, if people want to deploy proprietary software on that free-and-opensource platform, I welcome them too, because everyone should be able to use the FLOSS platform for everything they want. Also, it is better if more people use a free platform and be aware of it – this is the best way to prevent stuff like locked hardware devices and the UEFI hell. More app developers will certainly help in adopting the Linux platform. That’s why I welcome Valve on Linux too, although they’re proprietary and I don’t play games.

I always prefer free software over proprietary software, but there are some valid arguments for developing closed-source stuff too, and I don’t want to exclude these developers and add impose limits for our users to get software.

Regarding Linux and GNU/Linux

What is “Linux”? Many people already did lots of fighting about the name – I don’t want to do this here.

Sure, Linux is a OS kernel – technically. But what is Linux in society? For me, “Linux” is not a kernel or OS, instead it’s a value and brand, which defines all Linux OS’es. When I talk about “Linux” I refer to all components a Linux distribution is made of, the kernel, the plumbing layer, the toolkits, the desktop the GPL and the freedom we want to give to our users, as well as the great community. In population, Linux is a know word for “free operating system”, Linux is recognized – many small distributions are not. If I want to speak about the kernel, I just say “the Kernel” or “the Linux Kernel”.

People port software to “Linux”, people use “Linux” etc. – Linux is used as a brand and word for a free operating system.

So, it is important that software companies can target the Linux market if they want – not only a small fragment of it called “Ubuntu” or “Fedora”, but the whole Linux world using just one software build. People will see a download link for Linux software, not just a simple tarball and recognize Linux as equal (better!) to Windows and Mac.

The Linux world is big – it’s just not recognized as big. The Humble Indie Bundles already showed Linux users pay for software (I never doubted that and it doesn’t really surprise me), so we just need more app developers now.

Mark Shuttleworth recognized the potential Linux has – with programs to promote the Ubuntu desktop, workshops for app developers, the PPA-way to submit software and app-cebtric software management, they’re doing everything right. But of course, they’re Ubuntu-centric, software is developed “for Ubuntu”, you won’t find the word “Linux” anywhere on the Ubuntu homepage. Having a universal software distribution format for Linux would also be fair for smaller distributions, so they can receive new applications very fast too, although they don’t support Ubuntu PPAs.

Listaller changes

Listaller is currently designed to do exactly that – create packages to target the whole Linux “market”. These packages provide FLOSS developers a great way to push new releases directly to their end-users, removing the “distributor” part in the software release pipeline. Proprietary developers instead get a great way to target any Linux distribution.

The package format is based on standards like DOAP, so every application which ships DOAP informations (i.E. every GNOME application) can generate an IPK package really fast.

Of course, there are some limits: Listaller is designed to install applications, and only applications. Installing core infrastructure is not supported. This means you can’t install the full KDE with it or install systemd. You also can’t execute arbitrary code at install-time, most-used tasks are hardcoded.

Removing flexibility means adding security here.

Brought to you by…

The recent changes in Listaller wouldn’t be possible if the Linux desktop isn’t already becoming more app-developer friendly and more consistent: There is an increased awareness of ABI compatibility, systemd is unifying many OS functions which have been completely random before and Freedesktop is a great project which allows collaboration in components which work on all distributions.

For everything not yet covered, Listaller provides tools to make applications binary-compatible with many distributions, but I hope these things won’t be used in future anymore.

The future

There is a lot of new development in the GNOME community, which will make the life of a cross-distro packager and therefore Listaller easier: Stable APIs, a “GNOME SDK” etc. These changes might make it possible to make Listaller packages containing pure GNOME applications just depend on “GNOMESDK>=3.8″ instead of handling every single dependency separately.

On the KDE side, the project is splitting into “Workspaces”, “Frameworks” and “Applications”, which is also a great move forward for cross-distro apps and will make packaging KDE apps in general easier.

So, with this new development, I think it will finally be possible to build a cross-distro software installer with a design driven by user’s needs and not creating a big hack and workaround-set to overcome all technical limitations of cross-distro software: We might be able to fix these problems at the source. (The increased caring about ABI stability in FLOSS community already helps a lot)

Also, Listaller itself needs to be future-proof: We have two developments which affect Listaller:

  • a shifted focus towards WebApps instead of traditional applications
  • distribution through software-stores

For the first point: I do not believe in WebApps replacing traditional software – there are just too many limitations, and I also don’t like the idea of web apps storing much personal data on servers I don’t control. (Although there are efforts to fix that) I also think the Web technology is in many points inferior to what we use on the desktop today (be it Qt or GTK+). Instead, I think we’ll have more and more “hybrid” applications soon, which e.g. have logic implemented in C and run a UI built with web technology. Also QML is a very innovative and great approach for UI construction.

For the second point: I consider every application store as limitation of user’s freedom, as long as it is the only possible appstore and/or adding other stores is impossible. Competition between appstore vendors is very important.

So I imagine the following (far away) scenario: Distributions ship Listaller with their own software-store enabled, if they have one. The base OS is made of native packages, all additional software (also newer versions) is delivered through Listaller packages. (This doesn’t mean there are no applications like Firefox in the native packages – everything stays as-it-is, Listaller packages are an addition)

Because Listaller packages are distro-independent, OpenSUSE users can also use Ubuntu’s appstore-source, so there is no longer any dominance of a distribution regarding availability of precompiled software.

Ideally, one organisation creates a “Linux AppStore”, which is added to many distributions by default. (but maybe disabled and easy-to-enable) This Software Store will sell applications by commercial software vendors and – because it should be carried by a non-profit-organisation – sends most of the earned money back to distributions, which itself could use it to improve their distribution.

This software store would be very attractive for software vendors, because they could target the whole Linux market and would be very visible to potential customers, on distributions from Gentoo to Debian. Also Linux itself would be much more visible to others.

Listaller packages are by design very secure, so these packages can’t harm the system. (they’re usually signed and don’t install stuff in system directories – also a sandbox interface exists (but is not used at time)) Upgrades will work without problems, because the native package configuration is never touched, so distributors will receive less bug reports about those issues. PPAs will instead be used by people who want the latest Linux infrastructure, like a new KDE, GNOME, systemd, PackageKit etc. version, and not by people who just want the latest GIMP.

Because the setup process is 100% controlled by Listaller (no custom user scripts in packages, many things handled automagically), distributors can control every single 3rd-party software setup on their distribution just by adjusting Listaller to their needs. So distributors still have control about every piece of software installed on their distribution.

Also, Listaller is nicely integrated in all existing application-managers if they use PackageKit, so extra-UIs aren’t needed and users don’t even know that the software is there.

Of course, installing standalone software packages is still possible – they might of course be run in a sandbox by default.

Listaller will have logic to maintain the different software sources and provide information in a way frontends can display it nicely.


Of course I’m working on the code to make it ready for future use. The first step was adding support for package-updates using package deltas in Listaller 0.5.6 – unfortunately this feature has been delayed because of my SoC project and because I completely broke the API Listaller uses to talk to PackageKit during my work on PackageKit. So the next release will be without a major new feature, but with many bugfixes.

To make Listaller releases more frequent and because Listaller depends completely on PackageKit and internal APIs of PackageKit, I’ll sync the Listaller release cycle with the one of PackageKit. This means you will get a new Listaller version one week after a PackageKit release. By doing this I also hope new changes will reach people faster – and there will never be the case where Listaller and PackageKit are incompatible. (At time, Listaller 0.5.5 won’t work with PackageKit 0.8.x series)

Of course, the project needs contributors! :-D At time, “we” are only me and a few part-time people who create probably one small commit every two months, since the main contributors started to write their bachelor thesis or do other stuff (at work).

I would be very happy about comments regarding the plan above, which is of course very rough at time… To make it more concrete, I’ll try to talk to people at GNOME about their vision, because it seems they also have similar plans or at least have one for application management improvements, which could at least be valuable input.

And finally: Is the Linux desktop dead? The answer is NO! There might have been mistakes in the past, but both KDE and GNOME have clear goals for the future, are more end-user focused than ever, care about ABI stability, collaborate on an increasing amount of projects (although it could be more ^^), prepare for targeting mobile devices and are used by millions of people. We should be very happy :-) Let’s see what we achieve in future, even if it is not desktop-dominance over all other OSes, it will be great anyway! :)

(Of course I also care about Xfce, LXDE and E17, just before someone asks ;-))

If you want to contact me, you’ll find me on IRC: #PackageKit (Freenode) There’s also a Google Group about Listaller, but I’m looking for a traditional mailinglist at time… Reaching me via mail is always the easiest way ;-)

August 30, 2012

This year I did a Google Summer of Code Project for OpenSUSE as part of their cross-distribution collaboration track. (Again, many thanks for letting me work on this and for doing a cross-distro track!)

So, what did I achieve this summer? (Leaving out all the problems and stuff which didn’t work as I expected :P )

I did work on three components: AppStream, the Software Center and PackageKit.

At the beginning of the SoC, I thought I would be spending most of my time at the Software Center and in seeing Python code. Instead, I put lots of effort into PackageKit and writing C code. The reason for that was that when I started, all operations I performed in the Software Center were very slow, because PackageKit was slow. Also, some things weren’t possible, e.g. I couldn’t fetch details about an application from PackageKit while an installation was going on, because the installation blocked all other requests on PackageKit. So, I first focused on PackageKit.


I implemented various small speed optimizations, which made PackageKit a little bit faster. (on Ubuntu, it’s faster than Aptd now, but that’s not a fair comparison) I also added a package sqlcache, which can be used to access package data very fast, if enabled.

The biggest thing I implemented was support for parallel transactions. Parallel transactions allow PackageKit backends to run some transactions in parallel, if they support this feature. This means that you can now query package details or search the package database using PackageKit while installing or removing packages at the same time.

It also enabled frontends to query more data faster, which will speed up every PackageKit client if the PackageKit backend supports parallelization. At time, the Zif backend fully supports this feature and the Aptcc backend supports it partially.

Together with Richard Hughes I refactored the backend API, so backends now have some more options to optimize they way they handle cache openings and jobs. (most of the credits for these API changes go to Richard)

Software Center

The Software Center is a fork of the original Ubuntu Software Center, because it was not possible to have Canonical drop the CLA they apply on Software Center’s code. (and which is a problem for new contributors, especially those employed by companies)

At the Software Center side I did lots of bugfixing and speed improvements, for example the SC now starts really fast (if the PackageKit backend is fast too) and data fetching is super-fast too. I also removed some problems preventing the SC to work properly on non-Ubuntu distributions and ported the code to our newly created APIs.

I am still not happy with the state of the Software Center, as there are many unsolved bugs and the tool is not yet user-friendly. But I also think these problems will vanish when distributions start to ship AppStream data to fill the Software Center database and when PackageKit backends are improved.

Here are some screenshots of the PackageKit-based Software Center running on Debian and Fedora:


You can already test the code if you have at least PackageKit 0.8.4 installed. Unfortunately distributions don’t provide all AppStream data (the information which matches package names with applications and contains icons & stuff) yet, so using it is quite difficult at time.


For the AppStream project itself I created infrastructure to create & maintain the AppStream Xapian database. This database is used by application managers to query data about applications. It also makes searching applications really easy. At the beginning, all code which created this database was hard-wired in the original Ubuntu Software Center. So, if you wanted AppStream data, you had to install the USC. Now, I wrote a nice tool in C++ and Vala, which will build the database from various sources and which allows querying the database using a very simple interface.

These changes will allow alternative Software Center implementations to use AppStream and it will make some other interesting development possible (e.g. application support in Apper, which I implemented after my SoC project was completed, see this post for details.)

There are already some new software centers in progress, for example the Light Software Center, which was designed for use with PackageKit from the beginning.

At time I’m running a request at Freedesktop for space to present the new project & APIs, as well as to upload release tarballs, so this code can hit distribution repositories soon. :)

Project conclusion

It bothers me a little that I cannot present you a 100% bug-free end-user-usable Software Center with the end of my project. But instead, I killed all the big technical problems which made implementing a Software Center on other distributions impossible before.

I also did lots of changes which will make using PackageKit even more pleasant to use now and I created Software Center infrastructure, which will allow many projects to implement AppStream features. For example, GNOME developers can get their application-centric software managers, and KDE people will receive an AppStream-ready version of Apper soon.

So in the end, this project was very successful and I’m really happy with the results. Also, it feels like the AppStream project itself gained momentum now, and I will of course continue developing stuff around AppStream – stay tuned, there will be more news later :-)

GSoC conclusion

I can only recommend every student out there to consider applying for the Summer of Code! It has been an absolutely great time, meeting new people and discussing stuff with them has always been fun for me, but also the feeling of being able to really move a project forward and taking the time to do that is awesome!

During my SoC project, I really had some problems to find a balance between university stuff and GSoC work, because university was giving me a hard time (lots of stuff to learn!). In the end, even a harddrive crash in the hottest development period couldn’t stop me ;-) (now I have a new drive and I was fortunately able to restore most of the old data)

Working with Vincent and of course Richard (who I’m working with for years now) has been a lot of fun, and also interacting with various people in the community was great, but also sometimes very exhausting – in cross-distro and cross-desktop world you really have to put lots of energy in convincing other people to make certain changes happen. But during my SoC I achieved more things than ever in the area of application-centric software management, so it was totally worth it.

When I look at the projects fellow SoC students completed, there are even more great projects to see! (Just take a look at it!) Some other students I talked to also agree that the experience alone was already worth doing their SoC project.

So, if you are a student and are able to write code or do some other cool stuff in FLOSS projects: Consider applying for GSoC 2013!

So, that’s it for now, I already planned some new blogposts later, so stay tuned!

August 26, 2012

As some of might now: I did a GSoC project for OpenSUSE this summer. The goal was to improve AppStream and the Software Center stuff on all Linux distributions.

During my SoC project, I also created appstream-core, a small library and infrastructure to create & use the AppStream application database. This project enables all other tools to use AppStream data and combine it with PackageKit information.

So it was just natural to bring this stuff to Apper too, our favorite PackageKit-based KDE package manager ;-) Apper has had a function like this before, but it was limited to Kubuntu. Now, the AppStream stuff will work on all distributions which support AppStream and ship at least PackageKit 0.8.4 (currently unreleased).

The result of Apper+AppStream looks like this:

As you can see, Apper is showing applications now instead of packages. (All packages which don’t ship applications are still presented as packages) It also uses the Debian screenshot service to display a screenshot. This feature is experimental and still not completely finished on the AppStream side.

Additional credits for this go to Daniel Nicoletti, because I based my work on previous work done by him.

The new code is in Apper Git already. You will need PackageKit>=0.8.4 and AppStream-Core to test it, but because these changes are experimental at time I’d suggest waiting for your distribution to ship it. (many software is even unreleased at time)

So, this is just a sneak preview of the cool stuff to come. :) Stay tuned!

August 12, 2012


My SoC project is nearly finished now, which unfortunately doesn’t mean that we will have a completely usable Software Center on all distributions – but the most difficult steps are done, all specification issues solved and I talked to many people from other distributions about the AppStream project and we made lots of progress. I’ll publish details in my GSoC final report ;-) This post is meant as a guide which steps are necessary to make distribution X support the AppStream project and – using AppStream – an application-centric software management solution and Software-Center-like applications.

1. Write/update your PackageKit backend

We rely on PackageKit for package management, so to support your distribution, we need a backend for your package manager.

Make sure that your backend supports parallelization! This feature is used heavily in application-management frontends like the Ubuntu Software Center and will generally make your backend much faster.

2. Package AppStream components and ship them

First of all you will need of course to ship PackageKit. Also we need AppStream-Core (with UAI) to be installed. (I’ll make a first alpha release of this after my GSoC project is complete and I discussed some things regarding this piece of software with my mentor(s)) If you want, you can also ship the cross-distro version of Software Center – you’ll need to package it’s dependencies too, some extra Python components aren’t present in all distributions.

3. Generate AppStream data

This is the most important step. AppStream data can be provided in the Ubuntu AppInstall format or Debian DEP-11, but we suggest you use the AppStream XML spec which is supported on every distro. You need a script which generates this data from packages and application’s desktop files which fits your distribution’s needs. (This script needs to run on the distribution’s servers) OpenSUSE and Fedora already have these generator tools. You also need to make this script extract icons from packages, so the software-center-implementations can display icons of not-installed packages.

The data should probably be regenerated automatically every week on distro branches which are rolling-release or in development. For stable distros where package names don’t change and no applications are added or removed, generating the data at release-time should be enough.

4. Ship AppStream data

You can ship AppStream data any way you want. You can package it as a regular package (done in Ubuntu) or advise the package manager to download it as part of the repository metadata (planned on-request for Debian). You just need to make sure the XML files end up in /usr/share/app-info/xmls and the icons are placed in /usr/share/app-info/icons . It makes sense to encode the repository name in the XML file name to avoid filename-conflicts if another (3rd-party?) repository wants to install application data with the same name.

If you use the package-way to make the data available, make sure to depend on the “update-appstream-index” tool and execute “appstream-index –refresh –nowait” in the package postinstall script, so the AppStream index is rebuilt. If you use another way to deliver the data, make sure to trigger the cache rebuild there too.

5. (Optional) Adjust the Software Center application

If you use the cross-distro non-CLA-fork of the Ubuntu Software Center, you maybe need to add a profile for your distribution to it. At time we support Ubuntu, Debian, Fedora and OpenSUSE already. You also might want to do other adjustments to fit your distribution’s needs.

If you don’t use the USC, you can skip this step and just use another implementation. (A “Light Software Center” by the Xubuntu/Lubuntu team is already in progress, as well as something done by the Elementary project. I’ll write a blog post about these projects as soon as they become more mature)

6. Test it!

There might be some quirks in your data which need to be removed, e.g. applications listed which don’t belong there, wrong descriptions, apps listed twice, system services (KDE!) listed, very slow PackageKit backend etc. So, this feature just needs testing now. Otherwise everything should be set up and ready now :)

Please note: I’ll update this blogpost with new information if I find something I forgot to mention above. If I do changes, I’ll add a short note about the changes here.

August 09, 2012

With the most recent release of PackageKit, PackageKit 0.8.3 (published last Monday!), all my changes regarding parallelization have been merged into our master branch, which means parallelization features are now available for backends to use! Yay!

The best thing about the new parallelization is that it will have an incredibly high impact on PackageKit’s speed: Frontends are now able to process many resolve-requests at the same time. Also, you will no longer have to wait for PackageKit to finish installing packages before you can continue browsing the list of installed packages again. (This was the main reason to develop this feature in my SoC, as it is crucial for Software-Center-like applications)

Internally, many other things have changed also, which allow backends to do very clever cache-handling, so the cache can stay open for a longer time, so we can avoid the delay while a backend reopens the cache when running many transactions. We now have a “Backend” and multiple “BackendJobs”, which process specific requests, for example Resolve() or InstallPackages() which provides backend authors with a flexible way to handle transactions.

Parallel processing is implemented in a way where it is nearly impossible to get to a situation where everything is waiting and the daemon is dead-locked. (only a bug in PK or the backend could cause this) Avoiding this situation was – of course – very important, as otherwise the package database could be damaged or a database lock could just be never released.

The new changes also require some work of our backend authors. By default, parallel-transactions are disabled for backends which haven’t declared that they support it. Backend authors need to explicitly enable the feature as soon as their backend supports it. We highly recommend to enable parallelization, as PackageKit frontends might start to rely on it and will be much slower without a parallelized PackageKit backend. Also, backend authors will have to port to the new PkBackendJob infrastructure. Most necessary changes have already been made by automatic scripts, but of course it is better if people using the backend and knowing the package-manager would take a look at it.

So, backend authors: We want you! (to fix the backends ;-) ) Richard and I have created a porting guide for backends which summarizes all changes required for backends to be PK 0.8.x compatible. (backends/PORTING.txt)

Parallelization is a very invasive change and we have only one release with it enabled, so please help testing to find possible remaining bugs! :-)

And end-users can be happy about a much faster PackageKit soon, which will also be working as Software-Center engine ;-)

July 09, 2012

It’s been a very long time since my last blogpost, but you can be sure I haven’t gone lost! ;-) I was very busy with writing exams (and preparing for them) the last weeks, and university was giving me a hard time. Now I still have to write two more exams, but one of them is in August and I won’t have lectures anymore with the end of next week, so I have much more time again to work on my GSoC project on the Software Center.

So, what has happened so far? (If you read the mailinglists, you might want to skip that part)

I looked into the code, updated some very odd parts the SC used to access PackageKit, but there are still many pieces of code left which don’t look good and need optimization. During my work on the SC, I found out that with the current way PackageKit works it would be impossible to implement the Software Center in an user-friendly way. For example not being able to see the details of an application while another one is installing sucks. Also, the round-trips to the daemon slowed the whole thing down.

First I did some optimizations on the code which loads the package cache, so the SC now starts very fast. (It had an incredible startup time of ~30 minutes on my machine before, now it’s down to 4sec and faster (to show the UI, to be ready it takes ~20 more seconds)) I also did a few modifcations on PackageKit, which saved us some msecs to secs (depending on the requested action). To solve the general issues with PackageKit, I wrote a module for PK to create a cache of all packages. This solution resulted in a massive speed gain for the Software Center, but slowed down PackageKit actions a little, for example Refresh(). Also, the cache suffered – naturally – from all problems of a cache, e.g. it was going out of sync extremely fast. (You just needed to use a native package management tool) Additionally it was duplicating or even triplicating package data, and generally disliked by many people, including me in the first place. I first chose to implement the cache, because it was the easiest way to get the needed functionality (parallel access to package-data and fast cache-loading) in-time for my SoC project. But a workaround solution serves nobody in the long run. So I sat down with Richard Hughes, thinking about how we could change PackageKit to serve the needs of a Software Center. He rejected some of my proposals and we had a very extensive discussion about a suggestion by Daniel Nicoletti, but in the end we ended up with a solution which will allow PackageKit to execute software management tasks in parallel, if the backend supports it. The chosen solution is one which requires massive changes on the backend API and the most invasive changes in general, but it’s also a solution which does things right(tm), without any workarounds or more layers to access package information.

Implementing this functionality is not exactly trivial, we broke the PackageKit backend API completely, so all backends will now need massive changes to support the new functions and to even compile again. So, here’s a call for backend authors: Please fix your backends! You can see how the Yum, Zif and Aptcc backends were changed, there’s also very simple documentation of required changes in backends/PORTING.txt (file will be updated soon). The implementation is not complete, I’m working on the missing pieces now, so some changes might still happen.

All these changes mean PackageKit will soon be able to execute several actions in parallel if the backend supports it, for example running GetDetails() on a package while InstallPackages() is running too. We count on backend authors to implement this functionality, otherwise distributions running PackageKit without a backend which supports parallelization will not deliver a good “Software-Center experience”.

At the Software Center side, I’m currently implementing the PackageKit history feature, after that I will have to do some polishing and many ichanges on the code which talks to PackageKit, I guess – there’s lots of room for improvements! Also, there are some Ubuntu-specifics which need to be solved, as well as it is needed to split out the code which generates the AppStream Xapian database, so other alternative Software Center implementations are possible without the need to install the “original” Ubuntu Software Center. (I will work with Vincent and Michael on this)

If you want to try the current SC, just grab the code from Gitorious and try it – but be careful! You’ll need the latest development version of PackageKit (master branch) to make it work, but at the moment the only working backends are Yum, Zif, Aptcc and Dummy. So, Fedora, Debian and Ubuntu users are happy at time, all other distributions will need to update their backends as soon as the dust settled down. (There are also some more new and cool features in the unstable series at time, for example the systemd-support Richard has implemented, and the ofline-upgrade feature.) If your PackageKit is not the latest version, the SC will just crash on startup – I haven’t found out how to check version numbers of imported GObject Introspection data in Python to throw a proper error message. (in general it seems like version-checking is not used very much in Python-world) At the moment, using the SC is still not trivial, but in the end of this project everyone should be able to make use of it. (which is not the case at time, to be honest ;-) )

During the next weeks you can expect more code changes again, as the discussion part seems to be over, and I have more time again. (Doing cross-distro projects is 60% talking and 40% coding) I’ll also try to blog more, so everyone can stay informed without having to read mailinglist threads.

In the end, (and I think I should make that my signature) kudos to Richard Hughes for beeing an excellent maintainer and for helping me so much with the most difficult changes on PK. Also, once again thank you OpenSUSE for letting me work on this!