Planet Tanglu
July 16, 2014

appstream-logoToday I am very happy to announce the release of AppStream 0.7, the second-largest release (judging by commit number) after 0.6. AppStream 0.7 brings many new features for the specification, adds lots of good stuff to libappstream, introduces a new libappstream-qt library for Qt developers and, as always, fixes some bugs.

Unfortunately we broke the API/ABI of libappstream, so please adjust your code accordingly. Apart from that, any other changes are backwards-compatible. So, here is an overview of what’s new in AppStream 0.7:

Specification changes

Distributors may now specify a new <languages/> tag in their distribution XML, providing information about the languages a component supports and the completion-percentage for the language. This allows software-centers to apply smart filtering on applications to highlight the ones which are available in the users native language.

A new addon component type was added to represent software which is designed to be used together with a specific other application (think of a Firefox addon or GNOME-Shell extension). Software-center applications can group the addons together with their main application to provide an easy way for users to install additional functionality for existing applications.

The <provides/> tag gained a new dbus item-type to expose D-Bus interface names the component provides to the outside world. This means in future it will be possible to search for components providing a specific dbus service:

$ appstream-index what-provides dbus org.freedesktop.PackageKit.desktop system

(if you are using the cli tool)

A <developer_name/> tag was added to the generic component definition to define the name of the component developer in a human-readable form. Possible values are, for example “The KDE Community”, “GNOME Developers” or even the developer’s full name. This value can be (optionally) translated and will be displayed in software-centers.

An <update_contact/> tag was added to the specification, to provide a convenient way for distributors to reach upstream to talk about changes made to their metadata or issues with the latest software update. This tag was already used by some projects before, and has now been added to the official specification.

Timestamps in <release/> tags must now be UNIX epochs, YYYYMMDD is no longer valid (fortunately, everyone is already using UNIX epochs).

Last but not least, the <pkgname/> tag is now allowed multiple times per component. We still recommend to create metapackages according to the contents the upstream metadata describes and place the file there. However, in some cases defining one component to be in multiple packages is a short way to make metadata available correctly without excessive package-tuning (which can become difficult if a <provides/> tag needs to be satisfied).

As small sidenote: The multiarch path in /usr/share/appdata is now deprecated, because we think that we can live without it (by shipping -data packages per library and using smarter AppStream metadata generators which take advantage of the ability to define multiple <pkgname/> tags)

Documentation updates

In general, the documentation of the specification has been reworked to be easier to understand and to include less duplication of information. We now use excessive crosslinking to show you the information you need in order to write metadata for your upstream project, or to implement a metadata generator for your distribution.

Because the specification needs to define the allowed tags completely and contain as much information as possible, it is not very easy to digest for upstream authors who just want some metadata shipped quickly. In order to help them, we now have “Quickstart pages” in the documentation, which are rich of examples and contain the most important subset of information you need to write a good metadata file. These quickstart guides already exist for desktop-applications and addons, more will follow in future.

We also have an explicit section dealing with the question “How do I translate upstream metadata?” now.

More changes to the docs are planned for the next point releases. You can find the full project documentation at Freedesktop.

AppStream GObject library and tools

The libappstream library also received lots of changes. The most important one: We switched from using LGPL-3+ to LGPL-2.1+. People who know me know that I love the v3 license family of GPL licenses – I like it for tivoization protection, it’s explicit compatibility with some important other licenses and cosmetic details, like entities not loosing their right to use the software forever after a license violation. However, a LGPL-3+ library does not mix well with projects licensed under other open source licenses, mainly GPL-2-only projects. I want libappstream to be used by anyone without forcing the project to change its license. For some reason, using the library from proprietary code is easier than using it from a GPL-2-only open source project. The license change was also a popular request of people wanting to use the library, so I made the switch with 0.7. If you want to know more about the LGPL-3 issues, I recommend reading this blogpost by Nikos (GnuTLS).

On the code-side, libappstream received a large pile of bugfixes and some internal restructuring. This makes the cache builder about 5% faster (depending on your system and the amount of metadata which needs to be processed) and prepares for future changes (e.g. I plan to obsolete PackageKit’s desktop-file-database in the long term).

The library also brings back support for legacy AppData files, which it can now read. However, appstream-validate will not validate these files (and kindly ask you to migrate to the new format).

The appstream-index tool received some changes, making it’s command-line interface a bit more modern. It is also possible now to place the Xapian cache at arbitrary locations, which is a nice feature for developers.

Additionally, the testsuite got improved and should now work on systems which do not have metadata installed.

Of course, libappstream also implements all features of the new 0.7 specification.

With the 0.7 release, some symbols were removed which have been deprecated for a few releases, most notably as_component_get/set_idname, as_database_find_components_by_str, as_component_get/set_homepage and the “pkgname” property of AsComponent (which is now a string array and called “pkgnames”). API level was bumped to 1.

Appstream-Qt

A Qt library to access AppStream data has been added. So if you want to use AppStream metadata in your Qt application, you can easily do that now without touching any GLib/GObject based code!

Special thanks to Sune Vuorela for his nice rework of the Qt library!

And that’s it with the changes for now! Thanks to everyone who helped making 0.7 ready, being it feedback, contributions to the documentation, translation or coding. You can get the release tarballs at Freedesktop. Have fun!

July 05, 2014

Five months ago I announced the very first release of Cutelyst, a web framework powered by Qt 5. Time passes and I started shaping my real world applications written with it, I should probably release a newer version earlier but it’s not very nice to keep releasing API that changes a lot, however next version (0.3.0) will still contains API changes. What changed from 0.1.0 to 0.2.0:

  • I have setup a documentation website http://cutelyst.org
  • I started a WordPress like blog written with Cutelyst https://gitorious.org/cutelyst/untitled this right now is a nice example app, if you like it please help :)
  • Changed the API to expose the forked behavior thus being able to setup database connection in each new process avoiding a mess
  • UWSGI got a patch to deal with workers that don’t want to work, in other words when something on post-fork fails like reaching the dababase connection limit the worker can exit and won’t be restarted.
  • A bunch of bugfixed of course
  • Make use of categorized logging (ie qCDebug)
  • Allow for UWSGI to restart the application as soon as your application .so file changes, this greatly improve development speed :)
  • Encapsulated the post body into a QIODevice subclass on UWSGI engine this means a QFile is –post-buffering limit is reached, or a custom IODevice that deals with the request memory buffer
  • A parser to get uploaded content “multipart/form-data” which is fast and can work with lots of data

For the next version I’ll try to get a properQt loop integration. Download here Have fun!


June 18, 2014
bartholomea-annulata

Bartholomea annulata | (c) Kevin Bryant

It is time for a new Tanglu update, which has been overdue for a long time now!

Many things happened in Tanglu development, so here is just a short overview of what was done in the past months.

Infrastructure

Debile

The whole Tanglu distribution is now built with Debile, replacing Jenkins, which was difficult to use for package building purposes (although Jenkins is great for other things). You can see the Tanglu builders in action at buildd.tg.o.

The migration to Debile took a lot of time (a lot more than expected), and blocked the Bartholomea development at the beginning, but now it is working smoothly. Many thanks to all people who have been involved with making Debile work for Tanglu, especially Jon Severinsson. And of course many thanks to the Debile developers for helping with the integration, Sylvestre Ledru and of course Paul Tagliamonte.

Archive Server Migration

Those who read the tanglu-announce mailinglist know this already: We moved the main archive server stuff at archive.tg.o to to a new location, and to a very powerful machine. We also added some additional security measures to it, to prevent attacks.

The previous machine is now being used for the bugtracker at bugs.tg.o and for some other things, including an archive mirror and the new Tanglu User Forums. See more about that below :-)

Transitions

There is huge ongoing work on package transitions. Take a look at our transition tracker and the staging migration log to get a taste of it.

Merging with Debian Unstable is also going on right now, and we are working on merging some of the Tanglu changes which are useful for Debian as well (or which just reduce the diff to Tanglu) back to their upstream packages.

Installer

Work on the Tanglu Live-Installer, although badly needed, has not yet been started (it’s a task ready for taking by anyone who likes to do it!) – however, some awesome progress has been made in making the Debian-Installer work for Tanglu, which allows us to perform minimal installations of the Tanglu base systems and allows easier support of alternative Tanglu falvours. The work on d-i also uncovered a bug which appeared with the latest version of findutils, which has been reported upstream before Debian could run into it. This awesome progress was possible thanks to the work of Philip Muškovac and Thomas Funk (in really hard debug sessions).

Tanglu ForumsTanglu Users

We finally have the long-awaited Tanglu user forums ready! As discussed in the last meeting, a popular demand on IRC and our mailing lists was a forum or Stackexchange-like service for users to communicate, since many people can work better with that than with mailinglists.

Therefore, the new English TangluUsers forum is now ready at TangluUsers.org. The forum software is in an alpha version though, so we might experience some bugs which haven’t been uncovered in the testing period. We will watch how the software performs and then decide if we stick to it or maybe switch to another one. But so far, we are really happy with the Misago Forums, and our usage of it already led to the inclusion of some patches against Misago. It also is actively maintained and has an active community.

Misc Thingstanglu logo pure

KDE

We will ship with at least KDE Applications 4.13, maybe some 4.14 things as well (if we are lucky, since Tanglu will likely be in feature-freeze when this stuff is released). The other KDE parts will remain on their latest version from the 4.x series. For Tanglu 3, we might update KDE SC 4.x to KDE Frameworks 5 and use Plasma 5 though.

GNOME

Due to the lack manpower on the GNOME flavor, GNOME will ship in the same version available in Debian Sid – maybe with some stuff pulled from Experimental, where it makes sense. A GNOME flavor is planned to be available.

Common infrastructure

We currently run with systemd 208, but a switch to 210 is planned. Tanglu 2 also targets the X.org server in version 1.16. For more changes, stay tuned. The kernel release for Bartholomea is also not yet decided.

Artwork

Work on the default Tanglu 2 design has started as well – any artwork submissions are most welcome!

Tanglu joins the OIN

The Tanglu project is now a proud member (licensee)  of the Open Invention Network (OIN), which build a pool of defensive patents to protect the Linux ecosystem from companies who are trying to use patents against Linux. Although the Tanglu community does not fully support the generally positive stance the OIN has about software patents, the OIN effort is very useful and we agree with it’s goal. Therefore, Tanglu joined the OIN as licensee.


And that’s the stuff for now! If you have further questions, just join us on #tanglu or #tanglu-devel on Freenode, or write to our newly created forum! – You can, as always, also subscribe to our mailinglists to get in touch.

May 26, 2014

And again, another KDE-AppStream post ;-) If you want to know more about AppStream metadata and why adding it to your project is a good idea, you might be interested in this blogpost (and several previous ones I wrote).

Originally, my plan was to directly push metadata to most KDE projects. The problem is that there is no way to reach all maintainers and have them opt-out for getting metadata pushed to their repositories. There is also no technical policy for a KDE project, since “KDE” is really only about the community right now, and there are no technical criteria a project under the KDE umbrella has to fulfill (at least to my knowledge, in theory, even GTK+ projects are perfectly fine within KDE).

Since I feel very uncomfortable in touching other people’s repositories without sending them a note first, I think the best way forward is an opt-in approach.

So, if you want your KDE project to ship metadata, follow these simple steps:

1. Check if there already is metadata for your project

That’s right – we already have some metadata available. Checkout the kde-appstream-metadata-templates repository at Github. You can take the XML file from there, if you want. Just make sure that there are no invalid tags in the description field (no <a/> nodes allowed, for example – the content is not HTML!), check if you have an SPDX-compliant <project_license/> tag, check if the provided public interfaces in the <provides/> tag match your project and also test if the URLs work.

Then you can copy the modified AppStream metadata to your project.

2. Write new metadata

How to write new metadata is described in detail at this TechBase Wiki page. Just follow the instructions.

In case you need help or want me to push the metadata to your project if you don’t have the time, you can also write me an email: matthias [{AT}] tenstral . net – or alternatively file a bug against the Github project linked above.

Don’t forget to have CMake install your shiny new metadata into /usr/share/appdata/.

All metadata you add to your project will automatically get translated by the KDE l10n scripts, no further action is required. So far, projects like Ark, K3b and Calligra are shipping metadata, and the number of AppStream-aware projects in KDE is growing constantly, which greatly improves their visibility in software centers, and will help distributions a lot in organizing installed software.

If you have further questions, please ask! .-)

May 05, 2014

This new release doesn’t bring new features but it’s an important one since it supports PackageKit 0.9.x API, also this is the first release which uses the first 100% async calls on PackageKit-Qt.

Making Pk-Qt sync-free wasn’t so complicated but since I had to rethink the API the client code of Apper also needed to be adapted, I must say that I was lazy sometimes and just created a QEventLoop in Apper to wait for the reply… and this basically means that there will be new bugs in Apper, and hopefully some will be fixed.

It’s interesting that using DBus doesn’t means the code talking to it is async, the proxy classes generated by qdbus2cpp do a blocking call everytime you ask for a property, so you have to avoid it and do a GetProperties call which returns to a callback, although this worked well for PackageKitQt I failed to do the same for libnm-qt, and that is because it’s nearly impossible to make it somehow reliable due to NetworkManager itself not using gdbus, so you can’t have a reliable snapshot of it, unless you do an introspection call which is also insane, so I prefer to wait for when they switch to gdbus :P

Apper 0.9.0 depends on Qt4, KDE4 libs and PackageKitQt 0.9.2.

I’m not giving it much attention lately since I’m slowly building Apper on Qt5 which will be using QtQuick2, as my time is short supporting this old version is mainly to make packagers happy :)

http://download.kde.org/stable/apper/0.9.0/src/apper-0.9.0.tar.xz.mirrorlist


May 03, 2014

kde-appstreamIt took some time, but now it’s finally done: KDE has translations for AppStream upstream metadata!

AppStream is a Freedesktop project to extend metadata about the software projects which is available in distributions, especially regarding applications. Distributions compile a metadata file from data collected from packages, .desktop files and possibly other information sources, and create an AppStream XML file from it, which is then – directly or via a Xapian cache – read by software-center-like applications such as GNOME-Software or KDEs Apper.

Since the metadata available from current sources is not standardized and rather poor, upstream projects can ship small XML files, AppStream upstream metadata or AppData in short. These files contain additional information about a project, such as a long description and links to screenshots. They also provide hints about public interfaces a software provides, for example binaries and libraries, making it possible for distributors to give users exactly the right package name in case they are missing a software component.

So, in order to represent graphical KDE applications like they deserve it in the new software centers making use of AppStream, we need to ship AppData files, with long descriptions, screenshots and a few URLs.

But how can you create these metadata files? In case you want your graphical KDE app to ship an AppData file, there is now a help page on the Techbase Wiki which provides all information needed to get started!

For non-visual stuff or software which just wants to publish it’s provided interfaces with AppStream metadata, there is a dedicated page for that as well. Shipping metadata for non-GUI apps will help programmers to satisfy depedencies in order to compile new software, enhance bash-completion for missing binaries and provides some other neat stuff (take a look at this blogpost to get a taste of it).

And if you want to read a FAQ about the metadata stuff and get the bigger picture, just go to the Techbase Wiki page about AppStream metadata as well.

The pages are not 100% final, so if you have questions, please write me a mail and I’ll update the pages, or simply correct/refine it by yourself (it’s a wiki afterall).

And now to the best thing: As soon as you ship an AppStream upstream metadata file (*.appdata.xml* for apps or *.metainfo.xml* for other stuff), the KDE l10n-script (Scripty!) will automatically start translating it, just like we already do with .desktop files. No further actions are necessary.

I already have a large amount of metadata files here, partially auto-generated, which show that we have about 160+ applications in KDE which could get an AppData file, not counting any frameworks or other non-GUI stuff yet. Since that is a bit much to submit via Reviewboard (which I originally planned to do), I hope I can commit the changes directly to the respective repositories, where the maintainers can take a look at it and adjust it to their liking. If that idea does not receive approval, I will just publish a set of data at some place for the KDE app maintainers to take as a reference (the auto-generated stuff needs some fixup to be commit-ready (which I’d do in case I can just commit changes)). Either way, it is safe now to write and ship AppData files in KDE projects!

In order to get your stuff translated, it is necessary that you follow the AppStream 0.6 metadata specification, and not one of ther older revisions. You can easily detect 0.6 metadata by the <component> root node, instead of <application>, or by it having a metadata_license tag. We don’t support the older versions simply because it’s not necessary, as there were only two KDE projects shipping AppData before, which are now using 0.6 data as well. Since 0.6, the metadata XML format is guaranteed to be stable, and the only reason which could make me change it in an incompatible way is to prevent something as bad as the end of the world from happening (== won’t happen ;-) ). You can find the full specification (upstream and distro data) here.

All parsers are able to handle 0.6 data now, and the existing tools are almost all migrated already (might take a few months to hit the distributions though).

So, happy metadata-writing! :-)

Thanks to all people who helped with making this happen, and especially Burkhard Lück and Albert Astals Cid for their patch-review and help with integrating the necessary changes into the KDE l10n-script.

April 25, 2014

Today I released AppStream and libappstream 0.6.1, which feature mostly bugfixes, so nothing incredibly exciting to see there (but this also means no API/ABI breaks). The release clarifies some paragraphs in the spec which people found confusing, and fixes a few issues (like one example in the docs not being valid AppStream metadata). As only spec extension, we introduce a “priority” property in distro metadata to allow metadata from one repository to override data shipped by another one. This is used (although with a similar syntax) in Fedora already to have “placeholder” data for non-free stuff, which gets overridden by the real metadata if a new application was added. In general, the property tag was added to make the answer to the question “which data is preferred” much less magic.

The libappstream library got some new API to query component data in different ways, and I also brought back support for Vala (so if you missed the Vapi file: It’s back now, although you have to manually enable this feature).

The CLI tool also got some extensions to query AppStream data. Here is a brief introduction:

First of all, we need to make sure the database is up-to-date, which should be the case already (it is rebuilt automatically):

$ sudo appstream-index refresh

The database will only be rebuilt when necessary, if you want to force a rebuild anyway, use the “–force” parameter.

Now imagine we want to search for an app containing the word “media” (in description, keywords, summary, …):

$ appstream-index s media

which will return:

Identifier: gnome-media-player.desktop [desktop-app]
Name: GNOME Media Player
Summary: A simple media player for GNOME
Package: gnome-media-player
----
Identifier: mediaplayer-app.desktop [desktop-app]
Name: Media Player
Summary: Media Player
Package: mediaplayer-app
----
Identifier: kde4__plasma-mediacenter.desktop [desktop-app]
Name: Plasma Media Center
Summary: A mediacenter user interface written with the Plasma framework
Package: plasma-mediacenter
----
etc.

If we already know the name if a .desktop-file or the ID of a component, we can have the tool print out information about the application, including which package it was installed from:

$ appstream-index get lyx.desktop

If we want to see more details, including e.g. a screenshot URL and a longer description, we can pass “–details” to the tool:

Identifier: lyx.desktop [desktop-app]
Name: LyX
Summary: An advanced document processor with the power of LaTeX.
Package: lyx-common
Homepage: http://www.lyx.org/
Icon: lyx.png
Description: LyX is a document processor that encourages an approach to writing
 based on the structure of your documents (WYSIWYM) and not simply
 their appearance (WYSIWYG).
 
 LyX combines the power and flexibility of TeX/LaTeX[...]
Sample Screenshot URL: http://alt.fedoraproject.org/pub/alt/screenshots/f21/source/lyx-ea535ddf18b5c7328c5e88d2cd2cbd8c.png
License: GPLv2+

(I truncated the results slightly ;-) )

Okay, so far so good. But now it gets really exciting (and this is a feature added with 0.6.1): We can now query a component by the items it provides. For example, I want to know which software provides the library libfoo.2:

appstream-index what-provides lib libfoo.so.2

This also works with binaries, or Python modules:

appstream-index what-provides bin apper

This stuff works distribution-agnostic, and as long as software ships upstream metadata with a valid <provides/> field, or the distributor adds it while generating AppStream distro metadata.
This means that software can – as soon as we have sufficient metadata of this kind – declare it’s dependencies upstream in form of a simple text file, referencing the needed components to build and run it on any Linux distribution. Users can simply install missing stuff by passing that file to their package manager, which can look up the components->packaging mapping and versions and do the right thing in installing the dependencies. So basically, this allows things “pip -r” does for Python, but for any application (not only Python stuff), and based on the distributors package database.

With the provides-items, we can also scan software to detect it’s dependencies automatically (and have it in a distro-agnostic form directly). We can also easily search for missing mimetype-handlers, missing kernel-modules, missing firmware etc. to install it on-demand, making the system much smarter in handling it’s dependencies. And users don’t need to do search orgies to find the right component for a given task.

Also on my todo list for the future, based on this feature: A small tool telling upstream authors which distribution has their application in which version, using just one command (and AppStream data from multiple distros).

Also planned: A cross-distro information page showing which distros ship which library versions, Python modules and application versions (and also the support status of the distro), so developers know which library versions (or GCC versions etc.) they should at least support to make their application easily available on most distributions.

As always, you can get the releases on Freedesktop, as well es the AppStream specification.

April 19, 2014

Hi,

this is just a quick bug fix release, the last one depending on PackageKit 0.8 series, it doesn’t have new features apart from having some fixed support for appstream 0.6.

For the next 0.8.3 version PackageKit 0.9 will be required.

For more information you can look at the git logs.

http://download.kde.org/stable/apper/0.8.2/src/apper-0.8.2.tar.xz.mirrorlist

Have fun :)


February 12, 2014

FOSDEM is over, and it has been a great time! I talked to many people (thanks to full devrooms ;-) Graph Databases and Config management was crazy, JavaScript as well…) and there might be some cool new things to come soon, which we discussed there :-)
I also got some good and positive feedback on the projects I work on, and met many people from Debian, Kubuntu, KDE and GNOME (haven’t seen some of them for almost 3 years) – one of the best things of being at FOSDEM is, that you not only see people “of your own kind” – for example, for me as Debian developer it was great to see Fedora people and discuss things with them, something which rarely happens at Debian conferences. Also, having GNOME and KDE closely together again (litterally, their stands were next to each other…) is something I missed since the last Desktop Summit in 2011.

My talks were also good, except for the beamer-slides-technical-error at the beginning, which took quite some time (I blame KScreen ;-) ).

In case you’re interested in the slides, here they are: slides for FOSDEM’14 AppStream/Listaller talks.

The slides can likely be understood without the talk, they are way too detailed (usually I only show images on slides, but that doesn’t help people who can’t see the talk ^^)

I hope I can make it to FOSDEM’15 as well – I’ve been there only once, but it already is my favourite FOSS-conference (and I love Belgian waffles) ;-)

February 11, 2014

Three months ago Custelyst was started,

it was somehow a proof of concept to see if I could build something that could be used in the real world, but the more I progressed on getting all the pieces together the happier with the overall result I got. And the initial result is here today!

I have made a few benchmarks comparing some simple pages delivery using Django (python) and Perl Catalyst, and overall the time to first byte was around 3 times faster than both, comparing RAM usage it was like 2MB vs 50MB from the Perl version and 20MB of the Python one, the CPU time was also much smaller if compared to both which probably means it could handle more requests, but this isn’t a completely fair benchmark (as always) because there are several other things to measure…

Below I’ll try to clarify some doubts raised from my last post and what makes this new Web Framework unique:

But there is already QDjango and Tufao…

I didn’t know about these ones when I first started Cutelyst, but after taking a look at them, both misses a templating system (or integration with one), I also didn’t like the way one would register actions to be dispatched, IMO the way it’s done in Cutelyst is easier and nicer. And they doesn’t count with WSGI integration, they have their own HTTP parser/server. I also didn’t like looking at code :P

Why not DJango or any other well established framework out there?….

Apart from QDjango and Tufao there are no other (AFAIK) Qt based frameworks, there are indeed some C++ ones that I knew before but I didn’t like the way of doing things there, in fact when I started with Perl Catalyst I really liked how it was written. So in short I don’t know well enough other languages and I have no plans wasting my time learning other languages, I’d rather try to master at least one (although I know I’m far from it).

What’s in this release?

  • Glad you asked (duh!)
  • Production ready integration with uWSGI, meaning Cutelyst has a uWSGI plugin that will load your application (which will also be a plugin), and thus thanks to uWSGI it will have out of the box support for FastCGI, HTTP(s), uWSGI (protocol) and probably some other.
  • ClearSilver templates integration, this templating system is written in C and it’s amazingly fast, but it’s also incredible limited to what you can do in the view.
  • Grantlee (Django) templates integration, it’s used in KDE PIM to allow for easy templating of HTML, it’s slower than ClearSilver but offers much more fexibilities on your views, and since it’s also written in Qt the introspection comes for free.
  • Complete Path dispatcher, which means if you Port a Perl Catalyst Application that was using Path actions it will work as expected. This dispatcher matches an url to a Q_INVOKABLE method depending on it’s signature, The Catalyst Chained dispatcher is not implemented yet.
  • Session plugin, easy to use way of persisting data across requests, right now it’s done with a QSettings file, but integration with Redis and MongoDB are planned.
  • Authentication plugin, allowing you to authenticate your users using different Realms/stores, currently a StoreMinimal is implemented which can be used mostly for testing purposes (ie on the code you add your users)
  • StaticSimple plugin, allows for serving static content reducing the complexity of serving static content while developing your app.

If you are concerned about API/ABI stability I don’t promise it just right now it’s almost stable, probably 0.2 or 0.3 versions will have a promised stable API.

And yes, I have just put it on production http://serial.igloobox.com.br is running it right now (but the site is user restricted).

My plans now is to setup cutelyst.org with docs and examples, as well as writing a web blog, bug tracker and wiki based on it.

Have fun!

Download


January 26, 2014

After years of missing this conference, I will finally make it to Brussels this time!

I will even have some talks, one about Listaller (in the lightning-talks section) and one about AppStream and Listaller in the Distributions track. The lightning-talk will explain why distributing (some) applications independent from the distribution makes sense, and how Listaller does this. I will also very briefly talk about the concepts behind Listaller and which tools it offers for application developers to create & package cross-distro apps.

The AppStream & Listaller talk will be much more detailed. It will cover the rationale for AppStream, what AppData is good for and how AppData files relate to AppStream. I will also reflect the AppData adoption in GNOME/Fedora and why GNOME-Software is the right way to go forward in developing software-centers. It will of course also include our future plans for AppStream.

On the Listaller side, I will talk about how Listaller is connected to AppStream and PackageKit and why distributions should ship a way to install cross-distro packaged apps at all. I will explain module-definitions and why they are useful. An overview of the internals of Listaller and it’s system integration is also included, as well as how it can be compared to competing installation solutions.

If you are at FOSDEM and have questions about AppStream/PackageKit/Listaller/Tanglu/Debian/etc., please ask them! ;-) See you there!

FOSDEM

January 19, 2014

While writing this post, I recognize that we might need a news-blog for tanglu logo pureTanglu :-)

Since yesterday, we have an (installable!) Beta2 release of Tanglu!

Compared to the previous snapshot, it comes with a huge amount of improvements and bugfixes:

  • The systemd journal is now used for logging exclusively. You can install rsyslog to get the old behaviour back, and switch the journal to non-persistent mode. If you want to test the journal, type “journalctl -b” to get all messages from last boot. If we experience issues, we might revert to rsyslog for the final release.
  • KDE received a new theming for KDM/KSplash, so we can now show a face-browser in KDM, which is considered more user-friendly. User faces are now loaded from the user’s settings, instead of system directories by default. If you want to load a large user list (> 100) over network, you might not want this, but in that case you can easily revert the change.
  • Tanglu does no longer include an MTA by default. If you want one, install one and configure it. On desktop systems, having an MTA makes no sense, especially when it is not properly configured (we follow Ubuntu/Fedora here).
  • We ship with a functional live-installer now! You can install Tanglu easily to try it out. Unfortunately, the Debian installer is still not working, so you will have to boot into the live session to install Tanglu.
  • GNOME is now working. GDM and g-s-d will no longer crash, and Mutter is able to utilize llvmpipe for rendering in a virtual machine. The Shell and various other GNOME parts were updated to GNOME 3.10
  • We now ship the 1.14 version of Xorg now
  • Linux 3.12 is now default

Known issues:

  • People with ATi graphics cards might have trouble with installing the proprietary fglrx driver
  • In some very rare cases, the live-cd won’t auto-login. In that case, just reboot and try again, or use the password “live” to access the account.
  • Some packages are not installable, there is a list compiled here, with detailed info why they are broken (they will be removed from the release if they are not fixed) Some transitions are still pending (causes broken/not-coinstallable packages)
  • Users with a non-English locale will have an additional “Public” folder in their home-directory on KDE
  • We don’t support UEFI at time, please use the legacy mode to boot
  • We just noticed that the GNOME images won’t start the GNOME-Session for some reason. [UPDATE:21.01]: We published new live-cd images which have this issue fixed.

You can download the release from one of these mirrors:

Mirror1Mirror2Mirror3

Additionally, there is now a release schedule for Tanglu Aequorea, and we will do a vote on the name for the next release soon (proposed names).

As a note to Debian: systemd is working pretty well for Tanglu so far ;-)

I am pretty happy with this Beta2 release, because Tanglu is shaping out to be the distribution we imagined in the beginning.

Have fun! And as bonus, here are some images from Beta2:

tanglu-beta2-kde0

tanglu-beta2-kde1 tanglu-beta2-kde2

 

 

 

 

tanglu-beta2-gnome2tanglu-beta2-gnome1

 

 

 

 

 

 

November 04, 2013

appstream-logoWhile reading stuff posted by others about AppStream, and because of the discussion happening about AppData on kde-core-devel right now, I feel the need to clarify a few things. Especially because some are implementing AppStream in a way which is not really ideal right now. This is to some extend my fault, because I should have communicated this in a much more visible way.

To those people who don’t know it already: AppStream is a Freedesktop project aiming at providing basic building blocks to create distro- and desktop-agnostic software centers.

So, let’s answer some questions about AppStream!

Is AppStream GNOME or $distro specific?

No, not at all! It was originally created by people from at least 4 different distributions, and I took great care of it not becoming specific to any desktop or distribution. GNOME just happened to go ahead and implement the specs, which was absolutely necessary, since there was less progress in implementing AppStream for a long time.

How does AppStream actually work?

AppStream is a bunch of things, so I will only focus on what we have specified right now and what is working.

Basically, the distributor compiles a list of applications available in the repositories, and makes it available in some defined directories on the system. AppStream defines an XML specification for that, but since some peple don’t want to use it or can’t use it, there are also others ways to publish AppStream application data. For example, Debian will likely use YAML for that.

This data is taked by the AppStream software (running as a PackageKit plugin) and transformed into a Xapian database. This database is then in turn used by software-centers, in combination with PackageKit, to present applications.

This is the reason why it is bad to use the XML data directly – it might not be available on every distribution. The Xapian database is what matters. The database can be accessed using libappstream, a GLib based library (so far, there was no need for a Qt version).

Why is GNOME using the XML data directly then?

The libappsream stuff was under heavy construction, and GNOME wanted to be fast and ship the stuff with Fedora in their next release. They’ll likely switch to the Xapian db soon, or offer it as backend.

Is AppStream already used in KDE?

Yes, Apper can utilize it, see one of my previous blogposts.

What is AppData?

AppData is an initiative started by Richard Hughes in order to enhance the quality of applications descriptions shipped with AppStream. It defines a small file $appname.appdata.xml, which describes the application, sets screenshots etc. These files can be parsed at the distribution’s side in order to enhance the app metadata. They can also be translated upstream.

AppData might be merged into the XDG AppStream pages later, but that is to be discussed.

Are application authors forced to ship AppData files?

No, nobody is forced ;-) However, the GNOME Software application-center prefers applications shipping more metadata, and “punishes” the others, so shipping an AppData file makes sense for higher lising in GS. This is a policy decision by GNOME, KDE can make it’s own ones here.

Shipping AppData files makes sense, in general, because it enhances the metadata distributed with your application. It is also the file-format used by Listaller (well, a superset of it) in order to generate cross-distro app-packages, so you might want to consider adding one.

Are there any rules for the AppData files?

Yes, you can find them on the AppData specification page. However, these are more recommendations than “forced”, and it is currently aimed at GNOME apps. I later want to generalize that and create an own page with recommendations for KDE (martin had some good ideas already).

Where do software-centers get the screenshots from?

The screenshots are defined in AppData files, and the cached by the distributor. If there are no “official” screenshots, user-provided screenshots will be taken, using a screenshots-service with screenshots.d.n-like API.

Where do I find the specs?

The official AppStream spec, the thing which distributors should implement, can be found on Freedesktop.

The AppData spec can be found here. It also includes some nice hints on how to handle screenshots etc. and includes it’s own FAQ.

Where do I find libappstream?

The source code can be found on Gitorious, releases are available on fd.o software. Contributions to the code are welcome!

I want to change X in the spec, because of argument Y which brings improvement Z

Great! Please get in contact with me or Richard. The only feature we would not consider for the official standard is desktop/distro-specific stuff (which should be obvious ;-) ).

I will extend this FAQ, if I feel the need for it, so this article might change a bit.

November 01, 2013

Do you recall that “Qt everywhere” branding?

Well that’s one of the things I really like in Qt, being able to deploy everywhere with minimal to zero changes in source code. But sometimes your application needs a web interface some sort of web backend or web admin, and when that happens it breaks that slogan (AFAIK)…

I got in contact with web languages (not HTML) only when at university, ColdFusion and PHP where the first ones, and while PHP was sort of cool because it has a syntax close to C, I never really like it. I’m not a web designer nor know how to do pretty websites, but in today’s world building web apps is inevitable.

So years ago I learned about RubyOnRails, and people where talking all the good stuff about it, it’s was the framework of the hype, although I knew Ruby wasn’t the fastest scripted language my need didn’t demand that, so I got a book, read it cover to cover and happily started my web app, a week latter I got a complex problem with an apparently impossible solution. It seemed that RoR makes easy things stupidly easy and complex things impossible…

The I met Catalyst a Perl framework, got exited again, but I knew nothing about Perl so I read more 3 books and was started to love the language, I still do, but last weekend when trying to resurrect and old Catalyst application I got an error which I had no idea how to fix, on IRC I was told the problem could be that I mixed CPAN packages with distro packages, not very convinced I realized the bigger problem I was actually facing, I’m not fluent in Perl, well no matter how many books you read about a language you only get fluency in it after speaking it almost everyday, this applies to computing languages too of course.

So if 99% of the time (or less now with QML) I’m programming C++ it means that I know a lot more about how to debug it than with Perl, so Cutelyst! Writting web apps in Qt/C++ will be much more efficient to me and hopefully to you too.

I share lots of the Catalyst API and design ideas, tho the implementation is of course completely different. Also for the sake of HTML templating I plan to create a Grantlee View plugin.

You can take a look at the proof of concept:

https://gitorious.org/cutelyst

Have fun.


October 28, 2013

Long time with no article about Tanglu! This was mainly because we were busy with the project itself, improving various aspects of the distribution.

So, here is a new summary on what has been done and what you can expect from the first release ;-)

Tanglu QA

We further improved the automatic archive QA. There is now qa.tanglu.org, which constantly monitors the number of uninstallable or unbuildable packages in the Tanglu suites. It also provides tanglu-platform-smallstatus information on the metapackage generator, which helps us in finding out which packages are available on the live-cds. Furthermore, information about the staging->devel migration process is provided, to answer the question why a package does not migrate (this still needs some improvements, but it is being worked on).

We also use some code from Ubuntu to monitor package versions in Debian and upstream, which helps to see if others have released newer versions of software shipped with Tanglu.

This already resulted in many improvements: The Tanglu Aequorea suite does not contain unbuildable packages (at least not due to build-dependency changed), and all live-cds are working well.

We will soon migrate the archive to a new server, which frees some server capacities we can use for automated QA and things like automatic live-CD building.

Live-CDs, Installer, Alpha-Releases

We currently don’t do Alpha-Releases of Tanglu, but we create Live-CD snapshots of Tanglu, which are available at releases.tanglu.org (or mirror1, mirror2). These snapshots still have issues and are just early previews. They also ship without and installer, we are still working on that part. Please note that “CD” more or less means DVD or USB-Stick right now (and this won’t change – the expected image size will be around 800MB).

Release Planning

I am happy to announce that we will do a release this year, most likely in December. But what can you expect from the release?

KDE 4.11kde-konqui

We will ship with KDE 4.11, which will be the only desktop we officially support so far. The reason is simply lack of manpower – we could promise to support more, but that would just be not realistic for the small team. So we focus on KDE (Plasma Shells) right now, and try to make it awesome. Also, the team consists mostly of KDE people right now, which contributed to that decision ;-).

If you want to try Tanglu, right now the KDE live-images are the best to try it out.

GNOME 3.8

gnome-logoWe will also provide images with GNOME. The problem with GNOME is, that the GNOME team does not have enough manpower to maintain the whole desktop or to upgrade it to the latest version (it is essentially just me looking at it from time to time). So GNOME will be available in a “preview” state. We invite everyone with GNOME knowledge to join the project and help improving Tanglu-GNOME – GNOMErs, we want you!

systemd >= 204

We ship with systemd by default, which works nicely so far, although more testing needs to be done. The logind service will be used to replace ConsoleKit, if we manage to get everything in place and working in time (if there are issues, we might switch back to CK for one release). There are some plans to use a higher systemd version, due to some improvements made there, but if this will be done is still unclear (Debian will most likely stick to 204 for some time, because with systemd > 205, running it as pid 1 will be mandatory to use logind (and Debian is just in the process to decide which init-system we will use there)).

Systemd will run in SysVInit compatibility mode for most of the available services. This will improve in later Tanglu releases. Of course, systemd is usable, even if not every init-script has been converted to a service file. It just has an impact on startup times, so Tanglu will not be the distributions with the fastest startup times (yet ;-) ).

Debian Package Mixdebian_logo1

Tanglu consists mostly of packages from Debian Testing (Jessie), but we take full advantage of the Debian archive, so you will also find package versions from Unstable or even Experimental (where it made sense). A very small portion of packages has also been merged with Ubuntu. Although stuff has been changed, the incompatibilities with Debian are almost zero, so if you are installing Tanglu, it will currently feel like an more up-to-date Debian Testing with some extras.

Still, the differences are large enough that upgrading a Debian system to Tanglu might result in some issues.

Installer

Right now, the installer is a major field of work. Tanglu will most likely ship with the Debian-Installer, because it is the easiest thing to do right now.

For later releases, it is planned to also offer the Ubiquity installer (the thing Ubuntu uses), or a new installer with a similar UI and concept.

Other Cool Things266px-Wayland_Logo

Tanglu will ship with a fully working Qt5 (which is currently being tested and updated) and the latest version of Wayland/Weston as a preview for developers to play around with.

We also ship with Perl 5.18 and Haskell GHC 7.6, as well as with GCC 4.8 as default compiler (although the whole distribution does not yet compile with GCC 4.8). We might ship with Linux 3.12, but this also depends on other things. The Linux kernel build will be the same as in Debian. There might be more things to come :-)

Please do always keep in mind that Tanglu is a new project, so the first release might still have some teething problems – but we’re working on it. The Aequorea release will be supported one to two more months after the next release is out.

Tanglu Policy

I started to draft a Tanglu policy, which defines stuff like the procedures to become a Tanglu project member and/or developer, some basic values of the distribution, decision making processes etc. This work is at a very early stage right now, and will need lots of feedback later. But as soon as it is done, joining the project will be easier and what Tanglu is will be well-defined.

The policy will also include a Code-of-Conduct as an additional document from the start.

We need help! ;-)

First of all, thanks to everyone working on Tanglu! You are amazing! Also, many thanks to every Debian developer or contributor who helped a lot in setting the project up and contributed knowledge. And of course, thanks to everyone else who contributed by creating awesome artwork, helped with code, TangluTanglu Logo (big) archive mirrors, buildd-servers or by testing the distribution and providing feedback! (given the state the aequorea suite was in at the beginning, testing was a really frustrating activity – but people still did it!)

So, if you would like to help, just find us on #tanglu-devel on Freenode or join the tg-devel mailinglist. We also really need some people taking care of the Tanglu website and updating it with some recent information from time to time, so people can see what is going on. Before we have the Tanglu policy finished and the Tanglu members and developers directory in place (a software which allows us to track all registered developers and gives them an @tanglu.org mailalias), the start might be a bit confusing, but we do our best to make it easy for new people to join. The best way is asking people.

Tanglu is still created by an incredibly small team, which has a large task to accomplish. Help is welcome :-)

August 25, 2013

Tanglu Logo (big)Long time since I published the last update about Tanglu. So here is a short summary about what we did meanwhile (even if you don’t hear that much, there is lots of stuff going on!)

Better QA in the archive

We now use a modified version of Debian’s Britney tool to migrate packages from the newly-created “staging” suite to our current development branch “aequorea”. This ensures that all packages are completely built on all architectures and don’t break other packages.

New uploads and syncs/merges now happen through the staging area and can be tested there as well as being blocked on demand, so our current aequorea development branch stays installable and usable for development. People who want the *very* latest stuff, can embed the staging-sources into their sources.list (but we don’t encourage you to do that).

Improved syncs from Debian

The Synchrotron toolset we use to sync packages with Debian recently gained the ability to sync packages using regular expressions. This makes it possible to sync many packages quickly which match a pattern.

Tons of infrastructure bugfixes

The infrastructure has been tweaked a lot to remove quirks, and it now works quite smoothly. Also, all Debian tools now work flawless in the Tanglu environment.

A few issues are remaining, but nothing really important is affected anymore (and some problems are merely cosmetic).

Long term we plan to replace the Jenkins build-infrastructure with the software which is running Paul Tagliamontes debuild.me (only the buildd service, the archive will still be managed by dak). This requires lots of work, but will result in software not only usable by Tanglu, but also by Debian itself and everyone who wants a build-service capable of building a Debian-distro.

KDE 4.10 and GNOME 3.8

Tanglu now offers KDE 4.10 by default, a sync of GNOME 3.8 is currently in progress. The packages will be updated depending on our resources and otherwise just be synced from Debian unstable/experimental (and made working for Tanglu).

systemd 204 & libudev1

Tanglu now offers systemd 204 as default init system, and we transitioned the whole distribution to the latest version of udev. This even highlighted a few issues which could be fixed before the latest systemd reached Debian experimental. The udev transition went nicely, and hopefully Debian will fix bug#717418 too, soon, so both distributions run with the same udev version (which obviously makes things easier for Tanglu ^^)

Artwork

We now have a plymouth-bootscreen and wallpapers and stuff is in progress :-)

Alpha-release & Live-CD…?

This is what we are working on – we have some issues in creating a working live-cd, since live-build does have some issues with our stack. We are currently resolving issues, but because of the lack of manpower, this is progressing slowly (all contributors also work on other FLOSS projects and of course also have their work :P)

As soon as we have working live-media, we can do a first alpha release and offer installation media.

You!

Tanglu is a large task. And we need every help we can get, right now especially technical help from people who can build packages (Debian Developers/Maintainers, Ubuntu developers, etc.) We especially need someone to take care of the live-cd.

But also the website needs some help, and we need more artwork or improved artwork ;-) In general, if you have an idea how to make Tanglu better (of course, something which matches it’s goals :P ) and want to realize it, just come back to us! You can reach us on the mailiglists (tanglu-project for stuff related to the project, tanglu-devel for development of the distro) or in #tanglu / #tanglu-devel (on Freenode).

August 06, 2013

Some days ago I was listening the radio and a famous Brazilian comedian was saying how stupid it is to express you thoughts about things because latter you probably regret what you said, and realize how dumb you are… although I partly agree with him I decided to share this as maybe someone is facing the same problems I’m facing.

I don’t want to spread FUD, so if my tests/conclusion or everything I said is wrong please explain and I’ll update or drop the text entirely :P

Lately I’ve been working a lot on an Qt5/Quick2 application and I decided to share some thoughts about this new piece of tech. The application that I’m working on basically has to index >16k of videos and music files, and show their covers so the user can choose what to listen/see, it’s an app for jukebox machines. Xapian rocks!

First I need to say that I really like Quick2, the name is right about how fast can you write your app,  on Quick2 we have some very cool graphics effects like blur, shadows and others, and those are quite interesting for this kind of app. I can blur the background, draw shadows on the CD covers and so on. The multimedia part is also very easy to use, so how to play a video or music file is not something you worry about (tho I need to find a way to play files that I encrypt stupid recorders).

Most people also probably heard about Scene Graph which is the new rendering of QML 2 which uses OpenGL (EGL iirc), this brings many improvements on performance and quality, and expands the QML2 possibilities.

However maybe you are considering it for your own projects, the QWidgets vs QtQuick is something many developers don’t know the answer yet. To me it’s now a bit more clear on what to choose.

Every now and then I listen people saying that OSX is good because it’s software made for the hardware, and while in parts I agree with that the same is valid for Quick2. Martin recently wrote an interesting blog about “what makes a lightweight desktop lightweight“, which describes the problem of properly describing a technology.

My point is that with Quick2 we need to set a barrier between legacy and supported hardware, to me this is quite complicated because you can have some very powerful CPUs from like 8yo coupled with very slow GPU, and brand new hardware with the opposite configuration, Raspberry PI for example has a rather slow CPU with a great GPU.

So while QWidgets/Quick1 performed quite well on those old machines Quick2 doesn’t. To exemplify I have a computer to test my jukebox application that has a crap VIA graphics with a 3.2 intel D CPU. When I went to deploy/test my super cool and animated application there, I almost cried, after removing blur/shadow/transparency effects I could get a maximun of 10fps on fullscreen with 1024×768. Toggling the composite switch didn’t make any difference (nice work KWin team), and using other Window Managers also didin’t improve the poor experience.

Aaron give me a hint about llvmpipe, which is something I had heard but never truly understand what it was, basically llvmpipe is a software render for this exactly case where the GPU can’t handle the OpenGL instructions, but after some investigation it seems all new distros have it to render on these poor GPUs. Now that I knew about it I went to look at CPU usage and it was pretty high and leaking…

Before you jump on the wrong conclusion that Quick2 is bad please note that for any hardware capable of OpenGL it’s an excellent choice because it can offload the rendering stuff from your CPU to the GPU. And this is where we have a new era of “lightweight” of desktops as they use less CPU.

Something that does bother me is that Plasma 2 will need newer hardware specs, I won’t be able to run KDE SC 5 (or whatever it will be called) on such hardware, I went on thinking that it would be nice to have something like lxde-qt as our QWidget shell and still use Qt5 with our KDE stuff. On the long run I believe we won’t be able to share much of the GUI apps as Quick2 will probably become more and more common, not to mention it’s a nicer way of developing GUI stuff.

And if you wonder what I’m going to do, I not totally sure, but right now I have 2 user interfaces on the same binary, one QWidget based and another Quick2. I thought about going back to Quick1/Qt4, but then I wouldn’t have the cool Quick2 stuff. Thanks to the model based approach I’m sharing most code between the two interfaces. And yes, since QWidget on Qt5 still uses raster rendering it provides a much more responsive experience for the application.

I hope this gives you some clarity if you might be targeting hardware like that, in the future I hope to be able to deploy on Raspberry PIs but the lack of RTC doesn’t make it an option right now.

If you know some magic way to improve FPS please let me know, I’d love to have a single GUI :D


July 30, 2013

Hi all,

I finally managed to roll a new release of Apper :)

Here is a small list of changes:

  • AppStream support (experimental)
  • Updater plasmoid now as a pure QML applet, with some nice improvements
  • Don’t create KJobs for when simulating the updates
  • Allow for fetching showing screenshot from openSUSE servers
  • and several bugfixes

This release is almost a bug-fix only one, more features are planned but I lack time right now…

Here is the download link:

http://download.kde.org/stable/apper/0.8.1/apper-0.8.1.tar.bz2.mirrorlist

Enjoy.

PS. My wife is back at home, thank you all who donated a bit for Litteras campaign, but it is still far from reaching the goal, and I believe it won’t reach it. As I said I’ll continue it’s development but not the way I planned since I’ll need to manage other ways to fund myself.


July 25, 2013

I don’t even know how to start this one but wow, when I think my life can’t get worst it surprises me again.

So last month I started a campaign in Indeigogo  about Litteras a new email client with EWS (Exchange) support. It was great, in like 2 days I had 5% of the goal, but then it stuck completely, that was probably because I failed to blog more about it and it’s progress, but it also showed how I was wrong on thinking people dislike Akonadi, I got lots of feedback of users that like it, I gave it a second try and it was more or less what I had experienced in the past. It worked with my Gmail account but still I don’t like it much, I was able to make kmail unusable by killing the mysql process, I should fill a bug but lots of things happened after this.

One of the greatest things of the campaign was that I was informed about KDSoap which is not on “Qt Soap”google search, right now I didn’t start to use it but it will help a lot developing the libews-qt library, the API of the lib is quite nice to use, and it is a bit close to the .Net one but async :)

Soon after the campaign stopped receiving funds I got a proposal to write a software for an specific machine, the software isn’t that hard to do (it’s almost done) and it will allow for me to finally run my own company (which I plan to have a few paid people making KDE stuff at certain hours). Then I got my vacations and went to meet this people so soon I expect it to be selling.

My vacations finished, thanks God last month I managed to get rid of my bank debts (still have my car and the German lawyer to pay), my wife was to go to Argentina to deal with the issue that happened in Germany but at the beginning of the year she felt over her feet and broke it, she got some metal in it and till today she says it hurts sometimes. I was counting now I’d had money to pay her trip and the German/Argetinian lawyers and it happened again…

She was with my kid shopping some stuff and with my kid letting her crazy like most kids she mistakenly put some stuff she was buying on the baby car, and not to help that happened in two stores, she even called me if she could buy what she wanted and I said ok. But then the woman on the second store started to scream loudly and the mess was made. She was waiting for me to get out of work to go to the first store and buy the stuff.

The police was called and since it had things from two stores she had to stay arrested. Impossible not to remember when it happened to me in Germany but I’m sure here is probably worse, not to mention we are having the coldest days this year (it even snowed at some places), luckly I had some cloths to leave with her.

Now I’ll have to spend 5k USD with the lawyer and probably some more cash for the police, I’m glad tomorrow I get my pay check and next week my vacations, but still it’s a lot of money so it isn’t going to be easy to deal with this.

Most of the people that read my blog probably don’t believe in God, I have my own logical and non logical reasons to believe and I really feel that something is trying to set my family apart, in two years after my daughter died we were finally starting to love each other again tanks to this vacations, debts gone and lots of talking, and now this… If that’s not evil surely it’s real bad luck..

So I would really appreciate if you could help us (again! thank you very much if you did it the last times) a bit on this and donate a penny for that campaign, due to the feedback I realized that not supporting akonadi was a bad decision, so my plans for it now are:

* Make libews-qt have it’s own repo

* Make libews-qt use KDSoap

* Cover more of the EWS API

* Implement an Akonadi resource for managing contacts

* _MAYBE_ Implement an Akonadi resource for managing calendar, if the above proves to be as easy as the docs say

This is already lots of work, and I plan to do this as soon as I have my company up and running (hopefully in 1mo maximum), since we are moving to Qt5 it will probably make more sense not to promise more stuff since some stuff might change, if everything goes well I’ll also try to make use of the lib in KMail, but I don’t know how long would it take so I prefer not to promise that, the contacts I know I can manage to implement it. And btw folder syncing is mostly working on litteras already (not the emails in it).

If you can’t help with money please pray for my family or just wish us luck.

Thank you.


June 03, 2013

Before you jump to the comments section and start a flame war, please let me give you a little ground.

Two years ago when I got back to Brazil I went to the same job but hired by a different company, the new company uses Microsoft Exchange as the mail solution. No matter how hard I could try it’d almost impossible to think they would ever move away from it, I’d need to convince people that I don’t know personally which are also in another continent…

I believe I’m not alone in this, so, instead of dreaming with the IMAP day I decided to take a look at what email clients could talk to Exchange, at that time there was ZERO Linux email clients supporting it, and to be fair I’m not talking about MAPI, which is another protocol used to communicate with Exchange but it’s disabled/blocked on my company and in many others.

There is some program that takes the OWA (Outlook Web Access) and convert it to IMAP, but I didn’t like the idea nor had the will to setup it as it looked complex. So for some time I just gave up and used the webmail, but it’s really bad, especially in  2007 version it doesn’t auto refresh the page, and even if it did it wouldn’t notify me about new emails.

So when my boss told me he could easily setup it on his OSX Mail I got intrigued, how could it talk to exchange if MAPI was not an option? After some research I became aware of EWS (Exchange Web Services), which is a SOAP specification to talk to Exchange using HTTP. I then tried to use gSOAP library to auto generate the interfaces and code to talk to it (as I have used it before for another SOAP project), but as soon as the code was also linked to any KDE library I got some DSO weird error from the linker… I tried to find how to fix this linker issue but couldn’t get help or find a solution.

I then sort of gave up and time passed again, and Evolution Mail got EWS support, but it also didn’t work with my company setup, no idea why, but still it didn’t work. Recently a new version of it started to work with my company’s server, and I started using it, but well besides the fact that it’s a GTK app it doesn’t work well for me, it’s slow, the address completion is quite useless…

So time to give Litteras another try, but wait! Why not give KMail EWS support instead of the yet another email client?

UPDATE EDIT:

To put it simple: I don’t like MySQL on my servers (you can imagine how I fell about it on a desktop), even if it was PostgreSQL I simply think it’s wrong to store my mails on a SQL database. Granted KMail works great for lots of users, but I myself don’t like the underground tech, it’s probably much more a matter of taste.

So, KMail developer are telling me the emails are stored as regular files which is something I do know. But then there is a dependency with akonadi, which is the one that can use MySQL, sqlite… So to not spread FUD I’ll try to put it another way, I myself while trying to use it didn’t find a way to avoid akonadi, and I saw lots of other people not being able to do it too. Every place I look for information it says that Akonadi will cache the email information to be easily retrieved in other places, so it’s not the same as storing the emails on the data base. Still I myself (as mentioned earlier) don’t like this idea much. If one can still disable akonadi I actually find hard to believe it’s possible since all information I have found is that right now Akonadi has the resources to fetch the emails. So basically: continue using what you have been using and is working for you, in my taste I just don’t like, and that’t the reason why not everyone uses GTK or Qt or PostgreSQL, there are sometimes technical reasons, but it’s not entirely the case here, MySQL might perform well for this use case but past experience with it on servers gave me a trauma.

PS: I hope I clarified this part, as I knew it would be hard to explain it’s a personal matter (tho I ended the line stating this), and I needed to state my reasons for not willing to go the KMail/Akonadi way right now as that would be 1st question.

—————-

BUT If you like KMail and would like EWS support be happy. I’m building an EWS Qt library so that this will benefit any KDE/Qt developer willing to write a yet another mail client, and adding this kind of support for KMail should be much easier when the library is in place, I could even try to do it myself.

So what about Litteras?

Litteras already is a KDE application (as it uses KDE libraries), and I want it to feature EWS, POP3 and IMAP support. Locally I plan to store the emails in MailDir format, and index them with Xapian. I also want it to feature a clean user interface and most importantly do lot’s of magic when setting up an email account.

litteras

Right now you can just type your email and password and it will find the EWS server if it was deployed following Microsoft specification, it’s actually even better than EWS than Evolution in this regard right now as they didn’t do DNS SRV search to find the right server (which is the case needed for my company). It also download your folders and messages but doesn’t get the body yet, nor store then locally.

Email composer...

Email composer…

Now let’s do business :D I want your support:

  • Do you want EWS support using Qt technologies?
  • Want a library to make it easier for KMail use it?
  • A different KDE/Qt mail client?
  • Even Plasma Active/Tizen could benefit from this Exchange support as not having Exchange support might be a no go for many users.

Here is the indiegogo campaign to support two months of development, an amount of 6500 USD. Details on what I’ll do can be found there.

Go grab the code and hack it if you want: https://gitorious.org/litteras

Thank you.


May 29, 2013

For those who doesn’t know print-manager is a project I started back in 2010 (yes it’s 3 years old now!), but only got included into KDE 4.10 (the current stable release). The reason for that is that since it was meant to replace system-config-printer-kde it needed to provide at least the most used features, and that was only possible as the logic to find a good PPD for a given printer was then exposed through DBus, so we could use that but not it’s GUI.

I’m very pleased to say that my expectation of receiving bug reports of missing features, surprised me by being just one of the few bugs it got (sadly it’s not the easiest to one to fix). The number of bugs was also quite low and it’s acceptance quite positive, which helped me managing fixing most of them then in a short time. Currently there are only 2 standing bugs, and none of them are crashes (all were fixed), one is a missing convenience feature and the other is half fixed but I failed to setup a similar environment to figure out what was happening, but this next version will include some debug info to try to figure that out.

What about new stuff?

So CUPS is not a new project that earns features everyday, actually we fear to have more features removed, that being said print-manager already covers most of it’s functionality. One CUPS feature I didn’t make use in 4.10 was the ability to reprint canceled/completed jobs, this feature also only works if the job is preserved which is something that can be disabled on the server. Of course we only enable the button if the job can do that: p-m-queue-reprint Notice the Reprint button, now there’s something even cooler than that, the last column “From Hostname” is also new, but the coolest part of it was that this was actually a feature request from a KDE 3 (kprint) bug, funny enough I fixed it on the exact same day were it completed 10 years :D (I accept cookies…) Next we have a bunch under the hood changed and fixes.

When I first presented the print-manager plasmoid it would have a feature to configure which jobs to show. Having “Active jobs”, “Completed jobs” and “All jobs” ended up becoming an issue, since if the completed job list had like 20 entries the plasmoid was taking several seconds to load, so since the “Active jobs” is always the smallest list I removed the other options to avoid bug reports. Latter Apper gained a plasmoid and I run into the same slowness very quickly, actually a list of 200 updates (which is not uncommon) was taking half a minute to load.

My previous investigations showed the Plasma DataEngines were too slow, in fact I believe it’s the mapping between the DataEngine and QML since plasmaengineexplorer is not _that_ slow. This was a no go for Apper, so I created a hybird C++/QML plasmoid, and quickly I noticed that this would also be the best thing for printer manager. It’s not only is blazing fast now, but also has the important benefit of not keeping four models implementations as the QWidgets dialogs already had them, so now I fix one model and both plasmoid/KCM/print-queue get it. p-m-plasmoid-configIf you have used the 4.10 version you will also notice an important improvement on the above, instead of a weird LineEdit to select which printers the plasmoid will display, now you have a nice checkable list.

Bonus points to Kai Uwe Broulik that added the second icon there (and did some more stuff) which is the full System Settings printers module so you don’t need to open system settings. And yes I’d like to have a different icon for the first option but I failed to find the option (if it even exists) to do that as Plasma uses the plasmoid icon for the first item. The internal library that talks to CUPS also got several improvements and fixes which made it Qt5 safe, due to QHash getting items stored randomly (yes Qt4 says we can’t rely on the order but we know we can :P). To be honest I don’t really know how well this worked with the mess it was… The plasmoid also got several improvements:

  • It’s fully usable from the keyboard now
  • The highlights/states work more reliable
It isn't empty anymore if no printer is configured

It isn’t empty anymore if no printer is configured


You get this if you filter some printers out and they get removed

It detects if CUPS has stopped and show a proper error message

It detects if CUPS has stopped and show a proper error message

And finally the plasmoid full of printers and a few jobs, with the new NIHSwitch, fully draggable and with I/O visual to avoid confusing, done my real world testing and so far nobody got confused:

print-manager plasmoid in KDE SC 4.11

print-manager plasmoid in KDE SC 4.11

Until KDE Frameworks 5 isn’t released (and probably also packaged), the development of print-manager will continue in the next SC 4.x releases, of course the list of TODOs is quite small and if you are willing to give a hand send me an email.

Enjoy.


May 12, 2013

I was asked by some people to write a status report about the whole PK/AS/LI stuff – sorry guys that it took so much time to write it ;-).

PackageKit

(PackageKit is an abstraction layer for package-management systems, allowing applications to access the package-manager using simple DBus calls)Package (with list)

PackageKit is an incredibly successful project. With the 0.8.x series, it received many performance improvements, and has now the same speed on my computer than the distribution’s native tools. PackageKit is used in almost all major Linux distributions, except for Ubuntu[1]. But even Ubuntu has written some compatibility layer, so most calls to PackageKit will work.

The only major distro where PackageKit is currently not available, seems to be Gentoo (and I am not sure about the shape of the Gentoo PackageKit backend too).

Debian Wheezy includes PackageKit by default, and in Jessy we are going to replace some distribution-specific tools with PackageKit frontends (mostly the old and unmaintained update-notifier and Software-Updater – no worries, we are not going for a Synaptic replacement ;-) (currently this won’t be possible with PK anyway))

Unfortunately, some PackageKit backends are still not adjusted for the 0.8.x API and are only running on 0.7.x. This is bad, since 0.8.x is a huge step forward for PackageKit. But the situation is slowly improving, with the latest OpenSUSE release, the Zypper backend is now available on 0.8.x too.

Being able to run a PackageKit from the 0.8.x series is a requirement for both AppStream and Listaller.

AppStream

(AppStream is a cross-distro effort for building Software-Center-like Software Center Logoapplications. It contains stuff like a screenshot-service, ratings&reviews etc. The most important component is a Xapian database, storing information about all available applications in the distribution’s repositories. The Xapian DB is distro-agnostik, but distributors need to provide data to fill it. AppStream offers an application-centric way to look at a package database)

The AppStream side doesn’t look incredibly great, but the situation is improving. As far as I know, OpenSUSE is shipping AppStream XML to generate the database. Ubuntu ships the desktop-files, and I am working on AppStream support in Debian’s Archive Kit. On the Fedora side, negotiations with the infrastructure-team are still going on. I haven’t heard anything from Mageia and other AppStream participants yet.

Unfortunately, at least for OpenSUSE, the AppStream efforts seem to be stalled, due to people having moved to different tasks. But efforts to add the remaining missing things exist.

On the software side, Apper (KDE PackageKit frontend) has full support for AppStream. Apper just needs to be compiled with some extra flags to make it use the AppStream database.

On the GNOME-side, GNOME-Software is being developed. The tool will make use of the AppStream database, on distributions where it is available.

Also, a Software-Center for Elementary and other GTK+-based desktops is being developed, which is based on AppStream (already quite usable!).

Using the Ubuntu Software Center on not-Ubuntu-based distributions ist still not much fun, but with the AppStream database available and a working PackageKit 0.8.x with a backend which supports parallel transactions, it is possible to use it.

On the infrastructure side: I recently landed some patches in AppStream-Core, which will improve the search function a lot. AppStream-Core contains all tools necessary to generate the AppStream database. It also contains libappstream, a GObject-based library which can be used to access the AppStream database.

Also, we discuss dropping PackageKit’s internal desktop-file-cache in favour of using the AppStream database. If we do that, we will also add software descriptions to the AppStream db, to improve search results and to speed up search for applications. Because we have to deprecate API for that, I expect this change to happen with PackageKit 0.9.x.

As soon as the Freedesktop-Wiki is alive again and my account is re-enabled, I will create compatibility-list, showing which distribution implements what of the PK/AS/LI stuff, especially focusing on components needed for AppStream.

Only a few distributions package AppStream-Core so far. Although it is beta-software, creating packages for it and shipping the required data to generate the AppStream database would be a very huge step forward.

Listaller

(Listaller is a cross-distro 3rd-party software installer, which integrates into Listaller-LogoPackageKit and AppStream. It allows installing 3rd-party applications, which are not part of the distributor’s repositories, using standard tools used also for native-package handling. Everything which uses PackageKit can make use of Listaller packages too. Listaller also allows sandboxing of new applications, and uses an AppDir-like approach for installing software.)

Listaller is currently undergoing it’s last transition before a release with stable API and specifications can be made. Dependency solving will be improved a lot during the current release-cycle, making it less powerful, but working on all distributions instead. (Fedora always had an advantage in dependency-solving, due to RPM providing more package metadata for Listaller to use) This change was delayed due to discussing a possible use of ZeroInstall-feeds to provide missing dependencies with the ZeroInstall team. We did not come to a conclusion about extending the XML, so Listaller will contain an own format to define a dependency, which can reference a ZeroInstall feed. That should be a good solution for everyone.

All these changes will result in IPK1.2, a new version of the IPK spec with small changes in the pkoptions file syntax and huge changes in dependency-handling. The new code is slowly stabilizing in a separate branch, and will soon be merged into master.

The next Listaller release will be the last one of the 0.5.x series, we will start 0.6.x then. KDE currently has support for Listaller through Apper, which is enabled on a few distributions. In GNOME, optional Listaller support is being developed and will be available in one of the upcoming releases.

Currently, to my knowledge, only a few distributions package Listaller. This should improve, so it is easier for application developers to deploy IPK packages.

The upcoming changes in KDE and GNOME to build stable developer platforms will help Listaller a lot in finding matching dependencies, and for stuff which only depends on one software frameworks, installations should be a matter of seconds.

As you can see, lots of things are happening, and there is improvement in all components related to installing and presenting software on Linux machines. However, all these projects have a severe lack of manpower, especially AppStream and Listaller have the lowest number of developers working on the tools (at time, only two active developers). This is the main reason for the slow development. But I am confident that we will have something shipped in the next distribution releases. At least AppStream should be ready then.

[1]: I don’t blame Ubuntu for that – during the time they wrote an own solution, PackageKit did not have all the required features. (This situation has changed now, fortunately.)

NOTE: I might extend this post with feedback from the different distributions, as soon as I get it.

April 26, 2013

Hi,

this is short news mostly for packagers, but well I want to thank José Manuel Santamaría Lema which did most (if not all) of the changes in this release, he found a few bugs and extended the debconf protocol support. I didn’t manage to release this soon as I’m way too busy. His work was made when I got arrested in Germany last year so I’m quite late here…

Enjoy:  http://download.kde.org/stable/libdebconf-kde/0.3.0/src/libdebconf-kde-0.3.tar.bz2.mirrorlist

Next week I’ll probably do a new colord-kde release stay tunned!


April 08, 2013

Hello everyone! I am very excited to report about the awesome progress we made with Tanglu, the new Debian-based Linux-Distribution. :)

First of all, some off-topic info: I don’t feel comfortable with posting too much Tanglu stuff to Planet-KDE, as this is often not KDE-related. So, in future Planet-KDE won’t get Tanglu information, unless it is KDE-related ;-) You might want to take a look at Planet-Tanglu for (much) more information.

So, what happened during the last weeks? Because I haven’t had lectures, I nearly worked full-time on Tanglu, setting up most of the infrastructure we need. (this will change with the next week, where I have lectures again, and I also have work to do on other projects, not just Tanglu ^^) Also, we already have an awesome community of translators, designers and developers. Thanks to them, the Tanglu-Website is now translated to 6 languages, more are in the pipeline and will be merged later. Also, a new website based on the Django framework is in progress.

The Logo-Contest

We’ve run a logo-contest, to find a new and official Tanglu logo, as the previous logo draft was too close to the Debian logo (I asked the trademark people at Debian). More than 30 valid votes (you had to be subscribed to a Tanglu Mailinglist) were received for 7 logo proposals, and we now have a final logo:

Tanglu Logo (Text)

I like it very much :)

Fun with dak

I decided to use dak, the Debian Archive Kit, to handle the Tanglu archive. Choosing dak over smaller and easy-to-use solutions had multiple reasons, the main reason is that dak is way more flexible than the smaller solution (like reprepro or min-dak) and able to handle the large archive of Tanglu. Also, dak is lightning fast. And I would have been involved with dak sooner or later anyway, because I will implement the DEP-11 extension to the Debian Archive later (making the archive application-friendly).

duckWorking with dak is not exactly fun. The documentation is not that awesome, and dak contains many hardcoded stuff for Debian, e.g. it often expects the “unstable” suite to be present. Also, running dak on Debian Wheezy turned out to be a problem, as the Python module apt_pkg changed the API and dak naturally had problems with that. But with the great help of some Debian ftpmasters (many thanks to that!), dak is now working for Tanglu, managing the whole archive. There are still some quirks which need to be fixed, but the archive is in an usable state, accepting and integrating packages.

The work on dak is also great for Debian: I resolved many issues with non-Debian dak installations, and made many parts of dak Wheezy-proof. Also, I added a few functions which might also be useful for Debian itself. All patches have of course been submitted to upstream-dak. ;-)

Wanna-build and buildds

GearThis is also nearly finished :) Wanna-build, the software which manages all buildds for an archive, is a bit complicated to use. I still have some issues with it, but it does it’s job so far. (need to talk to the Debian wanna-build admins for help, e.g. wanna-build seems to be unable to handle arch:all-only packages, also, build logs are only submitted in parts)

The status of Tanglu builds can be viewed at the provisoric Buildd-Status pages.

Setting up a working buildd is also a tricky thing, it involves patching sbuild to escape bug #696841 and applying various workarounds to make the buildd work and upload packages correctly. I will write instructions how to set-up and maintain a buildd soon. At time, we have only one i386 buildd up and running, but more servers (in numbers: 3) are prepared and need to be turned into buildds.

After working on Wanna-build and dak, I fully understand why Canonical developed Launchpad and it’s Soyuz module for Ubuntu. But I think we might be able to achieve something similar, using just the tools Debian already uses (maybe a little less confortable than LP, but setting up an own LP instance would have been much more trouble).

Debian archive import

The import of packages from the Debian archive has finished. Importing the archive resulted in many issues and some odd findings (I didn’t know that there are packages in the archive which didn’t receive an upload since 2004!), but it has finally finished, and the archive is in a consistent state at time. To have a continuous package import from Debian while a distribution is in development, we need some changes to wanna-build, which will hopefully be possible.

Online package search

The Online-Package-Search is (after resolving many issues, who expected that? :P) up and running. You can search for any package there. Some issues are remaining, e.g. the file-contents-listing doesn’t work, and changelog support is broken, but the basic functionality is there.

Tanglu Bugtracker

We now also have a bugtracker which is based on the Trac software. The Tanglu-Bugtracker is automatically synced with the Tanglu archive, meaning that you find all packages in Trac to report bugs against them. The dak will automatically update new packages every day. Trac still needs a few confort-adjustments, e.g. submitting replies via email or tracking package versions.

Tanglu base system

tanglu-platform-smallThe Tanglu metapackages have been published in a first alpha version. We will support GNOME-3 and KDE4, as long as this is possible (= enough people working on the packaging). The Tanglu packages will also depend on systemd, which we will need in GNOME anyway, and which also allows some great new features in KDE.

Side-effect of using systemd is – at least for the start – that Tanglu boots a bit slow, because we haven’t done any systemd adjustments, and because systemd is very old. We will have to wait for the systemd and udev maintainers to merge the packages and release a new version first, before this will improve. (I don’t want to do this downstream in Tanglu, because I don’t know the plans for that at Debian (I only know the information Tollef Fog Heen & Co. provided at FOSDEM))

The community

The community really surprised me! We got an incredible amount of great feedback on Tanglu, and most of the people liked the idea of Tanglu. I think we are one of the less-flamed new distributions ever started ;-). Also, without the very active community, kickstarting Tanglu would not have been possible. My guess was that we might be able to have something running next year. Now, with the community help, I see a chance for a release in October :)

The only thing people complained about was the name of the distribution. And to be really honest, I am not too happy with the name. But finding a name was an incredibe difficult process (finding something all parties liked), and Tanglu was a good compromise. “Tanglu” has absolutely no meaning, it was taken because it sounded interesting. The name was created by combining the Brazilian “Tangerine” (Clementine) and the German “Iglu” (igloo). I also dont think the name matters that much, and I am more interested in the system itself than the name of it. Also, companies produce a lot of incredibly weird names, Tanglu is relatively harmless compared to that ;-).

In general, thanks to everyone participating in Tanglu! You are driving the project forward!

The first (planned) release

I hereby announce the name of the first Tanglu release, 1.1 “Aequorea Victoria“. It is Daniel’s fault that Tanglu releases will be named after jellyfishes, you can ask him why if you want ;-) I picked Aequorea, because this kind of jellyfish was particularly important for research in molecular biology. The GFP protein, a green fluorescent protein (=GFP), caused a small revolution in science and resulted in a Nobel Prize in 2008 for the researchers involved in GFP research (for the interested people: You can tag proteins with GFP and determine their position using light microscopy. GFP also made many other fancy new lab methods possible).

Because Tanglu itself is more or less experimental at time, I found the connection to research just right for the very first release. We don’t have a time yet when this version will be officially released, but I expect it to be in October, if the development speed increases a little and more developers get interested and work on it.

Project Policy

We will need to formalize the Tanglu project policy soon, both the technical and the social policies. In general, regarding free software and technical aspects, we strictly adhere to the Dbian Free Software Guidelines, the Debian Social Contract and the Debian Policy. Some extra stuff will be written later, please be patient!

Tanglu OIN membership

I was approached by the Open Invention Network, to join it as member. In general, I don’t have objections to do that, because it will benefit Tanglu. However, the OIN has a very tolerant public stance on software patents, which I don’t like that much. Debian did not join the OIN for this reason. For Tanglu, I think we could still join the OIN without someone thinking that we support the stance on software patents. Joining would simply be pragmatic: We support the OIN as a way to protect the Linux ecosystem from software patents, even if we don’t like the stance on software patents and see it differently.

Because this affects the whole Tanglu project, I don’t want to decide this alone, but get some feedback from the Tanglu community before making a decision.

Can I install Tanglu now?

Yes and no. We don’t provide installation images yet, so trying Tanglu is a difficult task (you need to install Debian and then upgrade it to Tanglu) – if you want to experiment with it, I recomment trying Tanglu in a VM.

I want to help!

Great, then please catch one of us on IRC or subscribe to the mailinglists. The best thing is not to ask for work, but suggest something you want to do, others will then tell you if that is possible and maybe help with the task.

Packages can for now only be uploaded by Debian Developers, Ubuntu Developers or Debian Maintainers who contacted me directly and whose keys have been verified. This will be changed later, but at the current state of the Tanglu archive (= less safety checks for packages), I only want people to upload stuff who definitely have the knowledge to create sane packages (you can also proove that otherwise, of course). We will later establish a new-member process.

If you want to provide a Tanglu archive mirror, we would be very happy, so that the main server doesn’t have to carry all the load.

If you have experience in creating Linux Live-CDs or have worked with the Ubiquity installer, helping with these parts would be awesome!

Unfortunately, we cannot reuse parts of Linux Mint Debian, because many of their packages don’t build from source and are repackaged binaries, which is a no-go for the Tanglu main archive.

Sneak peek

And here is a screenshot of the very first Tanglu installation (currently more Debian than Tanglu):

tanglu_sneak-peek_screenshot

Something else…

I am involved in Debian for a very long time now, first as Debian Maintainer and then as Debian Developer – and I never thought much about the work the Debian system administrators do. I didn’t know how dak worked or how Wanna-build handles the buildds and what exactly the ftpmasters have to do. By not knowing, I mean I knew the very basic theory and what these people do. But this is something different than experiencing how much work setting up and maintaining the infrastructure is and what an awesome job the people do for Debian, keeping it all up and running and secure! Kudos for that, to all people maintaining Debian infrastructure! You rock! (And I will never ever complain about slow buildds or packages which stay in NEW for too long ;-) )

April 01, 2013
I am just considering some modifications to the first round of sketches, the original post is here.

Different color:

Different font:

Playing nice with other shapes (examples):

March 26, 2013
After several attempts I am satisfied with this one (for now), so below is my proposal for the Tanglu logo.
Note: the text part is just for positioning reference, used just plain Vera Sans font but think the font part needs some care.

Variations of the above one and other possible uses:

A shortened "making of" and some draft versions:




Link to the post on the tanglu-devel mailing list with more description about the logo.

Download the above images in editable SVG format:
tanglu-logo-proposal-inkapnea-A.svg
tanglu-logo-proposal-inkapnea-B-variations.svg
tanglu-logo-proposal-inkapnea-makingof-1to4.svg


Creative Commons License
Tanglu logo proposal by Ink Apnea is licensed under a Creative Commons Attribution-ShareAlike 3.0 Unported License.
March 21, 2013
If not, read this. This sounds like it's going to be an awesome project, and I'm going to contribute in any way I can.

To quote some highlights from the initial announcement:
Tanglu will be based on Debian Testing and follow the Debian development closely. It will have a 6-months release-cycle and it’s target audience are Linux desktop users. We will make installing and setting up the distro as easy as possible. 
Tanglu will be usable for both developers of upstream software and the average Linux user and Linux newbie. This is possible because in our opinion developers and users don’t have different needs for a desktop system. Both kinds of users like a polished desktop which “just works”. We will, hwever, not apply any kind of fancy modification on upstream software, we will basically just distribute what upstream created, so users can get an almost “pure” GNOME and KDE experience.
Tanglu is designed to be able to solve the issue that Debian is frozen for a long time and Debian Developers can’t make new upstream versions available for testing easily. During a Debian freeze, DDs can upload their software to the current Tanglu development version and later start the new Debian cycle with already tested packages from Tanglu. The delta between Tanglu and Debian should be kept as minimal as possible. However, Tanglu is not meant as experimental distribution for Debian, so please upload experimental stuff to Experimental. Only packages good enough for a release should go into Tanglu.
And the best part (to me, anyway :-) ):

Which desktop will you use? 
Everyone can add a new desktop to Tanglu, as long as the desktop-environment is present in Debian. Long term, we will have to offer Linux-newbies a default flavour, probably by setting a default download on the website. But as long as there is a community for a given desktop-environment, the desktop is considered as supported.
At the beginning, we will focus on KDE, as many people have experience with it. But adding vanilla GNOME is planned too.

Yay for another option for a Debian based, pure KDE distro. I'm running both openSUSE and Kubuntu now on different machines. I'm surely more comfortable with Debian, but Kubuntu is simply more up-to-date, and KDE is really well integrated. I'm trying openSUSE, but having used Debian based distros for going on 10 years now, I still don't have the level of comfort, so it lives on my (less critical) Laptop.

I see this as great because it will help the overall Debian ecosystem, which feeds not only Debian, but The 'buntu's, Mint CrunchBang and many others.

I just think it's awesome to have something with a pure KDE more closely aligned with Debian coming soon.

Does this all sound as exciting to you as it does to me? Because, man, I'm pumped!

If so, do like I did and sign up for the mailing list, join the Freenode channel and get ready to contribute.
March 14, 2013

Hi everyone!

Today I make an announcement I thought I would never ever make. But things changed.

Discussion about this has a long history, starting as a non-serious suggestion at DesktopSummit 2011, continued with people on IRC, but it was decided back then that it wouldn’t be worth the effort. This has changed too, and a small team has formed to work on it.

We hereby announce Tanglu, a new Debian-based-Linux distribution.

A new logo?Why do we need another one? Let me explain the concepts of that distro:

Tanglu will be based on Debian Testing and follow the Debian development closely. It will have a 6-months release-cycle and it’s target audience are Linux desktop users. We will make installing and setting up the distro as easy as possible.

Tanglu will be usable for both developers of upstream software and the average Linux user and Linux newbie. This is possible because in our opinion developers and users don’t have different needs for a desktop system. Both kinds of users like a polished desktop which “just works”. We will, hwever, not apply any kind of fancy modification on upstream software, we will basically just distribute what upstream created, so users can get an almost “pure” GNOME and KDE experience.

Tanglu is designed to be able to solve the issue that Debian is frozen for a long time and Debian Developers can’t make new upstream versions available for testing easily. During a Debian freeze, DDs can upload their software to the current Tanglu development version and later start the new Debian cycle with already tested packages from Tanglu. The delta between Tanglu and Debian should be kept as minimal as possible. However, Tanglu is not meant as experimental distribution for Debian, so please upload experimental stuff to Experimental. Only packages good enough for a release should go into Tanglu.

Ideally, Tanglu and Debian should be working well together in mixed environments, where you for example have Debian servers and multiple Tanglu desktops with the new software, targeted at desktop user. Since the differences between Tanglu and Debian should not be very high, administering both systems should be very easy (if you know Debian).

Tanglu will be an open project, driven by community. At the beginning of each cycle, people can make suggestions for release goals they want to implement (similar to Fedora, but without FESCo). These proposals are discussed in public and are rejected if there are major technical concerns. If consensus about a certain proposal is lacking, a vote is done for it. The proposal can be accepted with absolute majority. If this does not happen, the proposal is postponed for the next release, where people can vote for it again. If nobody wants that function, it is rejected. In general, decisions made by Debian are superior and have to be followed.

We don’t think we know every package and every software better than the original upstream. That’s why it makes much sense to rely on feedback from others and to have a community-based and peer-reviewed distribution, instead of secretly developing stuff and dumping it on the community. Tanglu will have a highly predictable set of features, defined at the beginning of a cycle, so you will know what you can expect from the next release as soon as possible and plan for it.

Tanglu will make it easy to deploy applications for it. It will contain a software-center, similar to what Ubuntu has. We will also try to establish a solution for a “Linux-AppCenter”, a place for Linux applications, which will be open not only for Tanglu, but can be implemented in any other distribution too. Possible income will flow back into development of the platform.

Now, let’s answer the FQA (Future Questions Asked):

Why don’t you contribute to Debian directly and create yet another distribution?

First of all, we contribute to Debian ;-) And for me, I can say that I will contribute to Debian even more. The point is that Debian can not cover all possible use-cases, and with Tanglu we want to make a distro which solves this. You might ask why we have to create a new distro for that, instead of creating improvements inside Debian? Creating a new distro allows us to do stuff we can never do in Debian. For example, we will include proprietary firmware in that distro, we will make installations of proprietary stuff possible easily (but don’t ship with it by default) and we will have a time-based release cycle. These are already things which are a no-go for Debian, and that’s fine. We don’t want Debian to support these cases, as it is already a great distribution. We want to offer a distro as close to Debian as possible, but with a few modifications for use-cases which are not covered by Debian itself. Of course we will participate in DEX.

If Debian Developers contribute to Tanglu, freezes will take even longer!

This is an often-heard concern, it comes up on every mailinglist discussion about continuing development while freeze. I would disagree here, packaging new upstream stuff is not slowing down testing and improving of packages in Testing. Also, Tanglu is an offer for Debian developers to participate (we will sync privileges for their packages) – we don’t expect anyone to work on it, but as we think DDs know their packages best, we will make it possible for them to participate without extra barriers. We hope that Tanglu can add value to Debian and that Debian cycles can start with better-tested packages.

You said you are a small team – you cannot develop a whole distribution with it!

Let’s put that to the test! ;-) All people working on this are well aware of the issue that the project can not survive without much community-involvement on the long run. But we see a chance that many people are interested in it and that there is a high demand for it.

At the beginning, we will just start with a small set of packages. We will also sync many packages from Ubuntu, to reduce workload. For example, it is planned to use the Ubuntu-Kernel and KDE packaging. By doing this, we keep the workload at the beginning low. We also reduce duplicate work with that.

We even have some possible sponsors for the new distribution. But nothing is set in stone yet, so just wait for it to happen. :)

Why not participate in Arch, OpenSUSE $other_distro?

These are not Debian ;-) . I know, it sounds odd, but if you like the Debian way of doing things, you want to use a Debian-based distribution. There is nothing wrong with OpenSUSE. And Debian has issues too. But we want to be close to Debian and use it’s tools and way of doing things.

I hate you!!! You are doing it wrong!! The project is useless!

Well, that’s fine. But there is no reason for hating us. If you dislike our idea, there are basically two options: First, you hate us but the project is successful. In that case, you have been wrong with hate, as there are definitely people who liked the project and contributed to it. Second, you hate us and we fail. In this case, there is no reason for hate, as the project will just vanish and you don’t have to worry about it. So hating it would’ve been just a big waste of energy.

Also keep in mind that forking is a way to keep development healthy and to adapt software to new use-cases which it didn’t target before. And we are not introducing incompatibilities here (like e.g. writing our own display server could). Instead, we want to stay close to Debian and reuse as much code as possible.

Which desktop will you use?

Everyone can add a new desktop to Tanglu, as long as the desktop-environment is present in Debian. Long term, we will have to offer Linux-newbies a default flavour, probably by setting a default download on the website. But as long as there is a community for a given desktop-environment, the desktop is considered as supported.

At the beginning, we will focus on KDE, as many people have experience with it. But adding vanilla GNOME is planned too.

Can you say something about the software used in Tanglu?

Yes, but this is still in flow, so I can’t promise final decisions here. On the server, side, we will try to use the Dak for repository management, as soon as we have enough server capacity. We will also use the standard Debian repository GUI and basically reuse most of the software there, to diverge less from Debian.

The distribution itself could probably use a Linux Kernel from Ubuntu and systemd as the primary init system, as well as the login manager logind. It will be based on current Debian Testing with some fresh packages from Unstable and Experimental. We might also use the Ubuntu driver packages and KDE packaging. We expect to have a very rough start with the first release, but there will be enough time to polish Tanglu.

UPDATE20140214: Just because this pops up online incredibly often: Tanglu does not and will likely not use an Ubuntu Kernel. Tanglu 1 (Aequorea Victoria) ships with Linux 3.12 derived directly from Debian.

Nice idea! How can I help?

Well, you can help with basically anything at time – from writing manuals, over designing logos and pages to administering a webserver and create packages. We are at an early stage of development at the moment, but we wanted to go public with it as soon as possible, to include the community and receive feedback so we can make that distro community-centric from the beginning. Most of the infrastructure is currently in progress too.

So, if you want to get started with Tanglu, subscribe to our mailinglist tanglu-devel and write a mail to it, intruducing yourself. We can then include you in the loop. Generally, if you want to get access to our machines, a trusted GPG-signature will help a lot.

If you want to talk to us, join #tanglu-devel or Freenode! Most discussions are currently happening there.

And that’s it! Tanglu will be awesome!

Some other projects of mine will develop a bit slower because I am now involved in Tanglu. But nothing will stop, and there is some pretty cool stuff coming for both GNOME and KDE (and I still have to implement DEP-11 for Debian). :)

March 06, 2013

Disclaimer: This post just sums up a concept for a new distribution which matches certain ideals. It is not the announcement of a new distribution. These are just abstract ideas. (However, if there is high interest in a project like this, it might of course develop into something real…)

I have been involved in Debian and Ubuntu for a long time now. When Ubuntu started, I was a Debian Testing user, and I immediately switched to Ubuntu when it started, because I liked the idea of a short-release-cycle, user-centric company-supported Debian based Linux distribution. However, I am now back to Debian for a long time, because of many reasons which nearly all had to do with Canonical policy. But this is not a post to criticise Ubuntu, so I’ll leave out most of that part. I am highly disappointed on how Ubuntu develops – not only the technical decisions are at least questionable, but also the social and community part is not that great anymore. There is a high asymetry in the relation between Canonical and other developers, Ubuntu mailinglists basically don’t create meaningful results, they sometimes even mutate to a Canonical Q/A session. The community does not seem to have a large influence on decisions about core services, and it can’t have it if there are things developed behind closed doors. (This is all, of course my subjective impression)

But really nobody can argue against the basic idea of Ubuntu and the great things Ubuntu created Also, many of the processes Ubuntu uses to develop the distribution are just great and well-working, as well as there is a highly active community around it. As you simply cannot argue with Canonical to change their policy (they are a company and have hidden plans, also they have every right to apply whatever policy they want), the natural way in any OSS project would be to fork it. But doing that blindly would just create another distribution, which would almost certainly vanish again soon, since there are already many Ubuntu derivatives which cover many use-cases using an Ubuntu base.

I discussed this stuff with Daniel some time ago, and we did some kind of brainstorming about what a perfect distribution would look like, from the perspective of a developer who wants to use a Debian-based distribution.

Here is a list of points which would define such a project:

  • Every available package complies with the DFSG and Debian policy.
  • Packages of DISTRO stay in close sync with Debian packages, changes are preferrably applied in Debian. DISTRO might work as a playground for new technology while Debian is in freeze.
  • DISTRO stays as close to upstream as possible. It applies as less patches as possible, to deploy desktop environments which look like the thing upstream intended it to look like. Changes for DISTRO are developed upstream and only applied downstream if doing that doesn’t make sense or changes are distribution-specific and can’t be abstracted.
  • All desktop environments are treated equally. There is no preferred DE.
  • DISTRO stays in sync with release cycles of KDE and GNOME, to provide developers the latest stable development environment and users a recent version of fresh upstream software.
  • DISTRO is user-centric. It tries to make the distribution work on as many hardware as possible and to fix any usability quirks which are found. But work on that should go upstream, of course.
  • At the beginning of a release cycle, packages are merged from Debian and selectively merged from Ubuntu, where it makes sense.
  • All Debian developers automatically have upload access to DISTRO, so they can upload versions of their packages to the cutting-edge DISTRO and also maintain the Debian things.
  • All DISTRO developers can upload any package, but are responsible for everything they might break. Also, changes should be discussed with the original maintainer at Debian – this is the person who knows the package best.
  • DISTRO is not a testing environment for Debian, it is developed like a short-cycle-less-stable-but-usable Debian. DISTRO recommends Debian Stable as “LTS” version.
  • No CLA is enforced on anyone. People are free to contribute, as well as companies. The project structure is meritocratic.
  • Features are discussed in the open. If there is disagreement about the direction of the project or about any technical issue, a public voting on this feature is created, where every project member can vote. The vote result is final (but can be changed in future, of course).
  • Create a well-maintained core, make it possible to install the newest applications on that distribution (by using native packages, Listaller or Glick) Think about rolling releases, where applications are constantly updated and the core stuff is refreshed once a year/every 6 months. With this users have new apps immediately, but don’t live in fear that some core API or a desktop workflow changes immediately. (Also not desired for desktops in schools, universities and companies – maintaining a moving target in these larger environments is a pain)
  • Create an Application-Center and application-ecosystem based on AppStream and Listaller around it. Encourage upstream developers to publish fresh applications there. Make it possible for every distribution to use this solution and help others to adapt it. Don’t try to lock people on that one platform by providing stuff exclusively for DISTRO or making it harder to use it.
  • Encourage using additional commercial stuff like Amazon searches, magazine stores etc., but make these things available on a separate repository and make them entirely opt-in. Never enable stuff like this by default, but add simple instructions how to use these things for people who want to use them.

Debian DevelopmentThis is basically what we would like to have in a new distribution. :) If you take a closer look, you will see that an effort like this would basically create a close-to-upstream, user-centric short-cycle Debian-based and close-to-Debian distribution, which would cover many use cases, including fixing the “use experimental for new packages during freeze” issue at Debian (DISTRO could be used as environment to run cutting-edge technology, which is generally stable enough, but not yet “Debian-Stable”-stable). Something like this does not exist at the moment. If you take a second look at the list above, you will also see that I mixed technical aspects with organizational aspects. This is intentional. This is just brainstorming, because it is good to know what you would like to have, instead of complaing about the status quo of other projects.

But maybe there will be a distribution which matches some of the above points, to create an upstream-friendly entirely community based Ubuntu.

November 29, 2012
Maecenas sed egestas erat. Curabitur suscipit massa varius libero condimentum eget consequat erat elementum. Duis velit nunc, venenatis non consequat in, viverra scelerisque ipsum. Sed eget lectus non justo pharetra ultrices. Maecenas eget mi eget magna mattis pulvinar. Class aptent taciti sociosqu ad litora torquent per conubia nostra, per inceptos himenaeos. Maecenas magna libero, facilisis at hendrerit at, placerat at ipsum. Ut interdum euismod scelerisque. Vivamus sed enim in augue facilisis aliquet. Etiam lectus leo, aliquam vel interdum at, ultricies id dui. Donec quis diam neque. Lorem ipsum dolor sit amet, consectetur adipiscing elit. Fusce aliquet faucibus nibh et pellentesque.
November 12, 2012

“Once again darkened and the sun set in the heart I could not bear, I miss my home, I prayed to God in the affliction He opened my heart was when someone touched me and heard Jesus say this: It’s just a little more, a little more, then I’ll get you there, love is real, there is a river of peace, there is a heaven that God called him home. Again dawned and the sun was born of faith in me and when I think about quitting Jesus says: It’s just a little more, a little more, then I’ll get you there, there’s real love there is a river of peace, there is a heaven that God called home …”
King’s Heralds

Love you so much my husband Daniel Nicoletti, God will get you out there and sure ara sent angels take care of you … I will move heaven and earth,’m going through, I will do everything, would give my life for you … love you so much … miss …

Elisabeth Nicoletti …


November 09, 2012

Hello! This blogpost is personal and not technology-related at all, but I would appreciate it if you would read it anyway, in this case. Generally, I don’t like sending off-topic posts to the Planets, but this is an extreme situation and I feel it is important that you know what has happened.

First of all, who is Daniel Nicoletti? He is a KDE and Freedesktop developer, the author of many great things like Apper, the PackageKit-based KDE package manager, a new KDE Print Manager, colord-kde and the Aptcc backend PackageKit uses on Debian-based systems. Also, he is a friend of mine and we work on many projects together.

Recently, he was traveling to the Linux Color Management Hackfest 2012, but he never reached Brno, instead he was arrested by Interpol at the airport in Munich.

The reason is a tragic accident about a year ago. If you are a KDE developer/user, you might know the story already. Daniel and his family wanted to move to Argentina. On the way, a car suddenly stopped on the road. To avoid a collision, Daniel tried to dodge the car, which then collided with a truck. His daughter died later in cause of the accident. The KDE release 4.6.3 was dedicated in memory of her.

After these events, the family decided to move back to Brazil for various reasons. Meanwhile – and without Daniel knowing anything about it – Argentina filed an Interpol request to get him, accusing him for murdering his daughter. (As – of course – he was no longer in Argentina)

At time, he is arrested in Munich, and Argentina asks for his extradition, but Germany does not totally agree with his guilt.

Thanks to our really awesome community (really, I can’t express how impressed and proud I am) he received some help quickly, after his wife posted a desperate request for help via his Google+ account. – At first, she didn’t even know what exactly had happened, because no communication was allowed to Daniel (and still is not), and we all assumed an accident.

Now he has a lawyer who is communicating with his wife. In the worst case, Daniel will be struck for at least another 6 months, unless his wife can come over and intercede for him.

This surreal story really came out of nowhere. The day before he left for the hackfest, I was reviewing a patch with him and everything was fine, and he was excited to go to the CM Hackfest.

At time, his family is collecting money so his wife is able to go to Munich and clear the situation, because due to these incidents, there is not much money left. So, if you want you can donate for him.
Click here to lend your support to:  URGENT flights to go to the Brazilian Consulate in Munich and make a donation at www.pledgie.com !

I really appreciate any help to get Daniel back and clear this injustice, and I wish him and his family the all the best! And I hope that this nightmare will end quickly.

Thank you for reading through this…

UPDATE: Daniel’s wife will soon travel to Germany with an one-way ticket. She will meet a member of the community at the airport who can show her around and help in communicating with the locals. (And offer her a place to stay) At time, they seek a lawyer who is specialized in this international affairs. (Brazilian citizen, accused in Argentina, currently sitting in Germany…)

Because some people asked this, you can find the whole story about the accident on Daniel’s Blog. Also, a local newspaper reported about it. (But my Portuguese or Spanish is as good as the Google Translator allows it) Also, as far as I know communication between the Brazilian side and Germany is going on.

Currently, nobody can speak to Daniel. I’ll keep you updated.

Also, for Brazilians: You can also donate here, if Pledgie does not work for you. (requested some times) – I just hope they will get the money even if goals are not reached, I don’t know the rules these platforms apply.

October 04, 2012

Many awesome new releases have been made this week so far!

PackageKit 0.8.4

The new PackageKit features lots of bugfixes and small improvements. It also contains a few new features, like showing PolicyKit dialogs on the command-line, even if no GUI is running.

If you want to try the new PackageKit and are a Debian user, you can install it from Debian Experimental. (Packages are available) The next Apper release will support this PackageKit series too.

Listaller 0.5.6

The “we-break-everything” Listaller release. Contains a finalized IPK1.1 spec which is frozen now, so if you create packages, you can be sure future Listaller releases will support them too. Also the Listaller API was refreshed so it is much easier to use.

This new release does not include the promised support for IPK-repositories/software-updates, this feature was not ready for 0.5.6, but it might come with 0.5.7 – then it will also include deltaIPKs, one of the most-requested features by game developers (and probably LibreOffice users :P).

Instead the new Listaller contains rewritten support for GPG-key handling, giving distributors and users full control of the GPG public keys Listaller trusts. Also, Listaller won’t install packages with security level “Low” anymore, by default. Of course you can change this setting, if you want.

The documentation of Listaller is currently rewritten, so we can kill the Wiki, so you will soon have excellent documentation of Listaller’s features. Also, lots of other improvements happened, check the release notes for details.

The new Listaller requires at least PackageKit 0.8.4, because it requires some changes done in this PackageKit version.

AppStream-Core 0.1

I am really excited about this one! AppStream-Core is a collection of tools I started writing during my SoC which allows creating & accessing the AppStream database. In combination with PackageKit, AppStream-Core offers a nice way to build Software-Center-like applications. AppStream-Core is also a central place where all changes for AppStream support can be implemented in a distro-agnostic way. If your distribution wants to support software-centers, it should ship this collection of tools. (of course, AppStream data still needs to be provided by distribution-specific tools)

The 0.1 release is not meant as a stable release, it is only something you can play with at time. Right after 0.1 was released, I already did some important changes in the master branch, so the whole project is moving at time, and there is no API-stability-promise yet. But of course I suggest everyone to test the project ;-)

AppStream-Core is a prerequisite to make stuff like application-display in KDE’s Apper work or to help GNOME implementing a tool like GNOME-Software. (it is also recommended to run the USC on other distros)

September 03, 2012

Nearly everyone who’s reading this blog will know about Listaller, a project started by me to make cross-distro 3rd-party software installations possible & secure.

This project has been started years ago, and lots of things have changed in the Linux desktop world and down the stack, so I think it is time to look back now and summarize what has happened – and also to finally write down my evil plan about what Listaller should achieve. People who know me and who I’m working with already know what I would like to achieve and pushed me a little to write this “vision” down.

I’m not a big fan of project “visions”, but in this case to define a final goal visible to everyone (and not just me) will help a lot.

So, if you want to read something about the state of the Linux desktop and my view on the recent “Linux desktop has failed” discussion as well as lots of history and future visions of the Listaller Project, keep on reading! (If not, go here and enjoy the day ;-))

The Past

When Listaller was started in December 2007 (I guess, I don’t have an exact date), software management on Linux distributions was in a very sad state. It was usual to manage software in distributions using a native package manager, like Synaptic or YaST. Also, extending the software repository was not really easy. Ubuntu did it’s first experiments with an application-centric software manager that time and PPAs weren’t that common. (I don’t know if Launchpad already offered this service back then – I think it was implemented later)

Regarding cross-distro 3rd-party software installing, there were a few competing projects like LOKI/MoJo, Autopackage, ZeroInstall, Klik, etc. which all weren’t that common and only a few people used that software. Also, they didn’t solve the main issue: Packages aren’t user-friendly! A “package” is a somewhat abstract concept which is hard to understand for non-technical users. These projects only focused on installing cross-distro packages, but I wanted more.

So I started the Listaller project. Listaller was designed to build a cache of applications installed on a distribution and offer uninstalling these apps, no matter how they were installed. Listaller implemented it’s own package-manager abstraction layer, which basically worked with plugins which then called native distribution tools like “dpkg”, “rpm”, “apt-get” or “yum”. The only distributions supported were Debian, Ubuntu, SUSE and we had very poor support for Fedora.

Listaller also contained a new format to install applications, which was designed to abstract package management complexity. The Listaller package format was able to carry links to existing native packages and download & install them in order. It also was able to contain any binary installer, so it could also carry Autopackage or LOKI installers and perform necessary steps to execute these binaries and make sure everything is set up correctly. Finally, it was also possible to include the application install files directly into a Listaller package (file extension *.lipkg at the beginning, later changed to *.ipk) – It was planned that this solution should later be the only possible way to install software, the other IPK-package-modes were made to ease transition. The Listaller package generator also produced a logobutton, showing all distributions this package supported, which developers were able to put on their websites, because the generated Listaller packages were sometimes still limited to some distributions. (Listaller was able to detect that to a certain degree) I found only one of these buttons on my disk. It looked like the thing on the right side.

The software manager was very similar to later tools managing applications instead of packages, which was the goal of this effort.

Here’s a screenshot of the very early Listaller application manager (and it’s crappy UI):

With the rise of PackageKit, the whole situation shifted a little: Now we finally had one abstraction layer for all package managers distributions offered, which was much more powerful than the poor thing I created. I followed the PackageKit project since version 0.1.18 (I think) and later joined the project as developer and implemented support for it in Listaller.

During that time, the Klik installer project died and Autopackage merged into Listaller (after both projects first only did collaboration), leaving only very few projects with cross-distro software management in scope.

The the AppStream project was created (unfortunately I wasn’t able to attend the meeting due to exams) and suddenly all points the Listaller Project wanted to achieve back then had been solved – except for one: Cross-distro software installations. (AppStream implements the app-centric software management in a cross-distro way, so this Listaller goal is now achieved)

So I shifted the scope of Listaller to 3rd-party software-installations only and started a new implementation of it from scratch using the new Vala language. I also extended PackageKit with support for Listaller, because having multiple UIs for package management is highly confusing for users and the new design of Listaller made it possible to extend any PackageKit GUI with the ability to manage Listaller packages.

Listaller was extended to use ZeroInstall feeds to satisfy dependencies too, so resolving dependencies would work better, and both ZeroInstall and Listaller could benefit from the availability of more feeds. The Listaller support in PackageKit support was then split out into a plugin, so distributions are able to enable/disable Listaller as they like. (and also the plugin code made PackageKit’s code much better and easier to handle!)

Many other changes happened too, which I won’t summarize, this blogpost is already too long ;-) – Unfortunately the rewrite made Listaller unusable for a very long time and some people already considered the project dead.

The Present

So, many people might now ask the following:

Why do we need cross-distro packages?

I consider cross-distro software management a key element for the success of Linux desktops.

At time we have a very old protocol how software reaches end-users: Upstream creates it, downstream the distributions package it and deliver it to their users. This unfortunately does not leave any choice for people to selectively install or update the software they want. It is also not easily possible to install software which is not in the repositories. Of course, someone could create packages – but then he has to build his software for a multitude of distributions, which is lots of work (you can’t even target all RPM-based distributions using just one spec-file without doing many distro-specific checks)

It is important that developers can get quick feedback from their users on new software, without needing to go into the details of package management and having to wait for distributors to ship new software. (Take Debian stable as an example) Also, users should be able to install software without having to worry about compromising their system. (By installing a foreign package, you’re basically giving someone else root access to your computer)

The OpenBuildService, although being a great tool, is not a solution for this – also PPAs aren’t, because any change on the native package configuration can make the system unusable, install malicious software or break upgrade paths, and they’re hard to handle for non-technical users.

Where are the app developers?

There is a big lack of commercial software for Linux. Why? Usually we hear the argument that “Linux has not a big enough market share”. I don’t think this argument counts that much, as Apple also had a very small market share and they had commercial apps too back then. The situation we have in Linux is that you – as a proprietary software vendor – not just have to support “Linux”. You need to support a multitude of different configurations and distributions, and this fragmented world is very hard to handle.

First, some clarifications about my point of view on certain “social” aspects in the Linux community, so you can understand me easier:

Regarding FLOSS vs. proprietary software

I think both is important and we should fully support both. For me, I want the OS platform as open as possible to have unlimited possibilities regarding stuff I can do with it. And there should never ever be something limiting my freedom do make any change I want on the Linux platform. BUT, if people want to deploy proprietary software on that free-and-opensource platform, I welcome them too, because everyone should be able to use the FLOSS platform for everything they want. Also, it is better if more people use a free platform and be aware of it – this is the best way to prevent stuff like locked hardware devices and the UEFI hell. More app developers will certainly help in adopting the Linux platform. That’s why I welcome Valve on Linux too, although they’re proprietary and I don’t play games.

I always prefer free software over proprietary software, but there are some valid arguments for developing closed-source stuff too, and I don’t want to exclude these developers and add impose limits for our users to get software.

Regarding Linux and GNU/Linux

What is “Linux”? Many people already did lots of fighting about the name – I don’t want to do this here.

Sure, Linux is a OS kernel – technically. But what is Linux in society? For me, “Linux” is not a kernel or OS, instead it’s a value and brand, which defines all Linux OS’es. When I talk about “Linux” I refer to all components a Linux distribution is made of, the kernel, the plumbing layer, the toolkits, the desktop the GPL and the freedom we want to give to our users, as well as the great community. In population, Linux is a know word for “free operating system”, Linux is recognized – many small distributions are not. If I want to speak about the kernel, I just say “the Kernel” or “the Linux Kernel”.

People port software to “Linux”, people use “Linux” etc. – Linux is used as a brand and word for a free operating system.

So, it is important that software companies can target the Linux market if they want – not only a small fragment of it called “Ubuntu” or “Fedora”, but the whole Linux world using just one software build. People will see a download link for Linux software, not just a simple tarball and recognize Linux as equal (better!) to Windows and Mac.

The Linux world is big – it’s just not recognized as big. The Humble Indie Bundles already showed Linux users pay for software (I never doubted that and it doesn’t really surprise me), so we just need more app developers now.

Mark Shuttleworth recognized the potential Linux has – with programs to promote the Ubuntu desktop, workshops for app developers, the PPA-way to submit software and app-cebtric software management, they’re doing everything right. But of course, they’re Ubuntu-centric, software is developed “for Ubuntu”, you won’t find the word “Linux” anywhere on the Ubuntu homepage. Having a universal software distribution format for Linux would also be fair for smaller distributions, so they can receive new applications very fast too, although they don’t support Ubuntu PPAs.

Listaller changes

Listaller is currently designed to do exactly that – create packages to target the whole Linux “market”. These packages provide FLOSS developers a great way to push new releases directly to their end-users, removing the “distributor” part in the software release pipeline. Proprietary developers instead get a great way to target any Linux distribution.

The package format is based on standards like DOAP, so every application which ships DOAP informations (i.E. every GNOME application) can generate an IPK package really fast.

Of course, there are some limits: Listaller is designed to install applications, and only applications. Installing core infrastructure is not supported. This means you can’t install the full KDE with it or install systemd. You also can’t execute arbitrary code at install-time, most-used tasks are hardcoded.

Removing flexibility means adding security here.

Brought to you by…

The recent changes in Listaller wouldn’t be possible if the Linux desktop isn’t already becoming more app-developer friendly and more consistent: There is an increased awareness of ABI compatibility, systemd is unifying many OS functions which have been completely random before and Freedesktop is a great project which allows collaboration in components which work on all distributions.

For everything not yet covered, Listaller provides tools to make applications binary-compatible with many distributions, but I hope these things won’t be used in future anymore.

The future

There is a lot of new development in the GNOME community, which will make the life of a cross-distro packager and therefore Listaller easier: Stable APIs, a “GNOME SDK” etc. These changes might make it possible to make Listaller packages containing pure GNOME applications just depend on “GNOMESDK>=3.8″ instead of handling every single dependency separately.

On the KDE side, the project is splitting into “Workspaces”, “Frameworks” and “Applications”, which is also a great move forward for cross-distro apps and will make packaging KDE apps in general easier.

So, with this new development, I think it will finally be possible to build a cross-distro software installer with a design driven by user’s needs and not creating a big hack and workaround-set to overcome all technical limitations of cross-distro software: We might be able to fix these problems at the source. (The increased caring about ABI stability in FLOSS community already helps a lot)

Also, Listaller itself needs to be future-proof: We have two developments which affect Listaller:

  • a shifted focus towards WebApps instead of traditional applications
  • distribution through software-stores

For the first point: I do not believe in WebApps replacing traditional software – there are just too many limitations, and I also don’t like the idea of web apps storing much personal data on servers I don’t control. (Although there are efforts to fix that) I also think the Web technology is in many points inferior to what we use on the desktop today (be it Qt or GTK+). Instead, I think we’ll have more and more “hybrid” applications soon, which e.g. have logic implemented in C and run a UI built with web technology. Also QML is a very innovative and great approach for UI construction.

For the second point: I consider every application store as limitation of user’s freedom, as long as it is the only possible appstore and/or adding other stores is impossible. Competition between appstore vendors is very important.

So I imagine the following (far away) scenario: Distributions ship Listaller with their own software-store enabled, if they have one. The base OS is made of native packages, all additional software (also newer versions) is delivered through Listaller packages. (This doesn’t mean there are no applications like Firefox in the native packages – everything stays as-it-is, Listaller packages are an addition)

Because Listaller packages are distro-independent, OpenSUSE users can also use Ubuntu’s appstore-source, so there is no longer any dominance of a distribution regarding availability of precompiled software.

Ideally, one organisation creates a “Linux AppStore”, which is added to many distributions by default. (but maybe disabled and easy-to-enable) This Software Store will sell applications by commercial software vendors and – because it should be carried by a non-profit-organisation – sends most of the earned money back to distributions, which itself could use it to improve their distribution.

This software store would be very attractive for software vendors, because they could target the whole Linux market and would be very visible to potential customers, on distributions from Gentoo to Debian. Also Linux itself would be much more visible to others.

Listaller packages are by design very secure, so these packages can’t harm the system. (they’re usually signed and don’t install stuff in system directories – also a sandbox interface exists (but is not used at time)) Upgrades will work without problems, because the native package configuration is never touched, so distributors will receive less bug reports about those issues. PPAs will instead be used by people who want the latest Linux infrastructure, like a new KDE, GNOME, systemd, PackageKit etc. version, and not by people who just want the latest GIMP.

Because the setup process is 100% controlled by Listaller (no custom user scripts in packages, many things handled automagically), distributors can control every single 3rd-party software setup on their distribution just by adjusting Listaller to their needs. So distributors still have control about every piece of software installed on their distribution.

Also, Listaller is nicely integrated in all existing application-managers if they use PackageKit, so extra-UIs aren’t needed and users don’t even know that the software is there.

Of course, installing standalone software packages is still possible – they might of course be run in a sandbox by default.

Listaller will have logic to maintain the different software sources and provide information in a way frontends can display it nicely.

Implementation!

Of course I’m working on the code to make it ready for future use. The first step was adding support for package-updates using package deltas in Listaller 0.5.6 – unfortunately this feature has been delayed because of my SoC project and because I completely broke the API Listaller uses to talk to PackageKit during my work on PackageKit. So the next release will be without a major new feature, but with many bugfixes.

To make Listaller releases more frequent and because Listaller depends completely on PackageKit and internal APIs of PackageKit, I’ll sync the Listaller release cycle with the one of PackageKit. This means you will get a new Listaller version one week after a PackageKit release. By doing this I also hope new changes will reach people faster – and there will never be the case where Listaller and PackageKit are incompatible. (At time, Listaller 0.5.5 won’t work with PackageKit 0.8.x series)

Of course, the project needs contributors! :-D At time, “we” are only me and a few part-time people who create probably one small commit every two months, since the main contributors started to write their bachelor thesis or do other stuff (at work).

I would be very happy about comments regarding the plan above, which is of course very rough at time… To make it more concrete, I’ll try to talk to people at GNOME about their vision, because it seems they also have similar plans or at least have one for application management improvements, which could at least be valuable input.

And finally: Is the Linux desktop dead? The answer is NO! There might have been mistakes in the past, but both KDE and GNOME have clear goals for the future, are more end-user focused than ever, care about ABI stability, collaborate on an increasing amount of projects (although it could be more ^^), prepare for targeting mobile devices and are used by millions of people. We should be very happy :-) Let’s see what we achieve in future, even if it is not desktop-dominance over all other OSes, it will be great anyway! :)

(Of course I also care about Xfce, LXDE and E17, just before someone asks ;-))

If you want to contact me, you’ll find me on IRC: #PackageKit (Freenode) There’s also a Google Group about Listaller, but I’m looking for a traditional mailinglist at time… Reaching me via mail is always the easiest way ;-)

August 30, 2012

This year I did a Google Summer of Code Project for OpenSUSE as part of their cross-distribution collaboration track. (Again, many thanks for letting me work on this and for doing a cross-distro track!)

So, what did I achieve this summer? (Leaving out all the problems and stuff which didn’t work as I expected :P )

I did work on three components: AppStream, the Software Center and PackageKit.

At the beginning of the SoC, I thought I would be spending most of my time at the Software Center and in seeing Python code. Instead, I put lots of effort into PackageKit and writing C code. The reason for that was that when I started, all operations I performed in the Software Center were very slow, because PackageKit was slow. Also, some things weren’t possible, e.g. I couldn’t fetch details about an application from PackageKit while an installation was going on, because the installation blocked all other requests on PackageKit. So, I first focused on PackageKit.

PackageKit

I implemented various small speed optimizations, which made PackageKit a little bit faster. (on Ubuntu, it’s faster than Aptd now, but that’s not a fair comparison) I also added a package sqlcache, which can be used to access package data very fast, if enabled.

The biggest thing I implemented was support for parallel transactions. Parallel transactions allow PackageKit backends to run some transactions in parallel, if they support this feature. This means that you can now query package details or search the package database using PackageKit while installing or removing packages at the same time.

It also enabled frontends to query more data faster, which will speed up every PackageKit client if the PackageKit backend supports parallelization. At time, the Zif backend fully supports this feature and the Aptcc backend supports it partially.

Together with Richard Hughes I refactored the backend API, so backends now have some more options to optimize they way they handle cache openings and jobs. (most of the credits for these API changes go to Richard)

Software Center

The Software Center is a fork of the original Ubuntu Software Center, because it was not possible to have Canonical drop the CLA they apply on Software Center’s code. (and which is a problem for new contributors, especially those employed by companies)

At the Software Center side I did lots of bugfixing and speed improvements, for example the SC now starts really fast (if the PackageKit backend is fast too) and data fetching is super-fast too. I also removed some problems preventing the SC to work properly on non-Ubuntu distributions and ported the code to our newly created APIs.

I am still not happy with the state of the Software Center, as there are many unsolved bugs and the tool is not yet user-friendly. But I also think these problems will vanish when distributions start to ship AppStream data to fill the Software Center database and when PackageKit backends are improved.

Here are some screenshots of the PackageKit-based Software Center running on Debian and Fedora:

 

You can already test the code if you have at least PackageKit 0.8.4 installed. Unfortunately distributions don’t provide all AppStream data (the information which matches package names with applications and contains icons & stuff) yet, so using it is quite difficult at time.

AppStream

For the AppStream project itself I created infrastructure to create & maintain the AppStream Xapian database. This database is used by application managers to query data about applications. It also makes searching applications really easy. At the beginning, all code which created this database was hard-wired in the original Ubuntu Software Center. So, if you wanted AppStream data, you had to install the USC. Now, I wrote a nice tool in C++ and Vala, which will build the database from various sources and which allows querying the database using a very simple interface.

These changes will allow alternative Software Center implementations to use AppStream and it will make some other interesting development possible (e.g. application support in Apper, which I implemented after my SoC project was completed, see this post for details.)

There are already some new software centers in progress, for example the Light Software Center, which was designed for use with PackageKit from the beginning.

At time I’m running a request at Freedesktop for space to present the new project & APIs, as well as to upload release tarballs, so this code can hit distribution repositories soon. :)

Project conclusion

It bothers me a little that I cannot present you a 100% bug-free end-user-usable Software Center with the end of my project. But instead, I killed all the big technical problems which made implementing a Software Center on other distributions impossible before.

I also did lots of changes which will make using PackageKit even more pleasant to use now and I created Software Center infrastructure, which will allow many projects to implement AppStream features. For example, GNOME developers can get their application-centric software managers, and KDE people will receive an AppStream-ready version of Apper soon.

So in the end, this project was very successful and I’m really happy with the results. Also, it feels like the AppStream project itself gained momentum now, and I will of course continue developing stuff around AppStream – stay tuned, there will be more news later :-)

GSoC conclusion

I can only recommend every student out there to consider applying for the Summer of Code! It has been an absolutely great time, meeting new people and discussing stuff with them has always been fun for me, but also the feeling of being able to really move a project forward and taking the time to do that is awesome!

During my SoC project, I really had some problems to find a balance between university stuff and GSoC work, because university was giving me a hard time (lots of stuff to learn!). In the end, even a harddrive crash in the hottest development period couldn’t stop me ;-) (now I have a new drive and I was fortunately able to restore most of the old data)

Working with Vincent and of course Richard (who I’m working with for years now) has been a lot of fun, and also interacting with various people in the community was great, but also sometimes very exhausting – in cross-distro and cross-desktop world you really have to put lots of energy in convincing other people to make certain changes happen. But during my SoC I achieved more things than ever in the area of application-centric software management, so it was totally worth it.

When I look at the projects fellow SoC students completed, there are even more great projects to see! (Just take a look at it!) Some other students I talked to also agree that the experience alone was already worth doing their SoC project.

So, if you are a student and are able to write code or do some other cool stuff in FLOSS projects: Consider applying for GSoC 2013!

So, that’s it for now, I already planned some new blogposts later, so stay tuned!

August 26, 2012

As some of might now: I did a GSoC project for OpenSUSE this summer. The goal was to improve AppStream and the Software Center stuff on all Linux distributions.

During my SoC project, I also created appstream-core, a small library and infrastructure to create & use the AppStream application database. This project enables all other tools to use AppStream data and combine it with PackageKit information.

So it was just natural to bring this stuff to Apper too, our favorite PackageKit-based KDE package manager ;-) Apper has had a function like this before, but it was limited to Kubuntu. Now, the AppStream stuff will work on all distributions which support AppStream and ship at least PackageKit 0.8.4 (currently unreleased).

The result of Apper+AppStream looks like this:

As you can see, Apper is showing applications now instead of packages. (All packages which don’t ship applications are still presented as packages) It also uses the Debian screenshot service to display a screenshot. This feature is experimental and still not completely finished on the AppStream side.

Additional credits for this go to Daniel Nicoletti, because I based my work on previous work done by him.

The new code is in Apper Git already. You will need PackageKit>=0.8.4 and AppStream-Core to test it, but because these changes are experimental at time I’d suggest waiting for your distribution to ship it. (many software is even unreleased at time)

So, this is just a sneak preview of the cool stuff to come. :) Stay tuned!

August 12, 2012

Hi!

My SoC project is nearly finished now, which unfortunately doesn’t mean that we will have a completely usable Software Center on all distributions – but the most difficult steps are done, all specification issues solved and I talked to many people from other distributions about the AppStream project and we made lots of progress. I’ll publish details in my GSoC final report ;-) This post is meant as a guide which steps are necessary to make distribution X support the AppStream project and – using AppStream – an application-centric software management solution and Software-Center-like applications.

1. Write/update your PackageKit backend

We rely on PackageKit for package management, so to support your distribution, we need a backend for your package manager.

Make sure that your backend supports parallelization! This feature is used heavily in application-management frontends like the Ubuntu Software Center and will generally make your backend much faster.

2. Package AppStream components and ship them

First of all you will need of course to ship PackageKit. Also we need AppStream-Core (with UAI) to be installed. (I’ll make a first alpha release of this after my GSoC project is complete and I discussed some things regarding this piece of software with my mentor(s)) If you want, you can also ship the cross-distro version of Software Center – you’ll need to package it’s dependencies too, some extra Python components aren’t present in all distributions.

3. Generate AppStream data

This is the most important step. AppStream data can be provided in the Ubuntu AppInstall format or Debian DEP-11, but we suggest you use the AppStream XML spec which is supported on every distro. You need a script which generates this data from packages and application’s desktop files which fits your distribution’s needs. (This script needs to run on the distribution’s servers) OpenSUSE and Fedora already have these generator tools. You also need to make this script extract icons from packages, so the software-center-implementations can display icons of not-installed packages.

The data should probably be regenerated automatically every week on distro branches which are rolling-release or in development. For stable distros where package names don’t change and no applications are added or removed, generating the data at release-time should be enough.

4. Ship AppStream data

You can ship AppStream data any way you want. You can package it as a regular package (done in Ubuntu) or advise the package manager to download it as part of the repository metadata (planned on-request for Debian). You just need to make sure the XML files end up in /usr/share/app-info/xmls and the icons are placed in /usr/share/app-info/icons . It makes sense to encode the repository name in the XML file name to avoid filename-conflicts if another (3rd-party?) repository wants to install application data with the same name.

If you use the package-way to make the data available, make sure to depend on the “update-appstream-index” tool and execute “appstream-index –refresh –nowait” in the package postinstall script, so the AppStream index is rebuilt. If you use another way to deliver the data, make sure to trigger the cache rebuild there too.

5. (Optional) Adjust the Software Center application

If you use the cross-distro non-CLA-fork of the Ubuntu Software Center, you maybe need to add a profile for your distribution to it. At time we support Ubuntu, Debian, Fedora and OpenSUSE already. You also might want to do other adjustments to fit your distribution’s needs.

If you don’t use the USC, you can skip this step and just use another implementation. (A “Light Software Center” by the Xubuntu/Lubuntu team is already in progress, as well as something done by the Elementary project. I’ll write a blog post about these projects as soon as they become more mature)

6. Test it!

There might be some quirks in your data which need to be removed, e.g. applications listed which don’t belong there, wrong descriptions, apps listed twice, system services (KDE!) listed, very slow PackageKit backend etc. So, this feature just needs testing now. Otherwise everything should be set up and ready now :)

Please note: I’ll update this blogpost with new information if I find something I forgot to mention above. If I do changes, I’ll add a short note about the changes here.

August 09, 2012

With the most recent release of PackageKit, PackageKit 0.8.3 (published last Monday!), all my changes regarding parallelization have been merged into our master branch, which means parallelization features are now available for backends to use! Yay!

The best thing about the new parallelization is that it will have an incredibly high impact on PackageKit’s speed: Frontends are now able to process many resolve-requests at the same time. Also, you will no longer have to wait for PackageKit to finish installing packages before you can continue browsing the list of installed packages again. (This was the main reason to develop this feature in my SoC, as it is crucial for Software-Center-like applications)

Internally, many other things have changed also, which allow backends to do very clever cache-handling, so the cache can stay open for a longer time, so we can avoid the delay while a backend reopens the cache when running many transactions. We now have a “Backend” and multiple “BackendJobs”, which process specific requests, for example Resolve() or InstallPackages() which provides backend authors with a flexible way to handle transactions.

Parallel processing is implemented in a way where it is nearly impossible to get to a situation where everything is waiting and the daemon is dead-locked. (only a bug in PK or the backend could cause this) Avoiding this situation was – of course – very important, as otherwise the package database could be damaged or a database lock could just be never released.

The new changes also require some work of our backend authors. By default, parallel-transactions are disabled for backends which haven’t declared that they support it. Backend authors need to explicitly enable the feature as soon as their backend supports it. We highly recommend to enable parallelization, as PackageKit frontends might start to rely on it and will be much slower without a parallelized PackageKit backend. Also, backend authors will have to port to the new PkBackendJob infrastructure. Most necessary changes have already been made by automatic scripts, but of course it is better if people using the backend and knowing the package-manager would take a look at it.

So, backend authors: We want you! (to fix the backends ;-) ) Richard and I have created a porting guide for backends which summarizes all changes required for backends to be PK 0.8.x compatible. (backends/PORTING.txt)

Parallelization is a very invasive change and we have only one release with it enabled, so please help testing to find possible remaining bugs! :-)

And end-users can be happy about a much faster PackageKit soon, which will also be working as Software-Center engine ;-)

July 09, 2012

It’s been a very long time since my last blogpost, but you can be sure I haven’t gone lost! ;-) I was very busy with writing exams (and preparing for them) the last weeks, and university was giving me a hard time. Now I still have to write two more exams, but one of them is in August and I won’t have lectures anymore with the end of next week, so I have much more time again to work on my GSoC project on the Software Center.

So, what has happened so far? (If you read the mailinglists, you might want to skip that part)

I looked into the code, updated some very odd parts the SC used to access PackageKit, but there are still many pieces of code left which don’t look good and need optimization. During my work on the SC, I found out that with the current way PackageKit works it would be impossible to implement the Software Center in an user-friendly way. For example not being able to see the details of an application while another one is installing sucks. Also, the round-trips to the daemon slowed the whole thing down.

First I did some optimizations on the code which loads the package cache, so the SC now starts very fast. (It had an incredible startup time of ~30 minutes on my machine before, now it’s down to 4sec and faster (to show the UI, to be ready it takes ~20 more seconds)) I also did a few modifcations on PackageKit, which saved us some msecs to secs (depending on the requested action). To solve the general issues with PackageKit, I wrote a module for PK to create a cache of all packages. This solution resulted in a massive speed gain for the Software Center, but slowed down PackageKit actions a little, for example Refresh(). Also, the cache suffered – naturally – from all problems of a cache, e.g. it was going out of sync extremely fast. (You just needed to use a native package management tool) Additionally it was duplicating or even triplicating package data, and generally disliked by many people, including me in the first place. I first chose to implement the cache, because it was the easiest way to get the needed functionality (parallel access to package-data and fast cache-loading) in-time for my SoC project. But a workaround solution serves nobody in the long run. So I sat down with Richard Hughes, thinking about how we could change PackageKit to serve the needs of a Software Center. He rejected some of my proposals and we had a very extensive discussion about a suggestion by Daniel Nicoletti, but in the end we ended up with a solution which will allow PackageKit to execute software management tasks in parallel, if the backend supports it. The chosen solution is one which requires massive changes on the backend API and the most invasive changes in general, but it’s also a solution which does things right(tm), without any workarounds or more layers to access package information.

Implementing this functionality is not exactly trivial, we broke the PackageKit backend API completely, so all backends will now need massive changes to support the new functions and to even compile again. So, here’s a call for backend authors: Please fix your backends! You can see how the Yum, Zif and Aptcc backends were changed, there’s also very simple documentation of required changes in backends/PORTING.txt (file will be updated soon). The implementation is not complete, I’m working on the missing pieces now, so some changes might still happen.

All these changes mean PackageKit will soon be able to execute several actions in parallel if the backend supports it, for example running GetDetails() on a package while InstallPackages() is running too. We count on backend authors to implement this functionality, otherwise distributions running PackageKit without a backend which supports parallelization will not deliver a good “Software-Center experience”.

At the Software Center side, I’m currently implementing the PackageKit history feature, after that I will have to do some polishing and many ichanges on the code which talks to PackageKit, I guess – there’s lots of room for improvements! Also, there are some Ubuntu-specifics which need to be solved, as well as it is needed to split out the code which generates the AppStream Xapian database, so other alternative Software Center implementations are possible without the need to install the “original” Ubuntu Software Center. (I will work with Vincent and Michael on this)

If you want to try the current SC, just grab the code from Gitorious and try it – but be careful! You’ll need the latest development version of PackageKit (master branch) to make it work, but at the moment the only working backends are Yum, Zif, Aptcc and Dummy. So, Fedora, Debian and Ubuntu users are happy at time, all other distributions will need to update their backends as soon as the dust settled down. (There are also some more new and cool features in the unstable series at time, for example the systemd-support Richard has implemented, and the ofline-upgrade feature.) If your PackageKit is not the latest version, the SC will just crash on startup – I haven’t found out how to check version numbers of imported GObject Introspection data in Python to throw a proper error message. (in general it seems like version-checking is not used very much in Python-world) At the moment, using the SC is still not trivial, but in the end of this project everyone should be able to make use of it. (which is not the case at time, to be honest ;-) )

During the next weeks you can expect more code changes again, as the discussion part seems to be over, and I have more time again. (Doing cross-distro projects is 60% talking and 40% coding) I’ll also try to blog more, so everyone can stay informed without having to read mailinglist threads.

In the end, (and I think I should make that my signature) kudos to Richard Hughes for beeing an excellent maintainer and for helping me so much with the most difficult changes on PK. Also, once again thank you OpenSUSE for letting me work on this!