The Evil Blog

Leading up to our last Zeitgeist release (0.3.1) I hacked up our new Blacklisting- and Monitoring APIs, both things quite fun work and very useful API if I might say so my self But I regret not blogging about it as I wrote it – we gotta keep them olde hype-wheels a’turnin’. So here we go about the next feature on my plate…

Storage Awareness

So what does the buzz wordy term “Storage Awareness” cover? We had a few requests from application developers like:

1. “I don’t want to show online resources when there is no network interface up”
2. “I don’t want to show work related to files on a disconnected USB drive”
3. Or another one I just came up with: “When I plug in my USB-drive show my recent activity on that device”
4. Very much related is how deleted files should be handled in the results, but I will not discuss that right now
   

Since Zeitgeist is a log and not a snapshot of your environment we will keep the information around even if you delete files or detach your storage devices. So you might indeed get data about subjects that are not readily available when you query the log. However, the use cases above seems valid and applications stat()ing each file:// URL in the result set seems like a very bad idea, so it would be nice if we could help a bit with this.   Even though we are “just” a log doesn’t mean we can’t provide some nifty API for application developers.

So our query API has flag that filters events to only those events with subjects that are “available right now”, it has not been functional until now, but it will be so for 0.3.2. Since we also log information about what storage medium each event subject uses one can also ask for recent- or popular stuff on a given storage medium.

Storage Identifiers.. Help?

We associate each subject URL with a storage medium via a unique string identifier. For stuff like USB drives we have the UUID readily available from GIO. For online resources we simply use the id “net” and I use NetworkManager to check for network availability (ConnMan should be easy too).

So far so good, but I have not been able to handle CDs (both audio and data) and DVDs properly yet. I am not a storage format expert so I don’t know if it’s even possible to obtain a unique identifier for a given data CD (or what have we) – at least I can only get the disk label from GIO and that is not unique (but it might very well be “unique enough” for this to work well in practice). So any help on obtaining real unique ids for CDs and/or DVDs would be appreciated. Note that I would like to use the G* stack and not introduce funky dependencies – and I am also not going to read the first N bytes and checksum those.

Next bump on the road is that it seems that I can not get the disk label from within from gio.VolumeMonitor’s "volume-removed" signal handler (calling volume.get_identifier()). I just get a None whereas I get the right label in the "volume-added" handler. I can probably figure this one out, but any ideas are appreciated.

The Code

Hold on… While you can indeed dig out the code from Launchpad, it’s not a secret,  I would recommend that you wait a bit just yet. It’s not ready for testing (and not even wired up in the engine so you have to do that yourself). So no code pointer for you, sorry

Yesterday we pushed out version 0.3.1 of the Zeitgeist engine. I am pretty hyped about this, but since I try to avoid posting too many release announcements on my blog I will talk about something completely unrelated in stead. Read the Zeitgeist 0.3.1 announcement in the Gnome Announce archives.

Big Numbers

As we where talking at the dinner table today I asked Liv (just 4y) “What’s the BIGGEST number you know?”. It was obviously that she enjoyed getting that question, and I almost felt I could look inside her mind as she was contemplating it:

With utmost self satisfaction (clearly believing that no one could possibly think of a bigger number) she replied “As big as the TABLE!”.

Gravity or No Such Thing?

Most intrigued by Verlinde’s claim that Gravity is nothing but an emergent force arising from entropy. It has everything a good theory should have: simple, and based on some novel ideas. I really really hope that he is right, because if he is then it might just be the biggest scientific breakthrough (in theoretical physics at least) I am going to see in my lifetime. His claims has one other trait that worries me though – when things look too good to be true, most people, who doesn’t live in a fairy tale, will know that they are indeed too good to be true. Follow the buzz on Google blog search.

In between sick children, work, my daughters birthday, my son starting in kindergarten, and what have seemed to be an endless stream of bumps in the road which is otherwise known as life, I stole a few moments this afternoon and rolled up our first development release of the new Zeitgeist release series – Zeitgeist 0.3.0.

I am pasting the release announcement here because I know all you lazy ass busy readers out there wont bother have time to click a link to the gnome-announce-list archives [2]

Hi,

On behalf of the Zeitgeist team I am proud to announce our first
development release, Zeitgeist 0.3.0, leading up to what will be our
stable series which will be 0.4. It is our intent to aim for a 1.0
release as soon as we feel good about the stable series, but that is
still a bit in the future. Now that we've crossed the initial hurdle
in the rewrite we expect the release cycle to be much shorter than
this one, although we have not settled on something strict yet.

As many of you know the bulk work on this release was done in the
Zeitgeist hackfest in Bolzano. Since we came back we been busy little
bees polishing it up and fixing bugs - trying not to flame each others
too much when discussing the designs   Working face to face in
Bolzano gave us a unique chance to really discuss things through and
get to the bottom of the details. This will also affect other
developers a bit since...

We were bad boys and decided to change both our internal database
structures as well as our public DBus API. Sorry - but after long
discussions we all agreed that this was for the best. The new design
is leaps and bounds better than the old one. This means that you both
have to give up on your old log database, and accept that there are no
GUI written for the new API just yet. This is being worked on as you
read this though!

Something that might come as a shock to some other developers is that
we decided not to store annotations and bookmarks within Zeitgeist.
This should be done in Tracker or some other semantic metadata
storage[1]. Zeitgeist answers only when and how data was accessed, but
stores no information about the current state of the metadata. We will
be working very closely with Tracker from now on since 0.7 is a
blessed dependency for GNOME 2.30. Congrats to the Tracker Devs.

You can download the release from: https://launchpad.net/zeitgeist/+download

The NEWS entry reads:

First development release leading up to the stable 0.4 series. This
release features:

 - Complete rework of engine and DBus API. Read: apps written against 0.2.*
   will most certainly need an update (see fx.
   http://mail.gnome.org/archives/desktop-devel-list/2009-November/msg00019.html)
 - Public Python client API defined in zeitgeist.datamodel and
   zeitgeist.client modules
 - Documented public API with Sphinx (we'll have an URL for you shortly)
 - Changed Ontology from XESAM to Nepomuk.
 - Removed the Storm backend (obsoleted in 0.2.1).
 - Removed the Querymancer backend.
 - Support for event payloads (binary attachments to events)
 - An extension API for the core engine, allowing extensions direct
   access to the DB. There are already a handful extensions things in
   the works here, you will hear more about this later

There are a few DISCLAIMERS that needs to be attached to this:

 - The event notification/signals are not yet ready in the new DBus API.
   We expect to have that ready for 0.3.1.
 - We plan to support querying only for available items (eg. filtering out
   deleted files, not listing files on detached USB storage, etc.). However this
   feature is not fully supported yet, even though it is exposed in the API.
 - While we are pretty satisfied with the database layout, there may still be
   changes in the ontologies or concrete data extraction methods. This might
   require that users delete their log databases in order to rebuild them
   with the new definitions. Of course this will no longer happen when we
   go stable
 - Much related to the point above our event ontologies are not yet set in stone,
   and minor changes are expected
 - We have only one logger enabled for now. Namely the one monitoring your
   recent files. In coming releases this logger may well be deprecated in favour
   of application specific plugins.
 - And finally. Please note that this is a *development release*. We can not
   guarantee stability of services nor APIs, although we strive hard to keep
   things stable.

Cheers,

Mikkel

[1]: There have been talk about defining (and implementing) a very
simple DBus API for storing semantic annotations (bookmarks, tags,
comments, ratings, etc). In our internal speak such a service is
called a Repository. Tracker or Soprano would expose this API in most
cases, but on platforms where they are not available the simple
Repository implementation would be most handy. This being said, it is
currently not a high priority to implement a Repository, there are
alternatives ready in Tracker and Soprano.

[2]: Sorry it’s not a direct link to the announcement, because I am still waiting for my mail to get through. If you must see it in a mail archive then check the Zeitgeist list archives on Launchpad.

There is a tangible confusion around as to what Zeitgeist is and what it isn’t; what it can do and what it can’t do. This is partly our own fault because we could have communicated this whole thing better, for instance we have some very outdated wiki pages lying around that you should probably stay away from until we updated them. In this post I aim to give a semi technical run down of the core Zeitgeist functionality and how we expose it for you to work with. This should hopefully clear out some confusion.

Events

The Zeitgeist daemon (also known as the engine) is a process that exposes an event logging framework as a DBus API. The structure of these events is that they have a block of metadata that describe the event itself (this is known simply as the event metadata) and another block of metadata that describes the subject, or subjects, that this event happened to (this part is known as the subject metadata). The metadata for the event looks like:

• Timestamp – When did this happen. Milliseconds since the Unix Epoch. Note that we see events as single points in time, meaning that events don’t have a duration
• Interpretation – Abstract interpretation of this event; what happened. Fx. “opened”, “saved”, “closed”, “send”, etc.
• Manifestation – How the event happened. Fx. “user activity”, “notification”, or “scheduled activity”.
• Actor – Who triggered it. This will typically point to the .desktop file of the acting application. It will most likely be an application, but it is not required to be so.
• Payload – A free-form binary blob that you may attach to the event. This is specifically application specific and mainly intended to be a “back door” for people to do all sorts of funky hacks.

Each event has one or more subjects associated. For each subject we store:

• URI – You guessed it! The URI of the subject
• Interpretation – Abstract interpretation of the subject. This could be “Document”, “Image”, “Video”, “Email”, “Instant message”, “Contact”, anything.
• Manifestation – How the subject is stored. This could be something like “File”, “Mailbox”, “Web page”.
• Origin – A URI pointing to the origin or “patron” of the subject. For files this would be the parent folder. For YouTube videos it would be http://youtube.com
• Mimetype – The format of the datastream representing the subject. Fx. text/plain, application/xml.
• Text – Textual information added to the subject. This is not applicable for for types of subjects.
• Storage – Identifier for the storage medium this subject resides on. We use this to make it possible for queries that return only events for subjects that are “available now”. Fx. some clients don’t want to show events for files that are stored on you USB pen drive when it is not connected.

Ontology – Or Data Description

In reality the metadata fields we store don’t contain simple strings like “Document” for the subject interpretation. It’s a bit more complex than that – sorry! We store a URI pointing to a formal definition of something categorized as a Document. This formal categorization is called an ontology if you want a word to confuse your friends with. We are fortunate enough that someone already wrote such a spec, namely the Nepomuk Ontology. So instead of just “Document” we store the string”http://www.semanticdesktop.org/ontologies/2007/03/22/nfo/#Document“.

Since Tracker also uses the Nepomuk ontologies you may take these formal classification strings and plug them directly into Tracker to find everything that Tracker considers a document.

We will also have an ontology for the event metadata as this is not covered by Nepomuk. We are actively working on this.

Getting Data Out – Querying the Event Log

We employ a template based query API for searching our log data. You send us a list of event templates you want to look for and how you want it sorted and we give you the results. So if you want to find all “open” events on subjects of type “Document” simply create an Event object, set the interpretaion to “open”-event and add a subject to the event template with the interpretation set to “document”. All other fields should be left blank. Send this template to us and we will give you the matches.

The list of event templates is collected into a big OR-query to imbue the consumers with more power.

Getting Data Into Zeitgeist

There are really no limits to what kind of events we could store. If you have a spare mobile with a in-built accelerometer and glue it to your front door then you could send an event over bluetooth to your desktop each time your front door opens. Probably there are better use cases?

The point is that the usefulness of Zeitgeist stands and falls with the events that you push into us. We can store anything that you can model using the structures I outlined above. I am pretty certain that people will not agree on the kinds of data they want logged, but we are ready for anything

Normal users would of course not need to think about getting their data into Zeitgeist. What developers need to know is that we have a simple DBus API to insert events (surprisingly called InsertEvents). It is called InsertEvents and not AppendEvents or something like that for a reason. Namely that you are allowed to insert events that are in the past. This is useful if you want to import your Firefox history or what ever. If you try to log an event twice the engine will throw an error at you, so no need to worry about dupes.

Ok. I think that about wraps up what I intended to say for now. Hope it’s useful to at least one person out there!

Sitting here on my final day of the Zeitgeist hackfest. I really wanted to blog more, but I’ve been so busy hacking that I didn’t find the time. I hardly slept or ate either! I will follow up on this post with more technical details from the hackfest.

The stay here in Bolzano has been very nice. Fresh mountain air and very warm conditions compared to my usual Danish habitat. I want to give a warm thanks to our sponsors and organizers for making this possible, the whole arrangement has been run very smoothly and I don’t have a single thing to complain about. So big props go to sponsors:

And let us not forget the horde of local Bolzanites (what’s the correct term for people living in Bolzano again?) who helped us around the city and helped us at the CTS (our hacking venue). They showed an extraordinary display of patience and hospitality helping the flock of geeks around Thanks guys and girls!

As I mentioned in the first paragraph we’ve been extraordinarily busy hacking. More or less skipping lunch and supper, hacking until we collapsed from fatigue. I was working fully in the engine team and I really really wanted us to have the new engine in a working and unit-tested state after the hackfest (so we can give you a development release during the next week hopefully). I had a moment of despair in the middle of the week where bugs just continued to pop up and we had a hard time coming up with the right architecture. With a relentless effort from the entire engine team we pulled it through in the end though, and I am really happy about our new design and API. Perhaps I am just a tiny bit biased, but I really feel that our new design feels “just right”

The UI team where very busy as well and I saw lots of very cool stuff being hacked out. Let’s hope that they choose to share some of the bling with us I know as a matter of fact that they made huge progress on the Gnome Activity Journal and I can’t wait to have a release of this ready on top of the new engine.

That’s all for now, I intend to follow this post up with a more technical post later. Ciao!

I’ve prepped up for the upcoming Zeitgeist hackfest in Bolzano, bags packed, laptop charged, and kissed the kids goodnight and extra time. I have to get up at 3.40 this night in order to catch my plane.

It’s going to be great to catch up with the other developers and finally meet them face to face. It’s always an odd feeling meeting people in real life for the first time when you talked so much with them on various online mediums.

My personal aim for the hackfest is to do a lot of coding and really push Zeitgeist closer to production readiness. We done a lot of drafting and discussion lately, so we are really set to get the actual coding done now. I have a lot of travelling time before I get there so I expect to get a head start on the hacking

Huge props to the sponsors and organizers!

I just posted a long status update on the Zeitgeist project to Gnome’s desktop-devel mailing list. I bring it here in bloggified form to help spread the word past the desktop-devel crowd.

With the 2.30 module deadline passed it seems appropriate that we give a status report from the Zeitgeist team.

Since there have been a good deal of confusion about what Zeitgeist is, and isn’t, about I will try clear this up in this mail as well. I will try to stay low on the buzz word factor and leave some of the more exotic use cases out to avoid too much speculation.

Zeitgeist in 1 sentence

Zeitgeist is an event logging framework used to keep a log of user activity in a structured way.

What new services do we provide for UIs and applications

Zeitgeist provides a DBus API to query and update the activity log. Clients can query on time ranges, the acting applications, mimetypes, and Nepomuk classifications of the subjects and events. Sorting can be done on various criteria such as usage frequency and recent usage.

Concrete examples could be “Get me most used files of mimetypes x,y or z between the months January till March”

One can also query for documents that are used in context with others. As in “Which documents/websites are used with http://youtube.com within the last week”.

It is also possible for the applications to get notified when the log is updated. This is for instance used by the Parental Control application as well as the GNOME Activity Journal.

What Problems can we solve

The straight forward use case is as a GtkRecentManager on drugs. Zeitgeist removes the need for each application to parse a big XML file to retrieve recently used documents. It also removes the need to ever truncate your usage history, our database format is compact and can easily contain years of history. My estimation is that 1M log entries will take up about 80mb (give or take 20mb).

Open up for a range of query capabilities that GtkRecentManager doesn’t provide. Instead of simply storing the most recent usage event on a resource we store all usage events. This way we can not only answer when the most recent use case was, but also account for the entire usage history.

One use case that is already in the works is having the most used resources within the last 3 weeks for an app in the context menu in a window list. This is for example done in Docky.

Looking past just logging resource usage we will also start monitoring window and document focus times. This opens up to a whole new world of contextual relevancy that I wont elaborate on here. I am trying to stick to the more down to earth aspects of Zeitgeist.

Which processes/daemons do we run

Zeitgeist itself is a single DBus daemon. Where the picture gets a little more fuzzy is how we collect events. The long term goal is for apps to submit events, maybe hooking directly into GtkRecentManager, or in any case provide a very convenient way for apps to do this. Apps like Pidgin or Empathy would probably need some plugin for logging usage statistics of your contacts.

Right now we resort to less elegant ways of collecting events, like running a separate daemon harvesting Firefox’s history, GtkRecentlyUsed’s and other applications’ history (this daemon is also known as the datahub). The datahub is already on its way to becoming redundant now that a Firefox extension is in the works (and one for Epiphany already exist). It is our intent that the datahub should eventually go away as application support becomes widespread, but it
may eventually still prove useful for usage together with online service.

How resource hungry are we

Normal memory usage is around 5-10mb for the core Zeitgeist daemon. The datahub process (and I repeat; we want to get rid of this) is about 12mb.

What dependencies

Right now the daemon depends on SQLite, Python 2.5, python-gobject, python-xdg, and python-dbus. For the datahub we additionally need python-gconf and python-gtk2, but the datahub is optional.

Future plans

We have spend a lot of time planning and designing lately. When we have a stable reference implementation of our design in Python we plan to use that as a template for a C implementation. To be clear – the C version will be log-format and API compatible with the Python version.

We plan to make good use of the upcoming Zeitgeist hackfest and should have a 0.3 development release ready shortly after. If we are happy about the 0.3 series we will rename it to 0.9 and go for a 1.0.

Regarding Gnome 3.0 I think we are in a situation much like Owen Taylor recently outlined for Gnome Shell on the release-team mailing list. If we are desperate for Zeitgeist to be included in a Gnome 3.0 this March I believe it would be doable. It will require that we really bust our backs and cut some corners, but it’s doable. Personally (not speaking for the Zeitgeist team here) I am not sure it would be a very good idea for the same reasons Owen mention.

Relation to Tracker and Other Semantic Technologies

The very short version of this is that Tracker and Zeitgeist does not depend on each other in any way. The catch however is that either one becomes a whole lot more powerful when working together. To take an example consider tagging. Zeitgeist is just a log so we don’t manage your tags, we are however fully equipped to understand events concerning your tags. So you manage the tags via Tracker and track their usage in Zeitgeist. The combined power enables one to reason about what tags relate to resources in a temporal manner, even with resources that are not tagged.

In the Zeitgeist world we call an application like Tracker a Repository. Nepomuk or Desktop-CouchDB might work as other Repositories. If there is some confusion in this area it is understandable, since we do have some Repository-like features in our 0.2 series. This is however removed from the 0.3 series. It is still undecided if we want to define a minimal Repostiory DBus API for Zeitgeist and then ship a reference impl. of this API (which would run in a separate process). Any full fledged Repository would be able own the Repository service on the bus and Zeitgeist would not run its own. But again let me stress that a Repository is not needed for the Zeitgeist Log daemon to be useful.

Of course you’ve read today’s XKCD. Those of you in the know might even have gone to xkcd.com to view it – geocities old school style. But the real gem comes when you click view source (at least if you are into web development). Now go, you know you wanna! Peruse- and enjoy the details.

- In which our hero gets to dust of his dormant math geek genes.

Not so long ago Alessio Bolognino approached me with a patch adding a preliminary spatial window switching to WinWrangler, like Thomas blogged about a while ago in the Metacity blog. Before we move on let me quickly clarify that by “spatial window switching” I mean that you switch windows not by focus-order, but by selecting neighbouring windows in the directions up, down, left, or right. This makes good sense in a tiled window setup (like WinWrangler provides for any EWMH compliant WM).

The problems start to arise when you begin thinking about just how to select the expected neighbour window. The problem is easily solved if the windows are tiled in a perfect grid. Here the first problem lies – even though WinWrangler has manually constructed the window grid it is not guaranteed to be perfectly aligned. A skew grid can happen if the WM respects the size increment step hints from the apps. Concretely; Gnome Terminal likes to increment its size  in steps corresponding to the current character size. Metacity respects these hints – Compiz doesn’t.

The initial approach used edge-detection of the windows, which led to some unexpected behavior in the skew grid case, as well as relying on some very strict logic to select the neighbour window. Strict logic maps surprisingly bad to human perception – surprisingly, yes; you think it maps bad, but in fact it maps really bad.

I tried porting the window selection to something which was based on the windows’ centers of gravity instead of edge detection. This worked a lot better, but now I turned my testing towards a normal non-tiled session. Consider the setup below:

Window 1 has focus and I hit Left

Undoubtedly this is very subjective, but I would expect window 2 to get the focus. But as you can tell the red X (center of gravity) in 1 is closer to the X in 3 than it is to the X in 2. Meaning that my algorithm would select window 3 as the neighbour. Bummer.

While walking the dog I was thinking about hideously complicated algorithms to always “do what I think”. It was then I had one of those EUREKA moments. Just change the geometry!

This needs some explanation. You see I am really a Math major in Topology. I think that normal flat Euclidian geometry is for sissies. When measuring distances in a flat plane the distance between points (x1, y1) and (x2, y2) is found as we all learn in kindergarten:

Distance between two points in Euclidian geometry

But what if the points do not lie in a flat plane? Maybe we stretch the plane like a rubber membrane so the points lie on a curved surface:

Windows' centers of gravity on a curved surface

In this case the distance down to point 3 might actually be longer because the plane has been stretched. On such a plane differences in the Y-axis will yield a relatively bigger distance than it would in Euclidian space. The distance formular could now look like:

Non-Euclidian distance function, strecthing the plane in the Y-axis

As it turns out this actually selects the expected neighbour – “expected” for my deranged mind at least

Needless to say we need to stretch the plane in the X-axis when we want to select windows up or down from the current active one. This is left as an exercise for the eager reader.

I really like this solution. It was a very simple code change to get it working and I got to go all nostalgic on my Topology past You can try it out from WinWrangler trunk (bzr branch lp:winwrangler) – it requires GtkHotkey 0.2.* it will not compile with GtkHotkey from trunk. You trigger the switching with the hotkey <Control><Super>Up|Down|Left|Right.

Happy window switching!

I received a patch for WinWrangler from Alessio ‘molok’ Bolognino adding a new layout called “2/3 Layout” – inspired by a similar feature in XMonad. The basic idea is that you expand the active window to fill 2/3 of the desktop and fit the remaining windows into the last 1/3.

If I have 4 windows on a workspace with a Gnome Terminal being the active window and activate the 2/3 Layout it looks like this:

Warning: The 2/3 layout is quite addictive – I have never been into the tiling-window-manager-deal, but Metacity+WinWrangler really hits the spot for me

You activate the 2/3 Layout by pressing <Control><Super>3.

WinWrangler 0.2.3

I just pushed WinWrangler 0.2.3 online with the new hotness enabled – get it before the cool kids at launchpad.net/winwrangler/0.2/0.2.3

Secret tip

If you desperately want to change the hotkeys of WinWrangler you don’t need to recompile it – simply edit ~/.config/hotkeys/winwrangler.hotkeys