Lenses in Unity-5.0. Porting and New Features
January 18th, 2012Hi there. More gibberish about Unity lenses. You’d think that I don’t experience much else in life, huh? I am, believe you me, but for some reason it is so much easier to blog about technical matters 
Now, as some have picked up, the Unity Lenses API has changed slightly in Unity-5.0 (the version that’ll be in Ubuntu 12.04). First of all; sorry! As I’ll outline why we did this you’ll hopefully learn to appreciate it. Flames and pitchforks can go in my general direction if not. And if you still have hate to spare after that you can direct it at Michal 
Before we start, do note that the Unity-5.0 API overview on developer.ubuntu.com is already updated, and Michal is updating the wiki documentation.
I’ll take this on in the form of a case study; updating my unity-lens-bliss. This is a Python lens, which makes for a good example – identical changes apply to lenses written in Vala or C. Let’s roll.
Porting unity-lens-bliss to Unity-5.0
We introduced a new signal “search-changed” on the Unity.Scope class. The old property notification (on “notify::active-search” and “notify::active-global-search”) is still not available anymore, as the properties has been removed. The reason for the new signal was that the property notification scheme was racy in some subtle ways that would require some tricky GObject magic in the scopes to work correctly in all circumstances. The race manifested itself in lenses that dispatched the property change notification to an async handler of some sort. If the scope received another search while the async handler was still running we’d have re-entrancy issues in the async handler. This was the reason why you might have seen some mysterious calls to self.freeze_notify() and self.thaw_notify(). It seemed that no one really understood this, and I think we can all agree that having to know the intricacies of GObject property notifications is not a nice requirement for an API that should be simple.
For unity-lens-bliss, what was before:
self.connect ("notify::active-search", self._on_search_changed) self.connect ("notify::active-global-search", self._on_global_search_changed)
Should now become:
self.connect ("search-changed", self._on_search_changed)
The callback _on_search_changed() changes signature from:
def _on_search_changed (self, scope, param_spec):
to
def _on_search_changed (self, scope, search, search_type, cancellable):
The search parameter is a LensSearch instance. The LensSearch class has grown some new public properties. Of particular interest is the “results-model” property. This property will hold the correct model depending on whether it is a global- or in-scope search. You can figure out what kind of search this is by looking at the search_type parameter which is an enumeration Unity.SearchType with values Unity.SearchType.GLOBAL and Unity.SearchType.DEFAULT for global- and in-scope searches respectively.
So the implementation of the _on_search_changed() callback changes from:
def _on_search_changed (self, scope, search, search_type, cancellable): search = self.get_search_string() results = scope.props.results_model print "Search changed to: '%s'" % search self._update_results_model (search, results) self.search_finished()
to
def _on_search_changed (self, scope, search, search_type, cancellable): search_string = search.props.search_string results = search.props.results_model print "Search changed to: '%s'" % search_string self._update_results_model (search_string, results) search.emit("finished")
And this is all there is to it! Bliss will work fine in Unity-5.0 with these changes.
We haven’t yet mentioned the new parameter cancellable. Unsurprisingly (hopefully ) this is a Gio.Cancellable instance. If you’re writing a fully synchronous lens (like bliss is) it wont be of interest to you, but if you’re dispatching to async methods from your “search-changed” handler (like fx. both unity-lens-files and unity-lens-applications does) then read the next section carefully.
Concurrency and Cancellation
Before we get too deep into this, let’s make it clear what I mean by asynchronous. A method being async means that GLib will spin the mainloop while waiting for the method to return. This means that your app/daemon will continue to handle events (in particular requests from Unity to update the search) while your methods are running. Why would one want to use async methods if it is so complicated? Good question. If you’re just writing a simple scope or lens then chances are that it may not be worth it. But if you go beyond “simple” then it may matter.
Let’s imagine a slightly more complex scope. Maybe something that puts webapps inside the applications lens. The listing is done by first asynchronously querying a web service to list the apps and then asynchronously querying Zeitgeist to sort by popularity. If we didn’t do the web- and Zeitgeist queries asynchronously then the scope would block all requests while any queries were running. This would mean slower responses if the user changes the query under you (which is very likely when we’re talking live searching here), and also you’d run the chance of showing “outdated” results and doing work that you’ll discard immediately anyway. What you want is to be told in the middle of everything that “hey, there’s a new query; stop what you’re doing and do this in stead!”.
An alternative case where’d you want async searching is if you wrote a scope that was living inside an application. There’s really no reason why this wouldn’t work. And it circumvents some of the intricacies of sharing a datastore between a scope daemon and an app. Anyways, back to the example with the web apps. In simplified form it would look something like this:
def _on_search_changed (self, scope, search, search_type, cancellable): # Dispatch an async call with callback _on_web_apps_received() self._query_web_service_async (search, self._on_web_apps_received) def _on_web_apps_received (self, search, list_of_webapps): # Web apps listed by remote server. # Now sort them async with zeitgeist, with callback _on_webapps_sorted_received() self._sort_webaps_with_zeitgeist_async (list_of_webapps, search, self._on_webapps_sorted_received) def _on_webapps_sorted_received (self, search, sorted_list_of_webapps): # We now have the web apps sorted by popularity, # add them to the results model results_model = search.props.results_model results_model.clear () for app in sorted_list_of_webapps: results_model.append(...) search.finished ()
If you did something like this in the Unity-4.0 API then have to deal with all the re-entrancy, cancellation, and concurrent search handling yourself. Probably by elaborate application of freeze/thaw_notify() and Gio.Cancellables. Tricky stuff. In Unity-5.0 this is a breeze! Contrary to the old way with “notify::active-search” libunity goes out of its way to make the “search-changed” signal nice to use for scope authors (and no, it wouldn’t be technically possible to do the same with the old property notification system).
Firstly, libunity wont call you again before you’ve called search.finished(). So we’re re-entrancy safe in the example already. What’s more – libunity will cancel the cancellable parameter when you get a new query. So sprinkling some if cancellable.is_cancelled(): return lines around will make sure that you don’t do work in vain. We could fx. insert one right after we receive the results from the web service. Note that you don’t have to call search.finished() if you have been cancelled (libunity will ignore it if you do):
def _on_search_changed (self, scope, search, search_type, cancellable): self._query_web_service_async (search, cancellable, self._on_web_apps_received) def _on_web_apps_received (self, search, cancellable, list_of_webapps): # NOTE: The new parameter ^^^^^^^^^^^ if (cancellable.is_cancelled()): return self._sort_webaps_with_zeitgeist_async (list_of_webapps, search, cancellable, self._on_webapps_sorted_received) def _on_webapps_sorted_received (self, search, cancellable, sorted_list_of_webapps): # NOTE: The new parameter ^^^^^^^^^^^ if (cancellable.is_cancelled()): return ...
Filters
Bliss doesn’t use filters, so I didn’t touch on that yet. If your scope is using filters, the correct thing is in 99.99% of all scopes to connect the “filters-changed” signal to calling self.queue_search_changed(Unity.SearchType.DEFAULT). In Python:
def __init__ (self): ... self.connect ("filters-changed", self._on_filters_changed) def _on_filters_changed (self, scope): self.queue_search_changed(Unity.SearchType.DEFAULT)
Personally I’d probably do it with lambdas:
... self.connect ("filters-changed", lambda scope: self.queue_search_changed(Unity.SearchType.DEFAULT)
Out of Band Result Changes
Many scopes feature result sets that can change through external means. Fx. if you are listing the contents of a directory, listing browser bookmarks, listing recent stuff from Zeitgeist, etc. All can change when the user is doing something else than searching the lenses. When the result set should be updated, disregarding whether the search string has changed, you can call self.invalidate_search().
Search String Change Checking
In the previous paragraph I wrote “disregarding whether the search string has changed”. But when has the search string changed? Does appending a white space change the search string? Most lenses strips the search string from white spaces anyway; so in essence the strings “xyz” and “xyz “ are identical, seen from the scope. We don’t want to fire off a new search for these kinds of changes. Going further down this road – is “XYZ” and “xYz” the same as well? For most scopes, they will be. The problem is that this is highly dependent on the particular scope.
Doing change checking on search strings was a recurring chunk of similar code in all the default Unity lenses. In order to make this easier for our selves and everyone we baked it into libunity by means of the “generate-search-key” signal on the Unity.Scope class. This is a particular kind of signal that has a return value. The signal takes a Unity.LensSearch as input and returns a “normalized” version of the search string. This could typically be lower casing and chugging off white space at the ends. In code:
def __init__ (self): ... self.connect ("generate-search-key", self._generate_search_key) def _generate_search_key (self, scope, search): return search.props.search_string.lower().strip()
Cancellation and Transactions
Considering again the example with async searches and cancellation. One could easily imagine a scenario where you had a bunch of async methods, some of which added rows to the results model and then going on to dispatch more async searches before calling search.finished(). If we got cancelled in the middle of all this, the results model would be left in a dirty state with only half the results of the search. Enter Dee.Transaction.
Dee.Transaction is new class in Dee that implements the Dee.Model interface. You create a new Transaction instance, txn, from your results model, then go on clearing and adding rows to the txn model as you go through your chain of async calls. The real results model will not be updated before you call txn.commit(). So if you’re cancelled somewhere in the middle you just let txn go out of scope (or unref it if you’re writing in C) and it’ll vanish like the Cheshire cat. If you make it all the way to the end you call txn.commit() right before you call search.finished(). So with an example:
def _on_search_changed (self, scope, search, search_type, cancellable): txn = Dee.Transaction.new (search.props.results_model) self._query_web_service1_async (search, txn, cancellable, self._on_web_apps1_received) def _on_web_apps1_received (self, search, txn, cancellable, list_of_webapps): # First set of results retrieved, add them to the transaction # and then fetch some more results from another web service if cancellable.is_cancelled(): return txn.clear () for app in list_of_webapps: txn.append(...) self._query_web_service2_async (search, txn, cancellable, self._on_web_apps2_received) def _on_web_apps2_received (self, search, txn, cancellable, list_of_webapps): # Second batch of results if cancellable.is_cancelled(): return for app in list_of_webapps: txn.append(...) txn.commit () search.finished ()
Fin!
Wow, you’ve made it to the end of this blog post! You surely are an impressively patient person
Please feel free to ask questions or post corrections in the comments. Or catch me, kamstrup, or Michal, mhr3, on #ayatana on FreeNode if you’re into IRC.
Now as a bonus for your patience you’ll get… A FREE picture of Me Looking At A Webcam!
