On desktop data services

A few days a go a few students asked me to help them with designing a better EDS. I know I’m going to be hated for doing this blog item by some people (because, well, I’m pointing to some flaws in the architecture of some components. People don’t always want you to identify flaws).

Although I’m not very focused on calendaring and other things related to what EDS offers, here’s my try on the subject:

Evolution data server will via the notify_matched_object_cb in the ECalData lib issue a notifyObjectsAdded for all matched objects that your query (ECalView) wants. It seems it’s not doing that for just the ones that became recently visible.

+ e_cal_view_start
|- CalView_start (goes over the IPC)
| - impl_EDataCalView_start
|  + foreach: notify_matched_object_cb
|  |- notifyObjectsAdded (goes over the IPC)
|  | - impl_notifyObjectsAdded
|  |  - g_signal_emit "objects-added"
|  + end foreach
+ end e_cal_view_start

After that you will also receive notifications when an item that matches your query gets removed, changed, added, yadi yada.

Although that sounds reasonable, on a desktop, there’s a problem with this on mobiles: Unless you limit the query to exactly what you will see on the view, you’ll needlessly transfer a lot of iCal data over the IPC and worse, you’ll need to store it in the memory of the user interface (the client).

Depending on the backend implementation of EDS, this means that it’s in memory twice. Given that EDS is a locally running service, that’s a little bit stupid (if it would be a service running over a slow GPRS connection, I would better understand the need to fully cache everything in the client).

Another reason why you want to keep the memory at the service, is that the service is the centralized infrastructure. All clients using it, share the same memory. If your clients always need their own copies, you are effectively doubling all memory consumption for calendaring.

Although the user will do queries that are much larger (like: give me all items of this month), a mobile device’s view will most often display only a few calendar items. Which of course makes a good developer think about the other ones: do you really need them in memory at all times? Or is a proxy at the client good enough? A proxy that will get the real thing from the service, by asking a factory, at the time the user starts using it.

Therefore wouldn’t for example a cursor style remote API be much better? The model of the view would get the currently visible one by simply iterating to it, and only then getting it.

A cursor is quite simple and looks a lot like a C++, Java or .NET iterator indeed:

c = create_cursor (expression)
c.move_next, c.move_prev
c.get_current

The view would get a model that utilizes that cursor efficiently. For example: the view asks the model for the 100th item, the current position of the model is 80. So the model will do a c.move_next 20 times and then give the view c.get_current. Finally the view unreferences the instance as soon as the item is not visible anymore.

That iterator doesn’t have to be implemented using only remote calls. It can be emulated by storing the query result as long as the cursor is kept alive (or let it die on timeout or something) in the service, and implementing a get_current that takes a query id and an “nth” parameter. The move_next and move_prev are implemented locally (just keeping a “current nth” or position status as an integer).

Is this slow? Probably will the experience for the user be a lot faster than having to initially download the entire result-set of a query. It’s true that the performance would be slow when a lot items are visible: that’s because a lot c.get_current calls would happen. But then again, most mobile devices have small screens and therefore can’t display a lot calendar items in a meaningful way to the user.

Also, as a solution for that, you can make a proxy that has the first 10 characters of each once received item’s description cached. The model can now instead of returning c.get_current, return a proxy. The view can once the item gets invisible clear the real from the proxy. If the view is set to display a lot items, it would only ask for those first 10 characters: the proxy would only the first time need to get the real to fulfill that API. Zooming in, though, would make the view asks the proxy for information that it doesn’t necessarily have (any more), so the proxy would ask the model (or a factory) to do a c.get_current (getting the real) to fulfill the interface of the type for the view.

But really. Instead of an implementation like EDS, both KDE and GNOME experts should stick their heads together and create a D-BUS specification for this. Perhaps one that copes with that cursor idea?

Clearly, both teams are most likely not going to agree on sharing one implementation soon.

…

I see frightened people screaming and yelling after I just said that. That’s not necessary. See, guys, dear users, we developers do talk with each other at conferences. We love each other! We love competing! Competing makes both sides better and sharper. Don’t you sometimes do friendly competition with your partner?

With a good specification, we could (and eventually would too) compete on implementation. It’s like agreeing on the rules of a game of Pool with your partner. Or bowling.

That is why I told those students to focus on a very good D-BUS spec. Perhaps do a proof of concept initial implementation to proof test your new D-BUS protocol?