Iterators and tree models! Shocking!

Iterators

Now that Murray has posted about the iterators idea I can no longer hide.

Together with Jürg, the Vala man and Murray, who before becoming a pregnant guy was and still is coding on things like Glom and gtkmm, I’ve been writing up a document. This document explains iterators like the ones you’ll find in .NET and Java.

This is related to the Iterators concept because the document about iterators could be part of a solution for this.

I once had to write a custom GtkTreeModel (I think I still suffer from vertigo) and therefore I sent treeview Kris an analysis about this.

There’s a rule that an interface should be a contract for solving exactly one problem.

Use interfaces to say “What objects can do” or “What can be done to an object”. You can let a class implement multiple interfaces. There’s absolutely no need to make huge interfaces. An interface should define that your class “Does One Thing”. Another interface defines that your class “Does Another Thing, too”, and a last interface defines that your class “Also Does This”. My class Z can do D, E and F. Therefore: class Z implements D, E and F.

Instead, the contract being GtkTreeModel requires you to solve five problems:

  • being iterable: you can go to the next node
  • being a container: you can add and remove things from it
  • being observable: it notifies a component about its changes
  • being a tree: sometimes items can parent other items
  • having columns: it’s something that describes columns

Summarizing: interface TreeModel means BEING a D, E, F, G and H at the same time. Correct would be if you specified that TreeView requires a model that does D, does E and perhaps also does F. In case treeview is showing the things as a tree, it requires that the model also does E and does H.

GtkTreeModel should have been five different interfaces.

Shocking! But in my opinion, true.

The consequence is that having to implement a GtkTreeModel has become a difficult task.

“You have to implement solutions for all five problems anyway!”, you might think. That’s true. However! If there were five interfaces, other people would have made default solutions for four other interfaces. Meanwhile “you” focus on “your” specialization. For example Rhythmbox and Banshee are specialized in huge models that contain a specific set of data that must be searchable. Preferably search and sort are done by a database instead of a model-filter.

Banshee and Rhythmbox developers:

  • … don’t have to care about the Columns problem
  • … don’t have to care at all about the Tree problem
  • … don’t have to care about the Container problem: they never need to add nor remove things from it because they set up their model instantly by passing a query to a data source. Maybe the feature “remove from playlist” requires solving the Container problem?
  • … unless they support inotify on the MP3 folder of the disk, they don’t even have to care about the observer problem

Yet with GtkTreeModel you have put the burden on application developers to solve all five the problems in their model. No wonder Banshee developers decide to code their own TreeView in .NET!

Durham, a beautiful city

Since last week I’m staying in Durham together with Tinne for the next two months.

Tinne went to Durham last month because she’s doing her work placement. University college students in Belgium have to do three months of work placement in order to graduate. Tinne studies languages, so it was especially interesting to do her work placement in another country.

For me it doesn’t matter where I do my job, so I decided to join her in Durham city. We made some pictures to make you guys jealous.

Cathedral, taken from Framwellgate Bridge

I kinda wonder why we organized Guadec 2007 in Birmingham, given that Durham is such a beautiful city.

Almost very happy

After being jealous about my girlfriend’s Fusion install on her cute MacOS X, I’m almost happy with VMWare 6.5 beta 1. As you can see on this screenshot, it for example played tricks on each Window that I touched. In the gnome-panel, they all start blinking and don’t stop blinking any more.

Moving Microsoft’s Word window to my second screen was not a very good idea because after I did that, I was unable to move the window any longer. The Window also got a piece of guest-OS desktop-background at the left (of about 150 pixels) and was itself shifted to the right within the window that Unity seems to create for it in X11.

Moving the notepad window over Microsoft’s Word window made Microsoft’s word window stop being redrawn until I activated it. I had to “paint” the Window or changing a piece of text or selecting text to get it back in shape.

Given that Tinne’s Fusion’s Unity feature does actually work perfect, I guess I’m still going to be quite jealous for a few more months. Until, I guess, people like ChipX86 fix all this kind of problems in the beta.

But, it sure looks promising. Cool. etc etc. Sadly is the beta version being ran in debugging mode. Therefore my machine becomes dog slow as soon as I start using VMWare. I hope I can still downgrade to my old VMWare Workstation, because the VMWare tools upgrade has been installing quite a lot of new drivers in the guest Windows OS. Lucky me I have made myself a snapshot before all this.

Proposing to make a snapshot is perhaps something VMWare Workstation could ask its users if it notices that a beta is being started? Just an idea, of course.

Reference counting

There’s nothing wrong with adding a reference to the GObjects that you’ll pass to a deferred call.

For example (also read GdkLock, for the gdk_threads_enter thing):

static gboolean
callback (gpointer user_data) {
  GObject *instance = user_data;
  gdk_threads_enter ();
  /* work with instance and don't worry */
  gdk_threads_leave ();
  return FALSE;
}

static void
MyMethod (GObject *instance) {
  g_timeout_add_full (G_PRIORITY_DEFAULT, 5*1000,
		callback, g_object_ref (instance),
		(GDestroyNotify) g_object_unref);
}

Don’t do this:

static gboolean
callback (gpointer user_data) {
  GObject *instance = user_data;
  /* work with instance AND worry */
  return FALSE;
}

static void
MyMethod (GObject *instance) {
  g_timeout_add_full (G_PRIORITY_DEFAULT, 5*1000,
		callback, instance, NULL);
}

The problem is that most applications have multiple contexts. Either are those contexts threads or multiple code-paths for ‘instance’ to become finalized. For example D-Bus handlers, threads (yes, some people use them) or callbacks caused by GtkWidgets. Your timeout’s callback depends on that ‘instance’, therefore it’s required that you add a reference. Just like how a garbage collected higher language would add a reference to make sure that the instance wont be garbage collected during ‘callback’.

When I look at a lot of GLib/Gtk+ consuming code, I conclude that some people seem to be afraid of g_object_ref. Some people seem to believe that overuse of it creates memory leaks. The reality is much more simple: one misuse of it, causes a memory leak. Using it when required, doesn’t. Just be careful for cyclic references: an instance ‘A’ that references another instance ‘B’, that itself references ‘A’. Sometimes there is a type in between ‘A’ and ‘B’, like a container type. Use weak references for those situations in stead. Do use references when they are required, though.

In the example above nothing guarantees you that callback wont be interrupted. Nothing even guarantees you that callback will be started with a valid instance in its user_data. I have seen people putting G_IS_OBJECT checks at the top of their callback implementation. This is wrong and doesn’t really make it any better. Right after that runtime check your callback can get interrupted and you can again find the instance becoming finalized while using it in ‘callback’. By adding a reference, you are sure about the reference that you added. You are also sure that g_timeout_add_full will make sure that the GDestroyNotify will eventually execute. So you are sure that the g_object_ref is not causing any reference leak.

Also note that code like this is fine, indeed. Sure the two g_object_ref calls are atomic increments and therefore a little bit slower than avoiding one of the two. But it’s consistent. When a new developer reads that code he’ll immediately recognize that my_thing_get_instance returns a reference. It’s my opinion that readability is more important than a micro optimization hack.

static gboolean
callback (gpointer user_data) {
  GObject *instance = user_data;
  gdk_threads_enter ();
  /* work with instance and don't worry */
  gdk_threads_leave ();
  return FALSE;
}

static void
MyMethod (void) {
  GObject *instance = my_thing_get_instance ();
  ...
  g_timeout_add_full (G_PRIORITY_DEFAULT, 5*1000,
		callback, g_object_ref (instance),
		(GDestroyNotify) g_object_unref);
  ...
  g_object_unref (instance);
}

RE: EDS and Memory

Federico, I’m not going to make yet another long and technical blog post about this, because it’s rather obvious how it works. But you can make a db-cursor or a cursor-like API for a query that is solved at the service and kept in a session.

My opinion is also that 484 bytes actually is a lot of memory per item. Wrap that in a proxy instance and lazy load the proxy, and you reduce that to an actual average of 50 bytes per item. It wouldn’t even matter if when parsed the size exploded. Only the items that are needed right now would be exploded in memory, in stead of all items (which is now the case). You could even cache the strings that are needed to render an overview of the items in the proxy, and avoid creating the real this way. This would give you a solution that could scale to 100,000 items (although that’s probably stupid to render in a view, sure).

Note that for this proxy concept, you don’t even need a cursor-like API. Just an API to fetch one real item, and a way to fetch just the minimal info for your proxies, so that they know how to fetch their real.

About the structures that tend to explode in size when you parse them: is the library using GStringChunk to store all the tiny little strings? Else I kinda know already where all that heap-admin is going to. It’s a lot, if you measure it. That’s because all of them are stored in parsed structures, right? Each mini string needs four bytes for the pointer, at least four bytes for heap admin, etc. I can also imagine that this data contains a lot of duplicate strings, perhaps take a look at how camel_pstring_add works? If GStringChunk is sufficient, then go with that, of course.

Or, even more simple:

If you’d replace all \n characters with \0 characters in the original VCard string, you could also just make them point to an offset in the original buffer. Note sure about line wrapping and whether the format could support this easily (not my field of expertise).

Also note that the prevailing model for E-mail might be that you display a usually huge linear list of messages, but that the market is increasingly demanding a search-based model. Especially for mobiles with small screens.

Clarification:

  • A proxy here is an instance that has the same interface as the actual thing. To fulfill that interface it might have to get a real first, and then proxy the command to the real. A proxy usually only contains the things that are necessary to fetch a real from the service. It can also contain extra data (I call this a micro cache) so that it can avoid having to fetch the real for common strings. For example strings that are displayed in an overview screen. A proxy must be small.
  • A real is an instance with (all its) data fetched from the service. Perhaps not pixbuf image data if in this case the card contains image data. Perhaps you want to lazy load that data, in the real, too. A real can be really, really large.

Nonetheless, there’s something I don’t like about the architecture of Evolution Data Server.

What I don’t like about it, is that it more or less requires each client of the service to keep a local copy of the calendar or contact data in its own memory. At least all the data that matched the search query. This data is not really small per item. In fact, it’s quite large. It never really surprised me a lot that both the calendar component of Evolution’s shell AND the Evolution Data Server consume quite a lot of memory.

This is a little bit silly if you think about it. You also give the reason why yourself, Ross: it’s a local daemon. Does it really make sense to have a copy of the item in each client if the daemon that already contains a copy is guaranteed to be local and therefore really quickly accessible?

Does it really make a lot of sense if you have a mobile device with a lot of applications that want to tightly integrate with calendaring and contact information? Your messenger, your E-mail client, your TODO-list app, your desktop searcher, your menu probably too will all be duplicating this data in their local address space.

What I would want is to have a cursor-like API (like a database cursor). We all know that certain people have been warning us about the extraordinary insanely unimaginably huge roundtrip cost of D-BUS and that therefore this is absolutely definitely not a good idea. My opinion is that this black and white point of view on that is quite … wrong.

I remember you being enthusiastic about Polymer, so I’ll use your favourite E-mail client as an example:

Mulberry and Polymer use that incredibly complicated IPC over for example GPRS. GPRS, a network with roundtrips measured in whole seconds. Let me illustrate: sending the query and waiting for its answer sometimes takes a longer time than you standing up from your chair, running around your chair quickly and going back to sitting.

Now that’s a real roundtrip!

Nonetheless are E-mail clients like Mulberry and Polymer using a concept that is similar to a db-cursor. The reason why they don’t suffer from a lot of roundtrip problems, is because they made sure that their queries are returning a carefully measured amount of items. On top of that will Polymer pipeline the queries (not sure about Mulberry’s pipelining).

Nonetheless is the concept very similar if not equal to cursor based data access: You get a finger that points to your current location (I usually call that, my iterator). Around that pointer you fetch the things that you need “right now”. For example the things that are visible, indeed. Each time you scroll you do a few iterator.Next(), iterator.Next(), iterator.Next() and you render what you just got. With GtkTreeModel you can actually do this. If you have a local daemon and definitely if on top of that you fetch a small amount of items in a group, per each roundtrip, and do some micro caching, this will be fast enough. Just make a proxy and let your proxy instances keep the real for a few seconds, before disposing it. Let the tree model hint the proxies in the unref_node about disposing the real.

Net result: only the visible ones and the cached ones will be in the memory of the client.

This indeed means doing the sorting at the service. Therefore keeping a session at the service, too. You can actually integrate this cursor-over-IPC with a cursor on the SQLLite table, if you’d write a database backend. Your database engine just needs to support multiple cursors.

Net result: only the visible ones and the cached ones will be in the memory of the client, and the service will be using a cursor too. Which has a memory usage similar to mmap (only data that you are using right now, is actually loaded). Usually are these database engines really good at making sure that you still get the data really fast.

ps. It’s less important to update the clients in case of events, if at the service the cursor is updated. The next scroll event will do new “Next(), Next(), next()”‘s, and those will be updated already. You can of course still provide a “I’ll call you” mechanism to make the view update itself even faster.

Woah, this is probably going to unleash some sort of huge flamewar. Let’s try to keep it polite if you make replies in the comments, okay?

Wed 2008/Mar/12

  • I’m surrounded by beautiful hackers these days.








Hmm, yeah, nice try. Maybe I’m idd nonetheless a little bit jealous on Federico.

The case against webmail

Yesterday I posted how to make a Web 2.0 E-mail client. Although a joke, some people kinda agreed that this is the future.

Let’s consider a few use cases and let me try to make a point about this:

When OpenMoko asked me about an E-mail client, they told me it would be nice if they could show the history of activity of a person when you clicked on a person in the contact application. This included phonecalls, chat sessions, text messages and therefore also E-mail.

If everything was Google, this would mean that the entire contact management would have to be done by Google. All the data the phone needs would have to be managed by Google. All your phonenumbers, all your calls, all of your history, every every everything would have to suddenly be managed by Google. Not by your personal phone, but by Google. The one big, the one, the gigantic, Google.

I’m sure Google’s employees who have stock options are now seeing little dollar signs in their eyes. Me personally, I don’t really like the idea.

The other option is using Google’s online APIs. That could work, but then suddenly another user of my software would want to use yahoo mail, an yet another would want to use Hotmail. And another would want to use Thismail and another Thatmail. As an application developer, I would have to learn to use all the existing online APIs, and worse, all future ones too. On top of that would these E-mail monopolies get quite a large portion of your privacy too (they can easily log all the activity that you create on their server while using your phone).

Let’s try another example. Back when I was working as a subcontractor for technology companies, I remember not having the permission to and a firewall blocking me from using my E-mails from outside of the walls of the building of the company for whom I was doing contract working. And to be honest, that makes a lot of sense. I remember one of the companies forbidding the use of MSN because all the traffic was centralized at Microsoft’s servers. At first all Instant Messaging was forbidden, then they learned about XMPP (the point being: decentralization and guaranteed encryption).

You think these companies would allow me to receive their E-mails on Hotmail? On GMail? On Yahoo mail? Please keep in mind that often they are directly or indirectly competing with Microsoft, Google or Yahoo.

While everybody congratulated Jabber for decentralizing Instant Messaging, now a lot of people are telling E-mail client developers that the centralization and the monopolization of E-mail by Hotmail, Yahoo mail and GMail is a good thing.

That kinda doesn’t make a lot of sense. Does it? Anyway, I don’t think that their black and white point of view is realistic. GMail is a good service, but let us make them compete. If we don’t, nothing guarantees us that Google will keep playing nice.

I’m not one of those religious conspiracists but we’re talking here about possible privacy abuse and security concerns. These are real concerns that companies have and serious barriers to the migration of E-mail to one entity. They are among the reasons why E-mail will be decentralized and why GMail wont ever be the single solution for E-mail. No matter how many Web 2.0 enthusiasts start blogging.

It’s simply not black and white.

Web 2.0 !!!

A few days ago I got this reply on one of my blog posts:

Phil:

Your post — and your work — miss the point entirely. Nobody cares how email works, they just want it to work.

Gmail (and most other webmail applications) makes everything else obsolete. I can’t imagine why Evolution is even shipped with Gnome anymore.

Web-based email clients are the standard.

-Anon

I just finished the E-mail client the guy wants. Here it is!

using GLib;
using Gtk;
using WebKit;

class Web20EmailClient : Window {
	WebView view;
	construct {
		view = new WebView ();
		view.open ("http://gmail.com");
		view.set_size_request (640, 480);
		add (view);
	}
	static void main (string[] args) {
		Gtk.init (ref args);
		var win = new Web20EmailClient ();
		win.show_all ();
		Gtk.main ();
	}
}

Ideas? Plenty of ideas!

Introduction

Unlike what some strange people think wont E-mail disappear anytime soon. I also believe that more and more of the use cases of the desktop computer will migrate to specialized devices. Examples are music, navigation, simple messaging, movies, contact and meeting/agenda management. With devices like the Blackberry we are seeing a market for E-mail too.

I want to create the next generation of E-mail clients for mobile devices. Perhaps even do more with the technology than what people nowadays imagine E-mail can do.

CONVERT

IMAP will soon be one of the few content services that will make it possible to do conversions at the server. A mobile device often can’t ship with support for many document formats. Often are certain document formats inadequate and too large for the popular wireless data networks being used in mobile. Just to make a document easily editable can a format needlessly grow in size. Often is this capability not required on the mobile. Another problem is too much data quality. A device that has poor audio capabilities does not need a CD quality WAV file. A device that has a screen of only a few hundred pixels, does not need to download images that got attached to an E-mail as-is when they where taken from the digital camera. If just for the purpose of displaying on the device, they can be made a lot smaller in size.

Selective summary downloading

Not only CONVERT but also techniques that have been possible for ages, can be applied with IMAP. Mulberry, Pine and especially Polymer have done interesting experiments with existing IMAP capabilities like pipelining, envelope downloading in a specific way and downloading of individual MIME parts of a message as they are needed. Two of those three techniques are not surprisingly missing in my own project, Tinymail.

Tinymail will download the entire folder’s summary because it wants to serve as infrastructure to write E-mail clients that can work offline in a convenient way. This doesn’t mean that the envelope downloading technique being used by Pine, Mulberry and Polymer is not suitable for offline E-mail clients. I have, however, never found a good E-mail client that combines this technique with a fully offline available summary of your folder. I know Dave Cridland is interested in experimenting with Polymer to make it become an offline E-mail client too.

What is this technique about? In stead of doing a simple query that requests all summary items top to bottom, you start with requesting the items that are near the ones the user wants to view as soon as possible (the visible ones). After that you continue in the background with receiving less important items. This works surprisingly good for E-mail clients that require being online. You more or less integrate it with your summary viewing component’s scrollbar and you make the magic do its work. To avoid roundtrips you try to group items together in your queries (to end up having to send less queries). The better implementation will on top of that pipeline queries and responses.

For an offline E-mail client you need to store that, of course. Once stored you’ll also need to pick a strategy for deciding what the next most important group of items will be. It’s for example likely that you already have the group locally cached. Storing them is less trivial than it sounds. Especially if you want to use the items efficiently in your memory. For example by mmap()-ing the file that stores the cache, or by using one of those embedded database engines (which makes this a lot more simple, of course).

It’s likely that the user will be scrolling up and down. It’s equally likely that the user wont make huge leaps in his scrolling behaviour. Therefore you can make the assumption that you want to start growing the group of locally available items around the spot where the user is currently viewing things. Somehow, store it in that retrieval order.

Sorting, threading

Meanwhile you want to ask for a SORT and a THREAD which will assist your E-mail client with sorting and threading in a much faster way than it could do this by itself (remember that we are on a mobile with limited memory bandwidth and CPU capacity to reduce battery consumption). You also want to request all the possible sorts that the user can request, and store the results offline.

Selective MIME part downloading

It’s quite common that the user is not interested in the entire message. A good example is a message with five image attachments. The user will often pick just one of the images. With IMAP’s BODYSTRUCTURE and FETCH program you can selectively download the exact MIME part that the user is requesting, in stead of the entire message. For that reason you want to fetch BODYSTRUCTURE together with ENVELOPE when dealing with summary retrievals. That way your summary will also serve as the skeletons for your messages: you can prepare your user interface components before even having to download the actual content of the messages. Also important is that you can do this without having to ‘store’ the content of the messages: storage capacity is rather limited on a mobile compared to the typical E-mail account of a lot of people. Not having to store and not having to download the attachments, is kinda nice. Don’t you think?

The problem, current state of affairs

There’s not really a problem. A lot of the ideas that I had in mind last year are now implemented in Tinymail. Examples are support for CONDSTORE, QRESYNC, IDLE, Selective MIME downloading (if you enable it) and BINARY (if you enabled selective MIME downloading).

One problem is funding. The ideas that I have in mind imply developing on these ideas full time for at least another two years. Another problem is the experience level in E-mail client development: it’s no secret that all competent E-mail client developers seem to have the tendency of running away from E-mail client development. Often they join forces with IMAP server developers (hi there Dave), or they no longer like E-mail client development (hi Jeffrey!). Leaving the scene with inexperienced developers, like me. It seems those people get more experienced and end up doing a new project, they burn out or they get hired by server developers. It looks like they don’t often focus their experience on a exciting new kind of E-mail client, based on their in-depth experience with IMAP.

I finally kinda understand IMAP, I think (but I’m not sure). I of course think that if I’d start an E-mail client infrastructure now, it would be excellent stuff. But I also have to survive, so I found myself a contract that is mostly unrelated to E-mail client development. Starting April will Tinymail be my non-professional pet project. Which means that the speed of development will probably drastically go down.

Being mostly a technical guy, I don’t know how to market these ideas in such a way that somebody wants to pay me for a few years to make it happen. Unlike what most people want to believe, would it indeed take a few years to do all this right.

A few months of Google-IMAP, let’s evaluate!

IMAP’s biggest problem is poor server implementations and big vendors who are not really supportive towards each other being the ones defining it. Not because they are the only ones at IETF who are vocal about the protocol. It’s because if the big players don’t implement the new enhancements, it basically boils down to not being used and the E-mail clients not adopting it. Quite funny is the fact that all those big players have their own closed protocols for E-mail too. That all those big players seem to consider their IMAP support to be of secondary priority.

If a new feature would make IMAP really a lot better than the closed solutions, then it does happen that the big vendors will especially not implement it. I suspect that’s because that would pose a direct risk for their own offerings.

Take a look at GMail’s CAPABILITY line if you think that Google is the God of good, the great and the brave. Their IMAP server is probably one of the poorest imaginable. You would think that Google would want to make a name for itself by introducing a really good IMAP server? Given that they hired thousands really excellent software developers and even at least one of the people who have been really closely involved in the creation of the IMAP standard, you would expect that. No?

Well, no. For Google it’s only the words “IMAP” and “support” that really matter. Just to get the sound of those two words in the minds of the people, and then that’s it. Nonetheless for IMAP’s name-reputation it’s actually good that Google did this. Google’s IMAP server doesn’t really show what a modern IMAP server would be capable of.

To show this, it lacks capabilities like COMPRESS and/or do compression over TLS. It lacks capabilities like CONDSTORE, QRESYNC and BINARY. It will probably not adopt anything the Lemonade group is doing at this moment, like CONVERT and NOTIFY. It also doesn’t do TLS correctly: it only supports wrapped mode for SSL. Making it hard for E-mail client developers to standardize on data encryption. And what about THREAD, SORT, UIDPLUS?

Probably because otherwise people would make fun of Google’s IMAP team, it looks like they did make it support IDLE.

Not very surprisingly are they offering mobile E-mail solutions that use their secret closed protocol. So much for Google being the most cool company humans ever created. In a way, they are doing exactly the same as Microsoft once did with embrace and extend. But in a more subtile way. In such a way that people keep believing: but hey, Google is cool! Summer of code! yeej! They do IMAP! And XMPP! Open formats! Open protocols!

Well, Microsoft does IMAP too. Their IMAP support in Exchange is actually better than Google’s IMAP server. It’s hard to be worse than Microsoft’s support for open protocols, but to get away with it for free? Just like any other IMAP server implementer has Microsoft got some of their things wrong. They usually actually do fix this. We’ll see how Google will play their IMAP game. I’m for sure going to be critical, no matter what.

Foster parents for Tinymail

I have found myself foster parents for Tinymail.

One of the difficult jobs of a maintainer is delegating tasks to the right people. In my opinion a good project leader is someone who motivates people by giving out responsibilities.

This is why Igalia is doing Tinymail’s 0.0.8 pre-release this time.

Microsoft goes open?!

Microsoft promises to expand interoperability: the register, Microsoft’s PR site.

Don’t ask me, I’m still finding out what this is all about.

Making it look nice

Wouldn’t it be nice if the code for your E-mail client would look like this?

private void GetHeadersCallback (Tny.Folder folder, bool cancel, Tny.List model, IntPtr user_data)
{
  if (!cancel)
    this.headers_treeview.Model = (Gtk.TreeModel) model;
}

private void StatusCallback (GLib.Object sender, Tny.Status status)
{
  this.progressbar.Fraction = status.Fraction;
}

private void GetMsgCallBack (Tny.Folder folder, bool cancel, Tny.Msg msg, IntPtr user_data)
{
  if (msg != null && !cancel)
    this.msg_view.Msg = msg;
}

private void OnMailSelected (object o, EventArgs args)
{
  Tny.Ui.GTK.HeaderListModel model = (o as Gtk.TreeSelection).TreeView.Model
          as Tny.Ui.GTK.HeaderListModel;
  Tny.Header header = model.GetHeader (o as Gtk.TreeSelection);
  if (header != null) {
    Console.WriteLine ("Message selected: " + header.From);
    this.cur_folder.GetMsgAsync (header, GetMsgCallBack, StatusCallback);
  }
}

private void OnFolderChanged (object o, EventArgs args)
{
  Tny.Ui.GTK.FolderStoreTreeModel model = (o as Gtk.TreeSelection).TreeView.Model
          as Tny.Ui.GTK.FolderStoreTreeModel;
  Tny.Folder folder = model.GetFolder (o as Gtk.TreeSelection);
  if (folder != null) {
    Tny.Ui.GTK.HeaderListModel headers_model = new Tny.Ui.GTK.HeaderListModel();
    Console.WriteLine ("Folder selected: " + folder.Name);
    this.cur_folder = folder;
    folder.GetHeadersAsync (headers_model, true, GetHeadersCallback, StatusCallback);
  }
}

It would

The mandatory screenshot

I just finished making Tinymail‘s .NET bindings actually usable. To test whether the binding works I wrote a small demo user interface in C#.

I added minimal documentation too.

Because Tinymail does quite a bit more with the GTypeInterface GType than Gtk+, Mike’s GtkSharp code generator for this had a few problems. I expected it to fail a lot worse, though. Without any improvements it did already get pretty far by itself! Mike and me have been pushing its code to support even the most crazy kind of things that Tinymail is doing with interfaces in GObject-C. There is still a lot of room for improvement for both Tinymail’s resulting binding itself and GtkSharp‘s Gapi parser and code generator.

With current trunk of GtkSharp and after applying two patches for its generator, it’ll make you a quite nice .NET API now. I’m of course working with Mike on getting proper solutions for what I had to patch upstream in GtkSharp’s Gapi.

I would like to thank Mike for his patience and help while I was making this binding work. I hope other library maintainers will use Tinymail’s one as knowhow-offset to get their own bindings working in .NET too.

TnySharp

There are still a few rough edges, but I’m confident that I’ll soon have those fixed. Other than that, here it is: the first in C# programmed E-mail client that uses Tinymail.

Took me around two minutes to throw the UI together and connect a few signal handlers to the treeviews with MonoDevelop.

A few interesting things that happened today

I decided to just do it and throw together .NET bindings for Tinymail. They are nearly finished and looking great. I might never have told anybody, but a really sweet looking E-mail API for my favorite programming language C# was my original reason when I started Tinymail.

While I was doing Tinymail I of course learned about D, so now C# as a programming language might have to compete with D. Meanwhile Jürg did Vala too. To make sure OLPC would have a reason to take a look at Tinymail I fiddled around with Python bindings too. Mark Doffman made these bindings actually usable. Meanwhile Jürg made the Vala bindings for Tinymail. Just for fun.

That brings us at three bindings for Tinymail: .NET, Python and Vala. I’m hoping to add C++ to that list soon. I’m planning to do this in a way that makes the library sensible to use for Qt developers. This of course implies dealing with mainloop integration and providing a few standard models and views to get a typical Qt E-mail client developer quickly going. I’m not a big fan of C++ but I was pragmatic about GObject and C when I started Tinymail too.

I have some sort of goal in my mind and if I have to do twice as much work to get it working without my favorite tools, then well, that’s what it takes. Politics and religion just makes it more difficult to reach your goals, so I’m not very interested in .NET fans, Python fans, C++ fans, Qt fans and GNOME fans cheerleading because their favorite tech is being used (which is not the point of technology and innovation anyway). I do care about the technical implications of depending on something like a virtual machine, though. Especially since Tinymail has a focus on mobile.

Mono has an ARM port, so it’s usually not really a technical limitation. On top of that I found out about GC.AddMemoryPressure in .NET. Tinymail holds significant large resources that your garbage collector doesn’t know about. With GC.AddMemoryPressure you can easily teach the .NET garbage collector about these non-managed resources.

Something like GC.AddMemoryPressure is really a missing feature in Python in my opinion. When your E-mail client uses mostly the managed higher programming language just for invoking functionality implemented in C, not a lot of Python executions will occur during runtime. This means that Python’s garbage collector is only rarely called to do a collect. Unless you do gc.collect() at an intelligent location, your Python E-mail client will keep instances around for a very long time. In .NET, with GC.AddMemoryPressure, you can tell the virtual machine’s garbage collector that an external resource is significant enough to care about it more. It’s not as drastic as doing a GC.Collect() in .NET. GC.Collect is not recommended by Microsoft, since the .NET garbage collector is supposed to learn from the past to try to predict the future. If you GC.Collect(), you circumvent this. Using GC.AddMemoryPressure you don’t really do anything bad for the garbage collector. You just inform it about the significance of an external resource.

Regretfully, and I already asked Miguel, doesn’t the Mono runtime care about GC.AddMemoryPressure. But the API is available for when Mono’s garbage collector will be adapted to do care about it. Microsoft’s virtual machine does care, by the way.

Nice would be porting GObject to WinCE and then bringing Tinymail to that platform. Making a Tinymail based E-mail client with either Compact Framework .NET or Mono sounds like a fun project.

Meanwhile, Modest is doing great too. I’m still waiting a little bit with Tinymail’s first release. I would like to give the Modest team as gift when they release their first, Tinymail’s 1.0 release. I think we are nearing that moment.

Finally an opportunity!

So, if I get this right … Phones will use Qt, tablets will stay Maemo. That’s finally an opportunity for us to work together with the Qt people! I’m looking forward to a future where Nokia will fund all this jing jang. Great!

I guess I’ll soon be preparing Tinymail to be very usable for Qt developers too. Except for a few isolated places where it now requires GMainLoop integration, which I guess on devices will be shared for both Qt and Glib anyway, this should be possible already. The API just needs some C++ wrapping and perhaps an implementation of libtinymailui that uses Qt’s components.

I also started putting .NET bindings in place for the API. These bindings ain’t finished yet. Mike’s Gapi2 GTypeInterface binding generating code works very nice. Just a few compilation errors that I need to fix now (check gtk-sharp’s mailing list).

I sensed some enthusiasm in Rob’s post about all this Nokia and Trolltech stuff. My guess is that Rob is trying to tell us: “jeej! stuff is moving! Let’s go for it!”, but I’m not sure of course. But if that’s what you mean Rob, let’s indeed go for it and make the things in technology move!

All your privacy are belong to me!

I’ve been using Google analytics for Tinymail.org for a few months now.

I was mostly interested in results per city. As expected is Helsinki scoring high. Since a lot of Modest’s developers live in Spain there are a few cities with a lot of visits in Spain too. Now I know where you guys live!

Nothing surprising. Except maybe the visitors from the Indian cities Hyderabad and Bangalore. I wonder what Indian company is working on a mobile E-mail client? The visitors from South America are interesting too! Are you guys working on one for OLPC?

I also have a lot of Brooklyn and Tempe visitors. That’s Red Hat, right?

What Nokia division can we find in Oulu by the way? And Sydney, is that jdub visiting?

Cute and I guess typical are all European cities. All major cities in Europe had a lot of visitors. Just never really a lot, unless they are located in Finland and are called either Helsinki or Oulu.

With one single exception for Europe: a city in my own country, Heist-Op-Den-Berg. So, who’s that Tinymail fan in Heist-Op-Den-Berg? Let’s get a drink somewhere? What about FOSDEM this year? It was not me, my own home city scored like all other European cities.

Disappointing is Russia. The visits for all of Russia compares to one European city, all Russian visitors came from either Moscow, Tula or Lisichansk. In Russia E-mail libraries code you?

I had three visits from Honolulu!

What is strange is that Google analytic’s analysis of amount of visitors doesn’t really match my actual Apache logs if I manually count them. Something like 60% less unique visits on Google analytics. I wonder at what point will Google analytics start grouping the hits of a user as an actual visit?

Shaken or stirred?

I just released Tinymail’s pre-release 0.0.7. This release is mostly a bugfix release. We’ve been fixing memory problems and a few crashers in features that we introduced in pre 0.0.6. And because I have been telling people this about every pre release, this one too is probably one of the last pre releases before a final release!

The release notes can be found here. Downloads here. API documentation here. And the friendly guys doing Modest have made a beta release based on Tinymail’s trunk of today (which is the same version of the code as this pre-release 0.0.7). Install Modest on your N800 and N810 by tapping here.