Category: meego

Quim Gil announces why.

It’s a little bit frightening … I guess quite a lot of people will now start using our code.

I’m also proud of having been involved with such a great team of software developers working on Modest and Tinymail.

I think I succeeded at creating a healthy free software project suitable for both commercial and purely free software appliances. People are contributing for and from different angles. Different companies are involved. I am trying to blend the contributions a bit more in order to get more “passion contributions” rather than “just the minimum must work, here’s a quick-fix”-patches. It’s improving, though.

Nokia (however) gave me ‘the’ opportunity to create Tinymail. They allowed me to work on it full time for more than a year. Everything was always immediately free software, and happened completely in the open too. Go check if you don’t believe that.

You can accuse Nokia of a lot of things, but they did great at fostering several free software projects. They made mistakes, of course. In my opinion, the handling of this project was done in a very healthy way: very open minded and very much the way free software can be commercially successful. I hope they learned free software won’t do wonders but that it can help you finding experts in specific fields, working together on the bigger picture: the actual consumer device. Working together with free software inspires a certain kind of passion among engineers. Capitalizing on that is fine, as long as you have respect for the passion. Just like the engineers have respect for the passion of solving the problem of producing and bringing millions of devices to the customers in time. Both enemy and friend know no company can do this better than Nokia.

I don’t think there’s anything controversial in respecting the hardware dudes’s jobs. In the end we humans are all about our passions.

I didn’t succeed at making a release before Modest was thrown to the Maemo users. That should happen very soon now, though. I’m very much in agreement with that “release often, release fast” – meme. But I also only want to call something a one-point-zero if it’s really stuff that works, behaves as expected and has all API that people’s E-mail clients require. The problem is that as an engineer I’ll never think that any of my software is finished. Everything can be better. Living near Eindhoven I grew up with the slogan “Let’s make things better“. I agree with the meme in that slogan.

Neither did I reach a bunch of technical goals. I have so many things in mind that would make Tinymail based E-mail clients by far the best kind of mobile E-mail clients ever created. Although Polymer, damn you Dave Cridland! :-), makes it hard to really be the best. Subjects ranging from pipelining to supporting CONVERT, forward without download, streaming media as attachments, and many more. IMAP has a few interesting limitations that I would like to address too, like Lemonade does. And then you have the IMAP servers … grmbl.

Regretfully a lot of people think of E-mail as a simple subject. They should figure out then why so many teams have tried to write good E-mail clients, yet relatively few succeeded. I’ll spare you the details and I’ll just point out that a lot of ’stuff’ is involved that you won’t expect. But really, if it wasn’t a challenge, I wouldn’t have been interested anyway. I’m not complaining, I enjoyed and I am enjoying a hard task ahead of me.

Meanwhile, I enjoy working on Tracker for Nokia. It was time for something different. Incubation time for Tinymail, perhaps? I’m working a lot with SQLite, who knows what will come out of that symbiosis? Perhaps I learn that it’s not suitable? Perhaps it is? I haven’t had the chance to explore cursors in SQLite. That’s what I’d be interested in most for things like Tinymail.

I should stop thinking about it too much, before I end up hacking until eight in the morning again.

I have been whining about features that I want in Vala to Jürg. To make up for all the time he lost listening to me I decided to fix two Vala bugs.

The first bug I fixed was using a recursive mutex for lock statements. Code like this will work as expected now:

public class LockMe : GLib.Object { }
public class Executer : GLib.Object {
	LockMe o { get; set; }
	construct { o = new LockMe (); }
	void Internal () {
		lock (o) { }
	}
	public void Method () {
		lock (o) { Internal (); }
	}
}
public class App : GLib.Object {
	static void main (string[] args) {
		Executer e = new Executer ();
		e.Method ();
	}
}

Here’s a gdb session that most GLib programmers will recognize:

Breakpoint 1, 0x08048a87 in executer_Method ()
(gdb) break g_static_rec_mutex_lock
Breakpoint 2 at 0xb7e4d0e6
(gdb) cont
Continuing.
Breakpoint 2, 0xb7e4d0e6 in g_static_rec_mutex_lock ()
   from /usr/lib/libglib-2.0.so.0
(gdb) bt
#0  0xb7e4d0e6 in g_static_rec_mutex_lock () from /usr/lib/libglib-2.0.so.0
#1  0x08048b04 in executer_Method ()
#2  0x08049046 in app_main ()
#3  0x0804908a in main ()
(gdb) cont
Continuing.
Breakpoint 2, 0xb7e4d0e6 in g_static_rec_mutex_lock ()
   from /usr/lib/libglib-2.0.so.0
(gdb) bt
#0  0xb7e4d0e6 in g_static_rec_mutex_lock () from /usr/lib/libglib-2.0.so.0
#1  0x08048a6e in executer_Internal ()
#2  0x08048b0f in executer_Method ()
#3  0x08049046 in app_main ()
#4  0x0804908a in main ()
(gdb) cont
Continuing.
Program exited normally.
(gdb)

The second bug is supporting interfaces for D-Bus services in Vala. It goes like this:

using GLib;
[DBus (name = "org.gnome.TestServer")]
public interface TestServerAPI {
	public abstract int64 ping (string msg);
}
public class TestServer : Object, TestServerAPI {
	int64 counter;
	public int64 ping (string msg) {
		message (msg);
		return counter++;
	}
}
void main () {
	MainLoop loop = new MainLoop (null, false);
	try {
		var conn = DBus.Bus.get (DBus.BusType.SESSION);
		dynamic DBus.Object bus = conn.get_object (
			"org.freedesktop.DBus", "/org/freedesktop/DBus",
			"org.freedesktop.DBus");
		uint request_name_result = bus.RequestName ("org.gnome.TestService", 0);
		if (request_name_result == DBus.RequestNameReply.PRIMARY_OWNER) {
			// start server
			var server = new TestServer ();
			conn.register_object ("/org/gnome/test", server);
			loop.run ();
		} else {  // client
			dynamic DBus.Object test_server_object =
				conn.get_object ("org.gnome.TestService",
					"/org/gnome/test", "org.gnome.TestServer");
			int64 pong = test_server_object.ping ("Hello from Vala");
			message (pong.to_string ());
		}
        } catch (Error foo) { }
}

In Vala you can define interfaces just like in C# and Java. Interfaces imply that you can have class types that implement one or more such interfaces. Vala does not force you to implement its interfaces in Vala. You can also implement them in good-old GObject C.

Here’s a detailed example how you implement a type that implements two Vala interfaces in GObject/C:

Your application used to be single threaded and is consuming a resource that is not thread-safe. You’re splitting your application up into two or more threads. Both threads want to consume the non-thread-safe resource.

In this GNOME-Live item I explain how to use GThreadPool for this.

It’s a wiki so if you find any discrepancies in the sample and or text, just correct them. I’m subscribed so I’ll review it that way.

The GNOME-Live item is done in a similar way to the item about using asynchronous DBus bindings and the AsyncWorker item.

Summer of code 2008

I’m going to mentor three Summer of Code applications.

• E-mail activity for OLPC by Shikhar Bhushan
• Seahorse integration in Evolution by Zhang Shunchang
• SuperRandom, a predictive listening plugin for Rhythmbox by Charlotte Curtis

One of the Igalians who developed Modest and among many of his Tinymail contributions implemented the libcst implementation for handling certificates in Tinymail, José Dapena Paz, is going to mentor Zhang Shunchang’s application together with me.

Picking up what I left in 2005

I also just picked up AsyncWorker. I made it a little page and with the help of Tinne I improved its API documentation. Perhaps I will do a release someday (I never did, actually). Thing is that I hate the work involved with releasing. Especially since most of our development tools stink.

Note that my super fantastic lovely girlfriend, Tinne, has her own blog now. It contains a bunch of photos of our stay in Durham UK. In a few minutes, she just told me, she will put online a funny photo of me holding a fish bowl filled with cocktail.

While I was gathering some info about a DBus related task that I’m doing at this moment, I wrote down whatever I found about DBus’s glib bindings in tutorial format.

A few other people have done similar things in their blogs. This one explains how to use org.freedesktop.DBus.GLib.Async a little bit too.

If you find any mistakes in the document, it’s a wiki page so please just correct them.

I just released Tinymail’s pre 0.0.9, enjoy!

I noticed that more and more people from several specific cities are visiting Tinymail pages, and I know at least a few companies and organisations who are using Tinymail right now.

My personal opinion on development frameworks is that if they come without documentation, they are worth as much as vaporware.That’s why I started first, at an early stage, with writing the API documentation of Tinymail and then Tinymail’s trac, which holds a collection of examples and on top of the API documentation also explains most of its types and how to use them.

This was not sufficient. I wanted to write a test E-mail client to find the source of bugs in Modest. While I was doing that I decided that this test E-mail client was going to be documentation too. Documentation in the form of source code that itself required documentation.

I started TMut’s trac to deposit that documentation. Yesterday I mentioned that I implemented simple account management in TMut, today this is ~ finished. Here is the documentation about that source code.

New items:

• TMut’s account store
• Account manager
• Account editing

Former items:

• Creating messages
• Select a folder window
• TMut’s menu, its first screen
• Showing the summary of a folder
• The message view component

You guys remember TMut? It’s an E-mail client for small screens that uses Tinymail. It serves as an actual E-mail client, as some code that you can use to peek at while developing your E-mail client and as a piece of code where you can derive your stuff from (although TMut’s current build is not set up to build TMut’s classes into a library, you could easily do this and then subclass TMut’s high level components).

What makes TMut unusable for non-software developers is that it has no account management. You need to do that in for example GConf (depending on what implementation of TnyAccountStore your TMut uses).

At Modest we need to test Tinymail’s account management capabilities without executing all of the extra code involved in what Modest does whenever you manage its accounts. Therefore I started putting in place some code to have basic account management in TMut.

It’s, as usual with the things that I blog about, unfinished.

Iterators

Now that Murray has posted about the iterators idea I can no longer hide.

Together with Jürg, the Vala man and Murray, who before becoming a pregnant guy was and still is coding on things like Glom and gtkmm, I’ve been writing up a document. This document explains iterators like the ones you’ll find in .NET and Java.

This is related to the Iterators concept because the document about iterators could be part of a solution for this.

I once had to write a custom GtkTreeModel (I think I still suffer from vertigo) and therefore I sent treeview Kris an analysis about this.

There’s a rule that an interface should be a contract for solving exactly one problem.

Use interfaces to say “What objects can do” or “What can be done to an object”. You can let a class implement multiple interfaces. There’s absolutely no need to make huge interfaces. An interface should define that your class “Does One Thing”. Another interface defines that your class “Does Another Thing, too”, and a last interface defines that your class “Also Does This”. My class Z can do D, E and F. Therefore: class Z implements D, E and F.

Instead, the contract being GtkTreeModel requires you to solve five problems:

• being iterable: you can go to the next node
• being a container: you can add and remove things from it
• being observable: it notifies a component about its changes
• being a tree: sometimes items can parent other items
• having columns: it’s something that describes columns

Summarizing: interface TreeModel means BEING a D, E, F, G and H at the same time. Correct would be if you specified that TreeView requires a model that does D, does E and perhaps also does F. In case treeview is showing the things as a tree, it requires that the model also does E and does H.

GtkTreeModel should have been five different interfaces.

Shocking! But in my opinion, true.

The consequence is that having to implement a GtkTreeModel has become a difficult task.

“You have to implement solutions for all five problems anyway!”, you might think. That’s true. However! If there were five interfaces, other people would have made default solutions for four other interfaces. Meanwhile “you” focus on “your” specialization. For example Rhythmbox and Banshee are specialized in huge models that contain a specific set of data that must be searchable. Preferably search and sort are done by a database instead of a model-filter.

Banshee and Rhythmbox developers:

• … don’t have to care about the Columns problem
• … don’t have to care at all about the Tree problem
• … don’t have to care about the Container problem: they never need to add nor remove things from it because they set up their model instantly by passing a query to a data source. Maybe the feature “remove from playlist” requires solving the Container problem?
• … unless they support inotify on the MP3 folder of the disk, they don’t even have to care about the observer problem

Yet with GtkTreeModel you have put the burden on application developers to solve all five the problems in their model. No wonder Banshee developers decide to code their own TreeView in .NET!

Federico, I’m not going to make yet another long and technical blog post about this, because it’s rather obvious how it works. But you can make a db-cursor or a cursor-like API for a query that is solved at the service and kept in a session.

My opinion is also that 484 bytes actually is a lot of memory per item. Wrap that in a proxy instance and lazy load the proxy, and you reduce that to an actual average of 50 bytes per item. It wouldn’t even matter if when parsed the size exploded. Only the items that are needed right now would be exploded in memory, in stead of all items (which is now the case). You could even cache the strings that are needed to render an overview of the items in the proxy, and avoid creating the real this way. This would give you a solution that could scale to 100,000 items (although that’s probably stupid to render in a view, sure).

Note that for this proxy concept, you don’t even need a cursor-like API. Just an API to fetch one real item, and a way to fetch just the minimal info for your proxies, so that they know how to fetch their real.

About the structures that tend to explode in size when you parse them: is the library using GStringChunk to store all the tiny little strings? Else I kinda know already where all that heap-admin is going to. It’s a lot, if you measure it. That’s because all of them are stored in parsed structures, right? Each mini string needs four bytes for the pointer, at least four bytes for heap admin, etc. I can also imagine that this data contains a lot of duplicate strings, perhaps take a look at how camel_pstring_add works? If GStringChunk is sufficient, then go with that, of course.

Or, even more simple:

If you’d replace all \n characters with \0 characters in the original VCard string, you could also just make them point to an offset in the original buffer. Note sure about line wrapping and whether the format could support this easily (not my field of expertise).

Also note that the prevailing model for E-mail might be that you display a usually huge linear list of messages, but that the market is increasingly demanding a search-based model. Especially for mobiles with small screens.

Clarification:

• A proxy here is an instance that has the same interface as the actual thing. To fulfill that interface it might have to get a real first, and then proxy the command to the real. A proxy usually only contains the things that are necessary to fetch a real from the service. It can also contain extra data (I call this a micro cache) so that it can avoid having to fetch the real for common strings. For example strings that are displayed in an overview screen. A proxy must be small.
• A real is an instance with (all its) data fetched from the service. Perhaps not pixbuf image data if in this case the card contains image data. Perhaps you want to lazy load that data, in the real, too. A real can be really, really large.

What I don’t like about it, is that it more or less requires each client of the service to keep a local copy of the calendar or contact data in its own memory. At least all the data that matched the search query. This data is not really small per item. In fact, it’s quite large. It never really surprised me a lot that both the calendar component of Evolution’s shell AND the Evolution Data Server consume quite a lot of memory.

This is a little bit silly if you think about it. You also give the reason why yourself, Ross: it’s a local daemon. Does it really make sense to have a copy of the item in each client if the daemon that already contains a copy is guaranteed to be local and therefore really quickly accessible?

Does it really make a lot of sense if you have a mobile device with a lot of applications that want to tightly integrate with calendaring and contact information? Your messenger, your E-mail client, your TODO-list app, your desktop searcher, your menu probably too will all be duplicating this data in their local address space.

What I would want is to have a cursor-like API (like a database cursor). We all know that certain people have been warning us about the extraordinary insanely unimaginably huge roundtrip cost of D-BUS and that therefore this is absolutely definitely not a good idea. My opinion is that this black and white point of view on that is quite … wrong.

I remember you being enthusiastic about Polymer, so I’ll use your favourite E-mail client as an example:

Mulberry and Polymer use that incredibly complicated IPC over for example GPRS. GPRS, a network with roundtrips measured in whole seconds. Let me illustrate: sending the query and waiting for its answer sometimes takes a longer time than you standing up from your chair, running around your chair quickly and going back to sitting.

Now that’s a real roundtrip!

Nonetheless are E-mail clients like Mulberry and Polymer using a concept that is similar to a db-cursor. The reason why they don’t suffer from a lot of roundtrip problems, is because they made sure that their queries are returning a carefully measured amount of items. On top of that will Polymer pipeline the queries (not sure about Mulberry’s pipelining).

Nonetheless is the concept very similar if not equal to cursor based data access: You get a finger that points to your current location (I usually call that, my iterator). Around that pointer you fetch the things that you need “right now”. For example the things that are visible, indeed. Each time you scroll you do a few iterator.Next(), iterator.Next(), iterator.Next() and you render what you just got. With GtkTreeModel you can actually do this. If you have a local daemon and definitely if on top of that you fetch a small amount of items in a group, per each roundtrip, and do some micro caching, this will be fast enough. Just make a proxy and let your proxy instances keep the real for a few seconds, before disposing it. Let the tree model hint the proxies in the unref_node about disposing the real.

Net result: only the visible ones and the cached ones will be in the memory of the client.

This indeed means doing the sorting at the service. Therefore keeping a session at the service, too. You can actually integrate this cursor-over-IPC with a cursor on the SQLLite table, if you’d write a database backend. Your database engine just needs to support multiple cursors.

Net result: only the visible ones and the cached ones will be in the memory of the client, and the service will be using a cursor too. Which has a memory usage similar to mmap (only data that you are using right now, is actually loaded). Usually are these database engines really good at making sure that you still get the data really fast.

ps. It’s less important to update the clients in case of events, if at the service the cursor is updated. The next scroll event will do new “Next(), Next(), next()”‘s, and those will be updated already. You can of course still provide a “I’ll call you” mechanism to make the view update itself even faster.

Woah, this is probably going to unleash some sort of huge flamewar. Let’s try to keep it polite if you make replies in the comments, okay?

Yesterday I posted how to make a Web 2.0 E-mail client. Although a joke, some people kinda agreed that this is the future.

Let’s consider a few use cases and let me try to make a point about this:

When OpenMoko asked me about an E-mail client, they told me it would be nice if they could show the history of activity of a person when you clicked on a person in the contact application. This included phonecalls, chat sessions, text messages and therefore also E-mail.

If everything was Google, this would mean that the entire contact management would have to be done by Google. All the data the phone needs would have to be managed by Google. All your phonenumbers, all your calls, all of your history, every every everything would have to suddenly be managed by Google. Not by your personal phone, but by Google. The one big, the one, the gigantic, Google.

I’m sure Google’s employees who have stock options are now seeing little dollar signs in their eyes. Me personally, I don’t really like the idea.

The other option is using Google’s online APIs. That could work, but then suddenly another user of my software would want to use yahoo mail, an yet another would want to use Hotmail. And another would want to use Thismail and another Thatmail. As an application developer, I would have to learn to use all the existing online APIs, and worse, all future ones too. On top of that would these E-mail monopolies get quite a large portion of your privacy too (they can easily log all the activity that you create on their server while using your phone).

Let’s try another example. Back when I was working as a subcontractor for technology companies, I remember not having the permission to and a firewall blocking me from using my E-mails from outside of the walls of the building of the company for whom I was doing contract working. And to be honest, that makes a lot of sense. I remember one of the companies forbidding the use of MSN because all the traffic was centralized at Microsoft’s servers. At first all Instant Messaging was forbidden, then they learned about XMPP (the point being: decentralization and guaranteed encryption).

You think these companies would allow me to receive their E-mails on Hotmail? On GMail? On Yahoo mail? Please keep in mind that often they are directly or indirectly competing with Microsoft, Google or Yahoo.

While everybody congratulated Jabber for decentralizing Instant Messaging, now a lot of people are telling E-mail client developers that the centralization and the monopolization of E-mail by Hotmail, Yahoo mail and GMail is a good thing.

That kinda doesn’t make a lot of sense. Does it? Anyway, I don’t think that their black and white point of view is realistic. GMail is a good service, but let us make them compete. If we don’t, nothing guarantees us that Google will keep playing nice.

I’m not one of those religious conspiracists but we’re talking here about possible privacy abuse and security concerns. These are real concerns that companies have and serious barriers to the migration of E-mail to one entity. They are among the reasons why E-mail will be decentralized and why GMail wont ever be the single solution for E-mail. No matter how many Web 2.0 enthusiasts start blogging.

It’s simply not black and white.

A few days ago I got this reply on one of my blog posts:

Phil:

Your post — and your work — miss the point entirely. Nobody cares how email works, they just want it to work.

Gmail (and most other webmail applications) makes everything else obsolete. I can’t imagine why Evolution is even shipped with Gnome anymore.

Web-based email clients are the standard.

-Anon

I just finished the E-mail client the guy wants. Here it is!

using GLib;
using Gtk;
using WebKit;

class Web20EmailClient : Window {
	WebView view;
	construct {
		view = new WebView ();
		view.open ("http://gmail.com");
		view.set_size_request (640, 480);
		add (view);
	}
	static void main (string[] args) {
		Gtk.init (ref args);
		var win = new Web20EmailClient ();
		win.show_all ();
		Gtk.main ();
	}
}

Introduction

Unlike what some strange people think wont E-mail disappear anytime soon. I also believe that more and more of the use cases of the desktop computer will migrate to specialized devices. Examples are music, navigation, simple messaging, movies, contact and meeting/agenda management. With devices like the Blackberry we are seeing a market for E-mail too.

I want to create the next generation of E-mail clients for mobile devices. Perhaps even do more with the technology than what people nowadays imagine E-mail can do.

CONVERT

IMAP will soon be one of the few content services that will make it possible to do conversions at the server. A mobile device often can’t ship with support for many document formats. Often are certain document formats inadequate and too large for the popular wireless data networks being used in mobile. Just to make a document easily editable can a format needlessly grow in size. Often is this capability not required on the mobile. Another problem is too much data quality. A device that has poor audio capabilities does not need a CD quality WAV file. A device that has a screen of only a few hundred pixels, does not need to download images that got attached to an E-mail as-is when they where taken from the digital camera. If just for the purpose of displaying on the device, they can be made a lot smaller in size.

Selective summary downloading

Not only CONVERT but also techniques that have been possible for ages, can be applied with IMAP. Mulberry, Pine and especially Polymer have done interesting experiments with existing IMAP capabilities like pipelining, envelope downloading in a specific way and downloading of individual MIME parts of a message as they are needed. Two of those three techniques are not surprisingly missing in my own project, Tinymail.

Tinymail will download the entire folder’s summary because it wants to serve as infrastructure to write E-mail clients that can work offline in a convenient way. This doesn’t mean that the envelope downloading technique being used by Pine, Mulberry and Polymer is not suitable for offline E-mail clients. I have, however, never found a good E-mail client that combines this technique with a fully offline available summary of your folder. I know Dave Cridland is interested in experimenting with Polymer to make it become an offline E-mail client too.

What is this technique about? In stead of doing a simple query that requests all summary items top to bottom, you start with requesting the items that are near the ones the user wants to view as soon as possible (the visible ones). After that you continue in the background with receiving less important items. This works surprisingly good for E-mail clients that require being online. You more or less integrate it with your summary viewing component’s scrollbar and you make the magic do its work. To avoid roundtrips you try to group items together in your queries (to end up having to send less queries). The better implementation will on top of that pipeline queries and responses.

For an offline E-mail client you need to store that, of course. Once stored you’ll also need to pick a strategy for deciding what the next most important group of items will be. It’s for example likely that you already have the group locally cached. Storing them is less trivial than it sounds. Especially if you want to use the items efficiently in your memory. For example by mmap()-ing the file that stores the cache, or by using one of those embedded database engines (which makes this a lot more simple, of course).

It’s likely that the user will be scrolling up and down. It’s equally likely that the user wont make huge leaps in his scrolling behaviour. Therefore you can make the assumption that you want to start growing the group of locally available items around the spot where the user is currently viewing things. Somehow, store it in that retrieval order.

Sorting, threading

Meanwhile you want to ask for a SORT and a THREAD which will assist your E-mail client with sorting and threading in a much faster way than it could do this by itself (remember that we are on a mobile with limited memory bandwidth and CPU capacity to reduce battery consumption). You also want to request all the possible sorts that the user can request, and store the results offline.

Selective MIME part downloading

It’s quite common that the user is not interested in the entire message. A good example is a message with five image attachments. The user will often pick just one of the images. With IMAP’s BODYSTRUCTURE and FETCH program you can selectively download the exact MIME part that the user is requesting, in stead of the entire message. For that reason you want to fetch BODYSTRUCTURE together with ENVELOPE when dealing with summary retrievals. That way your summary will also serve as the skeletons for your messages: you can prepare your user interface components before even having to download the actual content of the messages. Also important is that you can do this without having to ‘store’ the content of the messages: storage capacity is rather limited on a mobile compared to the typical E-mail account of a lot of people. Not having to store and not having to download the attachments, is kinda nice. Don’t you think?

The problem, current state of affairs

There’s not really a problem. A lot of the ideas that I had in mind last year are now implemented in Tinymail. Examples are support for CONDSTORE, QRESYNC, IDLE, Selective MIME downloading (if you enable it) and BINARY (if you enabled selective MIME downloading).

One problem is funding. The ideas that I have in mind imply developing on these ideas full time for at least another two years. Another problem is the experience level in E-mail client development: it’s no secret that all competent E-mail client developers seem to have the tendency of running away from E-mail client development. Often they join forces with IMAP server developers (hi there Dave), or they no longer like E-mail client development (hi Jeffrey!). Leaving the scene with inexperienced developers, like me. It seems those people get more experienced and end up doing a new project, they burn out or they get hired by server developers. It looks like they don’t often focus their experience on a exciting new kind of E-mail client, based on their in-depth experience with IMAP.

I finally kinda understand IMAP, I think (but I’m not sure). I of course think that if I’d start an E-mail client infrastructure now, it would be excellent stuff. But I also have to survive, so I found myself a contract that is mostly unrelated to E-mail client development. Starting April will Tinymail be my non-professional pet project. Which means that the speed of development will probably drastically go down.

Being mostly a technical guy, I don’t know how to market these ideas in such a way that somebody wants to pay me for a few years to make it happen. Unlike what most people want to believe, would it indeed take a few years to do all this right.

IMAP’s biggest problem is poor server implementations and big vendors who are not really supportive towards each other being the ones defining it. Not because they are the only ones at IETF who are vocal about the protocol. It’s because if the big players don’t implement the new enhancements, it basically boils down to not being used and the E-mail clients not adopting it. Quite funny is the fact that all those big players have their own closed protocols for E-mail too. That all those big players seem to consider their IMAP support to be of secondary priority.

If a new feature would make IMAP really a lot better than the closed solutions, then it does happen that the big vendors will especially not implement it. I suspect that’s because that would pose a direct risk for their own offerings.

Take a look at GMail’s CAPABILITY line if you think that Google is the God of good, the great and the brave. Their IMAP server is probably one of the poorest imaginable. You would think that Google would want to make a name for itself by introducing a really good IMAP server? Given that they hired thousands really excellent software developers and even at least one of the people who have been really closely involved in the creation of the IMAP standard, you would expect that. No?

Well, no. For Google it’s only the words “IMAP” and “support” that really matter. Just to get the sound of those two words in the minds of the people, and then that’s it. Nonetheless for IMAP’s name-reputation it’s actually good that Google did this. Google’s IMAP server doesn’t really show what a modern IMAP server would be capable of.

To show this, it lacks capabilities like COMPRESS and/or do compression over TLS. It lacks capabilities like CONDSTORE, QRESYNC and BINARY. It will probably not adopt anything the Lemonade group is doing at this moment, like CONVERT and NOTIFY. It also doesn’t do TLS correctly: it only supports wrapped mode for SSL. Making it hard for E-mail client developers to standardize on data encryption. And what about THREAD, SORT, UIDPLUS?

Probably because otherwise people would make fun of Google’s IMAP team, it looks like they did make it support IDLE.

Not very surprisingly are they offering mobile E-mail solutions that use their secret closed protocol. So much for Google being the most cool company humans ever created. In a way, they are doing exactly the same as Microsoft once did with embrace and extend. But in a more subtile way. In such a way that people keep believing: but hey, Google is cool! Summer of code! yeej! They do IMAP! And XMPP! Open formats! Open protocols!

Well, Microsoft does IMAP too. Their IMAP support in Exchange is actually better than Google’s IMAP server. It’s hard to be worse than Microsoft’s support for open protocols, but to get away with it for free? Just like any other IMAP server implementer has Microsoft got some of their things wrong. They usually actually do fix this. We’ll see how Google will play their IMAP game. I’m for sure going to be critical, no matter what.

I have found myself foster parents for Tinymail.

One of the difficult jobs of a maintainer is delegating tasks to the right people. In my opinion a good project leader is someone who motivates people by giving out responsibilities.

This is why Igalia is doing Tinymail’s 0.0.8 pre-release this time.

Wouldn’t it be nice if the code for your E-mail client would look like this?

private void GetHeadersCallback (Tny.Folder folder, bool cancel, Tny.List model, IntPtr user_data)
{
  if (!cancel)
    this.headers_treeview.Model = (Gtk.TreeModel) model;
}

private void StatusCallback (GLib.Object sender, Tny.Status status)
{
  this.progressbar.Fraction = status.Fraction;
}

private void GetMsgCallBack (Tny.Folder folder, bool cancel, Tny.Msg msg, IntPtr user_data)
{
  if (msg != null && !cancel)
    this.msg_view.Msg = msg;
}

private void OnMailSelected (object o, EventArgs args)
{
  Tny.Ui.GTK.HeaderListModel model = (o as Gtk.TreeSelection).TreeView.Model
          as Tny.Ui.GTK.HeaderListModel;
  Tny.Header header = model.GetHeader (o as Gtk.TreeSelection);
  if (header != null) {
    Console.WriteLine ("Message selected: " + header.From);
    this.cur_folder.GetMsgAsync (header, GetMsgCallBack, StatusCallback);
  }
}

private void OnFolderChanged (object o, EventArgs args)
{
  Tny.Ui.GTK.FolderStoreTreeModel model = (o as Gtk.TreeSelection).TreeView.Model
          as Tny.Ui.GTK.FolderStoreTreeModel;
  Tny.Folder folder = model.GetFolder (o as Gtk.TreeSelection);
  if (folder != null) {
    Tny.Ui.GTK.HeaderListModel headers_model = new Tny.Ui.GTK.HeaderListModel();
    Console.WriteLine ("Folder selected: " + folder.Name);
    this.cur_folder = folder;
    folder.GetHeadersAsync (headers_model, true, GetHeadersCallback, StatusCallback);
  }
}

It would

I just finished making Tinymail‘s .NET bindings actually usable. To test whether the binding works I wrote a small demo user interface in C#.

I added minimal documentation too.

Because Tinymail does quite a bit more with the GTypeInterface GType than Gtk+, Mike’s GtkSharp code generator for this had a few problems. I expected it to fail a lot worse, though. Without any improvements it did already get pretty far by itself! Mike and me have been pushing its code to support even the most crazy kind of things that Tinymail is doing with interfaces in GObject-C. There is still a lot of room for improvement for both Tinymail’s resulting binding itself and GtkSharp‘s Gapi parser and code generator.

With current trunk of GtkSharp and after applying two patches for its generator, it’ll make you a quite nice .NET API now. I’m of course working with Mike on getting proper solutions for what I had to patch upstream in GtkSharp’s Gapi.

I would like to thank Mike for his patience and help while I was making this binding work. I hope other library maintainers will use Tinymail’s one as knowhow-offset to get their own bindings working in .NET too.