Category: english

While developing on top of Tinymail you need to make a platform factory, a device type and an account store. The platform factory will create instances of types that are specific for your device. The device type will tell the Tinymail framework about the state of your device (online, offline and state changes) and the account store is where the accounts are requested from.

These types are interfaces in Tinymail: you don’t want me to tell you what configuration libraries to use for putting account configuration in, neither do you want me to tell you what Network detection to use. A cellphone’s GPRS connectivity might or might not work the way NetworkManager works! What about the future? Think about the kids!

Perhaps I want to decorate an existing local account configuration by synchronizing it using data from an ACAP server? What about doing this for all the account stores that I ever wrote?

By itself that doesn’t mean that you are on your own while starting your application. Of course does Tinymail have a bunch of default implementations. For example a device implementation that uses NetworkManager and an account store that uses GConf. Obviously there’s also documentation in the format of a manual that explains how to implement your platform factory, how to implement your device type and how to implement an account store.

Dirk-Jan told his audience (slides, page 4) at GUADEC ’07 that this makes your application and Tinymail look a bit like a chessboard. That’s right, you do need implement certain Tinymail types to build your application on top of it. This is not really abnormal in a modern framework. Consider things like Spring and Hibernate. For those frameworks you need to integrate your code with the framework too. Although such frameworks enjoy the fantastic world of virtual machine capabilities. Like reflection.

That’s a lot of interface implementing and type inheritance to support in a language binding. And this is exactly what Mark (Doffman) is providing in the Python binding of Tinymail.

• A demo Python account store
• A demo Python Tny.List
• A demo Python platform factory

I’m sure a lot of people don’t care about the platform, a lot of other people are interested in seeing our components running on WinCE targets.

A reason why I believe making our GMAE products work on WinCE is interesting, is because of the existing market: mom, pop, bro and sis are not going to flash their expensive devices into a for themselves nearly unusable state. Maybe if you’d pay the shop that distributes the devices to flash it, you might convert a single percentage of the users to your absolute freedom Utopia. Most of them, will not really care and go with the flow.

That doesn’t mean that I don’t like initiatives like what Openmoko and Maemo are doing. Those are great, for the developers. The actual users, usually don’t care that much about the freedom aspect of it. (sure, comment that you disagree and that users do care. Please also post the actual numbers that prove it). The availability of applications that comes with opening a mobile, however, is what its users care about indeed. If this is in an integrated way (with a nice website where they can find the point-and-click installable applications).

We can, however, make software for people who made the decision to buy a WinCE based device. We just have to finish the WinCE port of pthread-win32, get iconv compiling and then build a glib for it. With the relatively new project cegcc it’s easy to set up an environment for cross compiling to a WinCE target. It works a lot like Mingw32. They even have a compiler that is compatible with it (and which seems to be working with autotools nicely).

My first experiments gave me the typical results for iconv and pthread-win32: WinCE has no errno.h, and when faking it you get unresolved symbols about _errno. Such problems are relatively easy to fix. The pthread-win32 library tried to link with wsock32, which I had to replace with winsock. There are some platform dependent parts in the pthread_cancel implementation of pthread-win32. I don’t think glib uses pthread_cancel so you can perhaps initially just ignore this and make it compile with an empty implementation.

The latest version of iconv seemed to be in an even better state. I think the only problem that I had while trying to compile it was the unresolved symbol _errno. Although I think WinCE has its own typical iconv related things.

Making Gtk+ work on WinCE might also be worthwhile. Although my opinion is that D-BUS and glib (including gobject, gmainloop and gthread) are initially more important. Tor did some work on porting Gdk/Gtk+ to Win32, perhaps is a bunch of that stuff reusable?

Once finished, we could run GMAE applications and components on WinCE phones, PocketPCs with native O.S. and a lot of tablet computers. I’m for example very interested in the many ruggedized embedded and mobile devices. For example the Psion Teklogix ones. A lot of companies let their employees use such ruggadized devices in the field. This might be an interesting market to explore with GMAE? And why not? Those companies usually see no reason whatsoever to flash their device with Linux, if their custom Compact Framework .NET application already just works for them. Although Mono might be very interesting for the company who developed that software?

Mark Doffman wrote a roadmap for the Python bindings of Tinymail. He just committed the possibility to implement Tinymail’s types in Python. Although I haven’t tested this yet, it looks like he’s making the bindings kick ass.

I guess this means that good Python bindings will be a feature of the first version of Tinymail. Thanks a lot Mark!

IMAP has the STATUS command which can give you information like how many of the messages in a folder are not flagged with \Seen. That’s cute, but not as useful as it might sound to most people. This is the problem:

What I want to avoid on a mobile device is having to do a synchronization of a folder. A synchronization the first time means retrieving the summary information like the Uid, the flags and some headers. What use is all this information if this is a folder that will never be viewed by the user of a mobile device? Why waste bandwidth on retrieving this, especially when bandwidth over GPRS is expensive for many people?

How will you know (before said synchronization) whether a message is unread but not deleted and how many of that unseen count, that you got from STATUS, are like this? It’s perfectly possible for a message to be marked both deleted (but not yet expunged) and unread.

“Simple”, I imagining hearing somebody in the audience say: “just mark all messages that you delete as \Seen!”. The problem here is that a mobile E-mail client is most of the times not the primary E-mail client of a user and I can’t control the behaviour of the many E-mail clients being used in the field.

The nice people at Isode specified ESEARCH for things like this. To all developers of IMAP servers: please hurry up implementing this one. With ESEARCH I can get an accurate unread count suitable for mobile devices without having to do a folder synchronization for each and every folder.

As some people at GUADEC asked me to continue the technical stories about Tinymail, here’s a preview on how it goes:

When you open a local E-mail folder that is stored on a MMC card, you obviously need to open some resources. Those include, for example, the summary’s mmapable file. You might also need to opendir() the directory and perhaps even chdir() to it? Perhaps rename items from a “new” or “tmp” to a “cur” subdirectory? In the process you might have left a directory or two open?

Suddenly … you get bug reports from system testers telling you that if the E-mail client is running, they can’t unmount the MMC card.

Today, Tinymail’s implementation for a few local E-mail folders makes sure that no directory related resources are left open, chdir() always goes to the root folder and the mapped files are closed as soon as the last component stopped using the folder.

Mark Doffman wants to maintain and fixed the Python bindings of Tinymail. He committed a huge patch restructuring some things and making things work again. I guess this means that very soon and perhaps already you can write your own cute little E-mail client in Python?

I know that for example one person plans to make a mutt-like using Tinymail. Maybe that one can be done in Python? I think the Python bindings will be most interesting for the OLPC project, though.

Vala bindings will of course be made very soon too. Though, my current plan for Tinymail 2.0 is to rewrite all interfaces and refactor a few of them to abstract classes … in Vala. I hope that by that time Vala will generate a bunch of bindings to other languages. If I look at the progress that the authors of this programming language are making recently, I’m confident it will.

I just noticed that it has been since May that I did a video demo. I’ll try to make something soon. What about Tinymail on the Openmoko phone?

For most of the organizations that needed it, it comes a few years to late (when i started Tinymail, they wanted it yesterday). I promised to start the process of making early releases the first time one full year ago. I completely underestimated it, indeed.

Finally, a first pre-release of Tinymail. I hear people think “What? Just a pre-release? Not even a final release?!”. The pre-releases are to aid people like the packagers with preparing their environments and to give a solid basis for application developers who want to start focusing on an API that will not change a lot anymore.

It also means that everything involved with releasing a final will now start. You can find the announcement with all the information on the mailing list.

ps. I have no idea when or if a pre-release of Modest will be made. I have no idea when or if packages will be made available.

I have been reading “The Transparent Society” by David Brin and a .NET book “Framework Design Guidelines” by Brad Abrams and Krzysztof Cwalina. On page 10 of the former book I found something interesting:

“In all of history, we have found just one cure for error — a partial antidote against making and repeating grand, foolish mistakes, a remedy against self-deception, That antidote is criticism. Scientists have known this for a long time. It is the keystone of their success. A scientific theory gains respect only by surviving repeated attempts to demolish it. (page 10)”

I have been closely following the criticism on Tinymail. I indeed noticed that there has been certain difficulties. The book continues with:

“Only after platoons of clever critics have striven to come up with refuting evidence, forcing changes, do a few hypotheses eventually graduate from mere theories to accepted models of the world. (page 11)”

This is why I’m a critic myself. This is what makes me want to listen to criticism for my own things. Because criticism is a partial antidote against making and repeating grand, foolish mistakes. Against self-deception.

My plan is simple: I’ll take the criticism, I’ll learn from it. I will educate myself using the .NET book and I will improve both my own skills and the framework.

I like the .NET book because of an interesting quote:

“Eight out of eight failed to read text from a file in the 30 minutes we allocated for the task. We believe this was due in part to problems with the documentation search engine and insufficient sample coverage; however, it is clear that the API itself had several usability problems. (page 17 and 18)”.

They explain that they designed the .NET API scenario driven. This means that they wrote the code of a scenario and only then constructed the classes and interfaces.

I think this is an important lesson in framework design: do usability studies on your API and you’ll learn that with your own frameworks eight out of eight will fail while performing the simplest tasks. The authors of the .NET book write that their goal was to provide a pit of success for their customers, not a pit of failure. I want the same for Tinymail: to have a pit of success for the application developers.

ps. The former book is not about a technical subject like designing a framework of course (I know, you don’t have to tell me). Rather about privacy vs. technology. The connection between the two subjects is only about these quotes: I’m reading both books at the same time.

Murray’s blog spurred some controversy about Evolution and the adapted Camel that Tinymail internally uses.

Having done most of the adjustments I’d like to point out that some of the changes are indeed focused on mobile use. Some of the changes alter the IMAP communication in such a way that less bandwidth is consumed. Examples are omitting the references header while downloading so-called summary information per message and the support for CONDSTORE.

CONDSTORE is a recent enhancement to the IMAP protocol that makes it possible to ask the IMAP servers for just the differences since a certain known state. Rather than asking for all the flags of an item, comparing that with your local version and synchronizing the two, you ask for only what has changed and you synchronize the recent local changes remotely.

I removed the support for the references header in Tinymail because this was poorly implemented in my opinion: a doubly linked list was used for each reference. This as you know consumes eight extra bytes per reference for the pointers (for the purists: on a 32bit computer). I think just storing the string and using strtok would be better here. The current E-mail clients being developed using Tinymail, however, don’t need the threading granularity of a desktop E-mail client.

Most modern E-mail clients are also less and less utilising this feature the way most experienced E-mail users know it: usually E-mails are grouped together based on relevance (flagging by a human and determining this with an algorithm) or on indeed the discussion thread. Full threading is often overkill: most people just want the messages to be grouped together somehow (in a flat list per group, not really the actual tree). Mind that most mobile devices have rather limited display capabilities and that a tree wastes a lot of whitespace this way. What is the use of message threading if you can only display at most five rows of E-mail summary info?

This “references information” must be downloaded and GPRS sometimes costs the end-user money per downloaded megabyte. Those reasons combined made me decide to simply not request this information. Later I might make the projection of the actual query definable by the application developer who builds on top of Tinymail.

My conclusion? Certain things can most likely be backported to Evolution. Other things might just not be interesting. Cooperating is definitely more interesting than not doing so. Right now is an internal highly adapted copy of Camel the pragmatic right solution for Tinymail.

I promised my precious Tinne that I was going to take two weeks of vacation. I’ll start with the Belgian rock festival Pukkelpop and after that, two weeks of holiday together with Tinne.

I promised a release candidate of Tinymail last week so I guess I will still be doing that in the evenings. Other than that I’m planning not to work on anything for the first time in .. already a few years I think (all the holidays that I took at my former employer have been going to Tinymail development). I kinda forgot how holidays feel. I’m even a little bit scared of getting bored.

Well, no. I can’t get bored! I have a bunch of things to read, A girlfriend who’ll finally get some attention and a release to make. It’s not much to do, but it’s something.

I’m at the point of (naively) believing that Tinymail‘s thread difficulties are all solved. The developers working on Modest seem to be telling the same thing (today’s quotes from the developers: “Modest is quite robust today”, “yes!!! it works”). Whoo!

The POP code had some difficulties too, mostly related to deleted and deleting messages. These have all been fixed too. We have bugs like a translation bug about the SSL-question text: it’s not yet translated to your locale. That type of bugs. Still too much such bugs, in my opinion. An E-mail client has quite a lot of functions to support, if you just want something very basic (you’d be amazed). We’re getting there and I have this (again naive) feeling Modest is going to please most people.

Next week will be an interesting one as I’m planning to do a release candidate of Tinymail. That’ll be an actual tarball and a branch tag in a “releases” sub directory of Tinymail’s repository. We’ll make sure Modest’s repository will be synchronized and tagged with this RC. I’ll make sure all the build information will be available. Chances are high that we’ll put some packages for N800 online too. That’s not a promise, though. Especially not for Modest (I don’t make such decisions for Modest, but I can probably prepare packages for Tinymail on N800).

I’m expecting quite a lot of bug reports. Especially if we’ll provide packages, as that will increase the amount of people who’ll try it out of course. I will make test accounts available for people who don’t yet trust Tinymail with their precious E-mail data. I will also prepare a QEmu image with an IMAP server and some test data. Although this QEmu image will only be bundled with a release officially when I’ll do the final onepointzero release.

It has been a year and a half of hard work, especially last few months, but I think it’s getting ready now for beta phase, yes.

If people want to help with all the “make distcheck” work, let me know. I’m not very optimistic that a lot people will, doing a release is really boring work. That’s because, and I repeat, all our typical development tools (autotools, gtk-doc, etc) suck and are absolutely not synchronized with how developers really work.

Jürg Billeter finally has his blog setup. He finished the first building blocks of a DBUS binding for Vala and made a sample. Go check it out!