Month: January 2007

I must be a little bit crazy and anti-paranoid about security for advertising this, but I also think that testing infrastructure is very important.

That’s why I have begun setting up different test IMAP servers for tinymail development. The domains might still have to be distributed to your providers DNS servers, I still have to make the mailbox for the user, etc etc.

I already created a little web interface that allows you start and stop the IMAP servers. Of course if I see too many planet users trying that out at the same time, I will disable it for a day or two. Just assume that it works :-).

The unit test environment of tinymail will probably get some scripts that will turn off and on certain IMAP servers for in case they will be testing capabilities that are specific for a specific IMAP server. They will indeed test online and reset the mailbox to its original state (which reminds me that I still have to add that to the web script).

Other projects are allowed to use the IMAP servers too (but do read the abuse item on the wiki page). Though there’s no promise that the services will always be available at all times and in its current shape.

I configured three virtual machines for this. Two of them are already running IMAP server software: one is running Dovecot and one Cyrus (the webscript allows for multiple installs to be stopped and started). Both the very latest and most unstable version that I could find. Yes I’m anti-paranoid about security and I do understand the risk that I’m taking here (and by blogging about it, I just increased that risk).

I welcome help from the server developers. If they want root on the machine and perform upgrades to the install of their softwares, then we can definitely discuss that. I’m also going to allow commercial IMAP softwares. But once many such servers have to run on the hardware simultaneously, I will not sponsor the hardware myself any longer. I guess that the commercial ones will have to get me hardware then.

So you can check more about it out here. I will be fine tuning it this evening and I might already change some unit tests to start using them too.

With the focus on Lemonade IMAP features and (therefore) wanting to optimize bandwidth utilization, I added support for the BINARY IMAP capability to tinymail’s IMAP code. The binary retrieval reduces retrieval size of certain messages (encoded is typically larger than the original message). Together with the existing support for CONDSTORE, STARTTLS with OpenSSL or NSS (which should compress too) and the optimizations for when you are retrieving the summary, this should mean that tinymail is indeed getting better and better at making sure that it’s not wasting your expensive GPRS connection.

I’m planning to write or steal somebodies ENVELOPE response parser so that I can use that in stead of the optimized summary retrieval code. This ain’t going to make a huge difference (just a few bytes per summary item). Those few bytes do add when retrieving really large folders, so it’s still worth the development time (… indeed, and I agree with you. Don’t worry).

Other plans include again rewriting the summary code (that’s the mmap stuff). This time I want to store segments of the summary data in mmap()ed files. For example 1000 items per such file. Then read-only mmap all of the frozen ones. Then when the next 1000th summary item is added, store the last 1000th in a file and mmap it too. Adding it to the list of segments.

The reason for this is that with the Lemonade IDLE code, adding summary items is going to happen more sporadically. Right now it’s very predictable when lots of summary items are going to be added to the summary: when you are retrieving or synchronizing your summary of course. With Lemonade’s IDLE there is less control over it. It’s the IMAP server that will “Push” this to the client.

I want to get it that stable and good that while this is happening, I want the view to start updating itself. This is what the TnyFolderObserver and TnyFolderMonitor are going to be responsible for: the IDLE event will trigger the invocation of the update method on all registered observers of the folder instance. The monitor will be such an observer that will deliver it to the list models of your views (those are TnyList implementations like TnyGtkHeaderListModel which is one that also implements GtkTreeModel).

Another reason is that that way I will also put the changeable data, which is only the flags, in a separate file. This will reduce level wearing on file systems that run on flash devices whenever changes to the summary happen. That’s because all the flags together will easily fit in a few blocks, and it’s not interesting to keep integers in the mmap. The pointer to that integer would consume as much memory as the integer itself. Leaving it out of the mmap()s will reduce the filesize of the segments, and the complexity to parse through it.

A last reason is that I can make looking up a summary item using the message’s uid a lot faster this way: I would store the summary items sorted on uid in those segments. So a little bit of simple math to find out in which segment it will belong (which would serve as a first index) and then utilizing a search that is optimized for sorted data (preferable one that doesn’t make big steps while comparing, as that would thrash possible caches). This is also going to help me a lot with mimicking a VFolder feature in tinymail, that works offline too (not just SEARCH commands on IMAP servers), that is — which is important — fast enough (and will also push found items to the views using the TnyFolderObserver stuff, while searching the read-only mmap()s in the background).

Equally important about such a search feature is that tinymail can NOT, no absolutely NOT, consume as much memory as Evolution does by keeping all summary of all folders in memory at all times.

There are a few other todo items too. Which are more bound to tinymail itself and less to tinymail’s camel-lite code. But those are all documented here already.

I hope the plans sound good :-). But I’m absolutely interested in your technical opinions. The future is going to be challenging, but definitely not undoable. The undoable part (starting the project and not giving up on it), is already done. Right?

For the Maemo ui, I’ll let Dirk-Jan surprise you.

Recently I was talking with Dave about using ENVELOPE in stead of what Evolution’s Camel does for fetching the summary information. The camel-lite of tinymail, however, already reduced the query a lot by cutting away everything that was not being used.

Months ago I cut away headers like REFERENCES, IN-REPLY-TO, MIME-VERSION, CONTENT-TYPE and in some situations also X-MAILING-LIST, X-LOOP, LIST-ID, LIST-POST, MAILING-LIST, ORIGINATOR, X-LIST, SENDER, RETURN-PATH, X-BEENTHERE. Though that last list was conditional they do get received by most people’s Evolution. Only if you have put “basic_headers” in the “url string” of your IMAP account, this last list of headers isn’t used in Evolution.

In contrast with that I figured early on that an E-mail client for a mobile-device does not need all the information, per message, about the mailing list. The summary view was not showing the reply-to field and tinymail doesn’t yet support showing the headers in a thread view. Which is what the references header is used for.

I’m mentioning this because I want to get rid of the reputation that Evolution has when it comes to bandwidth utilization. Tinymail is using an extensively modified Camel for a reason, indeed.

This was before I learned about Lemonade. You must know that back when I started with tinymail, I was not very experienced in all this E-mail & IMAP stuff. I’m still not, nor is it easy to know everything about it.

I got an account on one of the Isode servers for testing. I used their server for testing this, because that server (I assume) is a very efficient one when it comes to interpreting the IMAP specification versus bandwidth utilization (they are one of the companies that are pushing Lemonade, hence the assumption).

The most efficient way to ask for the summary is probably to use something like this (Don’t worry, people like Dave are going to correct me if it’s wrong):

A01 UID FETCH 1:2 (UID FLAGS RFC822.SIZE ENVELOPE)

ENVELOPE is often cached information, so it’s fast. In bandwidth, which is what matters most for tinymail because IMAP “servers” usually have enough power when comparing it with a mobile device, this consumed something like 800 bytes depending on the headers themselves. I’ll use the actual numbers of the account that I got from Isode.

This, however, is what tinymail is using today. You don’t want to see the query that Evolution is using :-). In a few words, it’s something like three times larger in what it asks for. You can probably compare it with a “TOP uid 0” on a POP3 server (most headers). It’s not about bashing Evolution. I’m just comparing facts here.

A01 UID FETCH 1:2 (FLAGS RFC822.SIZE INTERNALDATE BODY.PEEK[HEADER.FIELDS (DATE FROM TO CC SUBJECT MESSAGE-ID)])

This consumed 825 bytes. Don’t forget that back when I was optimizing this, I wasn’t into writing a parser for ENVELOPE nor did I actually know about that IMAP feature. Remember: I was not very experienced with IMAP at all. My focus was on getting tinymail to do something interesting in terms of memory consumption while wrapping that in the nicest API that I can imagine. You know, I’m just one person with only two hands.

Luckily only a few of the things that are needed do consume a lot bandwidth. Important are receiving the summary, updating the flags and knowing about expunges and other such changes. In other words: synchronizing the local offline status with the online status.

And that is precisely what Lemonade is improving in the specification. Condstore, for example, returns me a sequence number that uniquely identifies the current status of the mailbox. I can create a very simple query that says: well, give me all changes that have happened since a state. Important here is that I only receive the changes. I don’t have to get all of it and start comparing things.

The difference of 25 bytes sure is significant enough to want to improve this. This will eventually happen. I also have to check whether using ENVELOPE on all IMAP servers works correctly of course. With worst-case servers around (experts know which server I mean), this isn’t self evident. This indeed, is what makes developing an IMAP client difficult. My opinion, and I like to express that as people already know, is that it’s better not to create an IMAP server than to create one that sucks. The problem is that users will depend on your defects. And that E-mail client developers need to cope with it.

So, why did I write this lengthy blog item?

Because I wanted to tell people not to get the focus of the tinymail framework wrong. Memory consumption is only one aspect of the project. Bandwidth utilization is actually even more important. Other aspects are design, testing, documentation, language bindings, flexibility and being adaptive to change.

By the way, didn’t I whine a lot about being adaptive to change a few months ago? I might have irritated some people, right? Guess what Lemonade brings us?

Change

This proofs my point. Once I implement support for the IDLE Lemonade feature, I already have a TnyFolderObserver type ready for this.

E-mail nor any other technology is in a stable state. E-mail has never been in that state though it’s one of the oldest Internet standards. Change is among us.

Be adaptive.

Support for the Lemonade IMAP condstore capability (RFC 4551) has been added to tinymail. This is the first support for a Lemonade feature. I added a point of view page on the subject to the tinymail documentation. It outlines the strategy that tinymail developers (as I’m no longer the only one) will take to one day let tinymail become a MUA framework that will outperform existing commercial solutions like the Blackberry E-mail software. That’s, indeed, what I’m aiming at.

From a financial point of view I’m taking things more seriously now. I now understand that there might be commercial interest in the framework. I also understand the simple fact that I could die in a car accident tomorrow, is a major risk for companies interested in adopting tinymail. For this risk reason, I’m building up a network of TnyPartners who can help customers with tinymail consultancy. I’m being at FOSDEM next month, so catch me there if this sounds interesting to you.

Rather than see this as competition, I’m planning to actively help both customers and those partners finding each other. If I would make this a competition-battle between me and them, the risk reason would turn it against the project. I’m aware of that.

My biggest concern is keeping the project a healthy free software one. Ever heard of Peter Principle? I have and I’m aware of my own limits. I’m not (yet) much of a project-leader (but hey, I have started reading books on emotional intelligence. Soon maybe). I think most people by now understand that I’m the engineer type of guy, who passionately and unstoppable enjoys building software.

This is one of the reasons why I haven’t yet done any releases: it would put an extra burden on me and it would awake the crazy “I hate and bash everything”-people. I’m not interested in that. Since tinymail is for software developers, I figured that those guys can checkout from the subversion repository. Some day, however, I will release something. Nevertheless, if somebody wants to be that project leader or release maintainer, call for it.

For people who want to experiment with LEMONADE-style IMAP servers, I created a little developers guide to installing one that supports the IDLE and CONDSTORE capabilities. You can also try M-Box or others. It’s a wiki, so people are allowed to improve the guide.

This doesn’t mean that I already have support for these two capabilities covered in tinymail. It does mean, however, that I’m preparing my test environment. A software developer usually prepares a test environment for a reason. That reason is, indeed, that I’m on it (implementing support for lemonade imap features in tinymail).

It will be developed piece by piece. I definitely have other todo items and priorities too. Since only a few running IMAP servers support this already (and by running I mean IMAP servers that are actually running, not just an software version of an IMAP server, but an active and working deployment at a site), unless you install your own, it’s not yet extremely important. However, I do agree, fully, that the lemonade proposals and drafts are absolutely interesting and are probably going to be very important for the mobile world (that includes laptop users on shitty wireless networks).

I’ll guide people to how they can help Evolution (and therefore also tinymail) get support for at least certain interesting Lemonade IMAP protocol features. You can all start looking at me to do it for you, but as my todo list just keeps growing and growing, you’ll be looking for a longer time than that it would take you implementing it yourself.

I’ll explain the CONDSTORE capability. CONDSTORE is interesting because the thing that does change about summary of messages are the flags. For example the \Recent and the \Seen flag of your E-mails, change. The CONDSTORE capability makes it possible to quickly synchronize these flags with what you have been storing locally (without having to retrieve all of them one by one again).

In Camel’s camel-imap-folder.c there’s a not-very-difficult implementation for this, called imap_rescan. And in Camel’s camel-imap-store.c you’ll find a static global table called “capabilities”. Simply adding the “CONDSTORE” capability there and putting an if (store->capabilities & IMAP_CAPABILITY_CONDSTORE) { /* your new code */ } else { /* original camel code */ } at that location, would probably be sufficient to actually implement it.

imap_rescan (CamelFolder *folder, int exists, CamelException *ex)

The existing implementation uses a trick to avoid calling the hashtable lookup that I mentioned yesterday. But for a few lookups (only the changed ones since last sync) it’s probably better to simply utilize camel_folder_summary_uid to get the CamelMessageInfo instance for a specific uid. Now use camel_message_info_set_flags on that instance and feed it the new flags which you just received from the server. You even already have the helper functions that turn strings like “\Seen” and “\Recent” into the correct bitfield flags.

If you need to add “(CONDSTORE)” to the “SELECT”-IMAP command: The “SELECT” command is implemented in camel_imap_command (a quick search in the code for “SELECT %F” will do wonders). Just make sure that you only add it in case you have IMAP_CAPABILITY_CONDSTORE in the store->capabilities flags.

So why am I whining about this? Because well, honestly .. not much people are working on this type of things. Yet this is something that both our desktop and mobile devices desperately need.

I decided to link to Camel’s SVN this time because I’m pretty certain that the Evolution Mailer maintainer would accept a good patch that adds support for the CONDSTORE capability. If you are more into experimenting with things that are rather unlikely to go in Evolution’s Camel soon, you can of course join me on experimenting with it in camel-lite. This isn’t a pure isolated fork. I’m trying to extract useful deltas out of those experiments and pushing them upstream. Which takes time on a per-patch basis, of course.

I’m willing to dedicate my entire FOSDEM-time to co-developing Lemonade IMAP support with interested people. I mean, we have developer rooms over there, right? Why not? We can even continue at night if you are not afraid of spending the night coding. Somebody?

In the end, it’s really about “just doing it”.

Nokia N800

Thanks to Nokia and more specifically Dirk-Jan C. Binnema for getting me a discount for the N800. I will of course make sure tinymail runs on it. Not that I expect a lot changes for that.

FWD: Misinformation

Jono makes this interesting argument about misinformation. I recently started reading this book about emotional intelligence written by Daniel Goleman. This is is, In my opinion, a good followup subject on Jono’s point.

I read an older book of Daniel Goleman on emotional intelligence before, but this one is more practical (I think I have the Dutch version of Working with Emotional Intelligence).

Psychology is interesting, indeed.

Telomer & Polymer

I’ll immediately respond on Telomer before I get flooded with questions on why doing tinymail if it’s all that simple:

Telomer is interesting of course, but it does not cache E-mail locally. This means that you must always be online to read E-mail. Getting the summary information and getting messages (actually, getting MIME parts) is not the hardest part of an E-mail client. The hardest part is parsing the MIME parts and dealing with the offline cache in an efficient way. Which is what tinymail is really about. Actually, from a technical pov it’s what camel(-lite) is about.

Tinymail of course also provides code for connectivity with IMAP servers (also for POP3 and NNTP) otherwise it can not get this information in that cache.

Another already working solution when your mobile device is always online, is also called webmail like Yahoo mail, GMail, Hotmail, Squirrel, Roundcube, IMP, etc. The problem is, however, that mobile devices are not always online.

You could argue that making a simple quick ‘n dirty cache is quickly implementable. The implementation of this E-mail client or library will, however, get more complex once synchronizing changes to that offline cache must somehow be reflected to the IMAP server once there is connectivity. And once you want to do this efficiently. Etc. But I’m interested, as it’s on the TODO list for Telomer & Polymer (I just promise that it wont be easy, hehe).

The Lemonade stuff, however, is very interesting. For example the P-IMAP (Push-IMAP concept) for when new messages arrive. You can be certain that I’m already looking into ways to use these IMAP protocol features on IMAP servers that support it (emphasis on that).

It’s very nice and extremely interesting to see Polymer and Telomer use and push the Lemonade IMAP protocol features. In time tinymail will obviously support all of it.

Anybody who wants to work on reimplementing the IMAP provider of camel(-lite) or on improving the existing one by for example adding support for the Lemonade protocol features is absolutely invited.

It’s time for some monthly bashing-others about memory wasting. Moehaha! But hey, Camel was not done that bad. This is a friendly one. Some people in the past had this strange idea that I was questioning the competence of those who’ve built Camel. That’s absolutely not true. There’s a reason why I use it myself, don’t forget that (and some day, people will bash the fuck out of tinymail, right? Good! keeps us going).

Some developers enjoy the fact that glib makes their lives more easy a lot, it seems. And I agree with them! You should leverage the fact that glib is a well tested library. Who wants to implement his own hashtable or doubly-linked list each project he starts? I honestly don’t.

However. In the CamelFolderSummary, the number one piece of code that consumes most memory of E-mail clients that are being build on top of Camel (that includes Evolution and tinymail with its camel-lite), we surprisingly see a GHashTable being used in parallel with a GPtrArray! Both holds the exact same instances?!

Somebody like me … questions that. Because both in memory and in performance this (in this case) is a loose-loose situation: adding it to the hashtable takes time, searching a GPtrArray is (in this case) not going to take a lot longer (read below for more information on the “in this case” situation).

Consider the memory caused by each item you add to the GHashTable:

struct _GHashNode {
  gpointer   key;
  gpointer   value;
  GHashNode *next;
  guint      key_hash;
};

That’s on a typical x86 architecture 4 + 4 + 4 + 4 bytes, right? 16 bytes per item (Or am I calculating this wrong? There’s no waste caused by mem alignment here I think, and it uses g_slice_new so there’s also not a lot heap admin). Multiply that with 10,000 headers, that’s 152KB of memory. Nothing you say? Okay, I agree (well, except that for a mobile application this is a significant memory improvement).

However, consider that each and every summary item that you can see in your Evolution consumes this. With the help of some mailing list subscriptions, I use Evolution to manage up to 1,000,000 messages that way: 15,625KB or 15MB of GHashNode instances (gah, I must be miscalculating because that’s much more than what I expected).

Because this:

static CamelMessageInfo*
find_message_info_with_uid (CamelFolderSummary *s, const char *uid)
{
	CamelMessageInfo *retval = NULL;
	guint i = 0, len = strlen (uid);
	for (i=0; i < s->messages->len; i++) {
		CamelMessageInfo *info = s->messages->pdata[i];
		if (info && !strncmp (info->uid, uid, len)) {
			retval = info;
			break;
		}
	}
	return retval;
}

Is more difficult than this:

g_hash_table_lookup(s->messages_uid, uid)

I don’t whine without a patch, right? right!

Update: and on the tinymail front, I removed the need for often calling this function (update: which is an important pre-condition for applying patches like this). This made loading large folders a lot faster when using the framework for this. You can svn diff -r 1451:1452 to check what I changed for that.

Update on CPU consumption (because some people have questions about that): The hashtable lookup would indeed be faster, but the hashtable lookup can (and usually is) avoided in Camel. Not sure whether it’s also avoided (and avoidable) in Evolution code (that’s a lot code to check), but all occurrences where the hashtable lookup is needed are avoidable and are therefore avoided by tinymail. In other words: looking up by uid is not often needed. But you are right that a hashtable lookup is faster than that find_message_info_with_uid implementation.

Today the tinymail project is exactly one year old (it depends a little on how you count, but today one year ago, the first .c and .h files in my ~/Projects/tinymail directory where written).

I just implemented partial message retrieval (only receiving the message of an E-mail, not the attachments) for POP in tinymail. This was an uncertainty that I had with some of the Nokia requirements: In contrast with IMAP is POP3 really a dumb protocol that knows nothing about MIME parts. Yet I succeeded in actually implementing this. Next to summary support in the camel-lite’s POP provider (which uses the TOP capability if that is available, most POP3 servers have this, and else it too uses this new partial message retrieval feature).

The summary of your E-mail client is only the basic information for viewing a summary of a folder. You obviously don’t need to receive all message data of all messages for that. That is what this summary support for POP3 is all about.

My technique, for partially retrieving messages from POP3, is to search for the boundary in the Content-Type header and storing that parameter. Once after the headers, the second time I find this boundary string I simply disconnect from the POP3 server, make sure the already retrieved message data is correctly written to the file system as cache and then I simply let the software reconnect once the next command is unleashed. I still need to add detecting whether a message is partially retrieved and retrieving it again in case the strategy for receiving messages has since switched.

For example say the user received it partially, but now learned that the message is interesting and would therefore like to fully receive it. Including the attachments. For that I need to add some administration code that stores the retrieval status of the message and makes the correct decisions based on that.

Nevertheless has the milestone be reached: partial message retrieval for at least IMAP and POP3. Both are now implemented. The IMAP one fully and the POP3 one, well in a few hours this will be 100% correct and fully working with retrieval status detection and code that gets it right.

Update: it’s implemented now

These are in my opinion important features for mobile and embedded that very few other smaller E-mail solutions implement. Done in such a way that it’s flexible for you, the application developer (strategy design pattern), to decide when and if partial retrieval is to be utilized.

Gnome mobile and embedded people, we have work to do: Steve Jobs is showing us his new little wonder. This time I think we will have to cooperate with the guys doing crazy 3D effects and hardware people too. I’m more than certain that with our technology we can actually compete with devices like this. The only thing we really lack, in my opinion, is coordination and cooperation between each other. Technical leadership (and I know this is the subject nobody wants to talk about).

I’m for sure going to try providing the E-mail infrastructure (not only for gtk+ based devices). But that is only an extremely tiny piece of the puzzle. The cool ui effects, like rotating the X11 window when the device physically rotates, is something for our cool X11 developers. The cool scroll effects (when using your finger) is something gtk+ can implement (if this ain’t a patented idea of Apple of course). Input techniques: same (also about the patents). Hardware: Nokia, Palm, OpenMoko? Media players, we have plenty of them, but do we have one that has a really suitable ui for mobile devices? Honest question. GSM & phone functionality, I guess that is going to be hard and will depend on the hardware a lot (but not impossible). VoIP, aha! We have Telepathy! Presence, sure: Galago! Cool art: yep we have nice SVG icons and I’m sure we would find a lot artists who could create us very good stuff for such devices.

What are we waiting for? To be as usual late for the next big thing? Steve is clearly on it already. It’s not the first time that he showed us what that next big thing in technology is, right? In combination with full wireless Internet coverage in bigger cities (which will most likely happen sooner or later), I think devices like this are one such next big thing for the coming years.

Some of the core functionalities that I have in mind (I on purpose decided not to put a lot project names in the bullet points):

• A Windows, MacOS X AND Linux client for uploading music and other files from and to the device;
• VoIP, Presence;
• E-mail & SMS capabilities;
• Small browser, good Flash & AJAX support;
• An easy-to-use media player;
• A small X11 with some effects;
• A camera and a modest photo management tool;
• A Java VM like J2ME, maybe .NET and Python too;
• A good development platform for it. One that Windows developers can use too (“emulator.exe” -style);

OpenMoko and Nokia are, indeed, a good start. In my opinion, we need a lot more like this.

My opinion? Let us not wait and sit, in other words stare at it, like lame ducks. We do have the competence in our group to compete with this!

Maybe cooperate with Steve? I’m sure that such devices need standards like configuration standards, standards for transferring files and deploying music (insert more here). Might be Utopian thinking to want to get this standardized. Yet I believe that this standardization process would unleash the mobile world. Let the others fight each other with their standards while we create the real ones behind their backs?

Think different, be different, be creative and a few years later: Right now the new is you. Your time is limited so don’t waste it living someone else’s live. Have the courage to follow your own heart and intuition. Everything else, is secondary.

Finally, a last great quote from that recent speech:

Stay hungry, stay foolish.

Partially retrieving messages is something important to support if you are targeting the mobile market. The reason for that is that not all devices come with huge flash disks, yet all E-mail boxes can contain huge E-mail messages. Imagine receiving an E-mail with three MP3 attachments and two CID embedded 750kb jpeg images on a mobile phone that has a flash disk of five megabytes.

I implemented a feature that makes it possible to switch between retrieval modes. I implemented two retrieval modes that are put in the framework: only getting the first mime part and getting everything.

You basically set the receive strategy of a folder to either a full message receive strategy, a partial message receive strategy or your own implementation that, for example, classifies between these two based on the message size (the size can be made available using the camel-lite CamelMessageInfo API, in other words: it’s available in the summary of the folder). More info here.

I also reimplemented a Camel IMAP implementation in camel-lite that fetched messages from an IMAP server. The original one copied the message in memory a few times. The new implementation will use the CamelStream API to immediately stream it to the file system while it receives the message.

This means that camel-lite, when compared with Evolution’s Camel, now improves memory and bandwidth consumption in four significant ways:

• The summary is read-only mmap()ed;
• When retrieving summary information from the IMAP server, tinymail’s camel-lite uses significantly less upstream bandwidth and it uses a lot less requests (sending a lot requests isn’t a very good idea on for example GPRS connections);
• When retrieving summary information from the IMAP server, tinymail’s camel-lite will periodically dump newly received information into the mmap while receiving is happening;
• When retrieving messages from the IMAP server, tinymail’s camel-lite does not copy the message in memory but rather streams it straight from the tcp/ip connection into the local cache-file. On top of that does camel-lite implement partially retrieving messages (which of course also reduces file system consumption if used).

I will now start working on getting POP support working with summaries. At this moment will an E-mail client like Evolution copy all (or filter, in the end it’s the same) the messages that are on the POP server into a local mail folder. On a mobile device with a small flash disk this isn’t a very good idea. It’s more interesting to store a summary and only cache the E-mails that the user really wants to cache. For IMAP this is but for POP this isn’t implemented in tinymail (for POP it is a lot harder to get it right). This is going to change of course. Partially retrieving messages from a POP service is also planned, but I’m uncertain if this is actually doable in a sane way.

Other things that are on the agenda for tinymail: uh, eum .. a lot. Check out the trac. For example: backing up and merging cache, removing attachments from an existing cache to reduce file system consumption, dealing with all sorts of strange error situations (like failing connectivity, batteries dying and the power of the device shutting down), GtkHTML support (still needs some work), I started trying to get some glib dependencies working on WinCE (so who knows, maybe someday tinymail on WinCE), maybe a default mail composer component, searching the summary of a folder, more status feedback, language bindings!, more documentation, etc etc, integration with existing calendaring tools, etc

I created a new video. This time the video shows the tinymail demo user interface on the One Laptop Per Child laptop.

My little brother helped me a little bit with making this video nice. You know, with fading background sound from Karoliina’s excellent music samples and such a black end screen with a logo and the URL.

I hope tinymail will be used a lot on mobile devices and by projects like the One Laptop Per Child.

Project teams and companies should know that I am accepting and already working on custom tinymail development, enhancements and feature implementations. For the One Laptop Per Child project I will make an exception: I will try implementing what they need for free because I truly believe in their project. Yeah, I’m one of those guys who also believes that for example wikipedia is truly going to make a significant difference.

Tinymail on the one laptop per child laptop

Non-youtube versions of the videos can be found here

To prepare for the moment of releasing a first version, I started with my “modest public relations campaign of selling the framework” :-) by posting the existing demos on youtube.

In other words, please send these youtube links to your software development team responsible :-) (for example in case your company is interested in developing E-mail related functionality for a mobile device, an embedded appliance or on a desktop environment). You can also add a link to its main website, to its documentation and to its API reference manual.

ps. If your blog aggregator removes embed tags, you can find the same (youtube/flash) movies on this and this page on the tinymail website.