Tinymail’s pre 0.0.6

I just released Tinymail’s pre-release 0.0.6. I’m hoping (and thinking) that this will be one of the last pre-releases before I’ll do a final first version of Tinymail. Although I have probably been telling this about every previous pre-release.

This release has neat new features and a lot of important bugfixes. Among the new stuff you’ll find Vala bindings, Namespace support, symbol annotations, a lot of API documentation improvements, improved decoding & language support and it has endured a lot of testing for Modest’s beta releases (which might explain the many bugfixes).

You can find the release’s announcement here.

Namespaces in Tinymail

I just committed support for NAMESPACE in Tinymail. This means that now you can show your boss’s shared folders in Tinymail based E-mail clients.

The “Shared” and “Other” namespaces are often used to put large lists in like NNTP newsgroups, so I only support the list LSUB returns for those two namespaces. LSUB gives you only the mailboxes that you are subscribed to. For the “Personal” namespace, I get both the LIST and the LSUB or in case you have LIST-EXTENDED I ask for LIST (SUBSCRIBED) and filter-out the \NonExistent ones, and flag the \Subscribed ones.

The better way to use “Shared” and “Other” is probably by not recursively asking for the entire list. In stead, when the list is unfolded you ask for the flat list of children under the level of the list. This of course means that you can’t know whether or not the UI must show an unfold arrow. For that you have the \HasChildren flag in IMAP. I’m not yet using this, though. More work for me, yippee!

Once this is done, you could make a UI to help to user subscribe to folders in accounts that have very large amounts of folders.

Namespaces in Tinymail, in Evolution?, unfinished but works …

Evolution’s default IMAP provider does not support shared folders. The problem is of course in the code that handles namespaces. I fixed this in Tinymail‘s camel-lite. I know this is a much requested feature for Evolution too.

I would like to point out to the heroes who’ll, in stead of begging for me to port the patch to Evolution, this time port the patch themselves, that the patch is a.t.m. in a very ugly state (I made a few shortcuts in the code). That camel-lite’s LIST code for IMAP has been adapted in multiple places to consume less round trips (it might not apply easily on upstream Camel, you’ll have to do quite a lot of merging). That this stuff is experimental and that a raw port of the patch would most likely not get accepted upstream yet.

If you want to work on this perhaps join one of the IRC channels that I use and let’s discuss how we’ll clean it up and bring it upstream.

Edit: I’m making more intrusive changes for Tinymail’s version right now. Check Tinymail’s mailing list to follow up on this (for example this one).

Open-source-facts

In follow-up to what Luis wrote, i just took a look at open-source-facts and liked it a lot.

More space for hacking

I cleaned up my desk once more:

More space for hacking, less for bugs.

Modest’s first public beta release

Igalia and Dirk-Jan C. Binnema from Nokia just announced the availability of Modest‘s first public beta. You can find the one-click install link at their announcements.

Jürg Billeter’s Vala Tinymail demo E-mail client

The demo E-mail client can be found here, the Vala bindings for Tinymail can be found here.

Many thanks to Jürg!

Here are the instructions on how to get the binding regenerated.

Just do this to get the Vala demo E-mail client running:

  • Get Vala’s latest release here. You don’t need anything special. Just configure, make and sudo make install it with –prefix=/opt/vala for example.
  • export VALAC=/opt/vala/bin/valac
  • export PATH=$PATH:/opt/vala/bin
  • Build Tinymail: ./configure –prefix=/opt/tinymail –enable-vala-bindings && make && sudo make install
  • Setup your Tinymail (demoui) accounts (described here)
  • cd bindings/vala ; make && sudo make install
  • cd tests/vala-demo
  • make ; ./tinymail-vala-test

Some dude on IRC

[17:03 today] <juergbi> pvanhoof: i’m reading email with a vala tinymail client now :)

Development documentation for TMut

I just documented TMut. I think this documentation will help E-mail client developers who are planning to use Tinymail a lot. Some of them will probably take a look at how I did TMut and now they’ll have some documentation about how and why I did certain things.

Also the client developers who’ll start their E-mail client based on TMut will enjoy reading the documentation. If somebody wants to create a client based on TMut, and if for that we need to make TMut a static library in stead, just ask of course (or just send me the patch that adapts the Makefile.am for this).

You can enjoy my normal English as Tinne is at this moment sleeping and I don’t want to wake her up to correct my poor English. The language purists will have to wait for her language correcting until at least tomorrow, or correct it themselves (it’s a wiki, go ahead). Perhaps we should go over all of Tinymail’s and TMut’s documentation once more. It has been a while since we last did a massive spelling correction campaign.

Anyway, here are the most interesting added subjects:

Have fun

Push E-mail with IMAP

Note (edit): Some people prefer me to call IMAP’s Push-Email “notifications”. That’s fine of course. Just replace “Push” with “notification”.

Everybody who’s into IMAP already knows: IMAP does Push E-mail. Nowadays nearly all IMAP servers are implementing the necessary features for it.

Marketing people wanting to sell their proprietary mail server solution have been claiming that IMAP is not true push e-mail because it doesn’t push the entire E-mail to the client. This is incorrect and just marketing. It’s true that IMAP gives you as a client developer the opportunity to decide to waste bandwidth or not at the cost of one single round trip. Round trips are less important for Push E-mail, as a push event is a background event.

What happens is very simple. In stead of pumping the entire message through, it sends the new total count of your folder. This makes sense, because if you are not interested in push events, you only wasted a few bytes of bandwidth and if subsequent push events happen you get automatically upgraded. Just compare your local number with the new number and make a decision.

C: a IDLE
S: idling+
S: ...
S: * 3 EXISTS
C: DONE
C: a OK
C: b FETCH 3 (ENVELOPE)
S: * 3 FETCH (ENVELOPE ("Tue, 19 May...))
S: b OK
C: c IDLE
S: idling+

We see that we have to exit IDLE to fetch the message’s ENVELOPE. This is why NOTIFY is being introduced. With NOTIFY you simply set at the start of the session what you want the IMAP server to send you during a push event.

If you want it to pump the entire E-mail through, you can instruct it. True push E-mail according to the marketing guys! Nobody really cares about this. though. People with a technical understanding know that you just want to fetch the fields that are interesting and absolutely not the entire mail: that way a spammer could bankrupt you if he knows that you are using expensive GPRS while roaming.

NOTIFY makes the client implementation more simple. You can e.g. request the X-Priority header and/or your company’s custom header. “X-Came-From The-Boss: Yes”. Based on that info you can choose to make the phone perform certain actions.

C: a notify set (selected MessageNew (uid
            body.peek[header.fields (from to subject)]) (all)
            MessageExpunge) (subtree Lists MessageNew (uid) (all))
S: ...
S: * 127 FETCH (UID 127001 BODY[HEADER.FIELDS (From To Subject)] {75}
S: Subject: Re: good morning
S: From: alice@example.org
S: To: bob@example.org
S:
S: )
S: ...

GPRS and IDLE or NOTIFY are fine by itself too. When we wait long enough in the select(), most GPRS chips will automatically go into low power mode. This way you can wait up to 30 minutes before an IMAP server thinks that you got disconnected. Such very high timeout times are specified for e.g. mobile purposes and high latency situations (IMAP from Mars!). Misconfigured NAT gateways that make timeout decisions more early are a problem. The biggest problem is that there’s no way to find out what the timeout for a session at the NAT is.

IMAP as a protocol makes it possible to process alien Push events too. Most IMAP servers, including Cyrus, provide a small daemon that handles the pushing. Cyrus, for example, has “idled” for this purpose. Very often you can quickly make a plugin for the IMAP server such as converting the IDLE event to an SMS and sending it to the user’s phone.

Just remember that with a good GPRS chip with correct power management, this ain’t really needed

If you are a phone builder and you are integrating a GPRS, UMTS or EDGE chip, I recommend getting the power management of it right. In the future your users will stay connected with the IMAP server: the timeout for the NOOP can be high. I hope the people who maintain the NAT gateways of GPRS, UMTS and EDGE at the many mobile networks will consider such use cases in stead of closing sessions at extremely small timeouts.

Tinymail does Push E-mail with IMAP :-)