Month: November 2006

I recently tried out the TotalView debugger demo. I only have a few words for it: wow, nice, cool, etc.

This is an overview of the Memory analysis’s features. All of that is actually functional in the demo which you can download from their website. The product itself is, however, a little bit pricey. More than $2,000 for a license that has the memory analysis features. Note that I’m not such a religious anti software-that-costs-money developer. If it’s good software, I’m willing to pay for it. TotalView clearly is good software. But expensive too.

I agree that valgrind and ddd and/or gdb and/or gdbserver have a lot of the features that TotalView also has. Those tools, however, by far don’t integrate all the functionality as nice as TotalView does. And TotalView indeed has quite a few features that I haven’t experienced before. Very nice ones, and I only mostly tested the memory analysis tool.

It looks like tinymail is making a hacker happy. That’s why I’m doing it, that’s my very reason for doing free software and software development in general.

I polished my own little tool (edit: muppet gave me a much improved version, which is committed) for creating types in tinymail. I removed its requirement that your type names start with Tny. What it will do is outputting the .c file that implements all the GTypeInterfaces that you gave it on stdin, using stub implementations.

It detects to-implement methods, interface names, converts CamelCaseStyle to g_object_style / GObjectStyle / G_OBJECT_STYLE and it supports implementing multiple such interfaces. It expects the .h files of the GTypeInterfaces catted together to the stdin of the tool.

for example:

cd tinymail/trunk/tool
IFACES="../libtinymailui/tny-mime-part-view.h"
IFACES="$IFACES ../libtinymailui/tny-mime-part-saver.h"
./gtypeinterface-h-files-to-c-file.pl MyMimePartComponent $IFACES

Feel free to send me patches that fix the regular expressions that detect a method and a type in such a way that they would work with all possible GTypeInterfaces. This perl script at this moment only works with my personal style for GTypeInterface’s (tinymail’s style). I always add _func to each method’s function pointer in the interface’s struct (this isn’t like how Gtk+ developers do it) and my interface struct names end with Iface (this is like how Gtk+ developers do it).

It doesn’t yet differentiate between signals and methods (the virtual table). I might add something that reads the comments inside of the struct for detecting this soon (tinymail has, just like what gtk+ developers do, comments that indicate whether or not the function pointers are for signals or for methods). This is also a problem for the scanvirtuals.pl script of the Python bindings. Maybe we should, as a GObject writing community, define a standard /* comment */ for this?

My other idea for tinymail was to let the function pointer end on _method for methods and on _signal for signals.

The idea of the tool is to aid developers who will use tinymail for building an E-mail application with implementing the framework‘s interfaces. It being free software, however, means that you can use it freely for your own purposes.

A .h file generator for the GObject is of course very easy and something you can do with a little bit of search and replacing yourself. But feel free to send me a perl script that does that trivial thingy too. I’ll put it in the repository.

What is the difficulty here? Of course the simple fact that a message can contain a message and that any such recursive display must also display all pieces of the message that it itself is viewing. It’s indeed recursive. By now, all people who can’t grasp recursive loops should have decided to stop reading this. If not, you will suffer.

Let me start at the begin. A message is a mime part and a mime part can, but not must, contain one or more other mime parts. I defined a TnyMsgView, which is an interface that inherits from the TnyMimePartView interface. I created a bunch of implementations.

A TnyGtkMsgView which simply implements TnyMsgView and TnyMimePartView, a TnyGtkMsgWindow which decorates the TnyGtkMsgView (and also implements these interfaces, as that is the decorator design pattern), a TnyMozEmbedMsgView which inherits the TnyGtkMsgView.

I also implemented a bunch of mime part viewers that are not TnyMsgView implementations, just TnyMimePartView implementations: A TnyGtkTextMimePartView, a TnyMozEmbedHtmlMimePartView, A TnyGtkAttachmentMimePartView and soon we will also have a TnyICalMimePartView and others.

What will happen? Say you have a message with a text/plain body and a message/rfc822 mime part with a text/html mime part. This means that somebody clicked on a spam E-mail, pressed the forward button of his MUA, filled in your E-mail address and you received that E-mail. Not unusual, right?

I know you don’t (you don’t have to tell me Kris) but let us assume you want to see the HTML E-mail formatted using HTML. Yet you don’t want to double click on the attached E-mail, you want to view it inline in the message preview pane. All popular E-mail clients support this, right? The tinymail framework supports it out of the box too.

What will happen is that the developer using tinymail will have selected the constructor of TnyMozEmbedMsgView in the platform factory for creating a TnyMsgView instance. Maybe he created that implementation himself? Maybe he inherited from TnyGtkMsgView? Both are supported and possible, the factory returns the interface. We will assume he decided to use a standard component of tinymail.

When setting the message (the model) of this view, it will loop all mime parts. For each mime part in the message, the TnyMsgView implementation will have a method that creates a suitable TnyMimePartView instance. The HTML one overrides this method and creates one that can display HTML in case the mime part is a text/html one, else it calls the method on super. In case it’s an message/rfc822 mime part, the TnyMsgView implementation will simply create a new instance of its own type. It must also implement TnyMimePartView, remember? Therefore it can be used as a mime part view too. It becomes recursive without losing code readability.

To take it even further, the TnyGtkMsgWindow simply decorates any TnyMsgView implementation that eventually inherits from GtkWidget. It decorates this by wrapping it in a GtkWindow. Its implementation is extremely simple, yet that was all that was necessary to support opening messages in a window too. Transparent for the developer, because it too implements the same interfaces. Obviously shouldn’t this decorator really create a new instance of itself when a message/rfc822 mime part must be viewed. You can’t embed a GtkWindow in a GtkWindow (maybe you can, but we don’t have to do it this way). You simply keep decorating it. Also the method that creates a new instance of the self or this instance. That way it’ll return the decorated type, not the decorator type. Which is a GtkWindow.

From my web statistics I know a lot people are reading tinymail’s code using trac. For those people, if you want to learn the real purpose of all these funny design patterns and for-some-people not straight forward techniques, this is the code that you should read:

• TnyGtkMsgView is the base and default message viewer. It has to do classification in such a way that it doesn’t have to know about the for himself alien viewers. You will usually inherit this type. For each mime part it will create a TnyMimePartView using its create-mime-part-view-for method (which is overridden by types that inherit TnyGtkWindow). For message/rfc822 mime parts, the method will create a new instance of the same type as self using the create_new_inline_viewer method (which is also overridden by types that inherit TnyGtkWindow).
• TnyGtkWindow is a decorator for TnyGtkMsgView that wraps the widget in a GtkWindow (the screenshot demos this).
• TnyMozEmbedView overrides create-mime-part-view-for where it returns a TnyMozEmbedHtmlMimePartView. It’s of course just a sample HTML component. Multiple HTML components can be and will be supported by default in future. It also overrides the create_new_new_inline_viewer method so that it can return a HTML capable viewer for inlined messages too (the screenshot demos this).

From the Philip & Tinne studios, I give you … two video demos showing last week’s tinymail on the Nokia 770.

3..2..1 .. ja (which is dutch for, yes. Or, go or, well, whatever).

I will have video demos on GPE-capable devices like PDAs soon.

Last time I used some electro music, and got feature requests like “muting the sound of your E-mail client”, but remember that I promised to use Karoliina‘s music? Well, watch the new demos and listen! I had to make Tinne a little bit jealous for this (she plays the Clarinet, so she was like “hey, I can make music too! And this Karoliina-woman writes software, just like you, and and and music and .. pfff”.

You know, the typical female jealousy. In other words, cute :-). So I let her be the user on the video demo. After all, it’s her software too: I named it after her. Hence the Dutch pronunciation.

As you can see on this screenshot, tinymail‘s default ui components for viewing messages and for viewing mime parts are now aligned correctly for displaying message/rfc822 mime parts (like forwarded messages). They will now for example display attachments inside a forwarded message and they are hidden using a typical GtkExpander. Of course is all the behaviour of the default message view implemented for gtk+ inheritable and overridable.

ps. Yes, It’s still in 1974, I haven’t found the problem yet.

First, I would like to thank Sergio Villar Senin of Igalia for his recent contributions to the tinymail framework. It’s because people like you enjoy working on free software like tinymail, that we have a community and actual code that does something interesting.

His contributions implement various fixes and features including support for transferring messages between folders, subscribing and unsubscribing folders, support for adding queries to the current default folder view GtkTreeModel (for example letting it only show the subscribed folders, or only folders whose name and/or id match a regular expression) and a lot other things. He’s among the people who are making the ChangeLog look impressive.

This is of course a thank you to all the people who have been supporting me and who will support me in building tinymail. It grew from a little garage experiment, to what it is today. I promise that I will let it grow even more (in maturity, not in memory size).

It’s important to have contributors. Psychologically for me this means that people like my stuff so much that they want to put their own time in it. Thanks for your support and contributions Sergio.

Myself, I have been working on making it possible to display mssage/rfc822 mime parts (inlined messages like forwards). Well, it’s already working. I just need to fine tune the ui components a little bit now, and figure out why the second rfc822 mime part’s header is unusable. The interesting part is that the underlying tinymail infrastructure now supports, of course using existing Camel functionality, multipart mime parts like message/rfc822 and that the user interface components which ship with tinymail support viewing those correctly or will and/or can support it. In other words: that in design this is possible today. It’s just one of those milestones.

Hmm, I’m starting to wonder why that date is in 1974. Probably some converting problem. todo.prepend() :-\

Oh, Koen also made new testing packages for Ångström.

Screenshots of tinymail on devices

Koen created a screenshot of a tinymail demo ui showing his E-mails:

Inspired by Koen’s adventure, I decided to retry getting the demo ui working on the Nokia 770. My little brother has our camera at this moment, so it’s not a video demo but in stead this picture which I took with a crappy cellphone camera:

Debugging fun

Sometimes bug reports like this can still be useful. Just subtract the PC (program counter) value 0x40afdf54 with the start of the location of the library that is mmapped at that location, which in this case is libglib mmapped at 0x40b57000 until 0x40b58000. Now subtract the PC (0x40afdf54) with the begin of that mmap (0x40b57000) and you get 0x00024f54. SO I asked Koen to give me an objdump of his libglib-2.0.so.0.1200.3.so. We learn that at that location, you have the function “00024f54 [g_mapped_file_get_contents]:”.

I started looking at places where I called for the contents of a mapped file. It turned out that I didn’t check for errors after g_mapped_file_new. That was the location and reason of the crash. What the real bug is, we still have to figure out. It looks like his jffs2 filesystem sometimes didn’t create the summary.mmap file.

Feww, it wasn’t a ARM mmap data alignment problem. Those are hard to find. The kernel helps a little bit by trapping them. But still. Anyway, the first part of the fix is of course to check for errors after the g_mapped_file_new.

However! The real message is: why wasn’t Koen’s gdb working correctly on his device? :)

While doing design for tinymail, we create a simple wiki page, throw some sketches to it. Yes, hand-drawn & scanned in (doesn’t matter). Then I usually make a nice UML class diagram out of that. This time I used ArgoUML (None of the existing tools really work the way they should. I’m not saying ArgoUML is the good stuff, it’s just the least bad crap software that is available for this. — It’s still crap –). People like you guys get to make comments on it (hence me blogging about it). Finally, somebody (most likely me) will implement it.

This is the design page for “sending E-mails”. The tinymail framework is, indeed, going to aid the E-mail application developer with sending messages in an asynchronous way by implementing most of this in an inheritable, re-implementable, enhanceable and most importantly optional little inner framework or predefined method. You can still develop and design your own idea around this. Maybe in case your device or application needs a different way of sending E-mails that doesn’t use outbox, sentbox and draft folders? I don’t know. I can’t know. It’ll be both enhanceable and optional.

As usual it’s 2 ‘O clock at night and I’m still coding tinymail. I know this isn’t good. I’m fearing I’m a little bit addicted to coding. That’s nothing new of course. Maybe I shouldn’t blog at 2’O anymore, saves me a discussion about it with Tinne.

Anyway. I reimplemented the HTML messages support using Gecko in tinymail. It used to be quick ‘n dirty with temporary files and all that crap. Back then because I didn’t really have a lot time to figure out all the (mostly hidden) GtkMozEmbed API details. Nothing about time changed. It just finally got at the top of my todo items.

I replaced this with the nicer stream API of GtkMozEmbed. I also implemented not loading images and stuff like that. Of course you don’t want a HTML component to load images. Spammers would have an easy time figuring out whether or not your E-mail account is being read by somebody like you. Same for (Java) scripts.

Yet I’m interested in displaying images that tinymail has in its memory. Like as MIME part attached images. If the HTML components gives me a callback like: “draw the image that has this unique URL on this drawing area”, I could of course implement this. With GtkHTML I know something like this is possible. Tinymail is a framework that allows to implement multiple types of a (the functional same) mime part viewer. Note that a GtkHTML version of the TnyMsgView interface is implemented by Dirk-Jan and waiting for both cleaning up and licensing issues to go in the tinymail repository as LGPL. Webkit, GtkWebCore and Kit, WhateverKit they are all planned and possible, yes.

Again, the idea here is flexibility. If the vendor of the device already decided to use Gecko for all the HTML stuff, then what is little tinymail to contest that decision? The vendor wont, I repeat, he wont change that decision. That’s why these interfaces exists: you’ll use one of the default implementations, you’ll inherit and enhance one of the existing ones or you’ll implement it yourself. A good example: maybe the OLPC team wants to implement a mime part viewer as a web page or web page component? Sure! Possible. Because they don’t have to inherit from some Gtk+ type: they can implement from scratch without using any technology that they don’t need.

The TnyMozEmbedHtmlMimePartView is one such implementation of the TnyMimePartView interface. It also inherits GtkMozEmbed because, well, that’s what it is. It’s nature is indeed to be a GtkMozEmbed instance. Interface oriented development isn’t hard or more complex. In fact it makes things much more simple and straight forward. It avoids silly multiple inheritance.

Hmm, I should go to bed now.

For example as a google summer of code project, I think it would be cool to have a plugin for a few development environments like VS.NET, Eclipse, Anjuta, Emacs, GEdit and Vim to have code completion and LXR-like cross referencing that uses web services like google codesearch.

Like for example showing the implementation of a method when hovering over it (in the hint), or simple things like a popup-menu item when right-clicking on it that says: do a google codesearch on this function. Or a full pane in your development environment that in the background loads search results of the function near where your cursor is currently located. And many other such features.

I know this sounds like total overkill. Frankly, as a developer I really really don’t care how much bandwidth, memory and CPU my development environment uses. If that helps me develop applications better and faster, then yes I want it.

Privacy issues aside maybe a team could use statistical data from Google codesearch to get information about what developers have difficulties with, and based on that information develop interesting development tools or write documentation.

We lack a standard for delivering calendaring tools things like meeting requests. For example an E-mail client telling a calendaring tool to store a meeting request, and a calendaring tool telling an E-mail software to transfer meeting updates via a specific E-mail account to the attendees of the meeting.

I know a few projects implement calendaring for mobile devices. Like Dates, gpe-calendar and also a few Qtopia ones too (if you want me to be politically correct by mentioning the project names, then send me some). It would be a little bit silly for me to start creating patches for those implementations just so I can feed them with meeting requests.

A better solution would be a little D-BUS standard for it. An optional tinymail component could then implement supporting that standard for, for example, iCalendar attachments.

I personally wouldn’t really care if an application like Evolution would implement this. It’s not my goal at least. I do care, however, about Qtopia, GPE and Maemo endorsing such a standard for calendaring software. Maybe even get freedesktop.org onboard? Well, my personal goal really is mobile devices. But hey, it probably should have been a desktop standard a long time ago, right?

So the question: are the decision makers on those fronts interested in something like this? And if they are, where do we go from here? I mean, this standard can be typed in a few minutes. Implemented in a day for most calendaring tools and in something like 30 minutes for tinymail.

Let’s get it done?

Øystein Gisnås created Debian packages for tinymail. Again, it’s not a release nor is it a promise about the API, it’s just a preview. It’s also not an E-mail client. Tinymail is a framework, it’s not an E-mail client by itself. It has a demo written in C and one in Python.

I haven’t yet tested the packages myself and I strongly advise to build from source code or to rebuild the packages with what is current in the repository.

ps. For those who are now going to make blog templates of how I blog about tinymail: note that I have to keep repeating that tinymail is not an E-mail client because I simply keep getting E-mails from people who wanted me to create an E-mail client so much that they started believing that tinymail is that promised E-mail client. Tinymail is primarily focused at software developers who are going to create E-mail clients for mobile & embedded appliances. I repeat, I repeat, I repeat.

I’ve even read “it’s unusable because it doesn’t have a ui for configuring E-mail accounts”-like comments. Being a software developer who is going to use tinymail, that’s your task. Right? I fear that with the packages for GPE and Debian, that type of traffic is going to increase (more normal users will try it). That’s why I repeat, I repeat and I repeat.

Koen writes:

With some help from Philip I managed to get tinymail to build again in OE, and added it to the armv5te feed (more archs will follow later). If anyone feels an urge to write a mailreader, please consider using the tinymail framework :) A screenshot of the demo ui:

Thanks Koen. I will, using Bart’s hp2200, create a video demo soon. If people or companies want to get tinymail working on their device, contact me and try to get glib 2.8 on it (tinymail makes heavy use of gslice, which is only available since glib 2.8).

Note that it’s a good idea to try to make the packages yourself using what is current in Subversion. Tinymail is changing and improving a lot these days. A lot features and ideas are planned and being worked on at this very moment. Please look at the packages as a preview rather than the product (the framework) itself. It’s also not a release nor a API promise until I release something officially.

Packages and apt mirrors for the Nokia 770 will sooner or later also be available. About release dates: I’m still planning to in parallel with Modest do a 1.0 release. But feel free to take releasing over from me. If there is one thing I hate about doing free software development, it’s that I have to do releases and all the work involved with that. If I could, I would rather stick to pure development.

Non-coding

I bought myself Noam Chomsky‘s Failed States in Antwerp this weekend. Excellent book. Very interesting and something each U.S. citizen should read. I also bought a book about crimes against humanity by Jan Wouters and Bart Pattyn and one about the United Nations by Jan Wouters and Cedric Ryngaert. Poor Americans, I’m going to read about all the fucktup things their leaders are doing today. This is a good thing. People must know about this. And remember it. Americans (and Britons) should be deeply ashamed of their current politicians.

Coding/Tinymail

I made most of the methods in the tinymail framework virtual. This means that next to reimplementing a type using its interface, it’s now also easy to inherit from an existing one (and override methods and functionality).

Of course doesn’t GObject help a lot in making this easy. For example: you have to overwrite function pointers in the class initialization method and if you want to call the original one, you’ll need to store the function pointer yourself (for example in your own class structure).

A language bindings will of course make inheriting much more convenient in higher programming languages.

I also moved the functionality of removing messages to a strategy. I refactored this because what removing a message really means, might differ between devices, account types and E-mail clients. Removing a message on a local account will probably mean effectively removing it. Whereas removing a message from an IMAP account will probably mean marking the message as deleted (so that an expunge will effectively remove it) and removing the local cache of the message from the device’s storage.

A desktop developer who develops for users who have huge amounts of disk-space, doesn’t really have to care about this. Mobile & embedded developers, however, have different constraints.

The good thing is, is that tinymail is flexible for the developer. He can choose what tinymail will do without having to hack the framework. He just creates a remove strategy and attaches it to the folder. If he doesn’t do that, a default remove strategy is used.

For the same reason (disk-space and also bandwidth consumption) I’m planning to implement partial message retrieval. This means retrieving only the mime-part that most people call the body of the E-mail. Only when the attachments are needed will such an implementation get the remainder of the message from the remote service. Useful when your friend has just sent you a 5 MB attachment, and you want to view the body of the E-mail on your mobile phone that has only 2 MB of free storage space left.

There are a few other interesting features planned. But, I’m under NDA :). I can’t share everything. More announcements soon. Oh and sorry for not blogging as much as usual: I’m extremely busy at this moment. which is frustrating given that I have millions of ideas in my head.