Month: March 2006

Oh by the way. I forgot to mention it! Last week I managed to send the very first correct E-mail using libtinymail. Remember I told that the first person to receive an E-mail sent using tinymail would be my girlfriend Tinne? Well, I made a screenshot:

Sure there’s still a lot todo, but libtinymail is getting ready. It’s not yet doing multi-part messages and I haven’t yet added a “create a message ui” in the demo ui of tinymail. I’ll do that sooner or later.

You can learn how to send e-mails using libtinymail by looking at the bottom-part of the tny-test-anything.c example. Please note that a lot API details will probably change in future. Tinymail isn’t ready yet. If you’re interested in tinymail, you’re invited to the devel mailing list. I’ll do development related announcements on it. You can use the mailing list to discuss future and enhancements of tinymail.

Walking a little bit in that secondlife world. They released a Linux client.

I wonder if free software developers are now going to join the fun of extending that world using their programming know-how and free software attitude. You can create and develop some nice things.

If you’re interested, I named my character “Philip Potato”. You can find some information about the open source community at second life here.

A few days ago, vuntz asked us to start porting the GNOME applications to the GOption API.

I didn’t know such an API exists (so thanks vuntz) but, now I know I decided to teach it myself by helping out.

So I just finished a patch for nautilus and did one for Evolution yesterday.

There’s still a lot applications to patch guys. It’s not very difficult. So join the fun.

By the way, Nicolas wrote a nice tutorial about GOption commandline parsing.

Aha! The sysadmins of GNOME have the tinymail-devel mailing list up and running. If you are interested in tinymail development, you can now join the mailing list. For some reason I didn’t receive the administrator password for the list, but the list itself works. Thanks for creating the list guys.

I also started really preparing and implementing the infrastructure for sending e-mails. The CAMEL_PROVIDER_TRANSPORT type is handled different from the CAMEL_PROVIDER_STORE type, so at this moment is the account stuff in tinymail a little bit strange. I will have to refactor it a little bit. The interfaces are, however, already more or less correct. In this little test I implemented a small test procedure for sending an E-mail using libtinymail.

Anyway, now that there’s a bugzilla product and a mailing list, I guess you guys can start overflowing me with patches? No? I Wish that was reality. As all authors or free software, I guess.

Hey Chris, just wanted to say .. welcome on p.g.o and thanks a lot for your work on Dates. Let’s build solutions for mobile devices (and perhaps desktop users) that just work. I’m certain your lightweight calendaring tool for many people will.

I did some more work on tinymail this evening: I’ve been putting the infrastructure for sending mails into place. I made sure creating a message header, a mime part and a message is now possible. I refactored the account-type a little bit (there’s now a store-account and a transport-account). And I created a few unit tests that test the stream infrastructure (as this has to work if you want to create mime parts).

I’ve also been working on finishing the support for POP mailboxes. I adjusted the header type so that it supports folder types with support for summaries and without such support. I added a special case in the (now new) store-account camel implementation, as in camel POP accounts don’t handle folders like IMAP accounts (there’s only one real remote folder, the INBOX folder).

Hmm, if all the crap I just mentioned is working correctly, I might release some first version of the framework. Until then, don’t expect any API stability. Even after such a release it’s going to be likely that the API will change a lot.

Anyway, there’s some people (on IRC) who’ve been asking me questions like: why are you doing this?!.

The answer isn’t “to build a replacement for Evolution”. Nor would making it use libtinymail make the current implementation faster. Probably a little bit slower. The simple reality is that libtinymail would reduce the amount of features to an absolute minimum. It would improve memory usage. Probably a lot. But that isn’t an interesting conclusion as the evil eye of tinymail isn’t looking at Evolution. It sure looks like to much people are hoping it would start doing that. It wont. (It can’t, Evolution is multiple years of work for a single person)

That eye is looking only at the many mobile devices that don’t have a nice E-mail client at this moment. It’s not strictly focused at gtk+-capable such devices. It’s also not said that it’ll be camel-only. However, at this moment there’s no reason to switch that technology. I’m “using” camel because it supports many providers and because it has relatively small memory requirements.

Anyway, so why am I doing it personally? Because I started enjoying working on it? Because it’s a challenge? I can give you many reasons. Above all, I think .. reasons like “I want to show people how to do it”, “prove of concept”, “learning”, etcetera.

I’ve also been asked whether I’m planning to support message threading. I’m not planning it at this moment. Feel free to send me some analysis on whether the algorithm can be used with my “only load the header-info of visible rows” ideas. Probably not. On a mobile device, loading header-info of invisible rows isn’t a good idea when displaying large folders. I’ve been thinking about sorting when messages get received and storing the tree-index. Rather than sorting each time a folder is to be displayed. That way you can let the sorting (which basically means reading all header-info) happen on idle moments. Nevertheless would that consume device battery. So perhaps do it while the device is charging?

Yesterday evening I added support for POP boxes to tinymail. It’s a little bit unstable at this moment as the CamelPOP3Folder does not support summaries and the way tinymail handles IMAP was using this summary support in Camel. So some things had to change and I’ll have to do some more testing.

During the last few days I also added support for creating messages and mime parts in tinymail. I already promised my girlfriend Tinne that she will receive the first message sent by tinymail. But first things first: I need to implement some infrastructure for sending E-mails.

I also asked the GNOME administrators to add a tinymail product in bugzilla. And to create a tinymail-devel mailing list. But that mailing list ain’t created yet. Anyway, there’s some interesting people interested in joining tinymail development. That’s all I’m saying about it.

Probably because I’m using camel differently than Evolution, I’m finding some funny camel bugs. I’ve also started studying camel in depth as there’s some things about camel you really need to know that aren’t documented.

And cause of that, as usual, I want to start changing everything. Like the usage of threads in camel. I have this funny idea that a library shouldn’t try to implement much threading details by itself. In stead, camel implements a non portable wrapper and uses stuff like pthread_cancel. Therefore I hereby declare tor a hero for making that actually work on Win32.

I would someday like to replace that EThread stuff in Camel with something like asyncworker as as far as I can see, it’s basically the same as the E_THREAD_QUEUE type being used in CamelSession. The difference is that asyncworker uses the abstract and portable GThread type and has support for canceling at by-the-programmer defined cancellation points without using pthread_cancel.

The main problem with pthread_cancel is that it can cancel on various POSIX functions. Somebody on IRC told me it will only cancel on specifically pthread_trycancel if you have some default switch turned on, but the man page on pthread cancellation isn’t telling me that.

Anyway … I basically avoid using pthread_cancel. Asyncworker doesn’t use it. Search for the protected _o_async_worker_try_cancel method to learn how I implemented it (it’s using the gmainloop to recover from a cancellation).

Mikhail Zabaluev did some great work on codegen. He basically rewrote all the XSL templates and modified the internal XML format generator in such a way that codegen allows for nice XSL templates.

Codegen is a LGPL code generation framework that will generate boilerplate code from simple class diagrams using or XSL templates or other generator implementations.

At this moment will codegen generate C#, VB.NET, Ruby, PHP5, Java and GObject classes and interfaces. The GObject support is not finished, but we are working on this.

For example: Codegen will, using XSL templates, build a complete working Visual Studio .NET solution from your UML class diagram. Codegen supports both XMI and the uncompressed DIA format.

Mikhail updated the GObject template a little bit. He added some extras that will make it possible to fully correctly generate both GTypeInterface interfaces and GObject classes. I’m also planning to support generating configure.ac and Makefile.am files.

Some (but not all) features:

• Codegen supports fixing the UML types to the programming language specific types using so called “fixers” (A sample fixer for .NET). Such fixers can also fix other target specifics like inheritance and implementation details.
• Codegen supports grouping your types into namespaces and assemblies, libraries, “make targets” or however you call it in your favorite programming environment. Codegen uses the UML package tool for this.
• Codegen is rather a framework for code generation than a standalone application. Codegen is meant for integration with your own tools and/or environments. It has been designed for this. So codegen does not (yet) include fancy user interfaces or cool UML diagram editors. Mainly because that isn’t the scope of codegen. It can be used as plug in component for your own development environment.
• I’m at this moment very open to huge design changes, ideas and/or decisions for codegen. This can also be viewed as a feature if you are actually planning to start using codegen in some cool way.

Codegens scope isn’t to generate implementation code. You, as a programmer, should do that. But you can very easily modify the XSL templates and put your business implementation rules in it. You can, for example, let it use a in-house developed library or framework. That framework isn’t going to be generated by codegen. You’ll have to build that, if you want to use it.

I recommend existing frameworks like Spring and Hibernate for stuff like this. There are also .NET versions of these frameworks.

I don’t do frequent releases for codegen. So you’ll need to get a checkout from the Subversion repository. If you are interested in helping me do releases or just want to help coding codegen, contact me.

Abstractionitis. Yeh. But being flexible by design is also important. We all know all software projects have one thing in common: CHANGE.

So we have this situation where some people like the idea of having a d-bus service that basically does (as a service) what camel (and libtinymail) does: give you a simple-to-use interface to get information about your mailboxes.

Now if I wouldn’t have used interfaces for all types, this would have been more difficult. It would have meant a major redesign.

However, because I did use interfaces for all my types it’s simply creating remote proxy types by implementing the same interfaces using some d-bus code. And afterwards creating a d-bus service that uses the existing implementations as the real type in the remote proxy paradigm.

Now can a client that only depends on the interfaces use both the remote proxy- or the real types directly.

So the design is flexible for a change like this. I don’t yet have to worry about the idea of service oriented e-mail querying becoming popular. Nor about the idea of a service becoming less popular. The design is flexible, it could do it. And I wouldn’t have to change any of the client components that depend on the interfaces:

I’d implement intelligent caching in the client proxy types if that would be important. For example: the “from”, “to” and “subject” properties of the proxy instances that represent the visible rows in the gtktreeview are cached in the client proxy type. A simple and clean solution.
It’s is one of the advantages of working interface oriented: adding layers becomes more easy. Using the (remote) proxy pattern becomes possible.

So yeh, abstractionitis. I might be overabstracting things. But so far it hasn’t made me slower nor has it made the application slower or start using more memory.

Yet it did make the design adaptive to changes. For me, that is more important than getting something done quickly.

ps. Maemo is interested in writing a ui on top of libtinymail*. I’ve been on the phone with their product manager.

Some gnomies assured me I should keep talking about my programming experiences, as that is exactly what we do in the community of free software developers.

If I ever was abrasive, my apologizes for that. Sure I’m proud of my achievements. But I really like the quote “If I have seen further it is by standing on the shoulders of Giants”, for all of my achievements.

Anyway … I created a new page that explains about some of the memory-saving techniques being used in tinymail. So I’m posting it in the hopes others will learn from it. Which has always been my main reason for blogging about tinymail a lot. If only one person learns from it, it was worth the effort. On IRC, people tell me they are reading and learning from it. That is why. And yes, that (also) makes me proud.

This evening, Raphael created a initial python binding for a libtinymail and libtinymail-camel.

I haven’t tested whether it works nor did Raphael. So feel free to enhance and/or make it work.

Well, take a look at this case. I created a unit test that tests whether I can set the properties on a message header type implementation.

The test tests setting properties on the proxy instance like the BCC, CC, FROM, TO and finally the SUBJECT field. All of those properties except one worked: the subject field crashed the unit test.

I checked what the tests does and noticed there was one difference with the other properties: all the other fields forced me to create at least one CamelInternetAddress object before creating the CamelMimeMessage instance. Different from the TO, CC, BCC and FROM fields didn’t the SUBJECT test force me to create such a CamelInternetAddress instance first for setting the SUBJECT on the CamelMimeMessage instance.

It turns out camel_mime_message_new doesn’t work when you haven’t yet created another Camel object first. So basically, this can mean that camel_mime_message_new doesn’t initialize Camel like other CamelObjects do. Or something like that (I don’t know). The camel_type_init didn’t make the problem disappear. It does change the behavior of the crash.

A typical Evolution debugging session wouldn’t have showed this problem as Evolution probably initializes Camel very early. Only unit testing kills bugs like this before shipping the library and before declaring it stable. This is why the GNOME development community should use unit testing more often. We probably are shipping a lot such bugs in our core libraries.

It would be very doable to create a bunch of unit tests for camel. Just like how it would be very doable to create a bunch of unit tests for gtk+ and/or any other popular and important core GNOME library. Such unit tests could very easily be build to detect regression bugs early. Why aren’t we doing this? Why are we only using our girlfriends as our testers? This way, we aren’t going to convince ISVs about using GNOME as their target platform for customer software development.

Here’s my proposal for making Jeffs 10×10 concept succeed: let’s stop behaving like Ren & Stimpy hackers and start thinking like programmers.

Oh, before people get grumpy: there are a lot gnome-related companies who do unit testing already. And I’m not saying unit testing detects all bugs nor am I saying camel sucks. All human creations have bugs. It’s how you deal with it that makes the difference.

Tinymail has integrated unit testing. I used the GUnit framework. I have to add more tests but a first test-suite for the message header type has already been added. There’s support for the Hildon framework in GUnit, so I guess this is good news for the Maemo guys. Some API docs about GUnit can be found here.

I also started working on making it possible to create messages with tinymail. Once this is done, I’ll add a transport that supports SMTP. Soon we’ll be sending our first simple messages using tinymail.

And once that is possible, I might start thinking about preparing a first release. After that first release I’m planning to introduce support for a few HTML widgets. If you want to be ahead of me and do this yourself already, take a look at the text buffer stream-type in libtinymail-gtk. You will basically have to create a stream for your HTML widget (MIME parts get streamed in the widgets). A difficulty will be displaying image mime parts if they are referenced by an IMG HTML tag in the HTML MIME part. I don’t know whether all HTML widgets support doing that without writing the parts to a /tmp or something (which might not be a good idea on a mobile device).

Tinymail will need to support multiple possible HTML widgets because mobile devices often come with specialized HTML widgets. So if you want me to accept such a patch, don’t even think about making a coupling mistake. You basically have to implement this interface and put all the device-specific code in a new target. I’m thinking about creating an interface for a mime-viewer so that I can decouple the msgview-type from the display-implementation (for example a specific HTML widget). Feel free to join me on damaging our brains on this one. Note that quick’n dirty or hackerty hack ideas are not what I’m interested in. Tinymail is (going to be) 3000% correct and clean.

Some people asked me to setup a mailing list. If somebody is interested in doing that, tell me. I really dislike this sort of “project management” tasks. Me personally, I’d like to focus on coding. Really. I do, however, accept patches and help from others. Oh, only a -devel mailing list would do. I’m not yet interested in what users think of tinymail. Tinymail is pure development at this moment.