Modest on Garage & tinymail

Tinymail

Today I did mainly build environment related things. Like making sure tinymail is translatable using gettext and getting the gtk-doc API reference documentation generated. I had to do some nasty stuff to get that working. But eventually I got it working. I’ve put a version online. That version is, of course, already outdated. Because any online version is going to be outdated the first millisecond after somebody uploaded it.

Garage & modest

It looks like the Nokia/Maemo folks have put Garage online. On Garage you can find for example modest, their E-mail client that is being build on top of the tinymail framework. It looks like you can also check out the source code.

NNTP support in tinymail

Two new screenshots demonstrate the possibility to use NNTP with Tinymail.

These two new screenshots demonstrate the possibility to use NNTP with tinymail. Note that this is highly experimental. Well, all of tinymail is of course experimental.

Realtime memory monitor crap tool

Last night I wrote about a little tool for real-time monitoring of the memory consumption of a piece of software.

You can find it here. If you untar that you’ll find a valgrind_massif_ms_main.diff which you’ll have to apply to valgrinds massif tool release 3.1.1 (it’s the latest version on their website). I didn’t cleanup the crap code. So the code is as ugly as can be.

Oh by the way. Using it to monitor evolutions memory consumption gives funny results.

Showing “massif.PID.ps”-like memory usage while running your software

I wanted to see how tinymail consumes memory in real time.

By that I mean, let my system give me (the developer) a graphical idea of how much memory the software is using if I perform a specific action. And to check whether or not I’m getting the memory back when I’m done with the action. However. Not after a full session by viewing a .ps file. This is what a standard valgrind/massif makes possible.

No, I wanted to ‘see’ the curve grow ‘while’ running my software.

For example when I open a folder, I expect memory usage to grow. If I now click on an empty folder, I expect the memory usage to drop (because the first folder is now being uncached).

So I hacked massif a little bit and wrote a little Gtk+ tool that reads the information from massif and plots it to a drawingarea. It’s not really in a usable state but I created a screencast to give you guys the basic idea. It’s a 3.3 MB GIF animation which will eat your memory if you use Firefox for viewing it. But it’ll work.

I don’t know when nor if I’ll ever get this funny thingy in a usable state. Note that you also need a few small changes to the massif ‘ms_main.c‘ file. And the tool is a quick-put-together glade-2 generated piece of crap code at this moment. Oh bleh, I’ll probably make it usable. It’s just a little bit late now. And I have to go to work/my customer tomorrow. I hack to much late at night :-). So I already told myself I’m not going to finish it tonight. Don’t ask!

Fwd: spamorama & tinymail class diagrams

Dirk-Jan of the Maemo team in Helsinki posted a new update on the status of “modest”, their attempt to create an E-mail client for small devices.

I’ll again translate the tinymail-related stuff from Dutch to English:

This week there was a lot management tasks to do. Therefore I had less time for modest, our experimental E-mail application. We did, however, had some progress. Following Philip is/has become interesting — we use his tinymail project –, the high frequency of changes demands constant refactoring at our side. Not a big problem — This is more or less how the/our daily progress goes:

HERE, you see, it takes all the running YOU can do, to keep in the same place. If you want to get somewhere else, you must run at least twice as fast as that! — de “Red Queen” in “Through the Looking Glass, and What Alice Found There”, Lewis Carrol, 1872.

But as said, this is good. We try to run as fast as possible, and we are getting somewhere. The screenshot shows a little bit of what we’ve done. This screenshot is, by the way, our throwaway prototype, which we use to build the rest of the application. The “real” GTK+-version will follow the HIG — so have no fear :-) –. Professionally is the Maemo-version of course even more interesting, but I definitely also want a good GTK+-version.

The screenshot isn’t using the MozEmbed-stuff Philip in his screenshot uses. Therefore is viewing spam also less interesting :-). Luckily is tinymail very flexible. For example if I want to start using GtkHTML, it’s very easy to add/replace that. I can simply create a TnyGtkHTMLStream class which implements the TnyStreamIface-interface. And done! Let’s do that!

Other than that I’ve been adding some UML class diagrams that illustrate the design of tinymail. Soon will the gtk-doc API documentation also be finished.

Tinymail can now view spam, err HTML

Yesterday I got complaints about class diagrams not having enough bling. And that MacSlow has cooler blogs than I have :-)! People want anti aliased bling screenshots. Right? Yes? Is that true? Well … spam has a lot bling, no?

Therefore I just implemented the primary feature of all E-mail clients in tinymail: Viewing spam.

I’m using Gecko/GtkMozEmbed at this moment. I’m going to have to beautify the code a little bit before I commit it. There’s also some issues with finalizing the GtkMozEmbed instance. It makes everything hang in some recursive loop. Grmbl.

Update: It’s hanging in PR_WaitCondVar, does somebody know how to fix that?! It happens after the exit(0), so it’s probably some C++ destructor of the firefox/gecko widget? Does it happen on your platform? I’m testing on Dapper. Let me know if you test tinymail.

Update: The implementation is committed in subversion. Have fun.

A problem, however, is that now tinymail can view spam, you guys probably want me to also implement deleting messages, right? Crap.

Extreme rules

There’s (might be) some people who are interested in tinymail development. I just posted a plan-of-action. Please read if you are planning to (someday) join the software development iterations of tinymail. Note that this doesn’t mean that tomorrow a huge team of developers will be working on tinymail. But it’s my idea of how to be as Agile as possible during the tinymail development lifecycle.

I’m as unwise as Socrates (probably) thought he was. By that I mean that I might be wise only in so far as I know that I know nothing. Therefore I’m very interested in suggestions.

I’ve been busy last night

If you request an E-mail, Evolution will block the worker thread for getting your E-mail until the last folder you clicked is loaded. Try it — I’m not very sure what evolution will and what it wont do. Tinymail, however, will now, if you click an e-mail while you are loading a folder, restore the original folder and show the E-mail immediately. After it canceled the folder-loading request. It assumes the user wants to stay in the current folder if she’s going to read a message in that folder.

It has the same behaviour for switching a folder. So when you are loading a folder and you click a new folder, the request for loading the folder is canceled and the new request will happen instead. These two camel cancellations make tinymail very responsiveness.

I also removed the need for gdk_threads_enter in the status notification handler by throwing it to the GMainLoop. So the tinymail application developer does not have to care about threading issues in gdk/gtk+. The tinymail library takes care of that.

Update: About behdads announcement: I just added some tinymail proposals to the SoC ideas wiki page and applied for SoC mentor. I don’t know if it’s a good idea but if students are interested, feel free to propose it. Also note that I haven’t yet been accepted as mentor. Update-update: behdad accepted me as a SoC mentor.

Tinymail announcements

In this blog entry — which I’ll now translate –, Dirk basically tells us he hasn’t got to coding yet this week. Others, however, did. He says it’s nice that there’s progression being made on tinymail. He says that they are using tinymail for an experimental open-source e-mail application for the Maemo-platform (and for “normal” GTK+). Called “modest”.

Jesse is also asking for a snapshot release which the Fedora team might soon package in Fedora extras.

Tinymail: asynchronous folder synchronization with the service implemented

I know I often say I try to avoid threads in tinymail. This one is inevitable: Tinymail is going to support two methods for getting the headers.

The first method has been around for a while now: normal synchronous getting all the headers in a folder. This will return you a bunch of small proxy instances that can be used in for example the custom tree model. You have the option to synchronize or not to synchronize with the service. Synchronizing of course takes a noticeable amount of time. And unless you do something about it, your user interface will block during that time.

The second method is new. The second method or the asynchronous method will launch a small thread that will, in the background, synchronize your folder instance with the server. It will callback on the GMainLoop and it will tell you about the status (status notification is in-thread, the callback-on-finished is thrown to the GMainLoop). You are supposed to reassign the model of your treeview when the on-finished callback invokes. During that reassignment you’ll ask for a new list of proxy instances using the normal synchronous method. This time you (of course) no longer have to choose the synchronization option. So it will be fast.

I also made sure camel cancellation is integrated. What is that? Well, say the user clicks per accident on a folder. He notices it’s the wrong folder and rapidly selects a new folder. Does the user want to wait for the previous folder to be loaded? Probably not! Tinymail will automatically (when using the asynchronous method) cancel the request and will fulfill the new request instead.

The result is a very snappy tinymail with a non-blocked ui. Even if you never synchronized a specific folder with the service, the ui will not block. I didn’t yet implement a progressbar for the demo-ui. But using the asynchronous method this is very easy to implement. For status notifications, you’ll have to do the gdk thread locking because at this moment is the status notification in-thread and not thrown to the GMainLoop like what I did for the on-finished callback. I’m not sure if it would be a good idea to throw it to the GMainLoop. Maybe I will.

So.. I might refactor things a little bit. And as usual is this stuff experimental. No API promises.

You can of course svn co https://svn.cronos.be/svn/tinymail/ and try this new stuff out yourself. The default compilation flags are set in such a way that libtinymail is low on memory usage (a little bit slower) while the demo-ui will use the asynchronous method for getting its header information.

A libtinymail developer will not have to worry about all this. He’ll just have to code a few simple lines: take a look at the ASYNC_HEADERS ifdef. The ifdefs also show how small the difference between using simple synchronous and asynchronous can be.

Tinymail shows my large IMAP account using less than 3 MB ram

Look, the peak is gone and I’ve cut another two megabytes of ram memory from tinymail. I also improved the load speed (a lot) and the click-speed (snappy feeling). Now Tinymail even works fast when running it under the massif valgrind tool.

Tinymail in kb

Please check it out and try to find problems and issues. Maybe even help me a little bit. I’ve recently posted a todo list (some of the items are already solved) to the mailing list. I’m also looking at implementing a libtinymail-spruce. Nevertheless, this measurement proves that camel can do it. But I still think camel could use some API improvements like simplifying things. Which is why I’m looking at libspruce.

Drop caches

The newer Linux kernel releases will have a feature to drop caches in a clean way. Leio blogged about this. Check it out of you are into testing performance.

Clipboard copying the text/html target using gedit

I very often write my blog entries and E-mails using gedit. For a lot such posts and E-mails I use HTML. So I started thinking, maybe I should write a gedit plugin that will allow me to copy the HTML source text in gedit and paste it as a text/html target?!

So I did that. (you can follow-up using this bug)

As on IRC Paolo was quite interested, I guess Paolo and/or Steve are going to clean it up and add it to the available gedit plugins soon?

For now I didn’t add pasting HTML source code from a “text/html” TARGET capable selection owner (like OpenOffice.Org, Evolution or Firefox) to gedit. Shouldn’t be very hard to add that. Be ahead of me if you really need that.

Update and FYI: Paolo redid the plugin in Python because such a Python plugin does not need a compilation procedure.

Updates on the IList databinding for Gtk#

About the Adapter thingy of Friday: Matooo added support to the Gtk.ComboBox and the Gtk.ComboBoxEntry and fixed some annoyances. He also updated the demo so that it uses a multi-column Gtk.ComboBox, Gtk.ComboBoxEntry and a Gtk.TreeView.

The proof of concept is still an experiment so don’t count on anything becoming stable. Mike Kestner (one of the gtk-sharp developers) commented on the experiment here.

If you want to join experimenting, just ask me for a subversion account. I already gave Matooo one. So :-) update your checkout frequently.

The adapter design pattern

Somebody asked for it and since it might be interesting, I’ll blog it.

This is the UML Class diagram of the IList adapter POC which I mentioned yesterday and which you can use for databinding with the Gtk.TreeView in a typical .NET fashion (so using your own IList as true datastore or list model).

As you can see I used a variation of the adapter design pattern.

ps. I typically use KDE’s Umbrello for making UML Class diagram because well .. Dia is very limited for UML, ArgoUML and Poseidon are slow and Gaphor isn’t finished. Actually is Umbrello also very buggy and it doesn’t write valid XMI files (but no free software tools do, perhaps maybe ArgoUML). Perhaps I should try ArgoUML one more time?

DataBinding an IList as DataSource for the Gtk.TreeView

I did it again. I got arrogant while being excited about how we should make it more easy for language bindings to implement types that adapt the many C-isms we introduced in GObject and Gtk+. And as a result I got myself in the situation of having to prove the concept. Which most of the times isn’t a big problem. Often it’s just some work.

So by telling me that if I’d look at the NodeStore and NodeView implementations in gtk-sharp by Mike Kestner I could implement my ideas, toshok invited me to create such a proof of concept. I looked at it and decided that he was right about the fact that it’s possible. So I decided to do it.

The problem with the ListStore and TreeStore models is that in a correct Model View Controller paradigm, you don’t couple the model and the view. You decouple everything that is related to viewing the model, from the model. You put it in the view. That’s the purpose of the view! When implementing a GtkTreeModelIface you have to implement the typing of the columns in the model. “Columns”, is information about viewing the model. Fortunately for sanity but unfortunately for gtk-sharp this isn’t how it’s typically done in a programming environment like .NET.

In .NET (in this example System.Windows.Forms) you can do for example (say you have a Person object with a Name property):

DataGrid view = New DataGrid (); // Or use the .NET 2.0 DataGridView
DataGridTableStyle table = new DataGridTableStyle();
table.MappingName = "Person[]";
DataGridColumn column = new DataGridTextBoxColumn ();
column.MappingName = "Name";
table.DataGridColumns.Add (column);
view.DataGridTableStyles.Add (table);

Person [] persons = factory.GetPersons ();
view.DataSource = persons;

Types like the ComboBox also work like this. If you ask for the combo.SelectedItem, you get a Person instance. For the Gtk.TreeView the story is different: You can’t use a simple IList like the DataTable or collection types immediately. You have to flow them into a ListStore or a TreeStore first. And on top of that, you have to put column information in the model. Many OO designers will tell you that is wrong, others will tell you that now it no longer matters that the model and the view are two things. They are coupled for ever. So why do it in the first place? I’ve seen wrapper API’s that add a AddLine to the view. This is of course totally not the Model View Controller paradigm anymore.For a typical .NET developer, this is at least strange. It makes the model not reusable. So you end up creating user interface types and code that shouldn’t be there. And the Model View Controller paradigm eventually also gets lost. As the model is coupled with the view (it’s not reusable).

So basically: it’s not really how to do it correctly.

What you want is to set a datasource (the list model), and adjust the view so that it understands what to do with the datasource. But don’t require view related changes to the datasource (the list model). So I implemented a GtkTreeModelIface in C# and did it as an adaptor that will convert the C-isms to .NET.

The implementation will basically implement GtkTreeModelIface in C# (which was the hardest part). In stead of keeping my own store (for example by copying the references to an ArrayList), I inject the original IList into the custom tree model implementation and use that as store. Once done I simply use the integer index as TreeIter token and check for the Count property of the IList interface for letting the view know that it can ask for more rows/instances. That way it effectively adapts the IList to a TreeModel.

It’s an ugly ui application but read the code and run it while reading: you’ll understand what it does. You can get a checkout using this SVN repository. It should load in for example MonoDevelop.

So if I translate the above sample to this, you get something like this:

Person [] persons = factory.GetPersons ();
TreeViewAdaptor view = new TreeViewAdaptor ();
view.AppendColumn ("Full name", "Name");
view.DataSource = persons;

So when the GtkTreeView (the C thing) needs data from the model, it will ask the C# GtkTreeModelIface implementation for that data. And that implementation will use your Person instances and your IList instance. Not a copy. As a proof, feel free to put breakpoints at the getters and the setters in the Name, Surname and Number properties of that Person type. Or implement your own IList in stead of using the ArrayList (the demo is using an ArrayList) and follow when it’s asking for items by adding breakpoints in your IList implementation.

Know what? Simply check out the source of the demo. You’ll see.

A D compiler in a Fedora Core 4 package

I’ve been editing this gdc.spec in such a way that the resulting D compiler works correctly with your existing FC4 gcc installation. As the gdc patches don’t yet work on the gcc version of FC5, I didn’t repeat it for FC5 yet.

So I announce the direct availability of a gdc package for Fedora Core 4 and the stuff for building it (like the source RPM) yourself. Please do not consider this package as an official package. It’s not, I repeat, it’s not an official package. And I’ll repeat it again: it’s not an official package.

Please join the D.gnu newsgroup if you are interested in building packages for other distributions.

[root@zeus ~]# rpm -e gdc
[root@zeus ~]# gdc
-bash: /usr/bin/gdc: No such file or directory
[root@zeus ~]# rpm -Uvh /usr/src/redhat/RPMS/i386/gdc-0.17-0.i386.rpm
Preparing… ########################################### [100%]
1:gdc ########################################### [100%]
[root@zeus ~]# gdc test.d
[root@zeus ~]# ./a.out
Yes
[root@zeus ~]# cat test.d
import std.string;
extern (C) int strcmp(char* string1, char* string2);
int myDfunction(char[] s) {
return strcmp(std.string.toStringz(s), “foo”);
}
int main () {
if (myDfunction (“foo”) == 0)
printf (“Yes\n”);
else printf (“No\n”);
return 1;
}
[root@zeus ~]# cat /etc/issue | head -n 1
Fedora Core release 4 (Stentz)
[root@zeus ~]#

Getting myself a D compiler

I finally compiled myself gdc, a d frontend for gcc. Surprisingly it was very simple.

I basically had to unpack the gcc tarball, cd into gcc-4.0.3/gcc/ and unpack the gdc tarball which created a d sub directory. Then cd .. and run ./gcc/d/setup-gcc.sh which basically patched the gcc sources which added the d language.

Then I had to make a build directory, I decided to create /opt/gdc and then cd /opt/gdc. And then simply run (from that directory) /path/gcc-4.0.3/configure –prefix=/opt/gdc –enabled-languages=d,c,c++ followed by make and make install. Easy!

I’m planning to experiment with the d programming language and I will probably do all my concept-projects using this programming language. I’m for example planning to implement a reference implementation of deconf-spec in d. But I guess it depends on the results of the experiments I’m going to do.

User software configuration, why doing it?

Surprisingly some people where silently interested in the standard about configuration of user software. So I added some chapters and restructured the document a little bit. Just in case.

There’s still a lot specification-work to do. Most people probably can’t imagine why. Others simply start implementing. I don’t think the approach will work for this. But anyway, if people want to do that they should.

In my humble opinion can’t the configuration problem be solved by just hacking together an implementation. In fact aren’t the proposed solutions solving any real problems. GConf and KConfig solved those problems already. Unifying them is a.t.m. pointless. And actually as the proposed solutions introduce yet another ad-hoc standard, they in stead add to the problem.

More about this and also read this.

ps. I might soon also post some UML class diagrams of the current idea I have for the reference implementation of the service. I also have some working experiments. Vaporware–, it’s more about popping todo items.

Just another day in config paradise

Project deconf isn’t per definition implementing a replacement for existing systems like KConfig nor GConf. In fact do these systems work and don’t need a replacement.

Project deconf is about first specifying and then building infrastructure for futuristic computing environments. I foresee that most of the really used Information Technology will become embedded in specialized devices.

Take for example the iPod, cellphones, VoIP phones, personal data assistants, navigation assistance, digital television, printers that don’t need a desktop and soon instance messaging and video conferencing hardware. I have to conclude: Most people hate desktop computing. A lot people love the technology that it created. It’s not surprising Steve Jobs is again pioneering it with the iPod.

The focus of deconf (which will probably be renamed a few more times) is specifying and then building infrastructure that will survive and be useful in such a future computing environment. So this basically means that there’s a focus on distributed configuration management, on keeping the requirements small and most importantly, making every piece of technology replaceable.

Anyway. As a lot people hate it when you talk about configuration management and as I don’t really care about those people (and I certainly don’t want to spark yet another flamewar about it). Here’s what I really wanted to blog about.

By the way, I know I should cut my introductions.

As David was so smart to join a discussion about configuration management, I decided to for one evening put aside my work on tinymail and improve that funny specification of mine.

I actually even renamed it from A standard for desktop application configuration to A standard for user software configuration. It’s a much better description of what the specification is about. It’s about user software. Not just desktop applications.

I’m done, you guys can now continue calling things vaporware, flaming about configuration management and blocking innovation happening from the free software camps. So that soon, when Apple or Microsoft decide to do implement it, you can all start whining about how they should make that open. So that we can catch up, again.