Day: July 10, 2006

Hey, look at this cool thing I did, I sure hope the friendly Evolution maintainers commit it upstream so that we can all benefit in a happy orgy of peace and love. Note that without the mmap() patch, (my) Evolution consumes a lot more of course.

The patch, The mmap information from proc

cp -a $HOME/.evolution $HOME/evo.backup
cvs co evolution-data-server; cd evolution-data-server
cd camel; patch -p0 < patch ; cd ..
./autogen.sh --prefix=/opt/camel
make && sudo make install
export LD_LIBRARY_PATH=/opt/camel/lib

and then

/opt/tinymail/bin/tinymail

or

evolution --force-shutdown ; evolution

It will rewrite all “summary” files in your $HOME/.evolution/mail/type/$account directory. Don’t worry, if you go back to the old implementation, it will reverse that by doing the exact same thing. After that, go in offline mode. Shut down Evolution. Restart Evolution, for example using valgrind and massif. If things go terribly wrong, you still have the old files in “evo.backup” (right?). The patch only supports MBOX and IMAP. If you use another provider type, you will have to wait for support for that.

In online mode, Evolution will still (imho) consume to much memory. But I already have a very good idea where and how to fix that (imap_rescan for example).

Note that this is highly experimental. It will most likely crash. You will most likely have to launch Evolution in your debugger a few times, and fix things before you can get this working. Normal users don’t want this yet. I’m serious!

I created a new patch that fixes a few things (about the previous patch). The mailinglists are slow (the hackers and patches mailinglists of evolution are in CC) so I can’t yet point to the changelog E-mails. But I guess you can soon follow it up there.

Of course, what you guys care about is fancy graphs. The first shows tinymail opening and closing the ~650 headers folder a few times (this time without the leak). And the second is tinymail opening and closing the ~14550 headers folder.

While finding my leak I also learned that the original implementation of camel-folder-summary.c did an unnecessary strdup() and did some funny unneeded memory segmentation. I created this patch that fixes the unnecessary strdup. The memory segmentation fix is already committed upstream (and is for-free fixed by the mmap patch of course).

The strdup caused ~300kb unnecessary memory for the 14558 headers folder, and ~40kb for the 658 headers folder. In my opinion that’s a lot for memory that isn’t needed at all.

Everybody: please start using valgrinds massive tool aggressively on every change you make to your code. The difference between token = strdup(tokens[i]) and token = tokens[i] can be a lot memory! You can make little children, that will use such a OLPC device, happy with it.

If you want to make a real difference in Evolutions memory usage, check the imap_rescan method in camel-imap-folder.c. There’s huge room for improvement in memory usage in that method (or something that is related to “scanning messages”). Valgrind shows me up to 50MB of memory usage just for scanning for the new messages of ‘one’ folder (the ~14550 headers one).

Don’t think Evolution doesn’t need our community help. Let us fight this memory monster together shall we? Hows that for a positive message, jdub? ;-). There’s definitely very interesting components (like Camel) that aren’t very monolithic. You *can* make a difference, just checkout the code and start hacking on it.