I just finished the Evolution DBus metadata API‘s implementation. Information about this work can be found on this wiki page.
It’s currently not shipped as part of Evolution. Instead it’s done as an EPlugin compiled and installed by Tracker if your Evolution’s development package is 2.25.5 or later. At this moment that’s the versions in Evolution‘s and Evolution Data Server‘s Subversion trunks.
This API enables application developers to get notified not just about new E-mails, but also about any state changes of any E-mail Evolution handles. For example whenever the state of an E-mail changes from Unseen to Seen. You are invited to check out this Vala example to learn how to use the DBus API yourself.
Not only does the API give you “I’ll call you” or “pushed” hints about these state changes, it also gives you a lot of meta information about each E-mail: subject, sent, from, to, cached MIME filename, cc, service’s uid, seen, junk status, answered status, flagged status, forwarded status, deleted status, size, tags, keywords and flags.
The API will also inform you about message arrivals and about message expunges. Which are not the same as flagged for deletion.
With this information you have enough to build a simple E-mail client outside of Evolution that uses just this API to get itself informed about Evolution’s E-mails. That’s of course not the purpose of it, but it illustrates the completeness.
The purpose is to let a metadata engine like Tracker and Beagle get themselves fully informed and aware of all the state and metadata of your E-mails in real time. No more scanning of your $HOME/.evolution/mail directory. Which is by the way a for Evolution internal cache format that will change and has changed in the past. Scanning it yourself is for that reason too unreliable to depend on. This EPlugin uses Evolution’s own APIs, and runs in Evolution’s processes, to get the same information out and then pass it to one of Tracker’s processes over IPC.
“IPC??”, I hear you thinking, “isn’t that slow??”. Well we measured it. If a quite slow IPC like DBus can throw all of the metadata of 10,000 E-mails over in less time than Evolution needs to start up, knowing that we throw it over asynchronously with the user interface’s thread of Evolution, they I don’t think you should be too worried about that. But just in case you would still be worried have I specified the protocol in such a way that only the delta of changes since the last time Tracker registered itself are sent. While Evolution and Tracker run, Evolution will communicate new info with Tracker: Evolution pushes stuff into Tracker instead of Tracker pulling things out of Evolution.
What are you waiting for? Start adapting your $HOME/.evolution/mail scanning softwares! Now!
Nice. I’ve wondered. Does tracker export a nice api that could maybe be use to make an Evolution plugin that replaces the search box in Evolution. Maybe even an api that could be used for search-as-you-type? (maybe such a plugin already exists?)
Thanks for rocking :o)
Anders
@Anders: Tracker does, yes.
That would actually make a lot of sense. Tracker would be able to provide very fast full text search over all the content, as well, as the search over just few fields of the content.
http://svn.gnome.org/viewvc/tracker/trunk/data/dbus/tracker-search.xml?view=markup
method Query is probably what you are looking for. Service would be Mail or EvolutionMail (would need to check), and you could either crarft a query with the query condition (if you only search certain fields), or just use the search_text if you want full text search over all mail fields and content.
Anyway, things will improve dramatically from 0.7 onwards.
oh look, another exercise in reinventing the wheel. there is already akonadi, but when was the last time an existing technology that works ever stood in the way of a F/OSS project reimplementing everything from scratch (often making the same mistakes all over again)?
i know in our project we’ve worked damn hard to get rid of this counterproductive pattern of behviour, and it’s paid off rather nicely.
now if we could get a decent % of the rest of the F/OSS desktop world thinking similarly, we might all just have a chance at being mildly efficient and delivering user experiences that Don’t Suck(tm).
yes, i’m frustrated. yes, this is dissapointing. yes, i don’t want the next five years to be a repeat of the last five years.
@Aaron, this is not the same as akonadi. Akonadi could implement this, but it’s not the same at all. Not in purpose, nor in features nor in implementation.
For Evolution to start using Akonadi a lot more would have to happen. I don’t think that will happen soon enough. Nonetheless would Akonadi still need an API like this even if Evolution would…
first, Akonadi has such APIs. that’s the entire reason for its existence, even: to provide protocol based access to local caches of data.
second, i understand it would take a lot more work to get Evolution working with Akonadi. no disagreement there as it’s an obviously true statement.
what would make *sense* would b start looking at how to solve these problems properly by working on things like EvoAkonadi, and then you get things like what you just spent your time on for free.
however, by continuing to invest in the here-and-now implementation the end point of that effort continues to be delayed ever further.
what really pricked my ears up was the bit about “you could write an email client front end with it”, and if we’re all honest with ech other it’s pretty evident what the end game there is likely to turn to be.
we’ll have two incompatible solutions for no real purpose, diminishing the user experience (“why doesn’t FooCal work with BarMail? why does CoolTimeWidget only work with certain PIM apps? “) and dividing up our sparse resources in the PIM world.
it’s not an interesting or overly sexy area to work in, nor a layer where innovation is exacty the name of the game. it’s the right place to work together rather than ignore each other.
at LCA the other year, a GNOME developer gave a lightening talk during the gnome mini-conf (which i attended =) about considering using Akonadi. the Akonadi people were trying very hard to reach out to others, even offering up one of their GSoC slots to someone willing to write a gobject based client lib for Akonadi. i was hopeful.
since, though, i don’t see a lot of interest or attention being paid. i do see people reinventing wheels and fd.o saying “no” to hosting akonadi server dev due to this.
another end game is just to work at making applications that insist on their own world view irrelevent to the broader community. that would be hugely unfortunate and not a great way to service the userbase in the short term, but would at least get us to a useful long term solution.
Aaron! Does Akondi link to qt? if it does, i can perfectly understand why GNOME apps would not want to depend on it or fdo won’t want to host it because qt was not concidered (purely) Free SW for good reason until a few days ago.
@Aaron: I found akonadi to be really interesting, but I did not find information on Email-specific APIs, just generic ‘attributes storage’ APIs, any pointers?
@Zeeshan Ali: that’s bullshit; Qt has been Free Software for a long time now (9+ years); it just went GPL->LGPL
“Does Akonadi link to qt?”
What moron would deliberately link a PIM-framework to an arbitrary GUI toolkit that the developers _knew _ ahead of time that half of the Free Software community doesn’t even want to touch, because it gives them the “heebie-jeebies”?
And _then_ propose it be a desktop-agnostic FDO standard?
Really, you’d hope you guys would be past non-starter criticism like this by now. Give the KDE guys some credit. You may not agree with their implementation, you may be jaded with how the desktop-cooperation ecosystem can still seem a mostly hypocritical minefield, but I doubt their design methods are inherently stupid/malicious/without-any-intelligent-thought-whatsoever, as that statement seemed to imply.
(Unless of course they _did_ do all that. Then I’d be disappointed. But even being outside the development process, that seems like an incredibly obvious moronic accusation.)
@Judd:
You’re so right. No moron would be stupid enough to think smart people like KDE developers would design a storage mechanism like that. A smart person like you of course knows it would use a proper IPC mechanism so any client could use it.
(Unless of course you _didn’t_ know that. Then I’d be disappointed. But even being outside the development process, that seems like an incredibly obvious moronic accusation.)
@Gustavo:
Akonadi is designed to handle different data types. The transfer format will therefore depend on the data type, e.g. it currently is vcard for contacts, ical for calendar and I think (haven’t worked with mail myself yet) for mail it is the same as whatever is usually used for mail (multipart MIME?)
IRC Channel #akonadi on freenode network
Jud: You might check the sources of Akonadi sometime, but they do indeed depend on Qt – I just checked.
server: http://websvn.kde.org/trunk/kdesupport/akonadi/ (depends on Qt)
client: http://websvn.kde.org/trunk/KDE/kdepimlibs/akonadi/ (depends on Qt and KDE libs)
Let’s also note that Akonadi’s design is epic fail. Akonadi is basically the same exact thing as Evolution-Data-Server except that instead of offering decent client APIs (using DBus), it offers everything via IMAP – ical messages are in a Calendar folder, vcard messages are in a Addressbook folder, etc. This is the most retarded thing ever conceived. IMAP was not designed to serve data like this efficiently… and what’s worse is that neither GNOME nor KDE have even a half-way decent IMAP client implementation.
If the purpose of Akonadi is for concurrent access to messages, calendar data and vcards as MIME objects, a better design would have simply been to agree on a Maildir-like directory of messages on the local filesystem instead.
Even that, imho, would not be ideal however.
A better approach would be to agree to a specific Maildir location for mail and design similar concurrent access directory layouts for vcard and ical files and to agree to those locations as well.
Think about it – if the groupware clients are going to have to all implement the most complex mail access protocol known to man (IMAP) and they’re still going to have to implement a MIME parser, it’s not much more effort to talk to the real IMAP accounts themselves, is it? So what’s the point of talking to a daemon process using the same protocol they’d be using to talk to the server directly?
Akonadi is also inefficient by design because it forces 2 copies of the current message to be in memory (one in the akonadi server, one in the client), not to mention at least a third copy in memory for the renderer.
(KDE’s libmime++ parses entire MIME messages into RAM and iirc, so does Evolution’s).
I’m not a huge fan of Evolution, but the developers got at least 1 thing right: they didn’t make the retarded decision to make Evolution-Data-Server into a mail server.
In http://live.gnome.org/Evolution/Metadata, I read “This document is obsolete for Evolution’s Tracker EPlugin in Tracker’s master and > 0.7 releases: it doesn’t use this API anymore, instead it uses SPARQL Update directly against tracker-store.”. Does that mean that the hopes to get a dbus interface to evolution on a modern system are vain? Or is the plugin available from some other source (and working on recent Evolution)?
@Pietro: We obsoleted this API ourselves. You can perhaps still find the plugin in our git archive but we don’t use it ourselves that way. What you can do is simply query Tracker’s RDF service instead using SPARQL. Thanks to the Evolution plugin for it, will Tracker’s RDF store have the exact same information.
Thanks for the info… so I assume that the (current) file src/plugins/evolution/tracker-evolution-plugin.c is the _new_ (RDF) one, though last (non-cosmetic) change dates back to January 2009?
I’m a little confused…