Today we improved journal replaying from 1050s for my test of 25249 resources to 58s.
Journal replaying happens when your cache database gets corrupted. Also when you restore a backup: restore uses the same code the journal replaying uses, backup just makes a copy of your journal.
During the performance improvements we of course found other areas related to data entry. It looks like we’re entering a period of focus on performance, as we have a few interesting ideas for next week already. The ideas for next week will focus on performance of some SPARQL functions like regex.
Meanwhile are Michele Tameni and Roberto Guido working on a RSS miner for Tracker and has Adrien Bustany been working on other web miners like for Flickr, GData, Twitter and Facebook.
I think the first pieces of the RSS- and the other web miners will start becoming available in this week’s unstable 0.7 release. Martyn is still reviewing the branches of the guys, but we’re very lucky with such good software developers as contributors. Very nice work Michele, Roberto and Adrien!
Glad to see you’re using libgdata for the GData miner. If (When) you find any deficiencies in the API while writing the miner, please let me know. The API’s still slushy, so such problems can be fixed. :-)
Goodness! We are not alone with MFO ontology anymore! :)
Great!