At GCDS Jamie told us that he wants to make a plugin for tracker-store that writes all the triplets to a CouchDB instance.
Letting a CouchDB be a sort of offline backup isn’t very interesting. You want triples to go into the CouchDB at the moment of guaranteed storage: at commit time.
For the purpose of developing this we provide the following internal API.
typedef void (*TrackerStatementCallback) (const gchar *graph,
const gchar *subject,
const gchar *predicate,
const gchar *object,
GPtrArray *rdf_types,
gpointer user_data);
typedef void (*TrackerCommitCallback) (gpointer user_data);
tracker_data_add_insert_statement_callback (TrackerStatementCallback callback,
gpointer user_data);
tracker_data_add_delete_statement_callback (TrackerStatementCallback callback,
gpointer user_data);
tracker_data_add_commit_statement_callback (TrackerCommitCallback callback,
gpointer user_data);
You’ll need to make a plugin for tracker-store and make the hook at the initialization of your plugin.
Current behaviour is when graph is NULL, it means that the default graph is being used. If it’s not NULL, it means that you probably don’t want the data in CouchDB: it’s data that’s coming from a miner. You probably only want to store data that is coming from the user. His applications won’t use FROM and INTO for their SPARQL Update queries, meaning that graph is NULL.
Very important is that your callback handler works with bottom halves: put your expensive task on a queue and handle the queued item somewhere else. You can for example use a GThreadPool or a GQueue plus a g_idle_add_full with G_PRIORITY_LOW callback picking items one by one on the mainloop. You should never have a TrackerStatementCallback or a TrackerCommitCallback that blocks. Not even a tiny tiny bit of blocking: it’ll bring everything in tracker-store on its knees. It’s why we aren’t giving you a public plugin API with a way to install your own plugins outside of the Tracker project.
By the way: we want to see code instead of talk before we further optimize things for this purpose.