Although with SQLite WAL we have direct-access now, we don’t support direct-access for insert and delete SPARQL queries. Those queries when made using libtracker-sparql still go over D-Bus using Adrien’s FD passing D-Bus IPC technique. The library will do that for you.
After investigating a performance analysis by somebody from Intel we learned that there is still a significant overhead per each IPC call. In the analysis the person made miner-fs combine multiple insert transactions together and then send it over as a single big transaction. This was noticeably faster than making many individual IPC requests.
The problem with this is that if one of the many insert queries fail, they all fail: not good.
We’re now experimenting with a private API that allows you to pass n individual insert transactions, and get n errors back, using one IPC call.
The numbers are promising even on Desktop D-Bus (the test):
$ cd tests/functional-tests/ $ ./update-array-performance-test First run (first update then array) Array: 0.103675, Update: 0.139094 Reversing run (first array then update) Array: 0.290607, Update: 0.161749 $ ./update-array-performance-test First run (first update then array) Array: 0.105920, Update: 0.137554 Reversing run (first array then update) Array: 0.118785, Update: 0.130630 $ ./update-array-performance-test First run (first update then array) Array: 0.108501, Update: 0.136524 Reversing run (first array then update) Array: 0.117308, Update: 0.151192 $
We’re now deciding whether or not the API will become public; returning arrays of errors isn’t exactly ‘nice’ or ‘standard’.