Before
For returning the results of a SPARQL SELECT query we used to have a callback like this. I removed error handling, you can find the original here.
We need to marshal a database result_set to a GPtrArray because dbus-glib fancies that. This is a lot of boxing the strings into GValue and GStrv. It does allocations, so not good.
static void
query_callback(TrackerDBResultSet *result_set,GError *error,gpointer user_data)
{
TrackerDBusMethodInfo *info = user_data;
GPtrArray *values = tracker_dbus_query_result_to_ptr_array (result_set);
dbus_g_method_return (info->context, values);
tracker_dbus_results_ptr_array_free (&values);
}
void
tracker_resources_sparql_query (TrackerResources *self, const gchar *query,
DBusGMethodInvocation *context, GError **error)
{
TrackerDBusMethodInfo *info = ...; guint request_id;
TrackerResourcesPrivate *priv= ...; gchar *sender;
info->context = context;
tracker_store_sparql_query (query, TRACKER_STORE_PRIORITY_HIGH,
query_callback, ...,
info, destroy_method_info);
}
After
Last week I changed the asynchronous callback to return a database cursor. In SQLite that means an sqlite3_step(). SQLite returns const pointers to the data in the cell with its sqlite3_column_* APIs.
This means that now we’re not even copying the strings out of SQLite. Instead, we’re using them as const to fill in a raw DBusMessage:
static void
query_callback(TrackerDBCursor *cursor,GError *error,gpointer user_data)
{
TrackerDBusMethodInfo *info = user_data;
DBusMessage *reply; DBusMessageIter iter, rows_iter;
guint cols; guint length = 0;
reply = dbus_g_method_get_reply (info->context);
dbus_message_iter_init_append (reply, &iter);
cols = tracker_db_cursor_get_n_columns (cursor);
dbus_message_iter_open_container (&iter, DBUS_TYPE_ARRAY,
"as", &rows_iter);
while (tracker_db_cursor_iter_next (cursor, NULL)) {
DBusMessageIter cols_iter; guint i;
dbus_message_iter_open_container (&rows_iter, DBUS_TYPE_ARRAY,
"s", &cols_iter);
for (i = 0; i < cols; i++, length++) {
const gchar *result_str = tracker_db_cursor_get_string (cursor, i);
dbus_message_iter_append_basic (&cols_iter,
DBUS_TYPE_STRING,
&result_str);
}
dbus_message_iter_close_container (&rows_iter, &cols_iter);
}
dbus_message_iter_close_container (&iter, &rows_iter);
dbus_g_method_send_reply (info->context, reply);
}
Results
The test is a query on 13500 resources where we ask for two strings, repeated eleven times. I removed a first repeat from each round, because the first time the sqlite3_stmt still has to be created. This means that our measurement would get a few more milliseconds. I also directed the standard out to /dev/null to avoid the overhead created by the terminal. The results you see below are the value for “real”.
There is of course an overhead created by the “tracker-sparql” program. It does demarshaling using normal dbus-glib. If your application uses DBusMessage directly, then it can avoid the same overhead. But since for both rounds I used the same “tracker-sparql” it doesn’t matter for the measurement.
$ time tracker-sparql -q "SELECT ?u ?m { ?u a rdfs:Resource ;
tracker:modified ?m }" > /dev/null
Without the optimization:
0.361s, 0.399s, 0.327s, 0.355s, 0.340s, 0.377s, 0.346s, 0.380s, 0.381s, 0.393s, 0.345s
With the optimization:
0.279s, 0.271s, 0.305s, 0.296s, 0.295s, 0.294s, 0.295s, 0.244s, 0.289s, 0.237s, 0.307s
The improvement ranges between 7% and 40% with average improvement of 22%.
