Before
For returning the results of a SPARQL SELECT query we used to have a callback like this. I removed error handling, you can find the original here.
We need to marshal a database result_set to a GPtrArray because dbus-glib fancies that. This is a lot of boxing the strings into GValue and GStrv. It does allocations, so not good.
static void query_callback(TrackerDBResultSet *result_set,GError *error,gpointer user_data) { TrackerDBusMethodInfo *info = user_data; GPtrArray *values = tracker_dbus_query_result_to_ptr_array (result_set); dbus_g_method_return (info->context, values); tracker_dbus_results_ptr_array_free (&values); } void tracker_resources_sparql_query (TrackerResources *self, const gchar *query, DBusGMethodInvocation *context, GError **error) { TrackerDBusMethodInfo *info = ...; guint request_id; TrackerResourcesPrivate *priv= ...; gchar *sender; info->context = context; tracker_store_sparql_query (query, TRACKER_STORE_PRIORITY_HIGH, query_callback, ..., info, destroy_method_info); }
After
Last week I changed the asynchronous callback to return a database cursor. In SQLite that means an sqlite3_step(). SQLite returns const pointers to the data in the cell with its sqlite3_column_* APIs.
This means that now we’re not even copying the strings out of SQLite. Instead, we’re using them as const to fill in a raw DBusMessage:
static void query_callback(TrackerDBCursor *cursor,GError *error,gpointer user_data) { TrackerDBusMethodInfo *info = user_data; DBusMessage *reply; DBusMessageIter iter, rows_iter; guint cols; guint length = 0; reply = dbus_g_method_get_reply (info->context); dbus_message_iter_init_append (reply, &iter); cols = tracker_db_cursor_get_n_columns (cursor); dbus_message_iter_open_container (&iter, DBUS_TYPE_ARRAY, "as", &rows_iter); while (tracker_db_cursor_iter_next (cursor, NULL)) { DBusMessageIter cols_iter; guint i; dbus_message_iter_open_container (&rows_iter, DBUS_TYPE_ARRAY, "s", &cols_iter); for (i = 0; i < cols; i++, length++) { const gchar *result_str = tracker_db_cursor_get_string (cursor, i); dbus_message_iter_append_basic (&cols_iter, DBUS_TYPE_STRING, &result_str); } dbus_message_iter_close_container (&rows_iter, &cols_iter); } dbus_message_iter_close_container (&iter, &rows_iter); dbus_g_method_send_reply (info->context, reply); }
Results
The test is a query on 13500 resources where we ask for two strings, repeated eleven times. I removed a first repeat from each round, because the first time the sqlite3_stmt still has to be created. This means that our measurement would get a few more milliseconds. I also directed the standard out to /dev/null to avoid the overhead created by the terminal. The results you see below are the value for “real”.
There is of course an overhead created by the “tracker-sparql” program. It does demarshaling using normal dbus-glib. If your application uses DBusMessage directly, then it can avoid the same overhead. But since for both rounds I used the same “tracker-sparql” it doesn’t matter for the measurement.
$ time tracker-sparql -q "SELECT ?u ?m { ?u a rdfs:Resource ; tracker:modified ?m }" > /dev/null
Without the optimization:
0.361s, 0.399s, 0.327s, 0.355s, 0.340s, 0.377s, 0.346s, 0.380s, 0.381s, 0.393s, 0.345s
With the optimization:
0.279s, 0.271s, 0.305s, 0.296s, 0.295s, 0.294s, 0.295s, 0.244s, 0.289s, 0.237s, 0.307s
The improvement ranges between 7% and 40% with average improvement of 22%.