My KDE week

Debugging nepomuk/virtuoso’s CPU usage

Posted on: November 9, 2011

There was a lot of bug fixing regarding nepomuk and its indexing. However you might still get a high CPU-usage. Reporting this is a bit useless unless you can at least give some info about what’s happening.

So what you can do is query virtuoso’s status. On openSUSE it works like this: first find the .ini file currently in usage to get the port virtuoso is using, connect to virtuoso and finally query virtuoso for its status and running statements. The latter are unfortunately truncated so I would appreciate some hint on how to get around that.

ps aux | grep virtuoso

finds /usr/bin/virtuoso-t +foreground +configfile /tmp/virtuoso_T18122.ini +wait

cat /tmp/virtuoso_T18122.ini | grep Port

finds ServerPort=1111

isql-vt -H localhost -S 1111 -U dba -P dba

which connects to virtuoso

status('rhck');

which shows you some info and which queries are keeping the process busy. isql-vt is part of the virtuoso-server package but it might be that only recent packages have it compiled and older packages lack the tools.

trace_on ('client_sql');

will start logging every command send to virtuoso from the nepomuk service into the virtuoso.log file which should be next to your virtuoso.db, e.g. ~/.kde4/share/apps/nepomuk/repository/main/data/virtuosobackend/. Use trace_off(); to stop logging. See the virtuoso manual for more options.

Further you can do the following:

  • open kdebugdialog and search for nepomuk
  • use the quick-search to enable debug output for nepomuk
  • on a konsole do:  qdbus org.kde.NepomukServer /nepomukserver quit
  • wait until nepomuk and virtuoso are gone
  • restart it from the konsole: nepomukserver

UPDATE: There is yet another possibility to debug virtuoso-t, i.e. find out the currently active query. On a konsole issue the following commands:

  • qdbus org.kde.nepomuk.services.nepomukqueryservice
  • You will see something like /nepomukqueryservice/query64 if the query does not finish within a reasonable amount of time you can get the query string
  • qdbus org.kde.nepomuk.services.nepomukqueryservice /nepomukqueryservice/query64 queryString should then give you the query.

UPDATE 2: If virtuoso-t consumes CPU although there are no active queries, the analysis has to go a bit deeper. Virtuoso is started through Soprano with certain parameters which are set in a temporary ini-file (/tmp/virtuoso_XXXX.ini). Soprano needs to be modified manually to start Virtuoso with different parameters in the ini-file, e.g. to improve virtuoso-t’s behaviour by modifying backends/virtuoso/virtuosocontroller.cpp (Soprano) and setting NumberOfBuffers to 40000 (line 344) and SchedulerInterval to 0 (line 350).

After re-compiling soprano one has to attach gdb to virtuoso-t as soon as it starts consuming CPU and create a full threaded backtrace:

set logging file /tmp/virtuoso-t.out
set logging on
thread apply all bt full

Note: The above changes should not be used as default values for packaging!

If you have any other hints regarding this piece of software feel free to mention them and I will add them to the post.

About these ads

14 Responses to "Debugging nepomuk/virtuoso’s CPU usage"

finally, someone is working on nepomuk! :)

Hi,

I followed your instructions. Should I worry about

Lock Status: 1 deadlocks of which 0 2r1w, 2 waits,

I also get many

Transaction status: PENDING, 0 threads.

Cheers,
André

its isql-v on gentoo

That is one cool tip. Now I just have to understand the output. Will need to read a little on the weekend.

But looks kind of funky:
RPC: 769814 calls, -14 pending, 11 max until now, 0 queued, 146 burst reads (0%), 4 second brk=37875712

“-14″ pending?

And yes, virtuoso cpu usage has occured to me. System is up for 2 days, 2 minutes and vituoso has a cpu-time of nearly 110 hours. o_O

Yep, I wondered about the “-x pending” as well. It seems ahead of time. ;-)

All high cpu usage I have been seeing is related to indexing of PIM data. Perhaps you nepomuk guys should give a hand to the pimsters for a more performant integration.

I am getting the following errors.

“/usr/bin/nepomukservicestub(17589)” Soprano: “SQLExecDirect failed on query ‘sparql select distinct ?r ?reqProp1 (bif:concat(bif:search_excerpt(bif:vector(‘dyndns’), ?v7),bif:search_excerpt(bif:vector(‘dyndns’), ?v8),bif:search_excerpt(bif:vector(‘dyndns’), ?v10),bif:search_excerpt(bif:vector(‘dyndns’), ?v6))) as ?_n_f_t_m_ex_ where { { ‘ (iODBC Error: [OpenLink][Virtuoso iODBC Driver][Virtuoso Server]SQ074: Line 1: SP030: SPARQL compiler, line 1: syntax error at ‘;’ at ” immediately before end of statement)”

These errors come from a search in kmail and disapear when I remove the search. It seams that queries are being truncated.

You should report these things upstream at bugs.kde.org. Especially since you can reproduce it and know what causes the bug.

Everytime nepomuk is started, it starts over. (WHY OH WHY????)
After some time, it claims that initial indexing took some time.
After that, it settles on a constant CPU usage of 30-40% doing nothing, except chewing up ever-increasing amounts of RAM.

Unfortunately the archlinux package does not contain the isql command mentioned above. Looking at output, soprano spits out an endless amount of errors about some query being invalid.

Here is one (during indexing):

[/usr/bin/nepomukservicestub] “/usr/bin/nepomukservicestub(7375)” Soprano: “SQLExecDirect failed on query ‘sparql select distinct ?r ?reqProp1 (bif:concat(bif:search_excerpt(bif:vector(‘anders’), ?v4))) as ?_n_f_t_m_ex_ where { { ?r ?v2 . ?v2 <http://www.semanticdesktop.org/ontologies/2007/03/22/nco#hasEma&#039; (iODBC Error: [OpenLink][Virtuoso iODBC Driver][Virtuoso Server]SQ074: Line 1: syntax error at '#')"

I am using virtuoso 6.1.4 and KDE 4.7.3

Make sure that what you see right after logging into KDE is not just nepomuk checking whether all files are still in place which is different from indexing, i.e. much faster. Might still be considered a bug http://bugs.kde.org/show_bug.cgi?id=182064.

Regarding the permanent CPU usage, ask your distro where they included isq or file a bug to include it with virtuoso. With that info and what you got already you can file a bug at bugs.kde.org. The more information you can include the more likely it is that devs will pick it up.

great guide – this should find its way into the akonadi/nepomuk documentation in the KDE wiki…
btw: the command should be
isql-vt -H localhost -S 1111 -U dba -P dba
(-S denotes the port)

Thanks for the hint. I corrected the command.

No need to patch Soprano to increase the number of buffers. Nepomuk has configuration options for that.

Useless use of cat award! ;)

grep Port /tmp/virtuoso_T18122.ini

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

%d bloggers like this: