My KDE week

KDE: Week 33-39

Posted on: September 10, 2010

Nepomuk+strigi as desktop search

In the last weeks I – once again – got fed-up with strigi/nepomuk being of no use to me. Since KDE 4.0 I long for a desktop search, i.e. some way of finding files and getting a result list such as google etc. has it, i.e. including some context around the string found in the document and not just a file list. Anything else would just be a faster version of kfind for that task. And since I do not use tags, it is the only desktop search task for me.

So the first thing why it did not work for me was that strigi+nepomuk never finished to index my files because strigi 0.7.2 crashes while it indexes several filetypes such as email, flac or rpms.  Having a look at strigi’s website the project seemed dead. No news since 2008. This is a depressing first impression. Having a look at the SF project page continues with this, i.e. the latest available tarball is version 0.6.4 and the docs point to KDE’s playground svn repo if one wants to get strigi. The latter obviously fails because strigi is part of kdesupport.

Having a look at the bug tracker I found life though! And after I found out how to track down which file crashes strigi I could even provide some files to help fix the bugs – and fixed they were! Phreedom fixed one crash after the other so in the end I  had an almost working strigi which could index my files. Almost, because some symbol lookup error (symbol lookup error: /usr/lib64/libldap-2.4.so.2: undefined symbol: ldap_int_tls_destroy) was still crashing strigi, yet this is not strigi’s fault (same bug at Red Hat) and one can work around it by removing /usr/lib64/strigi/strigiea_vcf.so and strigiea_ics.so.

So if you find any crashes with strigi, use xmlindexer <folder or filename> or rdfindexer (both part of strigi) to track down which file causes it and report it at SF including the file – because strigi is alive and the devs are really nice people that fix bugs quickly if they are provided with testfiles! While reading along in #strigi I also gathered something about work on clucene and strigi to make it even faster. So it seems that the next releases of strigi et al. will result in some nice improvements for us users.

All crashes fixed and now I got a 1.4GB sized index with information about 130000 files in it, yet krunner and dolphin only return a file list which is as useful as if google only returned URLs in its results. If the search returns 20 OpenOffice documents and their filename does not tell me which of them is the document I am looking for, I would have to open all of them and use the OpenOffice find functionality to find the string within the document and check whether it is the paragraph I was looking for.

So I had a look at the nepomuk mailinglist in search for a search client to use and dolphin is apparently the best choice. And despite the fact that the term  “desktop search” (used for nepomuk+strigi in KDE’s systemsettings) has already been defined by other applications such as beagle or google, nepomuk+strigi+dolphin cannot display the search results with some context around the found string. For me this is the most basic functionality of a desktop search. Yet if I understood correctly nepomuk is more than that and has different and more ambitious aims. Thus this feature does not have a high priority.

IMHO it is absolutely understandable that nepomuk aims at something different and thus sets its priorities differently, especially because of the lack of developers involved in the project. Yet maybe providing these basic features would attract users and thus developers? And of course it would help a lot to gain some acceptance for all the CPU and disk space used by strigi+nepomuk. Without a wide acceptance nepomuk+strigi will stay turned off by a lot of distros by default.  Hence IMO nepomuk+strigi has a serious search client issue. There are some clients such as the crystal plasma applet within playground and indeed it does work and provides some context around the string found but only for a few files, i.e. it does not show any context for most of its results even though all of them are e.g. OO documents. And it has other issues as well, such as showing some bogus results. But it is a start and because it can be put into the panel it is very accessible for the users. It would be even more so if it would consist of a text-field rather than a click-able button to open a text-field.

I really would like to see a search client that enables nepomuk+strigi to be useful as a simple desktop search and gain acceptance from there. Start with the basics so to say. We will see what KDE SC 4.6 brings and since its their time and project it’s up to the devs to set their own pace and goals.

UPDATE: Sebastian Trüg just blogged about new features for dolphin’s search results view and guess what – there is some context shown! Great news! Now there only needs to be a text-field in the panel to e.g. start dolphin with the search terms entered. If things progress that nicely openSUSE 11.4 might even enable nepomuk+strigi by default.

Digikam 2

Since I was curious about the versioning and face recognition in the upcoming digikam 2.0 release, I compiled libkexiv2 and libkdcraw from trunk and got everything else needed from digikam2’s branch. Everything seems quite stable, i.e. I did not get any crashes yet despite the fact that there was not even a beta release yet. The latter is scheduled for October btw.

Neither versioning nor face detection are finished yet but the devs need feedback. So if you have some spare time, try those new features and report back to digikam’s devel mailinglist. From my point of view versioning already works quite well while face detection still needs some time until one can really test it. At least for me scanning the whole collection stalls after a few pictures and I cannot add names to the faces found, so it’s hard to test anything.

However, I think that those two and the other new features of digikam 2 will give that project yet another push to more popularity. Its development seems very lively to me and its great to see it improve from release to release. The only thing missing to make me completely happy is that one can keep tools opened when applying them to a picture. As in picasa doing so saves a lot of clicks if one e.g. goes through new pictures and wants to use cropping while maintaining the aspect ratio on some of them. This cannot be done with a batch tool since the user has to set the part of the image that should be cropped manually for every picture.

openSUSE 11.3 KDE (4.5.1) reloaded

Today openSUSE’s KDE reloaded LiveCD got published. It’s basically an installable  openSUSE 11.3 LiveCD with KDE 4.5.1 packages including all openSUSE patches and branding. The latter distinguishes it from KDE Four Live which is supposed to provide almost vanilla KDE packages.

The openSUSE 11.3 KDE reloaded LiveCD/USB is available for x86 and x86_64 from here. openSUSE users can install and run the package imagewriter to get a dead easy to use GUI to put the ISO image on an USB stick. If you want to provide feedback you can contact javier_ or alin in the #opensuse-kde IRC channel on freenode.

KDEPIM 4.4 translations for KDE SC 4.5

Since there is no KDEPIM 4.5 in KDE SC 4.5 yet, there are no translations either. Users updating to KDE SC 4.5 asked for translations for their KDEPIM 4.4 used with KDE SC 4.5 and thus KDE:Release:45 as well as KDE:Distro:Factory now contain translation packages for KDE SC 4.5 which have the KDEPIM transalations from KDE SC 4.4.

Desktop effects

Another note for others with intel graphics chipset using KDE trunk. I (945GME) had to disable functionality checks to get any desktop effects at all. According to other users this is also true for the 965GM and probably others . I think this is a known issue and caused by the buggy intel drivers. The problem with this is that I can enable the blur effect yet it only paints all plasma widget’s background in a dark grey rather than blurring it. So one has to disable blurring manually to get some semi-transparent blurring from the widget theme. I guess that disabling the functionality checks does also disable the blacklisting of the intel chipsets regarding the blurring effect.

20 Responses to "KDE: Week 33-39"

re: strigi – why not use recoll?

a) it works
b) there’s a kde4 (and qt4) gui client and
c) there’s a kio

Thanks for the post. I echo your comments about the longing for a working Desktop-Search with Nepomuk and Strigi. That is my use case and until KDE can offer 75% of what Google Desktop Search can I will have to stay with Google.

Desktop Search is a very key component of the KDE desktop experience – Windows, Apple all have this now and we are languishing. Gnome has several solutions all more functional than ours. For many non-developers who daily use a KDE desktop (such as me and my team in my office) it is search is vital find all the stuff we loose! I need it embedded in krunner, file dialogs, konqueror AND dolphin AND plasma apps etc.

I hope your rallying cry will stir us towards this end. I really can not offer any help other than a Use-Case and a general huzzah for anyone willing to help out in this direction.

Kevin

It’s pity, but Strigi has not been usable for years – it crashes all the time and there is no good front-end for it (Dolphin search box returns nemopuk’s objects with is useless) . I also got fed-up with Strigi and started using Tracker. It’s great, very fast, and stable tool. Only one bad thing is that there is no qt/kde front-end for it.

If a current svn checkout still crashes for you, find the file that crashes it and submit the bug. As I wrote, the devs are really quick on fixing bugs if they get the file to reproduce it.

You can use xmlindexer folder or xmlindexer filename to track down the file.

But yes, I agree with you that it is a pity that no matter how good strigi could be, without a proper client it is of no use.

Tracker is not an option for me for the same reason. I need a KDE client.

Thanks for giving me the tip. I’ll try this xmlindexer command. I have one more question. Is there any way to clean the Strigi database file from – how to say it – incorrect entries? Files that don’t exist on my hard disc any longer are still available( I guess ) in the database which makes it grow rapidly.

@GReg: If there are really old files in your database I would say it is a bug and you should file it at bugs.kde.org. Normally it should check for all files when it starts. That’s the actual reason why it hammers your harddisk on every KDE login.

Yet it might also be caused by strigi crashing before finishing the index. In that case you would have to get a working strigi first.

For me current svn and deleting the ics and vcf files results in such.

A search GUI was developed during this year’s GSoC and currently lives in playground.

Yes, there is such a search GUI. But unfortunately, I have not been able to compile it with KDE SC 4.5.1.

It is to be found here, btw.: http://websvn.kde.org/trunk/playground/base/nepomuk-kde/nepomuksearchgui/

I tried the other clients from playground as well. None of them compiled for me against KDE 4.4 which is what openSUSE 11.3 comes with except for crystal the plasma applet. And none of them worked for me with current KDE trunk.

Also I was surprised to see that dolphin is regarded the best client. maybe because those clients are still in playground for a reason.

Btw. can somebody tell me why compiling strigi fails on one computer for me with:

[ 1%] Building CXX object libstreams/lib/CMakeFiles/streamsstatic.dir/encodinginputstream.cpp.o
In file included from /usr/include/bits/errno.h:25:0,
from /usr/include/errno.h:36,
from /usr/include/c++/4.5/cerrno:43,
from /home/rabauke/devel/src/strigi/libstreams/lib/encodinginputstream.cpp:26:
/usr/include/linux/errno.h:4:23: fatal error: asm/errno.h: file or directory not found
compilation terminated.
make[2]: *** [libstreams/lib/CMakeFiles/streamsstatic.dir/encodinginputstream.cpp.o] Error 1
make[1]: *** [libstreams/lib/CMakeFiles/streamsstatic.dir/all] Error 2
make: *** [all] Error 2

I have searched for a solution and installed every kernel devel package there is for openSUSE including (linux-)glibc-devel, kernel-devel, kernel-default-devel, kernel-source but it does not work.😦

@ rabauke: Try linux-kernel-headers. That seems to me what you are missing.

That’s the package that came to my mind as well but there is no such package for openSUSE 11.3.😐

http://software.opensuse.org/search?q=kernel-headers&baseproject=openSUSE%3A11.3&lang=de&exclude_filter=home%3A&exclude_debug=true

On the computer it works on I just have kernel-desktop, kernel-desktop-devel, kernel-devel and linux-glibc-devel installed. That’s why I do not understand it. Even make cloneconfig fails with the above error. I guess I will take this “opportunity” to test the installation of openSUSE 11.3 KDE reloaded.🙂

I don’t know if it makes sense but I was thinking that you need the kernel-source package

It’s installed already.😦

@ rabauke: I use Arch Linux, so I don’t know what OpenSuse uses these days. I remember that it used to be linux-kernel-headers… Some googling indicates it is now called linux-glibc-devel, but you say you already installed it. Weird.

Oh, found this bug: https://bugzilla.novell.com/show_bug.cgi?id=571604
Supposedly, it is fixed, but maybe the symlink is missing for you?

So Strigi is dead? If so, that explains why Strigi can not yet do a realtime indexing and we can not get searched anything good with it.

GFam and Fam does not help either. And trying to get Strigi have a inotify support does not work.

Realtime index means that at the second the file gets saved/opened/moved/deleted it is processed trought Strigi and Nepomuk (added, modified, removed from index).

Realtime indexing seems to work here. And as I wrote, strigi is not dead. Just its webpages are outdated. Development is lively.

It would be nice to find out what actually makes the Strigi work with realtime. But as I remember, openSUSE has been distro what has taken care of that always. Example when the Beagle came out, they were only one having working version for KDE 3 at that time from it with nice kio-slaves and open/save dialoges.

I compiled strigi from svn myself because openSUSE 11.3 ships 0.7.2 which crashes. And I have both fam and gamin-devel installed at compile time.

I just tried again, i.e. saved a document from within firefox and searched for it a second later via the crystal plasmoid from playground which did find that document. The sorting is from oldest to newest though so it was the last item in the results list.

[…] Nepomuk+strigi as desktop search In the last weeks I – once again – got fed-up with strigi/nepomuk being of no use to me. Since KDE 4.0 I long for a desktop search, i.e. some way of finding files and getting a result list such as google etc. has it, i.e. including some context around the string found in the document and not just a file list. Anything else would just be a faster version of kfind for that task. And since I do not use tags, it is the only desktop search task for me. […]

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: