Hi,
I did the switch to KDE 4.13 and I seem to lost the tags that I've added to my files. This is most likely due to Nepomuk - > Baloo migration.
Could not find anything useful on the list or in the web, so :
Are the tags lost , or I need to dig deeper into how to attempt to retrieve them from some data store that Nepomuk had ?
If so, any hints or links on how to do it ?
Before any comments, found and tried some info, but nepomukbaloomigrator crashes on me, and I can't start nepomuk from my user, only from root (where tags aren't existing, obviously)
Piotr Gbyliczek wrote:
Hi,
I did the switch to KDE 4.13 and I seem to lost the tags that I've added to my files. This is most likely due to Nepomuk - > Baloo migration.
Could not find anything useful on the list or in the web, so :
Are the tags lost , or I need to dig deeper into how to attempt to retrieve them from some data store that Nepomuk had ?
If so, any hints or links on how to do it ?
The nepomukbaloomigrator tool is supposed to handle the transition. Maybe it didn't run properly?
-- Rex
On Thursday 21 August 2014 07:47:01 Rex Dieter wrote:
The nepomukbaloomigrator tool is supposed to handle the transition. Maybe it didn't run properly?
It crashes as mentioned. I have removed baloo entries from ~/.kde/share/config/nepomukserverrc and tried to start nepomuk to rerun, but it does not like it.
How I should start nepomuk as user ? It don't seem to be working, which is most likely an issue.
I have to add that i can add new tags and use them in Dolphin search, so it works, just my data is missing.
How to debug migrator ?
On Thursday 21 of August 2014 12:53:50 Piotr Gbyliczek wrote:
Hi,
I did the switch to KDE 4.13 and I seem to lost the tags that I've added to my files. This is most likely due to Nepomuk - > Baloo migration.
Could not find anything useful on the list or in the web, so :
Are the tags lost , or I need to dig deeper into how to attempt to retrieve them from some data store that Nepomuk had ?
If so, any hints or links on how to do it ?
Before any comments, found and tried some info, but nepomukbaloomigrator crashes on me, and I can't start nepomuk from my user, only from root (where tags aren't existing, obviously)
What happens when you start Nepomuk as user? Try it from console: nepomukctl start - it might tell you more about where the problem is. Maybe that is also the reason why the migrator crashes.
Dan
On Thursday 21 August 2014 15:04:30 Daniel Vratil wrote:
Before any comments, found and tried some info, but nepomukbaloomigrator crashes on me, and I can't start nepomuk from my user, only from root (where tags aren't existing, obviously)
What happens when you start Nepomuk as user? Try it from console: nepomukctl start - it might tell you more about where the problem is. Maybe that is also the reason why the migrator crashes.
I'm sure that I've tried it before several times and it did not started, which would be logical reason for the crash of the migrator.
This time however it did work, and this is the result :
Object::connect: No such signal org::freedesktop::UPower::DeviceAdded(QDBusObjectPath) Object::connect: No such signal org::freedesktop::UPower::DeviceRemoved(QDBusObjectPath) nepomukbaloomigrator(11775): "database is locked Unable to fetch row" nepomukbaloomigrator(11775): "database is locked Unable to fetch row" nepomukbaloomigrator(11775): "database is locked Unable to fetch row" nepomukbaloomigrator(11775): "database is locked Unable to fetch row" nepomukbaloomigrator(11775): Bananana Error nepomukbaloomigrator(11775): nepomukbaloomigrator(11775): Bananana Error nepomukbaloomigrator(11775): Serious Error: nepomukbaloomigrator(11775): Error opening table `/home/piotr/.local/share/baloo/file/postlist.': Couldn't open /home/piotr/.local/share/baloo/file/postlist.baseA: Too many open files Couldn't open /home/piotr/.local/share/baloo/file/postlist.baseB: Too many open files DatabaseOpeningError: Error opening table `/home/piotr/.local/share/baloo/file/postlist.': Couldn't open /home/piotr/.local/share/baloo/file/postlist.baseA: Too many open files Couldn't open /home/piotr/.local/share/baloo/file/postlist.baseB: Too many open files
KCrash: Application 'nepomukbaloomigrator' crashing... KCrash: Attempting to start /usr/libexec/kde4/drkonqi from kdeinit sock_file=/home/piotr/.kde/socket-takeshi.reconnix.com/kdeinit4__0 QSocketNotifier: Invalid socket 23 and type 'Read', disabling...
It seems that some work was done, as my tags are now present and files are tagged with them. However, the process crashed. Any way to check if it moved all the data before ?
On Thursday 21 of August 2014 14:30:03 Piotr Gbyliczek wrote:
On Thursday 21 August 2014 15:04:30 Daniel Vratil wrote:
Before any comments, found and tried some info, but nepomukbaloomigrator crashes on me, and I can't start nepomuk from my user, only from root (where tags aren't existing, obviously)
What happens when you start Nepomuk as user? Try it from console: nepomukctl start - it might tell you more about where the problem is. Maybe that is also the reason why the migrator crashes.
I'm sure that I've tried it before several times and it did not started, which would be logical reason for the crash of the migrator.
This time however it did work, and this is the result :
Object::connect: No such signal org::freedesktop::UPower::DeviceAdded(QDBusObjectPath) Object::connect: No such signal org::freedesktop::UPower::DeviceRemoved(QDBusObjectPath) nepomukbaloomigrator(11775): "database is locked Unable to fetch row" nepomukbaloomigrator(11775): "database is locked Unable to fetch row" nepomukbaloomigrator(11775): "database is locked Unable to fetch row" nepomukbaloomigrator(11775): "database is locked Unable to fetch row" nepomukbaloomigrator(11775): Bananana Error nepomukbaloomigrator(11775): nepomukbaloomigrator(11775): Bananana Error nepomukbaloomigrator(11775): Serious Error: nepomukbaloomigrator(11775): Error opening table `/home/piotr/.local/share/baloo/file/postlist.': Couldn't open /home/piotr/.local/share/baloo/file/postlist.baseA: Too many open files Couldn't open /home/piotr/.local/share/baloo/file/postlist.baseB: Too many open files DatabaseOpeningError: Error opening table `/home/piotr/.local/share/baloo/file/postlist.': Couldn't open /home/piotr/.local/share/baloo/file/postlist.baseA: Too many open files Couldn't open /home/piotr/.local/share/baloo/file/postlist.baseB: Too many open files
If the migrator cannot get lock, then it's probably before some other process is holding it - I assume you have the baloo_file process running, so stop it (just kill it, there's no other way) and run the migrator - this time it could work. Then start baloo_file again.
I at least committed a fix for Baloo to talk less bananas and more actual error messages :)
Dan
KCrash: Application 'nepomukbaloomigrator' crashing... KCrash: Attempting to start /usr/libexec/kde4/drkonqi from kdeinit sock_file=/home/piotr/.kde/socket-takeshi.reconnix.com/kdeinit4__0 QSocketNotifier: Invalid socket 23 and type 'Read', disabling...
It seems that some work was done, as my tags are now present and files are tagged with them. However, the process crashed. Any way to check if it moved all the data before ?
On Thursday 21 August 2014 17:23:24 Daniel Vratil wrote:
If the migrator cannot get lock, then it's probably before some other process is holding it - I assume you have the baloo_file process running, so stop it (just kill it, there's no other way) and run the migrator - this time it could work. Then start baloo_file again.
A bit better, no lock errors, but still crashing :
$ nepomukctl start $ nepomukctl status Nepomuk Server is running. Service storage is running. Service filewatch is running.
$ balooctl stop $ balooctl status Baloo File Indexer is NOT running Indexed 64273 / 64273 files Failed to index 0 files
$ nepomukbaloomigrator Object::connect: No such signal org::freedesktop::UPower::DeviceAdded(QDBusObjectPath) Object::connect: No such signal org::freedesktop::UPower::DeviceRemoved(QDBusObjectPath) nepomukbaloomigrator(14590): Bananana Error nepomukbaloomigrator(14590): nepomukbaloomigrator(14590): Bananana Error nepomukbaloomigrator(14590): Serious Error: nepomukbaloomigrator(14590): Error opening table `/home/piotr/.local/share/baloo/file/postlist.': Couldn't open /home/piotr/.local/share/baloo/file/postlist.baseA: Too many open files Couldn't open /home/piotr/.local/share/baloo/file/postlist.baseB: Too many open files DatabaseOpeningError: Error opening table `/home/piotr/.local/share/baloo/file/postlist.': Couldn't open /home/piotr/.local/share/baloo/file/postlist.baseA: Too many open files Couldn't open /home/piotr/.local/share/baloo/file/postlist.baseB: Too many open files
KCrash: Application 'nepomukbaloomigrator' crashing... KCrash: Attempting to start /usr/libexec/kde4/drkonqi from kdeinit sock_file=/home/piotr/.kde/socket-takeshi.reconnix.com/kdeinit4__0 QSocketNotifier: Invalid socket 23 and type 'Read', disabling...
Any ideas ?
On Thursday 21 August 2014 16:50:18 Piotr Gbyliczek wrote:
so stop it (just kill it, there's no other way) and run the migrator - this time it could work. Then start baloo_file again.
A bit better, no lock errors, but still crashing :
[...]
Couldn't open /home/piotr/.local/share/baloo/file/postlist.baseA: Too many open files
[...]
Any ideas ?
In fact, upping the file limit from 1024 with
$ulimit -n 4096
and going through the procedure again helped.
Rc file updated with migrated=true and tags are present.
Thanks.
On Thu, Aug 21, 2014 at 05:10:58PM +0100, Piotr Gbyliczek wrote:
On Thursday 21 August 2014 16:50:18 Piotr Gbyliczek wrote:
so stop it (just kill it, there's no other way) and run the migrator - this time it could work. Then start baloo_file again.
A bit better, no lock errors, but still crashing :
[...]
Couldn't open /home/piotr/.local/share/baloo/file/postlist.baseA: Too many open files
[...]
Any ideas ?
In fact, upping the file limit from 1024 with
$ulimit -n 4096
and going through the procedure again helped.
Rc file updated with migrated=true and tags are present.
I did recently go again through this after migrating one of my machines to F21.
Automatic migration did not happen or crashed without any notice. After some googling, trial and error I did
vim /etc/security/limits.conf # very big number of file descriptors
balooctl stop; ulimit -S -n 100000; rm ~/.kde/share/config/nepomukserverrc; nepomukctl start; sleep 600; nepomukbaloomigrator
I have repeated the commands several times because nepomukserver died or crashed or the migrator could not get some locks or whatever. Also, the migrator does not give any success or progress messages so I was not really confident if it finished succesfully.
The file descriptor limit definitely must be high enough so the migrator does not complain, otherwise the tags of some files simply won't be migrated. Afaics the number must be 3x the number of files with tags or comments so I was lucky that it still worked for me.
I did open a bug report upstreams about this: https://bugs.kde.org/show_bug.cgi?id=342832
Looking at the file descriptor leak I see something like this:
lr-x------. 1 rz rz 64 Jan 14 11:14 929 -> /home/rz/.local/share/baloo/file/record.DB lr-x------. 1 rz rz 64 Jan 14 11:14 93 -> /home/rz/.local/share/baloo/file/record.DB lr-x------. 1 rz rz 64 Jan 14 11:14 930 -> /home/rz/.local/share/baloo/file/termlist.DB lr-x------. 1 rz rz 64 Jan 14 11:14 931 -> /home/rz/.local/share/baloo/file/position.DB lr-x------. 1 rz rz 64 Jan 14 11:14 932 -> /home/rz/.local/share/baloo/file/postlist.DB lr-x------. 1 rz rz 64 Jan 14 11:14 933 -> /home/rz/.local/share/baloo/file/record.DB lr-x------. 1 rz rz 64 Jan 14 11:14 934 -> /home/rz/.local/share/baloo/file/termlist.DB lr-x------. 1 rz rz 64 Jan 14 11:14 935 -> /home/rz/.local/share/baloo/file/position.DB lr-x------. 1 rz rz 64 Jan 14 11:14 936 -> /home/rz/.local/share/baloo/file/postlist.DB lr-x------. 1 rz rz 64 Jan 14 11:14 937 -> /home/rz/.local/share/baloo/file/record.DB lr-x------. 1 rz rz 64 Jan 14 11:14 938 -> /home/rz/.local/share/baloo/file/termlist.DB lr-x------. 1 rz rz 64 Jan 14 11:14 939 -> /home/rz/.local/share/baloo/file/position.DB lr-x------. 1 rz rz 64 Jan 14 11:14 94 -> /home/rz/.local/share/baloo/file/termlist.DB lr-x------. 1 rz rz 64 Jan 14 11:14 940 -> /home/rz/.local/share/baloo/file/postlist.DB lr-x------. 1 rz rz 64 Jan 14 11:14 941 -> /home/rz/.local/share/baloo/file/record.DB lr-x------. 1 rz rz 64 Jan 14 11:14 942 -> /home/rz/.local/share/baloo/file/termlist.DB lr-x------. 1 rz rz 64 Jan 14 11:14 943 -> /home/rz/.local/share/baloo/file/position.DB lr-x------. 1 rz rz 64 Jan 14 11:14 944 -> /home/rz/.local/share/baloo/file/postlist.DB lr-x------. 1 rz rz 64 Jan 14 11:14 945 -> /home/rz/.local/share/baloo/file/record.DB lr-x------. 1 rz rz 64 Jan 14 11:14 946 -> /home/rz/.local/share/baloo/file/termlist.DB lr-x------. 1 rz rz 64 Jan 14 11:14 947 -> /home/rz/.local/share/baloo/file/position.DB lr-x------. 1 rz rz 64 Jan 14 11:14 948 -> /home/rz/.local/share/baloo/file/postlist.DB lr-x------. 1 rz rz 64 Jan 14 11:14 949 -> /home/rz/.local/share/baloo/file/record.DB lr-x------. 1 rz rz 64 Jan 14 11:14 95 -> /home/rz/.local/share/baloo/file/position.DB lr-x------. 1 rz rz 64 Jan 14 11:14 950 -> /home/rz/.local/share/baloo/file/termlist.DB lr-x------. 1 rz rz 64 Jan 14 11:14 951 -> /home/rz/.local/share/baloo/file/position.DB lr-x------. 1 rz rz 64 Jan 14 11:14 952 -> /home/rz/.local/share/baloo/file/postlist.DB lr-x------. 1 rz rz 64 Jan 14 11:14 953 -> /home/rz/.local/share/baloo/file/record.DB lr-x------. 1 rz rz 64 Jan 14 11:14 954 -> /home/rz/.local/share/baloo/file/termlist.DB lr-x------. 1 rz rz 64 Jan 14 11:14 955 -> /home/rz/.local/share/baloo/file/position.DB lr-x------. 1 rz rz 64 Jan 14 11:14 956 -> /home/rz/.local/share/baloo/file/postlist.DB lr-x------. 1 rz rz 64 Jan 14 11:14 957 -> /home/rz/.local/share/baloo/file/record.DB lr-x------. 1 rz rz 64 Jan 14 11:14 958 -> /home/rz/.local/share/baloo/file/termlist.DB lr-x------. 1 rz rz 64 Jan 14 11:14 959 -> /home/rz/.local/share/baloo/file/position.DB lr-x------. 1 rz rz 64 Jan 14 11:14 96 -> /home/rz/.local/share/baloo/file/postlist.DB lr-x------. 1 rz rz 64 Jan 14 11:14 960 -> /home/rz/.local/share/baloo/file/postlist.DB lr-x------. 1 rz rz 64 Jan 14 11:14 961 -> /home/rz/.local/share/baloo/file/record.DB lr-x------. 1 rz rz 64 Jan 14 11:14 962 -> /home/rz/.local/share/baloo/file/termlist.DB lr-x------. 1 rz rz 64 Jan 14 11:14 963 -> /home/rz/.local/share/baloo/file/position.DB lr-x------. 1 rz rz 64 Jan 14 11:14 964 -> /home/rz/.local/share/baloo/file/postlist.DB lr-x------. 1 rz rz 64 Jan 14 11:14 965 -> /home/rz/.local/share/baloo/file/record.DB lr-x------. 1 rz rz 64 Jan 14 11:14 966 -> /home/rz/.local/share/baloo/file/termlist.DB lr-x------. 1 rz rz 64 Jan 14 11:14 967 -> /home/rz/.local/share/baloo/file/position.DB lr-x------. 1 rz rz 64 Jan 14 11:14 968 -> /home/rz/.local/share/baloo/file/postlist.DB lr-x------. 1 rz rz 64 Jan 14 11:14 969 -> /home/rz/.local/share/baloo/file/record.DB lr-x------. 1 rz rz 64 Jan 14 11:14 97 -> /home/rz/.local/share/baloo/file/record.DB lr-x------. 1 rz rz 64 Jan 14 11:14 970 -> /home/rz/.local/share/baloo/file/termlist.DB lr-x------. 1 rz rz 64 Jan 14 11:14 971 -> /home/rz/.local/share/baloo/file/position.DB lr-x------. 1 rz rz 64 Jan 14 11:14 972 -> /home/rz/.local/share/baloo/file/postlist.DB lr-x------. 1 rz rz 64 Jan 14 11:14 973 -> /home/rz/.local/share/baloo/file/record.DB lr-x------. 1 rz rz 64 Jan 14 11:14 974 -> /home/rz/.local/share/baloo/file/termlist.DB lr-x------. 1 rz rz 64 Jan 14 11:14 975 -> /home/rz/.local/share/baloo/file/position.DB lr-x------. 1 rz rz 64 Jan 14 11:14 976 -> /home/rz/.local/share/baloo/file/postlist.DB lr-x------. 1 rz rz 64 Jan 14 11:14 977 -> /home/rz/.local/share/baloo/file/record.DB lr-x------. 1 rz rz 64 Jan 14 11:14 978 -> /home/rz/.local/share/baloo/file/termlist.DB lr-x------. 1 rz rz 64 Jan 14 11:14 979 -> /home/rz/.local/share/baloo/file/position.DB lr-x------. 1 rz rz 64 Jan 14 11:14 98 -> /home/rz/.local/share/baloo/file/termlist.DB lr-x------. 1 rz rz 64 Jan 14 11:14 980 -> /home/rz/.local/share/baloo/file/postlist.DB lr-x------. 1 rz rz 64 Jan 14 11:14 981 -> /home/rz/.local/share/baloo/file/record.DB lr-x------. 1 rz rz 64 Jan 14 11:14 982 -> /home/rz/.local/share/baloo/file/termlist.DB lr-x------. 1 rz rz 64 Jan 14 11:14 983 -> /home/rz/.local/share/baloo/file/position.DB lr-x------. 1 rz rz 64 Jan 14 11:14 984 -> /home/rz/.local/share/baloo/file/postlist.DB lr-x------. 1 rz rz 64 Jan 14 11:14 985 -> /home/rz/.local/share/baloo/file/record.DB lr-x------. 1 rz rz 64 Jan 14 11:14 986 -> /home/rz/.local/share/baloo/file/termlist.DB lr-x------. 1 rz rz 64 Jan 14 11:14 987 -> /home/rz/.local/share/baloo/file/position.DB lr-x------. 1 rz rz 64 Jan 14 11:14 988 -> /home/rz/.local/share/baloo/file/postlist.DB lr-x------. 1 rz rz 64 Jan 14 11:14 989 -> /home/rz/.local/share/baloo/file/record.DB lr-x------. 1 rz rz 64 Jan 14 11:14 99 -> /home/rz/.local/share/baloo/file/position.DB lr-x------. 1 rz rz 64 Jan 14 11:14 990 -> /home/rz/.local/share/baloo/file/termlist.DB lr-x------. 1 rz rz 64 Jan 14 11:14 991 -> /home/rz/.local/share/baloo/file/position.DB lr-x------. 1 rz rz 64 Jan 14 11:14 992 -> /home/rz/.local/share/baloo/file/postlist.DB lr-x------. 1 rz rz 64 Jan 14 11:14 993 -> /home/rz/.local/share/baloo/file/record.DB lr-x------. 1 rz rz 64 Jan 14 11:14 994 -> /home/rz/.local/share/baloo/file/termlist.DB lr-x------. 1 rz rz 64 Jan 14 11:14 995 -> /home/rz/.local/share/baloo/file/position.DB lr-x------. 1 rz rz 64 Jan 14 11:14 996 -> /home/rz/.local/share/baloo/file/postlist.DB lr-x------. 1 rz rz 64 Jan 14 11:14 997 -> /home/rz/.local/share/baloo/file/record.DB lr-x------. 1 rz rz 64 Jan 14 11:14 998 -> /home/rz/.local/share/baloo/file/termlist.DB lr-x------. 1 rz rz 64 Jan 14 11:14 999 -> /home/rz/.local/share/baloo/file/position.DB
Richard
--- Name and OpenPGP keys available from pgp key servers
-
Dan Vratil -
Piotr Gbyliczek -
Rex Dieter -
Richard Z