We are trying to reindex RHDS 9.1 after importing an updated index. Services were stopped. Started the /usr/lib64/dirsrv/slapd-instance/db2index.
Reindex starts but then consistently reports:
Processed 300,000 entries ( pass 1) Processed 300,000 entries ( pass 2)
and keeps repeating that sequence with same amount of entries. How long does it usually take to complete the process and is there a limit or issue with this feature? This happened once before and system was left overnight performing indexing only to still processing with new passes. We ended up breaking the process and just reinitializing the server.
Thank you, Paul M. Whitney E-mail: paul.whitney@mac.com
On 01/30/2014 09:58 AM, Paul Whitney wrote:
We are trying to reindex RHDS 9.1 after importing an updated index. Services were stopped. Started the /usr/lib64/dirsrv/slapd-instance/db2index.
Reindex starts but then consistently reports:
Processed 300,000 entries ( pass 1) Processed 300,000 entries ( pass 2)
and keeps repeating that sequence with same amount of entries. How long does it usually take to complete the process
Not this long.
and is there a limit or issue with this feature?
No.
This happened once before and system was left overnight performing indexing only to still processing with new passes. We ended up breaking the process and just reinitializing the server.
rpm -q 389-ds-base
Any messages in the errors log?
Thank you, Paul M. Whitney E-mail: paul.whitney@mac.com
-- 389 users mailing list 389-users@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/389-users
rpm -q 389-ds-base
389-ds-base-1.2.11.15-30.el6_5.x86_64
No errors, just a status:
reindex userRoot: Processed 315000 entries (pass 11) -- avg rate 15283456.5/sec, recent rate 0.0/sec. hit ration 0%
Then errors log states threshold has dropped belo 50%, ending pass 11, sweeps files for merging later, then restarts with pass 12.
There do not appear to be any glaring errors, just a constant processing and increment in pass number. Paul M. Whitney E-mail: paul.whitney@mac.com Cell: 410.493.9448
On Jan 30, 2014, at 11:57 AM, Rich Megginson rmeggins@redhat.com wrote:
On 01/30/2014 09:58 AM, Paul Whitney wrote: We are trying to reindex RHDS 9.1 after importing an updated index. Services were stopped. Started the /usr/lib64/dirsrv/slapd-instance/db2index.
Reindex starts but then consistently reports:
Processed 300,000 entries ( pass 1) Processed 300,000 entries ( pass 2)
and keeps repeating that sequence with same amount of entries. How long does it usually take to complete the process
Not this long.
and is there a limit or issue with this feature?
No.
This happened once before and system was left overnight performing indexing only to still processing with new passes. We ended up breaking the process and just reinitializing the server.
rpm -q 389-ds-base
Any messages in the errors log?
Thank you, Paul M. Whitney E-mail: paul.whitney@mac.com
-- 389 users mailing list 389-users@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/389-users
-- 389 users mailing list 389-users@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/389-users
On 1/30/2014 10:18 AM, Paul Whitney wrote:
rpm -q 389-ds-base
389-ds-base-1.2.11.15-30.el6_5.x86_64
No errors, just a status:
reindex userRoot: Processed 315000 entries (pass 11) -- avg rate 15283456.5/sec, recent rate 0.0/sec. hit ration 0%
Then errors log states threshold has dropped belo 50%, ending pass 11, sweeps files for merging later, then restarts with pass 12.
First thing to determine is : is it "stuck" for some reason, or actually performing work ?
Could you note and post here the machine load in the sulking period -- CPU, I/O stats, e.g the output from "iostat -x 1" ?
Also, if you could grab some process thread stack captures (use "pstack" , or gdb) from the indexing process and post the highlights here, that might give some insight into what's going on.
The output from "strace -p xxx" run on the indexing process could also be useful, but probably less useful than the information mentioned above.
On 01/30/2014 10:17 AM, David Boreham wrote:
On 1/30/2014 10:18 AM, Paul Whitney wrote:
rpm -q 389-ds-base
389-ds-base-1.2.11.15-30.el6_5.x86_64
No errors, just a status:
reindex userRoot: Processed 315000 entries (pass 11) -- avg rate 15283456.5/sec, recent rate 0.0/sec. hit ration 0%
Then errors log states threshold has dropped belo 50%, ending pass 11, sweeps files for merging later, then restarts with pass 12.
First thing to determine is : is it "stuck" for some reason, or actually performing work ?
Could you note and post here the machine load in the sulking period -- CPU, I/O stats, e.g the output from "iostat -x 1" ?
Also, if you could grab some process thread stack captures (use "pstack" , or gdb) from the indexing process and post the highlights here, that might give some insight into what's going on.
Yes, using the Debugging_Hangs instructions: http://port389.org/wiki/FAQ#Debugging_Hangs
The output from "strace -p xxx" run on the indexing process could also be useful, but probably less useful than the information mentioned above.
-- 389 users mailing list 389-users@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/389-users
Guys I appreciate you help in this issue. I unfortunately am hosting on a disconnected network and cannot post any of the information you are requesting without in essence "re-typing" it all in. In the interest of saving time, I am just reinitializing the directory server. In the past this has worked.
For the record, this db2index function does work fine on the RHDS 8.2. But on both RHDS 9.0 and 9.1, it appears to get stuck. Even after killing the process, the server attmepts to resume the process but makes no progress.
Paul M. Whitney E-mail: paul.whitney@mac.com
On Jan 30, 2014, at 12:48 PM, Rich Megginson rmeggins@redhat.com wrote:
On 01/30/2014 10:17 AM, David Boreham wrote: On 1/30/2014 10:18 AM, Paul Whitney wrote: rpm -q 389-ds-base 389-ds-base-1.2.11.15-30.el6_5.x86_64 No errors, just a status: reindex userRoot: Processed 315000 entries (pass 11) -- avg rate 15283456.5/sec, recent rate 0.0/sec. hit ration 0% Then errors log states threshold has dropped belo 50%, ending pass 11, sweeps files for merging later, then restarts with pass 12. First thing to determine is : is it "stuck" for some reason, or actually performing work ? Could you note and post here the machine load in the sulking period -- CPU, I/O stats, e.g the output from "iostat -x 1" ? Also, if you could grab some process thread stack captures (use "pstack" , or gdb) from the indexing process and post the highlights here, that might give some insight into what's going on.
Yes, using the Debugging_Hangs instructions: http://port389.org/wiki/FAQ#Debugging_Hangs
The output from "strace -p xxx" run on the indexing process could also be useful, but probably less useful than the information mentioned above. -- 389 users mailing list 389-users@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/389-users
-- 389 users mailing list 389-users@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/389-users
On 01/30/2014 11:33 AM, Paul Whitney wrote:
Guys I appreciate you help in this issue. I unfortunately am hosting on a disconnected network and cannot post any of the information you are requesting without in essence "re-typing" it all in. In the interest of saving time, I am just reinitializing the directory server. In the past this has worked.
For the record, this db2index function does work fine on the RHDS 8.2. But on both RHDS 9.0 and 9.1, it appears to get stuck. Even after killing the process, the server attmepts to resume the process but makes no progress.
If we cannot get large logs or debug traces, how do you suggest going about solving this problem? Can you give us your databases and index configuration?
I know for a fact that db2index works on RHDS 9.1, with very large databases. So the problem them becomes - what is it about your data and/or configuration that is causing problems?
Paul M. Whitney E-mail: paul.whitney@mac.com
On Jan 30, 2014, at 12:48 PM, Rich Megginson rmeggins@redhat.com wrote:
On 01/30/2014 10:17 AM, David Boreham wrote:
On 1/30/2014 10:18 AM, Paul Whitney wrote:
rpm -q 389-ds-base 389-ds-base-1.2.11.15-30.el6_5.x86_64 No errors, just a status: reindex userRoot: Processed 315000 entries (pass 11) -- avg rate 15283456.5/sec, recent rate 0.0/sec. hit ration 0% Then errors log states threshold has dropped belo 50%, ending pass 11, sweeps files for merging later, then restarts with pass 12.
First thing to determine is : is it "stuck" for some reason, or actually performing work ? Could you note and post here the machine load in the sulking period -- CPU, I/O stats, e.g the output from "iostat -x 1" ? Also, if you could grab some process thread stack captures (use "pstack" , or gdb) from the indexing process and post the highlights here, that might give some insight into what's going on.
Yes, using the Debugging_Hangs instructions: http://port389.org/wiki/FAQ#Debugging_Hangs
The output from "strace -p xxx" run on the indexing process could also be useful, but probably less useful than the information mentioned above. -- 389 users mailing list 389-users@lists.fedoraproject.org mailto:389-users@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/389-users
-- 389 users mailing list 389-users@lists.fedoraproject.org mailto:389-users@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/389-users
-- 389 users mailing list 389-users@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/389-users
That is a very good question. No cannot provide any of those items. I was just hoping maybe there would be some "known undocumented trick" to kickstart this server. Perhaps the next time Kent L. comes out here he can take a look and see if there is anything that stands out. Paul M. Whitney E-mail: paul.whitney@mac.com
On Jan 30, 2014, at 01:32 PM, Rich Megginson rmeggins@redhat.com wrote:
On 01/30/2014 11:33 AM, Paul Whitney wrote: Guys I appreciate you help in this issue. I unfortunately am hosting on a disconnected network and cannot post any of the information you are requesting without in essence "re-typing" it all in. In the interest of saving time, I am just reinitializing the directory server. In the past this has worked.
For the record, this db2index function does work fine on the RHDS 8.2. But on both RHDS 9.0 and 9.1, it appears to get stuck. Even after killing the process, the server attmepts to resume the process but makes no progress.
If we cannot get large logs or debug traces, how do you suggest going about solving this problem? Can you give us your databases and index configuration?
I know for a fact that db2index works on RHDS 9.1, with very large databases. So the problem them becomes - what is it about your data and/or configuration that is causing problems?
Paul M. Whitney E-mail: paul.whitney@mac.com
On Jan 30, 2014, at 12:48 PM, Rich Megginson rmeggins@redhat.com wrote:
On 01/30/2014 10:17 AM, David Boreham wrote: On 1/30/2014 10:18 AM, Paul Whitney wrote: rpm -q 389-ds-base 389-ds-base-1.2.11.15-30.el6_5.x86_64 No errors, just a status: reindex userRoot: Processed 315000 entries (pass 11) -- avg rate 15283456.5/sec, recent rate 0.0/sec. hit ration 0% Then errors log states threshold has dropped belo 50%, ending pass 11, sweeps files for merging later, then restarts with pass 12. First thing to determine is : is it "stuck" for some reason, or actually performing work ? Could you note and post here the machine load in the sulking period -- CPU, I/O stats, e.g the output from "iostat -x 1" ? Also, if you could grab some process thread stack captures (use "pstack" , or gdb) from the indexing process and post the highlights here, that might give some insight into what's going on.
Yes, using the Debugging_Hangs instructions: http://port389.org/wiki/FAQ#Debugging_Hangs
The output from "strace -p xxx" run on the indexing process could also be useful, but probably less useful than the information mentioned above. -- 389 users mailing list 389-users@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/389-users
-- 389 users mailing list 389-users@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/389-users
-- 389 users mailing list 389-users@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/389-users
-- 389 users mailing list 389-users@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/389-users
Hi,
I remeber s similar problem (with a similar DS), where import was looping on the same entry and just starting a new pass again. It was a problem in the import queue regulation, but to closer debug this we would neeed more data. What you could do is to try to change the cache sizes, which can have an effect on buffer size and hit ratio.
Regards, Ludwig
On 01/30/2014 07:57 PM, Paul Whitney wrote:
That is a very good question. No cannot provide any of those items. I was just hoping maybe there would be some "known undocumented trick" to kickstart this server. Perhaps the next time Kent L. comes out here he can take a look and see if there is anything that stands out. Paul M. Whitney E-mail: paul.whitney@mac.com
On Jan 30, 2014, at 01:32 PM, Rich Megginson rmeggins@redhat.com wrote:
On 01/30/2014 11:33 AM, Paul Whitney wrote:
Guys I appreciate you help in this issue. I unfortunately am hosting on a disconnected network and cannot post any of the information you are requesting without in essence "re-typing" it all in. In the interest of saving time, I am just reinitializing the directory server. In the past this has worked.
For the record, this db2index function does work fine on the RHDS 8.2. But on both RHDS 9.0 and 9.1, it appears to get stuck. Even after killing the process, the server attmepts to resume the process but makes no progress.
If we cannot get large logs or debug traces, how do you suggest going about solving this problem? Can you give us your databases and index configuration?
I know for a fact that db2index works on RHDS 9.1, with very large databases. So the problem them becomes - what is it about your data and/or configuration that is causing problems?
Paul M. Whitney E-mail:paul.whitney@mac.com
On Jan 30, 2014, at 12:48 PM, Rich Megginson rmeggins@redhat.com wrote:
On 01/30/2014 10:17 AM, David Boreham wrote:
On 1/30/2014 10:18 AM, Paul Whitney wrote:
rpm -q 389-ds-base 389-ds-base-1.2.11.15-30.el6_5.x86_64 No errors, just a status: reindex userRoot: Processed 315000 entries (pass 11) -- avg rate 15283456.5/sec, recent rate 0.0/sec. hit ration 0% Then errors log states threshold has dropped belo 50%, ending pass 11, sweeps files for merging later, then restarts with pass 12.
First thing to determine is : is it "stuck" for some reason, or actually performing work ? Could you note and post here the machine load in the sulking period -- CPU, I/O stats, e.g the output from "iostat -x 1" ? Also, if you could grab some process thread stack captures (use "pstack" , or gdb) from the indexing process and post the highlights here, that might give some insight into what's going on.
Yes, using the Debugging_Hangs instructions: http://port389.org/wiki/FAQ#Debugging_Hangs
The output from "strace -p xxx" run on the indexing process could also be useful, but probably less useful than the information mentioned above. -- 389 users mailing list 389-users@lists.fedoraproject.org mailto:389-users@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/389-users
-- 389 users mailing list 389-users@lists.fedoraproject.org mailto:389-users@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/389-users
-- 389 users mailing list 389-users@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/389-users
-- 389 users mailing list 389-users@lists.fedoraproject.org mailto:389-users@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/389-users
-- 389 users mailing list 389-users@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/389-users
389-users@lists.fedoraproject.org