Hi Gurus,
we are trying to understand are performance issues and start investigating the ACI's and indexes , I need to know if all "default indexes" showing in 389-console admin are necessary beside the one which we create for our application requirement : - there are a 1/2 dozen of default indexes which are were there with DS installation aka: mailalternateAddress, ntUserDomain, ntUserDomainid, seeAlso,numsubordinates etc... Can we just go an remove this indexes from DS ( using admin console?) , are they been used internally by database ?
-Is there a way to export all indexes in ldif file same we are doing with ACI's , how?
We are seeing running a ldapsearc cmd line for count of groups ( 7000 entries with memberOf plugin enabled) takes aprox 17-18 sec with server cfg fro multimaster replication and this is not acceptable by our users.
we are running 389DS version with OS :Linux proc5-02 2.6.32-431.el6.x86_64 #1 SMP Thu Nov 21 13:35:52 CST 2013 x86_64 x86_64 x86_64 GNU/Linu
389-ds-1.2.2-1.el6.noarch 389-ds-base-libs-1.2.11.15-34.el6_5.x86_64 389-ds-console-doc-1.2.6-1.el6.noarch 389-ds-base-cadc-1.3.3.3-sl6_00.x86_64 389-dsgw-1.1.11-1.el6.x86_64 389-ds-console-1.2.6-1.el6.noarch 389-ds-base-1.2.11.15-34.el6_5.x86_64
Thank you Isabella
Hi Isabella,
we are trying to understand are performance issues and start investigating the ACI's and indexes , I need to know if all "default indexes" showing in 389-console admin are necessary beside the one which we create for our application requirement :
- there are a 1/2 dozen of default indexes which are were there with DS
installation aka: mailalternateAddress, ntUserDomain, ntUserDomainid, seeAlso,numsubordinates etc... Can we just go an remove this indexes from DS ( using admin console?) , are they been used internally by database ?
-Is there a way to export all indexes in ldif file same we are doing with ACI's , how?
You can save the configuration of the indexes from cn=config, and just re-apply them later along with a task to re-index the database.
We are seeing running a ldapsearc cmd line for count of groups ( 7000 entries with memberOf plugin enabled) takes aprox 17-18 sec with server cfg fro multimaster replication and this is not acceptable by our users.
It's unlikely the indexes are the problem here. Indexes tend to create slow write conditions, not slow reads. Can you please give an example of a search from your slapd-<inst>/access log with an etime >0, so that we can see what is being attempted?
I suspect with the times you are quoting, it's likely that there is a lack-of -index on a certain key attribute in your search. You'll probably find that the log entry will have a field saying "notes=U". However, I'd like to see the logs for evidence of this.
On 09/16/2015 12:12 PM, William Brown wrote:
Hi Isabella,
we are trying to understand are performance issues and start investigating the ACI's and indexes , I need to know if all "default indexes" showing in 389-console admin are necessary beside the one which we create for our application requirement :
- there are a 1/2 dozen of default indexes which are were there with DS
installation aka: mailalternateAddress, ntUserDomain, ntUserDomainid, seeAlso,numsubordinates etc... Can we just go an remove this indexes from DS ( using admin console?) , are they been used internally by database ?
-Is there a way to export all indexes in ldif file same we are doing with ACI's , how?
You can save the configuration of the indexes from cn=config, and just re-apply them later along with a task to re-index the database.
We are seeing running a ldapsearc cmd line for count of groups ( 7000 entries with memberOf plugin enabled) takes aprox 17-18 sec with server cfg fro multimaster replication and this is not acceptable by our users.
It's unlikely the indexes are the problem here. Indexes tend to create slow write conditions, not slow reads. Can you please give an example of a search from your slapd-<inst>/access log with an etime >0, so that we can see what is being attempted?
I suspect with the times you are quoting, it's likely that there is a lack-of -index on a certain key attribute in your search. You'll probably find that the log entry will have a field saying "notes=U". However, I'd like to see the logs for evidence of this.
If there are internal searches that are unindexed, you'll need to turn on logging of internal operations in the access log.
-- 389 users mailing list 389-users@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/389-users
Rich, which internal logging you are referring ? I have auditing and access log on ,are other loggin option ? At thius time we suspect the slowness may be related to "memberOf" plugging in groups, we seen this by turning this plugging off and check for time exec again. Are other users seening performance degradation related to this plugging ? Isabella
/ Hi Isabella,
/>/ />>/ we are trying to understand are performance issues and start />>/ investigating the ACI's and indexes , I need to know if all "default />>/ indexes" showing in 389-console admin are necessary beside the one which />>/ we create for our application requirement : />>/ - there are a 1/2 dozen of default indexes which are were there with DS />>/ installation aka: />>/ mailalternateAddress, ntUserDomain, ntUserDomainid, />>/ seeAlso,numsubordinates etc... Can we just go an remove this indexes />>/ from DS ( using admin console?) , are they been used internally by />>/ database ? />>/ />>/ -Is there a way to export all indexes in ldif file same we are doing />>/ with ACI's , how? />/ You can save the configuration of the indexes from cn=config, and just re-apply />/ them later along with a task to re-index the database. />/ />>/ We are seeing running a ldapsearc cmd line for count of groups ( 7000 />>/ entries with memberOf plugin enabled) takes aprox 17-18 sec with server />>/ cfg fro multimaster replication and this is not acceptable by our users. />/ It's unlikely the indexes are the problem here. Indexes tend to create slow />/ write conditions, not slow reads. Can you please give an example of a search />/ from your slapd-<inst>/access log with an etime >0, so that we can see what is />/ being attempted? />/ />/ I suspect with the times you are quoting, it's likely that there is a lack-of />/ -index on a certain key attribute in your search. You'll probably find that the />/ log entry will have a field saying "notes=U". However, I'd like to see the logs />/ for evidence of this. / If there are internal searches that are unindexed, you'll need to turn on logging of internal operations in the access log.
On 09/16/2015 08:59 AM, ghiureai wrote:
Hi Gurus,
we are trying to understand are performance issues and start investigating the ACI's and indexes , I need to know if all "default indexes" showing in 389-console admin are necessary beside the one which we create for our application requirement :
- there are a 1/2 dozen of default indexes which are were there with DS
installation aka: mailalternateAddress, ntUserDomain, ntUserDomainid, seeAlso,numsubordinates etc... Can we just go an remove this indexes from DS ( using admin console?) , are they been used internally by database ?
-Is there a way to export all indexes in ldif file same we are doing with ACI's , how?
We are seeing running a ldapsearc cmd line for count of groups ( 7000 entries with memberOf plugin enabled) takes aprox 17-18 sec with server cfg fro multimaster replication and this is not acceptable by our users.
we are running 389DS version with OS :Linux proc5-02 2.6.32-431.el6.x86_64 #1 SMP Thu Nov 21 13:35:52 CST 2013 x86_64 x86_64 x86_64 GNU/Linu
389-ds-1.2.2-1.el6.noarch 389-ds-base-libs-1.2.11.15-34.el6_5.x86_64 389-ds-console-doc-1.2.6-1.el6.noarch 389-ds-base-cadc-1.3.3.3-sl6_00.x86_64 389-dsgw-1.1.11-1.el6.x86_64 389-ds-console-1.2.6-1.el6.noarch 389-ds-base-1.2.11.15-34.el6_5.x86_64
Thank you Isabella
On 09/17/2015 09:41 AM, ghiureai wrote:
Rich, which internal logging you are referring ? I have auditing and access log on ,are other loggin option ?
https://access.redhat.com/documentation/en-US/Red_Hat_Directory_Server/10/ht...
4 - Logging for internal access operations
At thius time we suspect the slowness may be related to "memberOf" plugging in groups, we seen this by turning this plugging off and check for time exec again. Are other users seening performance degradation related to this plugging ?
Not sure, but let's find out why you are having problems.
Isabella
/Hi Isabella, />//>>/we are trying to understand are performance issues and start />>/investigating the ACI's and indexes , I need to know if all "default />>/indexes" showing in 389-console admin are necessary beside the one which />>/we create for our application requirement : />>/- there are a 1/2 dozen of default indexes which are were there with DS />>/installation aka: />>/mailalternateAddress, ntUserDomain, ntUserDomainid, />>/seeAlso,numsubordinates etc... Can we just go an remove this indexes />>/from DS ( using admin console?) , are they been used internally by />>/database ? />>//>>/-Is there a way to export all indexes in ldif file same we are doing />>/with ACI's , how? />/You can save the configuration of the indexes from cn=config, and just
re-apply />/them later along with a task to re-index the database. />//>>/We are seeing running a ldapsearc cmd line for count of groups ( 7000 />>/entries with memberOf plugin enabled) takes aprox 17-18 sec with server />>/cfg fro multimaster replication and this is not acceptable by our users. />/It's unlikely the indexes are the problem here. Indexes tend to create slow />/write conditions, not slow reads. Can you please give an example of a search />/from your slapd-<inst>/access log with an etime >0, so that we can see what is />/being attempted? />//>/I suspect with the times you are quoting, it's likely that there is a lack-of />/-index on a certain key attribute in your search. You'll probably find that the />/log entry will have a field saying "notes=U". However, I'd like to see the logs />/for evidence of this. / If there are internal searches that are unindexed, you'll need to turn on logging of internal operations in the access log.
On 09/16/2015 08:59 AM, ghiureai wrote:
Hi Gurus,
we are trying to understand are performance issues and start investigating the ACI's and indexes , I need to know if all "default indexes" showing in 389-console admin are necessary beside the one which we create for our application requirement :
- there are a 1/2 dozen of default indexes which are were there with DS
installation aka: mailalternateAddress, ntUserDomain, ntUserDomainid, seeAlso,numsubordinates etc... Can we just go an remove this indexes from DS ( using admin console?) , are they been used internally by database ?
-Is there a way to export all indexes in ldif file same we are doing with ACI's , how?
We are seeing running a ldapsearc cmd line for count of groups ( 7000 entries with memberOf plugin enabled) takes aprox 17-18 sec with server cfg fro multimaster replication and this is not acceptable by our users.
we are running 389DS version with OS :Linux proc5-02 2.6.32-431.el6.x86_64 #1 SMP Thu Nov 21 13:35:52 CST 2013 x86_64 x86_64 x86_64 GNU/Linu
389-ds-1.2.2-1.el6.noarch 389-ds-base-libs-1.2.11.15-34.el6_5.x86_64 389-ds-console-doc-1.2.6-1.el6.noarch 389-ds-base-cadc-1.3.3.3-sl6_00.x86_64 389-dsgw-1.1.11-1.el6.x86_64 389-ds-console-1.2.6-1.el6.noarch 389-ds-base-1.2.11.15-34.el6_5.x86_64
Thank you Isabella
Rich, I turn the error log level to "Plug-ins" seeing this in ldap error log now: 17/Sep/2015:09:16:26 -0700] - cos_cache_vattr_types: failed to get class of service reference [17/Sep/2015:09:16:26 -0700] - cos_cache_vattr_types: failed to get class of service reference [17/Sep/2015:09:16:26 -0700] - cos_cache_vattr_types: failed to get class of service reference [17/Sep/2015:09:16:26 -0700] - cos_cache_vattr_types: failed to get class of service reference [17/Sep/2015:09:16:33 -0700] - cos_cache_vattr_types: failed to get class of service reference [17/Sep/2015:09:16:33 -0700] - cos_cache_vattr_types: failed to get class of service reference [17/Sep/2015:09:16:33 -0700] - cos_cache_vattr_types: failed to get class of service reference [17/Sep/2015:09:16:33 -0700] - cos_cache_vattr_types: failed to get class of service reference [17/Sep/2015:09:16:33 -0700] - cos_cache_vattr_types: failed to get class of service reference [17/Sep/2015:09:16:33 -0700] - cos_cache_vattr_types: failed to get class of service reference [17/Sep/2015:09:16:33 -0700] - cos_cache_vattr_types: failed to get class of service reference [17/Sep/2015:09:16:33 -0700] - cos_cache_vattr_types: failed to get class of service reference [17/Sep/2015:09:16:33 -0700] dse_search - node cn=replica,cn=o\3Dnetscaperoot,cn=mapping tree,cn=config was not found [17/Sep/2015:09:16:33 -0700] dse_search - node cn=replica,cn=dc\3Dtestcanfar,cn=mapping tree,cn=config was not found [17/Sep/2015:09:16:33 -0700] dse_search - node cn=replica,cn=dc\3Dcanfar\2Cdc\3Dnet,cn=mapping tree,cn=config was not found [17/Sep/2015:09:16:37 -0700] NS7bitAttr - MODIFY begin [17/Sep/2015:09:16:37 -0700] roles-plugin - --> roles_post_op [17/Sep/2015:09:16:37 -0700] roles-plugin - --> roles_cache_change_notify [17/Sep/2015:09:16:37 -0700] NS7bitAttr - MODIFY begin [17/Sep/2015:09:16:37 -0700] roles-plugin - <-- roles_cache_change_notify: not a role entry [17/Sep/2015:09:16:37 -0700] roles-plugin - <-- roles_post_op [17/Sep/2015:09:16:37 -0700] roles-plugin - --> roles_post_op [17/Sep/2015:09:16:37 -0700] roles-plugin - --> roles_cache_change_notify [17/Sep/2015:09:16:37 -0700] roles-plugin - <-- roles_cache_change_notify: not a role entry [17/Sep/2015:09:16:37 -0700] roles-plugin - <-- roles_post_op [17/Sep/2015:09:16:37 -0700] NS7bitAttr - MODIFY begin [17/Sep/2015:09:16:37 -0700] roles-plugin - --> roles_post_op [17/Sep/2015:09:16:37 -0700] roles-plugin - --> roles_cache_change_notify [17/Sep/2015:09:16:37 -0700] roles-plugin - <-- roles_cache_change_notify: not a role entry [17/Sep/2015:09:16:37 -0700] roles-plugin - <-- roles_post_op [17/Sep/2015:09:16:37 -0700] NS7bitAttr - MODIFY begin [17/Sep/2015:09:16:37 -0700] roles-plugin - --> roles_post_op [17/Sep/2015:09:16:37 -0700] roles-plugin - --> roles_cache_change_notify [17/Sep/2015:09:16:37 -0700] roles-plugin - <-- roles_cache_change_notify: not a role entry [17/Sep/2015:09:16:37 -0700] roles-plugin - <-- roles_post_op [17/Sep/2015:09:16:38 -0700] NS7bitAttr - MODIFY begin [17/Sep/2015:09:16:38 -0700] roles-plugin - --> roles_post_op [17/Sep/2015:09:16:38 -0700] roles-plugin - --> roles_cache_change_notify [17/Sep/2015:09:16:38 -0700] roles-plugin - <-- roles_cache_change_notify: not a role entry [17/Sep/2015:09:16:38 -0700] roles-plugin - <-- roles_post_op [17/Sep/2015:09:16:38 -0700] NS7bitAttr - MODIFY begin [17/Sep/2015:09:16:38 -0700] roles-plugin - --> roles_post_op [17/Sep/2015:09:16:38 -0700] roles-plugin - --> roles_cache_change_notify [17/Sep/2015:09:16:38 -0700] roles-plugin - <-- roles_cache_change_notify: not a role entry [17/Sep/2015:09:16:38 -0700] roles-plugin - <-- roles_post_op
On 09/17/2015 08:48 AM, Rich Megginson wrote:
On 09/17/2015 09:41 AM, ghiureai wrote:
Rich, which internal logging you are referring ? I have auditing and access log on ,are other loggin option ?
https://access.redhat.com/documentation/en-US/Red_Hat_Directory_Server/10/ht...
4 - Logging for internal access operations
At thius time we suspect the slowness may be related to "memberOf" plugging in groups, we seen this by turning this plugging off and check for time exec again. Are other users seening performance degradation related to this plugging ?
Not sure, but let's find out why you are having problems.
Isabella
/ Hi Isabella,
/>/ />>/ we are trying to understand are performance issues and start />>/ investigating the ACI's and indexes , I need to know if all "default />>/ indexes" showing in 389-console admin are necessary beside the one which />>/ we create for our application requirement : />>/ - there are a 1/2 dozen of default indexes which are were there with DS />>/ installation aka: />>/ mailalternateAddress, ntUserDomain, ntUserDomainid, />>/ seeAlso,numsubordinates etc... Can we just go an remove this indexes />>/ from DS ( using admin console?) , are they been used internally by />>/ database ? />>/ />>/ -Is there a way to export all indexes in ldif file same we are doing />>/ with ACI's , how? />/ You can save the configuration of the indexes from cn=config, and just re-apply />/ them later along with a task to re-index the database. />/ />>/ We are seeing running a ldapsearc cmd line for count of groups ( 7000 />>/ entries with memberOf plugin enabled) takes aprox 17-18 sec with server />>/ cfg fro multimaster replication and this is not acceptable by our users. />/ It's unlikely the indexes are the problem here. Indexes tend to create slow />/ write conditions, not slow reads. Can you please give an example of a search />/ from your slapd-<inst>/access log with an etime >0, so that we can see what is />/ being attempted? />/ />/ I suspect with the times you are quoting, it's likely that there is a lack-of />/ -index on a certain key attribute in your search. You'll probably find that the />/ log entry will have a field saying "notes=U". However, I'd like to see the logs />/ for evidence of this. / If there are internal searches that are unindexed, you'll need to turn on logging of internal operations in the access log.
On 09/16/2015 08:59 AM, ghiureai wrote:
Hi Gurus,
we are trying to understand are performance issues and start investigating the ACI's and indexes , I need to know if all "default indexes" showing in 389-console admin are necessary beside the one which we create for our application requirement :
- there are a 1/2 dozen of default indexes which are were there with DS
installation aka: mailalternateAddress, ntUserDomain, ntUserDomainid, seeAlso,numsubordinates etc... Can we just go an remove this indexes from DS ( using admin console?) , are they been used internally by database ?
-Is there a way to export all indexes in ldif file same we are doing with ACI's , how?
We are seeing running a ldapsearc cmd line for count of groups ( 7000 entries with memberOf plugin enabled) takes aprox 17-18 sec with server cfg fro multimaster replication and this is not acceptable by our users.
we are running 389DS version with OS :Linux proc5-02 2.6.32-431.el6.x86_64 #1 SMP Thu Nov 21 13:35:52 CST 2013 x86_64 x86_64 x86_64 GNU/Linu
389-ds-1.2.2-1.el6.noarch 389-ds-base-libs-1.2.11.15-34.el6_5.x86_64 389-ds-console-doc-1.2.6-1.el6.noarch 389-ds-base-cadc-1.3.3.3-sl6_00.x86_64 389-dsgw-1.1.11-1.el6.x86_64 389-ds-console-1.2.6-1.el6.noarch 389-ds-base-1.2.11.15-34.el6_5.x86_64
Thank you Isabella
On 09/17/2015 10:22 AM, ghiureai wrote:
Rich, I turn the error log level to "Plug-ins" seeing this in ldap error log now:
<snip>
Why did you turn the error log level to "Plug-ins"?
I'm trying to find out if the issue is related to unindexed searches. The plug-ins error log level will not be helpful for that.
You can use logconv.pl to analyze the access log for unindexed searches.
If the unindexed searches are due to internal operations, they will not be logged by default. You have to enable logging for internal access operations as per below. Then you can use logconv.pl to look for them.
On 09/17/2015 08:48 AM, Rich Megginson wrote:
On 09/17/2015 09:41 AM, ghiureai wrote:
Rich, which internal logging you are referring ? I have auditing and access log on ,are other loggin option ?
https://access.redhat.com/documentation/en-US/Red_Hat_Directory_Server/10/ht...
4 - Logging for internal access operations
At thius time we suspect the slowness may be related to "memberOf" plugging in groups, we seen this by turning this plugging off and check for time exec again. Are other users seening performance degradation related to this plugging ?
Not sure, but let's find out why you are having problems.
Isabella
/Hi Isabella, />//>>/we are trying to understand are performance issues and start />>/investigating the ACI's and indexes , I need to know if all "default />>/indexes" showing in 389-console admin are necessary beside the one
which />>/we create for our application requirement : />>/- there are a 1/2 dozen of default indexes which are were there with DS />>/installation aka: />>/mailalternateAddress, ntUserDomain, ntUserDomainid, />>/seeAlso,numsubordinates etc... Can we just go an remove this indexes />>/from DS ( using admin console?) , are they been used internally by />>/database ? />>//>>/-Is there a way to export all indexes in ldif file same we are doing />>/with ACI's , how? />/You can save the configuration of the indexes from cn=config, and just re-apply />/them later along with a task to re-index the database. />//>>/We are seeing running a ldapsearc cmd line for count of groups ( 7000 />>/entries with memberOf plugin enabled) takes aprox 17-18 sec with server />>/cfg fro multimaster replication and this is not acceptable by our users. />/It's unlikely the indexes are the problem here. Indexes tend to create slow />/write conditions, not slow reads. Can you please give an example of a search />/from your slapd-<inst>/access log with an etime >0, so that we can see what is />/being attempted? />//>/I suspect with the times you are quoting, it's likely that there is a lack-of />/-index on a certain key attribute in your search. You'll probably find that the />/log entry will have a field saying "notes=U". However, I'd like to see the logs />/for evidence of this. / If there are internal searches that are unindexed, you'll need to turn on logging of internal operations in the access log.
On 09/16/2015 08:59 AM, ghiureai wrote:
Hi Gurus,
we are trying to understand are performance issues and start investigating the ACI's and indexes , I need to know if all "default indexes" showing in 389-console admin are necessary beside the one which we create for our application requirement :
- there are a 1/2 dozen of default indexes which are were there with DS
installation aka: mailalternateAddress, ntUserDomain, ntUserDomainid, seeAlso,numsubordinates etc... Can we just go an remove this indexes from DS ( using admin console?) , are they been used internally by database ?
-Is there a way to export all indexes in ldif file same we are doing with ACI's , how?
We are seeing running a ldapsearc cmd line for count of groups ( 7000 entries with memberOf plugin enabled) takes aprox 17-18 sec with server cfg fro multimaster replication and this is not acceptable by our users.
we are running 389DS version with OS :Linux proc5-02 2.6.32-431.el6.x86_64 #1 SMP Thu Nov 21 13:35:52 CST 2013 x86_64 x86_64 x86_64 GNU/Linu
389-ds-1.2.2-1.el6.noarch 389-ds-base-libs-1.2.11.15-34.el6_5.x86_64 389-ds-console-doc-1.2.6-1.el6.noarch 389-ds-base-cadc-1.3.3.3-sl6_00.x86_64 389-dsgw-1.1.11-1.el6.x86_64 389-ds-console-1.2.6-1.el6.noarch 389-ds-base-1.2.11.15-34.el6_5.x86_64
Thank you Isabella
389-users@lists.fedoraproject.org
-
ghiureai -
Rich Megginson -
William Brown