Hi

I have made these changes to the directory:

dn: cn=config - nsslapd-sizelimit: 50000 - nsslapd-timelimit: 60 - nsslapd-maxdescriptors: 65535

dn: cn=config, cn=ldbm database, cn=plugins, cn=config - nsslapd-idlistscanlimit: 50000 - nsLookThroughLimit: 50000 - nsslapd-dbcachesize: 838860800 (=800MB) - nsslapd-allidsthreshold: 10000

dn: cn=database_name, cn=ldbm database, cn=plugins, cn=config (in the 26 databases) nsslapd-cachememsize: 125829120 (=120MB)

I have reindexed all databases. But the server freezes when making that query. The server is accepting connections and queries, but not responding them until all the results of the first query are displayed in the client (the results starts to display almost immediately, but keeps 5 minutes displaying the results on the client, and when the display finishes, the other queries are processed).

Any idea? Regards.

2009/10/26 Rich Megginson rmeggins@redhat.com:

Juan Asensio Sánchez wrote:

...

Hi

Samba is making a query to our 389 DS (v. 1.2.2, and too older versions) that makes the servers freeze. The server is running, and accepting connections, although the next queries are not processed until the Samba query is returned. This Samba query takes a long time to be returned, because it is searching all databases and all objects in the directory (more than 20000). The filter is "(&(uid=*)(objectClass=sambaSamAccount))". This query is done when executing the command "net user" from a Windows or Linux machine. This queries are executed manually, and intentionally, but should not make the server freeze. Why is this happening? Is there any option to avoid this?

I think you need to increase your nsslapd-idlistscanlimit - see http://www.redhat.com/docs/manuals/dir-server/8.1/cli/Configuration_Command_... and http://www.redhat.com/docs/manuals/dir-server/8.1/admin/Managing_Indexes.htm...

...

Regards.

-- 389 users mailing list 389-users@redhat.com https://www.redhat.com/mailman/listinfo/fedora-directory-users

-- 389 users mailing list 389-users@redhat.com https://www.redhat.com/mailman/listinfo/fedora-directory-users

Hi, thanks for your answer.

2009/10/27 Andrey Ivanov andrey.ivanov@polytechnique.fr:

Hi,

Do you make the ldapsearch on the same server where ldap server turns?

Yes, sure.

I think your server does not freeze. When you receive the result search entries the CPU of your server is occupied at 100%.

Yes, if I monitor the server, ONE if the cores is at 100%. I mean the LDAP service freezes, not the full server.

If it is a virtual machine that you are using  try to add another cpu.

No, this is a real machine with 8 cores (2 CPUs).

Instead of showing the result on the screen in order to have a more consistent of your test try to redirect it to /dev/null, smth like this :

ldapsearch -Y GSSAPI -h ldap-server.your.domain -b "dc=your,dc=domain" "(objectClass=*)" > /dev/null

From my computer:

time ldapsearch -H ldaps://ldapa1.sacyl.es -D "uid=adminsamba_XXXX,ou=dominio_samba,o=XXXX,dc=XXXXX,dc=XX" -w XXXXXX -b "dc=XXXXXX,dc=XXX" "(&(uid=*)(objectClass=sambasamaccount))" -LLL -x > /dev/null

real 6m23.429s user 0m4.060s sys 0m0.420s

While doing this query, I run one more, and until the first is not finished, the second did not respond.

And do your ldapsearch on another machine, not on the server...

The search is always done from other machine (the Samba server, or my computer).

Regards.

Juan Asensio Sánchez wrote:

Hi, thanks for your answer.

2009/10/27 Andrey Ivanov andrey.ivanov@polytechnique.fr:

...

Hi,

Do you make the ldapsearch on the same server where ldap server turns?

Yes, sure.

...

I think your server does not freeze. When you receive the result search entries the CPU of your server is occupied at 100%.

Yes, if I monitor the server, ONE if the cores is at 100%. I mean the LDAP service freezes, not the full server.

...

If it is a virtual machine that you are using try to add another cpu.

No, this is a real machine with 8 cores (2 CPUs).

...

Instead of showing the result on the screen in order to have a more consistent of your test try to redirect it to /dev/null, smth like this :

ldapsearch -Y GSSAPI -h ldap-server.your.domain -b "dc=your,dc=domain" "(objectClass=*)" > /dev/null

...

From my computer:

time ldapsearch -H ldaps://ldapa1.sacyl.es -D "uid=adminsamba_XXXX,ou=dominio_samba,o=XXXX,dc=XXXXX,dc=XX" -w XXXXXX -b "dc=XXXXXX,dc=XXX" "(&(uid=*)(objectClass=sambasamaccount))" -LLL -x > /dev/null

real 6m23.429s user 0m4.060s sys 0m0.420s

While doing this query, I run one more, and until the first is not finished, the second did not respond.

How many entries match this search filter? Is your nsslapd-idlistscanlimit high enough to hold both all of the uid=* entries and all of the objectClass=sambasamaccount entries?

...

And do your ldapsearch on another machine, not on the server...

The search is always done from other machine (the Samba server, or my computer).

Regards.

-- 389 users mailing list 389-users@redhat.com https://www.redhat.com/mailman/listinfo/fedora-directory-users

6 minutes for 25000 entries is obviously too much. On our server (HP of two years old ang 2Go of memory) 8300 entries are returned in 0.77 seconds (the filter is almost like yours - "(&(uid=*)(objectClass=inetOrgPerson))"). There is certainly some problem either with the disk access or with the memory sizing or with the indexed searches in your configuration... Do you have the PRESENCE index on uid?

2009/10/27 Juan Asensio Sánchez okelet@gmail.com

...

How many entries match this search filter? Is your

nsslapd-idlistscanlimit

...

high enough to hold both all of the uid=* entries and all of the objectClass=sambasamaccount entries?

We have about 25000 sambaSamAccount objects. nsslapd-idlistscanlimit is set to 50000 (total object are about 40000). Databases have been reindexed.

Regards.

-- 389 users mailing list 389-users@redhat.com https://www.redhat.com/mailman/listinfo/fedora-directory-users

Hi

I am already having poor performance when running this query. Any more ideas to try? Could be related due to the data is across almost 30 different databases?

Regards.

El día 28 de octubre de 2009 08:58, Juan Asensio Sánchez okelet@gmail.com escribió:

2009/10/27 Andrey Ivanov andrey.ivanov@polytechnique.fr:

...

6 minutes for 25000 entries is obviously too much. On our server  (HP of two years old ang 2Go of memory)  8300 entries are returned in 0.77 seconds (the filter is almost like yours - "(&(uid=*)(objectClass=inetOrgPerson))"). There is certainly some problem either with the disk access or with the memory sizing or with the indexed searches in your configuration... Do you have the PRESENCE index on uid?

For one moment I thought UID was not indexed, but I checked twice it is indexed. These are all the attributes indexed in out databases:

aci:    system: true    type: [pres]

entryDN:    system: true    type: [eq]

nscpEntryDN:    system: true    type: [eq]

nsds5ReplConflict:    system: true    type: [eq, pres]

nsUniqueId:    system: true    type: [eq]

numSubordinates:    system: true    type: [pres]

objectClass:    system: true    type: [eq, pres]

parentID:    system: true    type: [eq]

cn:    system: false    type: [pres, eq, sub]

displayName:    system: false    type: [pres, eq, sub]

gidNumber:    system: false    type: [pres, eq]

givenName:    system: false    type: [pres, eq, sub]

mail:    system: false    type: [pres, eq, sub]

mailAlternateAddress:    system: false    type: [pres, eq, sub]

memberOf:    system: false    type: [eq]

memberUid:    system: false    type: [eq]

ntUniqueId:    system: false    type: [eq]

ntUserDomainId:    system: false    type: [eq]

o:    system: false    type: [pres, eq, sub]

ou:    system: false    type: [pres, eq, sub]

sambaDomainName:    system: false    type: [pres, eq]

sambaGroupType:    system: false    type: [pres, eq]

sambaPrimaryGroupSID:    system: false    type: [pres, eq]

sambaSID:    system: false    type: [pres, eq]

sambaSIDList:    system: false    type: [pres, eq]

seeAlso:    system: false    type: [pres, eq, sub]

sn:    system: false    type: [pres, eq, sub]

telephoneNumber:    system: false    type: [pres, eq, sub]

uid:    system: false    type: [pres, eq, sub]

uidNumber:    system: false    type: [pres, eq]

uniqueMember:    system: false    type: [pres, eq, sub]

(This is part of a file we use to define database indexes, adding or removing necessary attributes to the default indexes).

Regards.

Hi

Yes, we have 30 different sub-suffixes, each with its own database, that are replicated over 30 differents centers (buildings). The same indexes (shown in a previous post) have been applied to all databases, included userroot, and all databases have been reindexed with those indexes. But that search keeps taking so long time (more than 6 minutes to return about 30000 objects, as said before).

Regards.

2009/11/4 Rich Megginson rmeggins@redhat.com:

Juan Asensio Sánchez wrote:

...

Hi

I am already having poor performance when running this query. Any more ideas to try? Could be related due to the data is across almost 30 different databases?

Could be.  What do you mean by "30 different databases"?  Chaining?  Sub-suffixes?  Can you provide more details?  Note that index configuration is specific to a database - so if you have sub-suffixes with their own databases, you will have to make sure your attributes are indexed correctly in all of those databases.

...

Regards.

El día 28 de octubre de 2009 08:58, Juan Asensio Sánchez okelet@gmail.com escribió:

...

2009/10/27 Andrey Ivanov andrey.ivanov@polytechnique.fr:

...

6 minutes for 25000 entries is obviously too much. On our server  (HP of two years old ang 2Go of memory)  8300 entries are returned in 0.77 seconds (the filter is almost like yours - "(&(uid=*)(objectClass=inetOrgPerson))"). There is certainly some problem either with the disk access or with the memory sizing or with the indexed searches in your configuration... Do you have the PRESENCE index on uid?

For one moment I thought UID was not indexed, but I checked twice it is indexed. These are all the attributes indexed in out databases:

aci:   system: true   type: [pres]

entryDN:   system: true   type: [eq]

nscpEntryDN:   system: true   type: [eq]

nsds5ReplConflict:   system: true   type: [eq, pres]

nsUniqueId:   system: true   type: [eq]

numSubordinates:   system: true   type: [pres]

objectClass:   system: true   type: [eq, pres]

parentID:   system: true   type: [eq]

cn:   system: false   type: [pres, eq, sub]

displayName:   system: false   type: [pres, eq, sub]

gidNumber:   system: false   type: [pres, eq]

givenName:   system: false   type: [pres, eq, sub]

mail:   system: false   type: [pres, eq, sub]

mailAlternateAddress:   system: false   type: [pres, eq, sub]

memberOf:   system: false   type: [eq]

memberUid:   system: false   type: [eq]

ntUniqueId:   system: false   type: [eq]

ntUserDomainId:   system: false   type: [eq]

o:   system: false   type: [pres, eq, sub]

ou:   system: false   type: [pres, eq, sub]

sambaDomainName:   system: false   type: [pres, eq]

sambaGroupType:   system: false   type: [pres, eq]

sambaPrimaryGroupSID:   system: false   type: [pres, eq]

sambaSID:   system: false   type: [pres, eq]

sambaSIDList:   system: false   type: [pres, eq]

seeAlso:   system: false   type: [pres, eq, sub]

sn:   system: false   type: [pres, eq, sub]

telephoneNumber:   system: false   type: [pres, eq, sub]

uid:   system: false   type: [pres, eq, sub]

uidNumber:   system: false   type: [pres, eq]

uniqueMember:   system: false   type: [pres, eq, sub]

(This is part of a file we use to define database indexes, adding or removing necessary attributes to the default indexes).

Regards.

-- 389 users mailing list 389-users@redhat.com https://www.redhat.com/mailman/listinfo/fedora-directory-users

-- 389 users mailing list 389-users@redhat.com https://www.redhat.com/mailman/listinfo/fedora-directory-users