So are you telling, your ds-389 isn't responding to simple ldapsearch for instance, even if there is no huge amount of logins to hosts? Just from refreshing cache on host clients? But if you doesn't have sssd (that do kernel-caching of privileges), therefore all your clients every time doing ldapsearch or something like this against ds-389 (but I could be wrong). Though I think ldap is really fast and could stand for thousands of requests. What access and errors logs of DS showing you?
2017-12-11 21:52 GMT+03:00 Aaron Hicks aaron.hicks@nesi.org.nz:
Hi Andrew,
I’m afraid it’s often happening during the initial population if the cache. Also these host are all LDAP only and caching with nscd, as they only need user and group name resolution. This was done to minimise changes to their software image as they’re stateless/diskless hosts.
Get Outlook for iOS https://aka.ms/o0ukef
*From:* Andrew Radygin randrewg@gmail.com *Sent:* Monday, December 11, 2017 7:54:45 PM *To:* FreeIPA users list *Cc:* Aaron Hicks *Subject:* Re: [Freeipa-users] FreeIPA connection limits?
Does sssd caching of privileges is working? I mean, suppose if there is no reply from IPA-server, it should use local cache for existing users.
2017-12-11 0:08 GMT+03:00 Aaron Hicks via FreeIPA-users < freeipa-users@lists.fedorahosted.org>:
Hello the list,
We’ve got a number (hundreds) of hosts inside a private network, these all query the FreeIPA server for user and group information using NAT and a gateway server.
However we’re having issues with the LDAP queries timing out or becoming unresponsive.
Is there a limit on the number of concurrent connections from a single host (e.g. the NAT gateway)?
Is there a way of increasing the number of simultaneous connections to FreeIPA/dirsrv?
Regards,
Aaron
FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org To unsubscribe send an email to freeipa-users-leave@lists.fedo rahosted.org
-- Best regards, Andrew.
Hi Andrew,
Single operations are fine. From the command line names resolve quickly, especially once cached, ldapsearch and other commands work when properly authenticated.
When the hosts behind the NAT process a job, it starts a burst of activity and initiating a large number of LDAP connections (multiple connections per host, about a hundred hosts) to refresh or initialise the credential cache. We’re seeing a large proportion of these initial connections timing out without a response, and the nscd cache not being populated, so then it happens again.
We’re not seeing errors in the FreeIPA or slapd logs either, well nothing that seems to be ‘timeout’ or ‘idle connections’ or ‘connection limit exceeded’ etc.
Regards,
Aaron
From: Andrew Radygin [mailto:randrewg@gmail.com] Sent: Tuesday, 12 December 2017 8:23 AM To: Aaron Hicks aaron.hicks@nesi.org.nz Cc: FreeIPA users list freeipa-users@lists.fedorahosted.org Subject: Re: [Freeipa-users] FreeIPA connection limits?
So are you telling, your ds-389 isn't responding to simple ldapsearch for instance, even if there is no huge amount of logins to hosts? Just from refreshing cache on host clients? But if you doesn't have sssd (that do kernel-caching of privileges), therefore all your clients every time doing ldapsearch or something like this against ds-389 (but I could be wrong).
Though I think ldap is really fast and could stand for thousands of requests.
What access and errors logs of DS showing you?
2017-12-11 21:52 GMT+03:00 Aaron Hicks <aaron.hicks@nesi.org.nz mailto:aaron.hicks@nesi.org.nz >:
Hi Andrew,
I’m afraid it’s often happening during the initial population if the cache. Also these host are all LDAP only and caching with nscd, as they only need user and group name resolution. This was done to minimise changes to their software image as they’re stateless/diskless hosts.
Get Outlook for iOS https://aka.ms/o0ukef
_____
From: Andrew Radygin <randrewg@gmail.com mailto:randrewg@gmail.com > Sent: Monday, December 11, 2017 7:54:45 PM To: FreeIPA users list Cc: Aaron Hicks Subject: Re: [Freeipa-users] FreeIPA connection limits?
Does sssd caching of privileges is working?
I mean, suppose if there is no reply from IPA-server, it should use local cache for existing users.
2017-12-11 0:08 GMT+03:00 Aaron Hicks via FreeIPA-users <freeipa-users@lists.fedorahosted.org mailto:freeipa-users@lists.fedorahosted.org >:
Hello the list,
We’ve got a number (hundreds) of hosts inside a private network, these all query the FreeIPA server for user and group information using NAT and a gateway server.
However we’re having issues with the LDAP queries timing out or becoming unresponsive.
Is there a limit on the number of concurrent connections from a single host (e.g. the NAT gateway)?
Is there a way of increasing the number of simultaneous connections to FreeIPA/dirsrv?
Regards,
Aaron
_______________________________________________ FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org mailto:freeipa-users@lists.fedorahosted.org To unsubscribe send an email to freeipa-users-leave@lists.fedorahosted.org mailto:freeipa-users-leave@lists.fedorahosted.org
On 12/11/2017 01:46 PM, Aaron Hicks via FreeIPA-users wrote:
When the hosts behind the NAT process a job, it starts a burst of activity and initiating a large number of LDAP connections (multiple connections per host, about a hundred hosts)
That seems like a relatively small number of connections, well within the default configuration's capabilities.
Performance tuning for the directory server for Linux is documented here:
http://directory.fedoraproject.org/docs/389ds/FAQ/performance-tuning.html
http://directory.fedoraproject.org/docs/389ds/howto/howto-systemd.html (see the last section of this page)
...but be aware that some of these settings are already higher on CentOS/RHEL 7 than the documentation suggests. Check the settings on your system before making any changes so that you don't actually impose a smaller limit than is currently in place.
On Tue, Dec 12, 2017 at 10:46:50AM +1300, Aaron Hicks via FreeIPA-users wrote:
Hi Andrew,
Single operations are fine. From the command line names resolve quickly, especially once cached, ldapsearch and other commands work when properly authenticated.
When the hosts behind the NAT process a job, it starts a burst of activity and initiating a large number of LDAP connections (multiple connections per host, about a hundred hosts) to refresh or initialise the credential cache. We’re seeing a large proportion of these initial connections timing out without a response, and the nscd cache not being populated, so then it happens again.
If there are really that many connections you might overrun the number of available worker threads. See nsslapd-threadnumber e.g. on http://directory.fedoraproject.org/docs/389ds/design/autotuning.html how to change the number, iirc the default is 30.
HTH
bye, Sumit
We’re not seeing errors in the FreeIPA or slapd logs either, well nothing that seems to be ‘timeout’ or ‘idle connections’ or ‘connection limit exceeded’ etc.
Regards,
Aaron
From: Andrew Radygin [mailto:randrewg@gmail.com] Sent: Tuesday, 12 December 2017 8:23 AM To: Aaron Hicks aaron.hicks@nesi.org.nz Cc: FreeIPA users list freeipa-users@lists.fedorahosted.org Subject: Re: [Freeipa-users] FreeIPA connection limits?
So are you telling, your ds-389 isn't responding to simple ldapsearch for instance, even if there is no huge amount of logins to hosts? Just from refreshing cache on host clients? But if you doesn't have sssd (that do kernel-caching of privileges), therefore all your clients every time doing ldapsearch or something like this against ds-389 (but I could be wrong).
Though I think ldap is really fast and could stand for thousands of requests.
What access and errors logs of DS showing you?
2017-12-11 21:52 GMT+03:00 Aaron Hicks <aaron.hicks@nesi.org.nz mailto:aaron.hicks@nesi.org.nz >:
Hi Andrew,
I’m afraid it’s often happening during the initial population if the cache. Also these host are all LDAP only and caching with nscd, as they only need user and group name resolution. This was done to minimise changes to their software image as they’re stateless/diskless hosts.
Get Outlook for iOS https://aka.ms/o0ukef
From: Andrew Radygin <randrewg@gmail.com mailto:randrewg@gmail.com > Sent: Monday, December 11, 2017 7:54:45 PM To: FreeIPA users list Cc: Aaron Hicks Subject: Re: [Freeipa-users] FreeIPA connection limits?
Does sssd caching of privileges is working?
I mean, suppose if there is no reply from IPA-server, it should use local cache for existing users.
2017-12-11 0:08 GMT+03:00 Aaron Hicks via FreeIPA-users <freeipa-users@lists.fedorahosted.org mailto:freeipa-users@lists.fedorahosted.org >:
Hello the list,
We’ve got a number (hundreds) of hosts inside a private network, these all query the FreeIPA server for user and group information using NAT and a gateway server.
However we’re having issues with the LDAP queries timing out or becoming unresponsive.
Is there a limit on the number of concurrent connections from a single host (e.g. the NAT gateway)?
Is there a way of increasing the number of simultaneous connections to FreeIPA/dirsrv?
Regards,
Aaron
FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org mailto:freeipa-users@lists.fedorahosted.org To unsubscribe send an email to freeipa-users-leave@lists.fedorahosted.org mailto:freeipa-users-leave@lists.fedorahosted.org
--
Best regards, Andrew.
--
Best regards, Andrew.
FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org To unsubscribe send an email to freeipa-users-leave@lists.fedorahosted.org
freeipa-users@lists.fedorahosted.org
-
Aaron Hicks -
Andrew Radygin -
Gordon Messmer -
Sumit Bose