On ma, 29 huhti 2019, John Desantis via FreeIPA-users wrote:
Sumit,
Thanks for your reply.
According to the logs you send there is an error during the first lookup when lookup up the group adglobalposixgroup: Is this really a group from the IPA domain? If yes and the SID is really missing (you can check with 'ipa group-show --all adglobalposixgroup) you might need the re-run the sidgen task to assign a SID.] has no SID, please check the ipaNTSecurityIdentifier attribute on the server-side. /var/log/sssd/sssd_ipa.domain.com.log:(Mon Apr 22 10:58:36 2019) [sssd[be[ipa.domain.com]]] [ipa_s2n_get_list_save_step] (0x0040): ipa_s2n_save_objects failed. /var/log/sssd/sssd_ipa.domain.com.log:(Mon Apr 22 10:58:36 2019) [sssd[be[ipa.domain.com]]] [ipa_s2n_get_list_next] (0x0040): ipa_s2n_get_list_save_step failed. /var/log/sssd/sssd_ipa.domain.com.log:(Mon Apr 22 10:58:36 2019) [sssd[be[ipa.domain.com]]] [ipa_s2n_get_list_done] (0x0040): s2n get_fqlist request failed.
Is this really a group from the IPA domain? If yes and the SID is really missing (you can check with 'ipa group-show --all adglobalposixgroup) you might need the re-run the sidgen task to assign a SID.
Yes, the group was created within the IPA domain via the cli, and this error is only manifest in the client log. However, the GID of the group (10001) is supplied via the AD trust using the POSIX range.
That isn't going to work at all.
For IPA groups POSIX IDs should be in IPA LDAP. You cannot have a non-POSIX group in IPA and POSIX ID supplied from AD LDAP.
Just to rule out any interference, I went ahead and removed the group. Once I did that, I was no longer able to resolve any AD user on the client in question (the IPA servers still had no issue). And, when I re-added the group using its specified GID of 10001, I was able to again resolve the users (with latency) on the client.
However, since this is a new lead, here is what I found in the slapd logs:
errors:[03/Apr/2019:11:23:22.941060612 -0400] - ERR - find_sid_for_ldap_entry - [file ipa_sidgen_common.c, line 522]: Cannot convert Posix ID [10001] into an unused SID. errors:[03/Apr/2019:11:23:23.015868654 -0400] - ERR - ipa_sidgen_add_post_op - [file ipa_sidgen.c, line 149]: Cannot add SID to new entry. errors:[04/Apr/2019:14:41:01.760929261 -0400] - ERR - find_sid_for_ldap_entry - [file ipa_sidgen_common.c, line 514]: Inconsistent objectclasses and attributes, nothing to do. errors:[05/Apr/2019:11:43:24.912535009 -0400] - ERR - find_sid_for_ldap_entry - [file ipa_sidgen_common.c, line 522]: Cannot convert Posix ID [1839611] into an unused SID. errors:[05/Apr/2019:11:43:24.931834879 -0400] - ERR - ipa_sidgen_add_post_op - [file ipa_sidgen.c, line 149]: Cannot add SID to new entry. errors:[29/Apr/2019:12:22:00.486070625 -0400] - ERR - ipa_sidgen_add_post_op - [file ipa_sidgen.c, line 128]: Missing target entry. errors:[29/Apr/2019:12:22:05.005693122 -0400] - ERR - find_sid_for_ldap_entry - [file ipa_sidgen_common.c, line 522]: Cannot convert Posix ID [10001] into an unused SID. errors:[29/Apr/2019:12:22:05.025295150 -0400] - ERR - ipa_sidgen_add_post_op - [file ipa_sidgen.c, line 149]: Cannot add SID to new entry.
These time stamps all correlate when I created the group, changed an override on the user, or deleted the group.
I looked at the URL's https://www.redhat.com/archives/freeipa-users/2017-February/msg00114.html and https://bugzilla.redhat.com/show_bug.cgi?id=1305533, but none of the suggestions functioned; I tried adding an extra ID range to potentially cover the GID in question (since it was supplied from AD), but there was no effect. I also cannot delete it now since it's associated with the trusted domain (I'm sure it's innocuous though).
Can you send the sssd.conf from the IPA server and client used to generate the debug logs (or the current version) as well?
Please find them attached. Currently, they both do function with the IPA server sssd.conf returning results immediately, while the client requires the population of the negative cache prior to returning a result.
Thanks for your help so far!
John DeSantis
Il giorno lun 29 apr 2019 alle ore 05:37 Sumit Bose via FreeIPA-users freeipa-users@lists.fedorahosted.org ha scritto:
On Thu, Apr 25, 2019 at 01:05:52PM -0400, John Desantis via FreeIPA-users wrote:
Hello all,
So, for anyone following this thread, I've been able to make some progress but not enough to consider the configuration production ready.
After watching sssd logs ([domain] debug_level = 10, [sssd] debug_level = 10, and [nss] debug_level = 10) on both the client and server, I am able to reduce by ~50% the time required and failures of user look-ups via `getent passwd` and `id` by configuring the [nss] option 'entry_negative_timeout = 1'.
Hi,
this option controls how long a "failed" lookup will be cached, The default is 15s which means that in your case when the first lookup fails SSSD will reply to every request for the same user during the next 15s with "User does not exists" and will not try to get fresh data from the IPA server. Making this value shorter allows you trigger a new request to the IPA server earlier and hopefully also have a proper result earlier. But it might also cause more unneeded request to the IPA server for users or groups which really do not exist. So I would not recommend to lower this value.
According to the logs you send there is an error during the first lookup when lookup up the group adglobalposixgroup:
/var/log/sssd/sssd_ipa.domain.com.log:(Mon Apr 22 10:58:36 2019) [sssd[be[ipa.domain.com]]] [ipa_s2n_save_objects] (0x0020): Cannot find SID of object. /var/log/sssd/sssd_ipa.domain.com.log:(Mon Apr 22 10:58:36 2019) [sssd[be[ipa.domain.com]]] [ipa_s2n_save_objects] (0x0020): Object [adglobalposixgroup@ipa.domain.com] has no SID, please check the ipaNTSecurityIdentifier attribute on the server-side. /var/log/sssd/sssd_ipa.domain.com.log:(Mon Apr 22 10:58:36 2019) [sssd[be[ipa.domain.com]]] [ipa_s2n_get_list_save_step] (0x0040): ipa_s2n_save_objects failed. /var/log/sssd/sssd_ipa.domain.com.log:(Mon Apr 22 10:58:36 2019) [sssd[be[ipa.domain.com]]] [ipa_s2n_get_list_next] (0x0040): ipa_s2n_get_list_save_step failed. /var/log/sssd/sssd_ipa.domain.com.log:(Mon Apr 22 10:58:36 2019) [sssd[be[ipa.domain.com]]] [ipa_s2n_get_list_done] (0x0040): s2n get_fqlist request failed.
Is this really a group from the IPA domain? If yes and the SID is really missing (you can check with 'ipa group-show --all adglobalposixgroup) you might need the re-run the sidgen task to assign a SID.
Can you send the sssd.conf from the IPA server and client used to generate the debug logs (or the current version) as well?
bye, Sumit
No matter what other options are configured on the client via sssd, the first look-up always fails. The server logs do not indicate that the client checked the server's sssd cache on the first look-up. Only after a negative entry has been introduced and then purged will a "true" client look-up receive a result. I cannot understand why this does not happen on the IPA servers with a default sssd configuration, but happens continually on a client's generated sssd configuration via the IPA installer. Even after removing the additional trusted domains and their ID ranges, the behaviour remains.
In order to rule out the hardware on the client and an older sssd version (1.15.2), I installed on new hardware and with the latest sssd version offered via our satellite server (1.16.2), and the behavior was the same.
Why is the first user lookup-up on the native IPA client failing to retrieve the entry that is properly cached on the IPA server?
Thanks, John DeSantis
Il giorno mer 24 apr 2019 alle ore 08:42 John Desantis desantis@mail.usf.edu ha scritto:
Hello all,
Doh! I realized that I hadn't actually attached the logs; so much for trouble-shooting!
Thanks, John DeSantis
Il giorno lun 22 apr 2019 alle ore 13:07 John Desantis desantis@mail.usf.edu ha scritto:
Hello all,
I've pretty much exhausted my searching in order to find a solution to a problem I've been working on for about a week now, and now I find myself grasping at straws.
Basically, AD trust user lookups on IPA clients fail several times in a row before finally returning results (after 8-20 seconds). However, this does not happen on the IPA servers - even after clearing caches. Furthermore, querying the same list of users against a non IPA Linux client that connects directly to our AD domain using nslcd has no issues querying the same list of users.
From what I understand regarding the anatomy of the FreeIPA - AD Trust relationship, the FreeIPA servers' sssd caches are queried first by FreeIPA clients and if there is no result, then the FreeIPA server queries the AD domain controllers, receives results, caches them, and then provides the results to the FreeIPA client.
I've tried adjusting the sssd.conf file on both the server and the client, without any expected results:
ignore_group_members = True ldap_purge_cache_timeout = (various values) memcache_timeout = (various values) cache_first = (various values) ldap_opt_timeout = (various values) ldap_search_timeout = (various values)
The trust was established using the range type of "ipa-ad-trust-posix" since each user has a unique Posix UID and a shared unique Posix GID (no AD groups are returned).
I've attached logs (dirsrv and sssd) from the IPA server I directly specified via the client sssd.conf and logs from the client itself.
Any pointers and/or suggestions would be extremely helpful!
Thank you, John DeSantis
FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org To unsubscribe send an email to freeipa-users-leave@lists.fedorahosted.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedorahosted.org/archives/list/freeipa-users@lists.fedorahoste...
FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org To unsubscribe send an email to freeipa-users-leave@lists.fedorahosted.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedorahosted.org/archives/list/freeipa-users@lists.fedorahoste...
[domain/ipa.domain.com]
debug_level = 10 krb5_store_password_if_offline = True ipa_domain = ipa.domain.com id_provider = ipa auth_provider = ipa access_provider = ipa ipa_hostname = client.ipa.domain.com chpass_provider = ipa #ipa_server = _srv_, ipaserver1.ipa.domain.com ipa_server = ipaserver2.ipa.domain.com ldap_tls_cacert = /etc/ipa/ca.crt krb5_auth_timeout = 3600
#ignore_group_members = True #ldap_purge_cache_timeout = 0 #ldap_use_tokengroups = false #ldap_search_timeout = 30 #ldap_network_timeout = 30 #ldap_group_nesting_level = 0 #ldap_opt_timeout = 30 #ldap_referrals = false #ipa_subdomains_search_base cn=ad.domain.com,cn=ad,cn=trusts,dc=ipa,dc=domain,dc=com #subdomain_inherit = ldap_purge_cache_timeout,ignore_group_members,ldap_use_tokengroups,ldap_group_nesting_level #reconnection_retries = 12 #get_domains_timeout = 120
[sssd] config_file_version = 2 debug_level = 10 services = nss, sudo, pam, ssh domains = ipa.domain.com #domain_resolution_order = ad.domain.com,ipa.domain.com full_name_format = %1$s
[nss] debug_level = 10 entry_negative_timeout = 1 override_homedir = /home/%l/%u filter_groups = adglobalposixgroup@ad.domain.com
[pam] pam_id_timeout = 3600
[sudo]
[autofs]
[ssh]
[pac]
[ifp]
[secrets]
[domain/ipa.domain.com]
debug_level = 10 krb5_store_password_if_offline = True ipa_domain = ipa.domain.com id_provider = ipa auth_provider = ipa access_provider = ipa ipa_hostname = ipaserver2.ipa.domain.com chpass_provider = ipa ipa_server = ipaserver2.ipa.domain.com ipa_server_mode = True ldap_tls_cacert = /etc/ipa/ca.crt #entry_cache_timeout = 30 #ldap_enumeration_refresh_timeout = 30 #ignore_group_members = True #ldap_purge_cache_timeout = 0 #ldap_use_tokengroups = False #ldap_group_nesting_level = 0 #subdomain_inherit = ignore_group_members,ldap_purge_cache_timeout,ldap_use_tokengroups,ldap_group_nesting_level
[sssd] debug_level = 10 services = nss, sudo, pam, ssh domains = ipa.domain.com #domain_resolution_order = ad.domain.com,ipa.domain.com full_name_format = %1$s
[nss] debug_level = 10 override_homedir = /home/%l/%u #entry_negative_timeout = 1 #filter_groups = *@ad.domain.com
[pam] pam_id_timeout = 3600
[sudo]
[autofs]
[ssh]
[pac]
[ifp]
[secrets]
[session_recording]
FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org To unsubscribe send an email to freeipa-users-leave@lists.fedorahosted.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedorahosted.org/archives/list/freeipa-users@lists.fedorahoste...