I’ve been struggling to get SSH to work with an AD user for over 3 weeks now. I've scraped the bowels of the internet for answers, still no dice.
The issue is pretty simple in itself, I can’t SSH to a freeipa joined Centos client 7.3 with an AD user. However, kinit with any AD users as well as su works just fine. I’m running two 4.4.0 IPA servers.
I made sure the entire setup is resolving DNS properly, NTP(external to freeipa) is in sync. I’m using FQDN for hostnames.
Here’s the output from journalctl -f:
Jul 27 04:37:10 centos.ipa.ad.com sshd[2633]: pam_unix(sshd:session): session opened for user root by (uid=0) Jul 27 04:37:35 centos.ipa.ad.com su[2652]: (to admin@ad.com) root on pts/1 Jul 27 04:37:35 centos.ipa.ad.com su[2652]: pam_unix(su-l:session): session opened for user admin@ad.com by root(uid=0) Jul 27 04:37:42 centos.ipa.ad.com su[2652]: pam_unix(su-l:session): session closed for user admin@ad.com Jul 27 04:38:35 centos.ipa.ad.com sshd[2677]: pam_sss(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruse r= rhost=localhost user=admin@ad.com Jul 27 04:38:35 centos.ipa.ad.com sshd[2677]: pam_sss(sshd:auth): received for user admin@ad.com: 6 (Permission denied) Jul 27 04:38:35 centos.ipa.ad.com sshd[2674]: error: PAM: Authentication failure for admin@ad.com from localhost Jul 27 04:38:38 centos.ipa.ad.com sshd[2674]: Connection closed by ::1 [preauth]
Config files:
/etc/krb5.conf
#File modified by ipa-client-install
includedir /etc/krb5.conf.d/
includedir /var/lib/sss/pubconf/krb5.include.d/
[libdefaults] default_realm = IP.AD.COM dns_lookup_realm = true dns_lookup_kdc = true rdns = false ticket_lifetime = 24h forwardable = true udp_preference_limit = 0 default_ccache_name = KEYRING:persistent:%{uid} [realms] IP.AD.COM = { pkinit_anchors = FILE:/etc/ipa/ca.crt
}
/etc/sssd/sssd.conf
[domain/ipa.ad.com] debug_level = 9 cache_credentials = True krb5_store_password_if_offline = True ipa_domain = ipa.ad.com id_provider = ipa auth_provider = ipa access_provider = ipa ipa_hostname = centos.ipa.ad.com chpass_provider = ipa dyndns_update = True ipa_server = _srv_, ipaserver02.ipa.ad.com dyndns_iface = ens192 ldap_tls_cacert = /etc/ipa/ca.crt[sssd] services = nss, sudo, pam, ssh debug_level = 9 domains = ipa.ad.com [nss] homedir_substring = /home
[pam] debug_level= 9
[sudo]
[autofs]
[ssh] debug_level=9
[pac]
[ifp]
/etc/ssh/sshd_config
HostKey /etc/ssh/ssh_host_rsa_key HostKey /etc/ssh/ssh_host_ecdsa_key HostKey /etc/ssh/ssh_host_ed25519_key SyslogFacility AUTHPRIV PermitRootLogin yes AuthorizedKeysFile .ssh/authorized_keys GSSAPICleanupCredentials no X11Forwarding yes UsePrivilegeSeparation sandbox # Default for new installations. AcceptEnv LANG LC_CTYPE LC_NUMERIC LC_TIME LC_COLLATE LC_MONETARY LC_MESSAGES AcceptEnv LC_PAPER LC_NAME LC_ADDRESS LC_TELEPHONE LC_MEASUREMENT AcceptEnv LC_IDENTIFICATION LC_ALL LANGUAGE AcceptEnv XMODIFIERS Subsystem sftp /usr/libexec/openssh/sftp-server KerberosAuthentication no PubkeyAuthentication yes UsePAM yes AuthorizedKeysCommand /usr/bin/sss_ssh_authorizedkeys GSSAPIAuthentication yes ChallengeResponseAuthentication yes AuthorizedKeysCommandUser nobody
I uploaded krb5_child.log and ldap_child.log to https://1drv.ms/f/s!AlZwwyQE2ZZ5p2b5ROa15PBkAEQD
I managed to ssh AD user login to works on both my freeipa servers. I had to modify the following files See changes in bold.
/etc/krb5.conf
includedir /etc/krb5.conf.d/ includedir /var/lib/sss/pubconf/krb5.include.d/
[logging] default = FILE:/var/log/krb5libs.log kdc = FILE:/var/log/krb5kdc.log admin_server = FILE:/var/log/kadmind.log
[libdefaults] default_realm = IPA.AD.COM * dns_lookup_realm = true* * dns_lookup_kdc = true* rdns = false ticket_lifetime = 24h forwardable = true udp_preference_limit = 0 default_ccache_name = KEYRING:persistent:%{uid}
[realms] IPA.AD.COM = { kdc = ipaserver01.ipa.ad.com:88 master_kdc = ipaserver01.ipa.ad.com:88 admin_server = ipaserver01.ipa.ad.com:749 default_domain = ipa.ad.com pkinit_anchors = FILE:/etc/ipa/ca.crt * auth_to_local = RULE:[1:$1@$0](^.*@AD.COM http://AD.COM)s/@AD.COM/@ad.com/ http://AD.COM/@ad.com/* * auth_to_local = DEFAULT* }
[domain_realm] .ipa.ad.com = IPA.AD.COM ipa.ad.com = IPA.AD.COM ipaserver02.ipa.ad.com = IPA.AD.COM
[dbmodules] IPA.AD.COM = { db_library = ipadb.so }
/etc/resolv.conf search ipa.ad.com ad.com nameserver 127.0.0.1 *nameserver 192.168.1.2 #Seconday IPA Server*
In /etc/named.conf, I disabled dnssec-validation(dnssec-validation no;)
Not sure those settings were at all necessary.
Adding the following line sunder the [realms] for krb5.conf on my centos client machine did not make a difference.
auth_to_local = RULE:[1:$1@$0](^.*@AD.COM)s/@AD.COM/@ad.com/ auth_to_local = DEFAULT
IPv6 has been disabled in /etc/sysctl.conf
net.ipv6.conf.all.disable_ipv6 = 1 net.ipv6.conf.default.disable_ipv6 = 1
If anyone has an idea what may be the issue or where to look, please reply.
Thanks Alex
On Thu, Jul 27, 2017 at 02:34:06AM -0400, Alexandre Pitre via FreeIPA-users wrote:
I uploaded krb5_child.log and ldap_child.log to https://1drv.ms/f/s!AlZwwyQE2ZZ5p2b5ROa15PBkAEQD
I think the child just times out during TGT validation, see: (Thu Jul 27 06:01:20 2017) [[sssd[krb5_child[2765]]]] [sss_child_krb5_trace_cb] (0x4000): [2765] 1501135280.837647: Sending request (2132 bytes) to AD.COM (Thu Jul 27 06:01:20 2017) [[sssd[krb5_child[2765]]]] [sss_child_krb5_trace_cb] (0x4000): [2765] 1501135280.838622: Resolving hostname RO1-INF-ADS-002.ad.com. (Thu Jul 27 06:01:20 2017) [[sssd[krb5_child[2765]]]] [sss_child_krb5_trace_cb] (0x4000): [2765] 1501135280.839154: Sending initial UDP request to dgram 10.248.40.11:88 (Thu Jul 27 06:01:21 2017) [[sssd[krb5_child[2765]]]] [sss_child_krb5_trace_cb] (0x4000): [2765] 1501135281.840215: Resolving hostname ns1-inf-ads-001.ad.com. (Thu Jul 27 06:01:21 2017) [[sssd[krb5_child[2765]]]] [sss_child_krb5_trace_cb] (0x4000): [2765] 1501135281.841223: Sending initial UDP request to dgram 10.3.200.10:88 (Thu Jul 27 06:01:22 2017) [[sssd[krb5_child[2765]]]] [sss_child_krb5_trace_cb] (0x4000): [2765] 1501135282.842291: Resolving hostname inf-p-sy2-ad-01.ad.com. (Thu Jul 27 06:01:22 2017) [[sssd[krb5_child[2765]]]] [sss_child_krb5_trace_cb] (0x4000): [2765] 1501135282.843245: Sending initial UDP request to dgram 192.168.1.10:88 (Thu Jul 27 06:01:23 2017) [[sssd[krb5_child[2765]]]] [sss_child_krb5_trace_cb] (0x4000): [2765] 1501135283.844311: Resolving hostname inf-p-sy2-ad-02.ad.com. (Thu Jul 27 06:01:23 2017) [[sssd[krb5_child[2765]]]] [sss_child_krb5_trace_cb] (0x4000): [2765] 1501135283.845251: Sending initial UDP request to dgram 192.168.1.11:88 (Thu Jul 27 06:01:24 2017) [[sssd[krb5_child[2765]]]] [sss_child_krb5_trace_cb] (0x4000): [2765] 1501135284.846318: Resolving hostname RO1-INF-ADS-001.ad.com. (Thu Jul 27 06:01:24 2017) [[sssd[krb5_child[2765]]]] [sss_child_krb5_trace_cb] (0x4000): [2765] 1501135284.847243: Sending initial UDP request to dgram 10.248.40.10:88 (Thu Jul 27 06:01:25 2017) [[sssd[krb5_child[2765]]]] [sss_child_krb5_trace_cb] (0x4000): [2765] 1501135285.848311: Resolving hostname ns1-inf-ads-002.ad.com. (Thu Jul 27 06:01:25 2017) [[sssd[krb5_child[2765]]]] [sss_child_krb5_trace_cb] (0x4000): [2765] 1501135285.849256: Sending initial UDP request to dgram 10.3.200.11:88
(This is the last message from PID 2765, so it was probably killed)
If the servers are reachable you can just increase the krb5_child timeout in sssd.conf..
Bull-eye Jakub, that did the trick. I should have posted for help on the mailing list sooner. Thanks you so much, you are saving my ass.
It makes sense to increase the krb5_auth_timeout as my AD domain controllers servers are worldwide. Currently they exist in 3 regions: North America, Europe and Asia.
The weird thing is it seems that when a linux host try to authenticate against my AD, it just randomly select an AD DC from the _kerberos SRV records. Normally, on the windows side, if "sites and services" are setup correctly with subnet defined and binded to sites, a windows client shouldn't try to authenticate against an AD DC that isn't local to his site. This mechanism doesn't seem to apply to my linux hosts. Is it because it's only available for windows hosts ? Is there another way to force linux clients to authenticate against AD DC local to their site ?
For now, I set the krb5_auth_timeout to 120 seconds. I had to completely stop sssd and start it again. A colleague mentioned that sssd has a known issue with restart apparently.
Also, I'm curious about ports requirements. Going from linux hosts to AD, I only authorize 88 TCP/UDP. I believe that's all I need.
Thanks, Alex
On Jul 27, 2017 04:08, "Jakub Hrozek via FreeIPA-users" < freeipa-users@lists.fedorahosted.org> wrote:
On Thu, Jul 27, 2017 at 02:34:06AM -0400, Alexandre Pitre via FreeIPA-users wrote:
I uploaded krb5_child.log and ldap_child.log to https://1drv.ms/f/s!AlZwwyQE2ZZ5p2b5ROa15PBkAEQD
I think the child just times out during TGT validation, see: (Thu Jul 27 06:01:20 2017) [[sssd[krb5_child[2765]]]] [sss_child_krb5_trace_cb] (0x4000): [2765] 1501135280.837647: Sending request (2132 bytes) to AD.COM (Thu Jul 27 06:01:20 2017) [[sssd[krb5_child[2765]]]] [sss_child_krb5_trace_cb] (0x4000): [2765] 1501135280.838622: Resolving hostname RO1-INF-ADS-002.ad.com. (Thu Jul 27 06:01:20 2017) [[sssd[krb5_child[2765]]]] [sss_child_krb5_trace_cb] (0x4000): [2765] 1501135280.839154: Sending initial UDP request to dgram 10.248.40.11:88 (Thu Jul 27 06:01:21 2017) [[sssd[krb5_child[2765]]]] [sss_child_krb5_trace_cb] (0x4000): [2765] 1501135281.840215: Resolving hostname ns1-inf-ads-001.ad.com. (Thu Jul 27 06:01:21 2017) [[sssd[krb5_child[2765]]]] [sss_child_krb5_trace_cb] (0x4000): [2765] 1501135281.841223: Sending initial UDP request to dgram 10.3.200.10:88 (Thu Jul 27 06:01:22 2017) [[sssd[krb5_child[2765]]]] [sss_child_krb5_trace_cb] (0x4000): [2765] 1501135282.842291: Resolving hostname inf-p-sy2-ad-01.ad.com. (Thu Jul 27 06:01:22 2017) [[sssd[krb5_child[2765]]]] [sss_child_krb5_trace_cb] (0x4000): [2765] 1501135282.843245: Sending initial UDP request to dgram 192.168.1.10:88 (Thu Jul 27 06:01:23 2017) [[sssd[krb5_child[2765]]]] [sss_child_krb5_trace_cb] (0x4000): [2765] 1501135283.844311: Resolving hostname inf-p-sy2-ad-02.ad.com. (Thu Jul 27 06:01:23 2017) [[sssd[krb5_child[2765]]]] [sss_child_krb5_trace_cb] (0x4000): [2765] 1501135283.845251: Sending initial UDP request to dgram 192.168.1.11:88 (Thu Jul 27 06:01:24 2017) [[sssd[krb5_child[2765]]]] [sss_child_krb5_trace_cb] (0x4000): [2765] 1501135284.846318: Resolving hostname RO1-INF-ADS-001.ad.com. (Thu Jul 27 06:01:24 2017) [[sssd[krb5_child[2765]]]] [sss_child_krb5_trace_cb] (0x4000): [2765] 1501135284.847243: Sending initial UDP request to dgram 10.248.40.10:88 (Thu Jul 27 06:01:25 2017) [[sssd[krb5_child[2765]]]] [sss_child_krb5_trace_cb] (0x4000): [2765] 1501135285.848311: Resolving hostname ns1-inf-ads-002.ad.com. (Thu Jul 27 06:01:25 2017) [[sssd[krb5_child[2765]]]] [sss_child_krb5_trace_cb] (0x4000): [2765] 1501135285.849256: Sending initial UDP request to dgram 10.3.200.11:88
(This is the last message from PID 2765, so it was probably killed)
If the servers are reachable you can just increase the krb5_child timeout in sssd.conf.. _______________________________________________ FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org To unsubscribe send an email to freeipa-users-leave@lists.fedorahosted.org
On Mon, Jul 31, 2017 at 05:47:11PM -0400, Alexandre Pitre wrote:
Bull-eye Jakub, that did the trick. I should have posted for help on the mailing list sooner. Thanks you so much, you are saving my ass.
It makes sense to increase the krb5_auth_timeout as my AD domain controllers servers are worldwide. Currently they exist in 3 regions: North America, Europe and Asia.
The weird thing is it seems that when a linux host try to authenticate against my AD, it just randomly select an AD DC from the _kerberos SRV records. Normally, on the windows side, if "sites and services" are setup correctly with subnet defined and binded to sites, a windows client shouldn't try to authenticate against an AD DC that isn't local to his site. This mechanism doesn't seem to apply to my linux hosts. Is it because it's only available for windows hosts ? Is there another way to force linux clients to authenticate against AD DC local to their site ?
We haven't implemented the site selection for the clients yet, only for servers, see: https://bugzilla.redhat.com/show_bug.cgi?id=1416528
For now, I set the krb5_auth_timeout to 120 seconds. I had to completely stop sssd and start it again. A colleague mentioned that sssd has a known issue with restart apparently.
I'm not aware of any such issue..
Also, I'm curious about ports requirements. Going from linux hosts to AD, I only authorize 88 TCP/UDP. I believe that's all I need.
Yes, from the clients, that should be enough. The servers need more ports open: https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/7/htm...
Turns out, I'm still getting the same problem. It works right away after I force clean the sssd cache: systemctl stop sssd ; rm -f /var/lib/sss/db/* /var/log/sssd/* ; systemctl start sssd
After some time, trying to log back on the same system I see the login prompt is much quicker when I type aduser@ad.com Instead of getting a simple "Password:" prompt I get aduser@ad.com@ centos.domain.ad.com's password.
If I login as root and stop/start and clean the sssd cache, it start working again.
/var/log/messages is filled with:
centos sssd_be: GSSAPI Error: Unspecified GSS failure. Minor code may provide more information (Server krbtgt/AD.COM@IPA.AD.COM not found in Kerberos database)
Any thoughts ?
Thanks, Alex
On Tue, Aug 1, 2017 at 2:58 AM, Jakub Hrozek jhrozek@redhat.com wrote:
On Mon, Jul 31, 2017 at 05:47:11PM -0400, Alexandre Pitre wrote:
Bull-eye Jakub, that did the trick. I should have posted for help on the mailing list sooner. Thanks you so much, you are saving my ass.
It makes sense to increase the krb5_auth_timeout as my AD domain controllers servers are worldwide. Currently they exist in 3 regions:
North
America, Europe and Asia.
The weird thing is it seems that when a linux host try to authenticate against my AD, it just randomly select an AD DC from the _kerberos SRV records. Normally, on the windows side, if "sites and services" are setup correctly with subnet defined and binded to sites, a windows client shouldn't try to authenticate against an AD DC that isn't local to his site. This mechanism doesn't seem to apply to my linux hosts. Is it because it's only available for windows hosts ? Is there another way to force linux clients to authenticate against AD DC local to their site ?
We haven't implemented the site selection for the clients yet, only for servers, see: https://bugzilla.redhat.com/show_bug.cgi?id=1416528
For now, I set the krb5_auth_timeout to 120 seconds. I had to completely stop sssd and start it again. A colleague mentioned that sssd has a known issue with restart apparently.
I'm not aware of any such issue..
Also, I'm curious about ports requirements. Going from linux hosts to
AD, I
only authorize 88 TCP/UDP. I believe that's all I need.
Yes, from the clients, that should be enough. The servers need more ports open: https://access.redhat.com/documentation/en-US/Red_Hat_ Enterprise_Linux/7/html/Linux_Domain_Identity_Authentication_and_Policy_ Guide/installing-ipa.html#prereq-ports
On 4 Aug 2017, at 23:08, Alexandre Pitre via FreeIPA-users freeipa-users@lists.fedorahosted.org wrote:
Turns out, I'm still getting the same problem. It works right away after I force clean the sssd cache: systemctl stop sssd ; rm -f /var/lib/sss/db/* /var/log/sssd/* ; systemctl start sssd
After some time, trying to log back on the same system I see the login prompt is much quicker when I type aduser@ad.com mailto:aduser@ad.com Instead of getting a simple "Password:" prompt I get aduser@ad.com mailto:aduser@ad.com@centos.domain.ad.com http://centos.domain.ad.com/'s password.
If I login as root and stop/start and clean the sssd cache, it start working again.
Are you sure cleaning the cache is needed? Because I think your issue is different. The fact that you get a faster login prompt and the “Server not found…” message both point to the sssd going offline.
You could run ‘sssctl domain-status’ to show if the domain is online or offline (requires the ‘ifp’ service to be enabled until RHEL-7.4/upstream 1.15.x) or look into the logs for messages like “Going offline”.
/var/log/messages is filled with:
centos sssd_be: GSSAPI Error: Unspecified GSS failure. Minor code may provide more information (Server krbtgt/AD.COM@IPA.AD.COM mailto:AD.COM@IPA.AD.COM not found in Kerberos database)
This is the trust principal. Are you sure all your replicas are either trust agents or you ran “ipa-adtrust-install” on them?
Any thoughts ?
Thanks, Alex
On Tue, Aug 1, 2017 at 2:58 AM, Jakub Hrozek <jhrozek@redhat.com mailto:jhrozek@redhat.com> wrote: On Mon, Jul 31, 2017 at 05:47:11PM -0400, Alexandre Pitre wrote:
Bull-eye Jakub, that did the trick. I should have posted for help on the mailing list sooner. Thanks you so much, you are saving my ass.
It makes sense to increase the krb5_auth_timeout as my AD domain controllers servers are worldwide. Currently they exist in 3 regions: North America, Europe and Asia.
The weird thing is it seems that when a linux host try to authenticate against my AD, it just randomly select an AD DC from the _kerberos SRV records. Normally, on the windows side, if "sites and services" are setup correctly with subnet defined and binded to sites, a windows client shouldn't try to authenticate against an AD DC that isn't local to his site. This mechanism doesn't seem to apply to my linux hosts. Is it because it's only available for windows hosts ? Is there another way to force linux clients to authenticate against AD DC local to their site ?
We haven't implemented the site selection for the clients yet, only for servers, see: https://bugzilla.redhat.com/show_bug.cgi?id=1416528 https://bugzilla.redhat.com/show_bug.cgi?id=1416528
For now, I set the krb5_auth_timeout to 120 seconds. I had to completely stop sssd and start it again. A colleague mentioned that sssd has a known issue with restart apparently.
I'm not aware of any such issue..
Also, I'm curious about ports requirements. Going from linux hosts to AD, I only authorize 88 TCP/UDP. I believe that's all I need.
Yes, from the clients, that should be enough. The servers need more ports open: https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/7/htm... https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/7/html/Linux_Domain_Identity_Authentication_and_Policy_Guide/installing-ipa.html#prereq-ports
-- Alexandre Pitre alexandre.pitre@gmail.com mailto:alexandre.pitre@gmail.com _______________________________________________ FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org mailto:freeipa-users@lists.fedorahosted.org To unsubscribe send an email to freeipa-users-leave@lists.fedorahosted.org mailto:freeipa-users-leave@lists.fedorahosted.org
Clearing the sssd cache make the AD login works for a short while, it's probably not necessary nor "production" ready. Looking at /var/log/sssd/ sssd_domain.ad.com. I do see offline messages:
(Mon Aug 7 15:19:47 2017) [sssd[be[domain.ad.com]]] [sdap_id_op_connect_done] (0x0020): Failed to connect, going offline (5 [Input/output error]) (Mon Aug 7 15:19:47 2017) [sssd[be[domain.ad.com]]] [be_mark_offline] (0x2000): Going offline! (Mon Aug 7 15:19:47 2017) [sssd[be[domain.ad.com]]] [be_mark_offline] (0x2000): Enable check_if_online_ptask. (Mon Aug 7 15:19:47 2017) [sssd[be[domain.ad.com]]] [be_ptask_enable] (0x0400): Task [Check if online (periodic)]: enabling task (Mon Aug 7 15:19:47 2017) [sssd[be[domain.ad.com]]] [be_ptask_schedule] (0x0400): Task [Check if online (periodic)]: scheduling task 65 seconds from now [1502119252] (Mon Aug 7 15:19:47 2017) [sssd[be[domain.ad.com]]] [be_run_offline_cb] (0x0080): Going offline. Running callbacks. (Mon Aug 7 15:19:47 2017) [sssd[be[domain.ad.com]]] [sdap_id_op_connect_done] (0x4000): notify offline to op #1 (Mon Aug 7 15:19:47 2017) [sssd[be[domain.ad.com]]] [ipa_subdomains_refresh_connect_done] (0x0020): Unable to connect to LDAP [11]: Resource temporarily unavailable (Mon Aug 7 15:19:47 2017) [sssd[be[domain.ad.com]]] [ipa_subdomains_refresh_connect_done] (0x0080): No IPA server is available, cannot get the subdomain list while offline (Mon Aug 7 15:19:47 2017) [sssd[be[domain.ad.com]]] [be_ptask_done] (0x0040): Task [Subdomains Refresh]: failed with [1432158212]: SSSD is offline (Mon Aug 7 15:19:47 2017) [sssd[be[domain.ad.com]]] [be_ptask_schedule] (0x0400): Task [Subdomains Refresh]: scheduling task 14400 seconds from now [1502133587] (Mon Aug 7 15:19:47 2017) [sssd[be[domain.ad.com]]] [sdap_id_release_conn_data] (0x4000): releasing unused connection (Mon Aug 7 15:19:47 2017) [sssd[be[domain.ad.com]]] [be_ptask_online_cb] (0x0400): Back end is online
I uploaded the full log file /var/log/sssd/sssd_domain.ad.com https://1drv.ms/f/s!AlZwwyQE2ZZ5p2ZmHLzmeKN7mBJ3
Both my IPA servers looks healthy.AD trust agent/controller server role are installed on both.
ipa trustdomain-find ad.com does return all of my AD domains on both IPA servers.
Thanks, Alex
On Sun, Aug 6, 2017 at 11:07 AM, Jakub Hrozek jhrozek@redhat.com wrote:
On 4 Aug 2017, at 23:08, Alexandre Pitre via FreeIPA-users < freeipa-users@lists.fedorahosted.org> wrote:
Turns out, I'm still getting the same problem. It works right away after I force clean the sssd cache: systemctl stop sssd ; rm -f /var/lib/sss/db/* /var/log/sssd/* ; systemctl start sssd
After some time, trying to log back on the same system I see the login prompt is much quicker when I type aduser@ad.com Instead of getting a simple "Password:" prompt I get aduser@ad.com@ centos.domain.ad.com's password.
If I login as root and stop/start and clean the sssd cache, it start working again.
Are you sure cleaning the cache is needed? Because I think your issue is different. The fact that you get a faster login prompt and the “Server not found…” message both point to the sssd going offline.
You could run ‘sssctl domain-status’ to show if the domain is online or offline (requires the ‘ifp’ service to be enabled until RHEL-7.4/upstream 1.15.x) or look into the logs for messages like “Going offline”.
/var/log/messages is filled with:
centos sssd_be: GSSAPI Error: Unspecified GSS failure. Minor code may provide more information (Server krbtgt/AD.COM@IPA.AD.COM not found in Kerberos database)
This is the trust principal. Are you sure all your replicas are either trust agents or you ran “ipa-adtrust-install” on them?
Any thoughts ?
Thanks, Alex
On Tue, Aug 1, 2017 at 2:58 AM, Jakub Hrozek jhrozek@redhat.com wrote:
On Mon, Jul 31, 2017 at 05:47:11PM -0400, Alexandre Pitre wrote:
Bull-eye Jakub, that did the trick. I should have posted for help on the mailing list sooner. Thanks you so much, you are saving my ass.
It makes sense to increase the krb5_auth_timeout as my AD domain controllers servers are worldwide. Currently they exist in 3 regions:
North
America, Europe and Asia.
The weird thing is it seems that when a linux host try to authenticate against my AD, it just randomly select an AD DC from the _kerberos SRV records. Normally, on the windows side, if "sites and services" are
setup
correctly with subnet defined and binded to sites, a windows client shouldn't try to authenticate against an AD DC that isn't local to his site. This mechanism doesn't seem to apply to my linux hosts. Is it because it's only available for windows hosts ? Is there another way to force linux clients to authenticate against AD DC local to their site ?
We haven't implemented the site selection for the clients yet, only for servers, see: https://bugzilla.redhat.com/show_bug.cgi?id=1416528
For now, I set the krb5_auth_timeout to 120 seconds. I had to completely stop sssd and start it again. A colleague mentioned that sssd has a
known
issue with restart apparently.
I'm not aware of any such issue..
Also, I'm curious about ports requirements. Going from linux hosts to
AD, I
only authorize 88 TCP/UDP. I believe that's all I need.
Yes, from the clients, that should be enough. The servers need more ports open: https://access.redhat.com/documentation/en-US/Red_Hat_Ente rprise_Linux/7/html/Linux_Domain_Identity_Authentication_ and_Policy_Guide/installing-ipa.html#prereq-ports
-- Alexandre Pitre alexandre.pitre@gmail.com _______________________________________________ FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org To unsubscribe send an email to freeipa-users-leave@lists.fedorahosted.org
On 7 Aug 2017, at 18:11, Alexandre Pitre alexandre.pitre@gmail.com wrote:
Clearing the sssd cache make the AD login works for a short while, it's probably not necessary nor "production" ready. Looking at /var/log/sssd/sssd_domain.ad.com http://sssd_domain.ad.com/.
Sure, but that’s not what I meant. What I meant is that just “systemctl restart sssd” clears the online/offline state.
Removing the sssd cache is not needed and can actually be quite dangerous. I wish people just stopped doing that, because in case credentials are cached, removing the cache also removes the credentials, leaving users with no way to authenticate.
I do see offline messages:
(Mon Aug 7 15:19:47 2017) [sssd[be[domain.ad.com http://domain.ad.com/]]] [sdap_id_op_connect_done] (0x0020): Failed to connect, going offline (5 [Input/output error]) (Mon Aug 7 15:19:47 2017) [sssd[be[domain.ad.com http://domain.ad.com/]]] [be_mark_offline] (0x2000): Going offline! (Mon Aug 7 15:19:47 2017) [sssd[be[domain.ad.com http://domain.ad.com/]]] [be_mark_offline] (0x2000): Enable check_if_online_ptask. (Mon Aug 7 15:19:47 2017) [sssd[be[domain.ad.com http://domain.ad.com/]]] [be_ptask_enable] (0x0400): Task [Check if online (periodic)]: enabling task (Mon Aug 7 15:19:47 2017) [sssd[be[domain.ad.com http://domain.ad.com/]]] [be_ptask_schedule] (0x0400): Task [Check if online (periodic)]: scheduling task 65 seconds from now [1502119252] (Mon Aug 7 15:19:47 2017) [sssd[be[domain.ad.com http://domain.ad.com/]]] [be_run_offline_cb] (0x0080): Going offline. Running callbacks. (Mon Aug 7 15:19:47 2017) [sssd[be[domain.ad.com http://domain.ad.com/]]] [sdap_id_op_connect_done] (0x4000): notify offline to op #1 (Mon Aug 7 15:19:47 2017) [sssd[be[domain.ad.com http://domain.ad.com/]]] [ipa_subdomains_refresh_connect_done] (0x0020): Unable to connect to LDAP [11]: Resource temporarily unavailable (Mon Aug 7 15:19:47 2017) [sssd[be[domain.ad.com http://domain.ad.com/]]] [ipa_subdomains_refresh_connect_done] (0x0080): No IPA server is available, cannot get the subdomain list while offline (Mon Aug 7 15:19:47 2017) [sssd[be[domain.ad.com http://domain.ad.com/]]] [be_ptask_done] (0x0040): Task [Subdomains Refresh]: failed with [1432158212]: SSSD is offline (Mon Aug 7 15:19:47 2017) [sssd[be[domain.ad.com http://domain.ad.com/]]] [be_ptask_schedule] (0x0400): Task [Subdomains Refresh]: scheduling task 14400 seconds from now [1502133587] (Mon Aug 7 15:19:47 2017) [sssd[be[domain.ad.com http://domain.ad.com/]]] [sdap_id_release_conn_data] (0x4000): releasing unused connection (Mon Aug 7 15:19:47 2017) [sssd[be[domain.ad.com http://domain.ad.com/]]] [be_ptask_online_cb] (0x0400): Back end is online
I uploaded the full log file /var/log/sssd/sssd_domain.ad.com http://sssd_domain.ad.com/ https://1drv.ms/f/s!AlZwwyQE2ZZ5p2ZmHLzmeKN7mBJ3 https://1drv.ms/f/s!AlZwwyQE2ZZ5p2ZmHLzmeKN7mBJ3
Both my IPA servers looks healthy.AD trust agent/controller server role are installed on both.
ipa trustdomain-find ad.com http://ad.com/ does return all of my AD domains on both IPA servers.
This is the failure in the logs: (Mon Aug 7 14:49:53 2017) [sssd[be[domain.ad.com]]] [sdap_process_result] (0x2000): Trace: end of ldap_result list (Mon Aug 7 14:49:53 2017) [sssd[be[domain.ad.com]]] [write_pipe_handler] (0x0400): All data has been sent! (Mon Aug 7 14:49:53 2017) [sssd[be[domain.ad.com]]] [read_pipe_handler] (0x0400): EOF received, client finished (Mon Aug 7 14:49:53 2017) [sssd[be[domain.ad.com]]] [sdap_get_tgt_recv] (0x0400): Child responded: 0 [FILE:/var/lib/sss/db/ccache_IPA.AD.COM], expired on [1502203793] (Mon Aug 7 14:49:53 2017) [sssd[be[domain.ad.com]]] [sdap_cli_auth_step] (0x0100): expire timeout is 900 (Mon Aug 7 14:49:53 2017) [sssd[be[domain.ad.com]]] [sdap_cli_auth_step] (0x1000): the connection will expire at 1502118293 (Mon Aug 7 14:49:53 2017) [sssd[be[domain.ad.com]]] [sasl_bind_send] (0x0100): Executing sasl bind mech: GSSAPI, user: host/centos.domain.ad.com (Mon Aug 7 14:49:53 2017) [sssd[be[domain.ad.com]]] [sasl_bind_send] (0x0020): ldap_sasl_bind failed (-2)[Local error] (Mon Aug 7 14:49:53 2017) [sssd[be[domain.ad.com]]] [sasl_bind_send] (0x0080): Extended failure message: [SASL(-1): generic failure: GSSAPI Error: Unspecified GSS failure. Minor c ode may provide more information (Server krbtgt/AD.COM@IPA.AD.COM not found in Kerberos database)]
Is your client hostname in the AD domain (centos.domain.ad.com http://centos.domain.ad.com/) or in the IPA domain (ipa.ad.com http://ipa.ad.com/) ?
Thanks, Alex
On Sun, Aug 6, 2017 at 11:07 AM, Jakub Hrozek <jhrozek@redhat.com mailto:jhrozek@redhat.com> wrote:
On 4 Aug 2017, at 23:08, Alexandre Pitre via FreeIPA-users <freeipa-users@lists.fedorahosted.org mailto:freeipa-users@lists.fedorahosted.org> wrote:
Turns out, I'm still getting the same problem. It works right away after I force clean the sssd cache: systemctl stop sssd ; rm -f /var/lib/sss/db/* /var/log/sssd/* ; systemctl start sssd
After some time, trying to log back on the same system I see the login prompt is much quicker when I type aduser@ad.com mailto:aduser@ad.com Instead of getting a simple "Password:" prompt I get aduser@ad.com mailto:aduser@ad.com@centos.domain.ad.com http://centos.domain.ad.com/'s password.
If I login as root and stop/start and clean the sssd cache, it start working again.
Are you sure cleaning the cache is needed? Because I think your issue is different. The fact that you get a faster login prompt and the “Server not found…” message both point to the sssd going offline.
You could run ‘sssctl domain-status’ to show if the domain is online or offline (requires the ‘ifp’ service to be enabled until RHEL-7.4/upstream 1.15.x) or look into the logs for messages like “Going offline”.
/var/log/messages is filled with:
centos sssd_be: GSSAPI Error: Unspecified GSS failure. Minor code may provide more information (Server krbtgt/AD.COM@IPA.AD.COM mailto:AD.COM@IPA.AD.COM not found in Kerberos database)
This is the trust principal. Are you sure all your replicas are either trust agents or you ran “ipa-adtrust-install” on them?
Any thoughts ?
Thanks, Alex
On Tue, Aug 1, 2017 at 2:58 AM, Jakub Hrozek <jhrozek@redhat.com mailto:jhrozek@redhat.com> wrote: On Mon, Jul 31, 2017 at 05:47:11PM -0400, Alexandre Pitre wrote:
Bull-eye Jakub, that did the trick. I should have posted for help on the mailing list sooner. Thanks you so much, you are saving my ass.
It makes sense to increase the krb5_auth_timeout as my AD domain controllers servers are worldwide. Currently they exist in 3 regions: North America, Europe and Asia.
The weird thing is it seems that when a linux host try to authenticate against my AD, it just randomly select an AD DC from the _kerberos SRV records. Normally, on the windows side, if "sites and services" are setup correctly with subnet defined and binded to sites, a windows client shouldn't try to authenticate against an AD DC that isn't local to his site. This mechanism doesn't seem to apply to my linux hosts. Is it because it's only available for windows hosts ? Is there another way to force linux clients to authenticate against AD DC local to their site ?
We haven't implemented the site selection for the clients yet, only for servers, see: https://bugzilla.redhat.com/show_bug.cgi?id=1416528 https://bugzilla.redhat.com/show_bug.cgi?id=1416528
For now, I set the krb5_auth_timeout to 120 seconds. I had to completely stop sssd and start it again. A colleague mentioned that sssd has a known issue with restart apparently.
I'm not aware of any such issue..
Also, I'm curious about ports requirements. Going from linux hosts to AD, I only authorize 88 TCP/UDP. I believe that's all I need.
Yes, from the clients, that should be enough. The servers need more ports open: https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/7/htm... https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/7/html/Linux_Domain_Identity_Authentication_and_Policy_Guide/installing-ipa.html#prereq-ports
-- Alexandre Pitre alexandre.pitre@gmail.com mailto:alexandre.pitre@gmail.com _______________________________________________ FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org mailto:freeipa-users@lists.fedorahosted.org To unsubscribe send an email to freeipa-users-leave@lists.fedorahosted.org mailto:freeipa-users-leave@lists.fedorahosted.org
-- Alexandre Pitre alexandre.pitre@gmail.com mailto:alexandre.pitre@gmail.com
The client is in the IPA domain. Although it's sub-domain of ad.com, I did delegate it and configure the IPA servers as name servers. It uses a different domain suffix than ipa realm which was specified by ipa-client-install:
ipa-client-install -U -p admin -w Passw0rd! --enable-dns-updates --mkhomedir --domain=domain.ad.com --realm=IPA.AD.COM --server= ipaserver01.ipa.ad.com --server=ipaserver02.ipa.ad.com --no-ntp --debug
On Mon, Aug 7, 2017 at 1:00 PM, Jakub Hrozek jhrozek@redhat.com wrote:
On 7 Aug 2017, at 18:11, Alexandre Pitre alexandre.pitre@gmail.com wrote:
Clearing the sssd cache make the AD login works for a short while, it's probably not necessary nor "production" ready. Looking at /var/log/sssd/ sssd_domain.ad.com.
Sure, but that’s not what I meant. What I meant is that just “systemctl restart sssd” clears the online/offline state.
Removing the sssd cache is not needed and can actually be quite dangerous. I wish people just stopped doing that, because in case credentials are cached, removing the cache also removes the credentials, leaving users with no way to authenticate.
I do see offline messages:
(Mon Aug 7 15:19:47 2017) [sssd[be[domain.ad.com]]] [sdap_id_op_connect_done] (0x0020): Failed to connect, going offline (5 [Input/output error]) (Mon Aug 7 15:19:47 2017) [sssd[be[domain.ad.com]]] [be_mark_offline] (0x2000): Going offline! (Mon Aug 7 15:19:47 2017) [sssd[be[domain.ad.com]]] [be_mark_offline] (0x2000): Enable check_if_online_ptask. (Mon Aug 7 15:19:47 2017) [sssd[be[domain.ad.com]]] [be_ptask_enable] (0x0400): Task [Check if online (periodic)]: enabling task (Mon Aug 7 15:19:47 2017) [sssd[be[domain.ad.com]]] [be_ptask_schedule] (0x0400): Task [Check if online (periodic)]: scheduling task 65 seconds from now [1502119252] (Mon Aug 7 15:19:47 2017) [sssd[be[domain.ad.com]]] [be_run_offline_cb] (0x0080): Going offline. Running callbacks. (Mon Aug 7 15:19:47 2017) [sssd[be[domain.ad.com]]] [sdap_id_op_connect_done] (0x4000): notify offline to op #1 (Mon Aug 7 15:19:47 2017) [sssd[be[domain.ad.com]]] [ipa_subdomains_refresh_connect_done] (0x0020): Unable to connect to LDAP [11]: Resource temporarily unavailable (Mon Aug 7 15:19:47 2017) [sssd[be[domain.ad.com]]] [ipa_subdomains_refresh_connect_done] (0x0080): No IPA server is available, cannot get the subdomain list while offline (Mon Aug 7 15:19:47 2017) [sssd[be[domain.ad.com]]] [be_ptask_done] (0x0040): Task [Subdomains Refresh]: failed with [1432158212]: SSSD is offline (Mon Aug 7 15:19:47 2017) [sssd[be[domain.ad.com]]] [be_ptask_schedule] (0x0400): Task [Subdomains Refresh]: scheduling task 14400 seconds from now [1502133587] (Mon Aug 7 15:19:47 2017) [sssd[be[domain.ad.com]]] [sdap_id_release_conn_data] (0x4000): releasing unused connection (Mon Aug 7 15:19:47 2017) [sssd[be[domain.ad.com]]] [be_ptask_online_cb] (0x0400): Back end is online
I uploaded the full log file /var/log/sssd/sssd_domain.ad.com https://1drv.ms/f/s!AlZwwyQE2ZZ5p2ZmHLzmeKN7mBJ3
Both my IPA servers looks healthy.AD trust agent/controller server role are installed on both.
ipa trustdomain-find ad.com does return all of my AD domains on both IPA servers.
This is the failure in the logs: (Mon Aug 7 14:49:53 2017) [sssd[be[domain.ad.com]]] [sdap_process_result] (0x2000): Trace: end of ldap_result list (Mon Aug 7 14:49:53 2017) [sssd[be[domain.ad.com]]] [write_pipe_handler] (0x0400): All data has been sent! (Mon Aug 7 14:49:53 2017) [sssd[be[domain.ad.com]]] [read_pipe_handler] (0x0400): EOF received, client finished (Mon Aug 7 14:49:53 2017) [sssd[be[domain.ad.com]]] [sdap_get_tgt_recv] (0x0400): Child responded: 0 [FILE:/var/lib/sss/db/ccache_IPA.AD.COM], expired on [1502203793] (Mon Aug 7 14:49:53 2017) [sssd[be[domain.ad.com]]] [sdap_cli_auth_step] (0x0100): expire timeout is 900 (Mon Aug 7 14:49:53 2017) [sssd[be[domain.ad.com]]] [sdap_cli_auth_step] (0x1000): the connection will expire at 1502118293 (Mon Aug 7 14:49:53 2017) [sssd[be[domain.ad.com]]] [sasl_bind_send] (0x0100): Executing sasl bind mech: GSSAPI, user: host/ centos.domain.ad.com (Mon Aug 7 14:49:53 2017) [sssd[be[domain.ad.com]]] [sasl_bind_send] (0x0020): ldap_sasl_bind failed (-2)[Local error] (Mon Aug 7 14:49:53 2017) [sssd[be[domain.ad.com]]] [sasl_bind_send] (0x0080): Extended failure message: [SASL(-1): generic failure: GSSAPI Error: Unspecified GSS failure. Minor c ode may provide more information (Server krbtgt/AD.COM@IPA.AD.COM not found in Kerberos database)]
Is your client hostname in the AD domain (centos.domain.ad.com) or in the IPA domain (ipa.ad.com) ?
Thanks, Alex
On Sun, Aug 6, 2017 at 11:07 AM, Jakub Hrozek jhrozek@redhat.com wrote:
On 4 Aug 2017, at 23:08, Alexandre Pitre via FreeIPA-users < freeipa-users@lists.fedorahosted.org> wrote:
Turns out, I'm still getting the same problem. It works right away after I force clean the sssd cache: systemctl stop sssd ; rm -f /var/lib/sss/db/* /var/log/sssd/* ; systemctl start sssd
After some time, trying to log back on the same system I see the login prompt is much quicker when I type aduser@ad.com Instead of getting a simple "Password:" prompt I get aduser@ad.com@ centos.domain.ad.com's password.
If I login as root and stop/start and clean the sssd cache, it start working again.
Are you sure cleaning the cache is needed? Because I think your issue is different. The fact that you get a faster login prompt and the “Server not found…” message both point to the sssd going offline.
You could run ‘sssctl domain-status’ to show if the domain is online or offline (requires the ‘ifp’ service to be enabled until RHEL-7.4/upstream 1.15.x) or look into the logs for messages like “Going offline”.
/var/log/messages is filled with:
centos sssd_be: GSSAPI Error: Unspecified GSS failure. Minor code may provide more information (Server krbtgt/AD.COM@IPA.AD.COM not found in Kerberos database)
This is the trust principal. Are you sure all your replicas are either trust agents or you ran “ipa-adtrust-install” on them?
Any thoughts ?
Thanks, Alex
On Tue, Aug 1, 2017 at 2:58 AM, Jakub Hrozek jhrozek@redhat.com wrote:
On Mon, Jul 31, 2017 at 05:47:11PM -0400, Alexandre Pitre wrote:
Bull-eye Jakub, that did the trick. I should have posted for help on
the
mailing list sooner. Thanks you so much, you are saving my ass.
It makes sense to increase the krb5_auth_timeout as my AD domain controllers servers are worldwide. Currently they exist in 3 regions:
North
America, Europe and Asia.
The weird thing is it seems that when a linux host try to authenticate against my AD, it just randomly select an AD DC from the _kerberos SRV records. Normally, on the windows side, if "sites and services" are
setup
correctly with subnet defined and binded to sites, a windows client shouldn't try to authenticate against an AD DC that isn't local to his site. This mechanism doesn't seem to apply to my linux hosts. Is it because it's only available for windows hosts ? Is there another way to force linux clients to authenticate against AD DC local to their site ?
We haven't implemented the site selection for the clients yet, only for servers, see: https://bugzilla.redhat.com/show_bug.cgi?id=1416528
For now, I set the krb5_auth_timeout to 120 seconds. I had to
completely
stop sssd and start it again. A colleague mentioned that sssd has a
known
issue with restart apparently.
I'm not aware of any such issue..
Also, I'm curious about ports requirements. Going from linux hosts to
AD, I
only authorize 88 TCP/UDP. I believe that's all I need.
Yes, from the clients, that should be enough. The servers need more ports open: https://access.redhat.com/documentation/en-US/Red_Hat_Ente rprise_Linux/7/html/Linux_Domain_Identity_Authentication_and _Policy_Guide/installing-ipa.html#prereq-ports
-- Alexandre Pitre alexandre.pitre@gmail.com _______________________________________________ FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org To unsubscribe send an email to freeipa-users-leave@lists.f edorahosted.org
-- Alexandre Pitre alexandre.pitre@gmail.com
On 7 Aug 2017, at 20:02, Alexandre Pitre via FreeIPA-users freeipa-users@lists.fedorahosted.org wrote:
The client is in the IPA domain. Although it's sub-domain of ad.com http://ad.com/, I did delegate it and configure the IPA servers as name servers. It uses a different domain suffix than ipa realm which was specified by ipa-client-install:
OK, but in the logs, I see: (Mon Aug 7 14:49:53 2017) [sssd[be[domain.ad.com http://domain.ad.com/]]] [sasl_bind_send] (0x0100): Executing sasl bind mech: GSSAPI, user: host/centos.domain.ad.com http://centos.domain.ad.com/
—> centos.domain.ad.com http://centos.domain.ad.com/
If your hosts are in the IPA subdomain, then I would have expected centos.ipa.ad.com
What is the relation between domain.ad.com http://domain.ad.com/ and ad.com http://ad.com/ and ipa.ad.com http://ipa.ad.com/?
(Mon Aug 7 14:49:53 2017) [sssd[be[domain.ad.com http://domain.ad.com/]]] [sasl_bind_send] (0x0020): ldap_sasl_bind failed (-2)[Local error] (Mon Aug 7 14:49:53 2017) [sssd[be[domain.ad.com http://domain.ad.com/]]] [sasl_bind_send] (0x0080): Extended failure message: [SASL(-1): generic failure: GSSAPI Error: Unspecified GSS failure. Minor c ode may provide more information (Server krbtgt/AD.COM@IPA.AD.COM mailto:krbtgt/AD.COM@IPA.AD.COM not found in Kerberos database)]
ipa-client-install -U -p admin -w Passw0rd! --enable-dns-updates --mkhomedir --domain=domain.ad.com http://domain.ad.com/ --realm=IPA.AD.COM http://ipa.ad.com/ --server=ipaserver01.ipa.ad.com http://ipaserver01.ipa.ad.com/ --server=ipaserver02.ipa.ad.com http://ipaserver02.ipa.ad.com/ --no-ntp --debug
On Mon, Aug 7, 2017 at 1:00 PM, Jakub Hrozek <jhrozek@redhat.com mailto:jhrozek@redhat.com> wrote:
On 7 Aug 2017, at 18:11, Alexandre Pitre <alexandre.pitre@gmail.com mailto:alexandre.pitre@gmail.com> wrote:
Clearing the sssd cache make the AD login works for a short while, it's probably not necessary nor "production" ready. Looking at /var/log/sssd/sssd_domain.ad.com http://sssd_domain.ad.com/.
Sure, but that’s not what I meant. What I meant is that just “systemctl restart sssd” clears the online/offline state.
Removing the sssd cache is not needed and can actually be quite dangerous. I wish people just stopped doing that, because in case credentials are cached, removing the cache also removes the credentials, leaving users with no way to authenticate.
I do see offline messages:
(Mon Aug 7 15:19:47 2017) [sssd[be[domain.ad.com http://domain.ad.com/]]] [sdap_id_op_connect_done] (0x0020): Failed to connect, going offline (5 [Input/output error]) (Mon Aug 7 15:19:47 2017) [sssd[be[domain.ad.com http://domain.ad.com/]]] [be_mark_offline] (0x2000): Going offline! (Mon Aug 7 15:19:47 2017) [sssd[be[domain.ad.com http://domain.ad.com/]]] [be_mark_offline] (0x2000): Enable check_if_online_ptask. (Mon Aug 7 15:19:47 2017) [sssd[be[domain.ad.com http://domain.ad.com/]]] [be_ptask_enable] (0x0400): Task [Check if online (periodic)]: enabling task (Mon Aug 7 15:19:47 2017) [sssd[be[domain.ad.com http://domain.ad.com/]]] [be_ptask_schedule] (0x0400): Task [Check if online (periodic)]: scheduling task 65 seconds from now [1502119252] (Mon Aug 7 15:19:47 2017) [sssd[be[domain.ad.com http://domain.ad.com/]]] [be_run_offline_cb] (0x0080): Going offline. Running callbacks. (Mon Aug 7 15:19:47 2017) [sssd[be[domain.ad.com http://domain.ad.com/]]] [sdap_id_op_connect_done] (0x4000): notify offline to op #1 (Mon Aug 7 15:19:47 2017) [sssd[be[domain.ad.com http://domain.ad.com/]]] [ipa_subdomains_refresh_connect_done] (0x0020): Unable to connect to LDAP [11]: Resource temporarily unavailable (Mon Aug 7 15:19:47 2017) [sssd[be[domain.ad.com http://domain.ad.com/]]] [ipa_subdomains_refresh_connect_done] (0x0080): No IPA server is available, cannot get the subdomain list while offline (Mon Aug 7 15:19:47 2017) [sssd[be[domain.ad.com http://domain.ad.com/]]] [be_ptask_done] (0x0040): Task [Subdomains Refresh]: failed with [1432158212]: SSSD is offline (Mon Aug 7 15:19:47 2017) [sssd[be[domain.ad.com http://domain.ad.com/]]] [be_ptask_schedule] (0x0400): Task [Subdomains Refresh]: scheduling task 14400 seconds from now [1502133587] (Mon Aug 7 15:19:47 2017) [sssd[be[domain.ad.com http://domain.ad.com/]]] [sdap_id_release_conn_data] (0x4000): releasing unused connection (Mon Aug 7 15:19:47 2017) [sssd[be[domain.ad.com http://domain.ad.com/]]] [be_ptask_online_cb] (0x0400): Back end is online
I uploaded the full log file /var/log/sssd/sssd_domain.ad.com http://sssd_domain.ad.com/ https://1drv.ms/f/s!AlZwwyQE2ZZ5p2ZmHLzmeKN7mBJ3 https://1drv.ms/f/s!AlZwwyQE2ZZ5p2ZmHLzmeKN7mBJ3
Both my IPA servers looks healthy.AD trust agent/controller server role are installed on both.
ipa trustdomain-find ad.com http://ad.com/ does return all of my AD domains on both IPA servers.
This is the failure in the logs: (Mon Aug 7 14:49:53 2017) [sssd[be[domain.ad.com http://domain.ad.com/]]] [sdap_process_result] (0x2000): Trace: end of ldap_result list (Mon Aug 7 14:49:53 2017) [sssd[be[domain.ad.com http://domain.ad.com/]]] [write_pipe_handler] (0x0400): All data has been sent! (Mon Aug 7 14:49:53 2017) [sssd[be[domain.ad.com http://domain.ad.com/]]] [read_pipe_handler] (0x0400): EOF received, client finished (Mon Aug 7 14:49:53 2017) [sssd[be[domain.ad.com http://domain.ad.com/]]] [sdap_get_tgt_recv] (0x0400): Child responded: 0 [FILE:/var/lib/sss/db/ccache_IPA.AD.COM http://ccache_ipa.ad.com/], expired on [1502203793] (Mon Aug 7 14:49:53 2017) [sssd[be[domain.ad.com http://domain.ad.com/]]] [sdap_cli_auth_step] (0x0100): expire timeout is 900 (Mon Aug 7 14:49:53 2017) [sssd[be[domain.ad.com http://domain.ad.com/]]] [sdap_cli_auth_step] (0x1000): the connection will expire at 1502118293 (Mon Aug 7 14:49:53 2017) [sssd[be[domain.ad.com http://domain.ad.com/]]] [sasl_bind_send] (0x0100): Executing sasl bind mech: GSSAPI, user: host/centos.domain.ad.com http://centos.domain.ad.com/ (Mon Aug 7 14:49:53 2017) [sssd[be[domain.ad.com http://domain.ad.com/]]] [sasl_bind_send] (0x0020): ldap_sasl_bind failed (-2)[Local error] (Mon Aug 7 14:49:53 2017) [sssd[be[domain.ad.com http://domain.ad.com/]]] [sasl_bind_send] (0x0080): Extended failure message: [SASL(-1): generic failure: GSSAPI Error: Unspecified GSS failure. Minor c ode may provide more information (Server krbtgt/AD.COM@IPA.AD.COM mailto:krbtgt/AD.COM@IPA.AD.COM not found in Kerberos database)]
Is your client hostname in the AD domain (centos.domain.ad.com http://centos.domain.ad.com/) or in the IPA domain (ipa.ad.com http://ipa.ad.com/) ?
Thanks, Alex
On Sun, Aug 6, 2017 at 11:07 AM, Jakub Hrozek <jhrozek@redhat.com mailto:jhrozek@redhat.com> wrote:
On 4 Aug 2017, at 23:08, Alexandre Pitre via FreeIPA-users <freeipa-users@lists.fedorahosted.org mailto:freeipa-users@lists.fedorahosted.org> wrote:
Turns out, I'm still getting the same problem. It works right away after I force clean the sssd cache: systemctl stop sssd ; rm -f /var/lib/sss/db/* /var/log/sssd/* ; systemctl start sssd
After some time, trying to log back on the same system I see the login prompt is much quicker when I type aduser@ad.com mailto:aduser@ad.com Instead of getting a simple "Password:" prompt I get aduser@ad.com mailto:aduser@ad.com@centos.domain.ad.com http://centos.domain.ad.com/'s password.
If I login as root and stop/start and clean the sssd cache, it start working again.
Are you sure cleaning the cache is needed? Because I think your issue is different. The fact that you get a faster login prompt and the “Server not found…” message both point to the sssd going offline.
You could run ‘sssctl domain-status’ to show if the domain is online or offline (requires the ‘ifp’ service to be enabled until RHEL-7.4/upstream 1.15.x) or look into the logs for messages like “Going offline”.
/var/log/messages is filled with:
centos sssd_be: GSSAPI Error: Unspecified GSS failure. Minor code may provide more information (Server krbtgt/AD.COM@IPA.AD.COM mailto:AD.COM@IPA.AD.COM not found in Kerberos database)
This is the trust principal. Are you sure all your replicas are either trust agents or you ran “ipa-adtrust-install” on them?
Any thoughts ?
Thanks, Alex
On Tue, Aug 1, 2017 at 2:58 AM, Jakub Hrozek <jhrozek@redhat.com mailto:jhrozek@redhat.com> wrote: On Mon, Jul 31, 2017 at 05:47:11PM -0400, Alexandre Pitre wrote:
Bull-eye Jakub, that did the trick. I should have posted for help on the mailing list sooner. Thanks you so much, you are saving my ass.
It makes sense to increase the krb5_auth_timeout as my AD domain controllers servers are worldwide. Currently they exist in 3 regions: North America, Europe and Asia.
The weird thing is it seems that when a linux host try to authenticate against my AD, it just randomly select an AD DC from the _kerberos SRV records. Normally, on the windows side, if "sites and services" are setup correctly with subnet defined and binded to sites, a windows client shouldn't try to authenticate against an AD DC that isn't local to his site. This mechanism doesn't seem to apply to my linux hosts. Is it because it's only available for windows hosts ? Is there another way to force linux clients to authenticate against AD DC local to their site ?
We haven't implemented the site selection for the clients yet, only for servers, see: https://bugzilla.redhat.com/show_bug.cgi?id=1416528 https://bugzilla.redhat.com/show_bug.cgi?id=1416528
For now, I set the krb5_auth_timeout to 120 seconds. I had to completely stop sssd and start it again. A colleague mentioned that sssd has a known issue with restart apparently.
I'm not aware of any such issue..
Also, I'm curious about ports requirements. Going from linux hosts to AD, I only authorize 88 TCP/UDP. I believe that's all I need.
Yes, from the clients, that should be enough. The servers need more ports open: https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/7/htm... https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/7/html/Linux_Domain_Identity_Authentication_and_Policy_Guide/installing-ipa.html#prereq-ports
-- Alexandre Pitre alexandre.pitre@gmail.com mailto:alexandre.pitre@gmail.com _______________________________________________ FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org mailto:freeipa-users@lists.fedorahosted.org To unsubscribe send an email to freeipa-users-leave@lists.fedorahosted.org mailto:freeipa-users-leave@lists.fedorahosted.org
-- Alexandre Pitre alexandre.pitre@gmail.com mailto:alexandre.pitre@gmail.com
-- Alexandre Pitre alexandre.pitre@gmail.com mailto:alexandre.pitre@gmail.com _______________________________________________ FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org mailto:freeipa-users@lists.fedorahosted.org To unsubscribe send an email to freeipa-users-leave@lists.fedorahosted.org mailto:freeipa-users-leave@lists.fedorahosted.org
If your hosts are in the IPA subdomain, then I would have expected centos.ipa.ad.com
The centos client has a hostname set to centos.domain.ad.com I'm using FQDN hostname based on the required DNS domain, not the IPA kerberos realm. Hence why centos.domain.ad.com.
To explain further more, It'll probably be easier to use ipa.ad.com for all for my clients and not deal with a different DNS domain than the IPA realm. However, we have this business requirement to use a a different domain than the IPA realm. My understanding is that supposed to be supported by using the --domain switch of ipa-client-install: http://www.unix.com/man-page/centos/1/ipa-client-install/ which basically just add static entires in /etc/sssd/sssd.conf and /etc/krb5.conf
Should I approach things differently ?
What is the relation between domain.ad.com and ad.com and ipa.ad.com?
ad.com is my Active Directory domain. domain.ad.com is a sub domain that was delegated from the AD DNS to the freeipa servers ipa.ad.com is also a sub domain that was delegated from the AD DNS to the freeipa servers and is the freeipa native kerberos realm.
On Wed, Aug 9, 2017 at 9:55 AM, Jakub Hrozek jhrozek@redhat.com wrote:
On 7 Aug 2017, at 20:02, Alexandre Pitre via FreeIPA-users < freeipa-users@lists.fedorahosted.org> wrote:
The client is in the IPA domain. Although it's sub-domain of ad.com, I did delegate it and configure the IPA servers as name servers. It uses a different domain suffix than ipa realm which was specified by ipa-client-install:
OK, but in the logs, I see: (Mon Aug 7 14:49:53 2017) [sssd[be[domain.ad.com]]] [sasl_bind_send] (0x0100): Executing sasl bind mech: GSSAPI, user: host/ centos.domain.ad.com
—> centos.domain.ad.com
If your hosts are in the IPA subdomain, then I would have expected centos.ipa.ad.com
What is the relation between domain.ad.com and ad.com and ipa.ad.com?
(Mon Aug 7 14:49:53 2017) [sssd[be[domain.ad.com]]] [sasl_bind_send] (0x0020): ldap_sasl_bind failed (-2)[Local error] (Mon Aug 7 14:49:53 2017) [sssd[be[domain.ad.com]]] [sasl_bind_send] (0x0080): Extended failure message: [SASL(-1): generic failure: GSSAPI Error: Unspecified GSS failure. Minor c ode may provide more information (Server krbtgt/AD.COM@IPA.AD.COM not found in Kerberos database)]
ipa-client-install -U -p admin -w Passw0rd! --enable-dns-updates --mkhomedir --domain=domain.ad.com --realm=IPA.AD.COM http://ipa.ad.com/ --server=ipaserver01.ipa.ad.com --server=ipaserver02.ipa.ad.com --no-ntp --debug
On Mon, Aug 7, 2017 at 1:00 PM, Jakub Hrozek jhrozek@redhat.com wrote:
On 7 Aug 2017, at 18:11, Alexandre Pitre alexandre.pitre@gmail.com wrote:
Clearing the sssd cache make the AD login works for a short while, it's probably not necessary nor "production" ready. Looking at /var/log/sssd/ sssd_domain.ad.com.
Sure, but that’s not what I meant. What I meant is that just “systemctl restart sssd” clears the online/offline state.
Removing the sssd cache is not needed and can actually be quite dangerous. I wish people just stopped doing that, because in case credentials are cached, removing the cache also removes the credentials, leaving users with no way to authenticate.
I do see offline messages:
(Mon Aug 7 15:19:47 2017) [sssd[be[domain.ad.com]]] [sdap_id_op_connect_done] (0x0020): Failed to connect, going offline (5 [Input/output error]) (Mon Aug 7 15:19:47 2017) [sssd[be[domain.ad.com]]] [be_mark_offline] (0x2000): Going offline! (Mon Aug 7 15:19:47 2017) [sssd[be[domain.ad.com]]] [be_mark_offline] (0x2000): Enable check_if_online_ptask. (Mon Aug 7 15:19:47 2017) [sssd[be[domain.ad.com]]] [be_ptask_enable] (0x0400): Task [Check if online (periodic)]: enabling task (Mon Aug 7 15:19:47 2017) [sssd[be[domain.ad.com]]] [be_ptask_schedule] (0x0400): Task [Check if online (periodic)]: scheduling task 65 seconds from now [1502119252] (Mon Aug 7 15:19:47 2017) [sssd[be[domain.ad.com]]] [be_run_offline_cb] (0x0080): Going offline. Running callbacks. (Mon Aug 7 15:19:47 2017) [sssd[be[domain.ad.com]]] [sdap_id_op_connect_done] (0x4000): notify offline to op #1 (Mon Aug 7 15:19:47 2017) [sssd[be[domain.ad.com]]] [ipa_subdomains_refresh_connect_done] (0x0020): Unable to connect to LDAP [11]: Resource temporarily unavailable (Mon Aug 7 15:19:47 2017) [sssd[be[domain.ad.com]]] [ipa_subdomains_refresh_connect_done] (0x0080): No IPA server is available, cannot get the subdomain list while offline (Mon Aug 7 15:19:47 2017) [sssd[be[domain.ad.com]]] [be_ptask_done] (0x0040): Task [Subdomains Refresh]: failed with [1432158212]: SSSD is offline (Mon Aug 7 15:19:47 2017) [sssd[be[domain.ad.com]]] [be_ptask_schedule] (0x0400): Task [Subdomains Refresh]: scheduling task 14400 seconds from now [1502133587] (Mon Aug 7 15:19:47 2017) [sssd[be[domain.ad.com]]] [sdap_id_release_conn_data] (0x4000): releasing unused connection (Mon Aug 7 15:19:47 2017) [sssd[be[domain.ad.com]]] [be_ptask_online_cb] (0x0400): Back end is online
I uploaded the full log file /var/log/sssd/sssd_domain.ad.com https://1drv.ms/f/s!AlZwwyQE2ZZ5p2ZmHLzmeKN7mBJ3
Both my IPA servers looks healthy.AD trust agent/controller server role are installed on both.
ipa trustdomain-find ad.com does return all of my AD domains on both IPA servers.
This is the failure in the logs: (Mon Aug 7 14:49:53 2017) [sssd[be[domain.ad.com]]] [sdap_process_result] (0x2000): Trace: end of ldap_result list (Mon Aug 7 14:49:53 2017) [sssd[be[domain.ad.com]]] [write_pipe_handler] (0x0400): All data has been sent! (Mon Aug 7 14:49:53 2017) [sssd[be[domain.ad.com]]] [read_pipe_handler] (0x0400): EOF received, client finished (Mon Aug 7 14:49:53 2017) [sssd[be[domain.ad.com]]] [sdap_get_tgt_recv] (0x0400): Child responded: 0 [FILE:/var/lib/sss/db/ccache_IPA.AD.COM http://ccache_ipa.ad.com/], expired on [1502203793] (Mon Aug 7 14:49:53 2017) [sssd[be[domain.ad.com]]] [sdap_cli_auth_step] (0x0100): expire timeout is 900 (Mon Aug 7 14:49:53 2017) [sssd[be[domain.ad.com]]] [sdap_cli_auth_step] (0x1000): the connection will expire at 1502118293 (Mon Aug 7 14:49:53 2017) [sssd[be[domain.ad.com]]] [sasl_bind_send] (0x0100): Executing sasl bind mech: GSSAPI, user: host/ centos.domain.ad.com (Mon Aug 7 14:49:53 2017) [sssd[be[domain.ad.com]]] [sasl_bind_send] (0x0020): ldap_sasl_bind failed (-2)[Local error] (Mon Aug 7 14:49:53 2017) [sssd[be[domain.ad.com]]] [sasl_bind_send] (0x0080): Extended failure message: [SASL(-1): generic failure: GSSAPI Error: Unspecified GSS failure. Minor c ode may provide more information (Server krbtgt/AD.COM@IPA.AD.COM not found in Kerberos database)]
Is your client hostname in the AD domain (centos.domain.ad.com) or in the IPA domain (ipa.ad.com) ?
Thanks, Alex
On Sun, Aug 6, 2017 at 11:07 AM, Jakub Hrozek jhrozek@redhat.com wrote:
On 4 Aug 2017, at 23:08, Alexandre Pitre via FreeIPA-users < freeipa-users@lists.fedorahosted.org> wrote:
Turns out, I'm still getting the same problem. It works right away after I force clean the sssd cache: systemctl stop sssd ; rm -f /var/lib/sss/db/* /var/log/sssd/* ; systemctl start sssd
After some time, trying to log back on the same system I see the login prompt is much quicker when I type aduser@ad.com Instead of getting a simple "Password:" prompt I get aduser@ad.com@ centos.domain.ad.com's password.
If I login as root and stop/start and clean the sssd cache, it start working again.
Are you sure cleaning the cache is needed? Because I think your issue is different. The fact that you get a faster login prompt and the “Server not found…” message both point to the sssd going offline.
You could run ‘sssctl domain-status’ to show if the domain is online or offline (requires the ‘ifp’ service to be enabled until RHEL-7.4/upstream 1.15.x) or look into the logs for messages like “Going offline”.
/var/log/messages is filled with:
centos sssd_be: GSSAPI Error: Unspecified GSS failure. Minor code may provide more information (Server krbtgt/AD.COM@IPA.AD.COM not found in Kerberos database)
This is the trust principal. Are you sure all your replicas are either trust agents or you ran “ipa-adtrust-install” on them?
Any thoughts ?
Thanks, Alex
On Tue, Aug 1, 2017 at 2:58 AM, Jakub Hrozek jhrozek@redhat.com wrote:
On Mon, Jul 31, 2017 at 05:47:11PM -0400, Alexandre Pitre wrote:
Bull-eye Jakub, that did the trick. I should have posted for help on
the
mailing list sooner. Thanks you so much, you are saving my ass.
It makes sense to increase the krb5_auth_timeout as my AD domain controllers servers are worldwide. Currently they exist in 3 regions:
North
America, Europe and Asia.
The weird thing is it seems that when a linux host try to authenticate against my AD, it just randomly select an AD DC from the _kerberos
SRV
records. Normally, on the windows side, if "sites and services" are
setup
correctly with subnet defined and binded to sites, a windows client shouldn't try to authenticate against an AD DC that isn't local to his site. This mechanism doesn't seem to apply to my linux hosts. Is it because it's only available for windows hosts ? Is there another way
to
force linux clients to authenticate against AD DC local to their site
?
We haven't implemented the site selection for the clients yet, only for servers, see: https://bugzilla.redhat.com/show_bug.cgi?id=1416528
For now, I set the krb5_auth_timeout to 120 seconds. I had to
completely
stop sssd and start it again. A colleague mentioned that sssd has a
known
issue with restart apparently.
I'm not aware of any such issue..
Also, I'm curious about ports requirements. Going from linux hosts to
AD, I
only authorize 88 TCP/UDP. I believe that's all I need.
Yes, from the clients, that should be enough. The servers need more ports open: https://access.redhat.com/documentation/en-US/Red_Hat_Ente rprise_Linux/7/html/Linux_Domain_Identity_Authentication_and _Policy_Guide/installing-ipa.html#prereq-ports
-- Alexandre Pitre alexandre.pitre@gmail.com _______________________________________________ FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org To unsubscribe send an email to freeipa-users-leave@lists.f edorahosted.org
-- Alexandre Pitre alexandre.pitre@gmail.com
-- Alexandre Pitre alexandre.pitre@gmail.com _______________________________________________ FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org To unsubscribe send an email to freeipa-users-leave@lists.fedorahosted.org
On 9 Aug 2017, at 16:26, Alexandre Pitre alexandre.pitre@gmail.com wrote:
If your hosts are in the IPA subdomain, then I would have expected centos.ipa.ad.com http://centos.ipa.ad.com/
The centos client has a hostname set to centos.domain.ad.com http://centos.domain.ad.com/ I'm using FQDN hostname based on the required DNS domain, not the IPA kerberos realm. Hence why centos.domain.ad.com http://centos.domain.ad.com/.
To explain further more, It'll probably be easier to use ipa.ad.com http://ipa.ad.com/ for all for my clients and not deal with a different DNS domain than the IPA realm. However, we have this business requirement to use a a different domain than the IPA realm. My understanding is that supposed to be supported by using the --domain switch of ipa-client-install: http://www.unix.com/man-page/centos/1/ipa-client-install/ http://www.unix.com/man-page/centos/1/ipa-client-install/ which basically just add static entires in /etc/sssd/sssd.conf and /etc/krb5.conf
Could you please paste once again the client’s sssd-users.conf?
I think the issue is that the hostname is used during the LDAP GSSAPI bind and it doesn’t match the client’s name on the IDM side (so, the client thinks it’s centos.domain.ad.com http://centos.domain.ad.com/ as far as its hostname is concerned but the IDM master only knows about centos.ipa.ad.com http://centos.ipa.ad.com/, right? That’s how the client is enrolled in IDM?)
If what I speculated above is true, then just adding: ipa_hostname = centos.ipa.ad.com http://centos.ipa.ad.com/ Into sssd.conf’s domain section might do the trick.
Should I approach things differently ?
I think it’s only a matter of configuration (and it took me three reads of the e-mails until I spotted how the domains are configured..)
What is the relation between domain.ad.com http://domain.ad.com/ and ad.com http://ad.com/ and ipa.ad.com http://ipa.ad.com/?
ad.com http://ad.com/ is my Active Directory domain. domain.ad.com http://domain.ad.com/ is a sub domain that was delegated from the AD DNS to the freeipa servers ipa.ad.com http://ipa.ad.com/ is also a sub domain that was delegated from the AD DNS to the freeipa servers and is the freeipa native kerberos realm.
On Wed, Aug 9, 2017 at 9:55 AM, Jakub Hrozek <jhrozek@redhat.com mailto:jhrozek@redhat.com> wrote:
On 7 Aug 2017, at 20:02, Alexandre Pitre via FreeIPA-users <freeipa-users@lists.fedorahosted.org mailto:freeipa-users@lists.fedorahosted.org> wrote:
The client is in the IPA domain. Although it's sub-domain of ad.com http://ad.com/, I did delegate it and configure the IPA servers as name servers. It uses a different domain suffix than ipa realm which was specified by ipa-client-install:
OK, but in the logs, I see: (Mon Aug 7 14:49:53 2017) [sssd[be[domain.ad.com http://domain.ad.com/]]] [sasl_bind_send] (0x0100): Executing sasl bind mech: GSSAPI, user: host/centos.domain.ad.com http://centos.domain.ad.com/
—> centos.domain.ad.com http://centos.domain.ad.com/
If your hosts are in the IPA subdomain, then I would have expected centos.ipa.ad.com http://centos.ipa.ad.com/
What is the relation between domain.ad.com http://domain.ad.com/ and ad.com http://ad.com/ and ipa.ad.com http://ipa.ad.com/?
(Mon Aug 7 14:49:53 2017) [sssd[be[domain.ad.com http://domain.ad.com/]]] [sasl_bind_send] (0x0020): ldap_sasl_bind failed (-2)[Local error] (Mon Aug 7 14:49:53 2017) [sssd[be[domain.ad.com http://domain.ad.com/]]] [sasl_bind_send] (0x0080): Extended failure message: [SASL(-1): generic failure: GSSAPI Error: Unspecified GSS failure. Minor c ode may provide more information (Server krbtgt/AD.COM@IPA.AD.COM mailto:krbtgt/AD.COM@IPA.AD.COM not found in Kerberos database)]
ipa-client-install -U -p admin -w Passw0rd! --enable-dns-updates --mkhomedir --domain=domain.ad.com http://domain.ad.com/ --realm=IPA.AD.COM http://ipa.ad.com/ --server=ipaserver01.ipa.ad.com http://ipaserver01.ipa.ad.com/ --server=ipaserver02.ipa.ad.com http://ipaserver02.ipa.ad.com/ --no-ntp --debug
On Mon, Aug 7, 2017 at 1:00 PM, Jakub Hrozek <jhrozek@redhat.com mailto:jhrozek@redhat.com> wrote:
On 7 Aug 2017, at 18:11, Alexandre Pitre <alexandre.pitre@gmail.com mailto:alexandre.pitre@gmail.com> wrote:
Clearing the sssd cache make the AD login works for a short while, it's probably not necessary nor "production" ready. Looking at /var/log/sssd/sssd_domain.ad.com http://sssd_domain.ad.com/.
Sure, but that’s not what I meant. What I meant is that just “systemctl restart sssd” clears the online/offline state.
Removing the sssd cache is not needed and can actually be quite dangerous. I wish people just stopped doing that, because in case credentials are cached, removing the cache also removes the credentials, leaving users with no way to authenticate.
I do see offline messages:
(Mon Aug 7 15:19:47 2017) [sssd[be[domain.ad.com http://domain.ad.com/]]] [sdap_id_op_connect_done] (0x0020): Failed to connect, going offline (5 [Input/output error]) (Mon Aug 7 15:19:47 2017) [sssd[be[domain.ad.com http://domain.ad.com/]]] [be_mark_offline] (0x2000): Going offline! (Mon Aug 7 15:19:47 2017) [sssd[be[domain.ad.com http://domain.ad.com/]]] [be_mark_offline] (0x2000): Enable check_if_online_ptask. (Mon Aug 7 15:19:47 2017) [sssd[be[domain.ad.com http://domain.ad.com/]]] [be_ptask_enable] (0x0400): Task [Check if online (periodic)]: enabling task (Mon Aug 7 15:19:47 2017) [sssd[be[domain.ad.com http://domain.ad.com/]]] [be_ptask_schedule] (0x0400): Task [Check if online (periodic)]: scheduling task 65 seconds from now [1502119252] (Mon Aug 7 15:19:47 2017) [sssd[be[domain.ad.com http://domain.ad.com/]]] [be_run_offline_cb] (0x0080): Going offline. Running callbacks. (Mon Aug 7 15:19:47 2017) [sssd[be[domain.ad.com http://domain.ad.com/]]] [sdap_id_op_connect_done] (0x4000): notify offline to op #1 (Mon Aug 7 15:19:47 2017) [sssd[be[domain.ad.com http://domain.ad.com/]]] [ipa_subdomains_refresh_connect_done] (0x0020): Unable to connect to LDAP [11]: Resource temporarily unavailable (Mon Aug 7 15:19:47 2017) [sssd[be[domain.ad.com http://domain.ad.com/]]] [ipa_subdomains_refresh_connect_done] (0x0080): No IPA server is available, cannot get the subdomain list while offline (Mon Aug 7 15:19:47 2017) [sssd[be[domain.ad.com http://domain.ad.com/]]] [be_ptask_done] (0x0040): Task [Subdomains Refresh]: failed with [1432158212]: SSSD is offline (Mon Aug 7 15:19:47 2017) [sssd[be[domain.ad.com http://domain.ad.com/]]] [be_ptask_schedule] (0x0400): Task [Subdomains Refresh]: scheduling task 14400 seconds from now [1502133587] (Mon Aug 7 15:19:47 2017) [sssd[be[domain.ad.com http://domain.ad.com/]]] [sdap_id_release_conn_data] (0x4000): releasing unused connection (Mon Aug 7 15:19:47 2017) [sssd[be[domain.ad.com http://domain.ad.com/]]] [be_ptask_online_cb] (0x0400): Back end is online
I uploaded the full log file /var/log/sssd/sssd_domain.ad.com http://sssd_domain.ad.com/ https://1drv.ms/f/s!AlZwwyQE2ZZ5p2ZmHLzmeKN7mBJ3 https://1drv.ms/f/s!AlZwwyQE2ZZ5p2ZmHLzmeKN7mBJ3
Both my IPA servers looks healthy.AD trust agent/controller server role are installed on both.
ipa trustdomain-find ad.com http://ad.com/ does return all of my AD domains on both IPA servers.
This is the failure in the logs: (Mon Aug 7 14:49:53 2017) [sssd[be[domain.ad.com http://domain.ad.com/]]] [sdap_process_result] (0x2000): Trace: end of ldap_result list (Mon Aug 7 14:49:53 2017) [sssd[be[domain.ad.com http://domain.ad.com/]]] [write_pipe_handler] (0x0400): All data has been sent! (Mon Aug 7 14:49:53 2017) [sssd[be[domain.ad.com http://domain.ad.com/]]] [read_pipe_handler] (0x0400): EOF received, client finished (Mon Aug 7 14:49:53 2017) [sssd[be[domain.ad.com http://domain.ad.com/]]] [sdap_get_tgt_recv] (0x0400): Child responded: 0 [FILE:/var/lib/sss/db/ccache_IPA.AD.COM http://ccache_ipa.ad.com/], expired on [1502203793] (Mon Aug 7 14:49:53 2017) [sssd[be[domain.ad.com http://domain.ad.com/]]] [sdap_cli_auth_step] (0x0100): expire timeout is 900 (Mon Aug 7 14:49:53 2017) [sssd[be[domain.ad.com http://domain.ad.com/]]] [sdap_cli_auth_step] (0x1000): the connection will expire at 1502118293 (Mon Aug 7 14:49:53 2017) [sssd[be[domain.ad.com http://domain.ad.com/]]] [sasl_bind_send] (0x0100): Executing sasl bind mech: GSSAPI, user: host/centos.domain.ad.com http://centos.domain.ad.com/ (Mon Aug 7 14:49:53 2017) [sssd[be[domain.ad.com http://domain.ad.com/]]] [sasl_bind_send] (0x0020): ldap_sasl_bind failed (-2)[Local error] (Mon Aug 7 14:49:53 2017) [sssd[be[domain.ad.com http://domain.ad.com/]]] [sasl_bind_send] (0x0080): Extended failure message: [SASL(-1): generic failure: GSSAPI Error: Unspecified GSS failure. Minor c ode may provide more information (Server krbtgt/AD.COM@IPA.AD.COM mailto:krbtgt/AD.COM@IPA.AD.COM not found in Kerberos database)]
Is your client hostname in the AD domain (centos.domain.ad.com http://centos.domain.ad.com/) or in the IPA domain (ipa.ad.com http://ipa.ad.com/) ?
Thanks, Alex
On Sun, Aug 6, 2017 at 11:07 AM, Jakub Hrozek <jhrozek@redhat.com mailto:jhrozek@redhat.com> wrote:
On 4 Aug 2017, at 23:08, Alexandre Pitre via FreeIPA-users <freeipa-users@lists.fedorahosted.org mailto:freeipa-users@lists.fedorahosted.org> wrote:
Turns out, I'm still getting the same problem. It works right away after I force clean the sssd cache: systemctl stop sssd ; rm -f /var/lib/sss/db/* /var/log/sssd/* ; systemctl start sssd
After some time, trying to log back on the same system I see the login prompt is much quicker when I type aduser@ad.com mailto:aduser@ad.com Instead of getting a simple "Password:" prompt I get aduser@ad.com mailto:aduser@ad.com@centos.domain.ad.com http://centos.domain.ad.com/'s password.
If I login as root and stop/start and clean the sssd cache, it start working again.
Are you sure cleaning the cache is needed? Because I think your issue is different. The fact that you get a faster login prompt and the “Server not found…” message both point to the sssd going offline.
You could run ‘sssctl domain-status’ to show if the domain is online or offline (requires the ‘ifp’ service to be enabled until RHEL-7.4/upstream 1.15.x) or look into the logs for messages like “Going offline”.
/var/log/messages is filled with:
centos sssd_be: GSSAPI Error: Unspecified GSS failure. Minor code may provide more information (Server krbtgt/AD.COM@IPA.AD.COM mailto:AD.COM@IPA.AD.COM not found in Kerberos database)
This is the trust principal. Are you sure all your replicas are either trust agents or you ran “ipa-adtrust-install” on them?
Any thoughts ?
Thanks, Alex
On Tue, Aug 1, 2017 at 2:58 AM, Jakub Hrozek <jhrozek@redhat.com mailto:jhrozek@redhat.com> wrote: On Mon, Jul 31, 2017 at 05:47:11PM -0400, Alexandre Pitre wrote:
Bull-eye Jakub, that did the trick. I should have posted for help on the mailing list sooner. Thanks you so much, you are saving my ass.
It makes sense to increase the krb5_auth_timeout as my AD domain controllers servers are worldwide. Currently they exist in 3 regions: North America, Europe and Asia.
The weird thing is it seems that when a linux host try to authenticate against my AD, it just randomly select an AD DC from the _kerberos SRV records. Normally, on the windows side, if "sites and services" are setup correctly with subnet defined and binded to sites, a windows client shouldn't try to authenticate against an AD DC that isn't local to his site. This mechanism doesn't seem to apply to my linux hosts. Is it because it's only available for windows hosts ? Is there another way to force linux clients to authenticate against AD DC local to their site ?
We haven't implemented the site selection for the clients yet, only for servers, see: https://bugzilla.redhat.com/show_bug.cgi?id=1416528 https://bugzilla.redhat.com/show_bug.cgi?id=1416528
For now, I set the krb5_auth_timeout to 120 seconds. I had to completely stop sssd and start it again. A colleague mentioned that sssd has a known issue with restart apparently.
I'm not aware of any such issue..
Also, I'm curious about ports requirements. Going from linux hosts to AD, I only authorize 88 TCP/UDP. I believe that's all I need.
Yes, from the clients, that should be enough. The servers need more ports open: https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/7/htm... https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/7/html/Linux_Domain_Identity_Authentication_and_Policy_Guide/installing-ipa.html#prereq-ports
-- Alexandre Pitre alexandre.pitre@gmail.com mailto:alexandre.pitre@gmail.com _______________________________________________ FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org mailto:freeipa-users@lists.fedorahosted.org To unsubscribe send an email to freeipa-users-leave@lists.fedorahosted.org mailto:freeipa-users-leave@lists.fedorahosted.org
-- Alexandre Pitre alexandre.pitre@gmail.com mailto:alexandre.pitre@gmail.com
-- Alexandre Pitre alexandre.pitre@gmail.com mailto:alexandre.pitre@gmail.com _______________________________________________ FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org mailto:freeipa-users@lists.fedorahosted.org To unsubscribe send an email to freeipa-users-leave@lists.fedorahosted.org mailto:freeipa-users-leave@lists.fedorahosted.org
-- Alexandre Pitre alexandre.pitre@gmail.com mailto:alexandre.pitre@gmail.com
On ke, 09 elo 2017, Jakub Hrozek via FreeIPA-users wrote:
On 9 Aug 2017, at 16:26, Alexandre Pitre alexandre.pitre@gmail.com wrote:
If your hosts are in the IPA subdomain, then I would have expected centos.ipa.ad.com http://centos.ipa.ad.com/
The centos client has a hostname set to centos.domain.ad.com http://centos.domain.ad.com/ I'm using FQDN hostname based on the required DNS domain, not the IPA kerberos realm. Hence why centos.domain.ad.com http://centos.domain.ad.com/.
To explain further more, It'll probably be easier to use ipa.ad.com http://ipa.ad.com/ for all for my clients and not deal with a different DNS domain than the IPA realm. However, we have this business requirement to use a a different domain than the IPA realm. My understanding is that supposed to be supported by using the --domain switch of ipa-client-install: http://www.unix.com/man-page/centos/1/ipa-client-install/ http://www.unix.com/man-page/centos/1/ipa-client-install/ which basically just add static entires in /etc/sssd/sssd.conf and /etc/krb5.conf
Could you please paste once again the client’s sssd-users.conf?
I think the issue is that the hostname is used during the LDAP GSSAPI bind and it doesn’t match the client’s name on the IDM side (so, the client thinks it’s centos.domain.ad.com http://centos.domain.ad.com/ as far as its hostname is concerned but the IDM master only knows about centos.ipa.ad.com http://centos.ipa.ad.com/, right? That’s how the client is enrolled in IDM?)
To close this thread, I helped Alexandre on the IRC. The basic issue is that one needs to plan domain space carefully when using trust to AD. Active Directory is more than just DNS zones, LDAP, Kerberos and friends. Active Directory domain controllers have internal assumptions on what belongs to AD namespace and what is not.
When IPA is handling DNS zones, we automatically add any domain added to IPA via 'ipa dnszone-add' to the list of IPA realm's domains, so it just works.
When trust is established, we tell AD that IPA 'owns' namespaces associated with IPA realm. In most cases a primary domain is advertised.
If you need more, 'ipa realmdomains-mod' command can be used to add more. However, you need to re-run 'ipa trust-add' to refresh this information to AD side and AD DCs run explicit verification of the topology conflicts at this step so it may report back some issues and fail to update the topology. In this case you'd get errors back in the error_log on IPA master that performed 'ipa trust-add' operation.
However, if you added new domains to IPA realm after trust was established, AD DCs have no idea these domains belong to IPA realm and cannot do any authentication routing to IPA. Running topology validation is expensive and Microsoft documentation says it is only done when an explicit call for validation is used. We decided to couple this step with 'ipa trust-add' because for most people this is a rare step to perform.
For example, if you have ipa.ad.com for IPA and ad.com for AD, we'd advertise that '*.ipa.ad.com' DNS namespace belongs to IPA realm. If you would add IPA clients to another domain, say, .domain.ad.com (and you only have IPA clients there, no Windows machines), AD DCs will still think .domain.ad.com belongs to AD, not IPA. To correct this thinking, add domain.ad.com to IPA realm mapping:
ipa realmdomains-mod --add-domain=domain.ad.com
and re-establish trust with 'ipa trust-add ...'. Running 'ipa trust-add' is enough -- it will automatically remove previous relationship and create a new one without affecting anything else.
If what I speculated above is true, then just adding: ipa_hostname = centos.ipa.ad.com http://centos.ipa.ad.com/ Into sssd.conf’s domain section might do the trick.
Should I approach things differently ?
I think it’s only a matter of configuration (and it took me three reads of the e-mails until I spotted how the domains are configured..)
What is the relation between domain.ad.com http://domain.ad.com/ and ad.com http://ad.com/ and ipa.ad.com http://ipa.ad.com/?
ad.com http://ad.com/ is my Active Directory domain. domain.ad.com http://domain.ad.com/ is a sub domain that was delegated from the AD DNS to the freeipa servers ipa.ad.com http://ipa.ad.com/ is also a sub domain that was delegated from the AD DNS to the freeipa servers and is the freeipa native kerberos realm.
On Wed, Aug 9, 2017 at 9:55 AM, Jakub Hrozek <jhrozek@redhat.com mailto:jhrozek@redhat.com> wrote:
On 7 Aug 2017, at 20:02, Alexandre Pitre via FreeIPA-users <freeipa-users@lists.fedorahosted.org mailto:freeipa-users@lists.fedorahosted.org> wrote:
The client is in the IPA domain. Although it's sub-domain of ad.com http://ad.com/, I did delegate it and configure the IPA servers as name servers. It uses a different domain suffix than ipa realm which was specified by ipa-client-install:
OK, but in the logs, I see: (Mon Aug 7 14:49:53 2017) [sssd[be[domain.ad.com http://domain.ad.com/]]] [sasl_bind_send] (0x0100): Executing sasl bind mech: GSSAPI, user: host/centos.domain.ad.com http://centos.domain.ad.com/
—> centos.domain.ad.com http://centos.domain.ad.com/
If your hosts are in the IPA subdomain, then I would have expected centos.ipa.ad.com http://centos.ipa.ad.com/
What is the relation between domain.ad.com http://domain.ad.com/ and ad.com http://ad.com/ and ipa.ad.com http://ipa.ad.com/?
(Mon Aug 7 14:49:53 2017) [sssd[be[domain.ad.com http://domain.ad.com/]]] [sasl_bind_send] (0x0020): ldap_sasl_bind failed (-2)[Local error] (Mon Aug 7 14:49:53 2017) [sssd[be[domain.ad.com http://domain.ad.com/]]] [sasl_bind_send] (0x0080): Extended failure message: [SASL(-1): generic failure: GSSAPI Error: Unspecified GSS failure. Minor c ode may provide more information (Server krbtgt/AD.COM@IPA.AD.COM mailto:krbtgt/AD.COM@IPA.AD.COM not found in Kerberos database)]
ipa-client-install -U -p admin -w Passw0rd! --enable-dns-updates --mkhomedir --domain=domain.ad.com http://domain.ad.com/ --realm=IPA.AD.COM http://ipa.ad.com/ --server=ipaserver01.ipa.ad.com http://ipaserver01.ipa.ad.com/ --server=ipaserver02.ipa.ad.com http://ipaserver02.ipa.ad.com/ --no-ntp --debug
On Mon, Aug 7, 2017 at 1:00 PM, Jakub Hrozek <jhrozek@redhat.com mailto:jhrozek@redhat.com> wrote:
On 7 Aug 2017, at 18:11, Alexandre Pitre <alexandre.pitre@gmail.com mailto:alexandre.pitre@gmail.com> wrote:
Clearing the sssd cache make the AD login works for a short while, it's probably not necessary nor "production" ready. Looking at /var/log/sssd/sssd_domain.ad.com http://sssd_domain.ad.com/.
Sure, but that’s not what I meant. What I meant is that just “systemctl restart sssd” clears the online/offline state.
Removing the sssd cache is not needed and can actually be quite dangerous. I wish people just stopped doing that, because in case credentials are cached, removing the cache also removes the credentials, leaving users with no way to authenticate.
I do see offline messages:
(Mon Aug 7 15:19:47 2017) [sssd[be[domain.ad.com http://domain.ad.com/]]] [sdap_id_op_connect_done] (0x0020): Failed to connect, going offline (5 [Input/output error]) (Mon Aug 7 15:19:47 2017) [sssd[be[domain.ad.com http://domain.ad.com/]]] [be_mark_offline] (0x2000): Going offline! (Mon Aug 7 15:19:47 2017) [sssd[be[domain.ad.com http://domain.ad.com/]]] [be_mark_offline] (0x2000): Enable check_if_online_ptask. (Mon Aug 7 15:19:47 2017) [sssd[be[domain.ad.com http://domain.ad.com/]]] [be_ptask_enable] (0x0400): Task [Check if online (periodic)]: enabling task (Mon Aug 7 15:19:47 2017) [sssd[be[domain.ad.com http://domain.ad.com/]]] [be_ptask_schedule] (0x0400): Task [Check if online (periodic)]: scheduling task 65 seconds from now [1502119252] (Mon Aug 7 15:19:47 2017) [sssd[be[domain.ad.com http://domain.ad.com/]]] [be_run_offline_cb] (0x0080): Going offline. Running callbacks. (Mon Aug 7 15:19:47 2017) [sssd[be[domain.ad.com http://domain.ad.com/]]] [sdap_id_op_connect_done] (0x4000): notify offline to op #1 (Mon Aug 7 15:19:47 2017) [sssd[be[domain.ad.com http://domain.ad.com/]]] [ipa_subdomains_refresh_connect_done] (0x0020): Unable to connect to LDAP [11]: Resource temporarily unavailable (Mon Aug 7 15:19:47 2017) [sssd[be[domain.ad.com http://domain.ad.com/]]] [ipa_subdomains_refresh_connect_done] (0x0080): No IPA server is available, cannot get the subdomain list while offline (Mon Aug 7 15:19:47 2017) [sssd[be[domain.ad.com http://domain.ad.com/]]] [be_ptask_done] (0x0040): Task [Subdomains Refresh]: failed with [1432158212]: SSSD is offline (Mon Aug 7 15:19:47 2017) [sssd[be[domain.ad.com http://domain.ad.com/]]] [be_ptask_schedule] (0x0400): Task [Subdomains Refresh]: scheduling task 14400 seconds from now [1502133587] (Mon Aug 7 15:19:47 2017) [sssd[be[domain.ad.com http://domain.ad.com/]]] [sdap_id_release_conn_data] (0x4000): releasing unused connection (Mon Aug 7 15:19:47 2017) [sssd[be[domain.ad.com http://domain.ad.com/]]] [be_ptask_online_cb] (0x0400): Back end is online
I uploaded the full log file /var/log/sssd/sssd_domain.ad.com http://sssd_domain.ad.com/ https://1drv.ms/f/s!AlZwwyQE2ZZ5p2ZmHLzmeKN7mBJ3 https://1drv.ms/f/s!AlZwwyQE2ZZ5p2ZmHLzmeKN7mBJ3
Both my IPA servers looks healthy.AD trust agent/controller server role are installed on both.
ipa trustdomain-find ad.com http://ad.com/ does return all of my AD domains on both IPA servers.
This is the failure in the logs: (Mon Aug 7 14:49:53 2017) [sssd[be[domain.ad.com http://domain.ad.com/]]] [sdap_process_result] (0x2000): Trace: end of ldap_result list (Mon Aug 7 14:49:53 2017) [sssd[be[domain.ad.com http://domain.ad.com/]]] [write_pipe_handler] (0x0400): All data has been sent! (Mon Aug 7 14:49:53 2017) [sssd[be[domain.ad.com http://domain.ad.com/]]] [read_pipe_handler] (0x0400): EOF received, client finished (Mon Aug 7 14:49:53 2017) [sssd[be[domain.ad.com http://domain.ad.com/]]] [sdap_get_tgt_recv] (0x0400): Child responded: 0 [FILE:/var/lib/sss/db/ccache_IPA.AD.COM http://ccache_ipa.ad.com/], expired on [1502203793] (Mon Aug 7 14:49:53 2017) [sssd[be[domain.ad.com http://domain.ad.com/]]] [sdap_cli_auth_step] (0x0100): expire timeout is 900 (Mon Aug 7 14:49:53 2017) [sssd[be[domain.ad.com http://domain.ad.com/]]] [sdap_cli_auth_step] (0x1000): the connection will expire at 1502118293 (Mon Aug 7 14:49:53 2017) [sssd[be[domain.ad.com http://domain.ad.com/]]] [sasl_bind_send] (0x0100): Executing sasl bind mech: GSSAPI, user: host/centos.domain.ad.com http://centos.domain.ad.com/ (Mon Aug 7 14:49:53 2017) [sssd[be[domain.ad.com http://domain.ad.com/]]] [sasl_bind_send] (0x0020): ldap_sasl_bind failed (-2)[Local error] (Mon Aug 7 14:49:53 2017) [sssd[be[domain.ad.com http://domain.ad.com/]]] [sasl_bind_send] (0x0080): Extended failure message: [SASL(-1): generic failure: GSSAPI Error: Unspecified GSS failure. Minor c ode may provide more information (Server krbtgt/AD.COM@IPA.AD.COM mailto:krbtgt/AD.COM@IPA.AD.COM not found in Kerberos database)]
Is your client hostname in the AD domain (centos.domain.ad.com http://centos.domain.ad.com/) or in the IPA domain (ipa.ad.com http://ipa.ad.com/) ?
Thanks, Alex
On Sun, Aug 6, 2017 at 11:07 AM, Jakub Hrozek <jhrozek@redhat.com mailto:jhrozek@redhat.com> wrote:
On 4 Aug 2017, at 23:08, Alexandre Pitre via FreeIPA-users <freeipa-users@lists.fedorahosted.org mailto:freeipa-users@lists.fedorahosted.org> wrote:
Turns out, I'm still getting the same problem. It works right away after I force clean the sssd cache: systemctl stop sssd ; rm -f /var/lib/sss/db/* /var/log/sssd/* ; systemctl start sssd
After some time, trying to log back on the same system I see the login prompt is much quicker when I type aduser@ad.com mailto:aduser@ad.com Instead of getting a simple "Password:" prompt I get aduser@ad.com mailto:aduser@ad.com@centos.domain.ad.com http://centos.domain.ad.com/'s password.
If I login as root and stop/start and clean the sssd cache, it start working again.
Are you sure cleaning the cache is needed? Because I think your issue is different. The fact that you get a faster login prompt and the “Server not found…” message both point to the sssd going offline.
You could run ‘sssctl domain-status’ to show if the domain is online or offline (requires the ‘ifp’ service to be enabled until RHEL-7.4/upstream 1.15.x) or look into the logs for messages like “Going offline”.
/var/log/messages is filled with:
centos sssd_be: GSSAPI Error: Unspecified GSS failure. Minor code may provide more information (Server krbtgt/AD.COM@IPA.AD.COM mailto:AD.COM@IPA.AD.COM not found in Kerberos database)
This is the trust principal. Are you sure all your replicas are either trust agents or you ran “ipa-adtrust-install” on them?
Any thoughts ?
Thanks, Alex
On Tue, Aug 1, 2017 at 2:58 AM, Jakub Hrozek <jhrozek@redhat.com mailto:jhrozek@redhat.com> wrote: On Mon, Jul 31, 2017 at 05:47:11PM -0400, Alexandre Pitre wrote:
Bull-eye Jakub, that did the trick. I should have posted for help on the mailing list sooner. Thanks you so much, you are saving my ass.
It makes sense to increase the krb5_auth_timeout as my AD domain controllers servers are worldwide. Currently they exist in 3 regions: North America, Europe and Asia.
The weird thing is it seems that when a linux host try to authenticate against my AD, it just randomly select an AD DC from the _kerberos SRV records. Normally, on the windows side, if "sites and services" are setup correctly with subnet defined and binded to sites, a windows client shouldn't try to authenticate against an AD DC that isn't local to his site. This mechanism doesn't seem to apply to my linux hosts. Is it because it's only available for windows hosts ? Is there another way to force linux clients to authenticate against AD DC local to their site ?
We haven't implemented the site selection for the clients yet, only for servers, see: https://bugzilla.redhat.com/show_bug.cgi?id=1416528 https://bugzilla.redhat.com/show_bug.cgi?id=1416528
For now, I set the krb5_auth_timeout to 120 seconds. I had to completely stop sssd and start it again. A colleague mentioned that sssd has a known issue with restart apparently.
I'm not aware of any such issue..
Also, I'm curious about ports requirements. Going from linux hosts to AD, I only authorize 88 TCP/UDP. I believe that's all I need.
Yes, from the clients, that should be enough. The servers need more ports open: https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/7/htm... https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/7/html/Linux_Domain_Identity_Authentication_and_Policy_Guide/installing-ipa.html#prereq-ports
-- Alexandre Pitre alexandre.pitre@gmail.com mailto:alexandre.pitre@gmail.com _______________________________________________ FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org mailto:freeipa-users@lists.fedorahosted.org To unsubscribe send an email to freeipa-users-leave@lists.fedorahosted.org mailto:freeipa-users-leave@lists.fedorahosted.org
-- Alexandre Pitre alexandre.pitre@gmail.com mailto:alexandre.pitre@gmail.com
-- Alexandre Pitre alexandre.pitre@gmail.com mailto:alexandre.pitre@gmail.com _______________________________________________ FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org mailto:freeipa-users@lists.fedorahosted.org To unsubscribe send an email to freeipa-users-leave@lists.fedorahosted.org mailto:freeipa-users-leave@lists.fedorahosted.org
-- Alexandre Pitre alexandre.pitre@gmail.com mailto:alexandre.pitre@gmail.com
FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org To unsubscribe send an email to freeipa-users-leave@lists.fedorahosted.org
On 12 Aug 2017, at 20:14, Alexander Bokovoy via FreeIPA-users freeipa-users@lists.fedorahosted.org wrote:
To close this thread, I helped Alexandre on the IRC. The basic issue is that one needs to plan domain space carefully when using trust to AD. Active Directory is more than just DNS zones, LDAP, Kerberos and friends. Active Directory domain controllers have internal assumptions on what belongs to AD namespace and what is not.
Thank you for driving this towards the end. I knew I was missing something and I’m glad I could learn something new as well.
Although, the explanation from Alexander Bokovoy made perfect sense, I'm still facing the issue after I re-established the AD trust successfully:
(Tue Aug 15 02:23:40 2017) [sssd[be[domain.ad.com]]] [sdap_cli_auth_step] (0x1000): the connection will expire at 1502764720 (Tue Aug 15 02:23:40 2017) [sssd[be[domain.ad.com]]] [sasl_bind_send] (0x0100): Executing sasl bind mech: GSSAPI, user: host/centos.domain.ad.com (Tue Aug 15 02:23:40 2017) [sssd[be[domain.ad.com]]] [sasl_bind_send] (0x0020): ldap_sasl_bind failed (-2)[Local error] (Tue Aug 15 02:23:40 2017) [sssd[be[domain.ad.com]]] [sasl_bind_send] (0x0080): Extended failure message: [SASL(-1): generic failure: GSSAPI Error: Unspecified GSS failure. Minor code may provide more information (Server krbtgt/AD.COM@TENANT.AD.COM not found in Kerberos database)] (Tue Aug 15 02:23:40 2017) [sssd[be[domain.ad.com]]] [child_sig_handler] (0x1000): Waiting for child [5197]. (Tue Aug 15 02:23:40 2017) [sssd[be[domain.ad.com]]] [child_sig_handler] (0x0100): child [5197] finished successfully. (Tue Aug 15 02:23:40 2017) [sssd[be[domain.ad.com]]] [sdap_cli_connect_recv] (0x0040): Unable to establish connection [1432158225]: Authentication Failed (Tue Aug 15 02:23:40 2017) [sssd[be[domain.ad.com]]] [_be_fo_set_port_status] (0x8000): Setting status: PORT_NOT_WORKING. Called from: src/providers/ldap/sdap_async_connection.c: sdap_cli_connect_recv: 2048 (Tue Aug 15 02:23:40 2017) [sssd[be[domain.ad.com]]] [fo_set_port_status] (0x0100): Marking port 0 of server 'ipaserver02.ipa.ad.com' as 'not working' (Tue Aug 15 02:23:40 2017) [sssd[be[domain.ad.com]]] [fo_set_port_status] (0x0400): Marking port 0 of duplicate server 'ipaserver02.ipa.ad.com' as 'not working' (Tue Aug 15 02:23:40 2017) [sssd[be[domain.ad.com]]] [sdap_handle_release] (0x2000): Trace: sh[0x7f4811278710], connected[1], ops[(nil)], ldap[0x7f481121f780], destructor_lock[0], release_memory[0] (Tue Aug 15 02:23:40 2017) [sssd[be[domain.ad.com]]] [remove_connection_callback] (0x4000): Successfully removed connection callback. (Tue Aug 15 02:23:40 2017) [sssd[be[domain.ad.com]]] [sdap_id_op_connect_done] (0x4000): attempting failover retry on op #1 (Tue Aug 15 02:23:40 2017) [sssd[be[domain.ad.com]]] [sdap_id_op_connect_step] (0x4000): beginning to connect (Tue Aug 15 02:23:40 2017) [sssd[be[domain.ad.com]]] [fo_resolve_service_send] (0x0100): Trying to resolve service 'IPA' (Tue Aug 15 02:23:40 2017) [sssd[be[domain.ad.com]]] [get_port_status] (0x1000): Port status of port 0 for server '(no name)' is 'neutral' (Tue Aug 15 02:23:40 2017) [sssd[be[domain.ad.com]]] [fo_resolve_service_activate_timeout] (0x2000): Resolve timeout set to 6 seconds (Tue Aug 15 02:23:40 2017) [sssd[be[domain.ad.com]]] [resolve_srv_send] (0x0200): The status of SRV lookup is neutral (Tue Aug 15 02:23:40 2017) [sssd[be[domain.ad.com]]] [resolv_discover_srv_next_domain] (0x0400): SRV resolution of service 'ldap'. Will use DNS discovery domain 'domain.ad.com' (Tue Aug 15 02:23:40 2017) [sssd[be[domain.ad.com]]] [resolv_getsrv_send] (0x0100): Trying to resolve SRV record of '_ldap._tcp.domain.ad.com' (Tue Aug 15 02:23:40 2017) [sssd[be[domain.ad.com]]] [sdap_id_release_conn_data] (0x4000): releasing unused connection (Tue Aug 15 02:23:40 2017) [sssd[be[domain.ad.com]]] [schedule_request_timeout] (0x2000): Scheduling a timeout of 6 seconds (Tue Aug 15 02:23:40 2017) [sssd[be[domain.ad.com]]] [schedule_timeout_watcher] (0x2000): Scheduling DNS timeout watcher (Tue Aug 15 02:23:40 2017) [sssd[be[domain.ad.com]]] [unschedule_timeout_watcher] (0x4000): Unscheduling DNS timeout watcher (Tue Aug 15 02:23:40 2017) [sssd[be[domain.ad.com]]] [request_watch_destructor] (0x0400): Deleting request watch (Tue Aug 15 02:23:40 2017) [sssd[be[domain.ad.com]]] [resolv_discover_srv_done] (0x0040): SRV query failed [4]: Domain name not found (Tue Aug 15 02:23:40 2017) [sssd[be[domain.ad.com]]] [fo_set_port_status] (0x0100): Marking port 0 of server '(no name)' as 'not working' (Tue Aug 15 02:23:40 2017) [sssd[be[domain.ad.com]]] [resolve_srv_done] (0x0040): Unable to resolve SRV [1432158235]: SRV record not found (Tue Aug 15 02:23:40 2017) [sssd[be[domain.ad.com]]] [set_srv_data_status] (0x0100): Marking SRV lookup of service 'IPA' as 'not resolved' (Tue Aug 15 02:23:40 2017) [sssd[be[domain.ad.com]]] [be_resolve_server_process] (0x0080): Couldn't resolve server (SRV lookup meta-server), resolver returned [1432158235]: SRV record not found (Tue Aug 15 02:23:40 2017) [sssd[be[domain.ad.com]]] [be_resolve_server_process] (0x1000): Trying with the next one! (Tue Aug 15 02:23:40 2017) [sssd[be[domain.ad.com]]] [fo_resolve_service_send] (0x0100): Trying to resolve service 'IPA' (Tue Aug 15 02:23:40 2017) [sssd[be[domain.ad.com]]] [get_server_status] (0x1000): Status of server 'ipaserver01.ipa.ad.com' is 'name resolved' (Tue Aug 15 02:23:40 2017) [sssd[be[domain.ad.com]]] [get_port_status] (0x1000): Port status of port 0 for server 'ipaserver01.ipa.ad.com' is 'not working' (Tue Aug 15 02:23:40 2017) [sssd[be[domain.ad.com]]] [get_server_status] (0x1000): Status of server 'ipaserver02.ipa.ad.com' is 'name resolved' (Tue Aug 15 02:23:40 2017) [sssd[be[domain.ad.com]]] [get_port_status] (0x1000): Port status of port 0 for server 'ipaserver02.ipa.ad.com' is 'not working' (Tue Aug 15 02:23:40 2017) [sssd[be[domain.ad.com]]] [get_port_status] (0x1000): Port status of port 0 for server '(no name)' is 'not working' (Tue Aug 15 02:23:40 2017) [sssd[be[domain.ad.com]]] [fo_resolve_service_send] (0x0020): No available servers for service 'IPA' (Tue Aug 15 02:23:40 2017) [sssd[be[domain.ad.com]]] [be_resolve_server_done] (0x1000): Server resolution failed: [5]: Input/output error (Tue Aug 15 02:23:40 2017) [sssd[be[domain.ad.com]]] [sdap_id_op_connect_done] (0x0020): Failed to connect, going offline (5 [Input/output error]) (Tue Aug 15 02:23:40 2017) [sssd[be[domain.ad.com]]] [be_mark_offline] (0x2000): Going offline!
These logs are from a system newly joined to FreeIPA.
I know for a fact it's related to the use of a different domain then my IPA realm. I tested with some systems with both dns and ipa realm set to ipa.ad.com and the authentication works just fine.
On Mon, Aug 14, 2017 at 4:22 AM, Jakub Hrozek via FreeIPA-users < freeipa-users@lists.fedorahosted.org> wrote:
On 12 Aug 2017, at 20:14, Alexander Bokovoy via FreeIPA-users <
freeipa-users@lists.fedorahosted.org> wrote:
To close this thread, I helped Alexandre on the IRC. The basic issue is that one needs to plan domain space carefully when using trust to AD. Active Directory is more than just DNS zones, LDAP, Kerberos and friends. Active Directory domain controllers have internal assumptions on what belongs to AD namespace and what is not.
Thank you for driving this towards the end. I knew I was missing something and I’m glad I could learn something new as well. _______________________________________________ FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org To unsubscribe send an email to freeipa-users-leave@lists.fedorahosted.org
On ma, 14 elo 2017, Alexandre Pitre via FreeIPA-users wrote:
Although, the explanation from Alexander Bokovoy made perfect sense, I'm still facing the issue after I re-established the AD trust successfully:
(Tue Aug 15 02:23:40 2017) [sssd[be[domain.ad.com]]] [sdap_cli_auth_step] (0x1000): the connection will expire at 1502764720 (Tue Aug 15 02:23:40 2017) [sssd[be[domain.ad.com]]] [sasl_bind_send] (0x0100): Executing sasl bind mech: GSSAPI, user: host/centos.domain.ad.com (Tue Aug 15 02:23:40 2017) [sssd[be[domain.ad.com]]] [sasl_bind_send] (0x0020): ldap_sasl_bind failed (-2)[Local error] (Tue Aug 15 02:23:40 2017) [sssd[be[domain.ad.com]]] [sasl_bind_send] (0x0080): Extended failure message: [SASL(-1): generic failure: GSSAPI Error: Unspecified GSS failure. Minor code may provide more information (Server krbtgt/AD.COM@TENANT.AD.COM not found in Kerberos database)]
Why do you use domain.ad.com as IPA domain here? Your IPA domain should be 'ipa.ad.com' when you enroll clients regardless which DNS domains they belong to. It is the realm they will be attached, so your sssd.conf on the machines in domain.ad.com should have
[domain/ipa.ad.com] ipa_domain = ipa.ad.com
And the client enrollment should use IPA domain too:
ipa-client-install --domain ipa.ad.com
Read http://www.freeipa.org/page/V4/IPA_Client_in_Active_Directory_DNS_domain for more.
Because your configuration now is wrong, krb5 library thinks your client is part of DOMAIN.AD.COM realm (TENANT.AD.COM in the log above, I guess you did not scrub it well) and not IPA.AD.COM instead. This is why it fails to find a cross-realm TGT to traverse up from DOMAIN.AD.COM to AD.COM realm hierarchically. Obviously, it should not be doing that.
(Tue Aug 15 02:23:40 2017) [sssd[be[domain.ad.com]]] [child_sig_handler] (0x1000): Waiting for child [5197]. (Tue Aug 15 02:23:40 2017) [sssd[be[domain.ad.com]]] [child_sig_handler] (0x0100): child [5197] finished successfully. (Tue Aug 15 02:23:40 2017) [sssd[be[domain.ad.com]]] [sdap_cli_connect_recv] (0x0040): Unable to establish connection [1432158225]: Authentication Failed (Tue Aug 15 02:23:40 2017) [sssd[be[domain.ad.com]]] [_be_fo_set_port_status] (0x8000): Setting status: PORT_NOT_WORKING. Called from: src/providers/ldap/sdap_async_connection.c: sdap_cli_connect_recv: 2048 (Tue Aug 15 02:23:40 2017) [sssd[be[domain.ad.com]]] [fo_set_port_status] (0x0100): Marking port 0 of server 'ipaserver02.ipa.ad.com' as 'not working' (Tue Aug 15 02:23:40 2017) [sssd[be[domain.ad.com]]] [fo_set_port_status] (0x0400): Marking port 0 of duplicate server 'ipaserver02.ipa.ad.com' as 'not working' (Tue Aug 15 02:23:40 2017) [sssd[be[domain.ad.com]]] [sdap_handle_release] (0x2000): Trace: sh[0x7f4811278710], connected[1], ops[(nil)], ldap[0x7f481121f780], destructor_lock[0], release_memory[0] (Tue Aug 15 02:23:40 2017) [sssd[be[domain.ad.com]]] [remove_connection_callback] (0x4000): Successfully removed connection callback. (Tue Aug 15 02:23:40 2017) [sssd[be[domain.ad.com]]] [sdap_id_op_connect_done] (0x4000): attempting failover retry on op #1 (Tue Aug 15 02:23:40 2017) [sssd[be[domain.ad.com]]] [sdap_id_op_connect_step] (0x4000): beginning to connect (Tue Aug 15 02:23:40 2017) [sssd[be[domain.ad.com]]] [fo_resolve_service_send] (0x0100): Trying to resolve service 'IPA' (Tue Aug 15 02:23:40 2017) [sssd[be[domain.ad.com]]] [get_port_status] (0x1000): Port status of port 0 for server '(no name)' is 'neutral' (Tue Aug 15 02:23:40 2017) [sssd[be[domain.ad.com]]] [fo_resolve_service_activate_timeout] (0x2000): Resolve timeout set to 6 seconds (Tue Aug 15 02:23:40 2017) [sssd[be[domain.ad.com]]] [resolve_srv_send] (0x0200): The status of SRV lookup is neutral (Tue Aug 15 02:23:40 2017) [sssd[be[domain.ad.com]]] [resolv_discover_srv_next_domain] (0x0400): SRV resolution of service 'ldap'. Will use DNS discovery domain 'domain.ad.com' (Tue Aug 15 02:23:40 2017) [sssd[be[domain.ad.com]]] [resolv_getsrv_send] (0x0100): Trying to resolve SRV record of '_ldap._tcp.domain.ad.com' (Tue Aug 15 02:23:40 2017) [sssd[be[domain.ad.com]]] [sdap_id_release_conn_data] (0x4000): releasing unused connection (Tue Aug 15 02:23:40 2017) [sssd[be[domain.ad.com]]] [schedule_request_timeout] (0x2000): Scheduling a timeout of 6 seconds (Tue Aug 15 02:23:40 2017) [sssd[be[domain.ad.com]]] [schedule_timeout_watcher] (0x2000): Scheduling DNS timeout watcher (Tue Aug 15 02:23:40 2017) [sssd[be[domain.ad.com]]] [unschedule_timeout_watcher] (0x4000): Unscheduling DNS timeout watcher (Tue Aug 15 02:23:40 2017) [sssd[be[domain.ad.com]]] [request_watch_destructor] (0x0400): Deleting request watch (Tue Aug 15 02:23:40 2017) [sssd[be[domain.ad.com]]] [resolv_discover_srv_done] (0x0040): SRV query failed [4]: Domain name not found (Tue Aug 15 02:23:40 2017) [sssd[be[domain.ad.com]]] [fo_set_port_status] (0x0100): Marking port 0 of server '(no name)' as 'not working' (Tue Aug 15 02:23:40 2017) [sssd[be[domain.ad.com]]] [resolve_srv_done] (0x0040): Unable to resolve SRV [1432158235]: SRV record not found (Tue Aug 15 02:23:40 2017) [sssd[be[domain.ad.com]]] [set_srv_data_status] (0x0100): Marking SRV lookup of service 'IPA' as 'not resolved' (Tue Aug 15 02:23:40 2017) [sssd[be[domain.ad.com]]] [be_resolve_server_process] (0x0080): Couldn't resolve server (SRV lookup meta-server), resolver returned [1432158235]: SRV record not found (Tue Aug 15 02:23:40 2017) [sssd[be[domain.ad.com]]] [be_resolve_server_process] (0x1000): Trying with the next one! (Tue Aug 15 02:23:40 2017) [sssd[be[domain.ad.com]]] [fo_resolve_service_send] (0x0100): Trying to resolve service 'IPA' (Tue Aug 15 02:23:40 2017) [sssd[be[domain.ad.com]]] [get_server_status] (0x1000): Status of server 'ipaserver01.ipa.ad.com' is 'name resolved' (Tue Aug 15 02:23:40 2017) [sssd[be[domain.ad.com]]] [get_port_status] (0x1000): Port status of port 0 for server 'ipaserver01.ipa.ad.com' is 'not working' (Tue Aug 15 02:23:40 2017) [sssd[be[domain.ad.com]]] [get_server_status] (0x1000): Status of server 'ipaserver02.ipa.ad.com' is 'name resolved' (Tue Aug 15 02:23:40 2017) [sssd[be[domain.ad.com]]] [get_port_status] (0x1000): Port status of port 0 for server 'ipaserver02.ipa.ad.com' is 'not working' (Tue Aug 15 02:23:40 2017) [sssd[be[domain.ad.com]]] [get_port_status] (0x1000): Port status of port 0 for server '(no name)' is 'not working' (Tue Aug 15 02:23:40 2017) [sssd[be[domain.ad.com]]] [fo_resolve_service_send] (0x0020): No available servers for service 'IPA' (Tue Aug 15 02:23:40 2017) [sssd[be[domain.ad.com]]] [be_resolve_server_done] (0x1000): Server resolution failed: [5]: Input/output error (Tue Aug 15 02:23:40 2017) [sssd[be[domain.ad.com]]] [sdap_id_op_connect_done] (0x0020): Failed to connect, going offline (5 [Input/output error]) (Tue Aug 15 02:23:40 2017) [sssd[be[domain.ad.com]]] [be_mark_offline] (0x2000): Going offline!
These logs are from a system newly joined to FreeIPA.
I know for a fact it's related to the use of a different domain then my IPA realm. I tested with some systems with both dns and ipa realm set to ipa.ad.com and the authentication works just fine.
On Mon, Aug 14, 2017 at 4:22 AM, Jakub Hrozek via FreeIPA-users < freeipa-users@lists.fedorahosted.org> wrote:
On 12 Aug 2017, at 20:14, Alexander Bokovoy via FreeIPA-users <
freeipa-users@lists.fedorahosted.org> wrote:
To close this thread, I helped Alexandre on the IRC. The basic issue is that one needs to plan domain space carefully when using trust to AD. Active Directory is more than just DNS zones, LDAP, Kerberos and friends. Active Directory domain controllers have internal assumptions on what belongs to AD namespace and what is not.
Thank you for driving this towards the end. I knew I was missing something and I’m glad I could learn something new as well. _______________________________________________ FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org To unsubscribe send an email to freeipa-users-leave@lists.fedorahosted.org
-- Alexandre Pitre alexandre.pitre@gmail.com
FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org To unsubscribe send an email to freeipa-users-leave@lists.fedorahosted.org
Hi Alexander,
You're correct, turns out I wasn't using the correct domain for the --domain parameter. I thought I was. Here's the command I used.
ipa-client-install -U -p admin -w Passw0rd! --enable-dns-updates --mkhomedir --domain=ipa.ad.com --realm=IPA.AD.COM --no-ntp --debug
All of my client hostname are set as "hostname.domain.ad.com", I didn't know that in itself that was enough of a requirement to join them to FreeIPA. Of course, given that the domain is also present in freeipa and the AD trust has been established AFTER the domain was added to freeipa.
I haven't tested yet without the realm parameter. It is possible that I don't need --domain nor --realm parameters ? Does that require the creation of *_ldap._tcp.* srv records in domain.ad.com dns zone?
Taken from the man page:
*When the client machine hostname is not in a subdomain of an IPA server, its domain can be passed with --domain https://www.mankier.com/1/ipa-client-install#--domain option. In that case, both SSSD and Kerberos components have the domain set in the configuration files and will use it to autodiscover IPA servers.*
That line miss directed me, not sure if that's my interpretation. Documentation could benefit from being clearer and having examples.
Setting krb5_auth_timeout to 120 seconds is also required in my environment as we're dealing with AD DC spreaded all over the globe. To make kerberos negotiation faster, I assume I could specify my AD.COM realm in /etc/krb5.conf with my local site AD DC ?
Big thanks to you and Jakub, my employer and I are very glad that this issue is finally resolved =)
On Tue, Aug 15, 2017 at 3:45 AM, Alexander Bokovoy abokovoy@redhat.com wrote:
On ma, 14 elo 2017, Alexandre Pitre via FreeIPA-users wrote:
Although, the explanation from Alexander Bokovoy made perfect sense, I'm still facing the issue after I re-established the AD trust successfully:
(Tue Aug 15 02:23:40 2017) [sssd[be[domain.ad.com]]] [sdap_cli_auth_step] (0x1000): the connection will expire at 1502764720 (Tue Aug 15 02:23:40 2017) [sssd[be[domain.ad.com]]] [sasl_bind_send] (0x0100): Executing sasl bind mech: GSSAPI, user: host/ centos.domain.ad.com (Tue Aug 15 02:23:40 2017) [sssd[be[domain.ad.com]]] [sasl_bind_send] (0x0020): ldap_sasl_bind failed (-2)[Local error] (Tue Aug 15 02:23:40 2017) [sssd[be[domain.ad.com]]] [sasl_bind_send] (0x0080): Extended failure message: [SASL(-1): generic failure: GSSAPI Error: Unspecified GSS failure. Minor code may provide more information (Server krbtgt/AD.COM@TENANT.AD.COM not found in Kerberos database)]
Why do you use domain.ad.com as IPA domain here? Your IPA domain should be 'ipa.ad.com' when you enroll clients regardless which DNS domains they belong to. It is the realm they will be attached, so your sssd.conf on the machines in domain.ad.com should have
[domain/ipa.ad.com] ipa_domain = ipa.ad.com
And the client enrollment should use IPA domain too:
ipa-client-install --domain ipa.ad.com
Read http://www.freeipa.org/page/V4/IPA_Client_in_Active_Director y_DNS_domain for more.
Because your configuration now is wrong, krb5 library thinks your client is part of DOMAIN.AD.COM realm (TENANT.AD.COM in the log above, I guess you did not scrub it well) and not IPA.AD.COM instead. This is why it fails to find a cross-realm TGT to traverse up from DOMAIN.AD.COM to AD.COM realm hierarchically. Obviously, it should not be doing that.
(Tue Aug 15 02:23:40 2017) [sssd[be[domain.ad.com]]] [child_sig_handler]
(0x1000): Waiting for child [5197]. (Tue Aug 15 02:23:40 2017) [sssd[be[domain.ad.com]]] [child_sig_handler] (0x0100): child [5197] finished successfully. (Tue Aug 15 02:23:40 2017) [sssd[be[domain.ad.com]]] [sdap_cli_connect_recv] (0x0040): Unable to establish connection [1432158225]: Authentication Failed (Tue Aug 15 02:23:40 2017) [sssd[be[domain.ad.com]]] [_be_fo_set_port_status] (0x8000): Setting status: PORT_NOT_WORKING. Called from: src/providers/ldap/sdap_async_connection.c: sdap_cli_connect_recv: 2048 (Tue Aug 15 02:23:40 2017) [sssd[be[domain.ad.com]]] [fo_set_port_status] (0x0100): Marking port 0 of server 'ipaserver02.ipa.ad.com' as 'not working' (Tue Aug 15 02:23:40 2017) [sssd[be[domain.ad.com]]] [fo_set_port_status] (0x0400): Marking port 0 of duplicate server 'ipaserver02.ipa.ad.com' as 'not working' (Tue Aug 15 02:23:40 2017) [sssd[be[domain.ad.com]]] [sdap_handle_release] (0x2000): Trace: sh[0x7f4811278710], connected[1], ops[(nil)], ldap[0x7f481121f780], destructor_lock[0], release_memory[0] (Tue Aug 15 02:23:40 2017) [sssd[be[domain.ad.com]]] [remove_connection_callback] (0x4000): Successfully removed connection callback. (Tue Aug 15 02:23:40 2017) [sssd[be[domain.ad.com]]] [sdap_id_op_connect_done] (0x4000): attempting failover retry on op #1 (Tue Aug 15 02:23:40 2017) [sssd[be[domain.ad.com]]] [sdap_id_op_connect_step] (0x4000): beginning to connect (Tue Aug 15 02:23:40 2017) [sssd[be[domain.ad.com]]] [fo_resolve_service_send] (0x0100): Trying to resolve service 'IPA' (Tue Aug 15 02:23:40 2017) [sssd[be[domain.ad.com]]] [get_port_status] (0x1000): Port status of port 0 for server '(no name)' is 'neutral' (Tue Aug 15 02:23:40 2017) [sssd[be[domain.ad.com]]] [fo_resolve_service_activate_timeout] (0x2000): Resolve timeout set to 6 seconds (Tue Aug 15 02:23:40 2017) [sssd[be[domain.ad.com]]] [resolve_srv_send] (0x0200): The status of SRV lookup is neutral (Tue Aug 15 02:23:40 2017) [sssd[be[domain.ad.com]]] [resolv_discover_srv_next_domain] (0x0400): SRV resolution of service 'ldap'. Will use DNS discovery domain 'domain.ad.com' (Tue Aug 15 02:23:40 2017) [sssd[be[domain.ad.com]]] [resolv_getsrv_send] (0x0100): Trying to resolve SRV record of '_ldap._tcp.domain.ad.com' (Tue Aug 15 02:23:40 2017) [sssd[be[domain.ad.com]]] [sdap_id_release_conn_data] (0x4000): releasing unused connection (Tue Aug 15 02:23:40 2017) [sssd[be[domain.ad.com]]] [schedule_request_timeout] (0x2000): Scheduling a timeout of 6 seconds (Tue Aug 15 02:23:40 2017) [sssd[be[domain.ad.com]]] [schedule_timeout_watcher] (0x2000): Scheduling DNS timeout watcher (Tue Aug 15 02:23:40 2017) [sssd[be[domain.ad.com]]] [unschedule_timeout_watcher] (0x4000): Unscheduling DNS timeout watcher (Tue Aug 15 02:23:40 2017) [sssd[be[domain.ad.com]]] [request_watch_destructor] (0x0400): Deleting request watch (Tue Aug 15 02:23:40 2017) [sssd[be[domain.ad.com]]] [resolv_discover_srv_done] (0x0040): SRV query failed [4]: Domain name not found (Tue Aug 15 02:23:40 2017) [sssd[be[domain.ad.com]]] [fo_set_port_status] (0x0100): Marking port 0 of server '(no name)' as 'not working' (Tue Aug 15 02:23:40 2017) [sssd[be[domain.ad.com]]] [resolve_srv_done] (0x0040): Unable to resolve SRV [1432158235]: SRV record not found (Tue Aug 15 02:23:40 2017) [sssd[be[domain.ad.com]]] [set_srv_data_status] (0x0100): Marking SRV lookup of service 'IPA' as 'not resolved' (Tue Aug 15 02:23:40 2017) [sssd[be[domain.ad.com]]] [be_resolve_server_process] (0x0080): Couldn't resolve server (SRV lookup meta-server), resolver returned [1432158235]: SRV record not found (Tue Aug 15 02:23:40 2017) [sssd[be[domain.ad.com]]] [be_resolve_server_process] (0x1000): Trying with the next one! (Tue Aug 15 02:23:40 2017) [sssd[be[domain.ad.com]]] [fo_resolve_service_send] (0x0100): Trying to resolve service 'IPA' (Tue Aug 15 02:23:40 2017) [sssd[be[domain.ad.com]]] [get_server_status] (0x1000): Status of server 'ipaserver01.ipa.ad.com' is 'name resolved' (Tue Aug 15 02:23:40 2017) [sssd[be[domain.ad.com]]] [get_port_status] (0x1000): Port status of port 0 for server 'ipaserver01.ipa.ad.com' is 'not working' (Tue Aug 15 02:23:40 2017) [sssd[be[domain.ad.com]]] [get_server_status] (0x1000): Status of server 'ipaserver02.ipa.ad.com' is 'name resolved' (Tue Aug 15 02:23:40 2017) [sssd[be[domain.ad.com]]] [get_port_status] (0x1000): Port status of port 0 for server 'ipaserver02.ipa.ad.com' is 'not working' (Tue Aug 15 02:23:40 2017) [sssd[be[domain.ad.com]]] [get_port_status] (0x1000): Port status of port 0 for server '(no name)' is 'not working' (Tue Aug 15 02:23:40 2017) [sssd[be[domain.ad.com]]] [fo_resolve_service_send] (0x0020): No available servers for service 'IPA' (Tue Aug 15 02:23:40 2017) [sssd[be[domain.ad.com]]] [be_resolve_server_done] (0x1000): Server resolution failed: [5]: Input/output error (Tue Aug 15 02:23:40 2017) [sssd[be[domain.ad.com]]] [sdap_id_op_connect_done] (0x0020): Failed to connect, going offline (5 [Input/output error]) (Tue Aug 15 02:23:40 2017) [sssd[be[domain.ad.com]]] [be_mark_offline] (0x2000): Going offline!
These logs are from a system newly joined to FreeIPA.
I know for a fact it's related to the use of a different domain then my IPA realm. I tested with some systems with both dns and ipa realm set to ipa.ad.com and the authentication works just fine.
On Mon, Aug 14, 2017 at 4:22 AM, Jakub Hrozek via FreeIPA-users < freeipa-users@lists.fedorahosted.org> wrote:
On 12 Aug 2017, at 20:14, Alexander Bokovoy via FreeIPA-users <
freeipa-users@lists.fedorahosted.org> wrote:
To close this thread, I helped Alexandre on the IRC. The basic issue is that one needs to plan domain space carefully when using trust to AD. Active Directory is more than just DNS zones, LDAP, Kerberos and friends. Active Directory domain controllers have internal assumptions on what belongs to AD namespace and what is not.
Thank you for driving this towards the end. I knew I was missing something and I’m glad I could learn something new as well. _______________________________________________ FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org To unsubscribe send an email to freeipa-users-leave@lists.fedo rahosted.org
-- Alexandre Pitre alexandre.pitre@gmail.com
FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org To unsubscribe send an email to freeipa-users-leave@lists.fedo rahosted.org
-- / Alexander Bokovoy
On Tue, Aug 15, 2017 at 10:05:50PM -0400, Alexandre Pitre wrote:
Hi Alexander,
You're correct, turns out I wasn't using the correct domain for the --domain parameter. I thought I was. Here's the command I used.
ipa-client-install -U -p admin -w Passw0rd! --enable-dns-updates --mkhomedir --domain=ipa.ad.com --realm=IPA.AD.COM --no-ntp --debug
All of my client hostname are set as "hostname.domain.ad.com", I didn't know that in itself that was enough of a requirement to join them to FreeIPA. Of course, given that the domain is also present in freeipa and the AD trust has been established AFTER the domain was added to freeipa.
I haven't tested yet without the realm parameter. It is possible that I don't need --domain nor --realm parameters ? Does that require the creation of *_ldap._tcp.* srv records in domain.ad.com dns zone?
Taken from the man page:
*When the client machine hostname is not in a subdomain of an IPA server, its domain can be passed with --domain https://www.mankier.com/1/ipa-client-install#--domain option. In that case, both SSSD and Kerberos components have the domain set in the configuration files and will use it to autodiscover IPA servers.*
That line miss directed me, not sure if that's my interpretation. Documentation could benefit from being clearer and having examples.
Since you had to deal with this kind of setup from a user perspective, would you mind proposing a better wording?
Setting krb5_auth_timeout to 120 seconds is also required in my environment as we're dealing with AD DC spreaded all over the globe. To make kerberos negotiation faster, I assume I could specify my AD.COM realm in /etc/krb5.conf with my local site AD DC ?
Yes, currently this is needed. Using the 'site affinity' on the clients is on the roadmap, but not implemented yet.
freeipa-users@lists.fedorahosted.org