New subject: Why is my sssd deployment not doing cross-subdomain AD authentication?

1 Jul 2018


      sssd subject matter experts,
Why is my sssd deployment not doing cross-subdomain AD authentication?
*Background:*
I have a parent AD domain DELL.COM with trusted subdomains AMER.DELL.COM,
APAC.DELL.COM, EMEA.DELL.COM and JAPN.DELL.COM  Each subdomain has a
transitive trust with DELL.COM.
So all subdomains trust each other.
I set up a first test VM deployment using sssd.  I set up the cross
subdomain auth as in:
https://lists.fedorahosted.org/pipermail/sssd-users/2015-February/002648.htm...
It worked great – allowed cross subdomain authentication.  The only thing
it would not do was use tokengroups.  That is, the VM was fully functional,
but I had to add ‘ldap_use_tokengroups = false’  to the sssd.conf file.
My AD experts have advised me that ‘tokengroups’ are an important AD
optimization and I should use them, if at all possible.
Using ldapsearch, I was able to verify that machine account didn’t have the
necessary privileges to query a user’s tokengroups.  Thus, the fault was
mine – that this first sssd deployment couldn’t use tokengroups.
So I did another sssd deployment, using another test VM.  Apparently, I did
the realm join command correct this time, as it’s able to use tokengroups.
BUT!  This second test VM is not allowing cross subdomain authentication
and login.    How do I fix this so that I have use of both tokengroups and
cross subdomain authentication?
(BTW -- Both test VMs are still up and operational, as described above.)
*Details:*
Here is the realm join command used in the second test VM (spikerealmd02):
kinit serviceunixinstall
realm join -v --automatic-id-mapping=no
--computer-ou='OU=Servers,OU=UNIX,DC=AMER,DC=DELL,DC=COM'
--user-principal="host/`hostname
--fqdn`@AMER.DELL.COM"  AMER.DELL.COM
Here is the /etc/realmd.conf file from this second test VM:
[root@spikerealmd02 etc]# cat realmd.conf
[AMER.DELL.COM]
computer-ou = OU=SERVERS,OU=UNIX,DC=AMER,DC=DELL,DC=COM
automatic-id-mapping = no
manage-system = no
fully-qualified-names = no
# THIS FAILS AT DELL;  serviceunixinstall apparently not allowed to create
UPNs associated with machine account.
# Set the user-prinicpal to yes to create userPrincipalName attributes for
the computer account in the realm, in the form host/computer@REALM
#user-principal = yes
[active-directory]
default_client = sssd
[service]
automatic-install = no
[users]
# shouldn't need this;  should be set in AD for each UNIX-enabled user.
default-home = /home/%U
# shouldn't need this;  should be set in AD for each UNIX-enabled user.
default-shell = /bin/bash
Here’s the /etc/sssd/sssd.conf file for this second test VM:
[root@spikerealmd02 sssd]# cat sssd.conf
[sssd]
debug_level = 6
domains = amer.dell.com,apac.dell.com,emea.dell.com,japn.dell.com
domain_resolution_order = amer.dell.com, emea.dell.com, apac.dell.com,
japn.dell.com
config_file_version = 2
services = nss,pam
#ldap_user_member_of = member
[pam]
pam_verbosity = 3
debug_level = 9
[nss]
debug_level = 9
filter_groups = root
filter_users = root
reconnection_retries = 3
#entry_cache_timeout = 300
entry_cache_nowait_percentage = 75
[domain/amer.dell.com]
debug_level = 9
auto_private_groups = True
use_fully_qualified_names = False
ad_domain = amer.dell.com
krb5_realm = AMER.DELL.COM
realmd_tags = joined-with-adcli
cache_credentials = True
id_provider = ad
auth_provider = ad
krb5_store_password_if_offline = True
default_shell = /bin/bash
ldap_id_mapping = False
fallback_homedir = /home/%u
access_provider = simple
#access_provider = ad
ldap_schema = rfc2307bis
ldap_sasl_authid = host/spikerealmd02.us.dell.com@AMER.DELL.COM
#ldap_sasl_authid = SPIKEREALMD02$@AMER.DELL.COM
#ldap_sasl_authid = spikerealmd02@AMER.DELL.COM
ad_enabled_domains = amer.dell.com,apac.dell.com,emea.dell.com,japn.dell.com
,dell.com
dyndns_update = False
subdomains_provider = none
ldap_use_tokengroups = true
simple_allow_groups = amerlinuxsup@AMER.DELL.COM, amerlinuxeng@AMER.DELL.COM,
emealinuxsup@EMEA.DELL.COM, AMER.DELL.COM, emealinuxeng@EMEA.DELL.COM,
apaclinuxsup@EMEA.DELL.COM, apaclinuxeng@EMEA.DELL.COM
# also look at
https://lists.fedorahosted.org/pipermail/sssd-users/2015-February/002648.htm...
[domain/apac.dell.com]
debug_level = 9
auto_private_groups = True
use_fully_qualified_names = False
ad_domain = apac.dell.com
krb5_realm = APAC.DELL.COM
cache_credentials = True
id_provider = ad
auth_provider = ad
krb5_store_password_if_offline = True
default_shell = /bin/bash
ldap_id_mapping = False
fallback_homedir = /home/%u
access_provider = simple
ldap_schema = rfc2307bis
ldap_sasl_authid = host/spikerealmd02.us.dell.com@AMER.DELL.COM
#ldap_sasl_authid = SPIKEREALMD02$@AMER.DELL.COM
#ldap_sasl_authid = spikerealmd02@AMER.DELL.COM
ad_enabled_domains = amer.dell.com, apac.dell.com, apac.dell.com,
japn.dell.com, dell.com
dyndns_update = False
subdomains_provider = none
ldap_use_tokengroups = false
simple_allow_groups = apaclinuxsup@APAC.DELL.COM, apaclinuxeng@APAC.DELL.COM
[domain/emea.dell.com]
debug_level = 9
auto_private_groups = True
use_fully_qualified_names = False
ad_domain = emea.dell.com
krb5_realm = EMEA.DELL.COM
cache_credentials = True
id_provider = ad
auth_provider = ad
krb5_store_password_if_offline = True
default_shell = /bin/bash
ldap_id_mapping = False
fallback_homedir = /home/%u
access_provider = simple
ldap_schema = rfc2307bis
ldap_sasl_authid = host/spikerealmd02.us.dell.com@AMER.DELL.COM
#ldap_sasl_authid = SPIKEREALMD02$@AMER.DELL.COM
#ldap_sasl_authid = spikerealmd02@AMER.DELL.COM
ad_enabled_domains = amer.dell.com, apac.dell.com, emea.dell.com,
japn.dell.com, dell.com
dyndns_update = False
subdomains_provider = none
ldap_use_tokengroups = true
simple_allow_groups = emealinuxsup@EMEA.DELL.COM, emealinuxeng@EMEA.DELL.COM
[domain/japn.dell.com]
debug_level = 9
auto_private_groups = True
use_fully_qualified_names = False
ad_domain = japn.dell.com
krb5_realm = JAPN.DELL.COM
cache_credentials = True
id_provider = ad
auth_provider = ad
krb5_store_password_if_offline = True
default_shell = /bin/bash
ldap_id_mapping = False
fallback_homedir = /home/%u
access_provider = simple
ldap_schema = rfc2307bis
ldap_sasl_authid = host/spikerealmd02.us.dell.com@AMER.DELL.COM
#ldap_sasl_authid = SPIKEREALMD02$@AMER.DELL.COM
#ldap_sasl_authid = spikerealmd02@AMER.DELL.COM
ad_enabled_domains = amer.dell.com, apac.dell.com, japn.dell.com,
japn.dell.com, dell.com
dyndns_update = False
subdomains_provider = none
ldap_use_tokengroups = true
simple_allow_groups = japnlinuxsup@JAPN.DELL.COM, japnlinuxeng@JAPN.DELL.COM
and here’s the /etc/krb5.conf file:
# Configuration snippets may be placed in this directory as well
includedir /etc/krb5.conf.d/
includedir /var/lib/sss/pubconf/krb5.include.d/
[logging]
default = FILE:/var/log/krb5libs.log
kdc = FILE:/var/log/krb5kdc.log
admin_server = FILE:/var/log/kadmind.log
[libdefaults]
# SW mod 5/12/2018
# dns_lookup_realm = false
dns_lookup_realm = false
ticket_lifetime = 24h
renew_lifetime = 7d
forwardable = true
rdns = false
# default_realm = EXAMPLE.COM
default_ccache_name = KEYRING:persistent:%{uid}
default_realm = AMER.DELL.COM
[realms]
# EXAMPLE.COM = {
#  kdc = kerberos.example.com
#  admin_server = kerberos.example.com
# }
# AMER.DELL.COM = {
# }
[domain_realm]
# .example.com = EXAMPLE.COM
# example.com = EXAMPLE.COM
amer.dell.com = AMER.DELL.COM
.amer.dell.com = AMER.DELL.COM
*Comparing with first VM that does cross subdomain auth:*
Here’s /etc/realmd.conf of first test VM that does cross subdomain auth
(spikerealmd01):
[root@spikerealmd01 krb5.include.d]# cat /etc/realmd.conf
[AMER.DELL.COM]
computer-ou = OU=SERVERS,OU=UNIX,DC=AMER,DC=DELL,DC=COM
automatic-id-mapping = no
manage-system = no
fully-qualified-names = no
# THIS FAILS AT DELL;  serviceunixinstall apparently not allowed to create
UPNs associated with machine account.
# Set the user-prinicpal to yes to create userPrincipalName attributes for
the computer account in the realm, in the form host/computer@REALM
#user-principal = yes
[active-directory]
default_client = sssd
[service]
automatic-install = no
[users]
# shouldn't need this;  should be set in AD for each UNIX-enabled user.
default-home = /home/%U
# shouldn't need this;  should be set in AD for each UNIX-enabled user.
default-shell = /bin/bash
Here’s /etc/sssd/sssd.conf file from same first test VM:
[root@spikerealmd01 sssd]# cat sssd.conf
[sssd]
debug_level = 6
domains = amer.dell.com, apac.dell.com, emea.dell.com, japn.dell.com
domain_resolution_order = amer.dell.com, emea.dell.com, apac.dell.com,
japn.dell.com
config_file_version = 2
services = nss, pam
#ldap_user_member_of = member
[pam]
pam_verbosity = 3
debug_level = 9
[nss]
debug_level = 9
filter_groups = root
filter_users = root
reconnection_retries = 3
#entry_cache_timeout = 300
entry_cache_nowait_percentage = 75
[domain/amer.dell.com]
debug_level = 9
auto_private_groups = True
use_fully_qualified_names = False
ad_domain = amer.dell.com
krb5_realm = AMER.DELL.COM
realmd_tags = joined-with-adcli
cache_credentials = True
id_provider = ad
auth_provider = ad
krb5_store_password_if_offline = True
default_shell = /bin/bash
ldap_id_mapping = False
fallback_homedir = /home/%u
access_provider = simple
ldap_schema = rfc2307bis
#ldap_sasl_authid = host/spikerealmd01.us.dell.com
#ldap_sasl_authid = SPIKEREALMD01$@AMER.DELL.COM
ldap_sasl_authid = spikerealmd01@AMER.DELL.COM
ad_enabled_domains = amer.dell.com, apac.dell.com, emea.dell.com,
japn.dell.com, dell.com
dyndns_update = False
subdomains_provider = none
ldap_use_tokengroups = false
simple_allow_groups = amerlinuxsup@AMER.DELL.COM, amerlinuxeng@AMER.DELL.COM
# also look at
https://lists.fedorahosted.org/pipermail/sssd-users/2015-February/002648.htm...
[domain/apac.dell.com]
debug_level = 9
auto_private_groups = True
use_fully_qualified_names = False
ad_domain = apac.dell.com
krb5_realm = APAC.DELL.COM
realmd_tags = joined-with-adcli
cache_credentials = True
id_provider = ad
auth_provider = ad
krb5_store_password_if_offline = True
default_shell = /bin/bash
ldap_id_mapping = False
fallback_homedir = /home/%u
access_provider = simple
ldap_schema = rfc2307bis
#ldap_sasl_authid = host/spikerealmd01.us.dell.com
#ldap_sasl_authid = SPIKEREALMD01$@AMER.DELL.COM
ldap_sasl_authid = spikerealmd01@AMER.DELL.COM
ad_enabled_domains = amer.dell.com, apac.dell.com, emea.dell.com,
japn.dell.com, dell.com
dyndns_update = False
subdomains_provider = none
ldap_use_tokengroups = false
simple_allow_groups = apaclinuxsup@APAC.DELL.COM, apaclinuxeng@APAC.DELL.COM
[domain/emea.dell.com]
debug_level = 9
auto_private_groups = True
use_fully_qualified_names = False
ad_domain = emea.dell.com
krb5_realm = EMEA.DELL.COM
realmd_tags = joined-with-adcli
cache_credentials = True
id_provider = ad
auth_provider = ad
krb5_store_password_if_offline = True
default_shell = /bin/bash
ldap_id_mapping = False
fallback_homedir = /home/%u
access_provider = simple
ldap_schema = rfc2307bis
#ldap_sasl_authid = host/spikerealmd01.us.dell.com
#ldap_sasl_authid = SPIKEREALMD01$@AMER.DELL.COM
ldap_sasl_authid = spikerealmd01@AMER.DELL.COM
ad_enabled_domains = amer.dell.com, apac.dell.com, emea.dell.com,
japn.dell.com, dell.com
dyndns_update = False
subdomains_provider = none
ldap_use_tokengroups = false
simple_allow_groups = emealinuxsup@EMEA.DELL.COM, emealinuxeng@EMEA.DELL.COM
[domain/japn.dell.com]
debug_level = 9
auto_private_groups = True
use_fully_qualified_names = False
ad_domain = japn.dell.com
krb5_realm = JAPN.DELL.COM
realmd_tags = joined-with-adcli
cache_credentials = True
id_provider = ad
auth_provider = ad
krb5_store_password_if_offline = True
default_shell = /bin/bash
ldap_id_mapping = False
fallback_homedir = /home/%u
access_provider = simple
ldap_schema = rfc2307bis
#ldap_sasl_authid = host/spikerealmd01.us.dell.com
#ldap_sasl_authid = SPIKEREALMD01$@AMER.DELL.COM
ldap_sasl_authid = spikerealmd01@AMER.DELL.COM
ad_enabled_domains = amer.dell.com, apac.dell.com, emea.dell.com,
japn.dell.com, dell.com
dyndns_update = False
subdomains_provider = none
ldap_use_tokengroups = false
simple_allow_groups = japnlinuxsup@JAPN.DELL.COM, japnlinuxeng@JAPN.DELL.COM,
linux-core-engineering, amer.dell.com
Here’s /etc/krb5.conf file:
[root@spikerealmd01 etc]# cat krb5.conf
# Configuration snippets may be placed in this directory as well
includedir /etc/krb5.conf.d/
includedir /var/lib/sss/pubconf/krb5.include.d/
[logging]
default = FILE:/var/log/krb5libs.log
kdc = FILE:/var/log/krb5kdc.log
admin_server = FILE:/var/log/kadmind.log
[libdefaults]
# SW mod 5/12/2018
# dns_lookup_realm = false
dns_lookup_realm = false
ticket_lifetime = 24h
renew_lifetime = 7d
forwardable = true
rdns = false
# default_realm = EXAMPLE.COM
default_ccache_name = KEYRING:persistent:%{uid}
default_realm = AMER.DELL.COM
[realms]
# EXAMPLE.COM = {
#  kdc = kerberos.example.com
#  admin_server = kerberos.example.com
# }
# AMER.DELL.COM = {
# }
[domain_realm]
# .example.com = EXAMPLE.COM
# example.com = EXAMPLE.COM
amer.dell.com = AMER.DELL.COM
.amer.dell.com = AMER.DELL.COM
[root@spikerealmd01 etc]#
*Other details:*
If I query group membership of an engineer in APAC:
id admjesse_chan
on the good VM (spikerealmd01) I see all expected groups and I see  this in
the /var/log/sssd/sssd_apac.dell.com.log file:
…
(Sun Jul  1 15:14:30 2018) [sssd[be[apac.dell.com]]]
[sdap_initgr_rfc2307bis_next_base] (0x0400): Searching for parent groups
for user [CN=AdmJesse_Chan,OU=ADMAccounts,DC=apac,DC=dell,DC=com] with base
[DC=apac,DC=dell,DC=com]
(Sun Jul  1 15:14:30 2018) [sssd[be[apac.dell.com]]]
[sdap_get_generic_ext_step] (0x0400): calling ldap_search_ext with
[(&(member=CN=AdmJesse_Chan,OU=ADMAccounts,DC=apac,DC=dell,DC=com)(objectClass=group)(sAMAccountName=*))][DC=apac,DC=dell,DC=com].
(Sun Jul  1 15:14:31 2018) [sssd[be[apac.dell.com]]]
[sdap_initgr_rfc2307bis_process] (0x1000): Found 4 parent groups for user [
AdmJesse_Chan@apac.dell.com]
(Sun Jul  1 15:14:31 2018) [sssd[be[apac.dell.com]]]
[sysdb_get_direct_parents] (0x2000): searching sysdb with filter
[(&(objectCategory=group)(member=name=AdmJesse_Chan@apac.dell.com
,cn=users,cn=apac.dell.com,cn=sysdb))]
(Sun Jul  1 15:14:31 2018) [sssd[be[apac.dell.com]]]
[sysdb_get_direct_parents] (0x1000): AdmJesse_Chan@apac.dell.com is a
member of 4 sysdb groups
(Sun Jul  1 15:14:31 2018) [sssd[be[apac.dell.com]]]
[save_rfc2307bis_user_memberships] (0x2000): Updating memberships for
AdmJesse_Chan@apac.dell.com
(Sun Jul  1 15:14:31 2018) [sssd[be[apac.dell.com]]] [sysdb_set_entry_attr]
(0x0200): Entry
[name=AdmJesse_Chan@apac.dell.com,cn=users,cn=apac.dell.com,cn=sysdb]
has set [ts_cache] attrs.
(Sun Jul  1 15:14:31 2018) [sssd[be[apac.dell.com]]]
[dp_table_value_destructor] (0x0400): Removing [0:1:0x0001:3::apac.dell.com:
name=admjesse_chan@apac.dell.com] from reply table
(Sun Jul  1 15:14:35 2018) [sssd[be[apac.dell.com]]]
[sdap_asq_search_parse_entry] (0x2000): Matched objectclass [user] on DN
[CN=AdmJesse_Chan,OU=ADMAccounts,DC=apac,DC=dell,DC=com], will use
associated map
(Sun Jul  1 15:14:35 2018) [sssd[be[apac.dell.com]]] [sdap_parse_entry]
(0x1000): OriginalDN:
[CN=AdmJesse_Chan,OU=ADMAccounts,DC=apac,DC=dell,DC=com].
(Sun Jul  1 15:14:35 2018) [sssd[be[apac.dell.com]]]
[sdap_asq_search_parse_entry] (0x2000): DN
[CN=AdmJesse_Chan,OU=ADMAccounts,DC=apac,DC=dell,DC=com] did not match the
objectClass [group]
(Sun Jul  1 15:14:36 2018) [sssd[be[apac.dell.com]]]
[sdap_nested_group_hash_insert] (0x4000): Inserting
[CN=AdmJesse_Chan,OU=ADMAccounts,DC=apac,DC=dell,DC=com] into hash table
[users]
(Sun Jul  1 15:14:37 2018) [sssd[be[apac.dell.com]]]
[sdap_get_primary_name] (0x0400): Processing object AdmJesse_Chan
(Sun Jul  1 15:14:37 2018) [sssd[be[apac.dell.com]]]
[sysdb_cache_search_users] (0x2000): Search users with filter:
(&(objectCategory=user)(originalDN=CN=AdmJesse_Chan,OU=ADMAccounts,DC=apac,DC=dell,DC=com))
(Sun Jul  1 15:14:37 2018) [sssd[be[apac.dell.com]]]
[sdap_find_entry_by_origDN] (0x4000): Searching cache for
[CN=AdmJesse_Chan,OU=ADMAccounts,DC=apac,DC=dell,DC=com].
(Sun Jul  1 15:14:37 2018) [sssd[be[apac.dell.com]]]
[sdap_fill_memberships] (0x1000):     member #2019 (CN=AdmJesse_Cha
,OU=ADMAccounts,DC=apac,DC=dell,DC=com): [name=AdmJesse_Chan@apac.dell.com
,cn=users,cn=apac.dell.com,cn=sysdb]
If I do the same query on the bad VM (spikerealmd02):
[root@spikerealmd02 sssd]# id admjesse_chan
id: admjesse_chan: no such user
and I see nothing in the sssd_apac.dell.com.log file:
[root@spikerealmd02 sssd]# grep -i admjesse_chan sssd_apac.dell.com.log
[root@spikerealmd02 sssd]#
Please help,
Spike