Hi,
I think this problem may be part (or related to) the "FreeIPA/SSSD LDAP cross-forest trust slow queries" issue, but I'm not sure.
We've been testing sssd on our RHEL6 and RHEL7 hosts, using the latest available packages. We have a fairly simple sssd configuration. We use the "ad" provider with LDAP id mapping:
[sssd] config_file_version = 2 debug_level = 0x0070 domains = example.org services = nss, pam
[nss] debug_level = 0x0070 default_shell = /bin/bash fallback_homedir = /home/%u
[pam] debug_level = 0x0070
[domain/example.org] access_provider = ad auth_provider = ad cache_credentials = true chpass_provider = ad debug_level = 0x0010 dyndns_update = false enumerate = true id_provider = ad krb5_realm = EXAMPLE.ORG ldap_id_mapping = true ldap_sasl_mech = GSSAPI ldap_schema = AD offline_failed_login_attempts = 3
Although we have about 1200 groups in Active Directory, only about 900 of them show up in the output of "getent -s sss group". For the remaining ones, attempting a lookup (e.g., with getent) returns no entry, and we see these lines in the sssd_example.org.log file:
(Wed May 6 00:03:06 2015) [sssd[be[example.org]]] [sdap_get_groups_next_base] (0x0400): Searching for groups with base [DC=example,DC=org] (Wed May 6 00:03:06 2015) [sssd[be[example.org]]] [sdap_get_generic_ext_step] (0x0400): calling ldap_search_ext with [(&(name=my-group)(objectClass=group)(name=*))][DC=example,DC=org]. (Wed May 6 00:03:06 2015) [sssd[be[example.org]]] [sdap_parse_entry] (0x1000): OriginalDN: [CN=my-group,OU=Distribution Groups,OU=Microsoft Exchange,DC=example,DC=org]. (Wed May 6 00:03:06 2015) [sssd[be[example.org]]] [sdap_nested_group_hash_group] (0x4000): AD group has type flags 0x8. (Wed May 6 00:03:06 2015) [sssd[be[example.org]]] [sdap_nested_group_hash_group] (0x0400): Filtering AD group. (Wed May 6 00:03:06 2015) [sssd[be[example.org]]] [sdap_nested_group_hash_group] (0x4000): The group's gid was zero (Wed May 6 00:03:06 2015) [sssd[be[example.org]]] [sdap_nested_group_hash_group] (0x2000): Marking group as non-posix and setting GID=0! (Wed May 6 00:03:06 2015) [sssd[be[example.org]]] [sdap_nested_group_hash_entry] (0x4000): Inserting [CN=my-group,OU=Distribution Groups,OU=Microsoft Exchange,DC=example,DC=org] into hash table [groups] (Wed May 6 00:03:06 2015) [sssd[be[example.org]]] [sdap_nested_group_process_send] (0x2000): About to process group [CN=my-group,OU=Distribution Groups,OU=Microsoft Exchange,DC=example,DC=org]
[user lookups omitted]
(Wed May 6 00:03:06 2015) [sssd[be[example.org]]] [sdap_get_primary_name] (0x0400): Processing object my-group (Wed May 6 00:03:06 2015) [sssd[be[example.org]]] [sdap_save_group] (0x0400): Processing group my-group (Wed May 6 00:03:06 2015) [sssd[be[example.org]]] [sdap_save_group] (0x4000): AD group [my-group] has type flags 0x8. (Wed May 6 00:03:06 2015) [sssd[be[example.org]]] [sdap_save_group] (0x0400): Filtering AD group [my-group]. (Wed May 6 00:03:06 2015) [sssd[be[example.org]]] [sdap_attrs_add_ldap_attr] (0x2000): Adding original DN [CN=my-group,OU=Distribution Groups,OU=Microsoft Exchange,DC=example,DC=org] to attributes of [my-group]. (Wed May 6 00:03:06 2015) [sssd[be[example.org]]] [sdap_attrs_add_ldap_attr] (0x2000): Adding original mod-Timestamp [20150429215241.0Z] to attributes of [my-group]. (Wed May 6 00:03:06 2015) [sssd[be[example.org]]] [sdap_process_ghost_members] (0x0400): The group has 5 members (Wed May 6 00:03:06 2015) [sssd[be[example.org]]] [sdap_process_ghost_members] (0x0400): Group has 5 members (Wed May 6 00:03:06 2015) [sssd[be[example.org]]] [sysdb_attrs_get_aliases] (0x2000): Domain is case-insensitive; will add lowercased aliases (Wed May 6 00:03:06 2015) [sssd[be[example.org]]] [sdap_save_group] (0x0400): Storing info for group my-group (Wed May 6 00:03:06 2015) [sssd[be[example.org]]] [sysdb_set_entry_attr] (0x0080): ldb_modify failed: [Attribute or value exists] (Wed May 6 00:03:06 2015) [sssd[be[example.org]]] [sysdb_set_entry_attr] (0x0040): Error: 17 (File exists) (Wed May 6 00:03:06 2015) [sssd[be[example.org]]] [sysdb_store_group] (0x1000): sysdb_set_group_attr failed. (Wed May 6 00:03:06 2015) [sssd[be[example.org]]] [sysdb_store_group] (0x0400): Error: 17 (File exists) (Wed May 6 00:03:06 2015) [sssd[be[example.org]]] [sdap_store_group_with_gid] (0x0040): Could not store group my-group (Wed May 6 00:03:06 2015) [sssd[be[example.org]]] [sdap_save_group] (0x0080): Could not store group with GID: [File exists] (Wed May 6 00:03:06 2015) [sssd[be[example.org]]] [sdap_save_group] (0x0080): Failed to save group [my-group]: [File exists] (Wed May 6 00:03:06 2015) [sssd[be[example.org]]] [sdap_save_groups] (0x0040): Failed to store group 0. Ignoring.
I can't find any pattern for what groups are correctly mapped versus the groups that aren't. But it's consistent across different sssd client hosts. Stopping sssd and removing its cache has no effect. Nor does disabling enumeration.
Since we just recently started testing sssd, we don't know for certain whether group lookups worked properly in the past. All we know is that it doesn't seem to work now.
Is this a known issue, or is this a new problem I should report to Bugzilla and/or Red Hat support?
If the former, is a patch available? Because if so, I'll rebuild our sssd RPMs locally with the patch until Red Hat releases official updated packages...
Thanks in advance for any assistance anyone might have...
On (06/05/15 01:12), James Ralston wrote:
Hi,
I think this problem may be part (or related to) the "FreeIPA/SSSD LDAP cross-forest trust slow queries" issue, but I'm not sure.
We've been testing sssd on our RHEL6 and RHEL7 hosts, using the latest available packages. We have a fairly simple sssd configuration. We use the "ad" provider with LDAP id mapping:
[sssd] config_file_version = 2 debug_level = 0x0070 domains = example.org services = nss, pam
[nss] debug_level = 0x0070 default_shell = /bin/bash fallback_homedir = /home/%u
[pam] debug_level = 0x0070
[domain/example.org] access_provider = ad auth_provider = ad cache_credentials = true chpass_provider = ad debug_level = 0x0010 dyndns_update = false enumerate = true
I Hope it was just for testing purposes. We do not recommend to enable enumeration.
id_provider = ad krb5_realm = EXAMPLE.ORG ldap_id_mapping = true
You can remove this line, id-mapping is enabled by default for id_provider ad
ldap_sasl_mech = GSSAPI ldap_schema = AD
Previous 2 lines are efault for id_provider ad as well
offline_failed_login_attempts = 3
This line shoudl be in [pam] section, it will have effect only if "cache_credentials" is enabled in domain section.
I would be curious where did you inspire in sssd.conf. So we can improve it.
Although we have about 1200 groups in Active Directory, only about 900 of them show up in the output of "getent -s sss group". For the remaining ones, attempting a lookup (e.g., with getent) returns no entry, and we see these lines in the sssd_example.org.log file:
(Wed May 6 00:03:06 2015) [sssd[be[example.org]]] [sdap_get_groups_next_base] (0x0400): Searching for groups with base [DC=example,DC=org] (Wed May 6 00:03:06 2015) [sssd[be[example.org]]] [sdap_get_generic_ext_step] (0x0400): calling ldap_search_ext with [(&(name=my-group)(objectClass=group)(name=*))][DC=example,DC=org]. (Wed May 6 00:03:06 2015) [sssd[be[example.org]]] [sdap_parse_entry] (0x1000): OriginalDN: [CN=my-group,OU=Distribution Groups,OU=Microsoft Exchange,DC=example,DC=org]. (Wed May 6 00:03:06 2015) [sssd[be[example.org]]] [sdap_nested_group_hash_group] (0x4000): AD group has type flags 0x8.
^^^^^^^^^ here
(Wed May 6 00:03:06 2015) [sssd[be[example.org]]] [sdap_nested_group_hash_group] (0x0400): Filtering AD group. (Wed May 6 00:03:06 2015) [sssd[be[example.org]]] [sdap_nested_group_hash_group] (0x4000): The group's gid was zero (Wed May 6 00:03:06 2015) [sssd[be[example.org]]] [sdap_nested_group_hash_group] (0x2000): Marking group as non-posix and setting GID=0! (Wed May 6 00:03:06 2015) [sssd[be[example.org]]] [sdap_nested_group_hash_entry] (0x4000): Inserting [CN=my-group,OU=Distribution Groups,OU=Microsoft Exchange,DC=example,DC=org] into hash table [groups] (Wed May 6 00:03:06 2015) [sssd[be[example.org]]] [sdap_nested_group_process_send] (0x2000): About to process group [CN=my-group,OU=Distribution Groups,OU=Microsoft Exchange,DC=example,DC=org]
[user lookups omitted]
(Wed May 6 00:03:06 2015) [sssd[be[example.org]]] [sdap_get_primary_name] (0x0400): Processing object my-group (Wed May 6 00:03:06 2015) [sssd[be[example.org]]] [sdap_save_group] (0x0400): Processing group my-group (Wed May 6 00:03:06 2015) [sssd[be[example.org]]] [sdap_save_group] (0x4000): AD group [my-group] has type flags 0x8.
^^^^^^^^^ and here is a type of group [1] Distributions grups are filtered by defaul. Technet[1] says: "Distribution groups are not security-enabled, which means that they cannot be listed in discretionary access control lists (DACLs). If you need a group for controlling access to shared resources, create a security group."
(Wed May 6 00:03:06 2015) [sssd[be[example.org]]] [sdap_save_group] (0x0400): Filtering AD group [my-group]. (Wed May 6 00:03:06 2015) [sssd[be[example.org]]] [sdap_attrs_add_ldap_attr] (0x2000): Adding original DN [CN=my-group,OU=Distribution Groups,OU=Microsoft Exchange,DC=example,DC=org] to attributes of [my-group]. (Wed May 6 00:03:06 2015) [sssd[be[example.org]]] [sdap_attrs_add_ldap_attr] (0x2000): Adding original mod-Timestamp [20150429215241.0Z] to attributes of [my-group]. (Wed May 6 00:03:06 2015) [sssd[be[example.org]]] [sdap_process_ghost_members] (0x0400): The group has 5 members (Wed May 6 00:03:06 2015) [sssd[be[example.org]]] [sdap_process_ghost_members] (0x0400): Group has 5 members (Wed May 6 00:03:06 2015) [sssd[be[example.org]]] [sysdb_attrs_get_aliases] (0x2000): Domain is case-insensitive; will add lowercased aliases (Wed May 6 00:03:06 2015) [sssd[be[example.org]]] [sdap_save_group] (0x0400): Storing info for group my-group (Wed May 6 00:03:06 2015) [sssd[be[example.org]]] [sysdb_set_entry_attr] (0x0080): ldb_modify failed: [Attribute or value exists]
^^^^^^^^^^^^^^^^^ here is a problem It is very likely upstream bug[2] with binary objectGUID
(Wed May 6 00:03:06 2015) [sssd[be[example.org]]] [sysdb_set_entry_attr] (0x0040): Error: 17 (File exists) (Wed May 6 00:03:06 2015) [sssd[be[example.org]]] [sysdb_store_group] (0x1000): sysdb_set_group_attr failed. (Wed May 6 00:03:06 2015) [sssd[be[example.org]]] [sysdb_store_group] (0x0400): Error: 17 (File exists) (Wed May 6 00:03:06 2015) [sssd[be[example.org]]] [sdap_store_group_with_gid] (0x0040): Could not store group my-group (Wed May 6 00:03:06 2015) [sssd[be[example.org]]] [sdap_save_group] (0x0080): Could not store group with GID: [File exists] (Wed May 6 00:03:06 2015) [sssd[be[example.org]]] [sdap_save_group] (0x0080): Failed to save group [my-group]: [File exists] (Wed May 6 00:03:06 2015) [sssd[be[example.org]]] [sdap_save_groups] (0x0040): Failed to store group 0. Ignoring.
I can't find any pattern for what groups are correctly mapped versus the groups that aren't. But it's consistent across different sssd client hosts. Stopping sssd and removing its cache has no effect. Nor does disabling enumeration.
For testing purposes you can try to use pre-release[3] of upstream 1.12.5 it shoudl be released within few days and it contains fix for bug[2] and also other fixes.
Since we just recently started testing sssd, we don't know for certain whether group lookups worked properly in the past. All we know is that it doesn't seem to work now.
LS
[1] https://technet.microsoft.com/en-us/library/cc781446%28v=ws.10%29.aspx [2] https://fedorahosted.org/sssd/ticket/2588 [3] https://copr.fedoraproject.org/coprs/lslebodn/sssd-1-12-latest/
On Wed, May 06, 2015 at 07:47:11AM +0200, Lukas Slebodnik wrote:
On (06/05/15 01:12), James Ralston wrote:
Hi,
I think this problem may be part (or related to) the "FreeIPA/SSSD LDAP cross-forest trust slow queries" issue, but I'm not sure.
We've been testing sssd on our RHEL6 and RHEL7 hosts, using the latest available packages. We have a fairly simple sssd configuration. We use the "ad" provider with LDAP id mapping:
[sssd] config_file_version = 2 debug_level = 0x0070 domains = example.org services = nss, pam
[nss] debug_level = 0x0070 default_shell = /bin/bash fallback_homedir = /home/%u
[pam] debug_level = 0x0070
[domain/example.org] access_provider = ad auth_provider = ad cache_credentials = true chpass_provider = ad debug_level = 0x0010 dyndns_update = false enumerate = true
I Hope it was just for testing purposes. We do not recommend to enable enumeration.
You know, just this morning, I was thinking about enumeration. It doesn't work for IPA views at all for example. It doesn't work for trusted domains at all either (except for some limited support in AD trusted domains that is very untested)
I wonder if we could just remove enumeration from IPA and AD back ends in some major release.
It's just a legacy feature, so those who need it can fall back to the LDAP provider..
Hi Lukas,
On Wed, May 6, 2015 at 1:47 AM, Lukas Slebodnik lslebodn@redhat.com wrote:
On (06/05/15 01:12), James Ralston wrote:
enumerate = true
I Hope it was just for testing purposes. We do not recommend to enable enumeration.
I know it's not recommended. I'll address this in a separate response.
ldap_id_mapping = true
You can remove this line, id-mapping is enabled by default for id_provider ad
ldap_sasl_mech = GSSAPI ldap_schema = AD
Previous 2 lines are efault for id_provider ad as well
I know. (I added those lines more for illustration purposes than anything else.)
offline_failed_login_attempts = 3
This line shoudl be in [pam] section, it will have effect only if "cache_credentials" is enabled in domain section.
Ah; good to know. I will correct that. Thanks.
I would be curious where did you inspire in sssd.conf. So we can improve it.
I created it myself. So, blame me. ;-)
Distributions grups are filtered by default.
Technet[1] says:
"Distribution groups are not security-enabled, which means that they cannot be listed in discretionary access control lists (DACLs). If you need a group for controlling access to shared resources, create a security group."
OK.
A suggestion: it would have been very helpful if the debug messages had contained some statement like "ignoring distribution group my-group".
That would have made it much more clear what was happening, which (as I understand it) was:
1. sssd was ignoring a distribution group. (This is normal, expected behavior.)
2. sssd was trying to cache the distribution group, but failing due to ticket/2588. (This is bug, not normal behavior.)
(Wed May 6 00:03:06 2015) [sssd[be[example.org]]] [sysdb_set_entry_attr] (0x0080): ldb_modify failed: [Attribute or value exists]
^^^^^^^^^^^^^^^^^ here is a problemIt is very likely upstream bug[2] with binary objectGUID
OK, thanks for the explanation and the pointer.
For testing purposes you can try to use pre-release[3] of upstream 1.12.5 it should be released within few days and it contains fix for bug[2] and also other fixes.
OK.
What do you recommend doing for RHEL6 (currently on 1.11.6-30.el6_6.4)?
1. Use your 1.12.5 packages on RHEL6?
2. Wait for Red Hat to backport the patch for ticket/2588 to their 1.11.6 branch?
3. Wait for Red Hat to rebase RHEL6 to 1.12.5?
4. Backport the patch for ticket/2588 to 1.11.6-30.el6_6.4 myself?
5. Something else?
Thanks...
On Wed, May 06, 2015 at 01:02:22PM -0400, James Ralston wrote:
What do you recommend doing for RHEL6 (currently on 1.11.6-30.el6_6.4)?
1. Use your 1.12.5 packages on RHEL6? 2. Wait for Red Hat to backport the patch for ticket/2588 to their 1.11.6 branch?
Unlikely to happen w/o a support case open and even so, a RHEL-6.6 update would have to be justified.
3. Wait for Red Hat to rebase RHEL6 to 1.12.5?
RHEL-6.7 will rebase to sssd-1-12. If you want to stay on the supported patch, this is the best option. btw 6.7 beta was released just today.
4. Backport the patch for ticket/2588 to 1.11.6-30.el6_6.4 myself?
The unsupported options are really up to you. If you can cherry-pick the single patch itself, that would create the smallest possible delta compared to the vanilla RHEL packages.
On Wed, May 6, 2015 at 1:26 PM, Jakub Hrozek jhrozek@redhat.com wrote:
On Wed, May 06, 2015 at 01:02:22PM -0400, James Ralston wrote:
- Wait for Red Hat to rebase RHEL6 to 1.12.5?
RHEL-6.7 will rebase to sssd-1-12. If you want to stay on the supported patch, this is the best option. btw 6.7 beta was released just today.
Agreed; we don't care about RHEL 6.6 if 6.7 is coming and has sssd 1.12.
On Wed, May 06, 2015 at 09:07:23PM -0400, James Ralston wrote:
On Wed, May 6, 2015 at 1:26 PM, Jakub Hrozek jhrozek@redhat.com wrote:
On Wed, May 06, 2015 at 01:02:22PM -0400, James Ralston wrote:
- Wait for Red Hat to rebase RHEL6 to 1.12.5?
RHEL-6.7 will rebase to sssd-1-12. If you want to stay on the supported patch, this is the best option. btw 6.7 beta was released just today.
Agreed; we don't care about RHEL 6.6 if 6.7 is coming and has sssd 1.12.
Thanks; any testing of 1.12 is greatly appreciated.
sssd-users@lists.fedorahosted.org
-
Jakub Hrozek -
James Ralston -
Lukas Slebodnik