On Wed, May 6, 2015 at 4:27 AM, Jakub Hrozek jhrozek@redhat.com wrote:
You know, just this morning, I was thinking about enumeration. It doesn't work for IPA views at all for example. It doesn't work for trusted domains at all either (except for some limited support in AD trusted domains that is very untested)
I wonder if we could just remove enumeration from IPA and AD back ends in some major release.
Please don't do this.
Enumeration is a very useful feature. It allows us to do things like this:
$ getent passwd | grep -i lastname
The equivalent ldapsearch command is much more tedious:
$ ldapsearch -z 0 -E pr=2147483647/noprompt -o ldif-wrap=no -L -L -H 'ldap:///dc%3Dexample%2Cdc%3Dorg -Y GSSAPI -N -b "dc=example,dc=org" "(&(objectClass=user)(cn=*lastname*))" dn cn sAMAccountName
More generically, enumeration is the way Unix/Linux has always worked. Even getting users to change from:
grep -i lastname /etc/passwd
To this:
getent passwd | grep -i lastname
...has been a struggle.
We also have various services that (unfortuantely) pre-load the passwd and group files at startup by enumerating them with getpwent_r() and getgrent_r(), instead of using the get*nam_r() and get*id_r() functions as-needed. These services break outright if enumeration is disabled.
(Yes, these services are broken. Yes, they shouldn't do that. But our ability to fix them is extremely limited at best, because we don't control them.)
Finally, we have many systems that cannot be joined to Active Directory (for policy reasons, not technical reasons). But we want to use the same passwd/group entries on those systems as returned by sssd on hosts that are joined to Active Directory. We do this by scraping the output of "getent -s sss passwd" and "getent -s sss group" and manually merging it into the local passwd and group files (respectively) on these hosts.
It's just a legacy feature, so those who need it can fall back to the LDAP provider..
But the LDAP provider doesn't support ID mapping; only the AD provider does. And ID mapping is the main reason we use sssd.
I'm not asking you to make enumeration the default. It shouldn't be; it should be something you only turn on if you need it, and you KNOW you need it. But if you need it, you NEED it. Please don't take it away.
----- Original Message -----
From: "James Ralston" ralston@pobox.com To: "End-user discussions about the System Security Services Daemon" sssd-users@lists.fedorahosted.org Sent: Wednesday, May 6, 2015 7:28:35 PM Subject: [SSSD-users] please do not remove enumeration from AD provider
On Wed, May 6, 2015 at 4:27 AM, Jakub Hrozek jhrozek@redhat.com wrote:
You know, just this morning, I was thinking about enumeration. It doesn't work for IPA views at all for example. It doesn't work for trusted domains at all either (except for some limited support in AD trusted domains that is very untested)
I wonder if we could just remove enumeration from IPA and AD back ends in some major release.
Please don't do this.
Enumeration is a very useful feature. It allows us to do things like this:
$ getent passwd | grep -i lastname
The equivalent ldapsearch command is much more tedious:
$ ldapsearch -z 0 -E pr=2147483647/noprompt -o ldif-wrap=no -L -L -H 'ldap:///dc%3Dexample%2Cdc%3Dorg -Y GSSAPI -N -b "dc=example,dc=org" "(&(objectClass=user)(cn=*lastname*))" dn cn sAMAccountName
More generically, enumeration is the way Unix/Linux has always worked. Even getting users to change from:
grep -i lastname /etc/passwdTo this:
getent passwd | grep -i lastname...has been a struggle.
We also have various services that (unfortuantely) pre-load the passwd and group files at startup by enumerating them with getpwent_r() and getgrent_r(), instead of using the get*nam_r() and get*id_r() functions as-needed. These services break outright if enumeration is disabled.
(Yes, these services are broken. Yes, they shouldn't do that. But our ability to fix them is extremely limited at best, because we don't control them.)
Finally, we have many systems that cannot be joined to Active Directory (for policy reasons, not technical reasons). But we want to use the same passwd/group entries on those systems as returned by sssd on hosts that are joined to Active Directory. We do this by scraping the output of "getent -s sss passwd" and "getent -s sss group" and manually merging it into the local passwd and group files (respectively) on these hosts.
It's just a legacy feature, so those who need it can fall back to the LDAP provider..
But the LDAP provider doesn't support ID mapping; only the AD provider does. And ID mapping is the main reason we use sssd.
ID mapping should work with LDAP provider (+ AD) But auto-discovery of domain SID does not work with ldap provider. So you need to configure it manually.
Here is a n example [domain/ad.example.com] debug_level = 0xFFF0 id_provider = ldap ldap_uri = ldap://$AD_SERVER1 ldap_schema = ad ldap_default_bind_dn = $AD_SERVER1_BINDDN ldap_default_authtok = $AD_SERVER1_BINDPASS ldap_tls_cacert = /etc/openldap/certs/ad_cert.pem
ldap_id_mapping = True ldap_idmap_default_domain_sid=S-1-5-21-1111111-2222222-3333333 ldap_idmap_default_domain=ad.example.com
But I would not recommend to use ldap+krb5 instead of ldap_defaul_bind_dn You can find some details in RHEL7 documentation[1]
LS
[1] https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/7/htm...
----- Original Message -----
From: "Lukas Slebodnik" lslebodn@redhat.com To: "End-user discussions about the System Security Services Daemon" sssd-users@lists.fedorahosted.org Sent: Wednesday, May 6, 2015 2:37:42 PM Subject: Re: [SSSD-users] please do not remove enumeration from AD provider
----- Original Message -----
From: "James Ralston" ralston@pobox.com To: "End-user discussions about the System Security Services Daemon" sssd-users@lists.fedorahosted.org Sent: Wednesday, May 6, 2015 7:28:35 PM Subject: [SSSD-users] please do not remove enumeration from AD provider
...
But the LDAP provider doesn't support ID mapping; only the AD provider does. And ID mapping is the main reason we use sssd.
ID mapping should work with LDAP provider (+ AD)
Yes, it does. "ldap_id_mapping = True".
But auto-discovery of domain SID does not work with ldap provider. So you need to configure it manually.
This statement is completely false. The domain SID is automatically detected. Setting it manually like this just means that instead of getting the automatically-determined ID range slice, it will always take slice 0.
...
But I would not recommend to use ldap+krb5 instead of ldap_defaul_bind_dn You can find some details in RHEL7 documentation[1]
I'm not sure what you were trying to say here. I think you meant to say "It's much preferred to use GSSAPI with LDAP and a kerberos keytab to secure your communication with Active Directory, the same way that the AD provider does."
----- Original Message -----
From: "James Ralston" ralston@pobox.com To: "End-user discussions about the System Security Services Daemon" sssd-users@lists.fedorahosted.org Sent: Wednesday, May 6, 2015 1:28:35 PM Subject: [SSSD-users] please do not remove enumeration from AD provider
On Wed, May 6, 2015 at 4:27 AM, Jakub Hrozek jhrozek@redhat.com wrote:
You know, just this morning, I was thinking about enumeration. It doesn't work for IPA views at all for example. It doesn't work for trusted domains at all either (except for some limited support in AD trusted domains that is very untested)
I wonder if we could just remove enumeration from IPA and AD back ends in some major release.
Please don't do this.
Enumeration is a very useful feature. It allows us to do things like this:
$ getent passwd | grep -i lastname
The equivalent ldapsearch command is much more tedious:
$ ldapsearch -z 0 -E pr=2147483647/noprompt -o ldif-wrap=no -L -L -H 'ldap:///dc%3Dexample%2Cdc%3Dorg -Y GSSAPI -N -b "dc=example,dc=org" "(&(objectClass=user)(cn=*lastname*))" dn cn sAMAccountName
To be fair, it's not that hard to turn that into a bash script that your users can use instead of learning the ldap syntax. But yes, that's still a change in behavior.
More generically, enumeration is the way Unix/Linux has always worked. Even getting users to change from:
grep -i lastname /etc/passwdTo this:
getent passwd | grep -i lastname...has been a struggle.
We also have various services that (unfortuantely) pre-load the passwd and group files at startup by enumerating them with getpwent_r() and getgrent_r(), instead of using the get*nam_r() and get*id_r() functions as-needed. These services break outright if enumeration is disabled.
(Yes, these services are broken. Yes, they shouldn't do that. But our ability to fix them is extremely limited at best, because we don't control them.)
Well, as the original post to this list noted, this is already broken, with no way to fix it. When we're talking to LDAP, there's no guarantee that the server will actually let us get all of the results. Many servers are configured with a limited number of records we can retrieve (though we work around that with paging controls on servers that support them). With AD, we can only enumerate the domain the host is joined to. If your users aren't part of the same domain as the host, enumeration won't find them.
Finally, we have many systems that cannot be joined to Active Directory (for policy reasons, not technical reasons). But we want to use the same passwd/group entries on those systems as returned by sssd on hosts that are joined to Active Directory. We do this by scraping the output of "getent -s sss passwd" and "getent -s sss group" and manually merging it into the local passwd and group files (respectively) on these hosts.
Sorry to sound glib, but "fix your policy". Let's be honest, any policy that boils down to "These machines are not allowed to function with proper security controls" is one that can only end in disaster.
It's just a legacy feature, so those who need it can fall back to the LDAP provider..
But the LDAP provider doesn't support ID mapping; only the AD provider does. And ID mapping is the main reason we use sssd.
As noted in the other reply, the LDAP provider does support ID mapping. However, you'll still have to face the same problems as above with regards to paging limits and domains other than the one the host is joined to.
I'm not asking you to make enumeration the default. It shouldn't be; it should be something you only turn on if you need it, and you KNOW you need it. But if you need it, you NEED it. Please don't take it away.
If you need it, you're already in bad shape. Have you heard the adage "If someone is irreplaceable, replace them immediately"? The same is true for software. If you have a bad system in place, it's best to rip it out as fast as possible, because otherwise the problem will continue to grow, accruing technical debt you can never hope to manage.
This is one of those cases: every band-aid we apply to the enumeration support causes this to limp along on life-support and provides consumers a false sense that this is something they can rely upon. Frankly, I think it may be time to rip off the aforementioned band-aid and amputate this gangrenous limb.
On Wed, May 06, 2015 at 03:10:05PM -0400, Stephen Gallagher wrote:
----- Original Message -----
From: "James Ralston" ralston@pobox.com To: "End-user discussions about the System Security Services Daemon" sssd-users@lists.fedorahosted.org Sent: Wednesday, May 6, 2015 1:28:35 PM Subject: [SSSD-users] please do not remove enumeration from AD provider
On Wed, May 6, 2015 at 4:27 AM, Jakub Hrozek jhrozek@redhat.com wrote:
You know, just this morning, I was thinking about enumeration. It doesn't work for IPA views at all for example. It doesn't work for trusted domains at all either (except for some limited support in AD trusted domains that is very untested)
I wonder if we could just remove enumeration from IPA and AD back ends in some major release.
Please don't do this.
Enumeration is a very useful feature. It allows us to do things like this:
$ getent passwd | grep -i lastname
The equivalent ldapsearch command is much more tedious:
$ ldapsearch -z 0 -E pr=2147483647/noprompt -o ldif-wrap=no -L -L -H 'ldap:///dc%3Dexample%2Cdc%3Dorg -Y GSSAPI -N -b "dc=example,dc=org" "(&(objectClass=user)(cn=*lastname*))" dn cn sAMAccountName
To be fair, it's not that hard to turn that into a bash script that your users can use instead of learning the ldap syntax. But yes, that's still a change in behavior.
More generically, enumeration is the way Unix/Linux has always worked. Even getting users to change from:
grep -i lastname /etc/passwdTo this:
getent passwd | grep -i lastname...has been a struggle.
We also have various services that (unfortuantely) pre-load the passwd and group files at startup by enumerating them with getpwent_r() and getgrent_r(), instead of using the get*nam_r() and get*id_r() functions as-needed. These services break outright if enumeration is disabled.
(Yes, these services are broken. Yes, they shouldn't do that. But our ability to fix them is extremely limited at best, because we don't control them.)
Well, as the original post to this list noted, this is already broken, with no way to fix it. When we're talking to LDAP, there's no guarantee that the server will actually let us get all of the results. Many servers are configured with a limited number of records we can retrieve (though we work around that with paging controls on servers that support them). With AD, we can only enumerate the domain the host is joined to. If your users aren't part of the same domain as the host, enumeration won't find them.
Finally, we have many systems that cannot be joined to Active Directory (for policy reasons, not technical reasons). But we want to use the same passwd/group entries on those systems as returned by sssd on hosts that are joined to Active Directory. We do this by scraping the output of "getent -s sss passwd" and "getent -s sss group" and manually merging it into the local passwd and group files (respectively) on these hosts.
Sorry to sound glib, but "fix your policy". Let's be honest, any policy that boils down to "These machines are not allowed to function with proper security controls" is one that can only end in disaster.
It's just a legacy feature, so those who need it can fall back to the LDAP provider..
But the LDAP provider doesn't support ID mapping; only the AD provider does. And ID mapping is the main reason we use sssd.
As noted in the other reply, the LDAP provider does support ID mapping. However, you'll still have to face the same problems as above with regards to paging limits and domains other than the one the host is joined to.
I'm not asking you to make enumeration the default. It shouldn't be; it should be something you only turn on if you need it, and you KNOW you need it. But if you need it, you NEED it. Please don't take it away.
If you need it, you're already in bad shape. Have you heard the adage "If someone is irreplaceable, replace them immediately"? The same is true for software. If you have a bad system in place, it's best to rip it out as fast as possible, because otherwise the problem will continue to grow, accruing technical debt you can never hope to manage.
This is one of those cases: every band-aid we apply to the enumeration support causes this to limp along on life-support and provides consumers a false sense that this is something they can rely upon. Frankly, I think it may be time to rip off the aforementioned band-aid and amputate this gangrenous limb.
Right. And half-broken-half-working implementation creates false expectations. We claim that we "support IPA ID views", but then enabling enumeration would simply break those.
As we keep adding features to AD and IPA back ends, they are no longer just wrappers around the LDAP provider with correct defaults for the particular server implementation.
I think it would be more fair towards our users to only support what we can actually support and what can actually work.. I'm not saying enumeration support should go away in the next release, but eventually we should remove it and advise the setups that require enumeration to migrate to using the LDAP provider.
On Wed, May 6, 2015 at 3:10 PM, Stephen Gallagher sgallagh@redhat.com wrote:
To be fair, it's not that hard to turn that [ldapsearch command] into a bash script that your users can use instead of learning the ldap syntax. But yes, that's still a change in behavior.
We could, but I think just using the ldapsearch command would cause less confusion in the long run.
Well, as the original post to this list noted, this is already broken, with no way to fix it. When we're talking to LDAP, there's no guarantee that the server will actually let us get all of the results. Many servers are configured with a limited number of records we can retrieve (though we work around that with paging controls on servers that support them). With AD, we can only enumerate the domain the host is joined to. If your users aren't part of the same domain as the host, enumeration won't find them.
I get it: there are multiple reasons why enumeration might not work in a particular environment, and it might not be obvious (until you try it) whether it will work.
But it *does* work for us. And it works for others.
Does it really make sense to remove a useful feature just because there are corner cases in which it doesn't work?
(Hell, more than half of the Linux kernel code would have to be excised if that's the standard we're using.)
I mean, how much time are the sssd developers spending fielding complaints about enumeration not working correctly? If the answer is "little to none", and the limitations are clearly spelled out (which they are), what harm is there in leaving it enabled?
[W]e have many systems that cannot be joined to Active Directory (for policy reasons, not technical reasons).
Sorry to sound glib, but "fix your policy". Let's be honest, any policy that boils down to "These machines are not allowed to function with proper security controls" is one that can only end in disaster.
Actually, the policy is a component of proper security controls. (Which is why the policy won't change.)
I'm not asking you to make enumeration the default. It shouldn't be; it should be something you only turn on if you need it, and you KNOW you need it. But if you need it, you NEED it. Please don't take it away.
If you need it, you're already in bad shape. Have you heard the adage "If someone is irreplaceable, replace them immediately"? The same is true for software. If you have a bad system in place, it's best to rip it out as fast as possible, because otherwise the problem will continue to grow, accruing technical debt you can never hope to manage.
Let me rephrase: to us, enumeration is a highly desirable feature, and sssd nicely provides that feature.
Could we survive if sssd removed enumeration? Yes. I could cobble something together to perform a global LDAP search against AD, extract the usernames and groupnames from the returned objects, and then perform a bajillion getpwnam(3) and getgrnam(3) calls on the object names.
But sssd already does that for us. And it does it more reliably, with less overhead, than whatever replacement I could throw together.
This is one of those cases: every band-aid we apply to the enumeration support causes this to limp along on life-support and provides consumers a false sense that this is something they can rely upon.
I would argue that enumeration *is* reliable: if it works for your configuration, it will work each time, every time.
I think what you mean is, "enumeration will only work for certain configurations". That's a fair statement, but it's not the same thing as "unreliable".
On Thu, 2015-05-14 at 17:11 -0400, James Ralston wrote:
On Wed, May 6, 2015 at 3:10 PM, Stephen Gallagher < sgallagh@redhat.com> wrote:
To be fair, it's not that hard to turn that [ldapsearch command] into a bash script that your users can use instead of learning the ldap syntax. But yes, that's still a change in behavior.
We could, but I think just using the ldapsearch command would cause less confusion in the long run.
Well, as the original post to this list noted, this is already broken, with no way to fix it. When we're talking to LDAP, there's no guarantee that the server will actually let us get all of the results. Many servers are configured with a limited number of records we can retrieve (though we work around that with paging controls on servers that support them). With AD, we can only enumerate the domain the host is joined to. If your users aren't part of the same domain as the host, enumeration won't find them.
I get it: there are multiple reasons why enumeration might not work in a particular environment, and it might not be obvious (until you try it) whether it will work.
But it *does* work for us. And it works for others.
Does it really make sense to remove a useful feature just because there are corner cases in which it doesn't work?
(Hell, more than half of the Linux kernel code would have to be excised if that's the standard we're using.)
I mean, how much time are the sssd developers spending fielding complaints about enumeration not working correctly? If the answer is "little to none", and the limitations are clearly spelled out (which they are), what harm is there in leaving it enabled?
Well, the SSSD developers are spending a moderate amount of time dealing with bugs in it, first of all. Secondly, the limitations aren't really clearly spelled out. We probably need to expand the manpages to describe how poorly this feature works. Right now, it only describes the negative performance impact, but not the fact that it simply doesn't work in some environments.
And the harm to leaving it enabled is that failures in the enumeration code are generally *silent* and therefore hard to debug. When an enumeration only completes partially, there's no way to know. If you have a system that is basing access control on a user being in (or not in) a particular group read through enumeration, then this may result in a security issue. (Example: you explicitly disallow members of the "untrusted" group from accessing sensitive machines. However, user jappleseed should be in this group, but enumeration didn't pick him up because of a peculiarity of cross-realm interaction. Now jappleseed has access to a sensitive machine. Ouch.)
[W]e have many systems that cannot be joined to Active Directory (for policy reasons, not technical reasons).
Sorry to sound glib, but "fix your policy". Let's be honest, any policy that boils down to "These machines are not allowed to function with proper security controls" is one that can only end in disaster.
Actually, the policy is a component of proper security controls. (Which is why the policy won't change.)
If they cannot be joined to an existing Active Directory domain, why can't they be joined to an isolated AD or FreeIPA domain? That's probably irrelevant to the current conversation, though.
I'm not asking you to make enumeration the default. It shouldn't be; it should be something you only turn on if you need it, and you KNOW you need it. But if you need it, you NEED it. Please don't take it away.
If you need it, you're already in bad shape. Have you heard the adage "If someone is irreplaceable, replace them immediately"? The same is true for software. If you have a bad system in place, it's best to rip it out as fast as possible, because otherwise the problem will continue to grow, accruing technical debt you can never hope to manage.
Let me rephrase: to us, enumeration is a highly desirable feature, and sssd nicely provides that feature.
Sure, I completely agree with you that *if it worked as advertised*, it would be a really nice feature. The fact is that it doesn't. And in some situations, we're discovering that it actually *can't* (and as noted above, may not be able to warn you of this fact).
Could we survive if sssd removed enumeration? Yes. I could cobble something together to perform a global LDAP search against AD, extract the usernames and groupnames from the returned objects, and then perform a bajillion getpwnam(3) and getgrnam(3) calls on the object names.
Well, I don't know the specific use-case here, but in most cases it's more reliable to just get the list of groups a user belongs to (which we *can* do reliably) and base decisions on that. Of course, this doesn't work if your goal is a user/group browser.
But sssd already does that for us. And it does it more reliably, with less overhead, than whatever replacement I could throw together.
That's certainly fair, but again: which is worse? 1) Not having the ability to list all users 2) Being told that you can, but it might not work - without any way to notify you that it failed.
This is one of those cases: every band-aid we apply to the enumeration support causes this to limp along on life-support and provides consumers a false sense that this is something they can rely upon.
I would argue that enumeration *is* reliable: if it works for your configuration, it will work each time, every time.
As long as your configuration doesn't change and start hitting one of the cases we can't support. Example: you suddenly add your 2001st user and don't notice that the LDAP server only allows clients to retrieve 2000 records. Enumerate will continue to appear to work, but it won't be able to get all the records. (Bonus points: some LDAP servers don't have a predictable order of which records you get back, so enumerate might see a different view at different times.)
I think what you mean is, "enumeration will only work for certain configurations". That's a fair statement, but it's not the same thing as "unreliable".
It's not just certain configurations but it may be certain states within an otherwise good configuration (such as the paging limitation I noted just above).
Following up on an issue from a while ago…
On Thu, May 14, 2015 at 9:32 PM, Stephen Gallagher sgallagh@redhat.com wrote:
[T]he SSSD developers are spending a moderate amount of time dealing with bugs in it [enumeration], first of all. Secondly, the limitations aren't really clearly spelled out. We probably need to expand the manpages to describe how poorly this feature works. Right now, it only describes the negative performance impact, but not the fact that it simply doesn't work in some environments.
And the harm to leaving it enabled is that failures in the enumeration code are generally *silent* and therefore hard to debug. When an enumeration only completes partially, there's no way to know. If you have a system that is basing access control on a user being in (or not in) a particular group read through enumeration, then this may result in a security issue. (Example: you explicitly disallow members of the "untrusted" group from accessing sensitive machines. However, user jappleseed should be in this group, but enumeration didn't pick him up because of a peculiarity of cross-realm interaction. Now jappleseed has access to a sensitive machine. Ouch.)
After spending many months running with enumeration enabled, we reached the conclusion that the cost of enumeration wasn't worth it, and disabled it.
We didn't encounter any of the corner cases where enumeration doesn't work, silently fails, returns incomplete group information, et. al. But what we *did* notice is that sssd pounded the host when enumeration was enabled, even if the host was otherwise idle. (We had a non-trivial number of mostly-idle hosts where sssd had the most CPU consumption of any service running on the system.)
While it is currently somewhat of a pain to perform iterative enumeration (due to cache performance issues), for the handful of hosts where we need to be enumerate all AD users and groups, it's still a better alternative than enabling enumeration in sssd and having it pound the hosts.
In conclusion, we no longer care if the enumeration feature is removed from sssd, because we are no longer using it.
On Wed, Jan 27, 2016 at 07:48:50PM -0500, James Ralston wrote:
In conclusion, we no longer care if the enumeration feature is removed from sssd, because we are no longer using it.
Just as a status update -- while the enumeration is still around, it's a) not being tested as much as the non-enum case and b) some features like IPA overrides simply are not implemented with enumeration..
Op 28 jan. 2016, om 01:48 heeft James Ralston ralston@pobox.com het volgende geschreven:
Following up on an issue from a while ago…
On Thu, May 14, 2015 at 9:32 PM, Stephen Gallagher sgallagh@redhat.com wrote:
[T]he SSSD developers are spending a moderate amount of time dealing with bugs in it [enumeration], first of all. Secondly, the limitations aren't really clearly spelled out. We probably need to expand the manpages to describe how poorly this feature works. Right now, it only describes the negative performance impact, but not the fact that it simply doesn't work in some environments.
And the harm to leaving it enabled is that failures in the enumeration code are generally *silent* and therefore hard to debug. When an enumeration only completes partially, there's no way to know. If you have a system that is basing access control on a user being in (or not in) a particular group read through enumeration, then this may result in a security issue. (Example: you explicitly disallow members of the "untrusted" group from accessing sensitive machines. However, user jappleseed should be in this group, but enumeration didn't pick him up because of a peculiarity of cross-realm interaction. Now jappleseed has access to a sensitive machine. Ouch.)
After spending many months running with enumeration enabled, we reached the conclusion that the cost of enumeration wasn't worth it, and disabled it.
We didn't encounter any of the corner cases where enumeration doesn't work, silently fails, returns incomplete group information, et. al. But what we *did* notice is that sssd pounded the host when enumeration was enabled, even if the host was otherwise idle. (We had a non-trivial number of mostly-idle hosts where sssd had the most CPU consumption of any service running on the system.)
While it is currently somewhat of a pain to perform iterative enumeration (due to cache performance issues), for the handful of hosts where we need to be enumerate all AD users and groups, it's still a better alternative than enabling enumeration in sssd and having it pound the hosts.
I wonder how this is done. In our case we have the requirement to list the users that are enabled for a system. Currently, both “getent passwd” and “getent group” do not show these users without enumeration. I don’t mind doing an iterative approach from either side (ie. group or user), but how do I know these users/groups beforehand so I have a starting position?
I wonder how this is done. In our case we have the requirement to list the users that are enabled for a system. Currently, both “getent passwd” and “getent group” do not show these users without enumeration. I don’t mind doing an iterative approach from either side (ie. group or user), but how do I know these users/groups beforehand so I have a starting position?
In our environment we do that kind of listing in lot of scripts on server side using direct ldap searches.
like in:
USER_FULL_LIST=`ldapsearch -x -LLL -h $LDAP_SERVER -b "dc=global" "(mail=*@somedomain.net)" uid | grep "^uid: " | cut -d" " -f2- | sort`
even if we have sssd configured, we prefer that approach. and we generally don't allow such kind of searches on the client (desktop) side.
maybe if you give some more info about what you need to achieve we can give you more accurate solutions.
anyway, I don't know how hard is to mantain this code but I don't like leting enumeration go away and I think software should always give options, not remove them. is the admin's job to decide if something can harm a system or not.
abosch
Op 28 jan. 2016, om 13:38 heeft Angel Bosch abosch@ticmallorca.net het volgende geschreven:
I wonder how this is done. In our case we have the requirement to list the users that are enabled for a system. Currently, both “getent passwd” and “getent group” do not show these users without enumeration. I don’t mind doing an iterative approach from either side (ie. group or user), but how do I know these users/groups beforehand so I have a starting position?
In our environment we do that kind of listing in lot of scripts on server side using direct ldap searches.
like in:
USER_FULL_LIST=`ldapsearch -x -LLL -h $LDAP_SERVER -b "dc=global" "(mail=*@somedomain.net)" uid | grep "^uid: " | cut -d" " -f2- | sort`
even if we have sssd configured, we prefer that approach. and we generally don't allow such kind of searches on the client (desktop) side.
maybe if you give some more info about what you need to achieve we can give you more accurate solutions.
anyway, I don't know how hard is to mantain this code but I don't like leting enumeration go away and I think software should always give options, not remove them. is the admin's job to decide if something can harm a system or not.
Our environment is a Hadoop cluster with FreeIPA as an identity provider with a trust against the corporate active directory. Business roles are created and reflected in active directory and then mapped to IPA. As mentioned in another thread one of the Hadoop components (Ranger) syncs all users and groups (including GIDs) on a regular basis to provide authorization.
We like the end to end security IPA provides for us (ie. across OS and Hadoop) and we prefer to use IPA as our source of truth, instead of a mashup of several sources, making sure the loop stays closed.
I would be able to do ldap searches on AD if IPA tells me where, but it also requires additional authentication which I would prefer sssd or IPA to handle.
On Thu, Jan 28, 2016 at 8:18 AM, Bolke de Bruin bdbruin@gmail.com wrote:
As mentioned in another thread one of the Hadoop components (Ranger) syncs all users and groups (including GIDs) on a regular basis to provide authorization.
Unfortunately, that is the problem. :-(
Apache Ranger assumes that the back-end database for the passwd/group services is capable of enumeration. That is true for the "files" database, but is not guaranteed to be true for other databases.
More simply put: there is no guarantee that getpwent()/getgrent() will enumerate all users/groups (respectively) known to the passwd/group services.
At our site, we have a team that uses Hadoop, and they encountered this issue when we first deployed sssd. Their work-around was to manually create local passwd/group entries for the users/groups they wanted to be visible within Hadoop. That worked for them, because their Hadoop cluster was for only a handful of users, but that solution isn't going to work for a production Hadoop cluster of any significant size.
I asked the developers on our Hadoop team to file a bug against Apache Ranger, but I don't know if they ever did.
Op 28 jan. 2016, om 20:42 heeft James Ralston ralston@pobox.com het volgende geschreven:
On Thu, Jan 28, 2016 at 8:18 AM, Bolke de Bruin bdbruin@gmail.com wrote:
As mentioned in another thread one of the Hadoop components (Ranger) syncs all users and groups (including GIDs) on a regular basis to provide authorization.
Unfortunately, that is the problem. :-(
Apache Ranger assumes that the back-end database for the passwd/group services is capable of enumeration. That is true for the "files" database, but is not guaranteed to be true for other databases.
More simply put: there is no guarantee that getpwent()/getgrent() will enumerate all users/groups (respectively) known to the passwd/group services.
At our site, we have a team that uses Hadoop, and they encountered this issue when we first deployed sssd. Their work-around was to manually create local passwd/group entries for the users/groups they wanted to be visible within Hadoop. That worked for them, because their Hadoop cluster was for only a handful of users, but that solution isn't going to work for a production Hadoop cluster of any significant size.
I asked the developers on our Hadoop team to file a bug against Apache Ranger, but I don't know if they ever did.
Ranger is actually even worse. It currently uses /etc/passwd and /etc/group directly - so no nss. I have a patch in the works that addresses this by using getent instead. Moreover, I am adding some config parameters that allow to sync/enumerate specific groups that ranger otherwise doesn’t see. It might help your guys in the future.
Still I think Ranger is a load of crap though, enumerating all users with over 50.000 in our corp directory that is not fun. I just try to make it a little bit more manageable.
On Thu, Jan 28, 2016 at 08:51:29PM +0100, Bolke de Bruin wrote:
Op 28 jan. 2016, om 20:42 heeft James Ralston ralston@pobox.com het volgende geschreven:
On Thu, Jan 28, 2016 at 8:18 AM, Bolke de Bruin bdbruin@gmail.com wrote:
As mentioned in another thread one of the Hadoop components (Ranger) syncs all users and groups (including GIDs) on a regular basis to provide authorization.
Unfortunately, that is the problem. :-(
Apache Ranger assumes that the back-end database for the passwd/group services is capable of enumeration. That is true for the "files" database, but is not guaranteed to be true for other databases.
More simply put: there is no guarantee that getpwent()/getgrent() will enumerate all users/groups (respectively) known to the passwd/group services.
At our site, we have a team that uses Hadoop, and they encountered this issue when we first deployed sssd. Their work-around was to manually create local passwd/group entries for the users/groups they wanted to be visible within Hadoop. That worked for them, because their Hadoop cluster was for only a handful of users, but that solution isn't going to work for a production Hadoop cluster of any significant size.
I asked the developers on our Hadoop team to file a bug against Apache Ranger, but I don't know if they ever did.
Ranger is actually even worse. It currently uses /etc/passwd and /etc/group directly - so no nss. I have a patch in the works that addresses this by using getent instead. Moreover, I am adding some config parameters that allow to sync/enumerate specific groups that ranger otherwise doesn’t see. It might help your guys in the future.
Still I think Ranger is a load of crap though, enumerating all users with over 50.000 in our corp directory that is not fun. I just try to make it a little bit more manageable.
What might also be a workaround for broken software like this is to include a filter in the ldap_search_base option to only match the required users (maybe coupled with a special attribute on the server or a group you can easily match with memberof:). Then sssd would really only match the filter and enumeration could be bearable. (it still wouldn't include features like id views though).
+1 here. I can also confirm that several services (Cadence, are you listening??) depend on enumeration so we need preserve this functionality - but I agree that relying on enumeration is a bad habit which should be avoided.
Ondrej
-----Original Message----- From: sssd-users-bounces@lists.fedorahosted.org [mailto:sssd-users-bounces@lists.fedorahosted.org] On Behalf Of James Ralston Sent: Wednesday, May 06, 2015 7:29 PM To: End-user discussions about the System Security Services Daemon Subject: [SSSD-users] please do not remove enumeration from AD provider
On Wed, May 6, 2015 at 4:27 AM, Jakub Hrozek jhrozek@redhat.com wrote:
You know, just this morning, I was thinking about enumeration. It doesn't work for IPA views at all for example. It doesn't work for trusted domains at all either (except for some limited support in AD trusted domains that is very untested)
I wonder if we could just remove enumeration from IPA and AD back ends in some major release.
Please don't do this.
Enumeration is a very useful feature. It allows us to do things like this:
$ getent passwd | grep -i lastname
The equivalent ldapsearch command is much more tedious:
$ ldapsearch -z 0 -E pr=2147483647/noprompt -o ldif-wrap=no -L -L -H 'ldap:///dc%3Dexample%2Cdc%3Dorg -Y GSSAPI -N -b "dc=example,dc=org" "(&(objectClass=user)(cn=*lastname*))" dn cn sAMAccountName
More generically, enumeration is the way Unix/Linux has always worked. Even getting users to change from:
grep -i lastname /etc/passwd
To this:
getent passwd | grep -i lastname
...has been a struggle.
We also have various services that (unfortuantely) pre-load the passwd and group files at startup by enumerating them with getpwent_r() and getgrent_r(), instead of using the get*nam_r() and get*id_r() functions as-needed. These services break outright if enumeration is disabled.
(Yes, these services are broken. Yes, they shouldn't do that. But our ability to fix them is extremely limited at best, because we don't control them.)
Finally, we have many systems that cannot be joined to Active Directory (for policy reasons, not technical reasons). But we want to use the same passwd/group entries on those systems as returned by sssd on hosts that are joined to Active Directory. We do this by scraping the output of "getent -s sss passwd" and "getent -s sss group" and manually merging it into the local passwd and group files (respectively) on these hosts.
It's just a legacy feature, so those who need it can fall back to the LDAP provider..
But the LDAP provider doesn't support ID mapping; only the AD provider does. And ID mapping is the main reason we use sssd.
I'm not asking you to make enumeration the default. It shouldn't be; it should be something you only turn on if you need it, and you KNOW you need it. But if you need it, you NEED it. Please don't take it away. _______________________________________________ sssd-users mailing list sssd-users@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/sssd-users
-----
The information contained in this e-mail and in any attachments is confidential and is designated solely for the attention of the intended recipient(s). If you are not an intended recipient, you must not use, disclose, copy, distribute or retain this e-mail or any part thereof. If you have received this e-mail in error, please notify the sender by return e-mail and delete all copies of this e-mail from your computer system(s). Please direct any additional queries to: communications@s3group.com. Thank You. Silicon and Software Systems Limited (S3 Group). Registered in Ireland no. 378073. Registered Office: South County Business Park, Leopardstown, Dublin 18.
On (11/05/15 07:01), Ondrej Valousek wrote:
+1 here. I can also confirm that several services (Cadence, are you listening??) depend on enumeration so we need preserve this functionality - but I agree that relying on enumeration is a bad habit which should be avoided.
Would it be acceptable for you to use ldap provider + krb5 with AD and enabled enumeration?
LS
Not sure why would I do it. The AD provider seems to be working well right now :) O.
-----Original Message----- From: sssd-users-bounces@lists.fedorahosted.org [mailto:sssd-users-bounces@lists.fedorahosted.org] On Behalf Of Lukas Slebodnik Sent: Monday, May 11, 2015 9:40 AM To: End-user discussions about the System Security Services Daemon Subject: Re: [SSSD-users] please do not remove enumeration from AD provider
On (11/05/15 07:01), Ondrej Valousek wrote:
+1 here. I can also confirm that several services (Cadence, are you listening??) depend on enumeration so we need preserve this functionality - but I agree that relying on enumeration is a bad habit which should be avoided.
Would it be acceptable for you to use ldap provider + krb5 with AD and enabled enumeration?
LS _______________________________________________ sssd-users mailing list sssd-users@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/sssd-users
-----
The information contained in this e-mail and in any attachments is confidential and is designated solely for the attention of the intended recipient(s). If you are not an intended recipient, you must not use, disclose, copy, distribute or retain this e-mail or any part thereof. If you have received this e-mail in error, please notify the sender by return e-mail and delete all copies of this e-mail from your computer system(s). Please direct any additional queries to: communications@s3group.com. Thank You. Silicon and Software Systems Limited (S3 Group). Registered in Ireland no. 378073. Registered Office: South County Business Park, Leopardstown, Dublin 18.
sssd-users@lists.fedorahosted.org
-
Angel Bosch -
Bolke de Bruin -
Jakub Hrozek -
James Ralston -
Lukas Slebodnik -
Ondrej Valousek -
Stephen Gallagher