Does anybody have a pointer to any performance comparisons between Sun DS and 389?
I was extremely happy with the performance boost using 389 on a Linux VM which is 5-8 times faster for ldapsearch operations than the older Sun machines with Sun DS 6.3.
In testing one of our most important use cases just now, I find that the ldapmodify speed is many many times slower. This doesn't make much sense, so I think I'm doing something wrong or having something misconfigured.
Earlier I improved the write performance by using large db cache sizes and moving the nsslapd-db-home-directory to tmpfs. Now most modify operations have very little I/O wait except when occasionally flushing the index files and such, and yet, there is a CPU pegged for very long periods of time, orders of magnitude higher than on Sun DS.
Is there any documentation on ldapmodify performance that I could review? Google searching seems eerily silent on the issue… (which also leads me to believe I have something misconfigured if nobody has been asking about the issue…)
The particular use case I am working with involves replacing large quantities of uniqueMember values on entries in ou=groups.
Thanks, Russ.
============================== Russell Beall Programmer Analyst IV Enterprise Identity Management University of Southern California beall@usc.edu ==============================
On 04/18/2012 12:10 PM, Russell Beall wrote:
Does anybody have a pointer to any performance comparisons between Sun DS and 389?
I was extremely happy with the performance boost using 389 on a Linux VM which is 5-8 times faster for ldapsearch operations than the older Sun machines with Sun DS 6.3.
Please note that earlier versions of the SunDS EULA forbade the publishing of competitive benchmark figures.
In testing one of our most important use cases just now, I find that the ldapmodify speed is many many times slower. This doesn't make much sense, so I think I'm doing something wrong or having something misconfigured.
Earlier
You mean, with SunDS? On Linux or Solaris?
I improved the write performance by using large db cache sizes and moving the nsslapd-db-home-directory to tmpfs. Now most modify operations have very little I/O wait except when occasionally flushing the index files and such, and yet, there is a CPU pegged for very long periods of time, orders of magnitude higher than on Sun DS.
Is there any documentation on ldapmodify performance that I could review? Google searching seems eerily silent on the issue… (which also leads me to believe I have something misconfigured if nobody has been asking about the issue…)
The particular use case I am working with involves replacing large quantities of uniqueMember values on entries in ou=groups.
Yeah, this particular operation has not been optimized. I believe SunDS added explicit optimizations for this particular case.
Thanks, Russ.
============================== Russell Beall Programmer Analyst IV Enterprise Identity Management University of Southern California beall@usc.edu mailto:beall@usc.edu ==============================
-- 389 users mailing list 389-users@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/389-users
On Apr 18, 2012, at 11:15 AM, Rich Megginson wrote:
You mean, with SunDS? On Linux or Solaris?
The Sun DS instance I'm talking about is on Sun v440 machines.
The 389 instance is on relatively new servers with RedHat Linux 6.
Yeah, this particular operation has not been optimized. I believe SunDS added explicit optimizations for this particular case.
It is becoming painfully apparent as I write more detailed tests. 389 takes time to add or delete uniquemember values proportionate to the number of values being operated on and is using about twice as much time to delete as it does to add. Sun DS appears to have perhaps an almost O(1) algorithm in play on both adding and deleting values.
Is there perhaps some kind of referential integrity setting that is being used and forcing some kind of lookup of each value, one that we could perhaps turn off? We wouldn't need such a check because our metadirectory process handles the integrity/consistency checking already.
This behavior is especially surprising given that I can run ldapdelete on the entry followed by ldapadd and the changes are almost instantaneous. I can delete and then re-add all 852 groups, some with 80K+ unique member attributes, in about ten minutes (using 389).
Why would membership changes on the group take forever to process if the entire group can be reconstructed in an instant?
Could this operation be optimized internally by turning the ldapmodify into a two-step process of ldapdelete followed by ldapadd (locking the entry during the processing and not actually replicating it that way so the group never appears to be missing)?
Thanks, Russ.
Hi Russel,
Le 18 avril 2012 23:06, Russell Beall beall@usc.edu a écrit :
On Apr 18, 2012, at 11:15 AM, Rich Megginson wrote:
Yeah, this particular operation has not been optimized. I believe SunDS added explicit optimizations for this particular case.
It is becoming painfully apparent as I write more detailed tests. 389 takes time to add or delete uniquemember values proportionate to the number of values being operated on and is using about twice as much time to delete as it does to add. Sun DS appears to have perhaps an almost O(1) algorithm in play on both adding and deleting values.
Is there perhaps some kind of referential integrity setting that is being used and forcing some kind of lookup of each value, one that we could perhaps turn off? We wouldn't need such a check because our metadirectory process handles the integrity/consistency checking already.
There is memberOf plugin that maintains the memberOf attribute for groups. I don't know whether it is activated by default or not. You could try to disable it. There is also referential integrity plugin, attribute uniqueness plugin, maybe USN plugin or custom indexes that could consume a lot of CPU. Make sure you've disabled them if you don't need them.
@+
Thanks for the tips. I scanned the dse.ldif for those plugins and I found definitions for them all, but they all have nsslapd-pluginEnabled: off.
There is something special about the uniquemember attribute that requires additional processing different from other attributes... Ldapmodify of other attributes runs pretty quick.
Regards, Russ.
On Apr 19, 2012, at 2:20 AM, Andrey Ivanov wrote:
Hi Russel,
Le 18 avril 2012 23:06, Russell Beall beall@usc.edu a écrit : On Apr 18, 2012, at 11:15 AM, Rich Megginson wrote:
Yeah, this particular operation has not been optimized. I believe SunDS added explicit optimizations for this particular case.
It is becoming painfully apparent as I write more detailed tests. 389 takes time to add or delete uniquemember values proportionate to the number of values being operated on and is using about twice as much time to delete as it does to add. Sun DS appears to have perhaps an almost O(1) algorithm in play on both adding and deleting values.
Is there perhaps some kind of referential integrity setting that is being used and forcing some kind of lookup of each value, one that we could perhaps turn off? We wouldn't need such a check because our metadirectory process handles the integrity/consistency checking already.
There is memberOf plugin that maintains the memberOf attribute for groups. I don't know whether it is activated by default or not. You could try to disable it. There is also referential integrity plugin, attribute uniqueness plugin, maybe USN plugin or custom indexes that could consume a lot of CPU. Make sure you've disabled them if you don't need them.
@+
389 users mailing list 389-users@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/389-users
On 04/19/2012 10:50 AM, Russell Beall wrote:
Thanks for the tips. I scanned the dse.ldif for those plugins and I found definitions for them all, but they all have nsslapd-pluginEnabled: off.
There is something special about the uniquemember attribute that requires additional processing different from other attributes... Ldapmodify of other attributes runs pretty quick.
Is uniquemember the only attribute using large numbers of multiple values in ldapmodify operations?
Regards, Russ.
On Apr 19, 2012, at 2:20 AM, Andrey Ivanov wrote:
Hi Russel,
Le 18 avril 2012 23:06, Russell Beall <beall@usc.edu mailto:beall@usc.edu> a écrit :
On Apr 18, 2012, at 11:15 AM, Rich Megginson wrote:Yeah, this particular operation has not been optimized. I believe SunDS added explicit optimizations for this particular case.It is becoming painfully apparent as I write more detailed tests. 389 takes time to add or delete uniquemember values proportionate to the number of values being operated on and is using about twice as much time to delete as it does to add. Sun DS appears to have perhaps an almost O(1) algorithm in play on both adding and deleting values. Is there perhaps some kind of referential integrity setting that is being used and forcing some kind of lookup of each value, one that we could perhaps turn off? We wouldn't need such a check because our metadirectory process handles the integrity/consistency checking already.There is memberOf plugin that maintains the memberOf attribute for groups. I don't know whether it is activated by default or not. You could try to disable it. There is also referential integrity plugin, attribute uniqueness plugin, maybe USN plugin or custom indexes that could consume a lot of CPU. Make sure you've disabled them if you don't need them.
@+
389 users mailing list 389-users@lists.fedoraproject.org mailto:389-users@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/389-users
-- 389 users mailing list 389-users@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/389-users
That is by far the largest example we have. We use groups with the uniquemember attribute linking to account entries and more than a few of the groups have tens of thousands of values for uniquemember. We create more of these groups regularly and it will be a problem for it to take many hours to construct such a group versus seconds/minutes with Sun DS. Our metadirectory process does not use ldapadd to create the group pre-populated, the group is created and then ldapmodify is run to add members. Three times a year at the change of semester, many thousands of group membership changes are processed and we already have a problem with it taking multiple days to process the entire set...
We also have large quantities of eduPersonEntitlement on account records, but those sets are not nearly as numerously populated. I can delete and re-add the eduPersonEntitlement attribute across 110,000 records in about 40 minutes (20 minutes each way -- with 389).
Russ.
On Apr 19, 2012, at 10:18 AM, Rich Megginson wrote:
On 04/19/2012 10:50 AM, Russell Beall wrote:
Thanks for the tips. I scanned the dse.ldif for those plugins and I found definitions for them all, but they all have nsslapd-pluginEnabled: off.
There is something special about the uniquemember attribute that requires additional processing different from other attributes... Ldapmodify of other attributes runs pretty quick.
Is uniquemember the only attribute using large numbers of multiple values in ldapmodify operations?
Regards, Russ.
On Apr 19, 2012, at 2:20 AM, Andrey Ivanov wrote:
Hi Russel,
Le 18 avril 2012 23:06, Russell Beall beall@usc.edu a écrit : On Apr 18, 2012, at 11:15 AM, Rich Megginson wrote:
Yeah, this particular operation has not been optimized. I believe SunDS added explicit optimizations for this particular case.
It is becoming painfully apparent as I write more detailed tests. 389 takes time to add or delete uniquemember values proportionate to the number of values being operated on and is using about twice as much time to delete as it does to add. Sun DS appears to have perhaps an almost O(1) algorithm in play on both adding and deleting values.
Is there perhaps some kind of referential integrity setting that is being used and forcing some kind of lookup of each value, one that we could perhaps turn off? We wouldn't need such a check because our metadirectory process handles the integrity/consistency checking already.
There is memberOf plugin that maintains the memberOf attribute for groups. I don't know whether it is activated by default or not. You could try to disable it. There is also referential integrity plugin, attribute uniqueness plugin, maybe USN plugin or custom indexes that could consume a lot of CPU. Make sure you've disabled them if you don't need them.
@+
389 users mailing list 389-users@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/389-users
-- 389 users mailing list 389-users@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/389-users
On 04/19/2012 11:38 AM, Russell Beall wrote:
That is by far the largest example we have. We use groups with the uniquemember attribute linking to account entries and more than a few of the groups have tens of thousands of values for uniquemember. We create more of these groups regularly and it will be a problem for it to take many hours to construct such a group versus seconds/minutes with Sun DS. Our metadirectory process does not use ldapadd to create the group pre-populated, the group is created and then ldapmodify is run to add members. Three times a year at the change of semester, many thousands of group membership changes are processed and we already have a problem with it taking multiple days to process the entire set...
OK. If you've ruled out the possibility that some plugin is interfering with the processing, then it must be something we will have to fix in the core server. Please file a ticket at https://fedorahosted.org/389
We also have large quantities of eduPersonEntitlement on account records, but those sets are not nearly as numerously populated. I can delete and re-add the eduPersonEntitlement attribute across 110,000 records in about 40 minutes (20 minutes each way -- with 389).
Russ.
On Apr 19, 2012, at 10:18 AM, Rich Megginson wrote:
On 04/19/2012 10:50 AM, Russell Beall wrote:
Thanks for the tips. I scanned the dse.ldif for those plugins and I found definitions for them all, but they all have nsslapd-pluginEnabled: off.
There is something special about the uniquemember attribute that requires additional processing different from other attributes... Ldapmodify of other attributes runs pretty quick.
Is uniquemember the only attribute using large numbers of multiple values in ldapmodify operations?
Regards, Russ.
On Apr 19, 2012, at 2:20 AM, Andrey Ivanov wrote:
Hi Russel,
Le 18 avril 2012 23:06, Russell Beall <beall@usc.edu mailto:beall@usc.edu> a écrit :
On Apr 18, 2012, at 11:15 AM, Rich Megginson wrote:Yeah, this particular operation has not been optimized. I believe SunDS added explicit optimizations for this particular case.It is becoming painfully apparent as I write more detailed tests. 389 takes time to add or delete uniquemember values proportionate to the number of values being operated on and is using about twice as much time to delete as it does to add. Sun DS appears to have perhaps an almost O(1) algorithm in play on both adding and deleting values. Is there perhaps some kind of referential integrity setting that is being used and forcing some kind of lookup of each value, one that we could perhaps turn off? We wouldn't need such a check because our metadirectory process handles the integrity/consistency checking already.There is memberOf plugin that maintains the memberOf attribute for groups. I don't know whether it is activated by default or not. You could try to disable it. There is also referential integrity plugin, attribute uniqueness plugin, maybe USN plugin or custom indexes that could consume a lot of CPU. Make sure you've disabled them if you don't need them.
@+
389 users mailing list 389-users@lists.fedoraproject.org mailto:389-users@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/389-users
-- 389 users mailing list 389-users@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/389-users
-- 389 users mailing list 389-users@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/389-users
I've been running some more tests before setting up the ticket, but I think I have enough information now. The uniqueMember attribute has extra processing overhead, but the necessary optimization might apply across the board for all attributes. I found also that adding large sets of values for other attributes also increases modification times heavily, though not quite as much as uniqueMember. Luckily, the modification delay is based on the size of the modification rather than the size of the entry, so even if the modification is done to a 100K-value attribute, if the modification is only to remove a few members and add a few others, then the change is still relatively quick. The delay is noticed most when first setting up a group, for instance, adding 100K members to an empty group takes 2.5 hours on 389 as opposed to 1 minute on Sun DS.
Also during this testing I have noticed a memory leak when running large quantities of ldapmodify operations. When I set up a loop to delete and then re-add the eduPersonEntitlement attribute across 100K entries, I found that memory consumption continuously increased and the server crashed after the fifth iteration of this loop. (And this one really is with ldapmodify and is not related to my earlier issues with excessively creating tombstones by deleting and adding entire entries). Before digging into this too deeply and making another ticket, I wanted to ask if this had been noticed and fixed in the 1.2.10 release? I am using the default 1.2.9.16 release. I'm guessing it hasn't since I didn't see it in the release notes.
I am starting up the server with the valgrind command you recommended a few messages back to see if I can spot the leak, though of course with valgrind in the mix, the overhead and runtimes are, as might be expected, much increased.
Regards, Russ.
On Apr 19, 2012, at 1:42 PM, Rich Megginson wrote:
OK. If you've ruled out the possibility that some plugin is interfering with the processing, then it must be something we will have to fix in the core server. Please file a ticket at https://fedorahosted.org/389
On 04/23/2012 08:01 AM, Russell Beall wrote:
I've been running some more tests before setting up the ticket, but I think I have enough information now. The uniqueMember attribute has extra processing overhead, but the necessary optimization might apply across the board for all attributes. I found also that adding large sets of values for other attributes also increases modification times heavily, though not quite as much as uniqueMember.
uniqueMember is a DN syntax attribute. DN syntax values are "expensive" to handle due to normalization overhead.
Luckily, the modification delay is based on the size of the modification rather than the size of the entry, so even if the modification is done to a 100K-value attribute, if the modification is only to remove a few members and add a few others, then the change is still relatively quick. The delay is noticed most when first setting up a group, for instance, adding 100K members to an empty group takes 2.5 hours on 389 as opposed to 1 minute on Sun DS.
That's very interesting. Does Sun DS have some sort of tuning parameter for number of values? That is, they may have some threshold for number of values in an attribute - once the number hits that threshold, they may switch to using some sort of ADT to store the values, like a AVL tree or hash table, rather than the simple linked list used by default.
Also during this testing I have noticed a memory leak when running large quantities of ldapmodify operations. When I set up a loop to delete and then re-add the eduPersonEntitlement attribute across 100K entries, I found that memory consumption continuously increased and the server crashed after the fifth iteration of this loop. (And this one really is with ldapmodify and is not related to my earlier issues with excessively creating tombstones by deleting and adding entire entries). Before digging into this too deeply and making another ticket, I wanted to ask if this had been noticed and fixed in the 1.2.10 release? I am using the default 1.2.9.16 release. I'm guessing it hasn't since I didn't see it in the release notes.
Try increasing your nsslapd-cachememsize and monitoring it closely. Using the size of id2entry.db4 is a good place to start, but that will not be enough.
http://docs.redhat.com/docs/en-US/Red_Hat_Directory_Server/9.0/html/Administ...
See also https://fedorahosted.org/389/ticket/51 and https://bugzilla.redhat.com/show_bug.cgi?id=697701
I am starting up the server with the valgrind command you recommended a few messages back to see if I can spot the leak, though of course with valgrind in the mix, the overhead and runtimes are, as might be expected, much increased.
Yes, and valgrind will report many false positives that are hard to weed through.
The issue you are seeing may not be a memory leak per se - see the ticket/bug above.
Regards, Russ.
On Apr 19, 2012, at 1:42 PM, Rich Megginson wrote:
OK. If you've ruled out the possibility that some plugin is interfering with the processing, then it must be something we will have to fix in the core server. Please file a ticket athttps://fedorahosted.org/389
-- 389 users mailing list 389-users@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/389-users
On Apr 23, 2012, at 10:28 AM, Rich Megginson wrote:
That's very interesting. Does Sun DS have some sort of tuning parameter for number of values? That is, they may have some threshold for number of values in an attribute - once the number hits that threshold, they may switch to using some sort of ADT to store the values, like a AVL tree or hash table, rather than the simple linked list used by default.
I've compared the dse.ldif of both servers looking specifically for attributes that I should transfer from our production environment to 389. The configurations for major components are virtually identical and I have seen no attribute that relates to the number of values in a multi-valued attribute. I expect that the optimization is a behind-the-scenes code improvement.
Also during this testing I have noticed a memory leak when running large quantities of ldapmodify operations. When I set up a loop to delete and then re-add the eduPersonEntitlement attribute across 100K entries, I found that memory consumption continuously increased and the server crashed after the fifth iteration of this loop. (And this one really is with ldapmodify and is not related to my earlier issues with excessively creating tombstones by deleting and adding entire entries). Before digging into this too deeply and making another ticket, I wanted to ask if this had been noticed and fixed in the 1.2.10 release? I am using the default 1.2.9.16 release. I'm guessing it hasn't since I didn't see it in the release notes.
Try increasing your nsslapd-cachememsize and monitoring it closely. Using the size of id2entry.db4 is a good place to start, but that will not be enough.
Early on in the process of setting up 389 I optimized the cachememsize. I configured a 12G cache, and the cache usage after loading all 600K entries is just under 10G. While the ldapmodify operations are in progress, I am pretty sure I did not have an increase in the cacheentryusage monitor attribute under cn=config, but I'd have to re-check to be sure.
Unfortunately, with valgrind attached, the server uses much extra memory on startup and does not complete the startup operation before running out of memory on my 32GB machine. I have had to reduce the cachememsize so that it will start. It's been starting up for two hours and finally stopped allocating more memory at 24G (with only a 3G cachememsize configured). I'll probably have to delete out a large quantity of entries to run the test within the bounds of the cachememsize.
http://docs.redhat.com/docs/en-US/Red_Hat_Directory_Server/9.0/html/Administ...
See also https://fedorahosted.org/389/ticket/51 and https://bugzilla.redhat.com/show_bug.cgi?id=697701
This bug appears very different from what I am looking at. The ldapmodify I run makes a single connection and transmits a large file of operations to perform value deletions on 100K entries, followed by a new connection to transmit value additions to 100K entries contained within a single large file, and then loop around to do the same thing again. This emulates the behavior of our directory synchronization script which calculates large quantities of necessary modifications and then submits them all in an ldif file.
I am starting up the server with the valgrind command you recommended a few messages back to see if I can spot the leak, though of course with valgrind in the mix, the overhead and runtimes are, as might be expected, much increased.
Yes, and valgrind will report many false positives that are hard to weed through.
The issue you are seeing may not be a memory leak per se - see the ticket/bug above.
Ok. I'll see if there is anything I can pull from the rough.
Regards, Russ.
On 04/23/2012 12:20 PM, Russell Beall wrote:
On Apr 23, 2012, at 10:28 AM, Rich Megginson wrote:
That's very interesting. Does Sun DS have some sort of tuning parameter for number of values? That is, they may have some threshold for number of values in an attribute - once the number hits that threshold, they may switch to using some sort of ADT to store the values, like a AVL tree or hash table, rather than the simple linked list used by default.
I've compared the dse.ldif of both servers looking specifically for attributes that I should transfer from our production environment to 389. The configurations for major components are virtually identical and I have seen no attribute that relates to the number of values in a multi-valued attribute. I expect that the optimization is a behind-the-scenes code improvement.
Also during this testing I have noticed a memory leak when running large quantities of ldapmodify operations. When I set up a loop to delete and then re-add the eduPersonEntitlement attribute across 100K entries, I found that memory consumption continuously increased and the server crashed after the fifth iteration of this loop. (And this one really is with ldapmodify and is not related to my earlier issues with excessively creating tombstones by deleting and adding entire entries). Before digging into this too deeply and making another ticket, I wanted to ask if this had been noticed and fixed in the 1.2.10 release? I am using the default 1.2.9.16 release. I'm guessing it hasn't since I didn't see it in the release notes.
Try increasing your nsslapd-cachememsize and monitoring it closely. Using the size of id2entry.db4 is a good place to start, but that will not be enough.
Early on in the process of setting up 389 I optimized the cachememsize. I configured a 12G cache, and the cache usage after loading all 600K entries is just under 10G. While the ldapmodify operations are in progress, I am pretty sure I did not have an increase in the cacheentryusage monitor attribute under cn=config, but I'd have to re-check to be sure.
You will see an increase due to replication metadata and possibly other factors.
Unfortunately, with valgrind attached, the server uses much extra memory on startup and does not complete the startup operation before running out of memory on my 32GB machine. I have had to reduce the cachememsize so that it will start. It's been starting up for two hours and finally stopped allocating more memory at 24G (with only a 3G cachememsize configured). I'll probably have to delete out a large quantity of entries to run the test within the bounds of the cachememsize.
Ok - so valgrind is probably not an option.
http://docs.redhat.com/docs/en-US/Red_Hat_Directory_Server/9.0/html/Administ...
See also https://fedorahosted.org/389/ticket/51 and https://bugzilla.redhat.com/show_bug.cgi?id=697701
This bug appears very different from what I am looking at. The ldapmodify I run makes a single connection and transmits a large file of operations to perform value deletions on 100K entries, followed by a new connection to transmit value additions to 100K entries contained within a single large file, and then loop around to do the same thing again. This emulates the behavior of our directory synchronization script which calculates large quantities of necessary modifications and then submits them all in an ldif file.
The thing in common is this - when the cache usage hits the cache max size, you see unbounded memory growth.
I am starting up the server with the valgrind command you recommended a few messages back to see if I can spot the leak, though of course with valgrind in the mix, the overhead and runtimes are, as might be expected, much increased.
Yes, and valgrind will report many false positives that are hard to weed through.
The issue you are seeing may not be a memory leak per se - see the ticket/bug above.
Ok. I'll see if there is anything I can pull from the rough.
Regards, Russ.
-- 389 users mailing list 389-users@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/389-users
Hi,
I've been doing a lot more testing to try to flesh out the issue here. I upgraded to the latest stable version from the rmeggins repo, but found the same memory growth behavior in that instance.
I have a few more details which clarify much better what I'm experiencing.
Unbounded memory growth for an endless chain of ldapmodify operations is seen in both cases where the cache size limit is reached as well as when the maximum cache size is well above the total data size of all entries and all entries are loaded.
On the contrary, when I reduce the cachememsize to nothing, (which then is reset for me to the minimum value of 512000), I see no memory growth at all, and the only memory consumed is just slightly larger than the DB cache size set.
I found that I can use some cache and still get a stable configuration by setting a cache size of only 3GB, and then the memory usage reaches a plateau of 24G (including a DB cache size that I don't remember).
A similar ratio is seen when setting a cachememsize of 1GB. The server starts out grabbing 4GB (including the 2GB of DB cache I set), and then grows to 9GB and then goes up and down between 8 and 9 GB while processing.
It seems that the server believes it can have an in-memory workspace equivalent to (6 * cachememsize), and this behavior seems directly linked to the cache management code.
I need to be able to set my server to use cachememsize=12GB or more, but I can't have the server believing it then has a right to 72GB of working memory. With 12GB set, the server quickly eats up the 32GB RAM, and goes well into the 16GB of swap before finally crashing.
Is this something I should just go ahead and file as a bug?
Thanks, Russ.
============================== Russell Beall Programmer Analyst IV Enterprise Identity Management University of Southern California beall@usc.edu ==============================
On Apr 24, 2012, at 6:53 AM, Rich Megginson wrote:
The thing in common is this - when the cache usage hits the cache max size, you see unbounded memory growth.
On 05/23/2012 10:27 AM, Russell Beall wrote:
Hi,
I've been doing a lot more testing to try to flesh out the issue here. I upgraded to the latest stable version from the rmeggins repo, but found the same memory growth behavior in that instance.
I have a few more details which clarify much better what I'm experiencing.
Unbounded memory growth for an endless chain of ldapmodify operations is seen in both cases where the cache size limit is reached as well as when the maximum cache size is well above the total data size of all entries and all entries are loaded.
But based on what you say later in the post, it's not unbounded, it's just not bounded by what you set as the cache size?
On the contrary, when I reduce the cachememsize to nothing, (which then is reset for me to the minimum value of 512000), I see no memory growth at all, and the only memory consumed is just slightly larger than the DB cache size set.
I found that I can use some cache and still get a stable configuration by setting a cache size of only 3GB, and then the memory usage reaches a plateau of 24G (including a DB cache size that I don't remember).
A similar ratio is seen when setting a cachememsize of 1GB. The server starts out grabbing 4GB (including the 2GB of DB cache I set), and then grows to 9GB and then goes up and down between 8 and 9 GB while processing.
It seems that the server believes it can have an in-memory workspace equivalent to (6 * cachememsize), and this behavior seems directly linked to the cache management code.
I need to be able to set my server to use cachememsize=12GB or more, but I can't have the server believing it then has a right to 72GB of working memory. With 12GB set, the server quickly eats up the 32GB RAM, and goes well into the 16GB of swap before finally crashing.
Is this something I should just go ahead and file as a bug?
Yes, please.
Thanks, Russ.
============================== Russell Beall Programmer Analyst IV Enterprise Identity Management University of Southern California beall@usc.edu mailto:beall@usc.edu ==============================
On Apr 24, 2012, at 6:53 AM, Rich Megginson wrote:
The thing in common is this - when the cache usage hits the cache max size, you see unbounded memory growth.
-- 389 users mailing list 389-users@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/389-users
On May 23, 2012, at 9:36 AM, Rich Megginson wrote:
But based on what you say later in the post, it's not unbounded, it's just not bounded by what you set as the cache size?
Yes. I guess unbounded was the wrong word now that the ratio of the boundary seems to be shown. The boundary of the memory growth seems to be (6 * cachememsize).
It doesn't grow unless it gets hit with ldapmodify operations. It seems to correctly use the cachememsize and the dbcachesize when reading in all entries and doesn't grow if just hit with ldapsearch.
Yes, please.
Ok.
Russ.
Thanks, Russ.
============================== Russell Beall Programmer Analyst IV Enterprise Identity Management University of Southern California beall@usc.edu ==============================
On Apr 24, 2012, at 6:53 AM, Rich Megginson wrote:
The thing in common is this - when the cache usage hits the cache max size, you see unbounded memory growth.
-- 389 users mailing list 389-users@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/389-users
On 05/23/2012 01:19 PM, Russell Beall wrote:
On May 23, 2012, at 9:36 AM, Rich Megginson wrote:
But based on what you say later in the post, it's not unbounded, it's just not bounded by what you set as the cache size?
Yes. I guess unbounded was the wrong word now that the ratio of the boundary seems to be shown. The boundary of the memory growth seems to be (6 * cachememsize).
It doesn't grow unless it gets hit with ldapmodify operations.
Have you tried modrdn? delete? I was just wondering if the problem is specific to ldapmodify.
It seems to correctly use the cachememsize and the dbcachesize when reading in all entries and doesn't grow if just hit with ldapsearch.
Ok, good to know.
Yes, please.
Ok.
Russ.
Thanks, Russ.
============================== Russell Beall Programmer Analyst IV Enterprise Identity Management University of Southern California beall@usc.edu mailto:beall@usc.edu ==============================
On Apr 24, 2012, at 6:53 AM, Rich Megginson wrote:
The thing in common is this - when the cache usage hits the cache max size, you see unbounded memory growth.
-- 389 users mailing list 389-users@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/389-users
-- 389 users mailing list 389-users@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/389-users
I have not tried modrdn.
Very early in my testing I thought I was seeing unbounded growth by performing endless deletes (and re-adds). That, I found out through your much-appreciated responses to this list, was causing an explosion in tombstone entries and thus the server was exploding in actual data requirements.
We don't actually place much importance on that use case because we hardly ever delete an entry, so the housekeeping process for tombstones is more than ample for that cleanup.
The ldapmodify use case is critical to us however, because we perform large quantities of ldapmodify operations every day.
This memory growth situation I have described here applies specifically and only to endless ldapmodify operations.
Regards, Russ.
On May 23, 2012, at 12:34 PM, Rich Megginson wrote:
Have you tried modrdn? delete? I was just wondering if the problem is specific to ldapmodify.
On 05/23/2012 02:11 PM, Russell Beall wrote:
I have not tried modrdn.
Very early in my testing I thought I was seeing unbounded growth by performing endless deletes (and re-adds). That, I found out through your much-appreciated responses to this list, was causing an explosion in tombstone entries and thus the server was exploding in actual data requirements.
We don't actually place much importance on that use case because we hardly ever delete an entry, so the housekeeping process for tombstones is more than ample for that cleanup.
The ldapmodify use case is critical to us however, because we perform large quantities of ldapmodify operations every day.
This memory growth situation I have described here applies specifically and only to endless ldapmodify operations.
ok - please file a ticket at https://fedorahosted.org/389
Regards, Russ.
On May 23, 2012, at 12:34 PM, Rich Megginson wrote:
Have you tried modrdn? delete? I was just wondering if the problem is specific to ldapmodify.
-- 389 users mailing list 389-users@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/389-users
Ditto!
/mrg
On Apr 19, 2012, at 13:38, Russell Beall wrote:
That is by far the largest example we have. We use groups with the uniquemember attribute linking to account entries and more than a few of the groups have tens of thousands of values for uniquemember. We create more of these groups regularly and it will be a problem for it to take many hours to construct such a group versus seconds/minutes with Sun DS. Our metadirectory process does not use ldapadd to create the group pre-populated, the group is created and then ldapmodify is run to add members. Three times a year at the change of semester, many thousands of group membership changes are processed and we already have a problem with it taking multiple days to process the entire set...
Hello:
I am struggling to set up 389DS on Centos 6.2. I can hardly find any information on how to do it, as everything is written for Centos 5. Due to the changes introduced in Centos 6 and the lack of information, I am finding it a nightmare, every step I make I run into new issues. Any help?
I had very few OS-related issues setting up on CentOS 6.2. I set a node up in this OS alongside a node in RedHat 6.2 and the settings for the directoryserver are identical. I was pleased at the very large quantity of documentation at the redhat site which describes every aspect of the product in very complete detail.
The documentation at RedHat never seemed very OS-specific to me, and the only differences I generally saw were where the documentation had differing instructions based on the older "fedora" version of the directory server.
What system differences did you encounter?
Is your issue related to being on a 32-bit system instead of a 64-bit system?
Regards, Russ.
On Apr 24, 2012, at 8:26 AM, Alberto Suárez wrote:
Hello:
I am struggling to set up 389DS on Centos 6.2. I can hardly find any information on how to do it, as everything is written for Centos 5. Due to the changes introduced in Centos 6 and the lack of information, I am finding it a nightmare, every step I make I run into new issues. Any help?
389 users mailing list 389-users@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/389-users
Thank you for your prompt reply. I am running Centos 6.2 64 bits. To be honest, I am a bit lost with so many things to take into account...
Right now I am a bit stuck at configuring TLS. I executed the script "setupssl2.sh" kindly provided by Rich Megginson and now i can no longer start the administration server. When I execute "service server-admin I get the following error message:
httpd.worker: Syntax error on line 735 of /etc/dirsrv/admin-serv/httpd.conf: Could not open configuartion file /etc/dirsrv/admin-serv/nss.conf: Permission denied
I have played with that file's permissions, even setting them as 777, but nothing changes. It is currently owned by user "fedora-ds" group "fedora-ds" and permissions are set to 400.
Thank you again.
Regards, Alberto.
Russell Beall wrote:
I had very few OS-related issues setting up on CentOS 6.2. I set a node up in this OS alongside a node in RedHat 6.2 and the settings for the directoryserver are identical. I was pleased at the very large quantity of documentation at the redhat site which describes every aspect of the product in very complete detail.
The documentation at RedHat never seemed very OS-specific to me, and the only differences I generally saw were where the documentation had differing instructions based on the older "fedora" version of the directory server.
What system differences did you encounter?
Is your issue related to being on a 32-bit system instead of a 64-bit system?
Regards, Russ.
On Apr 24, 2012, at 8:26 AM, Alberto Suárez wrote:
Hello:
I am struggling to set up 389DS on Centos 6.2. I can hardly find any information on how to do it, as everything is written for Centos 5. Due to the changes introduced in Centos 6 and the lack of information, I am finding it a nightmare, every step I make I run into new issues. Any help?
389 users mailing list 389-users@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/389-users
-- 389 users mailing list 389-users@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/389-users
Sure. Unfortunately you've gone beyond my limited experience with issues with the admin-server. I haven't configured SSL for the admin server, though I did get SSL working for the directory server.
I'll have to leave the rest of this to someone more experienced.
However, I have debugged issues like this, and one simple thing to make sure of is that the whole directory tree is accessible to the user that runs the process. Is the process owner of the admin server fedora-ds and does that user have read access to the whole tree? If you become that user can you see the files?
Regards, Russ.
On Apr 24, 2012, at 10:15 AM, Alberto Suárez wrote:
Thank you for your prompt reply. I am running Centos 6.2 64 bits. To be honest, I am a bit lost with so many things to take into account...
Right now I am a bit stuck at configuring TLS. I executed the script "setupssl2.sh" kindly provided by Rich Megginson and now i can no longer start the administration server. When I execute "service server-admin I get the following error message:
httpd.worker: Syntax error on line 735 of /etc/dirsrv/admin-serv/httpd.conf: Could not open configuartion file /etc/dirsrv/admin-serv/nss.conf: Permission denied
I have played with that file's permissions, even setting them as 777, but nothing changes. It is currently owned by user "fedora-ds" group "fedora-ds" and permissions are set to 400.
Thank you again.
Regards, Alberto.
Russell Beall wrote:
I had very few OS-related issues setting up on CentOS 6.2. I set a node up in this OS alongside a node in RedHat 6.2 and the settings for the directoryserver are identical. I was pleased at the very large quantity of documentation at the redhat site which describes every aspect of the product in very complete detail.
The documentation at RedHat never seemed very OS-specific to me, and the only differences I generally saw were where the documentation had differing instructions based on the older "fedora" version of the directory server.
What system differences did you encounter?
Is your issue related to being on a 32-bit system instead of a 64-bit system?
Regards, Russ.
On Apr 24, 2012, at 8:26 AM, Alberto Suárez wrote:
Hello:
I am struggling to set up 389DS on Centos 6.2. I can hardly find any information on how to do it, as everything is written for Centos 5. Due to the changes introduced in Centos 6 and the lack of information, I am finding it a nightmare, every step I make I run into new issues. Any help?
389 users mailing list 389-users@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/389-users
-- 389 users mailing list 389-users@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/389-users
-- 389 users mailing list 389-users@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/389-users
Hi Russell,
Thank you for your scripts.
I did not configure SSL for any particular bit of the software, I just followed the instructions given in the document "Configuring SSL Enabled Fedora Directory Server" found in fedora project's website. At a point it links to your script. I was wondering if the problem might come from likely differences between Centos 5 and Centos 5.
Thank you for your hint. I will look into it.
Alberto.
Russell Beall wrote:
Sure. Unfortunately you've gone beyond my limited experience with issues with the admin-server. I haven't configured SSL for the admin server, though I did get SSL working for the directory server.
I'll have to leave the rest of this to someone more experienced.
However, I have debugged issues like this, and one simple thing to make sure of is that the whole directory tree is accessible to the user that runs the process. Is the process owner of the admin server fedora-ds and does that user have read access to the whole tree? If you become that user can you see the files?
Regards, Russ.
On Apr 24, 2012, at 10:15 AM, Alberto Suárez wrote:
Thank you for your prompt reply. I am running Centos 6.2 64 bits. To be honest, I am a bit lost with so many things to take into account...
Right now I am a bit stuck at configuring TLS. I executed the script "setupssl2.sh" kindly provided by Rich Megginson and now i can no longer start the administration server. When I execute "service server-admin I get the following error message:
httpd.worker: Syntax error on line 735 of /etc/dirsrv/admin-serv/httpd.conf: Could not open configuartion file /etc/dirsrv/admin-serv/nss.conf: Permission denied
I have played with that file's permissions, even setting them as 777, but nothing changes. It is currently owned by user "fedora-ds" group "fedora-ds" and permissions are set to 400.
Thank you again.
Regards, Alberto.
Russell Beall wrote:
I had very few OS-related issues setting up on CentOS 6.2. I set a node up in this OS alongside a node in RedHat 6.2 and the settings for the directoryserver are identical. I was pleased at the very large quantity of documentation at the redhat site which describes every aspect of the product in very complete detail.
The documentation at RedHat never seemed very OS-specific to me, and the only differences I generally saw were where the documentation had differing instructions based on the older "fedora" version of the directory server.
What system differences did you encounter?
Is your issue related to being on a 32-bit system instead of a 64-bit system?
Regards, Russ.
On Apr 24, 2012, at 8:26 AM, Alberto Suárez wrote:
Hello:
I am struggling to set up 389DS on Centos 6.2. I can hardly find any information on how to do it, as everything is written for Centos 5. Due to the changes introduced in Centos 6 and the lack of information, I am finding it a nightmare, every step I make I run into new issues. Any help?
389 users mailing list 389-users@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/389-users
-- 389 users mailing list 389-users@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/389-users
-- 389 users mailing list 389-users@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/389-users
-- 389 users mailing list 389-users@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/389-users
Hi Alberto,
Le 24 avril 2012 16:15, Alberto Suárez asuapaz@gobiernodecanarias.org a écrit :
httpd.worker: Syntax error on line 735 of /etc/dirsrv/admin-serv/httpd.**conf: Could not open configuartion file /etc/dirsrv/admin-serv/nss.**conf: Permission denied
I have played with that file's permissions, even setting them as 777, but nothing changes. It is currently owned by user "fedora-ds" group "fedora-ds" and permissions are set to 400.
It may be SELinux-related
Thanks, Andrey. I will take a look at it.
Regards,
Alberto.
Andrey Ivanov wrote:
Hi Alberto,
Le 24 avril 2012 16:15, Alberto Suárez<asuapaz@gobiernodecanarias.org mailto:asuapaz@gobiernodecanarias.org> a écrit :
httpd.worker: Syntax error on line 735 of /etc/dirsrv/admin-serv/httpd.__conf: Could not open configuartion file /etc/dirsrv/admin-serv/nss.__conf: Permission denied I have played with that file's permissions, even setting them as 777, but nothing changes. It is currently owned by user "fedora-ds" group "fedora-ds" and permissions are set to 400.It may be SELinux-related
I've forgotten Linked Attributes plugin, you could also disable it.
Don't you have some exotic type of index activated for uniqueMember (like substring)? The default value is only the equality index in dse.ldif
dn: cn=uniquemember,cn=default indexes,cn=config,cn=ldbm database,cn=plugins,c n=config objectClass: top objectClass: nsIndex cn: uniquemember nsSystemIndex: false nsIndexType: eq
In any case, batch write loads are quite particular. You could try to play with nsslapd-db-checkpoint-interval and nsslapd-db-durable-transaction config attributes while you run your batch uniqueMember modifications. You could also try disabling completely or limiting logging intensity ( http://directory.fedoraproject.org/wiki/Named_Pipe_Log_Script) .
In my VM tests (RHEL5, 1 vCPU Xeon E5640 @ 2.67GHz, 1Gb mem, 389DS v1.2.10.6) with our production environment (~20k users, groups of ~6000 members created as in your case with perl scripts using ldapmodify running on the same VM) a group of 6000 uniqueMembers is created in 3 minutes 10 sec (190s) from scratch. Using "dstat" i see that the main problem is disk writes (transaction logs of db4): ----total-cpu-usage---- -dsk/total- -net/total- ---paging-- ---system-- usr sys idl wai hiq siq| read writ| recv send| in out | int csw 91 0 0 9 0 0| 0 40M| 120B 978B| 0 0 | 316 325 90 3 0 7 0 0| 0 40M| 60B 310B| 0 0 | 184 294 92 2 0 5 0 1| 0 40M| 60B 310B| 0 0 | 160 297 93 1 0 5 0 1| 0 40M| 60B 310B| 0 0 | 159 312 93 3 1 1 0 2| 0 24M| 60B 310B| 0 0 | 206 372 76 4 2 18 0 0| 0 40M| 60B 310B| 0 0 | 165 265 94 0 0 6 0 0| 0 40M| 60B 310B| 0 0 | 221 275 90 1 0 7 1 1| 0 41M| 60B 310B| 0 0 | 479 313 86 2 0 11 1 0| 0 40M| 60B 310B| 0 0 | 403 306 93 0 0 6 0 1| 0 20M| 120B 364B| 0 0 | 489 298 90 1 0 9 0 0| 0 40M| 60B 310B| 0 0 | 389 296 88 1 0 11 0 0| 0 40M| 60B 310B| 0 0 | 358 319 76 0 0 23 1 0| 0 41M| 60B 310B| 0 0 | 403 303
@+
Le 19 avril 2012 18:50, Russell Beall beall@usc.edu a écrit :
Thanks for the tips. I scanned the dse.ldif for those plugins and I found definitions for them all, but they all have nsslapd-pluginEnabled: off.
There is something special about the uniquemember attribute that requires additional processing different from other attributes... Ldapmodify of other attributes runs pretty quick.
Regards, Russ.
On Apr 19, 2012, at 2:20 AM, Andrey Ivanov wrote:
Hi Russel,
Le 18 avril 2012 23:06, Russell Beall beall@usc.edu a écrit :
On Apr 18, 2012, at 11:15 AM, Rich Megginson wrote:
Yeah, this particular operation has not been optimized. I believe SunDS added explicit optimizations for this particular case.
It is becoming painfully apparent as I write more detailed tests. 389 takes time to add or delete uniquemember values proportionate to the number of values being operated on and is using about twice as much time to delete as it does to add. Sun DS appears to have perhaps an almost O(1) algorithm in play on both adding and deleting values.
Is there perhaps some kind of referential integrity setting that is being used and forcing some kind of lookup of each value, one that we could perhaps turn off? We wouldn't need such a check because our metadirectory process handles the integrity/consistency checking already.
There is memberOf plugin that maintains the memberOf attribute for groups. I don't know whether it is activated by default or not. You could try to disable it. There is also referential integrity plugin, attribute uniqueness plugin, maybe USN plugin or custom indexes that could consume a lot of CPU. Make sure you've disabled them if you don't need them.
@+
389 users mailing list 389-users@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/389-users
-- 389 users mailing list 389-users@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/389-users
Hmm... Thanks again for the data.
I will see what happens with a modification of those attributes.
However, I tried running the modifications with and without the index (I deleted it entirely). When it was present, it matched the default index which only uses the eq index type. I think it went slightly faster without the index but the difference was negligible compared to the overall time.
Regards, Russ.
On Apr 19, 2012, at 11:47 AM, Andrey Ivanov wrote:
I've forgotten Linked Attributes plugin, you could also disable it.
Don't you have some exotic type of index activated for uniqueMember (like substring)? The default value is only the equality index in dse.ldif
dn: cn=uniquemember,cn=default indexes,cn=config,cn=ldbm database,cn=plugins,c n=config objectClass: top objectClass: nsIndex cn: uniquemember nsSystemIndex: false nsIndexType: eq
In any case, batch write loads are quite particular. You could try to play with nsslapd-db-checkpoint-interval and nsslapd-db-durable-transaction config attributes while you run your batch uniqueMember modifications. You could also try disabling completely or limiting logging intensity (http://directory.fedoraproject.org/wiki/Named_Pipe_Log_Script) .
In my VM tests (RHEL5, 1 vCPU Xeon E5640 @ 2.67GHz, 1Gb mem, 389DS v1.2.10.6) with our production environment (~20k users, groups of ~6000 members created as in your case with perl scripts using ldapmodify running on the same VM) a group of 6000 uniqueMembers is created in 3 minutes 10 sec (190s) from scratch. Using "dstat" i see that the main problem is disk writes (transaction logs of db4): ----total-cpu-usage---- -dsk/total- -net/total- ---paging-- ---system-- usr sys idl wai hiq siq| read writ| recv send| in out | int csw 91 0 0 9 0 0| 0 40M| 120B 978B| 0 0 | 316 325 90 3 0 7 0 0| 0 40M| 60B 310B| 0 0 | 184 294 92 2 0 5 0 1| 0 40M| 60B 310B| 0 0 | 160 297 93 1 0 5 0 1| 0 40M| 60B 310B| 0 0 | 159 312 93 3 1 1 0 2| 0 24M| 60B 310B| 0 0 | 206 372 76 4 2 18 0 0| 0 40M| 60B 310B| 0 0 | 165 265 94 0 0 6 0 0| 0 40M| 60B 310B| 0 0 | 221 275 90 1 0 7 1 1| 0 41M| 60B 310B| 0 0 | 479 313 86 2 0 11 1 0| 0 40M| 60B 310B| 0 0 | 403 306 93 0 0 6 0 1| 0 20M| 120B 364B| 0 0 | 489 298 90 1 0 9 0 0| 0 40M| 60B 310B| 0 0 | 389 296 88 1 0 11 0 0| 0 40M| 60B 310B| 0 0 | 358 319 76 0 0 23 1 0| 0 41M| 60B 310B| 0 0 | 403 303
@+
Le 19 avril 2012 18:50, Russell Beall beall@usc.edu a écrit : Thanks for the tips. I scanned the dse.ldif for those plugins and I found definitions for them all, but they all have nsslapd-pluginEnabled: off.
There is something special about the uniquemember attribute that requires additional processing different from other attributes... Ldapmodify of other attributes runs pretty quick.
Regards, Russ.
On Apr 19, 2012, at 2:20 AM, Andrey Ivanov wrote:
Hi Russel,
Le 18 avril 2012 23:06, Russell Beall beall@usc.edu a écrit : On Apr 18, 2012, at 11:15 AM, Rich Megginson wrote:
Yeah, this particular operation has not been optimized. I believe SunDS added explicit optimizations for this particular case.
It is becoming painfully apparent as I write more detailed tests. 389 takes time to add or delete uniquemember values proportionate to the number of values being operated on and is using about twice as much time to delete as it does to add. Sun DS appears to have perhaps an almost O(1) algorithm in play on both adding and deleting values.
Is there perhaps some kind of referential integrity setting that is being used and forcing some kind of lookup of each value, one that we could perhaps turn off? We wouldn't need such a check because our metadirectory process handles the integrity/consistency checking already.
There is memberOf plugin that maintains the memberOf attribute for groups. I don't know whether it is activated by default or not. You could try to disable it. There is also referential integrity plugin, attribute uniqueness plugin, maybe USN plugin or custom indexes that could consume a lot of CPU. Make sure you've disabled them if you don't need them.
@+
389 users mailing list 389-users@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/389-users
-- 389 users mailing list 389-users@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/389-users
-- 389 users mailing list 389-users@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/389-users
Hey russ, I've got the same problem for large groups using member... We are coming from an openldap world so not much use of uniquemember yet. On Apr 18, 2012 2:10 PM, "Russell Beall" beall@usc.edu wrote:
Does anybody have a pointer to any performance comparisons between Sun DS and 389?
I was extremely happy with the performance boost using 389 on a Linux VM which is 5-8 times faster for ldapsearch operations than the older Sun machines with Sun DS 6.3.
In testing one of our most important use cases just now, I find that the ldapmodify speed is many many times slower. This doesn't make much sense, so I think I'm doing something wrong or having something misconfigured.
Earlier I improved the write performance by using large db cache sizes and moving the nsslapd-db-home-directory to tmpfs. Now most modify operations have very little I/O wait except when occasionally flushing the index files and such, and yet, there is a CPU pegged for very long periods of time, orders of magnitude higher than on Sun DS.
Is there any documentation on ldapmodify performance that I could review? Google searching seems eerily silent on the issue… (which also leads me to believe I have something misconfigured if nobody has been asking about the issue…)
The particular use case I am working with involves replacing large quantities of uniqueMember values on entries in ou=groups.
Thanks, Russ.
============================== Russell Beall Programmer Analyst IV Enterprise Identity Management University of Southern California beall@usc.edu ==============================
-- 389 users mailing list 389-users@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/389-users
On 04/18/2012 01:33 PM, Michael Gettes wrote:
Hey russ, I've got the same problem for large groups using member... We are coming from an openldap world so not much use of uniquemember yet.
It's essentially the same problem - it doesn't matter if you use member or uniquemember.
On Apr 18, 2012 2:10 PM, "Russell Beall" <beall@usc.edu mailto:beall@usc.edu> wrote:
Does anybody have a pointer to any performance comparisons between Sun DS and 389? I was extremely happy with the performance boost using 389 on a Linux VM which is 5-8 times faster for ldapsearch operations than the older Sun machines with Sun DS 6.3. In testing one of our most important use cases just now, I find that the ldapmodify speed is many many times slower. This doesn't make much sense, so I think I'm doing something wrong or having something misconfigured. Earlier I improved the write performance by using large db cache sizes and moving the nsslapd-db-home-directory to tmpfs. Now most modify operations have very little I/O wait except when occasionally flushing the index files and such, and yet, there is a CPU pegged for very long periods of time, orders of magnitude higher than on Sun DS. Is there any documentation on ldapmodify performance that I could review? Google searching seems eerily silent on the issue… (which also leads me to believe I have something misconfigured if nobody has been asking about the issue…) The particular use case I am working with involves replacing large quantities of uniqueMember values on entries in ou=groups. Thanks, Russ. ============================== Russell Beall Programmer Analyst IV Enterprise Identity Management University of Southern California beall@usc.edu <mailto:beall@usc.edu> ============================== -- 389 users mailing list 389-users@lists.fedoraproject.org <mailto:389-users@lists.fedoraproject.org> https://admin.fedoraproject.org/mailman/listinfo/389-users-- 389 users mailing list 389-users@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/389-users
Dear all, i've got a little problem using the new wdmin Console ( http://port389.org/download/389-Console-1.1.6-i386.msi ) with my 389 Server:
[root@pserver ~]# rpm -qa | grep 389 389-ds-base-1.2.9.9-1.el5 389-ds-console-1.2.6-1.el5 389-dsgw-1.1.7-2.el5 389-console-1.1.7-3.el5 389-ds-console-doc-1.2.6-1.el5 389-admin-1.1.23-1.el5 389-admin-console-1.1.8-1.el5 389-admin-console-doc-1.1.8-1.el5 389-ds-1.2.1-1.el5 389-ds-base-libs-1.2.9.9-1.el5 389-adminutil-1.1.14-1.el5
When i try to connect to Directory Server, client issues an error called "Class Loader Error" with this txt:
" Failed to install a local copy of 389-ds-123.jar or one of its supporting files. ...."
What happen? Can anyone help me? Anyway i didn't have this error with the previous version of Admin Console (i think 1.1.4).... i'm looking in the website where is that file, but there isn't.
Thanks a lot.
Have a nice day,
Dear all, i've got a little problem using the new wdmin Console ( http://port389.org/download/389-Console-1.1.6-i386.msi ) with my 389 Server:
[root@pserver ~]# rpm -qa | grep 389 389-ds-base-1.2.9.9-1.el5 389-ds-console-1.2.6-1.el5 389-dsgw-1.1.7-2.el5 389-console-1.1.7-3.el5 389-ds-console-doc-1.2.6-1.el5 389-admin-1.1.23-1.el5 389-admin-console-1.1.8-1.el5 389-admin-console-doc-1.1.8-1.el5 389-ds-1.2.1-1.el5 389-ds-base-libs-1.2.9.9-1.el5 389-adminutil-1.1.14-1.el5
When i try to connect to Directory Server, client issues an error called "Class Loader Error" with this txt:
" Failed to install a local copy of 389-ds-123.jar or one of its supporting files. ...."
What happen? Can anyone help me? Anyway i didn't have this error with the previous version of Admin Console (i think 1.1.4).... i'm looking in the website where is that file, but there isn't.
Thanks a lot.
Have a nice day,
On Thu, 19 Apr 2012 13:52:08 +0200 (CEST) "Andrea Modesto Rossi" amrossi@linux.it wrote:
Dear all, i've got a little problem using the new wdmin Console ( http://port389.org/download/389-Console-1.1.6-i386.msi ) with my 389 Server:
[root@pserver ~]# rpm -qa | grep 389 389-ds-base-1.2.9.9-1.el5 389-ds-console-1.2.6-1.el5 389-dsgw-1.1.7-2.el5 389-console-1.1.7-3.el5 389-ds-console-doc-1.2.6-1.el5 389-admin-1.1.23-1.el5 389-admin-console-1.1.8-1.el5 389-admin-console-doc-1.1.8-1.el5 389-ds-1.2.1-1.el5 389-ds-base-libs-1.2.9.9-1.el5 389-adminutil-1.1.14-1.el5
When i try to connect to Directory Server, client issues an error called "Class Loader Error" with this txt:
" Failed to install a local copy of 389-ds-123.jar or one of its supporting files. ...."
What happen? Can anyone help me? Anyway i didn't have this error with the previous version of Admin Console (i think 1.1.4).... i'm looking in the website where is that file, but there isn't.
hello andrea
does work here windows 7 64 bit java 64bit 6u31
server is cento6 rpm -qa |grep 389 389-admin-1.1.25-1.el6.x86_64 389-ds-console-1.2.6-1.el6.noarch 389-adminutil-devel-1.1.14-2.el6.x86_64 389-ds-base-1.2.9.14-1.el6_2.2.x86_64 389-admin-console-1.1.8-1.el6.noarch 389-dsgw-1.1.7-2.el6.x86_64 389-ds-base-devel-1.2.9.14-1.el6_2.2.x86_64 389-admin-console-doc-1.1.8-1.el6.noarch 389-console-1.1.7-1.el6.noarch 389-ds-base-libs-1.2.9.14-1.el6_2.2.x86_64 389-adminutil-1.1.14-2.el6.x86_64 389-ds-console-doc-1.2.6-1.el6.noarch 389-ds-1.2.2-1.el6.noarch
-m
On Gio, 19 Aprile 2012 1:52 pm, Andrea Modesto Rossi wrote:
Dear all, i've got a little problem using the new wdmin Console ( http://port389.org/download/389-Console-1.1.6-i386.msi ) with my 389 Server:
[root@pserver ~]# rpm -qa | grep 389 389-ds-base-1.2.9.9-1.el5 389-ds-console-1.2.6-1.el5 389-dsgw-1.1.7-2.el5 389-console-1.1.7-3.el5 389-ds-console-doc-1.2.6-1.el5 389-admin-1.1.23-1.el5 389-admin-console-1.1.8-1.el5 389-admin-console-doc-1.1.8-1.el5 389-ds-1.2.1-1.el5 389-ds-base-libs-1.2.9.9-1.el5 389-adminutil-1.1.14-1.el5
When i try to connect to Directory Server, client issues an error called "Class Loader Error" with this txt:
" Failed to install a local copy of 389-ds-123.jar or one of its supporting files. ...."
What happen? Can anyone help me? Anyway i didn't have this error with the previous version of Admin Console (i think 1.1.4).... i'm looking in the website where is that file, but there isn't.
Thanks a lot.
Have a nice day,
Please, does anyone can help me to fix this boring issue?
I'm not able to manage my ldap server :-( (of course, through bash console i can...but ..)
Thanks in advantage to anyone help me...
389-users@lists.fedoraproject.org
-
Alberto Suárez -
Andrea Modesto Rossi -
Ivanov Andrey (M.) -
Maurizio Marini -
Michael Gettes -
Rich Megginson -
Russell Beall