389-commits March 2019

389-commits@lists.fedoraproject.org

1 participants
36 discussions

[389-ds-base] branch master updated: Bump version to 1.4.1.2
by pagure＠pagure.io 29 Mar '19

29 Mar '19

This is an automated email from the git hooks/post-receive script. mreynolds pushed a commit to branch master in repository 389-ds-base. The following commit(s) were added to refs/heads/master by this push: new 9a12661 Bump version to 1.4.1.2 9a12661 is described below commit 9a126614a5ed10b5d999680a2be2ae83d94eb55c Author: Mark Reynolds <mreynolds(a)redhat.com> AuthorDate: Fri Mar 29 16:06:14 2019 -0400 Bump version to 1.4.1.2 --- VERSION.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/VERSION.sh b/VERSION.sh index 4acdeb6..a0b867d 100644 --- a/VERSION.sh +++ b/VERSION.sh @@ -10,7 +10,7 @@ vendor="389 Project" # PACKAGE_VERSION is constructed from these VERSION_MAJOR=1 VERSION_MINOR=4 -VERSION_MAINT=1.1 +VERSION_MAINT=1.2 # NOTE: VERSION_PREREL is automatically set for builds made out of a git tree VERSION_PREREL= VERSION_DATE=$(date -u +%Y%m%d) -- To stop receiving notification emails like this one, please contact the administrator of this repository.

1 0

[389-ds-base] branch 389-ds-base-1.4.0 updated: Bump version to 1.4.0.22
by pagure＠pagure.io 29 Mar '19

29 Mar '19

This is an automated email from the git hooks/post-receive script. mreynolds pushed a commit to branch 389-ds-base-1.4.0 in repository 389-ds-base. The following commit(s) were added to refs/heads/389-ds-base-1.4.0 by this push: new 9d84a40 Bump version to 1.4.0.22 9d84a40 is described below commit 9d84a40dd3ceceafd619f9c6837b6ce41a1c02ee Author: Mark Reynolds <mreynolds(a)redhat.com> AuthorDate: Fri Mar 29 15:40:43 2019 -0400 Bump version to 1.4.0.22 --- VERSION.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/VERSION.sh b/VERSION.sh index 1b43006..e0fae64 100644 --- a/VERSION.sh +++ b/VERSION.sh @@ -10,7 +10,7 @@ vendor="389 Project" # PACKAGE_VERSION is constructed from these VERSION_MAJOR=1 VERSION_MINOR=4 -VERSION_MAINT=0.21 +VERSION_MAINT=0.22 # NOTE: VERSION_PREREL is automatically set for builds made out of a git tree VERSION_PREREL= VERSION_DATE=$(date -u +%Y%m%d) -- To stop receiving notification emails like this one, please contact the administrator of this repository.

1 0

[389-ds-base] branch 389-ds-base-1.3.9 updated: Ticket 50063 - Crash after attempting to restore a single backend
by pagure＠pagure.io 29 Mar '19

29 Mar '19

This is an automated email from the git hooks/post-receive script. mreynolds pushed a commit to branch 389-ds-base-1.3.9 in repository 389-ds-base. The following commit(s) were added to refs/heads/389-ds-base-1.3.9 by this push: new b5e599a Ticket 50063 - Crash after attempting to restore a single backend b5e599a is described below commit b5e599ae5a0bfaad77b8ba8a1846f1b56fa5993c Author: Mark Reynolds <mreynolds(a)redhat.com> AuthorDate: Fri Nov 30 11:45:39 2018 -0500 Ticket 50063 - Crash after attempting to restore a single backend Description: While we do not support backup/restore of individual backends, it should not crash the server either. PR_OpenDir will crash if the file name is NULL, so this fix just prevents the crash by returning an error if the filename is NULL. https://pagure.io/389-ds-base/issue/50063 Reviewed by: firstyear & tbordaz(Thanks!!) (cherry picked from commit d36f796a7b35bade5a05e197690abf4e49d212ce) --- ldap/servers/slapd/back-ldbm/dblayer.c | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/ldap/servers/slapd/back-ldbm/dblayer.c b/ldap/servers/slapd/back-ldbm/dblayer.c index fa931cc..64a4e9e 100644 --- a/ldap/servers/slapd/back-ldbm/dblayer.c +++ b/ldap/servers/slapd/back-ldbm/dblayer.c @@ -5292,6 +5292,11 @@ dblayer_delete_database_ex(struct ldbminfo *li, char *instance, char *cldir) } /* now smash everything else in the db/ dir */ + if (priv->dblayer_home_directory == NULL){ + slapi_log_err(SLAPI_LOG_ERR, "dblayer_delete_database_ex", + "dblayer_home_directory is NULL, can not proceed\n"); + return -1; + } dirhandle = PR_OpenDir(priv->dblayer_home_directory); if (!dirhandle) { slapi_log_err(SLAPI_LOG_ERR, "dblayer_delete_database_ex", "PR_OpenDir (%s) failed (%d): %s\n", -- To stop receiving notification emails like this one, please contact the administrator of this repository.

1 0

[389-ds-base] branch 389-ds-base-1.4.0 updated: Ticket 50308 - Revise memory leak fix
by pagure＠pagure.io 29 Mar '19

29 Mar '19

This is an automated email from the git hooks/post-receive script. mreynolds pushed a commit to branch 389-ds-base-1.4.0 in repository 389-ds-base. The following commit(s) were added to refs/heads/389-ds-base-1.4.0 by this push: new 11430af Ticket 50308 - Revise memory leak fix 11430af is described below commit 11430afdc0da682905d7dbaf7d3978506f2bc7a1 Author: Mark Reynolds <mreynolds(a)redhat.com> AuthorDate: Fri Mar 29 14:59:05 2019 -0400 Ticket 50308 - Revise memory leak fix Description; Turns out the previous commit did not address the changelog leak, and it introduced a compiler warning. This part of the fix is being reverted. https://pagure.io/389-ds-base/issue/50308 (cherry picked from commit 1808f317d5ef240fd0ac3947fcb90a895e46b436) --- ldap/servers/plugins/replication/cl5_clcache.c | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/ldap/servers/plugins/replication/cl5_clcache.c b/ldap/servers/plugins/replication/cl5_clcache.c index 6c18829..6b591fb 100644 --- a/ldap/servers/plugins/replication/cl5_clcache.c +++ b/ldap/servers/plugins/replication/cl5_clcache.c @@ -547,6 +547,7 @@ clcache_refresh_consumer_maxcsns(CLC_Buffer *buf) int i; for (i = 0; i < buf->buf_num_cscbs; i++) { + csn_free(&buf->buf_cscbs[i]->consumer_maxcsn); ruv_get_largest_csn_for_replica( buf->buf_consumer_ruv, buf->buf_cscbs[i]->rid, @@ -589,7 +590,6 @@ clcache_refresh_local_maxcsn(const ruv_enum_data *rid_data, void *data) /* this is the first time we have a local change for the RID * we need to check what the consumer knows about it. */ - csn_free(&buf->buf_cscbs[i]->consumer_maxcsn); ruv_get_largest_csn_for_replica( buf->buf_consumer_ruv, buf->buf_cscbs[i]->rid, @@ -833,7 +833,6 @@ clcache_skip_change(CLC_Buffer *buf) /* Send CSNs that are covered by the local RUV snapshot */ if (csn_compare(buf->buf_current_csn, cscb->local_maxcsn) <= 0) { skip = 0; - csn_free(&cscb->consumer_maxcsn); csn_dup_or_init_by_csn(&cscb->consumer_maxcsn, buf->buf_current_csn); break; } @@ -848,8 +847,7 @@ clcache_skip_change(CLC_Buffer *buf) { csn_init_by_csn(cscb->local_maxcsn, buf->buf_current_csn); if (cscb->consumer_maxcsn) { - csn_free(&cscb->consumer_maxcsn); - csn_dup_or_init_by_csn(cscb->consumer_maxcsn, buf->buf_current_csn); + csn_init_by_csn(cscb->consumer_maxcsn, buf->buf_current_csn); } skip = 0; break; -- To stop receiving notification emails like this one, please contact the administrator of this repository.

1 0

[389-ds-base] branch master updated: Ticket 50308 - Revise memory leak fix
by pagure＠pagure.io 29 Mar '19

29 Mar '19

This is an automated email from the git hooks/post-receive script. mreynolds pushed a commit to branch master in repository 389-ds-base. The following commit(s) were added to refs/heads/master by this push: new 1808f31 Ticket 50308 - Revise memory leak fix 1808f31 is described below commit 1808f317d5ef240fd0ac3947fcb90a895e46b436 Author: Mark Reynolds <mreynolds(a)redhat.com> AuthorDate: Fri Mar 29 14:59:05 2019 -0400 Ticket 50308 - Revise memory leak fix Description; Turns out the previous commit did not address the changelog leak, and it introduced a compiler warning. This part of the fix is being reverted. https://pagure.io/389-ds-base/issue/50308 --- ldap/servers/plugins/replication/cl5_clcache.c | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/ldap/servers/plugins/replication/cl5_clcache.c b/ldap/servers/plugins/replication/cl5_clcache.c index 6c18829..6b591fb 100644 --- a/ldap/servers/plugins/replication/cl5_clcache.c +++ b/ldap/servers/plugins/replication/cl5_clcache.c @@ -547,6 +547,7 @@ clcache_refresh_consumer_maxcsns(CLC_Buffer *buf) int i; for (i = 0; i < buf->buf_num_cscbs; i++) { + csn_free(&buf->buf_cscbs[i]->consumer_maxcsn); ruv_get_largest_csn_for_replica( buf->buf_consumer_ruv, buf->buf_cscbs[i]->rid, @@ -589,7 +590,6 @@ clcache_refresh_local_maxcsn(const ruv_enum_data *rid_data, void *data) /* this is the first time we have a local change for the RID * we need to check what the consumer knows about it. */ - csn_free(&buf->buf_cscbs[i]->consumer_maxcsn); ruv_get_largest_csn_for_replica( buf->buf_consumer_ruv, buf->buf_cscbs[i]->rid, @@ -833,7 +833,6 @@ clcache_skip_change(CLC_Buffer *buf) /* Send CSNs that are covered by the local RUV snapshot */ if (csn_compare(buf->buf_current_csn, cscb->local_maxcsn) <= 0) { skip = 0; - csn_free(&cscb->consumer_maxcsn); csn_dup_or_init_by_csn(&cscb->consumer_maxcsn, buf->buf_current_csn); break; } @@ -848,8 +847,7 @@ clcache_skip_change(CLC_Buffer *buf) { csn_init_by_csn(cscb->local_maxcsn, buf->buf_current_csn); if (cscb->consumer_maxcsn) { - csn_free(&cscb->consumer_maxcsn); - csn_dup_or_init_by_csn(cscb->consumer_maxcsn, buf->buf_current_csn); + csn_init_by_csn(cscb->consumer_maxcsn, buf->buf_current_csn); } skip = 0; break; -- To stop receiving notification emails like this one, please contact the administrator of this repository.

1 0

[389-ds-base] branch 389-ds-base-1.3.9 updated: Ticket 49873 - Contention on virtual attribute lookup
by pagure＠pagure.io 29 Mar '19

29 Mar '19

This is an automated email from the git hooks/post-receive script. tbordaz pushed a commit to branch 389-ds-base-1.3.9 in repository 389-ds-base. The following commit(s) were added to refs/heads/389-ds-base-1.3.9 by this push: new f6c0c84 Ticket 49873 - Contention on virtual attribute lookup f6c0c84 is described below commit f6c0c84e24096ca7e789af8ff03e11e2edc73a7e Author: Thierry Bordaz <tbordaz(a)redhat.com> AuthorDate: Tue Jan 15 11:13:42 2019 +0100 Ticket 49873 - Contention on virtual attribute lookup Bug Description: During lookup of the virtual attribute table (filter evaluation and returned attribute) the lock is acquired many times in read. For example it is acquired for each targetfilter aci and for each evaluated entry. Unfortunately RW lock is expensive and appears frequently on pstacks. The lock exists because the table can be updated but update is very rare (addition of a new service provider). So it slows down general proceeding for exceptional events. Fix Description: The fix is to acquire/release the read lock at the operation level and set a per-cpu flag, so that later lookup would just check the flag. SSL initialization does internal searches that access the vattr_global_lock Call of vattr_global_lock_create needs to be called before slapd_do_all_nss_ssl_init. Also, 'main' may or may not fork, the initialization fo the thread private variable is done either on the child or parent depending if main forks or not. The leak is fixed using a destructor callback of the private variable and so call PR_SetThreadPrivate only if there is no private variable. This patch is the merge of the four 49873 patches done in master https://pagure.io/389-ds-base/issue/49873 Reviewed by: Ludwig Krispenz, William Brown , Simon Pichugi (thanks !!) Platforms tested: F28 Flag Day: no Doc impact: no --- ldap/servers/slapd/detach.c | 9 +++ ldap/servers/slapd/opshared.c | 6 ++ ldap/servers/slapd/proto-slap.h | 5 ++ ldap/servers/slapd/vattr.c | 164 +++++++++++++++++++++++++++++++++++----- 4 files changed, 167 insertions(+), 17 deletions(-) diff --git a/ldap/servers/slapd/detach.c b/ldap/servers/slapd/detach.c index 681e6a7..d5c95a0 100644 --- a/ldap/servers/slapd/detach.c +++ b/ldap/servers/slapd/detach.c @@ -144,6 +144,10 @@ detach(int slapd_exemode, int importexport_encrypt, int s_port, daemon_ports_t * } break; } + /* The thread private counter needs to be allocated after the fork + * it is not inherited from parent process + */ + vattr_global_lock_create(); /* call this right after the fork, but before closing stdin */ if (slapd_do_all_nss_ssl_init(slapd_exemode, importexport_encrypt, s_port, ports_info)) { @@ -174,6 +178,11 @@ detach(int slapd_exemode, int importexport_encrypt, int s_port, daemon_ports_t * g_set_detached(1); } else { /* not detaching - call nss/ssl init */ + /* The thread private counter needs to be allocated after the fork + * it is not inherited from parent process + */ + vattr_global_lock_create(); + if (slapd_do_all_nss_ssl_init(slapd_exemode, importexport_encrypt, s_port, ports_info)) { return 1; } diff --git a/ldap/servers/slapd/opshared.c b/ldap/servers/slapd/opshared.c index 50b7ae8..cf6cdff 100644 --- a/ldap/servers/slapd/opshared.c +++ b/ldap/servers/slapd/opshared.c @@ -244,6 +244,7 @@ op_shared_search(Slapi_PBlock *pb, int send_result) int pr_idx = -1; Slapi_DN *orig_sdn = NULL; int free_sdn = 0; + PRBool vattr_lock_acquired = PR_FALSE; be_list[0] = NULL; referral_list[0] = NULL; @@ -511,6 +512,8 @@ op_shared_search(Slapi_PBlock *pb, int send_result) } slapi_pblock_set(pb, SLAPI_BACKEND_COUNT, &index); + vattr_rdlock(); + vattr_lock_acquired = PR_TRUE; if (be) { slapi_pblock_set(pb, SLAPI_BACKEND, be); @@ -969,6 +972,9 @@ free_and_return: } else if (be_single) { slapi_be_Unlock(be_single); } + if (vattr_lock_acquired) { + vattr_rd_unlock(); + } free_and_return_nolock: slapi_pblock_set(pb, SLAPI_PLUGIN_OPRETURN, &rc); diff --git a/ldap/servers/slapd/proto-slap.h b/ldap/servers/slapd/proto-slap.h index 7a429b2..79017e6 100644 --- a/ldap/servers/slapd/proto-slap.h +++ b/ldap/servers/slapd/proto-slap.h @@ -1409,6 +1409,11 @@ void subentry_create_filter(Slapi_Filter **filter); * vattr.c */ void vattr_init(void); +void vattr_global_lock_create(void); +void vattr_rdlock(); +void vattr_rd_unlock(); +void vattr_wrlock(); +void vattr_wr_unlock(); void vattr_cleanup(void); /* diff --git a/ldap/servers/slapd/vattr.c b/ldap/servers/slapd/vattr.c index f7c473a..852a887 100644 --- a/ldap/servers/slapd/vattr.c +++ b/ldap/servers/slapd/vattr.c @@ -102,6 +102,16 @@ int vattr_basic_sp_init(); void **statechange_api; +struct _vattr_map +{ + Slapi_RWLock *lock; + PLHashTable *hashtable; /* Hash table */ +}; +typedef struct _vattr_map vattr_map; + +static vattr_map *the_map = NULL; +static PRUintn thread_private_global_vattr_lock; + /* Housekeeping Functions, called by server startup/shutdown code */ /* Called on server startup, init all structures etc */ @@ -115,7 +125,136 @@ vattr_init() vattr_basic_sp_init(); #endif } +/* + * https://developer.mozilla.org/en-US/docs/Mozilla/Projects/NSPR/Reference/PR… + * It is called each time: + * - PR_SetThreadPrivate is call with a not NULL private value + * - on thread exit + */ +static void +vattr_global_lock_free(void *ptr) +{ + int *nb_acquired = ptr; + if (nb_acquired) { + slapi_ch_free((void **)&nb_acquired); + } +} +/* Create a private variable for each individual thread of the current process */ +void +vattr_global_lock_create() +{ + if (PR_NewThreadPrivateIndex(&thread_private_global_vattr_lock, vattr_global_lock_free) != PR_SUCCESS) { + slapi_log_err(SLAPI_LOG_ALERT, + "vattr_global_lock_create", "Failure to create global lock for virtual attribute !\n"); + PR_ASSERT(0); + } +} +static int +global_vattr_lock_get_acquired_count() +{ + int *nb_acquired; + nb_acquired = (int *) PR_GetThreadPrivate(thread_private_global_vattr_lock); + if (nb_acquired == NULL) { + /* if it was not initialized set it to zero */ + nb_acquired = (int *) slapi_ch_calloc(1, sizeof(int)); + PR_SetThreadPrivate(thread_private_global_vattr_lock, (void *) nb_acquired); + } + return *nb_acquired; +} +static void +global_vattr_lock_set_acquired_count(int nb_acquired) +{ + int *val; + val = (int *) PR_GetThreadPrivate(thread_private_global_vattr_lock); + if (val == NULL) { + /* if it was not initialized set it to zero */ + val = (int *) slapi_ch_calloc(1, sizeof(int)); + PR_SetThreadPrivate(thread_private_global_vattr_lock, (void *) val); + } + *val = nb_acquired; +} +/* The map lock can be acquired recursively. So only the first rdlock + * will acquire the lock. + * A optimization acquires it at high level (op_shared_search), so that + * later calls during the operation processing will just increase/decrease a counter. + */ +void +vattr_rdlock() +{ + int nb_acquire = global_vattr_lock_get_acquired_count(); + + if (nb_acquire == 0) { + /* The lock was not held just acquire it */ + slapi_rwlock_rdlock(the_map->lock); + } + nb_acquire++; + global_vattr_lock_set_acquired_count(nb_acquire); + +} +/* The map lock can be acquired recursively. So only the last unlock + * will release the lock. + * A optimization acquires it at high level (op_shared_search), so that + * later calls during the operation processing will just increase/decrease a counter. + */ +void +vattr_rd_unlock() +{ + int nb_acquire = global_vattr_lock_get_acquired_count(); + if (nb_acquire >= 1) { + nb_acquire--; + if (nb_acquire == 0) { + slapi_rwlock_unlock(the_map->lock); + } + global_vattr_lock_set_acquired_count(nb_acquire); + } else { + /* this is likely the consequence of lock acquire in read during an internal search + * but the search callback updated the map and release the readlock and acquired + * it in write. + * So after the update the lock was no longer held but when completing the internal + * search we release the global read lock, that now has nothing to do + */ + slapi_log_err(SLAPI_LOG_DEBUG, + "vattr_rd_unlock", "vattr lock no longer acquired in read.\n"); + } +} + +/* The map lock is acquired in write (updating the map) + * It exists a possibility that lock is acquired in write while it is already + * hold in read by this thread (internal search with updating callback) + * In such situation, the we must abandon the read global lock and acquire in write + */ +void +vattr_wrlock() +{ + int nb_read_acquire = global_vattr_lock_get_acquired_count(); + + if (nb_read_acquire) { + /* The lock was acquired in read but we need it in write + * release it and set the global vattr_lock counter to 0 + */ + slapi_rwlock_unlock(the_map->lock); + global_vattr_lock_set_acquired_count(0); + } + slapi_rwlock_wrlock(the_map->lock); +} +/* The map lock is release from a write write (updating the map) + */ +void +vattr_wr_unlock() +{ + int nb_read_acquire = global_vattr_lock_get_acquired_count(); + + if (nb_read_acquire) { + /* The lock being acquired in write, the private thread counter + * (that count the number of time it was acquired in read) should be 0 + */ + slapi_log_err(SLAPI_LOG_INFO, + "vattr_unlock", "The lock was acquired in write. We should not be here\n"); + PR_ASSERT(nb_read_acquire == 0); + } + slapi_rwlock_unlock(the_map->lock); +} /* Called on server shutdown, free all structures, inform service providers that we're going down etc */ void vattr_cleanup() @@ -1811,15 +1950,6 @@ typedef struct _vattr_map_entry vattr_map_entry; vattr_map_entry test_entry = {NULL}; -struct _vattr_map -{ - Slapi_RWLock *lock; - PLHashTable *hashtable; /* Hash table */ -}; -typedef struct _vattr_map vattr_map; - -static vattr_map *the_map = NULL; - static PRIntn vattr_hash_compare_keys(const void *v1, const void *v2) { @@ -1939,11 +2069,11 @@ vattr_map_lookup(const char *type_to_find, vattr_map_entry **result) } /* Get the reader lock */ - slapi_rwlock_rdlock(the_map->lock); + vattr_rdlock(); *result = (vattr_map_entry *)PL_HashTableLookupConst(the_map->hashtable, (void *)basetype); /* Release ze lock */ - slapi_rwlock_unlock(the_map->lock); + vattr_rd_unlock(); if (tmp) { slapi_ch_free_string(&tmp); @@ -1962,13 +2092,13 @@ vattr_map_insert(vattr_map_entry *vae) { PR_ASSERT(the_map); /* Get the writer lock */ - slapi_rwlock_wrlock(the_map->lock); + vattr_wrlock(); /* Insert the thing */ /* It's illegal to call this function if the entry is already there */ PR_ASSERT(NULL == PL_HashTableLookupConst(the_map->hashtable, (void *)vae->type_name)); PL_HashTableAdd(the_map->hashtable, (void *)vae->type_name, (void *)vae); /* Unlock and we're done */ - slapi_rwlock_unlock(the_map->lock); + vattr_wr_unlock(); return 0; } @@ -2105,13 +2235,13 @@ schema_changed_callback(Slapi_Entry *e __attribute__((unused)), void *caller_data __attribute__((unused))) { /* Get the writer lock */ - slapi_rwlock_wrlock(the_map->lock); + vattr_wrlock(); /* go through the list */ PL_HashTableEnumerateEntries(the_map->hashtable, vattr_map_entry_rebuild_schema, 0); /* Unlock and we're done */ - slapi_rwlock_unlock(the_map->lock); + vattr_wr_unlock(); } @@ -2131,7 +2261,7 @@ slapi_vattr_schema_check_type(Slapi_Entry *e, char *type) objAttrValue *obj; if (0 == vattr_map_lookup(type, &map_entry)) { - slapi_rwlock_rdlock(the_map->lock); + vattr_rdlock(); obj = map_entry->objectclasses; @@ -2148,7 +2278,7 @@ slapi_vattr_schema_check_type(Slapi_Entry *e, char *type) obj = obj->pNext; } - slapi_rwlock_unlock(the_map->lock); + vattr_rd_unlock(); } slapi_valueset_free(vs); -- To stop receiving notification emails like this one, please contact the administrator of this repository.

1 0

[389-ds-base] branch 389-ds-base-1.4.0 updated: Ticket 50308 - Fix memory leaks for repeat binds and replication
by pagure＠pagure.io 29 Mar '19

29 Mar '19

This is an automated email from the git hooks/post-receive script. mreynolds pushed a commit to branch 389-ds-base-1.4.0 in repository 389-ds-base. The following commit(s) were added to refs/heads/389-ds-base-1.4.0 by this push: new 85c3c30 Ticket 50308 - Fix memory leaks for repeat binds and replication 85c3c30 is described below commit 85c3c304b4f247dbd6521cd8ca6e7840bcc212bb Author: Mark Reynolds <mreynolds(a)redhat.com> AuthorDate: Thu Mar 28 18:15:10 2019 -0400 Ticket 50308 - Fix memory leaks for repeat binds and replication Description: Fixed two memory leaks: - If a worker thread had multiple binds the "bind dn" thread data was leaked. - Memory leak when processing changes in the changelog https://pagure.io/389-ds-base/issue/50308 Reviewed by: firstyear(Thanks!) (cherry picked from commit 6c2bb66f15d7ab8ab079effc66e0705c2513b1fd) --- ldap/servers/plugins/replication/cl5_clcache.c | 9 ++++++--- ldap/servers/slapd/thread_data.c | 2 ++ 2 files changed, 8 insertions(+), 3 deletions(-) diff --git a/ldap/servers/plugins/replication/cl5_clcache.c b/ldap/servers/plugins/replication/cl5_clcache.c index a8477a8..6c18829 100644 --- a/ldap/servers/plugins/replication/cl5_clcache.c +++ b/ldap/servers/plugins/replication/cl5_clcache.c @@ -589,6 +589,7 @@ clcache_refresh_local_maxcsn(const ruv_enum_data *rid_data, void *data) /* this is the first time we have a local change for the RID * we need to check what the consumer knows about it. */ + csn_free(&buf->buf_cscbs[i]->consumer_maxcsn); ruv_get_largest_csn_for_replica( buf->buf_consumer_ruv, buf->buf_cscbs[i]->rid, @@ -832,6 +833,7 @@ clcache_skip_change(CLC_Buffer *buf) /* Send CSNs that are covered by the local RUV snapshot */ if (csn_compare(buf->buf_current_csn, cscb->local_maxcsn) <= 0) { skip = 0; + csn_free(&cscb->consumer_maxcsn); csn_dup_or_init_by_csn(&cscb->consumer_maxcsn, buf->buf_current_csn); break; } @@ -842,11 +844,12 @@ clcache_skip_change(CLC_Buffer *buf) * are not sure if current_csn is the neighbor. */ if (csn_time_difference(buf->buf_current_csn, cscb->local_maxcsn) == 0 && - (csn_get_seqnum(buf->buf_current_csn) == - csn_get_seqnum(cscb->local_maxcsn) + 1)) { + (csn_get_seqnum(buf->buf_current_csn) == csn_get_seqnum(cscb->local_maxcsn) + 1)) + { csn_init_by_csn(cscb->local_maxcsn, buf->buf_current_csn); if (cscb->consumer_maxcsn) { - csn_init_by_csn(cscb->consumer_maxcsn, buf->buf_current_csn); + csn_free(&cscb->consumer_maxcsn); + csn_dup_or_init_by_csn(cscb->consumer_maxcsn, buf->buf_current_csn); } skip = 0; break; diff --git a/ldap/servers/slapd/thread_data.c b/ldap/servers/slapd/thread_data.c index 7babe36..fb9951d 100644 --- a/ldap/servers/slapd/thread_data.c +++ b/ldap/servers/slapd/thread_data.c @@ -98,6 +98,8 @@ slapi_td_get_plugin_locked() int32_t slapi_td_set_dn(char *value) { + char *dn = pthread_getspecific(td_requestor_dn); + slapi_ch_free_string(&dn); if (pthread_setspecific(td_requestor_dn, value) != 0) { return PR_FAILURE; } -- To stop receiving notification emails like this one, please contact the administrator of this repository.

1 0

[389-ds-base] branch 389-ds-base-1.4.0 updated: Ticket 49873 - (cont 3rd) cleanup debug log
by pagure＠pagure.io 28 Mar '19

28 Mar '19

1 0

[389-ds-base] branch master updated: Ticket 49873 - (cont 3rd) cleanup debug log
by pagure＠pagure.io 28 Mar '19

28 Mar '19

This is an automated email from the git hooks/post-receive script. tbordaz pushed a commit to branch master in repository 389-ds-base. The following commit(s) were added to refs/heads/master by this push: new 235bde9 Ticket 49873 - (cont 3rd) cleanup debug log 235bde9 is described below commit 235bde930b19eaf7868118123e35f07f5e1911a4 Author: Thierry Bordaz <tbordaz(a)redhat.com> AuthorDate: Thu Mar 28 17:58:46 2019 +0100 Ticket 49873 - (cont 3rd) cleanup debug log --- ldap/servers/slapd/vattr.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/ldap/servers/slapd/vattr.c b/ldap/servers/slapd/vattr.c index bc4d0e9..852a887 100644 --- a/ldap/servers/slapd/vattr.c +++ b/ldap/servers/slapd/vattr.c @@ -214,7 +214,7 @@ vattr_rd_unlock() * So after the update the lock was no longer held but when completing the internal * search we release the global read lock, that now has nothing to do */ - slapi_log_err(SLAPI_LOG_INFO, + slapi_log_err(SLAPI_LOG_DEBUG, "vattr_rd_unlock", "vattr lock no longer acquired in read.\n"); } } -- To stop receiving notification emails like this one, please contact the administrator of this repository.

1 0

[389-ds-base] branch 389-ds-base-1.4.0 updated: Ticket 49873 - (cont 2nd) Contention on virtual attribute lookup
by pagure＠pagure.io 27 Mar '19

27 Mar '19

This is an automated email from the git hooks/post-receive script. tbordaz pushed a commit to branch 389-ds-base-1.4.0 in repository 389-ds-base. The following commit(s) were added to refs/heads/389-ds-base-1.4.0 by this push: new 74490fb Ticket 49873 - (cont 2nd) Contention on virtual attribute lookup 74490fb is described below commit 74490fb28c5c091dcc4c6eb492f2cef2acaf4dde Author: Thierry Bordaz <tbordaz(a)redhat.com> AuthorDate: Tue Mar 19 14:17:52 2019 +0100 Ticket 49873 - (cont 2nd) Contention on virtual attribute lookup Bug Description: SSL initialization does internal searches that access the vattr_global_lock Thread private counter needs to be initialized by that time. Currently it is initialized after SSL init. Second problem was a leak of one 'int' per worker. It was used to keep the private counter. Fix Description: Call of vattr_global_lock_create needs to be called before slapd_do_all_nss_ssl_init. Also, 'main' may or may not fork, the initialization fo the thread private variable is done either on the child or parent depending if main forks or not. The leak is fixed using a destructor callback of the private variable and so call PR_SetThreadPrivate only if there is no private variable. https://pagure.io/389-ds-base/issue/49873 Reviewed by: Mark Reynolds, Simon Pichugi (thanks) Platforms tested: F28 Flag Day: no Doc impact: no Ticket foo --- ldap/servers/slapd/detach.c | 9 +++++++++ ldap/servers/slapd/main.c | 4 ---- ldap/servers/slapd/vattr.c | 18 ++++++++++++++++-- 3 files changed, 25 insertions(+), 6 deletions(-) diff --git a/ldap/servers/slapd/detach.c b/ldap/servers/slapd/detach.c index 681e6a7..d5c95a0 100644 --- a/ldap/servers/slapd/detach.c +++ b/ldap/servers/slapd/detach.c @@ -144,6 +144,10 @@ detach(int slapd_exemode, int importexport_encrypt, int s_port, daemon_ports_t * } break; } + /* The thread private counter needs to be allocated after the fork + * it is not inherited from parent process + */ + vattr_global_lock_create(); /* call this right after the fork, but before closing stdin */ if (slapd_do_all_nss_ssl_init(slapd_exemode, importexport_encrypt, s_port, ports_info)) { @@ -174,6 +178,11 @@ detach(int slapd_exemode, int importexport_encrypt, int s_port, daemon_ports_t * g_set_detached(1); } else { /* not detaching - call nss/ssl init */ + /* The thread private counter needs to be allocated after the fork + * it is not inherited from parent process + */ + vattr_global_lock_create(); + if (slapd_do_all_nss_ssl_init(slapd_exemode, importexport_encrypt, s_port, ports_info)) { return 1; } diff --git a/ldap/servers/slapd/main.c b/ldap/servers/slapd/main.c index 5a86e2e..185ba90 100644 --- a/ldap/servers/slapd/main.c +++ b/ldap/servers/slapd/main.c @@ -950,10 +950,6 @@ main(int argc, char **argv) return_value = 1; goto cleanup; } - /* The thread private counter needs to be allocated after the fork - * it is not inherited from parent process - */ - vattr_global_lock_create(); /* * Create our thread pool here for tasks to utilise. diff --git a/ldap/servers/slapd/vattr.c b/ldap/servers/slapd/vattr.c index ce63f50..bc4d0e9 100644 --- a/ldap/servers/slapd/vattr.c +++ b/ldap/servers/slapd/vattr.c @@ -125,11 +125,25 @@ vattr_init() vattr_basic_sp_init(); #endif } +/* + * https://developer.mozilla.org/en-US/docs/Mozilla/Projects/NSPR/Reference/PR… + * It is called each time: + * - PR_SetThreadPrivate is call with a not NULL private value + * - on thread exit + */ +static void +vattr_global_lock_free(void *ptr) +{ + int *nb_acquired = ptr; + if (nb_acquired) { + slapi_ch_free((void **)&nb_acquired); + } +} /* Create a private variable for each individual thread of the current process */ void vattr_global_lock_create() { - if (PR_NewThreadPrivateIndex(&thread_private_global_vattr_lock, NULL) != PR_SUCCESS) { + if (PR_NewThreadPrivateIndex(&thread_private_global_vattr_lock, vattr_global_lock_free) != PR_SUCCESS) { slapi_log_err(SLAPI_LOG_ALERT, "vattr_global_lock_create", "Failure to create global lock for virtual attribute !\n"); PR_ASSERT(0); @@ -155,9 +169,9 @@ global_vattr_lock_set_acquired_count(int nb_acquired) if (val == NULL) { /* if it was not initialized set it to zero */ val = (int *) slapi_ch_calloc(1, sizeof(int)); + PR_SetThreadPrivate(thread_private_global_vattr_lock, (void *) val); } *val = nb_acquired; - PR_SetThreadPrivate(thread_private_global_vattr_lock, (void *) val); } /* The map lock can be acquired recursively. So only the first rdlock * will acquire the lock. -- To stop receiving notification emails like this one, please contact the administrator of this repository.

1 0

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

389-commits March 2019