389-commits November 2012

389-commits@lists.fedoraproject.org

5 participants
38 discussions

Branch '389-ds-base-1.2.11' - ldap/schema
by Mark Reynolds 29 Nov '12

29 Nov '12

ldap/schema/02common.ldif | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) New commits: commit 77f126b85db760f0ad8e56575a8bac135b15fe49 Author: Mark Reynolds <mreynolds(a)redhat.com> Date: Thu Nov 29 10:49:33 2012 -0500 Ticket 216 - disable replication agreements Bug Description: Objectclass violation when trying to add "nsds5ReplicaEnabled" to a winSync agmt. Fix Description: Add this attribute to the "MAY" list for objectclass: nsDSWindowsReplicationAgreement https://fedorahosted.org/389/ticket/216 Reviewed by: richm(Thanks) (cherry picked from commit b3a2f4010d4eae7bd62a0c09576b1a643eca3901) diff --git a/ldap/schema/02common.ldif b/ldap/schema/02common.ldif index c6cc6b8..ffec7ce 100644 --- a/ldap/schema/02common.ldif +++ b/ldap/schema/02common.ldif @@ -180,7 +180,7 @@ objectClasses: ( 2.16.840.1.113730.3.2.99 NAME 'cosSuperDefinition' DESC 'Netsca objectClasses: ( 2.16.840.1.113730.3.2.100 NAME 'cosClassicDefinition' DESC 'Netscape defined objectclass' SUP cosSuperDefinition MAY ( cosTemplateDn $ cosspecifier ) X-ORIGIN 'Netscape Directory Server' ) objectClasses: ( 2.16.840.1.113730.3.2.101 NAME 'cosPointerDefinition' DESC 'Netscape defined objectclass' SUP cosSuperDefinition MAY ( cosTemplateDn ) X-ORIGIN 'Netscape Directory Server' ) objectClasses: ( 2.16.840.1.113730.3.2.102 NAME 'cosIndirectDefinition' DESC 'Netscape defined objectclass' SUP cosSuperDefinition MAY ( cosIndirectSpecifier ) X-ORIGIN 'Netscape Directory Server' ) -objectClasses: ( 2.16.840.1.113730.3.2.503 NAME 'nsDSWindowsReplicationAgreement' DESC 'Netscape defined objectclass' SUP top MUST ( cn ) MAY ( nsDS5ReplicaHost $ nsDS5ReplicaPort $ nsDS5ReplicaTransportInfo $ nsDS5ReplicaBindDN $ nsDS5ReplicaCredentials $ nsDS5ReplicaBindMethod $ nsDS5ReplicaRoot $ nsDS5ReplicatedAttributeList $ nsDS5ReplicaUpdateSchedule $ nsds5BeginReplicaRefresh $ description $ nsds50ruv $ nsruvReplicaLastModified $ nsds5ReplicaTimeout $ nsds5replicaChangesSentSinceStartup $ nsds5replicaLastUpdateEnd $ nsds5replicaLastUpdateStart $ nsds5replicaLastUpdateStatus $ nsds5replicaUpdateInProgress $ nsds5replicaLastInitEnd $ nsds5replicaLastInitStart $ nsds5replicaLastInitStatus $ nsds5debugreplicatimeout $ nsds5replicaBusyWaitTime $ nsds5replicaSessionPauseTime $ nsds7WindowsReplicaSubtree $ nsds7DirectoryReplicaSubtree $ nsds7NewWinUserSyncEnabled $ nsds7NewWinGroupSyncEnabled $ nsds7WindowsDomain $ nsds7DirsyncCookie $ winSyncInterval $ oneWaySync $ winSyncM oveAction) X-ORIGIN 'Netscape Directory Server' ) +objectClasses: ( 2.16.840.1.113730.3.2.503 NAME 'nsDSWindowsReplicationAgreement' DESC 'Netscape defined objectclass' SUP top MUST ( cn ) MAY ( nsDS5ReplicaHost $ nsDS5ReplicaPort $ nsDS5ReplicaTransportInfo $ nsDS5ReplicaBindDN $ nsDS5ReplicaCredentials $ nsDS5ReplicaBindMethod $ nsDS5ReplicaRoot $ nsDS5ReplicatedAttributeList $ nsDS5ReplicaUpdateSchedule $ nsds5BeginReplicaRefresh $ description $ nsds50ruv $ nsruvReplicaLastModified $ nsds5ReplicaTimeout $ nsds5replicaChangesSentSinceStartup $ nsds5replicaLastUpdateEnd $ nsds5replicaLastUpdateStart $ nsds5replicaLastUpdateStatus $ nsds5replicaUpdateInProgress $ nsds5replicaLastInitEnd $ nsds5replicaLastInitStart $ nsds5replicaLastInitStatus $ nsds5debugreplicatimeout $ nsds5replicaBusyWaitTime $ nsds5replicaSessionPauseTime $ nsds7WindowsReplicaSubtree $ nsds7DirectoryReplicaSubtree $ nsds7NewWinUserSyncEnabled $ nsds7NewWinGroupSyncEnabled $ nsds7WindowsDomain $ nsds7DirsyncCookie $ winSyncInterval $ oneWaySync $ winSyncM oveAction $ nsds5ReplicaEnabled ) X-ORIGIN 'Netscape Directory Server' ) objectClasses: ( 2.16.840.1.113730.3.2.128 NAME 'costemplate' DESC 'Netscape defined objectclass' SUP top MAY ( cn $ cospriority ) X-ORIGIN 'Netscape Directory Server' ) objectClasses: ( 2.16.840.1.113730.3.2.304 NAME 'nsView' DESC 'Netscape defined objectclass' SUP top AUXILIARY MAY ( nsViewFilter $ description ) X-ORIGIN 'Netscape Directory Server' ) objectClasses: ( 2.16.840.1.113730.3.2.316 NAME 'nsAttributeEncryption' DESC 'Netscape defined objectclass' SUP top MUST ( cn $ nsEncryptionAlgorithm ) X-ORIGIN 'Netscape Directory Server' )

1 0

ldap/schema
by Mark Reynolds 29 Nov '12

29 Nov '12

ldap/schema/02common.ldif | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) New commits: commit b3a2f4010d4eae7bd62a0c09576b1a643eca3901 Author: Mark Reynolds <mreynolds(a)redhat.com> Date: Thu Nov 29 10:49:33 2012 -0500 Ticket 216 - disable replication agreements Bug Description: Objectclass violation when trying to add "nsds5ReplicaEnabled" to a winSync agmt. Fix Description: Add this attribute to the "MAY" list for objectclass: nsDSWindowsReplicationAgreement https://fedorahosted.org/389/ticket/216 Reviewed by: richm(Thanks) diff --git a/ldap/schema/02common.ldif b/ldap/schema/02common.ldif index 7ebaa1e..c0ac751 100644 --- a/ldap/schema/02common.ldif +++ b/ldap/schema/02common.ldif @@ -180,7 +180,7 @@ objectClasses: ( 2.16.840.1.113730.3.2.99 NAME 'cosSuperDefinition' DESC 'Netsca objectClasses: ( 2.16.840.1.113730.3.2.100 NAME 'cosClassicDefinition' DESC 'Netscape defined objectclass' SUP cosSuperDefinition MAY ( cosTemplateDn $ cosspecifier ) X-ORIGIN 'Netscape Directory Server' ) objectClasses: ( 2.16.840.1.113730.3.2.101 NAME 'cosPointerDefinition' DESC 'Netscape defined objectclass' SUP cosSuperDefinition MAY ( cosTemplateDn ) X-ORIGIN 'Netscape Directory Server' ) objectClasses: ( 2.16.840.1.113730.3.2.102 NAME 'cosIndirectDefinition' DESC 'Netscape defined objectclass' SUP cosSuperDefinition MAY ( cosIndirectSpecifier ) X-ORIGIN 'Netscape Directory Server' ) -objectClasses: ( 2.16.840.1.113730.3.2.503 NAME 'nsDSWindowsReplicationAgreement' DESC 'Netscape defined objectclass' SUP top MUST ( cn ) MAY ( nsDS5ReplicaHost $ nsDS5ReplicaPort $ nsDS5ReplicaTransportInfo $ nsDS5ReplicaBindDN $ nsDS5ReplicaCredentials $ nsDS5ReplicaBindMethod $ nsDS5ReplicaRoot $ nsDS5ReplicatedAttributeList $ nsDS5ReplicaUpdateSchedule $ nsds5BeginReplicaRefresh $ description $ nsds50ruv $ nsruvReplicaLastModified $ nsds5ReplicaTimeout $ nsds5replicaChangesSentSinceStartup $ nsds5replicaLastUpdateEnd $ nsds5replicaLastUpdateStart $ nsds5replicaLastUpdateStatus $ nsds5replicaUpdateInProgress $ nsds5replicaLastInitEnd $ nsds5replicaLastInitStart $ nsds5replicaLastInitStatus $ nsds5debugreplicatimeout $ nsds5replicaBusyWaitTime $ nsds5replicaSessionPauseTime $ nsds7WindowsReplicaSubtree $ nsds7DirectoryReplicaSubtree $ nsds7NewWinUserSyncEnabled $ nsds7NewWinGroupSyncEnabled $ nsds7WindowsDomain $ nsds7DirsyncCookie $ winSyncInterval $ oneWaySync $ winSyncM oveAction) X-ORIGIN 'Netscape Directory Server' ) +objectClasses: ( 2.16.840.1.113730.3.2.503 NAME 'nsDSWindowsReplicationAgreement' DESC 'Netscape defined objectclass' SUP top MUST ( cn ) MAY ( nsDS5ReplicaHost $ nsDS5ReplicaPort $ nsDS5ReplicaTransportInfo $ nsDS5ReplicaBindDN $ nsDS5ReplicaCredentials $ nsDS5ReplicaBindMethod $ nsDS5ReplicaRoot $ nsDS5ReplicatedAttributeList $ nsDS5ReplicaUpdateSchedule $ nsds5BeginReplicaRefresh $ description $ nsds50ruv $ nsruvReplicaLastModified $ nsds5ReplicaTimeout $ nsds5replicaChangesSentSinceStartup $ nsds5replicaLastUpdateEnd $ nsds5replicaLastUpdateStart $ nsds5replicaLastUpdateStatus $ nsds5replicaUpdateInProgress $ nsds5replicaLastInitEnd $ nsds5replicaLastInitStart $ nsds5replicaLastInitStatus $ nsds5debugreplicatimeout $ nsds5replicaBusyWaitTime $ nsds5replicaSessionPauseTime $ nsds7WindowsReplicaSubtree $ nsds7DirectoryReplicaSubtree $ nsds7NewWinUserSyncEnabled $ nsds7NewWinGroupSyncEnabled $ nsds7WindowsDomain $ nsds7DirsyncCookie $ winSyncInterval $ oneWaySync $ winSyncM oveAction $ nsds5ReplicaEnabled ) X-ORIGIN 'Netscape Directory Server' ) objectClasses: ( 2.16.840.1.113730.3.2.128 NAME 'costemplate' DESC 'Netscape defined objectclass' SUP top MAY ( cn $ cospriority ) X-ORIGIN 'Netscape Directory Server' ) objectClasses: ( 2.16.840.1.113730.3.2.304 NAME 'nsView' DESC 'Netscape defined objectclass' SUP top AUXILIARY MAY ( nsViewFilter $ description ) X-ORIGIN 'Netscape Directory Server' ) objectClasses: ( 2.16.840.1.113730.3.2.316 NAME 'nsAttributeEncryption' DESC 'Netscape defined objectclass' SUP top MUST ( cn $ nsEncryptionAlgorithm ) X-ORIGIN 'Netscape Directory Server' )

1 0

ldap/servers
by Ludwig Krispenz 29 Nov '12

29 Nov '12

ldap/servers/slapd/config.c | 32 ++++----------------- ldap/servers/slapd/dse.c | 60 ++++++++++++++++++++++++++-------------- ldap/servers/slapd/proto-slap.h | 1 3 files changed, 47 insertions(+), 46 deletions(-) New commits: commit b3ca9eec5d50c2ca503582e55b6681b9b3ad6ad3 Author: Ludwig Krispenz <lkrispen(a)redhat.com> Date: Fri Nov 23 11:15:52 2012 +0100 Ticket 518 - dse.ldif is 0 length after server kill or machine kill Bug Description: If a machine is powered off while slapd is running, dse.ldif can have 0 bytes and server doesn't start even if a dse.ldif.tmp or dse.ldif.bak exists Fix Description: I see no way to prevent a 0 byte ldif in case of a machine crash from slapd code, but the server should try all avaialble backup dse.ldif files to be able to start, https://fedorahosted.org/389/ticket/518 Reviewed by: RichM (Thanks) diff --git a/ldap/servers/slapd/config.c b/ldap/servers/slapd/config.c index d97a575..3edc24b 100644 --- a/ldap/servers/slapd/config.c +++ b/ldap/servers/slapd/config.c @@ -165,6 +165,7 @@ slapd_bootstrap_config(const char *configdir) char *buf = 0; char *lastp = 0; char *entrystr = 0; + char tmpfile[MAXPATHLEN+1]; if (NULL == configdir) { slapi_log_error(SLAPI_LOG_FATAL, @@ -173,33 +174,14 @@ slapd_bootstrap_config(const char *configdir) } PR_snprintf(configfile, sizeof(configfile), "%s/%s", configdir, CONFIG_FILENAME); - if ( (rc = PR_GetFileInfo( configfile, &prfinfo )) != PR_SUCCESS ) - { - /* the "real" file does not exist; see if there is a tmpfile */ - char tmpfile[MAXPATHLEN+1]; - slapi_log_error(SLAPI_LOG_FATAL, "config", - "The configuration file %s does not exist\n", configfile); - PR_snprintf(tmpfile, sizeof(tmpfile), "%s/%s.tmp", configdir, + PR_snprintf(tmpfile, sizeof(tmpfile), "%s/%s.tmp", configdir, CONFIG_FILENAME); - if ( PR_GetFileInfo( tmpfile, &prfinfo ) == PR_SUCCESS ) { - rc = PR_Rename(tmpfile, configfile); - if (rc == PR_SUCCESS) { - slapi_log_error(SLAPI_LOG_FATAL, "config", - "The configuration file %s was restored from backup %s\n", - configfile, tmpfile); - } else { - slapi_log_error(SLAPI_LOG_FATAL, "config", - "The configuration file %s was not restored from backup %s, error %d\n", - configfile, tmpfile, rc); - return rc; /* Fail */ - } - } else { - slapi_log_error(SLAPI_LOG_FATAL, "config", - "The backup configuration file %s does not exist, either.\n", - tmpfile); - return rc; /* Fail */ - } + if ( (rc = dse_check_file(configfile, tmpfile)) == 0 ) { + PR_snprintf(tmpfile, sizeof(tmpfile), "%s/%s.bak", configdir, + CONFIG_FILENAME); + rc = dse_check_file(configfile, tmpfile); } + if ( (rc = PR_GetFileInfo( configfile, &prfinfo )) != PR_SUCCESS ) { PRErrorCode prerr = PR_GetError(); diff --git a/ldap/servers/slapd/dse.c b/ldap/servers/slapd/dse.c index 3d2955c..53d8736 100644 --- a/ldap/servers/slapd/dse.c +++ b/ldap/servers/slapd/dse.c @@ -651,6 +651,40 @@ dse_updateNumSubOfParent(struct dse *pdse, const Slapi_DN *child, int op) slapi_sdn_done(&parent); } +/* check if a file is valid, or if a provided backup file can be used. + * there is no way to determine if the file contents is usable, the only + * checks that can be done is that the file exists and that it is not size 0 + */ +int +dse_check_file(char *filename, char *backupname) +{ + int rc= 0; /* Fail */ + PRFileInfo prfinfo; + + if (PR_GetFileInfo( filename, &prfinfo ) == PR_SUCCESS) { + if ( prfinfo.size > 0) + return (1); + else { + rc = PR_Delete (filename); + } + } + + if (backupname) + rc = PR_Rename (backupname, filename); + else + return (0); + + if ( PR_GetFileInfo( filename, &prfinfo ) == PR_SUCCESS && prfinfo.size > 0 ) { + slapi_log_error(SLAPI_LOG_FATAL, "dse", + "The configuration file %s was restored from backup %s\n", filename, backupname); + return (1); + } else { + slapi_log_error(SLAPI_LOG_FATAL, "dse", + "The configuration file %s was not restored from backup %s, error %d\n", + filename, backupname, rc); + return (0); + } +} static int dse_read_one_file(struct dse *pdse, const char *filename, Slapi_PBlock *pb, int primary_file ) @@ -669,27 +703,11 @@ dse_read_one_file(struct dse *pdse, const char *filename, Slapi_PBlock *pb, if ( (NULL != pdse) && (NULL != filename) ) { - if ( (rc = PR_GetFileInfo( filename, &prfinfo )) != PR_SUCCESS ) - { - /* the "real" file does not exist; see if there is a tmpfile */ - if ( pdse->dse_tmpfile && - PR_GetFileInfo( pdse->dse_tmpfile, &prfinfo ) == PR_SUCCESS ) { - rc = PR_Rename(pdse->dse_tmpfile, filename); - if (rc == PR_SUCCESS) { - slapi_log_error(SLAPI_LOG_FATAL, "dse", - "The configuration file %s was restored from backup %s\n", - filename, pdse->dse_tmpfile); - rc = 1; - } else { - slapi_log_error(SLAPI_LOG_FATAL, "dse", - "The configuration file %s was not restored from backup %s, error %d\n", - filename, pdse->dse_tmpfile, rc); - rc = 0; - } - } else { - rc = 0; /* fail */ - } - } + /* check if the "real" file exists and cam be used, if not try tmp as backup */ + rc = dse_check_file(filename, pdse->dse_tmpfile); + if (!rc) + rc = dse_check_file(filename, pdse->dse_fileback); + if ( (rc = PR_GetFileInfo( filename, &prfinfo )) != PR_SUCCESS ) { slapi_log_error(SLAPI_LOG_FATAL, "dse", diff --git a/ldap/servers/slapd/proto-slap.h b/ldap/servers/slapd/proto-slap.h index d206c6d..a17f40d 100644 --- a/ldap/servers/slapd/proto-slap.h +++ b/ldap/servers/slapd/proto-slap.h @@ -661,6 +661,7 @@ struct dse *dse_new( char *filename, char *tmpfilename, char *backfilename, char struct dse *dse_new_with_filelist(char *filename, char *tmpfilename, char *backfilename, char *startokfilename, const char *configdir, char **filelist); int dse_deletedse(Slapi_PBlock *pb); int dse_destroy(struct dse *pdse); +int dse_check_file(char *filename, char *backupname); int dse_read_file(struct dse *pdse, Slapi_PBlock *pb); int dse_bind( Slapi_PBlock *pb ); int dse_unbind( Slapi_PBlock *pb );

1 0

ldap/servers
by Mark Reynolds 27 Nov '12

27 Nov '12

ldap/servers/plugins/replication/repl5.h | 2 + ldap/servers/plugins/replication/repl5_agmt.c | 10 ++++++ ldap/servers/plugins/replication/windows_inc_protocol.c | 23 +++++++--------- ldap/servers/plugins/replication/windows_private.c | 10 +++++- 4 files changed, 31 insertions(+), 14 deletions(-) New commits: commit 8799a86f36c40169f9077d1e836ba1b4f32080d7 Author: Mark Reynolds <mreynolds(a)redhat.com> Date: Tue Nov 27 15:16:45 2012 -0500 Ticket 393 - Change in winSyncInterval does not take immediate effect Bug Description: If you change the sync interval you must restart the DS for it to take effect. Fix Description: When we change certain config settings(like the interval), notify the repl protocol that the agmt has changed. This iputs the incr_run into the START state where we will pick the interval change. We do not want to notify the protocol when updating these attributes: nsds7WindowsReplicaSubtree nsds7DirectoryReplicaSubtree nsds7WindowsDomain Notifying the protocol automatically triggers these attributes to be updated, so we don't want to renotify the protocol and end up in an infinite loop. https://fedorahosted.org/389/ticket/393 Reviewed by: richm(Thanks!) diff --git a/ldap/servers/plugins/replication/repl5.h b/ldap/servers/plugins/replication/repl5.h index 4c57ee5..f1d6ef4 100644 --- a/ldap/servers/plugins/replication/repl5.h +++ b/ldap/servers/plugins/replication/repl5.h @@ -492,6 +492,8 @@ void prot_notify_window_closed (Repl_Protocol *rp); Object *prot_get_replica_object(Repl_Protocol *rp); void prot_replicate_now(Repl_Protocol *rp); +Repl_Protocol *agmt_get_protocol(Repl_Agmt *ra); + /* In repl5_replica.c */ typedef enum { diff --git a/ldap/servers/plugins/replication/repl5_agmt.c b/ldap/servers/plugins/replication/repl5_agmt.c index 935fb37..da68e16 100644 --- a/ldap/servers/plugins/replication/repl5_agmt.c +++ b/ldap/servers/plugins/replication/repl5_agmt.c @@ -2497,6 +2497,16 @@ void* agmt_get_connection(Repl_Agmt *ra) } } +Repl_Protocol * +agmt_get_protocol(Repl_Agmt *ra) +{ + if(ra){ + return ra->protocol; + } else { + return NULL; + } +} + int agmt_has_protocol(Repl_Agmt *agmt) { diff --git a/ldap/servers/plugins/replication/windows_inc_protocol.c b/ldap/servers/plugins/replication/windows_inc_protocol.c index b5fcadc..20a1c7a 100644 --- a/ldap/servers/plugins/replication/windows_inc_protocol.c +++ b/ldap/servers/plugins/replication/windows_inc_protocol.c @@ -147,7 +147,7 @@ static const char* event2name (int event); static const char* acquire2name (int code); static void periodic_dirsync(time_t when, void *arg); -static Slapi_Eq_Context dirsync; +static Slapi_Eq_Context dirsync = NULL; /* * It's specifically ok to delete a protocol instance that * is currently running. The instance will be shut down, and @@ -292,8 +292,8 @@ windows_inc_run(Private_Repl_Protocol *prp) PRBool use_busy_backoff_timer = PR_FALSE; long pausetime = 0; long busywaittime = 0; - /* Some operations should only be done the first time STATE_START is true. */ - static PRBool is_first_start = PR_TRUE; + unsigned long current_interval = 0; + unsigned long interval = 0; int one_way; PRBool run_dirsync = PR_FALSE; @@ -357,17 +357,16 @@ windows_inc_run(Private_Repl_Protocol *prp) w_set_pause_and_busy_time(&pausetime, &busywaittime); } - - if (is_first_start) { - unsigned long interval = windows_private_get_sync_interval(prp->agmt) * 1000; - /* - * The function, the arguments, the time (hence) when it is first to be called, - * and the repeat interval. - */ - /* DBDB: we should probably make this polling interval configurable */ + /* Check if the interval changed */ + interval = windows_private_get_sync_interval(prp->agmt) * 1000; + if(interval != current_interval){ + current_interval = interval; + if(dirsync){ + slapi_eq_cancel(dirsync); + } dirsync = slapi_eq_repeat(periodic_dirsync, (void*) prp, (time_t)0 , interval); - is_first_start = PR_FALSE; } + break; case STATE_WAIT_WINDOW_OPEN: diff --git a/ldap/servers/plugins/replication/windows_private.c b/ldap/servers/plugins/replication/windows_private.c index cc007aa..6b509a1 100644 --- a/ldap/servers/plugins/replication/windows_private.c +++ b/ldap/servers/plugins/replication/windows_private.c @@ -180,8 +180,9 @@ windows_parse_config_entry(Repl_Agmt *ra, const char *type, Slapi_Entry *e) { windows_private_set_create_users(ra, PR_FALSE); } - retval = 1; slapi_ch_free((void**)&tmpstr); + prot_notify_agmt_changed(agmt_get_protocol(ra), (char *)agmt_get_long_name(ra)); + retval = 1; } if (type == NULL || slapi_attr_types_equivalent(type,type_nsds7CreateNewGroups)) { @@ -194,8 +195,9 @@ windows_parse_config_entry(Repl_Agmt *ra, const char *type, Slapi_Entry *e) { windows_private_set_create_groups(ra, PR_FALSE); } - retval = 1; slapi_ch_free((void**)&tmpstr); + prot_notify_agmt_changed(agmt_get_protocol(ra), (char *)agmt_get_long_name(ra)); + retval = 1; } if (type == NULL || slapi_attr_types_equivalent(type,type_nsds7WindowsDomain)) { @@ -214,6 +216,7 @@ windows_parse_config_entry(Repl_Agmt *ra, const char *type, Slapi_Entry *e) if (NULL != tmpstr) { windows_private_set_sync_interval(ra,tmpstr); + prot_notify_agmt_changed(agmt_get_protocol(ra), (char *)agmt_get_long_name(ra)); } slapi_ch_free_string(&tmpstr); retval = 1; @@ -240,6 +243,7 @@ windows_parse_config_entry(Repl_Agmt *ra, const char *type, Slapi_Entry *e) windows_private_set_one_way(ra, ONE_WAY_SYNC_DISABLED); } slapi_ch_free((void**)&tmpstr); + prot_notify_agmt_changed(agmt_get_protocol(ra), (char *)agmt_get_long_name(ra)); retval = 1; } if (type == NULL || slapi_attr_types_equivalent(type,type_winsyncMoveAction)) @@ -266,8 +270,10 @@ windows_parse_config_entry(Repl_Agmt *ra, const char *type, Slapi_Entry *e) windows_private_set_move_action(ra, MOVE_DOES_NOTHING); } slapi_ch_free((void**)&tmpstr); + prot_notify_agmt_changed(agmt_get_protocol(ra), (char *)agmt_get_long_name(ra)); retval = 1; } + return retval; }

1 0

Branch '389-ds-base-1.2.11' - ldap/servers
by Mark Reynolds 27 Nov '12

27 Nov '12

ldap/servers/plugins/automember/automember.c | 44 +++++++++------ ldap/servers/slapd/errormap.c | 10 +++ ldap/servers/slapd/slapi-plugin.h | 75 +++++++++++++++++++++++++++ 3 files changed, 111 insertions(+), 18 deletions(-) New commits: commit bf2bfaab8e61f335442e4341ff33521a822bd830 Author: Mark Reynolds <mreynolds(a)redhat.com> Date: Tue Nov 27 10:24:02 2012 -0500 Ticket 20 - Allow automember to work on entries that have already been added Bug Description: If the server can not open a ldif for reading(mapping task), an incorrect error and file name is returned. Fix Description: Report the correct file name, and correctly grab the OS error/string. Also made slapd_pr_strerr() and slapd_system_strerr() public, so I refactored the function names to be "slapi_" - so a lot of files are touched but the main change for this ticket is still in automember.c https://fedorahosted.org/389/ticket/20 Reviewed by: richm(Thanks) diff --git a/ldap/servers/plugins/automember/automember.c b/ldap/servers/plugins/automember/automember.c index a11909f..424b507 100644 --- a/ldap/servers/plugins/automember/automember.c +++ b/ldap/servers/plugins/automember/automember.c @@ -2333,6 +2333,7 @@ void automember_export_task_thread(void *arg){ task_data *td = NULL; PRFileDesc *ldif_fd; int i = 0; + int rc = 0; td = (task_data *)slapi_task_get_data(task); slapi_task_begin(task, 1); @@ -2341,13 +2342,14 @@ void automember_export_task_thread(void *arg){ /* make sure we can open the ldif file */ if (( ldif_fd = PR_Open( td->ldif_out, PR_CREATE_FILE | PR_WRONLY, DEFAULT_FILE_MODE )) == NULL ){ - slapi_task_log_notice(task, "Automember export task could not open ldif file \"%s\" for writing %d\n", - td->ldif_out, PR_GetError() ); - slapi_task_log_status(task, "Automember export task could not open ldif file \"%s\" for writing %d\n", - td->ldif_out, PR_GetError() ); + rc = PR_GetOSError(); + slapi_task_log_notice(task, "Automember export task could not open ldif file \"%s\" for writing, error %d (%s)\n", + td->ldif_out, rc, slapi_system_strerror(rc)); + slapi_task_log_status(task, "Automember export task could not open ldif file \"%s\" for writing, error %d (%s)\n", + td->ldif_out, rc, slapi_system_strerror(rc) ); slapi_log_error( SLAPI_LOG_FATAL, AUTOMEMBER_PLUGIN_SUBSYSTEM, - "Could not open ldif file \"%s\" for writing %d\n", - td->ldif_out, PR_GetError() ); + "Could not open ldif file \"%s\" for writing, error %d (%s)\n", + td->ldif_out, rc, slapi_system_strerror(rc) ); result = SLAPI_DSE_CALLBACK_ERROR; goto out; } @@ -2516,6 +2518,7 @@ void automember_map_task_thread(void *arg){ task_data *td = NULL; PRFileDesc *ldif_fd_out = NULL; char *entrystr = NULL; + char *errstr = NULL; #if defined(USE_OPENLDAP) int buflen = 0; LDIFFP *ldif_fd_in = NULL; @@ -2534,29 +2537,34 @@ void automember_map_task_thread(void *arg){ /* make sure we can open the ldif files */ if(( ldif_fd_out = PR_Open( td->ldif_out, PR_CREATE_FILE | PR_WRONLY, DEFAULT_FILE_MODE )) == NULL ){ - slapi_task_log_notice(task, "The ldif file %s could not be accessed, error %d. Aborting task.\n", - td->ldif_out, rc); - slapi_task_log_status(task, "The ldif file %s could not be accessed, error %d. Aborting task.\n", - td->ldif_out, rc); + rc = PR_GetOSError(); + slapi_task_log_notice(task, "The ldif file %s could not be accessed, error %d (%s). Aborting task.\n", + td->ldif_out, rc, slapi_system_strerror(rc)); + slapi_task_log_status(task, "The ldif file %s could not be accessed, error %d (%s). Aborting task.\n", + td->ldif_out, rc, slapi_system_strerror(rc)); slapi_log_error( SLAPI_LOG_FATAL, AUTOMEMBER_PLUGIN_SUBSYSTEM, - "Could not open ldif file \"%s\" for writing %d\n", - td->ldif_out, PR_GetError() ); + "Could not open ldif file \"%s\" for writing, error %d (%s)\n", + td->ldif_out, rc, slapi_system_strerror(rc) ); result = SLAPI_DSE_CALLBACK_ERROR; goto out; } #if defined(USE_OPENLDAP) if(( ldif_fd_in = ldif_open(td->ldif_in, "r")) == NULL ){ + rc = errno; + errstr = strerror(rc); #else if(( ldif_fd_in = PR_Open( td->ldif_in, PR_RDONLY, DEFAULT_FILE_MODE )) == NULL ){ + rc = PR_GetOSError(); + errstr = slapi_system_strerror(rc); #endif - slapi_task_log_notice(task, "The ldif file %s could not be accessed, error %d. Aborting task.\n", - td->ldif_in, rc); - slapi_task_log_status(task, "The ldif file %s could not be accessed, error %d. Aborting task.\n", - td->ldif_in, rc); + slapi_task_log_notice(task, "The ldif file %s could not be accessed, error %d (%s). Aborting task.\n", + td->ldif_in, rc, errstr); + slapi_task_log_status(task, "The ldif file %s could not be accessed, error %d (%s). Aborting task.\n", + td->ldif_in, rc, errstr); slapi_log_error( SLAPI_LOG_FATAL, AUTOMEMBER_PLUGIN_SUBSYSTEM, - "Could not open ldif file \"%s\" for reading %d\n", - td->ldif_out, PR_GetError() ); + "Could not open ldif file \"%s\" for reading, error %d (%s)\n", + td->ldif_in, rc, errstr ); result = SLAPI_DSE_CALLBACK_ERROR; goto out; } diff --git a/ldap/servers/slapd/errormap.c b/ldap/servers/slapd/errormap.c index d05e495..c585d25 100644 --- a/ldap/servers/slapd/errormap.c +++ b/ldap/servers/slapd/errormap.c @@ -73,6 +73,11 @@ slapd_pr_strerror( const int prerrno ) return( s ); } +char * +slapi_pr_strerror( const int prerrno ) +{ + return slapd_pr_strerror(prerrno); +} /* * return the string equivalent of a system error @@ -92,6 +97,11 @@ slapd_system_strerror( const int syserrno ) return( s ); } +const char * +slapi_system_strerror( const int syserrno ) +{ + return slapd_system_strerror(syserrno); +} /* * return the string equivalent of an NSPR error. If "prerrno" is not diff --git a/ldap/servers/slapd/slapi-plugin.h b/ldap/servers/slapd/slapi-plugin.h index 4316833..4962b2a 100644 --- a/ldap/servers/slapd/slapi-plugin.h +++ b/ldap/servers/slapd/slapi-plugin.h @@ -65,6 +65,7 @@ extern "C" { #include "prtypes.h" #include "ldap.h" #include "prprf.h" +#include "nspr.h" NSPR_API(PRUint32) PR_snprintf(char *out, PRUint32 outlen, const char *fmt, ...) #ifdef __GNUC__ __attribute__ ((format (printf, 3, 4))); @@ -7156,6 +7157,80 @@ uint64_t slapi_str_to_u64(const char *s); void slapi_set_plugin_open_rootdn_bind(Slapi_PBlock *pb); +/* + * Public entry extension getter/setter functions + * + * Currently, only slapi_pw_get/set_entry_ext is implemented. + * The functions are in pw.c. Detailed usage of the factory + * is found in the comments at the top of factory.c. + * + * When you plan to add other entry extension code AND + * the type-value pair is managed via ordinary mod, + * setter, getter and copy function having the same API + * are supposed to be implemented, then add the set to + * attrs_in_extension list in entry.c. The set is called + * in slapi_entry_apply_mod_extension. + * + * Note: setter and getter are public, but copy function + * is not. (for the copy function, see pw_copy_entry_ext in pw.c) + */ +/* operation used in the entry extension setter */ +#define SLAPI_EXT_SET_ADD 0 +#define SLAPI_EXT_SET_REPLACE 1 + +/** + * Get entry extension + * + * \param entry is the entry to retrieve the extension from + * \param vals is the array of (Slapi_Value *), which directly refers the extension. Caller must duplicate it to use it for other than referring. + * + * \return LDAP_SUCCESS if successful. + * \return non-zero otherwise. + */ +int slapi_pw_get_entry_ext(Slapi_Entry *entry, Slapi_Value ***vals); + +/** + * Set entry extension + * + * \param entry is the entry to set the extension to + * \param vals is the array of (Slapi_Value *), which is consumed in slapi_pw_set_ext if the call is successful. + * \param flags: SLAPI_EXT_SET_ADD -- add vals to the existing extension if any. + * SLAPI_EXT_SET_REPLACE -- replace vals with the existing extension if any. + * No difference if there is no extension in the entry. + * + * \return LDAP_SUCCESS if successful. + * \return non-zero otherwise. + */ +int slapi_pw_set_entry_ext(Slapi_Entry *entry, Slapi_Value **vals, int flags); + +/** + * Get stashed clear password. + * If multiple of them are in the extension, the first one is returned. + * + * \param entry is the entry to retrieve the extension from + * + * \return a pointer to the clear password string. Caller is responsible to free the string. + */ +char *slapi_get_first_clear_text_pw(Slapi_Entry *entry); + +/** + * Return the string equivalent of an NSPR error + * * + * \param a NSPR error code + * + * \return a pointer to the error code string. + */ +char *slapi_pr_strerror( const PRErrorCode prerrno ); + +/** + * Return the string equivalent of an OS error + * + * \param a OS error code + * + * \return a pointer to the system error code string. + */ +const char *slapi_system_strerror( const int syserrno ); + #ifdef __cplusplus } #endif

1 0

ldap/servers
by Mark Reynolds 27 Nov '12

27 Nov '12

ldap/servers/plugins/automember/automember.c | 44 +++++++++++++++------------ ldap/servers/slapd/errormap.c | 10 ++++++ ldap/servers/slapd/slapi-plugin.h | 19 +++++++++++ 3 files changed, 55 insertions(+), 18 deletions(-) New commits: commit 2b98d672c11a5e4111e5d22a322f4e8a414ca104 Author: Mark Reynolds <mreynolds(a)redhat.com> Date: Tue Nov 27 10:24:02 2012 -0500 Ticket 20 - Allow automember to work on entries that have already been added Bug Description: If the server can not open a ldif for reading(mapping task), an incorrect error and file name is returned. Fix Description: Report the correct file name, and correctly grab the OS error/string. Also made slapd_pr_strerr() and slapd_system_strerr() public, so I refactored the function names to be "slapi_" - so a lot of files are touched but the main change for this ticket is still in automember.c https://fedorahosted.org/389/ticket/20 Reviewed by: richm(Thanks) diff --git a/ldap/servers/plugins/automember/automember.c b/ldap/servers/plugins/automember/automember.c index a11909f..424b507 100644 --- a/ldap/servers/plugins/automember/automember.c +++ b/ldap/servers/plugins/automember/automember.c @@ -2333,6 +2333,7 @@ void automember_export_task_thread(void *arg){ task_data *td = NULL; PRFileDesc *ldif_fd; int i = 0; + int rc = 0; td = (task_data *)slapi_task_get_data(task); slapi_task_begin(task, 1); @@ -2341,13 +2342,14 @@ void automember_export_task_thread(void *arg){ /* make sure we can open the ldif file */ if (( ldif_fd = PR_Open( td->ldif_out, PR_CREATE_FILE | PR_WRONLY, DEFAULT_FILE_MODE )) == NULL ){ - slapi_task_log_notice(task, "Automember export task could not open ldif file \"%s\" for writing %d\n", - td->ldif_out, PR_GetError() ); - slapi_task_log_status(task, "Automember export task could not open ldif file \"%s\" for writing %d\n", - td->ldif_out, PR_GetError() ); + rc = PR_GetOSError(); + slapi_task_log_notice(task, "Automember export task could not open ldif file \"%s\" for writing, error %d (%s)\n", + td->ldif_out, rc, slapi_system_strerror(rc)); + slapi_task_log_status(task, "Automember export task could not open ldif file \"%s\" for writing, error %d (%s)\n", + td->ldif_out, rc, slapi_system_strerror(rc) ); slapi_log_error( SLAPI_LOG_FATAL, AUTOMEMBER_PLUGIN_SUBSYSTEM, - "Could not open ldif file \"%s\" for writing %d\n", - td->ldif_out, PR_GetError() ); + "Could not open ldif file \"%s\" for writing, error %d (%s)\n", + td->ldif_out, rc, slapi_system_strerror(rc) ); result = SLAPI_DSE_CALLBACK_ERROR; goto out; } @@ -2516,6 +2518,7 @@ void automember_map_task_thread(void *arg){ task_data *td = NULL; PRFileDesc *ldif_fd_out = NULL; char *entrystr = NULL; + char *errstr = NULL; #if defined(USE_OPENLDAP) int buflen = 0; LDIFFP *ldif_fd_in = NULL; @@ -2534,29 +2537,34 @@ void automember_map_task_thread(void *arg){ /* make sure we can open the ldif files */ if(( ldif_fd_out = PR_Open( td->ldif_out, PR_CREATE_FILE | PR_WRONLY, DEFAULT_FILE_MODE )) == NULL ){ - slapi_task_log_notice(task, "The ldif file %s could not be accessed, error %d. Aborting task.\n", - td->ldif_out, rc); - slapi_task_log_status(task, "The ldif file %s could not be accessed, error %d. Aborting task.\n", - td->ldif_out, rc); + rc = PR_GetOSError(); + slapi_task_log_notice(task, "The ldif file %s could not be accessed, error %d (%s). Aborting task.\n", + td->ldif_out, rc, slapi_system_strerror(rc)); + slapi_task_log_status(task, "The ldif file %s could not be accessed, error %d (%s). Aborting task.\n", + td->ldif_out, rc, slapi_system_strerror(rc)); slapi_log_error( SLAPI_LOG_FATAL, AUTOMEMBER_PLUGIN_SUBSYSTEM, - "Could not open ldif file \"%s\" for writing %d\n", - td->ldif_out, PR_GetError() ); + "Could not open ldif file \"%s\" for writing, error %d (%s)\n", + td->ldif_out, rc, slapi_system_strerror(rc) ); result = SLAPI_DSE_CALLBACK_ERROR; goto out; } #if defined(USE_OPENLDAP) if(( ldif_fd_in = ldif_open(td->ldif_in, "r")) == NULL ){ + rc = errno; + errstr = strerror(rc); #else if(( ldif_fd_in = PR_Open( td->ldif_in, PR_RDONLY, DEFAULT_FILE_MODE )) == NULL ){ + rc = PR_GetOSError(); + errstr = slapi_system_strerror(rc); #endif - slapi_task_log_notice(task, "The ldif file %s could not be accessed, error %d. Aborting task.\n", - td->ldif_in, rc); - slapi_task_log_status(task, "The ldif file %s could not be accessed, error %d. Aborting task.\n", - td->ldif_in, rc); + slapi_task_log_notice(task, "The ldif file %s could not be accessed, error %d (%s). Aborting task.\n", + td->ldif_in, rc, errstr); + slapi_task_log_status(task, "The ldif file %s could not be accessed, error %d (%s). Aborting task.\n", + td->ldif_in, rc, errstr); slapi_log_error( SLAPI_LOG_FATAL, AUTOMEMBER_PLUGIN_SUBSYSTEM, - "Could not open ldif file \"%s\" for reading %d\n", - td->ldif_out, PR_GetError() ); + "Could not open ldif file \"%s\" for reading, error %d (%s)\n", + td->ldif_in, rc, errstr ); result = SLAPI_DSE_CALLBACK_ERROR; goto out; } diff --git a/ldap/servers/slapd/errormap.c b/ldap/servers/slapd/errormap.c index d05e495..c585d25 100644 --- a/ldap/servers/slapd/errormap.c +++ b/ldap/servers/slapd/errormap.c @@ -73,6 +73,11 @@ slapd_pr_strerror( const int prerrno ) return( s ); } +char * +slapi_pr_strerror( const int prerrno ) +{ + return slapd_pr_strerror(prerrno); +} /* * return the string equivalent of a system error @@ -92,6 +97,11 @@ slapd_system_strerror( const int syserrno ) return( s ); } +const char * +slapi_system_strerror( const int syserrno ) +{ + return slapd_system_strerror(syserrno); +} /* * return the string equivalent of an NSPR error. If "prerrno" is not diff --git a/ldap/servers/slapd/slapi-plugin.h b/ldap/servers/slapd/slapi-plugin.h index 978b02c..0e7220b 100644 --- a/ldap/servers/slapd/slapi-plugin.h +++ b/ldap/servers/slapd/slapi-plugin.h @@ -65,6 +65,7 @@ extern "C" { #include "prtypes.h" #include "ldap.h" #include "prprf.h" +#include "nspr.h" NSPR_API(PRUint32) PR_snprintf(char *out, PRUint32 outlen, const char *fmt, ...) #ifdef __GNUC__ __attribute__ ((format (printf, 3, 4))); @@ -7303,6 +7304,24 @@ int slapi_pw_set_entry_ext(Slapi_Entry *entry, Slapi_Value **vals, int flags); */ char *slapi_get_first_clear_text_pw(Slapi_Entry *entry); +/** + * Return the string equivalent of an NSPR error + * * + * \param a NSPR error code + * + * \return a pointer to the error code string. + */ +char *slapi_pr_strerror( const PRErrorCode prerrno ); + +/** + * Return the string equivalent of an OS error + * + * \param a OS error code + * + * \return a pointer to the system error code string. + */ +const char *slapi_system_strerror( const int syserrno ); + #ifdef __cplusplus } #endif

1 0

Branch '389-ds-base-1.2.11' - ldap/servers
by Mark Reynolds 26 Nov '12

26 Nov '12

ldap/servers/plugins/replication/cl5_api.c | 3 --- ldap/servers/plugins/replication/repl5_replica_config.c | 6 ++++-- 2 files changed, 4 insertions(+), 5 deletions(-) New commits: commit 7d22bc2eca8d71ac212879fb28d39c8a31a7be58 Author: Mark Reynolds <mreynolds(a)redhat.com> Date: Mon Nov 26 11:04:36 2012 -0500 Coverity Fixes 12626 13030 13114 13115 13116 Reviewed by: richm (Thanks Rich!) (cherry picked from commit 4850b2720a6d2a1cf65b2cbfa296e37f04f85c5d) diff --git a/ldap/servers/plugins/replication/cl5_api.c b/ldap/servers/plugins/replication/cl5_api.c index 6c94b3d..175eb80 100644 --- a/ldap/servers/plugins/replication/cl5_api.c +++ b/ldap/servers/plugins/replication/cl5_api.c @@ -6554,9 +6554,6 @@ cl5CleanRUV(ReplicaId rid){ ruv_delete_replica(file->maxRUV, rid); obj = objset_next_obj(s_cl5Desc.dbFiles, obj); } - if(obj){ - object_release (obj); - } slapi_rwlock_unlock (s_cl5Desc.stLock); } diff --git a/ldap/servers/plugins/replication/repl5_replica_config.c b/ldap/servers/plugins/replication/repl5_replica_config.c index 0fd785b..e234c15 100644 --- a/ldap/servers/plugins/replication/repl5_replica_config.c +++ b/ldap/servers/plugins/replication/repl5_replica_config.c @@ -1743,7 +1743,7 @@ check_replicas_are_done_cleaning(cleanruv_data *data ) { Object *agmt_obj; Repl_Agmt *agmt; - char csnstr[CSN_STRSIZE]; + char *csnstr = NULL; char *filter = NULL; int not_all_cleaned = 1; int interval = 10; @@ -1786,6 +1786,7 @@ check_replicas_are_done_cleaning(cleanruv_data *data ) interval = 14400; } } + slapi_ch_free_string(&csnstr); slapi_ch_free_string(&filter); } @@ -2356,7 +2357,7 @@ delete_cleaned_rid_config(cleanruv_data *clean_data) struct berval *vals[2]; struct berval val; char data[CSN_STRSIZE + 15]; - char csnstr[CSN_STRSIZE]; + char *csnstr = NULL; char *dn; int rc; @@ -2399,6 +2400,7 @@ delete_cleaned_rid_config(cleanruv_data *clean_data) } slapi_pblock_destroy (pb); slapi_ch_free_string(&dn); + slapi_ch_free_string(&csnstr); } /*

1 0

ldap/servers
by Mark Reynolds 26 Nov '12

26 Nov '12

ldap/servers/plugins/replication/cl5_api.c | 3 --- ldap/servers/plugins/replication/repl5_replica_config.c | 6 ++++-- 2 files changed, 4 insertions(+), 5 deletions(-) New commits: commit 4850b2720a6d2a1cf65b2cbfa296e37f04f85c5d Author: Mark Reynolds <mreynolds(a)redhat.com> Date: Mon Nov 26 11:04:36 2012 -0500 Coverity Fixes 12626 13030 13114 13115 13116 Reviewed by: richm (Thanks Rich!) diff --git a/ldap/servers/plugins/replication/cl5_api.c b/ldap/servers/plugins/replication/cl5_api.c index 6c94b3d..175eb80 100644 --- a/ldap/servers/plugins/replication/cl5_api.c +++ b/ldap/servers/plugins/replication/cl5_api.c @@ -6554,9 +6554,6 @@ cl5CleanRUV(ReplicaId rid){ ruv_delete_replica(file->maxRUV, rid); obj = objset_next_obj(s_cl5Desc.dbFiles, obj); } - if(obj){ - object_release (obj); - } slapi_rwlock_unlock (s_cl5Desc.stLock); } diff --git a/ldap/servers/plugins/replication/repl5_replica_config.c b/ldap/servers/plugins/replication/repl5_replica_config.c index f1baef2..aef9c13 100644 --- a/ldap/servers/plugins/replication/repl5_replica_config.c +++ b/ldap/servers/plugins/replication/repl5_replica_config.c @@ -1743,7 +1743,7 @@ check_replicas_are_done_cleaning(cleanruv_data *data ) { Object *agmt_obj; Repl_Agmt *agmt; - char csnstr[CSN_STRSIZE]; + char *csnstr = NULL; char *filter = NULL; int not_all_cleaned = 1; int interval = 10; @@ -1786,6 +1786,7 @@ check_replicas_are_done_cleaning(cleanruv_data *data ) interval = 14400; } } + slapi_ch_free_string(&csnstr); slapi_ch_free_string(&filter); } @@ -2356,7 +2357,7 @@ delete_cleaned_rid_config(cleanruv_data *clean_data) struct berval *vals[2]; struct berval val; char data[CSN_STRSIZE + 15]; - char csnstr[CSN_STRSIZE]; + char *csnstr = NULL; char *dn; int rc; @@ -2399,6 +2400,7 @@ delete_cleaned_rid_config(cleanruv_data *clean_data) } slapi_pblock_destroy (pb); slapi_ch_free_string(&dn); + slapi_ch_free_string(&csnstr); } /*

1 0

ldap/schema
by Ludwig Krispenz 22 Nov '12

22 Nov '12

ldap/schema/02common.ldif | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) New commits: commit da4e8686e530757e6c1005588e5e0c4e185f12e1 Author: Ludwig Krispenz <lkrispen(a)redhat.com> Date: Wed Nov 21 10:08:45 2012 +0100 Ticket 349 - nsViewFilter syntax issue in 389DS 1.2.5 Bug Description: current nsViewFilter syntax is IA5string, this is too restictive for definition of views in DIT using international chareacters Fix Description: change syntax of nsViewFilter to directory string NOTE: there could be deployments relying on the case sensitivity of IA5string although not very likely. Probably need to doc in RN https://fedorahosted.org/389/ticket/349 Reviewed by: ? diff --git a/ldap/schema/02common.ldif b/ldap/schema/02common.ldif index c6cc6b8..7ebaa1e 100644 --- a/ldap/schema/02common.ldif +++ b/ldap/schema/02common.ldif @@ -148,7 +148,7 @@ attributeTypes: ( 2.16.840.1.113730.3.1.1100 NAME 'oneWaySync' DESC 'Netscape de attributeTypes: ( 2.16.840.1.113730.3.1.2139 NAME 'winSyncMoveAction' DESC 'Netscape defined attribute type' SYNTAX 1.3.6.1.4.1.1466.115.121.1.15 SINGLE-VALUE X-ORIGIN 'Netscape Directory Server' ) attributeTypes: ( 1.3.6.1.1.4 NAME 'vendorName' EQUALITY 1.3.6.1.4.1.1466.109.114.1 SYNTAX 1.3.6.1.4.1.1466.115.121.1.15 SINGLE-VALUE NO-USER-MODIFICATION USAGE dSAOperation X-ORIGIN 'RFC 3045' ) attributeTypes: ( 1.3.6.1.1.5 NAME 'vendorVersion' EQUALITY 1.3.6.1.4.1.1466.109.114.1 SYNTAX 1.3.6.1.4.1.1466.115.121.1.15 SINGLE-VALUE NO-USER-MODIFICATION USAGE dSAOperation X-ORIGIN 'RFC 3045' ) -attributeTypes: ( 2.16.840.1.113730.3.1.3023 NAME 'nsViewFilter' DESC 'Netscape defined attribute type' SYNTAX 1.3.6.1.4.1.1466.115.121.1.26 X-ORIGIN 'Netscape Directory Server' ) +attributeTypes: ( 2.16.840.1.113730.3.1.3023 NAME 'nsViewFilter' DESC 'Netscape defined attribute type' SYNTAX 1.3.6.1.4.1.1466.115.121.1.15 X-ORIGIN 'Netscape Directory Server' ) attributeTypes: ( 2.16.840.1.113730.3.1.2063 NAME 'nsEncryptionAlgorithm' DESC 'Netscape defined attribute type' SYNTAX 1.3.6.1.4.1.1466.115.121.1.15 SINGLE-VALUE X-ORIGIN 'Netscape Directory Server' ) attributeTypes: ( 2.16.840.1.113730.3.1.2093 NAME 'nsslapd-changelogsuffix' DESC 'Netscape defined attribute type' SYNTAX 1.3.6.1.4.1.1466.115.121.1.12 X-ORIGIN 'Netscape' ) attributeTypes: ( 2.16.840.1.113730.3.1.2094 NAME 'nsslapd-parent-suffix' DESC 'Netscape defined attribute type' SYNTAX 1.3.6.1.4.1.1466.115.121.1.12 X-ORIGIN 'Netscape' )

1 0

Branch '389-ds-base-1.2.11' - ldap/servers
by Mark Reynolds 21 Nov '12

21 Nov '12

ldap/servers/plugins/replication/cl5_api.c | 21 ldap/servers/plugins/replication/repl5.h | 29 ldap/servers/plugins/replication/repl5_agmt.c | 99 - ldap/servers/plugins/replication/repl5_agmtlist.c | 11 ldap/servers/plugins/replication/repl5_connection.c | 6 ldap/servers/plugins/replication/repl5_init.c | 69 + ldap/servers/plugins/replication/repl5_replica.c | 132 -- ldap/servers/plugins/replication/repl5_replica_config.c | 1055 ++++++++++------ ldap/servers/plugins/replication/repl_extop.c | 266 +++- ldap/servers/plugins/replication/replutil.c | 47 ldap/servers/slapd/back-ldbm/ldbm_modify.c | 7 ldap/servers/slapd/back-ldbm/misc.c | 2 ldap/servers/slapd/back-ldbm/proto-back-ldbm.h | 1 ldap/servers/slapd/entry.c | 10 ldap/servers/slapd/slapi-private.h | 2 15 files changed, 1104 insertions(+), 653 deletions(-) New commits: commit 7a3014c0746134403f1fe9a25eac9581750aef0b Author: Mark Reynolds <mreynolds(a)redhat.com> Date: Tue Nov 20 16:14:42 2012 -0500 Ticket 337 - improve CLEANRUV functionality Bug Description: Without anonymous access allowed, then task could not contact other servers and retrieve data(ruv, maxcsns, etc). Fix Description: Changed all communication to use extended operations, and improved sychronization between the replicas. https://fedorahosted.org/389/ticket/337 Reviewed by: richm(Thanks Rich!) (cherry picked from commit ecbd8b7513d05dab5876a88ca56b102c08a7a67c) diff --git a/ldap/servers/plugins/replication/cl5_api.c b/ldap/servers/plugins/replication/cl5_api.c index a347c57..6c94b3d 100644 --- a/ldap/servers/plugins/replication/cl5_api.c +++ b/ldap/servers/plugins/replication/cl5_api.c @@ -1474,13 +1474,13 @@ int cl5WriteOperation(const char *replName, const char *replGen, } /* Name: cl5CreateReplayIterator - Description: creates an iterator that allows to retireve changes that should - to be sent to the consumer identified by ruv. The iteration is peformed by + Description: creates an iterator that allows to retrieve changes that should + to be sent to the consumer identified by ruv. The iteration is performed by repeated calls to cl5GetNextOperationToReplay. Parameters: replica - replica whose data we wish to iterate; ruv - consumer ruv; iterator - iterator to be passed to cl5GetNextOperationToReplay call - Return: CL5_SUCCESS, if function is successfull; + Return: CL5_SUCCESS, if function is successful; CL5_MISSING_DATA, if data that should be in the changelog is missing CL5_PURGED_DATA, if some data that consumer needs has been purged. Note that the iterator can be non null if the supplier contains @@ -1488,14 +1488,14 @@ int cl5WriteOperation(const char *replName, const char *replGen, CL5_NOTFOUND if the consumer is up to data with respect to the supplier CL5_BAD_DATA if invalid parameter is passed; CL5_BAD_STATE if db has not been open; - CL5_DB_ERROR if any other db error occured; + CL5_DB_ERROR if any other db error occurred; CL5_MEMORY_ERROR if memory allocation fails. Algorithm: Build a list of csns from consumer's and supplier's ruv. For each element of the consumer's ruv put max csn into the csn list. For each element of the supplier's ruv not in the consumer's ruv put min csn from the supplier's ruv into the list. The list contains, for each known replica, the starting point for changes to be sent to the consumer. - Sort the list in accending order. + Sort the list in ascending order. Build a hash which contains, for each known replica, whether the supplier can bring the consumer up to data with respect to that replica. The hash is used to decide whether a change can be sent to the consumer @@ -1503,7 +1503,7 @@ int cl5WriteOperation(const char *replName, const char *replGen, we can bring the consumer up to date. Position the db cursor on the change entry that corresponds to this csn. Hash entries are created for each replica traversed so far. sendChanges - flag is set to FALSE for all repolicas except the last traversed. + flag is set to FALSE for all replicas except the last traversed. */ int cl5CreateReplayIteratorEx (Private_Repl_Protocol *prp, const RUV *consumerRuv, @@ -6543,7 +6543,9 @@ bail: void cl5CleanRUV(ReplicaId rid){ CL5DBFile *file; - Object *obj; + Object *obj = NULL; + + slapi_rwlock_wrlock (s_cl5Desc.stLock); obj = objset_first_obj(s_cl5Desc.dbFiles); while (obj){ @@ -6552,6 +6554,11 @@ cl5CleanRUV(ReplicaId rid){ ruv_delete_replica(file->maxRUV, rid); obj = objset_next_obj(s_cl5Desc.dbFiles, obj); } + if(obj){ + object_release (obj); + } + + slapi_rwlock_unlock (s_cl5Desc.stLock); } void trigger_cl_trimming(ReplicaId rid){ diff --git a/ldap/servers/plugins/replication/repl5.h b/ldap/servers/plugins/replication/repl5.h index 61b51b0..bd582bc 100644 --- a/ldap/servers/plugins/replication/repl5.h +++ b/ldap/servers/plugins/replication/repl5.h @@ -97,8 +97,14 @@ /* cleanallruv extended ops */ #define REPL_CLEANRUV_OID "2.16.840.1.113730.3.6.5" #define REPL_ABORT_CLEANRUV_OID "2.16.840.1.113730.3.6.6" -#define CLEANRUV_NOTIFIED 0 -#define CLEANRUV_RELEASED 1 +#define REPL_CLEANRUV_GET_MAXCSN_OID "2.16.840.1.113730.3.6.7" +#define REPL_CLEANRUV_CHECK_STATUS_OID "2.16.840.1.113730.3.6.8" + +#define CLEANRUV_ACCEPTED "accepted" +#define CLEANRUV_REJECTED "rejected" +#define CLEANRUV_FINISHED "finished" +#define CLEANRUV_CLEANING "cleaning" +#define CLEANRUV_NO_MAXCSN "no maxcsn" /* DS 5.0 replication protocol error codes */ #define NSDS50_REPL_REPLICA_READY 0x00 /* Replica ready, go ahead */ @@ -230,6 +236,8 @@ int multimaster_extop_StartNSDS50ReplicationRequest(Slapi_PBlock *pb); int multimaster_extop_EndNSDS50ReplicationRequest(Slapi_PBlock *pb); int multimaster_extop_cleanruv(Slapi_PBlock *pb); int multimaster_extop_abort_cleanruv(Slapi_PBlock *pb); +int multimaster_extop_cleanruv_get_maxcsn(Slapi_PBlock *pb); +int multimaster_extop_cleanruv_check_status(Slapi_PBlock *pb); int extop_noop(Slapi_PBlock *pb); struct berval *NSDS50StartReplicationRequest_new(const char *protocol_oid, const char *repl_root, char **extra_referrals, CSN *csn); @@ -365,9 +373,6 @@ PRBool agmt_is_enabled(Repl_Agmt *ra); int agmt_set_enabled_from_entry(Repl_Agmt *ra, Slapi_Entry *e, char *returntext); char **agmt_get_attrs_to_strip(Repl_Agmt *ra); int agmt_set_attrs_to_strip(Repl_Agmt *ra, Slapi_Entry *e); -void agmt_set_cleanruv_notified_from_entry(Repl_Agmt *ra, Slapi_Entry *e); -int agmt_set_cleanruv_data(Repl_Agmt *ra, ReplicaId rid, int op); -int agmt_is_cleanruv_notified(Repl_Agmt *ra, ReplicaId rid); int agmt_set_timeout(Repl_Agmt *ra, long timeout); void agmt_update_done(Repl_Agmt *ra, int is_total); @@ -454,6 +459,7 @@ LDAP * conn_get_ldap(Repl_Connection *conn); void conn_lock(Repl_Connection *conn); void conn_unlock(Repl_Connection *conn); void conn_delete_internal_ext(Repl_Connection *conn); +const char* conn_get_bindmethod(Repl_Connection *conn); /* In repl5_protocol.c */ typedef struct repl_protocol Repl_Protocol; @@ -608,7 +614,7 @@ int replica_config_init(); void replica_config_destroy (); int get_replica_type(Replica *r); int replica_execute_cleanruv_task_ext(Object *r, ReplicaId rid); -void add_cleaned_rid(ReplicaId rid, Replica *r, char *maxcsn); +void add_cleaned_rid(ReplicaId rid, Replica *r, char *maxcsn, char *forcing); int is_cleaned_rid(ReplicaId rid); int replica_cleanall_ruv_abort(Slapi_PBlock *pb, Slapi_Entry *e, Slapi_Entry *eAfter, int *returncode, char *returntext, void *arg); @@ -616,17 +622,15 @@ void replica_cleanallruv_thread_ext(void *arg); void stop_ruv_cleaning(); int task_aborted(); void replica_abort_task_thread(void *arg); -void delete_cleaned_rid(Replica *r, ReplicaId rid, CSN *maxcsn); +void remove_cleaned_rid(ReplicaId rid); int process_repl_agmts(Replica *replica, int *agmt_info, char *oid, Slapi_Task *task, struct berval *payload, int op); int decode_cleanruv_payload(struct berval *extop_value, char **payload); -struct berval *create_ruv_payload(char *value); -void replica_add_cleanruv_data(Replica *r, char *val); -void replica_remove_cleanruv_data(Replica *r, char *val); -CSN *replica_get_cleanruv_maxcsn(Replica *r, ReplicaId rid); +struct berval *create_cleanruv_payload(char *value); void ruv_get_cleaned_rids(RUV *ruv, ReplicaId *rids); void add_aborted_rid(ReplicaId rid, Replica *r, char *repl_root); int is_task_aborted(ReplicaId rid); -void delete_aborted_rid(Replica *replica, ReplicaId rid, char *repl_root); +void delete_aborted_rid(Replica *replica, ReplicaId rid, char *repl_root, int skip); +int is_pre_cleaned_rid(ReplicaId rid); void set_cleaned_rid(ReplicaId rid); void cleanruv_log(Slapi_Task *task, char *task_type, char *fmt, ...); @@ -643,6 +647,7 @@ typedef struct _cleanruv_data char *repl_root; Slapi_DN *sdn; char *certify; + char *force; } cleanruv_data; /* replutil.c */ diff --git a/ldap/servers/plugins/replication/repl5_agmt.c b/ldap/servers/plugins/replication/repl5_agmt.c index d25f54b..935fb37 100644 --- a/ldap/servers/plugins/replication/repl5_agmt.c +++ b/ldap/servers/plugins/replication/repl5_agmt.c @@ -141,7 +141,6 @@ typedef struct repl5agmt { char **attrs_to_strip; /* for fractional replication, if a "mod" is empty, strip out these attributes: * modifiersname, modifytimestamp, internalModifiersname, internalModifyTimestamp, etc */ int agreement_type; - int cleanruv_notified[CLEANRIDSIZ + 1]; /* specifies if the replica has been notified of a CLEANALLRUV task */ } repl5agmt; /* Forward declarations */ @@ -252,7 +251,6 @@ agmt_new_from_entry(Slapi_Entry *e) Repl_Agmt *ra; Slapi_Attr *sattr; char *tmpstr; - char **clean_vals = NULL; char **denied_attrs = NULL; char *auto_initialize = NULL; char *val_nsds5BeginReplicaRefresh = "start"; @@ -431,20 +429,6 @@ agmt_new_from_entry(Slapi_Entry *e) ra->last_init_start_time = 0UL; ra->last_init_status[0] = '\0'; - /* cleanruv notification */ - clean_vals = slapi_entry_attr_get_charray(e, type_nsds5ReplicaCleanRUVnotified); - if(clean_vals){ - int i; - for (i = 0; i < CLEANRIDSIZ && clean_vals[i]; i++){ - ra->cleanruv_notified[i] = atoi(clean_vals[i]); - } - if(i < CLEANRIDSIZ) - ra->cleanruv_notified[i + 1] = 0; - slapi_ch_array_free(clean_vals); - } else { - ra->cleanruv_notified[0] = 0; - } - /* Fractional attributes */ slapi_entry_attr_find(e, type_nsds5ReplicatedAttributeList, &sattr); @@ -2623,89 +2607,6 @@ agmt_set_attrs_to_strip(Repl_Agmt *ra, Slapi_Entry *e) return -1; } -int -agmt_is_cleanruv_notified(Repl_Agmt *ra, ReplicaId rid){ - int notified = 0; - int i; - - PR_Lock(ra->lock); - for(i = 0; i < CLEANRIDSIZ && ra->cleanruv_notified[i]; i++){ - if(ra->cleanruv_notified[i] == rid){ - notified = 1; - break; - } - } - PR_Unlock(ra->lock); - - return notified; -} - -/* - * This will trigger agmt_set_cleanruv_notified_from_entry() to be called, - * which will update the in memory agmt. - * - * op can be: CLEANRUV_NOTIFIED or CLEANRUV_RELEASED - */ -int -agmt_set_cleanruv_data(Repl_Agmt *ra, ReplicaId rid, int op){ - Slapi_PBlock *pb; - LDAPMod *mods[2]; - LDAPMod mod; - struct berval *vals[2]; - struct berval val; - char data[6]; - int rc = 0; - - if(ra == NULL){ - return -1; - } - - if(op == CLEANRUV_NOTIFIED){ - /* add the cleanruv data */ - mod.mod_op = LDAP_MOD_ADD|LDAP_MOD_BVALUES; - } else { - /* remove the cleanruv data */ - mod.mod_op = LDAP_MOD_DELETE|LDAP_MOD_BVALUES; - } - - pb = slapi_pblock_new(); - val.bv_len = PR_snprintf(data, sizeof(data), "%d", (int)rid); - mod.mod_type = (char *)type_nsds5ReplicaCleanRUVnotified; - mod.mod_bvalues = vals; - vals [0] = &val; - vals [1] = NULL; - val.bv_val = data; - mods[0] = &mod; - mods[1] = NULL; - - slapi_modify_internal_set_pb_ext (pb, ra->dn, mods, NULL, NULL, - repl_get_plugin_identity (PLUGIN_MULTIMASTER_REPLICATION), 0); - slapi_modify_internal_pb (pb); - slapi_pblock_destroy(pb); - - return rc; -} - -void -agmt_set_cleanruv_notified_from_entry(Repl_Agmt *ra, Slapi_Entry *e){ - char **attr_vals = NULL; - int i; - - PR_Lock(ra->lock); - attr_vals = slapi_entry_attr_get_charray(e, type_nsds5ReplicaCleanRUVnotified); - if(attr_vals){ - for (i = 0; i < CLEANRIDSIZ && attr_vals[i]; i++){ - ra->cleanruv_notified[i] = atoi(attr_vals[i]); - } - if( i < CLEANRIDSIZ ) - ra->cleanruv_notified[i + 1] = 0; - slapi_ch_array_free(attr_vals); - } else { - ra->cleanruv_notified[0] = 0; - } - PR_Unlock(ra->lock); -} - /* this is called whenever an update (total/incremental) is completed */ void diff --git a/ldap/servers/plugins/replication/repl5_agmtlist.c b/ldap/servers/plugins/replication/repl5_agmtlist.c index 537cfd8..70f71a8 100644 --- a/ldap/servers/plugins/replication/repl5_agmtlist.c +++ b/ldap/servers/plugins/replication/repl5_agmtlist.c @@ -248,7 +248,12 @@ agmtlist_modify_callback(Slapi_PBlock *pb, Slapi_Entry *entryBefore, Slapi_Entry /* we don't allow delete attribute operations unless it was issued by the replication plugin - handled above */ if (mods[i]->mod_op & LDAP_MOD_DELETE) - { + { + if(strcasecmp (mods[i]->mod_type, type_nsds5ReplicaCleanRUVnotified) == 0){ + /* allow the deletion of cleanallruv agmt attr */ + continue; + } + slapi_log_error(SLAPI_LOG_REPL, repl_plugin_name, "agmtlist_modify_callback: " "deletion of %s attribute is not allowed\n", type_nsds5ReplicaInitialize); *returncode = LDAP_UNWILLING_TO_PERFORM; @@ -508,10 +513,6 @@ agmtlist_modify_callback(Slapi_PBlock *pb, Slapi_Entry *entryBefore, Slapi_Entry rc = SLAPI_DSE_CALLBACK_ERROR; } } - else if (slapi_attr_types_equivalent(mods[i]->mod_type, type_nsds5ReplicaCleanRUVnotified)) - { - agmt_set_cleanruv_notified_from_entry(agmt, e); - } else if (0 == windows_handle_modify_agreement(agmt, mods[i]->mod_type, e)) { slapi_log_error(SLAPI_LOG_REPL, repl_plugin_name, "agmtlist_modify_callback: " diff --git a/ldap/servers/plugins/replication/repl5_connection.c b/ldap/servers/plugins/replication/repl5_connection.c index 51a2bc5..5efd0e6 100644 --- a/ldap/servers/plugins/replication/repl5_connection.c +++ b/ldap/servers/plugins/replication/repl5_connection.c @@ -1760,6 +1760,12 @@ bind_method_to_mech(int bindmethod) return LDAP_SASL_SIMPLE; } +const char* +conn_get_bindmethod(Repl_Connection *conn) +{ + return (bind_method_to_mech(conn->bindmethod)); +} + /* * Check the result of an ldap BIND operation to see we it * contains the expiration controls diff --git a/ldap/servers/plugins/replication/repl5_init.c b/ldap/servers/plugins/replication/repl5_init.c index 7a80c6f..a0f02fe 100644 --- a/ldap/servers/plugins/replication/repl5_init.c +++ b/ldap/servers/plugins/replication/repl5_init.c @@ -123,7 +123,7 @@ static char *cleanruv_oid_list[] = { NULL }; static char *cleanruv_name_list[] = { - NSDS_REPL_NAME_PREFIX " Cleanruv", + NSDS_REPL_NAME_PREFIX " CleanAllRUV", NULL }; static char *cleanruv_abort_oid_list[] = { @@ -131,9 +131,26 @@ static char *cleanruv_abort_oid_list[] = { NULL }; static char *cleanruv_abort_name_list[] = { - NSDS_REPL_NAME_PREFIX " Cleanruv Abort", + NSDS_REPL_NAME_PREFIX " CleanAllRUV Abort", NULL }; +static char *cleanruv_maxcsn_oid_list[] = { + REPL_CLEANRUV_GET_MAXCSN_OID, + NULL +}; +static char *cleanruv_maxcsn_name_list[] = { + NSDS_REPL_NAME_PREFIX " CleanAllRUV Retrieve MaxCSN", + NULL +}; +static char *cleanruv_status_oid_list[] = { + REPL_CLEANRUV_CHECK_STATUS_OID, + NULL +}; +static char *cleanruv_status_name_list[] = { + NSDS_REPL_NAME_PREFIX " CleanAllRUV Check Status", + NULL +}; + /* List of plugin identities for every plugin registered. Plugin identity is passed by the server in the plugin init function and must be supplied @@ -403,6 +420,52 @@ multimaster_end_extop_init( Slapi_PBlock *pb ) return rc; } +int +multimaster_cleanruv_maxcsn_extop_init( Slapi_PBlock *pb ) +{ + int rc= 0; /* OK */ + void *identity = NULL; + + /* get plugin identity and store it to pass to internal operations */ + slapi_pblock_get (pb, SLAPI_PLUGIN_IDENTITY, &identity); + PR_ASSERT (identity); + + if (slapi_pblock_set( pb, SLAPI_PLUGIN_VERSION, SLAPI_PLUGIN_VERSION_01 ) != 0 || + slapi_pblock_set( pb, SLAPI_PLUGIN_DESCRIPTION, (void *)&multimasterextopdesc ) != 0 || + slapi_pblock_set( pb, SLAPI_PLUGIN_EXT_OP_OIDLIST, (void *)cleanruv_maxcsn_oid_list ) != 0 || + slapi_pblock_set( pb, SLAPI_PLUGIN_EXT_OP_NAMELIST, (void *)cleanruv_maxcsn_name_list ) != 0 || + slapi_pblock_set( pb, SLAPI_PLUGIN_EXT_OP_FN, (void *)multimaster_extop_cleanruv_get_maxcsn )) + { + slapi_log_error( SLAPI_LOG_PLUGIN, repl_plugin_name, "multimaster_cleanruv_extop_init failed\n" ); + rc= -1; + } + + return rc; +} + +int +multimaster_cleanruv_status_extop_init( Slapi_PBlock *pb ) +{ + int rc= 0; /* OK */ + void *identity = NULL; + + /* get plugin identity and store it to pass to internal operations */ + slapi_pblock_get (pb, SLAPI_PLUGIN_IDENTITY, &identity); + PR_ASSERT (identity); + + if (slapi_pblock_set( pb, SLAPI_PLUGIN_VERSION, SLAPI_PLUGIN_VERSION_01 ) != 0 || + slapi_pblock_set( pb, SLAPI_PLUGIN_DESCRIPTION, (void *)&multimasterextopdesc ) != 0 || + slapi_pblock_set( pb, SLAPI_PLUGIN_EXT_OP_OIDLIST, (void *)cleanruv_status_oid_list ) != 0 || + slapi_pblock_set( pb, SLAPI_PLUGIN_EXT_OP_NAMELIST, (void *)cleanruv_status_name_list ) != 0 || + slapi_pblock_set( pb, SLAPI_PLUGIN_EXT_OP_FN, (void *)multimaster_extop_cleanruv_check_status )) + { + slapi_log_error( SLAPI_LOG_PLUGIN, repl_plugin_name, "multimaster_cleanruv_extop_init failed\n" ); + rc= -1; + } + + return rc; +} + int multimaster_total_extop_init( Slapi_PBlock *pb ) @@ -679,6 +742,8 @@ int replication_multimaster_plugin_init(Slapi_PBlock *pb) rc= slapi_register_plugin("extendedop", 1 /* Enabled */, "multimaster_response_extop_init", multimaster_response_extop_init, "Multimaster replication extended response plugin", NULL, identity); rc= slapi_register_plugin("extendedop", 1 /* Enabled */, "multimaster_cleanruv_extop_init", multimaster_cleanruv_extop_init, "Multimaster replication cleanruv extended operation plugin", NULL, identity); rc= slapi_register_plugin("extendedop", 1 /* Enabled */, "multimaster_cleanruv_abort_extop_init", multimaster_cleanruv_abort_extop_init, "Multimaster replication cleanruv abort extended operation plugin", NULL, identity); + rc= slapi_register_plugin("extendedop", 1 /* Enabled */, "multimaster_cleanruv_maxcsn_extop_init", multimaster_cleanruv_maxcsn_extop_init, "Multimaster replication cleanruv maxcsn extended operation plugin", NULL, identity); + rc= slapi_register_plugin("extendedop", 1 /* Enabled */, "multimaster_cleanruv_status_extop_init", multimaster_cleanruv_status_extop_init, "Multimaster replication cleanruv status extended operation plugin", NULL, identity); if (0 == rc) { multimaster_initialised = 1; diff --git a/ldap/servers/plugins/replication/repl5_replica.c b/ldap/servers/plugins/replication/repl5_replica.c index 2b1d958..b3df831 100644 --- a/ldap/servers/plugins/replication/repl5_replica.c +++ b/ldap/servers/plugins/replication/repl5_replica.c @@ -88,7 +88,6 @@ struct replica { PRBool state_update_inprogress; /* replica state is being updated */ PRLock *agmt_lock; /* protects agreement creation, start and stop */ char *locking_purl; /* supplier who has exclusive access */ - char *repl_cleanruv_data[CLEANRIDSIZ + 1]; }; @@ -310,7 +309,6 @@ replica_destroy(void **arg) { Replica *r; void *repl_name; - int i; if (arg == NULL) return; @@ -397,10 +395,6 @@ replica_destroy(void **arg) csnplFree(&r->min_csn_pl);; } - for(i = 0;r->repl_cleanruv_data[i] != NULL; i++){ - slapi_ch_free_string(&r->repl_cleanruv_data[i]); - } - slapi_ch_free((void **)arg); } @@ -1502,7 +1496,7 @@ int replica_check_for_data_reload (Replica *r, void *arg) slapi_log_error(SLAPI_LOG_FATAL, repl_plugin_name, "replica_check_for_data_reload: " "Warning: for replica %s there were some differences between the changelog max RUV and the " "database RUV. If there are obsolete elements in the database RUV, you " - "should remove them using CLEANRUV task. If they are not obsolete, " + "should remove them using the CLEANALLRUV task. If they are not obsolete, " "you should check their status to see why there are no changes from those " "servers in the changelog.\n", slapi_sdn_get_dn(r->repl_root)); @@ -1819,12 +1813,13 @@ replica_check_for_tasks(Replica *r, Slapi_Entry *e) PRThread *thread = NULL; struct berval *payload = NULL; CSN *maxcsn = NULL; - char *csnpart; - char *iter; + ReplicaId rid; char csnstr[CSN_STRSIZE]; - char *ridstr; char *token = NULL; - ReplicaId rid; + char *forcing; + char *csnpart; + char *ridstr; + char *iter; int i; for(i = 0; i < CLEANRIDSIZ && clean_vals[i]; i++){ @@ -1833,7 +1828,6 @@ replica_check_for_tasks(Replica *r, Slapi_Entry *e) /* * Set the cleanruv data, and add the cleaned rid */ - r->repl_cleanruv_data[i] = slapi_ch_strdup(clean_vals[i]); token = ldap_utf8strtok_r(clean_vals[i], ":", &iter); if(token){ rid = atoi(token); @@ -1851,15 +1845,18 @@ replica_check_for_tasks(Replica *r, Slapi_Entry *e) maxcsn = csn_new(); csn_init_by_string(maxcsn, csnpart); csn_as_string(maxcsn, PR_FALSE, csnstr); - add_cleaned_rid(rid, r, csnstr); + forcing = ldap_utf8strtok_r(iter, ":", &iter); + if(forcing == NULL){ + forcing = "no"; + } slapi_log_error( SLAPI_LOG_FATAL, repl_plugin_name, "CleanAllRUV Task: cleanAllRUV task found, " "resuming the cleaning of rid(%d)...\n", rid); /* * Create payload */ - ridstr = slapi_ch_smprintf("%d:%s:%s", rid, slapi_sdn_get_dn(replica_get_root(r)), csnstr); - payload = create_ruv_payload(ridstr); + ridstr = slapi_ch_smprintf("%d:%s:%s:%s", rid, slapi_sdn_get_dn(replica_get_root(r)), csnstr, forcing); + payload = create_cleanruv_payload(ridstr); slapi_ch_free_string(&ridstr); if(payload == NULL){ @@ -1882,8 +1879,10 @@ replica_check_for_tasks(Replica *r, Slapi_Entry *e) data->rid = rid; data->task = NULL; data->maxcsn = maxcsn; - data->sdn = slapi_sdn_dup(r->repl_root); data->payload = payload; + data->sdn = slapi_sdn_dup(r->repl_root); + data->force = slapi_ch_strdup(forcing); + data->repl_root = NULL; thread = PR_CreateThread(PR_USER_THREAD, replica_cleanallruv_thread_ext, (void *)data, PR_PRIORITY_NORMAL, PR_GLOBAL_THREAD, @@ -1895,11 +1894,11 @@ replica_check_for_tasks(Replica *r, Slapi_Entry *e) csn_free(&maxcsn); slapi_sdn_free(&data->sdn); ber_bvfree(data->payload); + slapi_ch_free_string(&data->force); slapi_ch_free((void **)&data); } } } - r->repl_cleanruv_data[i] = NULL; done: slapi_ch_array_free(clean_vals); @@ -1909,13 +1908,12 @@ done: { PRThread *thread = NULL; struct berval *payload; - CSN *maxcsn = NULL; - char *iter; + ReplicaId rid; + char *certify = NULL; char *ridstr = NULL; - char *repl_root; char *token = NULL; - char *certify = NULL; - ReplicaId rid; + char *repl_root; + char *iter; int i; for(i = 0; clean_vals[i]; i++){ @@ -1926,21 +1924,27 @@ done: rid = atoi(token); if(rid <= 0 || rid >= READ_ONLY_REPLICA_ID){ slapi_log_error( SLAPI_LOG_FATAL, repl_plugin_name, "Abort CleanAllRUV Task: invalid replica id(%d) " - "aborting task.\n", rid); + "aborting abort task.\n", rid); goto done2; } } else { slapi_log_error( SLAPI_LOG_FATAL, repl_plugin_name, "Abort CleanAllRUV Task: unable to parse cleanallruv " - "data (%s), aborting task.\n",clean_vals[i]); + "data (%s), aborting abort task.\n",clean_vals[i]); goto done2; } repl_root = ldap_utf8strtok_r(iter, ":", &iter); certify = ldap_utf8strtok_r(iter, ":", &iter); + + if(!is_cleaned_rid(rid)){ + slapi_log_error( SLAPI_LOG_FATAL, repl_plugin_name, "Abort CleanAllRUV Task: replica id(%d) is not " + "being cleaned, nothing to abort. Aborting abort task.\n", rid); + delete_aborted_rid(r, rid, repl_root, 0); + goto done2; + } + + add_aborted_rid(rid, r, repl_root); stop_ruv_cleaning(); - maxcsn = replica_get_cleanruv_maxcsn(r, rid); - delete_cleaned_rid(r, rid, maxcsn); - csn_free(&maxcsn); slapi_log_error( SLAPI_LOG_FATAL, repl_plugin_name, "Abort CleanAllRUV Task: abort task found, " "resuming abort of rid(%d).\n", rid); @@ -1951,8 +1955,8 @@ done: if (data == NULL) { slapi_log_error( SLAPI_LOG_FATAL, repl_plugin_name, "Abort CleanAllRUV Task: failed to allocate cleanruv_data.\n"); } else { - ridstr = slapi_ch_smprintf("%d:%s", rid, repl_root); - payload = create_ruv_payload(ridstr); + ridstr = slapi_ch_smprintf("%d:%s:%s", rid, repl_root, certify); + payload = create_cleanruv_payload(ridstr); slapi_ch_free_string(&ridstr); if(payload == NULL){ @@ -2583,7 +2587,7 @@ replica_ruv_smods_for_op( Slapi_PBlock *pb, char **uniqueid, Slapi_Mods **smods must use the CLEANRUV task instead */ slapi_log_error(SLAPI_LOG_FATAL, repl_plugin_name, "replica_ruv_smods_for_op: attempted to directly modify the tombstone RUV " - "entry [%s] - use the CLEANRUV task instead\n", + "entry [%s] - use the CLEANALLRUV task instead\n", slapi_entry_get_dn_const(target_entry)); return (-1); } @@ -3430,7 +3434,7 @@ replica_replace_ruv_tombstone(Replica *r) if (rc != LDAP_SUCCESS) { - if ((rc != LDAP_NO_SUCH_OBJECT) || !replica_is_state_flag_set(r, REPLICA_IN_USE)) + if ((rc != LDAP_NO_SUCH_OBJECT && rc != LDAP_TYPE_OR_VALUE_EXISTS) || !replica_is_state_flag_set(r, REPLICA_IN_USE)) { slapi_log_error(SLAPI_LOG_FATAL, repl_plugin_name, "replica_replace_ruv_tombstone: " "failed to update replication update vector for replica %s: LDAP " @@ -3794,69 +3798,3 @@ replica_get_attr ( Slapi_PBlock *pb, const char* type, void *value ) return rc; } - -void -replica_add_cleanruv_data(Replica *r, char *val) -{ - int i; - - PR_Lock(r->repl_lock); - - for (i = 0; i < CLEANRIDSIZ && r->repl_cleanruv_data[i] != NULL; i++); /* goto the end of the list */ - if( i < CLEANRIDSIZ){ - r->repl_cleanruv_data[i] = slapi_ch_strdup(val); /* append to list */ - r->repl_cleanruv_data[i + 1] = 0; - } - - PR_Unlock(r->repl_lock); -} - -void -replica_remove_cleanruv_data(Replica *r, char *val) -{ - int i; - - PR_Lock(r->repl_lock); - - for(i = 0; i < CLEANRIDSIZ && r->repl_cleanruv_data[i] && strcmp(r->repl_cleanruv_data[i], val) != 0; i++); - if( i < CLEANRIDSIZ ){ - slapi_ch_free_string(&r->repl_cleanruv_data[i]); - for(; i < CLEANRIDSIZ; i++){ - /* rewrite entire array */ - r->repl_cleanruv_data[i] = r->repl_cleanruv_data[i + 1]; - } - } - - PR_Unlock(r->repl_lock); -} - -CSN * -replica_get_cleanruv_maxcsn(Replica *r, ReplicaId rid) -{ - CSN *newcsn; - char *csnstr; - char *token; - char *iter; - int repl_rid = 0; - int i; - - PR_Lock(r->repl_lock); - - for(i = 0; i < CLEANRIDSIZ && r->repl_cleanruv_data[i]; i++){ - token = ldap_utf8strtok_r(r->repl_cleanruv_data[i], ":", &iter); - if(token){ - repl_rid = atoi(token); - } - csnstr = ldap_utf8strtok_r(iter, ":", &iter); - if(repl_rid == rid){ - newcsn = csn_new(); - csn_init_by_string(newcsn, csnstr); - PR_Unlock(r->repl_lock); - return newcsn; - } - } - - PR_Unlock(r->repl_lock); - - return NULL; -} diff --git a/ldap/servers/plugins/replication/repl5_replica_config.c b/ldap/servers/plugins/replication/repl5_replica_config.c index c5ca965..0fd785b 100644 --- a/ldap/servers/plugins/replication/repl5_replica_config.c +++ b/ldap/servers/plugins/replication/repl5_replica_config.c @@ -64,6 +64,7 @@ int slapi_log_urp = SLAPI_LOG_REPL; static ReplicaId cleaned_rids[CLEANRIDSIZ + 1] = {0}; +static ReplicaId pre_cleaned_rids[CLEANRIDSIZ + 1] = {0}; static ReplicaId aborted_rids[CLEANRIDSIZ + 1] = {0}; static Slapi_RWLock *rid_lock = NULL; static Slapi_RWLock *abort_rid_lock = NULL; @@ -84,20 +85,27 @@ static int replica_execute_task (Object *r, const char *task_name, char *returnt static int replica_execute_cl2ldif_task (Object *r, char *returntext); static int replica_execute_ldif2cl_task (Object *r, char *returntext); static int replica_execute_cleanruv_task (Object *r, ReplicaId rid, char *returntext); -static int replica_execute_cleanall_ruv_task (Object *r, ReplicaId rid, Slapi_Task *task, char *returntext); +static int replica_execute_cleanall_ruv_task (Object *r, ReplicaId rid, Slapi_Task *task, const char *force_cleaning, char *returntext); static void replica_cleanallruv_thread(void *arg); -static void replica_send_cleanruv_task(Repl_Agmt *agmt, ReplicaId rid, Slapi_Task *task); +static void replica_send_cleanruv_task(Repl_Agmt *agmt, cleanruv_data *clean_data); static int check_agmts_are_alive(Replica *replica, ReplicaId rid, Slapi_Task *task); -static int check_agmts_are_caught_up(Replica *replica, ReplicaId rid, char *maxcsn, Slapi_Task *task); -static int replica_cleanallruv_send_extop(Repl_Agmt *ra, ReplicaId rid, Slapi_Task *task, struct berval *payload, int check_result); +static int check_agmts_are_caught_up(cleanruv_data *data, char *maxcsn); +static int replica_cleanallruv_send_extop(Repl_Agmt *ra, cleanruv_data *data, int check_result); static int replica_cleanallruv_send_abort_extop(Repl_Agmt *ra, Slapi_Task *task, struct berval *payload); -static int replica_cleanallruv_check_maxcsn(Repl_Agmt *agmt, char *rid_text, char *maxcsn, Slapi_Task *task); +static int replica_cleanallruv_check_maxcsn(Repl_Agmt *agmt, char *basedn, char *rid_text, char *maxcsn, Slapi_Task *task); static int replica_cleanallruv_replica_alive(Repl_Agmt *agmt); -static int replica_cleanallruv_check_ruv(Repl_Agmt *ra, char *rid_text, Slapi_Task *task); +static int replica_cleanallruv_check_ruv(char *repl_root, Repl_Agmt *ra, char *rid_text, Slapi_Task *task); static int get_cleanruv_task_count(); static int get_abort_cleanruv_task_count(); static int replica_cleanup_task (Object *r, const char *task_name, char *returntext, int apply_mods); static int replica_task_done(Replica *replica); +static void delete_cleaned_rid_config(cleanruv_data *data); +static int replica_cleanallruv_is_finished(Repl_Agmt *agmt, char *filter, Slapi_Task *task); +static void check_replicas_are_done_cleaning(cleanruv_data *data); +static void check_replicas_are_done_aborting(cleanruv_data *data ); +static CSN* replica_cleanallruv_find_maxcsn(Replica *replica, ReplicaId rid, char *basedn); +static int replica_cleanallruv_get_replica_maxcsn(Repl_Agmt *agmt, char *rid_text, char *basedn, CSN **csn); +static void preset_cleaned_rid(ReplicaId rid); static multimaster_mtnode_extension * _replica_config_get_mtnode_ext (const Slapi_Entry *e); /* @@ -385,6 +393,12 @@ replica_config_modify (Slapi_PBlock *pb, Slapi_Entry* entryBefore, Slapi_Entry* if (apply_mods) replica_set_legacy_consumer (r, PR_FALSE); } + else if (strcasecmp (config_attr, type_replicaCleanRUV) == 0 || + strcasecmp (config_attr, type_replicaAbortCleanRUV) == 0) + { + /* only allow the deletion of the cleanAllRUV config attributes */ + continue; + } else { *returncode = LDAP_UNWILLING_TO_PERFORM; @@ -891,7 +905,7 @@ static int replica_execute_task (Object *r, const char *task_name, char *returnt if (apply_mods) { Slapi_Task *empty_task = NULL; - return replica_execute_cleanall_ruv_task(r, (ReplicaId)temprid, empty_task, returntext); + return replica_execute_cleanall_ruv_task(r, (ReplicaId)temprid, empty_task, returntext, "no"); } else return LDAP_SUCCESS; @@ -1225,35 +1239,52 @@ replica_cleanall_ruv_task(Slapi_PBlock *pb, Slapi_Entry *e, Slapi_Entry *eAfter, Slapi_Task *task = NULL; const Slapi_DN *task_dn; Slapi_DN *dn = NULL; + ReplicaId rid; Object *r; + const char *force_cleaning; const char *base_dn; const char *rid_str; - ReplicaId rid; int rc = SLAPI_DSE_CALLBACK_OK; /* allocate new task now */ task = slapi_new_task(slapi_entry_get_ndn(e)); + task_dn = slapi_entry_get_sdn(e); if(task == NULL){ slapi_log_error(SLAPI_LOG_FATAL, repl_plugin_name, "cleanAllRUV_task: Failed to create new task\n"); rc = SLAPI_DSE_CALLBACK_ERROR; goto out; } - /* * Get our task settings */ if ((base_dn = fetch_attr(e, "replica-base-dn", 0)) == NULL){ + PR_snprintf(returntext, SLAPI_DSE_RETURNTEXT_SIZE, "Missing replica-base-dn attribute"); + cleanruv_log(task, CLEANALLRUV_ID, "%s", returntext); *returncode = LDAP_OBJECT_CLASS_VIOLATION; rc = SLAPI_DSE_CALLBACK_ERROR; goto out; } if ((rid_str = fetch_attr(e, "replica-id", 0)) == NULL){ + PR_snprintf(returntext, SLAPI_DSE_RETURNTEXT_SIZE, "Missing replica-id attribute"); + cleanruv_log(task, CLEANALLRUV_ID, "%s", returntext); *returncode = LDAP_OBJECT_CLASS_VIOLATION; rc = SLAPI_DSE_CALLBACK_ERROR; goto out; } - task_dn = slapi_entry_get_sdn(e); + if ((force_cleaning = fetch_attr(e, "replica-force-cleaning", 0)) != NULL){ + if(strcasecmp(force_cleaning,"yes") != 0 && strcasecmp(force_cleaning,"no") != 0){ + PR_snprintf(returntext, SLAPI_DSE_RETURNTEXT_SIZE, "Invalid value for replica-force-cleaning " + "(%s). Value must be \"yes\" or \"no\" for task - (%s)", + force_cleaning, slapi_sdn_get_dn(task_dn)); + cleanruv_log(task, CLEANALLRUV_ID, "%s", returntext); + *returncode = LDAP_OPERATIONS_ERROR; + rc = SLAPI_DSE_CALLBACK_ERROR; + goto out; + } + } else { + force_cleaning = "no"; + } /* * Check the rid */ @@ -1266,6 +1297,14 @@ replica_cleanall_ruv_task(Slapi_PBlock *pb, Slapi_Entry *e, Slapi_Entry *eAfter, rc = SLAPI_DSE_CALLBACK_ERROR; goto out; } + if(is_cleaned_rid(rid)){ + /* we are already cleaning this rid */ + PR_snprintf(returntext, SLAPI_DSE_RETURNTEXT_SIZE, "Replica id (%d) is already being cleaned", rid); + cleanruv_log(task, CLEANALLRUV_ID, "%s", returntext); + *returncode = LDAP_UNWILLING_TO_PERFORM; + rc = SLAPI_DSE_CALLBACK_ERROR; + goto out; + } /* * Get the replica object */ @@ -1279,7 +1318,7 @@ replica_cleanall_ruv_task(Slapi_PBlock *pb, Slapi_Entry *e, Slapi_Entry *eAfter, } /* clean the RUV's */ - rc = replica_execute_cleanall_ruv_task (r, rid, task, returntext); + rc = replica_execute_cleanall_ruv_task (r, rid, task, force_cleaning, returntext); out: if(rc){ @@ -1302,31 +1341,36 @@ out: * */ static int -replica_execute_cleanall_ruv_task (Object *r, ReplicaId rid, Slapi_Task *task, char *returntext) +replica_execute_cleanall_ruv_task (Object *r, ReplicaId rid, Slapi_Task *task, const char* force_cleaning, char *returntext) { - PRThread *thread = NULL; + struct berval *payload = NULL; Slapi_Task *pre_task = NULL; /* this is supposed to be null for logging */ - Replica *replica; - Object *ruv_obj; cleanruv_data *data = NULL; + PRThread *thread = NULL; CSN *maxcsn = NULL; - const RUV *ruv; - struct berval *payload = NULL; - char *ridstr = NULL; + Replica *replica; char csnstr[CSN_STRSIZE]; + char *ridstr = NULL; + char *basedn = NULL; int rc = 0; + cleanruv_log(pre_task, CLEANALLRUV_ID,"Initiating CleanAllRUV Task..."); + if(get_cleanruv_task_count() >= CLEANRIDSIZ){ /* we are already running the maximum number of tasks */ cleanruv_log(pre_task, CLEANALLRUV_ID, "Exceeded maximum number of active CLEANALLRUV tasks(%d)",CLEANRIDSIZ); return LDAP_UNWILLING_TO_PERFORM; } - /* * Grab the replica */ - replica = (Replica*)object_get_data (r); + if(r){ + replica = (Replica*)object_get_data (r); + } else { + cleanruv_log(pre_task, CLEANALLRUV_ID, "Replica object is NULL, aborting task"); + return -1; + } /* * Check if this is a consumer */ @@ -1342,32 +1386,25 @@ replica_execute_cleanall_ruv_task (Object *r, ReplicaId rid, Slapi_Task *task, c /* * Grab the max csn of the deleted replica */ - ruv_obj = replica_get_ruv(replica); - ruv = object_get_data (ruv_obj); - if(ruv_get_rid_max_csn(ruv, &maxcsn, rid) == RUV_BAD_DATA){ - /* no maxcsn, can not proceed */ - cleanruv_log(pre_task, CLEANALLRUV_ID, "Could not find maxcsn for rid (%d)", rid); - rc = -1; - object_release(ruv_obj); - goto fail; - } else { - object_release(ruv_obj); - if(maxcsn == NULL || csn_get_replicaid(maxcsn) == 0){ - /* - * This is for consistency with extop csn creation, where - * we want the csn string to be "0000000000000000000" not "" - */ - csn_free(&maxcsn); - maxcsn = csn_new(); - csn_init_by_string(maxcsn, ""); - } - csn_as_string(maxcsn, PR_FALSE, csnstr); + cleanruv_log(pre_task, CLEANALLRUV_ID, "Retrieving maxcsn..."); + basedn = (char *)slapi_sdn_get_dn(replica_get_root(replica)); + maxcsn = replica_cleanallruv_find_maxcsn(replica, rid, basedn); + if(maxcsn == NULL || csn_get_replicaid(maxcsn) == 0){ + /* + * This is for consistency with extop csn creation, where + * we want the csn string to be "0000000000000000000" not "" + */ + csn_free(&maxcsn); + maxcsn = csn_new(); + csn_init_by_string(maxcsn, ""); } + csn_as_string(maxcsn, PR_FALSE, csnstr); + cleanruv_log(pre_task, CLEANALLRUV_ID, "Found maxcsn (%s)",csnstr); /* * Create payload */ - ridstr = slapi_ch_smprintf("%d:%s:%s", rid, slapi_sdn_get_dn(replica_get_root(replica)), csnstr); - payload = create_ruv_payload(ridstr); + ridstr = slapi_ch_smprintf("%d:%s:%s:%s", rid, basedn, csnstr, force_cleaning); + payload = create_cleanruv_payload(ridstr); slapi_ch_free_string(&ridstr); if(payload == NULL){ @@ -1389,15 +1426,19 @@ replica_execute_cleanall_ruv_task (Object *r, ReplicaId rid, Slapi_Task *task, c data->replica = replica; data->rid = rid; data->task = task; - data->maxcsn = maxcsn; data->payload = payload; data->sdn = NULL; + data->maxcsn = maxcsn; + data->repl_root = slapi_ch_strdup(basedn); + data->force = slapi_ch_strdup(force_cleaning); thread = PR_CreateThread(PR_USER_THREAD, replica_cleanallruv_thread, (void *)data, PR_PRIORITY_NORMAL, PR_GLOBAL_THREAD, PR_UNJOINABLE_THREAD, SLAPD_DEFAULT_THREAD_STACKSIZE); if (thread == NULL) { rc = -1; + slapi_ch_free_string(&data->force); + slapi_ch_free_string(&data->repl_root); goto fail; } else { goto done; @@ -1441,11 +1482,11 @@ replica_cleanallruv_thread_ext(void *arg) static void replica_cleanallruv_thread(void *arg) { - Object *ruv_obj = NULL; + cleanruv_data *data = arg; Object *agmt_obj = NULL; + Object *ruv_obj = NULL; Repl_Agmt *agmt = NULL; RUV *ruv = NULL; - cleanruv_data *data = arg; char csnstr[CSN_STRSIZE]; char *returntext = NULL; char *rid_text = NULL; @@ -1466,7 +1507,7 @@ replica_cleanallruv_thread(void *arg) * the backends might not be online yet. */ PR_Lock( notify_lock ); - PR_WaitCondVar( notify_cvar, PR_SecondsToInterval(5) ); + PR_WaitCondVar( notify_cvar, PR_SecondsToInterval(10) ); PR_Unlock( notify_lock ); data->repl_obj = replica_get_replica_from_dn(data->sdn); if(data->repl_obj == NULL){ @@ -1488,17 +1529,25 @@ replica_cleanallruv_thread(void *arg) aborted = 1; goto done; } + if(data->repl_root == NULL){ + /* we must have resumed from start up, fill in the repl root */ + data->repl_root = slapi_ch_strdup(slapi_sdn_get_dn(replica_get_root(data->replica))); + } if(data->task){ slapi_task_begin(data->task, 1); } - rid_text = slapi_ch_smprintf("{replica %d ldap", data->rid); + /* + * Presetting the rid prevents duplicate thread creation, but allows the db and changelog to still + * process updates from the rid. set_cleaned_rid() blocks updates, so we don't want to do that... yet. + */ + preset_cleaned_rid(data->rid); + rid_text = slapi_ch_smprintf("%d", data->rid); csn_as_string(data->maxcsn, PR_FALSE, csnstr); - /* * Add the cleanallruv task to the repl config - so we can handle restarts */ + add_cleaned_rid(data->rid, data->replica, csnstr, data->force); /* marks config that we started cleaning a rid */ cleanruv_log(data->task, CLEANALLRUV_ID, "Cleaning rid (%d)...", data->rid); - add_cleaned_rid(data->rid, data->replica, csnstr); /* marks config that we started cleaning a rid */ /* * First, wait for the maxcsn to be covered */ @@ -1506,7 +1555,7 @@ replica_cleanallruv_thread(void *arg) ruv_obj = replica_get_ruv(data->replica); ruv = object_get_data (ruv_obj); while(data->maxcsn && !is_task_aborted(data->rid) && !is_cleaned_rid(data->rid) && !slapi_is_shutting_down()){ - if(csn_get_replicaid(data->maxcsn) == 0 || ruv_covers_csn_cleanallruv(ruv,data->maxcsn)){ + if(csn_get_replicaid(data->maxcsn) == 0 || ruv_covers_csn_cleanallruv(ruv,data->maxcsn) || strcasecmp(data->force,"yes") == 0){ /* We are caught up, now we can clean the ruv's */ break; } @@ -1517,6 +1566,8 @@ replica_cleanallruv_thread(void *arg) object_release(ruv_obj); /* * Next, make sure all the replicas are up and running before sending off the clean ruv tasks + * + * Even if we are forcing the cleaning, the replicas still need to be up */ cleanruv_log(data->task, CLEANALLRUV_ID,"Waiting for all the replicas to be online..."); if(check_agmts_are_alive(data->replica, data->rid, data->task)){ @@ -1528,7 +1579,7 @@ replica_cleanallruv_thread(void *arg) * Make sure all the replicas have seen the max csn */ cleanruv_log(data->task, CLEANALLRUV_ID,"Waiting for all the replicas to receive all the deleted replica updates..."); - if(check_agmts_are_caught_up(data->replica, data->rid, csnstr, data->task)){ + if(strcasecmp(data->force,"no") == 0 && check_agmts_are_caught_up(data, csnstr)){ /* error, aborted or shutdown */ aborted = 1; goto done; @@ -1555,10 +1606,11 @@ replica_cleanallruv_thread(void *arg) agmt_not_notified = 0; continue; } - if(replica_cleanallruv_send_extop(agmt, data->rid, data->task, data->payload, 1) == 0){ + if(replica_cleanallruv_send_extop(agmt, data, 1) == 0){ agmt_not_notified = 0; } else { agmt_not_notified = 1; + cleanruv_log(data->task, CLEANALLRUV_ID, "Failed to send task to replica (%s)",agmt_get_long_name(agmt)); break; } agmt_obj = agmtlist_get_next_agreement_for_replica (data->replica, agmt_obj); @@ -1609,10 +1661,11 @@ replica_cleanallruv_thread(void *arg) found_dirty_rid = 0; continue; } - if(replica_cleanallruv_check_ruv(agmt, rid_text, data->task) == 0){ + if(replica_cleanallruv_check_ruv(data->repl_root, agmt, rid_text, data->task) == 0){ found_dirty_rid = 0; } else { found_dirty_rid = 1; + cleanruv_log(data->task, CLEANALLRUV_ID,"Replica is not cleaned yet (%s)",agmt_get_long_name(agmt)); break; } agmt_obj = agmtlist_get_next_agreement_for_replica (data->replica, agmt_obj); @@ -1623,7 +1676,7 @@ replica_cleanallruv_thread(void *arg) goto done; } if(found_dirty_rid == 0){ - break; + break; } /* * need to sleep between passes @@ -1647,9 +1700,11 @@ done: */ if(!aborted){ trigger_cl_trimming(data->rid); - delete_cleaned_rid(data->replica, data->rid, data->maxcsn); + delete_cleaned_rid_config(data); + /* make sure all the replicas have been "pre_cleaned" before finishing */ + check_replicas_are_done_cleaning(data); cleanruv_log(data->task, CLEANALLRUV_ID, "Successfully cleaned rid(%d).", data->rid); - slapi_task_finish(data->task, rc); + remove_cleaned_rid(data->rid); } else { /* * Shutdown or abort @@ -1658,29 +1713,189 @@ done: cleanruv_log(data->task, CLEANALLRUV_ID,"Server shutting down. Process will resume at server startup"); } else { cleanruv_log(data->task, CLEANALLRUV_ID,"Task aborted for rid(%d).",data->rid); - } - if(data->task){ - slapi_task_finish(data->task, rc); + delete_cleaned_rid_config(data); + remove_cleaned_rid(data->rid); } } - + if(data->task){ + slapi_task_finish(data->task, rc); + } if(data->payload){ ber_bvfree(data->payload); } if(data->repl_obj && free_obj){ object_release(data->repl_obj); } + csn_free(&data->maxcsn); slapi_sdn_free(&data->sdn); + slapi_ch_free_string(&data->repl_root); + slapi_ch_free_string(&data->force); slapi_ch_free_string(&rid_text); - csn_free(&data->maxcsn); slapi_ch_free((void **)&data); } /* + * Loop over the agmts, and check if they are in the last phase of cleaning, meaning they have + * released cleanallruv data from the config + */ +static void +check_replicas_are_done_cleaning(cleanruv_data *data ) +{ + Object *agmt_obj; + Repl_Agmt *agmt; + char csnstr[CSN_STRSIZE]; + char *filter = NULL; + int not_all_cleaned = 1; + int interval = 10; + + cleanruv_log(data->task, CLEANALLRUV_ID, "Waiting for all the replicas to finish cleaning..."); + + csn_as_string(data->maxcsn, PR_FALSE, csnstr); + filter = PR_smprintf("(%s=%d:%s:%s)", type_replicaCleanRUV,(int)data->rid, csnstr, data->force); + while(not_all_cleaned && !is_task_aborted(data->rid) && !slapi_is_shutting_down()){ + agmt_obj = agmtlist_get_first_agreement_for_replica (data->replica); + if(agmt_obj == NULL){ + not_all_cleaned = 0; + break; + } + while (agmt_obj && !slapi_is_shutting_down()){ + agmt = (Repl_Agmt*)object_get_data (agmt_obj); + if(!agmt_is_enabled(agmt) || get_agmt_agreement_type(agmt) == REPLICA_TYPE_WINDOWS){ + agmt_obj = agmtlist_get_next_agreement_for_replica (data->replica, agmt_obj); + not_all_cleaned = 0; + continue; + } + if(replica_cleanallruv_is_finished(agmt, filter, data->task) == 0){ + not_all_cleaned = 0; + } else { + not_all_cleaned = 1; + break; + } + agmt_obj = agmtlist_get_next_agreement_for_replica (data->replica, agmt_obj); + } + if(not_all_cleaned == 0 || is_task_aborted(data->rid) ){ + break; + } + cleanruv_log(data->task, CLEANALLRUV_ID, "Not all replicas finished cleaning, retrying in %d seconds",interval); + PR_Lock( notify_lock ); + PR_WaitCondVar( notify_cvar, PR_SecondsToInterval(interval) ); + PR_Unlock( notify_lock ); + if(interval < 14400){ /* 4 hour max */ + interval = interval * 2; + } else { + interval = 14400; + } + } + slapi_ch_free_string(&filter); +} + +/* + * Search this replica for the nsds5ReplicaCleanruv attribute, we don't return + * an entry, we know it has finished cleaning. + */ +static int +replica_cleanallruv_is_finished(Repl_Agmt *agmt, char *filter, Slapi_Task *task) +{ + Repl_Connection *conn = NULL; + ConnResult crc = 0; + struct berval *payload = NULL; + int msgid = 0; + int rc = -1; + + if((conn = conn_new(agmt)) == NULL){ + return -1; + } + if(conn_connect(conn) == CONN_OPERATION_SUCCESS){ + payload = create_cleanruv_payload(filter); + crc = conn_send_extended_operation(conn, REPL_CLEANRUV_CHECK_STATUS_OID, payload, NULL, &msgid); + if(crc == CONN_OPERATION_SUCCESS){ + struct berval *retsdata = NULL; + char *retoid = NULL; + + crc = conn_read_result_ex(conn, &retoid, &retsdata, NULL, msgid, NULL, 1); + if (CONN_OPERATION_SUCCESS == crc ){ + char *response = NULL; + + decode_cleanruv_payload(retsdata, &response); + if(response && strcmp(response,CLEANRUV_FINISHED) == 0){ + /* finished cleaning */ + rc = 0; + } + if (NULL != retsdata) + ber_bvfree(retsdata); + slapi_ch_free_string(&response); + slapi_ch_free_string(&retoid); + } + } + } else { + rc = -1; + } + conn_delete_internal_ext(conn); + if(payload) + ber_bvfree(payload); + + return rc; +} + +/* + * Loop over the agmts, and check if they are in the last phase of aborting, meaning they have + * released the abort cleanallruv data from the config + */ +static void +check_replicas_are_done_aborting(cleanruv_data *data ) +{ + Object *agmt_obj; + Repl_Agmt *agmt; + char *filter = NULL; + int not_all_aborted = 1; + int interval = 10; + + cleanruv_log(data->task, ABORT_CLEANALLRUV_ID,"Waiting for all the replicas to finish aborting..."); + + filter = PR_smprintf("(%s=%d:%s)", type_replicaAbortCleanRUV, data->rid, data->repl_root); + + while(not_all_aborted && !slapi_is_shutting_down()){ + agmt_obj = agmtlist_get_first_agreement_for_replica (data->replica); + if(agmt_obj == NULL){ + not_all_aborted = 0; + break; + } + while (agmt_obj && !slapi_is_shutting_down()){ + agmt = (Repl_Agmt*)object_get_data (agmt_obj); + if(get_agmt_agreement_type(agmt) == REPLICA_TYPE_WINDOWS){ + agmt_obj = agmtlist_get_next_agreement_for_replica (data->replica, agmt_obj); + not_all_aborted = 0; + continue; + } + if(replica_cleanallruv_is_finished(agmt, filter, data->task) == 0){ + not_all_aborted = 0; + } else { + not_all_aborted = 1; + break; + } + agmt_obj = agmtlist_get_next_agreement_for_replica (data->replica, agmt_obj); + } + if(not_all_aborted == 0){ + break; + } + cleanruv_log(data->task, ABORT_CLEANALLRUV_ID, "Not all replicas finished aborting, retrying in %d seconds",interval); + PR_Lock( notify_lock ); + PR_WaitCondVar( notify_cvar, PR_SecondsToInterval(interval) ); + PR_Unlock( notify_lock ); + if(interval < 14400){ /* 4 hour max */ + interval = interval * 2; + } else { + interval = 14400; + } + } + slapi_ch_free_string(&filter); +} + +/* * Waits for all the repl agmts to be have have the maxcsn. Returns error only on abort or shutdown */ static int -check_agmts_are_caught_up(Replica *replica, ReplicaId rid, char *maxcsn, Slapi_Task *task) +check_agmts_are_caught_up(cleanruv_data *data, char *maxcsn) { Object *agmt_obj; Repl_Agmt *agmt; @@ -1688,10 +1903,10 @@ check_agmts_are_caught_up(Replica *replica, ReplicaId rid, char *maxcsn, Slapi_T int not_all_caughtup = 1; int interval = 10; - rid_text = slapi_ch_smprintf("{replica %d ldap", rid); + rid_text = slapi_ch_smprintf("%d", data->rid); - while(not_all_caughtup && !is_task_aborted(rid) && !slapi_is_shutting_down()){ - agmt_obj = agmtlist_get_first_agreement_for_replica (replica); + while(not_all_caughtup && !is_task_aborted(data->rid) && !slapi_is_shutting_down()){ + agmt_obj = agmtlist_get_first_agreement_for_replica (data->replica); if(agmt_obj == NULL){ not_all_caughtup = 0; break; @@ -1699,23 +1914,24 @@ check_agmts_are_caught_up(Replica *replica, ReplicaId rid, char *maxcsn, Slapi_T while (agmt_obj){ agmt = (Repl_Agmt*)object_get_data (agmt_obj); if(!agmt_is_enabled(agmt) || get_agmt_agreement_type(agmt) == REPLICA_TYPE_WINDOWS){ - agmt_obj = agmtlist_get_next_agreement_for_replica (replica, agmt_obj); + agmt_obj = agmtlist_get_next_agreement_for_replica (data->replica, agmt_obj); not_all_caughtup = 0; continue; } - if(replica_cleanallruv_check_maxcsn(agmt, rid_text, maxcsn, task) == 0){ + if(replica_cleanallruv_check_maxcsn(agmt, data->repl_root, rid_text, maxcsn, data->task) == 0){ not_all_caughtup = 0; } else { not_all_caughtup = 1; + cleanruv_log(data->task, CLEANALLRUV_ID, "Replica not caught up (%s)",agmt_get_long_name(agmt)); break; } - agmt_obj = agmtlist_get_next_agreement_for_replica (replica, agmt_obj); - } /* agmt while */ + agmt_obj = agmtlist_get_next_agreement_for_replica (data->replica, agmt_obj); + } - if(not_all_caughtup == 0 || is_task_aborted(rid) ){ + if(not_all_caughtup == 0 || is_task_aborted(data->rid) ){ break; } - cleanruv_log(task, CLEANALLRUV_ID, "Not all replicas caught up, retrying in %d seconds",interval); + cleanruv_log(data->task, CLEANALLRUV_ID, "Not all replicas caught up, retrying in %d seconds",interval); PR_Lock( notify_lock ); PR_WaitCondVar( notify_cvar, PR_SecondsToInterval(interval) ); PR_Unlock( notify_lock ); @@ -1728,7 +1944,7 @@ check_agmts_are_caught_up(Replica *replica, ReplicaId rid, char *maxcsn, Slapi_T } slapi_ch_free_string(&rid_text); - if(is_task_aborted(rid)){ + if(is_task_aborted(data->rid)){ return -1; } @@ -1763,6 +1979,7 @@ check_agmts_are_alive(Replica *replica, ReplicaId rid, Slapi_Task *task) not_all_alive = 0; } else { not_all_alive = 1; + cleanruv_log(task, CLEANALLRUV_ID, "Replica not online (%s)",agmt_get_long_name(agmt)); break; } agmt_obj = agmtlist_get_next_agreement_for_replica (replica, agmt_obj); @@ -1793,7 +2010,7 @@ check_agmts_are_alive(Replica *replica, ReplicaId rid, Slapi_Task *task) * Create the CLEANALLRUV extended op payload */ struct berval * -create_ruv_payload(char *value) +create_cleanruv_payload(char *value) { struct berval *req_data = NULL; BerElement *tmp_bere = NULL; @@ -1801,10 +2018,7 @@ create_ruv_payload(char *value) if ((tmp_bere = der_alloc()) == NULL){ goto error; } - if (ber_printf(tmp_bere, "{s", value) == -1){ - goto error; - } - if (ber_printf(tmp_bere, "}") == -1){ + if (ber_printf(tmp_bere, "{s}", value) == -1){ goto error; } if (ber_flatten(tmp_bere, &req_data) == -1){ @@ -1832,7 +2046,7 @@ done: * the CLEANALLRUV task. */ static void -replica_send_cleanruv_task(Repl_Agmt *agmt, ReplicaId rid, Slapi_Task *task) +replica_send_cleanruv_task(Repl_Agmt *agmt, cleanruv_data *clean_data) { Repl_Connection *conn; ConnResult crc = 0; @@ -1859,7 +2073,7 @@ replica_send_cleanruv_task(Repl_Agmt *agmt, ReplicaId rid, Slapi_Task *task) conn_delete_internal_ext(conn); return; } - val.bv_len = PR_snprintf(data, sizeof(data), "CLEANRUV%d", rid); + val.bv_len = PR_snprintf(data, sizeof(data), "CLEANRUV%d", clean_data->rid); sdn = agmt_get_replarea(agmt); mod.mod_op = LDAP_MOD_ADD|LDAP_MOD_BVALUES; mod.mod_type = "nsds5task"; @@ -1876,7 +2090,7 @@ replica_send_cleanruv_task(Repl_Agmt *agmt, ReplicaId rid, Slapi_Task *task) rc = ldap_modify_ext_s( ld, repl_dn, mods, NULL, NULL); if(rc != LDAP_SUCCESS){ - cleanruv_log(task, CLEANALLRUV_ID, "Failed to add CLEANRUV task replica " + cleanruv_log(clean_data->task, CLEANALLRUV_ID, "Failed to add CLEANRUV task replica " "(%s). You will need to manually run the CLEANRUV task on this replica (%s) error (%d)", agmt_get_long_name(agmt), agmt_get_hostname(agmt), rc); } @@ -1906,6 +2120,23 @@ is_cleaned_rid(ReplicaId rid) } int +is_pre_cleaned_rid(ReplicaId rid) +{ + int i; + + slapi_rwlock_rdlock(rid_lock); + for(i = 0; i < CLEANRIDSIZ && pre_cleaned_rids[i] != 0; i++){ + if(rid == pre_cleaned_rids[i]){ + slapi_rwlock_unlock(rid_lock); + return 1; + } + } + slapi_rwlock_unlock(rid_lock); + + return 0; +} + +int is_task_aborted(ReplicaId rid) { int i; @@ -1924,6 +2155,22 @@ is_task_aborted(ReplicaId rid) return 0; } +static void +preset_cleaned_rid(ReplicaId rid) +{ + int i; + + slapi_rwlock_wrlock(rid_lock); + for(i = 0; i < CLEANRIDSIZ; i++){ + if(pre_cleaned_rids[i] == 0){ + pre_cleaned_rids[i] = rid; + pre_cleaned_rids[i + 1] = 0; + break; + } + } + slapi_rwlock_unlock(rid_lock); +} + /* * Just add the rid to the in memory, as we don't want it to survive after a restart, * This prevent the changelog from sending updates from this rid, and the local ruv @@ -1948,7 +2195,7 @@ set_cleaned_rid(ReplicaId rid) * Add the rid and maxcsn to the repl config (so we can resume after a server restart) */ void -add_cleaned_rid(ReplicaId rid, Replica *r, char *maxcsn) +add_cleaned_rid(ReplicaId rid, Replica *r, char *maxcsn, char *forcing) { Slapi_PBlock *pb; struct berval *vals[2]; @@ -1965,7 +2212,7 @@ add_cleaned_rid(ReplicaId rid, Replica *r, char *maxcsn) /* * Write the rid & maxcsn to the config entry */ - val.bv_len = PR_snprintf(data, sizeof(data),"%d:%s", rid, maxcsn); + val.bv_len = PR_snprintf(data, sizeof(data),"%d:%s:%s", rid, maxcsn, forcing); dn = replica_get_dn(r); pb = slapi_pblock_new(); mod.mod_op = LDAP_MOD_ADD|LDAP_MOD_BVALUES; @@ -1977,13 +2224,11 @@ add_cleaned_rid(ReplicaId rid, Replica *r, char *maxcsn) mods[0] = &mod; mods[1] = NULL; - replica_add_cleanruv_data(r, val.bv_val); - slapi_modify_internal_set_pb (pb, dn, mods, NULL, NULL, repl_get_plugin_identity (PLUGIN_MULTIMASTER_REPLICATION), 0); slapi_modify_internal_pb (pb); slapi_pblock_get(pb, SLAPI_PLUGIN_INTOP_RESULT, &rc); - if (rc != LDAP_SUCCESS && rc != LDAP_TYPE_OR_VALUE_EXISTS){ + if (rc != LDAP_SUCCESS && rc != LDAP_TYPE_OR_VALUE_EXISTS && rc != LDAP_NO_SUCH_OBJECT){ slapi_log_error(SLAPI_LOG_FATAL, repl_plugin_name, "CleanAllRUV Task: failed to update replica " "config (%d), rid (%d)\n", rc, rid); } @@ -2036,7 +2281,7 @@ add_aborted_rid(ReplicaId rid, Replica *r, char *repl_root) repl_get_plugin_identity (PLUGIN_MULTIMASTER_REPLICATION), 0); slapi_modify_internal_pb (pb); slapi_pblock_get(pb, SLAPI_PLUGIN_INTOP_RESULT, &rc); - if (rc != LDAP_SUCCESS){ + if (rc != LDAP_SUCCESS && rc != LDAP_TYPE_OR_VALUE_EXISTS && rc != LDAP_NO_SUCH_OBJECT){ slapi_log_error(SLAPI_LOG_FATAL, repl_plugin_name, "Abort CleanAllRUV Task: failed to update " "replica config (%d), rid (%d)\n", rc, rid); } @@ -2047,7 +2292,7 @@ add_aborted_rid(ReplicaId rid, Replica *r, char *repl_root) } void -delete_aborted_rid(Replica *r, ReplicaId rid, char *repl_root){ +delete_aborted_rid(Replica *r, ReplicaId rid, char *repl_root, int skip){ Slapi_PBlock *pb; LDAPMod *mods[2]; LDAPMod mod; @@ -2061,117 +2306,126 @@ delete_aborted_rid(Replica *r, ReplicaId rid, char *repl_root){ if(r == NULL) return; - /* - * Remove this rid, and optimize the array - */ - slapi_rwlock_wrlock(abort_rid_lock); - for(i = 0; i < CLEANRIDSIZ && aborted_rids[i] != rid; i++); /* found rid, stop */ - for(; i < CLEANRIDSIZ; i++){ - /* rewrite entire array */ - aborted_rids[i] = aborted_rids[i + 1]; + if(skip){ + /* skip the deleting of the config, and just remove the in memory rid */ + slapi_rwlock_wrlock(abort_rid_lock); + for(i = 0; i < CLEANRIDSIZ && aborted_rids[i] != rid; i++); /* found rid, stop */ + for(; i < CLEANRIDSIZ; i++){ + /* rewrite entire array */ + aborted_rids[i] = aborted_rids[i + 1]; + } + slapi_rwlock_unlock(abort_rid_lock); + } else { + /* only remove the config, leave the in-memory rid */ + dn = replica_get_dn(r); + pb = slapi_pblock_new(); + data = PR_smprintf("%d:%s", (int)rid, repl_root); + + mod.mod_op = LDAP_MOD_DELETE|LDAP_MOD_BVALUES; + mod.mod_type = (char *)type_replicaAbortCleanRUV; + mod.mod_bvalues = vals; + vals [0] = &val; + vals [1] = NULL; + val.bv_val = data; + val.bv_len = strlen (data); + mods[0] = &mod; + mods[1] = NULL; + + slapi_modify_internal_set_pb(pb, dn, mods, NULL, NULL, repl_get_plugin_identity (PLUGIN_MULTIMASTER_REPLICATION), 0); + slapi_modify_internal_pb (pb); + slapi_pblock_get(pb, SLAPI_PLUGIN_INTOP_RESULT, &rc); + if (rc != LDAP_SUCCESS && rc != LDAP_NO_SUCH_OBJECT){ + slapi_log_error(SLAPI_LOG_FATAL, repl_plugin_name, "Abort CleanAllRUV Task: failed to remove replica " + "config (%d), rid (%d)\n", rc, rid); + } + slapi_pblock_destroy (pb); + slapi_ch_free_string(&dn); + slapi_ch_free_string(&data); } - slapi_rwlock_unlock(abort_rid_lock); +} + +/* + * Just remove the dse.ldif config, but we need to keep the cleaned rids in memory until we know we are done + */ +static void +delete_cleaned_rid_config(cleanruv_data *clean_data) +{ + Slapi_PBlock *pb; + LDAPMod *mods[2]; + LDAPMod mod; + struct berval *vals[2]; + struct berval val; + char data[CSN_STRSIZE + 15]; + char csnstr[CSN_STRSIZE]; + char *dn; + int rc; + /* * Prepare the mods for the config entry */ - dn = replica_get_dn(r); + csn_as_string(clean_data->maxcsn, PR_FALSE, csnstr); + val.bv_len = PR_snprintf(data, sizeof(data), "%d:%s:%s", (int)clean_data->rid, csnstr, clean_data->force); + dn = replica_get_dn(clean_data->replica); pb = slapi_pblock_new(); - data = PR_smprintf("%d:%s", (int)rid, repl_root); - mod.mod_op = LDAP_MOD_DELETE|LDAP_MOD_BVALUES; mod.mod_type = (char *)type_replicaAbortCleanRUV; + mod.mod_type = (char *)type_replicaCleanRUV; mod.mod_bvalues = vals; vals [0] = &val; vals [1] = NULL; val.bv_val = data; - val.bv_len = strlen (data); + mods[0] = &mod; mods[1] = NULL; - slapi_modify_internal_set_pb(pb, dn, mods, NULL, NULL, repl_get_plugin_identity (PLUGIN_MULTIMASTER_REPLICATION), 0); slapi_modify_internal_pb (pb); slapi_pblock_get(pb, SLAPI_PLUGIN_INTOP_RESULT, &rc); if (rc != LDAP_SUCCESS){ - slapi_log_error(SLAPI_LOG_FATAL, repl_plugin_name, "Abort CleanAllRUV Task: failed to remove replica " - "config (%d), rid (%d)\n", rc, rid); - } - slapi_pblock_destroy (pb); - slapi_ch_free_string(&dn); - slapi_ch_free_string(&data); -} + /* + * When aborting a task, we don't know if the "force" option was used. + * So we assume it is set to "no", but if this op fails, we'll try "yes" + */ + slapi_pblock_destroy (pb); + pb = slapi_pblock_new(); + memset(data,'0',sizeof(data)); + val.bv_len = PR_snprintf(data, sizeof(data), "%d:%s:yes", (int)clean_data->rid, csnstr); + slapi_modify_internal_set_pb(pb, dn, mods, NULL, NULL, repl_get_plugin_identity (PLUGIN_MULTIMASTER_REPLICATION), 0); + slapi_modify_internal_pb (pb); + slapi_pblock_get(pb, SLAPI_PLUGIN_INTOP_RESULT, &rc); + if (rc != LDAP_SUCCESS && rc != LDAP_NO_SUCH_OBJECT){ + slapi_log_error(SLAPI_LOG_FATAL, repl_plugin_name, "CleanAllRUV Task: failed to remove replica config " + "(%d), rid (%d)\n", rc, clean_data->rid); + } + } + slapi_pblock_destroy (pb); + slapi_ch_free_string(&dn); + } /* * Remove the rid from our list, and the config */ void -delete_cleaned_rid(Replica *r, ReplicaId rid, CSN *maxcsn) +remove_cleaned_rid(ReplicaId rid) { - Slapi_PBlock *pb; - Object *agmt_obj; - Repl_Agmt *agmt; - LDAPMod *mods[2]; - LDAPMod mod; - struct berval *vals[2]; - struct berval val; - char *dn; - char data[CSN_STRSIZE + 10]; - char csnstr[CSN_STRSIZE]; - int rc; int i; - - if(r == NULL || maxcsn == NULL) - return; - /* * Remove this rid, and optimize the array */ slapi_rwlock_wrlock(rid_lock); + for(i = 0; i < CLEANRIDSIZ && cleaned_rids[i] != rid; i++); /* found rid, stop */ for(; i < CLEANRIDSIZ; i++){ /* rewrite entire array */ cleaned_rids[i] = cleaned_rids[i + 1]; } - slapi_rwlock_unlock(rid_lock); - /* - * Prepare the mods for the config entry - */ - csn_as_string(maxcsn, PR_FALSE, csnstr); - val.bv_len = PR_snprintf(data, sizeof(data), "%d:%s", (int)rid, csnstr); - dn = replica_get_dn(r); - pb = slapi_pblock_new(); - mod.mod_op = LDAP_MOD_DELETE|LDAP_MOD_BVALUES; - mod.mod_type = (char *)type_replicaCleanRUV; - mod.mod_bvalues = vals; - vals [0] = &val; - vals [1] = NULL; - val.bv_val = data; - mods[0] = &mod; - mods[1] = NULL; - - replica_remove_cleanruv_data(r, data); - - slapi_modify_internal_set_pb(pb, dn, mods, NULL, NULL, repl_get_plugin_identity (PLUGIN_MULTIMASTER_REPLICATION), 0); - slapi_modify_internal_pb (pb); - slapi_pblock_get(pb, SLAPI_PLUGIN_INTOP_RESULT, &rc); - if (rc != LDAP_SUCCESS){ - slapi_log_error(SLAPI_LOG_FATAL, repl_plugin_name, "CleanAllRUV Task: failed to remove replica config " - "(%d), rid (%d)\n", rc, rid); - } - slapi_pblock_destroy (pb); - slapi_ch_free_string(&dn); - /* - * Now release the cleaned rid from the repl agmts - */ - agmt_obj = agmtlist_get_first_agreement_for_replica (r); - while (agmt_obj){ - agmt = (Repl_Agmt*)object_get_data (agmt_obj); - if(!agmt_is_enabled(agmt) || get_agmt_agreement_type(agmt) == REPLICA_TYPE_WINDOWS){ - agmt_obj = agmtlist_get_next_agreement_for_replica (r, agmt_obj); - continue; - } - agmt_set_cleanruv_data(agmt, rid, CLEANRUV_RELEASED); - agmt_obj = agmtlist_get_next_agreement_for_replica (r, agmt_obj); + /* now do the preset cleaned rids */ + for(i = 0; i < CLEANRIDSIZ && pre_cleaned_rids[i] != rid; i++); /* found rid, stop */ + for(; i < CLEANRIDSIZ; i++){ + /* rewrite entire array */ + pre_cleaned_rids[i] = pre_cleaned_rids[i + 1]; } + + slapi_rwlock_unlock(rid_lock); } /* @@ -2181,18 +2435,17 @@ int replica_cleanall_ruv_abort(Slapi_PBlock *pb, Slapi_Entry *e, Slapi_Entry *eAfter, int *returncode, char *returntext, void *arg) { - PRThread *thread = NULL; struct berval *payload = NULL; - Slapi_Task *task = NULL; - Replica *replica; - ReplicaId rid; cleanruv_data *data = NULL; + PRThread *thread = NULL; + Slapi_Task *task = NULL; Slapi_DN *sdn = NULL; + Replica *replica; + ReplicaId rid;; Object *r; - CSN *maxcsn = NULL; + const char *certify_all; const char *base_dn; const char *rid_str; - const char *certify_all; char *ridstr = NULL; int rc = SLAPI_DSE_CALLBACK_OK; @@ -2216,13 +2469,6 @@ replica_cleanall_ruv_abort(Slapi_PBlock *pb, Slapi_Entry *e, Slapi_Entry *eAfter rc = SLAPI_DSE_CALLBACK_ERROR; goto out; } - if ((base_dn = fetch_attr(e, "replica-base-dn", 0)) == NULL){ - PR_snprintf(returntext, SLAPI_DSE_RETURNTEXT_SIZE, "Missing required attr \"replica-base-dn\""); - cleanruv_log(task, ABORT_CLEANALLRUV_ID, "%s", returntext); - *returncode = LDAP_OBJECT_CLASS_VIOLATION; - rc = SLAPI_DSE_CALLBACK_ERROR; - goto out; - } certify_all = fetch_attr(e, "replica-certify-all", 0); /* * Check the rid @@ -2236,6 +2482,29 @@ replica_cleanall_ruv_abort(Slapi_PBlock *pb, Slapi_Entry *e, Slapi_Entry *eAfter rc = SLAPI_DSE_CALLBACK_ERROR; goto out; } + if ((base_dn = fetch_attr(e, "replica-base-dn", 0)) == NULL){ + PR_snprintf(returntext, SLAPI_DSE_RETURNTEXT_SIZE, "Missing required attr \"replica-base-dn\""); + cleanruv_log(task, ABORT_CLEANALLRUV_ID, "%s", returntext); + *returncode = LDAP_OBJECT_CLASS_VIOLATION; + rc = SLAPI_DSE_CALLBACK_ERROR; + goto out; + } + if(!is_cleaned_rid(rid) && !is_pre_cleaned_rid(rid)){ + /* we are not cleaning this rid */ + PR_snprintf(returntext, SLAPI_DSE_RETURNTEXT_SIZE, "Replica id (%d) is not being cleaned, nothing to abort.", rid); + cleanruv_log(task, ABORT_CLEANALLRUV_ID, "%s", returntext); + *returncode = LDAP_UNWILLING_TO_PERFORM; + rc = SLAPI_DSE_CALLBACK_ERROR; + goto out; + } + if(is_task_aborted(rid)){ + /* we are already aborting this rid */ + PR_snprintf(returntext, SLAPI_DSE_RETURNTEXT_SIZE, "Replica id (%d) is already being aborted", rid); + cleanruv_log(task, ABORT_CLEANALLRUV_ID, "%s", returntext); + *returncode = LDAP_UNWILLING_TO_PERFORM; + rc = SLAPI_DSE_CALLBACK_ERROR; + goto out; + } /* * Get the replica object */ @@ -2248,7 +2517,7 @@ replica_cleanall_ruv_abort(Slapi_PBlock *pb, Slapi_Entry *e, Slapi_Entry *eAfter goto out; } /* - * Check verify value + * Check certify value */ if(certify_all){ if(strcasecmp(certify_all,"yes") && strcasecmp(certify_all,"no")){ @@ -2260,13 +2529,13 @@ replica_cleanall_ruv_abort(Slapi_PBlock *pb, Slapi_Entry *e, Slapi_Entry *eAfter goto out; } } else { - certify_all = "no"; + certify_all = "yes"; } /* * Create payload */ ridstr = slapi_ch_smprintf("%d:%s:%s", rid, base_dn, certify_all); - payload = create_ruv_payload(ridstr); + payload = create_cleanruv_payload(ridstr); if(payload == NULL){ cleanruv_log(task, ABORT_CLEANALLRUV_ID, "Failed to create extended op payload, aborting task"); @@ -2278,8 +2547,6 @@ replica_cleanall_ruv_abort(Slapi_PBlock *pb, Slapi_Entry *e, Slapi_Entry *eAfter * Stop the cleaning, and delete the rid */ replica = (Replica*)object_get_data (r); - maxcsn = replica_get_cleanruv_maxcsn(replica, rid); - delete_cleaned_rid(replica, rid, maxcsn); add_aborted_rid(rid, replica, (char *)base_dn); stop_ruv_cleaning(); /* @@ -2308,11 +2575,11 @@ replica_cleanall_ruv_abort(Slapi_PBlock *pb, Slapi_Entry *e, Slapi_Entry *eAfter object_release(r); cleanruv_log(task, ABORT_CLEANALLRUV_ID,"Unable to create abort thread. Aborting task."); *returncode = LDAP_OPERATIONS_ERROR; + slapi_ch_free_string(&data->certify); rc = SLAPI_DSE_CALLBACK_ERROR; } out: - csn_free(&maxcsn); slapi_ch_free_string(&ridstr); slapi_sdn_free(&sdn); @@ -2337,6 +2604,8 @@ replica_abort_task_thread(void *arg) int interval = 10; int release_it = 0; + cleanruv_log(data->task, ABORT_CLEANALLRUV_ID, "Aborting task for rid(%d)...",data->rid); + /* * Need to build the replica from the dn */ @@ -2414,8 +2683,12 @@ done: /* * Clean up the config */ - delete_aborted_rid(data->replica, data->rid, data->repl_root); - cleanruv_log(data->task, ABORT_CLEANALLRUV_ID, "Successfully aborted cleanAllRUV task for rid(%d)", data->rid); + delete_aborted_rid(data->replica, data->rid, data->repl_root, 1); /* delete just the config, leave rid in memory */ + if(strcasecmp(data->certify, "yes") == 0){ + check_replicas_are_done_aborting(data); + } + delete_aborted_rid(data->replica, data->rid, data->repl_root, 0); /* remove the in-memory aborted rid */ + cleanruv_log(data->task, ABORT_CLEANALLRUV_ID, "Successfully aborted task for rid(%d)", data->rid); } if(data->task){ slapi_task_finish(data->task, agmt_not_notified); @@ -2437,10 +2710,10 @@ replica_cleanallruv_send_abort_extop(Repl_Agmt *ra, Slapi_Task *task, struct ber Repl_Connection *conn = NULL; ConnResult crc = 0; int msgid = 0; - int rc = 0; + int rc = -1; if((conn = conn_new(ra)) == NULL){ - return -1; + return rc; } if(conn_connect(conn) == CONN_OPERATION_SUCCESS){ crc = conn_send_extended_operation(conn, REPL_ABORT_CLEANRUV_OID, payload, NULL, &msgid); @@ -2462,186 +2735,262 @@ replica_cleanallruv_send_abort_extop(Repl_Agmt *ra, Slapi_Task *task, struct ber static int -replica_cleanallruv_send_extop(Repl_Agmt *ra, ReplicaId rid, Slapi_Task *task, struct berval *payload, int check_result) +replica_cleanallruv_send_extop(Repl_Agmt *ra, cleanruv_data *clean_data, int check_result) { Repl_Connection *conn = NULL; ConnResult crc = 0; int msgid = 0; - int rc = 0; + int rc = -1; if((conn = conn_new(ra)) == NULL){ - return -1; + return rc; } if(conn_connect(conn) == CONN_OPERATION_SUCCESS){ - crc = conn_send_extended_operation(conn, REPL_CLEANRUV_OID, payload, NULL, &msgid); + crc = conn_send_extended_operation(conn, REPL_CLEANRUV_OID, clean_data->payload, NULL, &msgid); if(crc == CONN_OPERATION_SUCCESS && check_result){ struct berval *retsdata = NULL; char *retoid = NULL; crc = conn_read_result_ex(conn, &retoid, &retsdata, NULL, msgid, NULL, 1); if (CONN_OPERATION_SUCCESS == crc ){ - struct berval **ruv_bervals = NULL; - struct berval *data = NULL; - char *data_guid = NULL; - - decode_repl_ext_response(retsdata, &rc, &ruv_bervals, &data_guid, &data); - /* just free everything, we only wanted "rc" */ - slapi_ch_free_string(&data_guid); - if(data) - ber_bvfree(data); - if (ruv_bervals) - ber_bvecfree(ruv_bervals); - - if(rc == 0 ){ /* rc == 1 is success */ - cleanruv_log(task, CLEANALLRUV_ID,"Replica %s does not support the CLEANALLRUV task. Sending replica CLEANRUV task...", + char *response = NULL; + + decode_cleanruv_payload(retsdata, &response); + if(response && strcmp(response,CLEANRUV_ACCEPTED) == 0){ + /* extop was accepted */ + rc = 0; + } else { + cleanruv_log(clean_data->task, CLEANALLRUV_ID,"Replica %s does not support the CLEANALLRUV task. Sending replica CLEANRUV task...", slapi_sdn_get_dn(agmt_get_dn_byref(ra))); /* * Ok, this replica doesn't know about CLEANALLRUV, so just manually * add the CLEANRUV task to the replica. */ - replica_send_cleanruv_task(ra, rid, task); - } else { - /* extop was accepted */ - rc = 0; + replica_send_cleanruv_task(ra, clean_data); } - if (NULL != retoid) - ldap_memfree(retoid); if (NULL != retsdata) ber_bvfree(retsdata); + slapi_ch_free_string(&retoid); + slapi_ch_free_string(&response); } - agmt_set_cleanruv_data(ra, rid, CLEANRUV_NOTIFIED); } else { /* * success or failure, just return the error code */ rc = crc; } - } else { - rc =-1; } conn_delete_internal_ext(conn); return rc; } +static CSN* +replica_cleanallruv_find_maxcsn(Replica *replica, ReplicaId rid, char *basedn) +{ + Object *agmt_obj; + Repl_Agmt *agmt; + char *rid_text; + CSN *maxcsn = NULL, *topcsn = NULL; + int done = 1, found = 0; + int interval = 10; + + rid_text = slapi_ch_smprintf("%d", rid); + + while(done && !is_task_aborted(rid) && !slapi_is_shutting_down()){ + agmt_obj = agmtlist_get_first_agreement_for_replica (replica); + if(agmt_obj == NULL){ + break; + } + while (agmt_obj && !slapi_is_shutting_down()){ + agmt = (Repl_Agmt*)object_get_data (agmt_obj); + if(!agmt_is_enabled(agmt) || get_agmt_agreement_type(agmt) == REPLICA_TYPE_WINDOWS){ + agmt_obj = agmtlist_get_next_agreement_for_replica (replica, agmt_obj); + done = 0; + continue; + } + if(replica_cleanallruv_get_replica_maxcsn(agmt, rid_text, basedn, &maxcsn) == 0){ + if(maxcsn == NULL){ + agmt_obj = agmtlist_get_next_agreement_for_replica (replica, agmt_obj); + continue; + } + found = 1; + if(topcsn == NULL){ + topcsn = maxcsn; + } else { + if(csn_compare(topcsn, maxcsn) < 0){ + csn_free(&topcsn); + topcsn = maxcsn; + } else { + csn_free(&maxcsn); + } + } + done = 0; + } else { + done = 1; + break; + } + agmt_obj = agmtlist_get_next_agreement_for_replica (replica, agmt_obj); + } /* agmt while */ + if(done == 0 || is_task_aborted(rid) ){ + break; + } + if(!found){ + /* we could not find any maxcsn's - already cleaned? */ + return NULL; + } + slapi_log_error(SLAPI_LOG_FATAL, repl_plugin_name, "CleanAllRUV Task: replica_cleanallruv_find_maxcsn: Not all replicas online, retrying in %d seconds\n",interval); + PR_Lock( notify_lock ); + PR_WaitCondVar( notify_cvar, PR_SecondsToInterval(interval) ); + PR_Unlock( notify_lock ); + + if(interval < 14400){ /* 4 hour max */ + interval = interval * 2; + } else { + interval = 14400; + } + } + slapi_ch_free_string(&rid_text); + + if(is_task_aborted(rid)){ + return NULL; + } + + return topcsn; +} + static int -replica_cleanallruv_check_maxcsn(Repl_Agmt *agmt, char *rid_text, char *maxcsn, Slapi_Task *task) +replica_cleanallruv_get_replica_maxcsn(Repl_Agmt *agmt, char *rid_text, char *basedn, CSN **csn) { Repl_Connection *conn = NULL; - LDAP *ld; - Slapi_DN *sdn; - struct berval **vals; - LDAPMessage *result = NULL, *entry = NULL; - BerElement *ber; - char *attrs[2]; - char *attr = NULL; - char *iter = NULL; - char *ruv_part = NULL; - int found_rid = 0; - int part_count = 0; - int rc = 0, i; + ConnResult crc = -1; + struct berval *payload = NULL; + CSN *maxcsn = NULL; + char *data = NULL; + int msgid = 0; if((conn = conn_new(agmt)) == NULL){ return -1; } + data = slapi_ch_smprintf("%s:%s",rid_text, basedn); + payload = create_cleanruv_payload(data); + if(conn_connect(conn) == CONN_OPERATION_SUCCESS){ - attrs[0] = "nsds50ruv"; - attrs[1] = NULL; - ld = conn_get_ldap(conn); - if(ld == NULL){ - conn_delete_internal_ext(conn); - return -1; - } - sdn = agmt_get_replarea(agmt); - rc = ldap_search_ext_s(ld, slapi_sdn_get_dn(sdn), LDAP_SCOPE_SUBTREE, - "(&(nsuniqueid=ffffffff-ffffffff-ffffffff-ffffffff)(objectclass=nstombstone))", - attrs, 0, NULL, NULL, NULL, 0, &result); - slapi_sdn_free(&sdn); - if(rc != LDAP_SUCCESS){ - cleanruv_log(task, CLEANALLRUV_ID,"Failed to contact " - "agmt (%s) error (%d), will retry later.", agmt_get_long_name(agmt), rc); - conn_delete_internal_ext(conn); - return -1; - } - entry = ldap_first_entry( ld, result ); - if ( entry != NULL ) { - for ( attr = ldap_first_attribute( ld, entry, &ber ); attr != NULL; attr = ldap_next_attribute( ld, entry, ber ) ){ - /* make sure the attribute is nsds50ruv */ - if(strcasecmp(attr,"nsds50ruv") != 0){ - ldap_memfree( attr ); - continue; + crc = conn_send_extended_operation(conn, REPL_CLEANRUV_GET_MAXCSN_OID, payload, NULL, &msgid); + if(crc == CONN_OPERATION_SUCCESS){ + struct berval *retsdata = NULL; + char *retoid = NULL; + + crc = conn_read_result_ex(conn, &retoid, &retsdata, NULL, msgid, NULL, 1); + if (CONN_OPERATION_SUCCESS == crc ){ + char *remote_maxcsn = NULL; + + decode_cleanruv_payload(retsdata, &remote_maxcsn); + if(remote_maxcsn && strcmp(remote_maxcsn, CLEANRUV_NO_MAXCSN)){ + maxcsn = csn_new(); + csn_init_by_string(maxcsn, remote_maxcsn); + *csn = maxcsn; + } else { + /* no csn */ + *csn = NULL; } - found_rid = 0; - if ((vals = ldap_get_values_len( ld, entry, attr)) != NULL ) { - for ( i = 0; vals[i] && vals[i]->bv_val; i++ ) { - /* look for this replica */ - if(strstr(vals[i]->bv_val, rid_text)){ - found_rid = 1; - /* get the max csn compare it to our known max csn */ - ruv_part = ldap_utf8strtok_r(vals[i]->bv_val, " ", &iter); - for(part_count = 1; ruv_part && part_count < 5; part_count++){ - ruv_part = ldap_utf8strtok_r(iter, " ", &iter); - } - if(part_count == 5 && ruv_part){ - /* we have the maxcsn */ - if(strcmp(ruv_part, maxcsn)){ - /* we are not caught up yet, free, and return */ - ldap_value_free_len(vals); - ldap_memfree( attr ); - ldap_msgfree( result ); - if(ber){ - ber_free( ber, 0 ); - } - conn_delete_internal_ext(conn); - return -1; - } else { - /* ok this replica has all the updates from the deleted replica */ - rc = 0; - } - } else { - /* there is no maxcsn for this rid - treat it as caught up */ - rc = 0; - } - } - } - if(!found_rid){ - /* must have been cleaned already */ + slapi_ch_free_string(&retoid); + slapi_ch_free_string(&remote_maxcsn); + if (NULL != retsdata) + ber_bvfree(retsdata); + } + } + } + conn_delete_internal_ext(conn); + slapi_ch_free_string(&data); + if(payload) + ber_bvfree(payload); + + return (int)crc; +} + +static int +replica_cleanallruv_check_maxcsn(Repl_Agmt *agmt, char *basedn, char *rid_text, char *maxcsn, Slapi_Task *task) +{ + Repl_Connection *conn = NULL; + ConnResult crc = 0; + struct berval *payload = NULL; + char *data = NULL; + int msgid = 0; + int rc = -1; + + if((conn = conn_new(agmt)) == NULL){ + return -1; + } + + data = slapi_ch_smprintf("%s:%s",rid_text, basedn); + payload = create_cleanruv_payload(data); + + if(conn_connect(conn) == CONN_OPERATION_SUCCESS){ + crc = conn_send_extended_operation(conn, REPL_CLEANRUV_GET_MAXCSN_OID, payload, NULL, &msgid); + if(crc == CONN_OPERATION_SUCCESS){ + struct berval *retsdata = NULL; + char *retoid = NULL; + + crc = conn_read_result_ex(conn, &retoid, &retsdata, NULL, msgid, NULL, 1); + if (CONN_OPERATION_SUCCESS == crc ){ + char *remote_maxcsn = NULL; + + decode_cleanruv_payload(retsdata, &remote_maxcsn); + if(remote_maxcsn && strcmp(remote_maxcsn, CLEANRUV_NO_MAXCSN)){ + CSN *max, *repl_max; + + max = csn_new(); + repl_max = csn_new(); + csn_init_by_string(max, maxcsn); + csn_init_by_string(repl_max, remote_maxcsn); + if(csn_compare (repl_max, max) < 0){ + /* we are not caught up yet, free, and return */ + cleanruv_log(task, CLEANALLRUV_ID,"Replica maxcsn (%s) is not caught up with deleted replica's maxcsn(%s)", + remote_maxcsn, maxcsn); + rc = -1; + } else { + /* ok this replica is caught up */ rc = 0; } - ldap_value_free_len(vals); + csn_free(&max); + csn_free(&repl_max); + } else { + /* no remote_maxcsn - return success */ + rc = 0; } - ldap_memfree( attr ); - } - if ( ber != NULL ) { - ber_free( ber, 0 ); - } - } - if(result) - ldap_msgfree( result ); - } else { - rc = -1; + slapi_ch_free_string(&retoid); + slapi_ch_free_string(&remote_maxcsn); + if (NULL != retsdata) + ber_bvfree(retsdata); + } + } } conn_delete_internal_ext(conn); + slapi_ch_free_string(&data); + if(payload) + ber_bvfree(payload); - return rc; + return rc; } + static int replica_cleanallruv_replica_alive(Repl_Agmt *agmt) { Repl_Connection *conn = NULL; LDAP *ld = NULL; LDAPMessage *result = NULL; - int rc = 0; + int rc = -1; if((conn = conn_new(agmt)) == NULL){ - return -1; + return rc; } if(conn_connect(conn) == CONN_OPERATION_SUCCESS){ ld = conn_get_ldap(conn); if(ld == NULL){ - slapi_log_error(SLAPI_LOG_REPL, repl_plugin_name, "cleanAllRUV_task: failed to get LDAP " + slapi_log_error(SLAPI_LOG_REPL, repl_plugin_name, "CleanAllRUV_task: failed to get LDAP " "handle from the replication agmt (%s). Moving on to the next agmt.\n",agmt_get_long_name(agmt)); conn_delete_internal_ext(conn); return -1; @@ -2655,8 +3004,6 @@ replica_cleanallruv_replica_alive(Repl_Agmt *agmt) } if(result) ldap_msgfree( result ); - } else { - rc = -1; } conn_delete_internal_ext(conn); @@ -2664,83 +3011,51 @@ replica_cleanallruv_replica_alive(Repl_Agmt *agmt) } static int -replica_cleanallruv_check_ruv(Repl_Agmt *ra, char *rid_text, Slapi_Task *task) +replica_cleanallruv_check_ruv(char *repl_root, Repl_Agmt *agmt, char *rid_text, Slapi_Task *task) { Repl_Connection *conn = NULL; - BerElement *ber = NULL; - struct berval **vals = NULL; - LDAPMessage *result = NULL, *entry = NULL; - LDAP *ld = NULL; - Slapi_DN *sdn; - char *attrs[2]; - char *attr = NULL; - int rc = 0, i; + ConnResult crc = 0; + struct berval *payload = NULL; + char *data = NULL; + int msgid = 0; + int rc = -1; - if((conn = conn_new(ra)) == NULL){ - return -1; + if((conn = conn_new(agmt)) == NULL){ + return rc; } + + data = slapi_ch_smprintf("%s:%s",rid_text, repl_root); + payload = create_cleanruv_payload(data); + if(conn_connect(conn) == CONN_OPERATION_SUCCESS){ - attrs[0] = "nsds50ruv"; - attrs[1] = NULL; - ld = conn_get_ldap(conn); - if(ld == NULL){ - cleanruv_log(task, CLEANALLRUV_ID,"Failed to get LDAP handle from " - "the replication agmt (%s). Moving on to the next agmt.",agmt_get_long_name(ra)); - rc = -1; - goto done; - } + crc = conn_send_extended_operation(conn, REPL_CLEANRUV_GET_MAXCSN_OID, payload, NULL, &msgid); + if(crc == CONN_OPERATION_SUCCESS){ + struct berval *retsdata = NULL; + char *retoid = NULL; - sdn = agmt_get_replarea(ra); - rc = ldap_search_ext_s(ld, slapi_sdn_get_dn(sdn), LDAP_SCOPE_SUBTREE, - "(&(nsuniqueid=ffffffff-ffffffff-ffffffff-ffffffff)(objectclass=nstombstone))", - attrs, 0, NULL, NULL, NULL, 0, &result); - slapi_sdn_free(&sdn); - if(rc != LDAP_SUCCESS){ - cleanruv_log(task, CLEANALLRUV_ID,"Failed to contact " - "agmt (%s) error (%d), will retry later.", agmt_get_long_name(ra), rc); - rc = -1; - goto done; - } - entry = ldap_first_entry( ld, result ); - if ( entry != NULL ) { - for ( attr = ldap_first_attribute( ld, entry, &ber ); attr != NULL; attr = ldap_next_attribute( ld, entry, ber ) ){ - /* make sure the attribute is nsds50ruv */ - if(strcasecmp(attr,"nsds50ruv") != 0){ - ldap_memfree( attr ); - continue; - } - if ((vals = ldap_get_values_len( ld, entry, attr)) != NULL ) { - for ( i = 0; vals[i] && vals[i]->bv_val; i++ ) { - /* look for this replica */ - if(strstr(vals[i]->bv_val, rid_text)){ - /* rid has not been cleaned yet, free and return */ - rc = -1; - ldap_value_free_len(vals); - ldap_memfree( attr ); - if ( ber != NULL ) { - ber_free( ber, 0 ); - ber = NULL; - } - goto done; - } else { - rc = 0; - } - } - ldap_value_free_len(vals); + crc = conn_read_result_ex(conn, &retoid, &retsdata, NULL, msgid, NULL, 1); + if (CONN_OPERATION_SUCCESS == crc ){ + char *remote_maxcsn = NULL; + + decode_cleanruv_payload(retsdata, &remote_maxcsn); + if(remote_maxcsn && strcmp(remote_maxcsn, CLEANRUV_NO_MAXCSN)){ + /* remote replica still has dirty RUV element */ + rc = -1; + } else { + /* no maxcsn = we're clean */ + rc = 0; } - ldap_memfree( attr ); - } /* for loop */ - if ( ber != NULL ) { - ber_free( ber, 0 ); + slapi_ch_free_string(&retoid); + slapi_ch_free_string(&remote_maxcsn); + if (NULL != retsdata) + ber_bvfree(retsdata); } } -done: - if(result) - ldap_msgfree( result ); - } else { - return -1; } conn_delete_internal_ext(conn); + slapi_ch_free_string(&data); + if(payload) + ber_bvfree(payload); return rc; } diff --git a/ldap/servers/plugins/replication/repl_extop.c b/ldap/servers/plugins/replication/repl_extop.c index 3a6f422..e842c62 100644 --- a/ldap/servers/plugins/replication/repl_extop.c +++ b/ldap/servers/plugins/replication/repl_extop.c @@ -1451,7 +1451,6 @@ multimaster_extop_abort_cleanruv(Slapi_PBlock *pb) cleanruv_data *data; Replica *r; ReplicaId rid; - CSN *maxcsn = NULL; struct berval *extop_payload = NULL; char *extop_oid; char *repl_root; @@ -1463,7 +1462,7 @@ multimaster_extop_abort_cleanruv(Slapi_PBlock *pb) slapi_pblock_get(pb, SLAPI_EXT_OP_REQ_OID, &extop_oid); slapi_pblock_get(pb, SLAPI_EXT_OP_REQ_VALUE, &extop_payload); - if (NULL == extop_oid || strcmp(extop_oid, REPL_CLEANRUV_OID) != 0 || + if (NULL == extop_oid || strcmp(extop_oid, REPL_ABORT_CLEANRUV_OID) != 0 || NULL == extop_payload || NULL == extop_payload->bv_val){ /* something is wrong, error out */ return LDAP_OPERATIONS_ERROR; @@ -1472,24 +1471,24 @@ multimaster_extop_abort_cleanruv(Slapi_PBlock *pb) * Decode the payload, and grab our settings */ if(decode_cleanruv_payload(extop_payload, &payload)){ - slapi_log_error(SLAPI_LOG_FATAL, repl_plugin_name, "Abort cleanAllRUV task: failed to decode payload. Aborting ext op\n"); + slapi_log_error(SLAPI_LOG_FATAL, repl_plugin_name, "Abort CleanAllRUV Task: failed to decode payload. Aborting ext op\n"); return LDAP_OPERATIONS_ERROR; } rid = atoi(ldap_utf8strtok_r(payload, ":", &iter)); repl_root = ldap_utf8strtok_r(iter, ":", &iter); certify_all = ldap_utf8strtok_r(iter, ":", &iter); - if(!is_cleaned_rid(rid) || is_task_aborted(rid)){ + if(!is_cleaned_rid(rid) || !is_pre_cleaned_rid(rid) || is_task_aborted(rid)){ /* This replica has already been aborted, or was never cleaned, or already finished cleaning */ goto out; } else { - slapi_log_error(SLAPI_LOG_FATAL, repl_plugin_name, "Abort cleanAllRUV task: aborting cleanallruv task for rid(%d)\n", rid); + slapi_log_error(SLAPI_LOG_FATAL, repl_plugin_name, "Abort CleanAllRUV Task: aborting cleanallruv task for rid(%d)\n", rid); } /* * Get the node, so we can get the replica and its agreements */ if((mtnode_ext = replica_config_get_mtnode_by_dn(repl_root)) == NULL){ - slapi_log_error(SLAPI_LOG_FATAL, repl_plugin_name, "Abort cleanAllRUV task: failed to get replication node " + slapi_log_error(SLAPI_LOG_FATAL, repl_plugin_name, "Abort CleanAllRUV Task: failed to get replication node " "from (%s), aborting operation\n", repl_root); rc = LDAP_OPERATIONS_ERROR; goto out; @@ -1498,14 +1497,14 @@ multimaster_extop_abort_cleanruv(Slapi_PBlock *pb) object_acquire (mtnode_ext->replica); release_it = 1; } else { - slapi_log_error(SLAPI_LOG_FATAL, repl_plugin_name, "Abort cleanAllRUV task: replica is missing from (%s), " + slapi_log_error(SLAPI_LOG_FATAL, repl_plugin_name, "Abort CleanAllRUV Task: replica is missing from (%s), " "aborting operation\n",repl_root); rc = LDAP_OPERATIONS_ERROR; goto out; } r = (Replica*)object_get_data (mtnode_ext->replica); if(r == NULL){ - slapi_log_error(SLAPI_LOG_FATAL, repl_plugin_name, "Abort cleanAllRUV task: replica is NULL, aborting task\n"); + slapi_log_error(SLAPI_LOG_FATAL, repl_plugin_name, "Abort CleanAllRUV Task: replica is NULL, aborting task\n"); rc = LDAP_OPERATIONS_ERROR; goto out; } @@ -1514,7 +1513,7 @@ multimaster_extop_abort_cleanruv(Slapi_PBlock *pb) */ data = (cleanruv_data*)slapi_ch_calloc(1, sizeof(cleanruv_data)); if (data == NULL) { - slapi_log_error( SLAPI_LOG_REPL, repl_plugin_name, "Abort cleanAllRUV task: failed to allocate " + slapi_log_error( SLAPI_LOG_REPL, repl_plugin_name, "Abort CleanAllRUV Task: failed to allocate " "abort_cleanruv_data. Aborting task.\n"); rc = LDAP_OPERATIONS_ERROR; goto out; @@ -1528,11 +1527,8 @@ multimaster_extop_abort_cleanruv(Slapi_PBlock *pb) data->repl_root = slapi_ch_strdup(repl_root); data->certify = slapi_ch_strdup(certify_all); /* - * Stop the cleaning, and delete the rid + * Set the aborted rid and stop the cleaning */ - maxcsn = replica_get_cleanruv_maxcsn(r, rid); - delete_cleaned_rid(r, rid, maxcsn); - csn_free(&maxcsn); add_aborted_rid(rid, r, repl_root); stop_ruv_cleaning(); /* @@ -1542,7 +1538,7 @@ multimaster_extop_abort_cleanruv(Slapi_PBlock *pb) (void *)data, PR_PRIORITY_NORMAL, PR_GLOBAL_THREAD, PR_UNJOINABLE_THREAD, SLAPD_DEFAULT_THREAD_STACKSIZE); if (thread == NULL) { - slapi_log_error( SLAPI_LOG_REPL, repl_plugin_name, "Abort cleanAllRUV task: unable to create abort " + slapi_log_error( SLAPI_LOG_REPL, repl_plugin_name, "Abort CleanAllRUV Task: unable to create abort " "thread. Aborting task.\n"); release_it = 1; /* have to release mtnode_ext->replica now */ slapi_ch_free_string(&data->repl_root); @@ -1583,10 +1579,11 @@ multimaster_extop_cleanruv(Slapi_PBlock *pb) struct berval *extop_payload; struct berval *resp_bval = NULL; BerElement *resp_bere = NULL; - char *extop_oid; - char *repl_root; char *payload = NULL; char *csnstr = NULL; + char *force = NULL; + char *extop_oid; + char *repl_root; char *iter; int release_it = 0; int rid = 0; @@ -1604,28 +1601,31 @@ multimaster_extop_cleanruv(Slapi_PBlock *pb) * Decode the payload */ if(decode_cleanruv_payload(extop_payload, &payload)){ - slapi_log_error(SLAPI_LOG_FATAL, repl_plugin_name, "cleanAllRUV_task: failed to decode payload. Aborting ext op\n"); + slapi_log_error(SLAPI_LOG_FATAL, repl_plugin_name, "CleanAllRUV Task: failed to decode payload. Aborting ext op\n"); goto free_and_return; } rid = atoi(ldap_utf8strtok_r(payload, ":", &iter)); repl_root = ldap_utf8strtok_r(iter, ":", &iter); csnstr = ldap_utf8strtok_r(iter, ":", &iter); + force = ldap_utf8strtok_r(iter, ":", &iter); + if(force == NULL){ + force = "no"; + } maxcsn = csn_new(); csn_init_by_string(maxcsn, csnstr); /* * If we already cleaned this server, just return success */ - if(is_cleaned_rid(rid)){ + if(is_cleaned_rid(rid) || is_pre_cleaned_rid(rid) || is_task_aborted(rid)){ csn_free(&maxcsn); rc = LDAP_SUCCESS; goto free_and_return; } - /* * Get the node, so we can get the replica and its agreements */ if((mtnode_ext = replica_config_get_mtnode_by_dn(repl_root)) == NULL){ - slapi_log_error(SLAPI_LOG_FATAL, repl_plugin_name, "cleanAllRUV_task: failed to get replication node " + slapi_log_error(SLAPI_LOG_FATAL, repl_plugin_name, "CleanAllRUV Task: failed to get replication node " "from (%s), aborting operation\n", repl_root); goto free_and_return; } @@ -1634,14 +1634,14 @@ multimaster_extop_cleanruv(Slapi_PBlock *pb) object_acquire (mtnode_ext->replica); release_it = 1; } else { - slapi_log_error(SLAPI_LOG_FATAL, repl_plugin_name, "cleanAllRUV_task: replica is missing from (%s), " + slapi_log_error(SLAPI_LOG_FATAL, repl_plugin_name, "CleanAllRUV Task: replica is missing from (%s), " "aborting operation\n",repl_root); goto free_and_return; } r = (Replica*)object_get_data (mtnode_ext->replica); if(r == NULL){ - slapi_log_error(SLAPI_LOG_FATAL, repl_plugin_name, "cleanAllRUV_task: replica is NULL, aborting task\n"); + slapi_log_error(SLAPI_LOG_FATAL, repl_plugin_name, "CleanAllRUV Task: replica is NULL, aborting task\n"); goto free_and_return; } @@ -1651,10 +1651,10 @@ multimaster_extop_cleanruv(Slapi_PBlock *pb) * * This will also release mtnode_ext->replica */ - slapi_log_error(SLAPI_LOG_FATAL, repl_plugin_name, "cleanAllRUV_task: launching cleanAllRUV thread...\n"); + slapi_log_error(SLAPI_LOG_FATAL, repl_plugin_name, "CleanAllRUV Task: launching cleanAllRUV thread...\n"); data = (cleanruv_data*)slapi_ch_calloc(1, sizeof(cleanruv_data)); if (data == NULL) { - slapi_log_error( SLAPI_LOG_FATAL, repl_plugin_name, "cleanAllRUV_task: failed to allocate " + slapi_log_error( SLAPI_LOG_FATAL, repl_plugin_name, "CleanAllRUV Task: failed to allocate " "cleanruv_Data\n"); goto free_and_return; } @@ -1664,15 +1664,19 @@ multimaster_extop_cleanruv(Slapi_PBlock *pb) data->task = NULL; data->maxcsn = maxcsn; data->payload = slapi_ch_bvdup(extop_payload); + data->force = slapi_ch_strdup(force); + data->repl_root = slapi_ch_strdup(repl_root); thread = PR_CreateThread(PR_USER_THREAD, replica_cleanallruv_thread_ext, (void *)data, PR_PRIORITY_NORMAL, PR_GLOBAL_THREAD, PR_UNJOINABLE_THREAD, SLAPD_DEFAULT_THREAD_STACKSIZE); if (thread == NULL) { - slapi_log_error( SLAPI_LOG_FATAL, repl_plugin_name, "cleanAllRUV_task: unable to create cleanAllRUV " + slapi_log_error( SLAPI_LOG_FATAL, repl_plugin_name, "CleanAllRUV Task: unable to create cleanAllRUV " "monitoring thread. Aborting task.\n"); ber_bvfree(data->payload); data->payload = NULL; + slapi_ch_free_string(&data->force); + slapi_ch_free_string(&data->repl_root); slapi_ch_free((void **)&data); } else { release_it = 0; /* thread will release data->repl_obj == mtnode_ext->replica */ @@ -1694,18 +1698,18 @@ multimaster_extop_cleanruv(Slapi_PBlock *pb) /* we've already been cleaned */ break; } - slapi_log_error(SLAPI_LOG_FATAL, repl_plugin_name, "cleanAllRUV_task: checking if we're caught up...\n"); - if(ruv_covers_csn_cleanallruv(ruv,maxcsn) || csn_get_replicaid(maxcsn) == 0){ + slapi_log_error(SLAPI_LOG_FATAL, repl_plugin_name, "CleanAllRUV Task: checking if we're caught up...\n"); + if(ruv_covers_csn_cleanallruv(ruv,maxcsn) || csn_get_replicaid(maxcsn) == 0|| strcmp(force,"yes") == 0){ /* We are caught up */ break; } else { char csnstr[CSN_STRSIZE]; csn_as_string(maxcsn, PR_FALSE, csnstr); - slapi_log_error( SLAPI_LOG_FATAL, repl_plugin_name, "cleanAllRUV_task: not ruv caught up maxcsn(%s)\n", csnstr); + slapi_log_error( SLAPI_LOG_FATAL, repl_plugin_name, "CleanAllRUV Task: not ruv caught up maxcsn(%s)\n", csnstr); } DS_Sleep(PR_SecondsToInterval(5)); } - slapi_log_error(SLAPI_LOG_FATAL, repl_plugin_name, "cleanAllRUV_task: we're caught up...\n"); + slapi_log_error(SLAPI_LOG_FATAL, repl_plugin_name, "CleanAllRUV Task: we're caught up...\n"); /* * Set cleaned rid in memory only - does not survive a server restart */ @@ -1721,8 +1725,8 @@ multimaster_extop_cleanruv(Slapi_PBlock *pb) * This read-only replica has no easy way to tell when it's safe to release the rid. * So we won't release it, not until a server restart. */ - slapi_log_error(SLAPI_LOG_FATAL, repl_plugin_name, "cleanAllRUV_task: You must restart the server if you want to reuse rid(%d).\n", rid); - slapi_log_error(SLAPI_LOG_FATAL, repl_plugin_name, "cleanAllRUV_task: Successfully cleaned rid(%d).\n", rid); + slapi_log_error(SLAPI_LOG_FATAL, repl_plugin_name, "CleanAllRUV Task: You must restart the server if you want to reuse rid(%d).\n", rid); + slapi_log_error(SLAPI_LOG_FATAL, repl_plugin_name, "CleanAllRUV Task: Successfully cleaned rid(%d).\n", rid); rc = LDAP_SUCCESS; } @@ -1737,10 +1741,120 @@ free_and_return: * Craft a message so we know this replica supports the task */ if ((resp_bere = der_alloc())){ + ber_printf(resp_bere, "{s}", CLEANRUV_ACCEPTED); + ber_flatten(resp_bere, &resp_bval); + slapi_pblock_set(pb, SLAPI_EXT_OP_RET_VALUE, resp_bval); + slapi_send_ldap_result(pb, LDAP_SUCCESS, NULL, NULL, 0, NULL); + /* resp_bere */ + if (NULL != resp_bere) + { + ber_free(resp_bere, 1); + } + /* resp_bval */ + if (NULL != resp_bval) + { + ber_bvfree(resp_bval); + } + /* tell extendop code that we have already sent the result */ + rc = SLAPI_PLUGIN_EXTENDED_SENT_RESULT; + } else { + rc = LDAP_OPERATIONS_ERROR; + } - ber_int_t response = 1; + return rc; +} + +/* + * Get the max csn for the designated repl area + */ +int +multimaster_extop_cleanruv_get_maxcsn(Slapi_PBlock *pb) +{ + Slapi_PBlock *search_pb = NULL; + Slapi_Entry **entries = NULL; + struct berval *resp_bval = NULL; + struct berval *extop_payload; + BerElement *resp_bere = NULL; + char **ruv_elements = NULL; + char *extop_oid = NULL; + char *ruv_part = NULL; + char *base_dn = NULL; + char *payload = NULL; + char *maxcsn = NULL; + char *filter = NULL; + char *ridstr = NULL; + char *iter = NULL; + char *attrs[2]; + int part_count = 0; + int rid = 0; + int res = 0; + int rc = LDAP_OPERATIONS_ERROR; + int i = 0; + + slapi_pblock_get(pb, SLAPI_EXT_OP_REQ_OID, &extop_oid); + slapi_pblock_get(pb, SLAPI_EXT_OP_REQ_VALUE, &extop_payload); - ber_printf(resp_bere, "{e}", response); + if (NULL == extop_oid || strcmp(extop_oid, REPL_CLEANRUV_GET_MAXCSN_OID) != 0 || + NULL == extop_payload || NULL == extop_payload->bv_val){ + /* something is wrong, error out */ + goto free_and_return; + } + /* + * Decode the payload + */ + if(decode_cleanruv_payload(extop_payload, &payload)){ + slapi_log_error(SLAPI_LOG_FATAL, repl_plugin_name, "CleanAllRUV Get MaxCSN Task: failed to decode payload. Aborting ext op\n"); + goto free_and_return; + } + rid = atoi(ldap_utf8strtok_r(payload, ":", &iter)); + base_dn = ldap_utf8strtok_r(iter, ":", &iter); + /* + * Get the maxruv from the database tombstone entry + */ + filter = "(&(nsuniqueid=ffffffff-ffffffff-ffffffff-ffffffff)(objectclass=nstombstone))"; + attrs[0] = "nsds50ruv"; + attrs[1] = NULL; + ridstr = slapi_ch_smprintf("{replica %d ldap", rid); + + search_pb = slapi_pblock_new(); + slapi_search_internal_set_pb(search_pb, base_dn, LDAP_SCOPE_SUBTREE, filter, attrs, 0, NULL, NULL, repl_get_plugin_identity(PLUGIN_MULTIMASTER_REPLICATION), 0); + slapi_search_internal_pb (search_pb); + slapi_pblock_get(search_pb, SLAPI_PLUGIN_INTOP_RESULT, &res); + + if ( LDAP_SUCCESS == res ) { + slapi_pblock_get(search_pb, SLAPI_PLUGIN_INTOP_SEARCH_ENTRIES, &entries); + if (NULL == entries || entries[0] == NULL) { + /* Hmmm, no tombstpne! Error out */ + } else { + /* find the right ruv element, and find the maxcsn */ + ruv_elements = slapi_entry_attr_get_charray(entries[0],attrs[0]); + for(i = 0; ruv_elements && ruv_elements[i] ; i++){ + if(strstr(ruv_elements[i], ridstr)){ + /* get the max csn */ + ruv_part = ldap_utf8strtok_r(ruv_elements[i], " ", &iter); + for(part_count = 1; ruv_part && part_count < 5; part_count++){ + ruv_part = ldap_utf8strtok_r(iter, " ", &iter); + } + if(part_count == 5 && ruv_part){/* we have the maxcsn */ + maxcsn = slapi_ch_strdup(ruv_part); + break; + } + } + } + slapi_ch_array_free(ruv_elements); + } + } else { + /* internal search failed */ + slapi_log_error(SLAPI_LOG_FATAL, repl_plugin_name, "CleanAllRUV Get MaxCSN Task: internal search failed (%d)\n", res); + } + if(maxcsn == NULL){ + maxcsn = slapi_ch_strdup(CLEANRUV_NO_MAXCSN); + } + /* + * Send the extended op response + */ + if ((resp_bere = der_alloc())){ + ber_printf(resp_bere, "{s}", maxcsn); ber_flatten(resp_bere, &resp_bval); slapi_pblock_set(pb, SLAPI_EXT_OP_RET_VALUE, resp_bval); slapi_send_ldap_result(pb, LDAP_SUCCESS, NULL, NULL, 0, NULL); @@ -1760,10 +1874,96 @@ free_and_return: rc = LDAP_OPERATIONS_ERROR; } +free_and_return: + slapi_free_search_results_internal(search_pb); + slapi_pblock_destroy(search_pb); + slapi_ch_free_string(&payload); + slapi_ch_free_string(&maxcsn); + slapi_ch_free_string(&ridstr); + return rc; } /* + * Search cn=config for the cleanallruv attributes (clean & abort) + */ +int +multimaster_extop_cleanruv_check_status(Slapi_PBlock *pb) +{ + Slapi_PBlock *search_pb = NULL; + Slapi_Entry **entries = NULL; + struct berval *resp_bval = NULL; + struct berval *extop_payload; + BerElement *resp_bere = NULL; + char *response = NULL; + char *filter = NULL; + char *extop_oid; + int res = 0; + int rc = LDAP_OPERATIONS_ERROR; + + slapi_pblock_get(pb, SLAPI_EXT_OP_REQ_OID, &extop_oid); + slapi_pblock_get(pb, SLAPI_EXT_OP_REQ_VALUE, &extop_payload); + + if (NULL == extop_oid || strcmp(extop_oid, REPL_CLEANRUV_CHECK_STATUS_OID) != 0 || + NULL == extop_payload || NULL == extop_payload->bv_val){ + /* something is wrong, error out */ + goto free_and_return; + } + /* + * Decode the payload - which should just be a filter + */ + if(decode_cleanruv_payload(extop_payload, &filter)){ + slapi_log_error(SLAPI_LOG_FATAL, repl_plugin_name, "CleanAllRUV Check Status Task: failed to decode payload. Aborting ext op\n"); + goto free_and_return; + } + + search_pb = slapi_pblock_new(); + slapi_search_internal_set_pb(search_pb, "cn=config", LDAP_SCOPE_SUBTREE, + filter, NULL, 0, NULL, NULL, repl_get_plugin_identity(PLUGIN_MULTIMASTER_REPLICATION), 0); + slapi_search_internal_pb (search_pb); + slapi_pblock_get(search_pb, SLAPI_PLUGIN_INTOP_RESULT, &res); + if ( LDAP_SUCCESS == res ) { + slapi_pblock_get(search_pb, SLAPI_PLUGIN_INTOP_SEARCH_ENTRIES, &entries); + if (NULL == entries || entries[0] == NULL) { + /* cleaning task has finished, send repsonse */ + response = CLEANRUV_FINISHED; + } else { + response = CLEANRUV_CLEANING; + } + /* + * Send the extended op response + */ + if ((resp_bere = der_alloc())){ + ber_printf(resp_bere, "{s}", response); + ber_flatten(resp_bere, &resp_bval); + slapi_pblock_set(pb, SLAPI_EXT_OP_RET_VALUE, resp_bval); + slapi_send_ldap_result(pb, LDAP_SUCCESS, NULL, NULL, 0, NULL); + /* resp_bere */ + if (NULL != resp_bere) + { + ber_free(resp_bere, 1); + } + /* resp_bval */ + if (NULL != resp_bval) + { + ber_bvfree(resp_bval); + } + /* tell extendop code that we have already sent the result */ + rc = SLAPI_PLUGIN_EXTENDED_SENT_RESULT; + } + } + +free_and_return: + + slapi_free_search_results_internal(search_pb); + slapi_pblock_destroy(search_pb); + slapi_ch_free_string(&filter); + + return rc; +} + + +/* * This plugin entry point is a noop entry * point. It's used when registering extops that * are only used as responses. We'll never receive diff --git a/ldap/servers/plugins/replication/replutil.c b/ldap/servers/plugins/replication/replutil.c index 5e8019c..d007f6c 100644 --- a/ldap/servers/plugins/replication/replutil.c +++ b/ldap/servers/plugins/replication/replutil.c @@ -765,33 +765,32 @@ repl_set_mtn_state_and_referrals( ldap_free_urldesc(lud); } - if (!referrals_to_set) { /* deleting referrals */ - /* Set state before */ - if (!chain_on_update) { - slapi_mtn_set_state(repl_root_sdn, (char *)mtn_state); + if (!referrals_to_set) { /* deleting referrals */ + /* Set state before */ + if (!chain_on_update) { + slapi_mtn_set_state(repl_root_sdn, (char *)mtn_state); + } + /* We should delete referral only if we want to set the + replica database in backend state mode */ + /* if chain on update mode, go ahead and set the referrals anyway */ + if (strcasecmp(mtn_state, STATE_BACKEND) == 0 || chain_on_update) { + rc = slapi_mtn_set_referral(repl_root_sdn, referrals_to_set); + if (rc == LDAP_NO_SUCH_ATTRIBUTE) { + /* we will get no such attribute (16) if we try to set the referrals to NULL if + there are no referrals - not an error */ + rc = LDAP_SUCCESS; } - /* We should delete referral only if we want to set the - replica database in backend state mode */ - /* if chain on update mode, go ahead and set the referrals anyway */ - if (strcasecmp(mtn_state, STATE_BACKEND) == 0 || chain_on_update) { - rc = slapi_mtn_set_referral(repl_root_sdn, referrals_to_set); - if (rc == LDAP_NO_SUCH_ATTRIBUTE) { - /* we will get no such attribute (16) if we try to set the referrals to NULL if - there are no referrals - not an error */ - rc = LDAP_SUCCESS; - } - } - } else { /* Replacing */ - rc = slapi_mtn_set_referral(repl_root_sdn, referrals_to_set); - if (rc == LDAP_SUCCESS && !chain_on_update){ - slapi_mtn_set_state(repl_root_sdn, (char *)mtn_state); - } - } + } + } else { /* Replacing */ + rc = slapi_mtn_set_referral(repl_root_sdn, referrals_to_set); + if (rc == LDAP_SUCCESS && !chain_on_update){ + slapi_mtn_set_state(repl_root_sdn, (char *)mtn_state); + } + } - if (rc != LDAP_SUCCESS && rc != LDAP_TYPE_OR_VALUE_EXISTS) { + if (rc != LDAP_SUCCESS && rc != LDAP_TYPE_OR_VALUE_EXISTS) { slapi_log_error(SLAPI_LOG_FATAL, repl_plugin_name, "repl_set_mtn_referrals: could " - "not set referrals for replica %s: %d\n", - slapi_sdn_get_dn(repl_root_sdn), rc); + "not set referrals for replica %s: %d\n", slapi_sdn_get_dn(repl_root_sdn), rc); } charray_free(referrals_to_set); diff --git a/ldap/servers/slapd/back-ldbm/ldbm_modify.c b/ldap/servers/slapd/back-ldbm/ldbm_modify.c index 6d2e6f6..5c9585f 100644 --- a/ldap/servers/slapd/back-ldbm/ldbm_modify.c +++ b/ldap/servers/slapd/back-ldbm/ldbm_modify.c @@ -71,6 +71,11 @@ void modify_init(modify_context *mc,struct backentry *old_entry) int modify_apply_mods(modify_context *mc, Slapi_Mods *smods) { + return modify_apply_mods_ignore_error(mc, smods, -1); +} + +int modify_apply_mods_ignore_error(modify_context *mc, Slapi_Mods *smods, int error) +{ int ret = 0; /* Make a copy of the entry */ PR_ASSERT(mc->old_entry != NULL); @@ -78,7 +83,7 @@ int modify_apply_mods(modify_context *mc, Slapi_Mods *smods) mc->new_entry = backentry_dup(mc->old_entry); PR_ASSERT(smods!=NULL); if ( mods_have_effect (mc->new_entry->ep_entry, smods) ) { - ret = entry_apply_mods( mc->new_entry->ep_entry, slapi_mods_get_ldapmods_byref(smods)); + ret = entry_apply_mods_ignore_error( mc->new_entry->ep_entry, slapi_mods_get_ldapmods_byref(smods), error); } mc->smods= smods; return ret; diff --git a/ldap/servers/slapd/back-ldbm/misc.c b/ldap/servers/slapd/back-ldbm/misc.c index a56069b..fd62df9 100644 --- a/ldap/servers/slapd/back-ldbm/misc.c +++ b/ldap/servers/slapd/back-ldbm/misc.c @@ -444,7 +444,7 @@ ldbm_txn_ruv_modify_context( Slapi_PBlock *pb, modify_context *mc ) modify_init( mc, bentry ); - if (modify_apply_mods( mc, smods )) { + if (modify_apply_mods_ignore_error( mc, smods, LDAP_TYPE_OR_VALUE_EXISTS )) { LDAPDebug( LDAP_DEBUG_ANY, "Error: ldbm_txn_ruv_modify_context failed to apply updates to RUV entry\n", 0, 0, 0 ); rc = -1; diff --git a/ldap/servers/slapd/back-ldbm/proto-back-ldbm.h b/ldap/servers/slapd/back-ldbm/proto-back-ldbm.h index 48b3164..a7fb06d 100644 --- a/ldap/servers/slapd/back-ldbm/proto-back-ldbm.h +++ b/ldap/servers/slapd/back-ldbm/proto-back-ldbm.h @@ -347,6 +347,7 @@ void modify_init(modify_context *mc,struct backentry *old_entry); int modify_apply_mods(modify_context *mc, Slapi_Mods *smods); int modify_term(modify_context *mc,backend *be); int modify_switch_entries(modify_context *mc,backend *be); +int modify_apply_mods_ignore_error(modify_context *mc, Slapi_Mods *smods, int error); /* * add.c diff --git a/ldap/servers/slapd/entry.c b/ldap/servers/slapd/entry.c index 6339a6f..58a9d51 100644 --- a/ldap/servers/slapd/entry.c +++ b/ldap/servers/slapd/entry.c @@ -3259,6 +3259,12 @@ int slapi_entry_apply_mod( Slapi_Entry *e, LDAPMod *mod ) int entry_apply_mods( Slapi_Entry *e, LDAPMod **mods ) { + return entry_apply_mods_ignore_error(e, mods, -1); +} + +int +entry_apply_mods_ignore_error( Slapi_Entry *e, LDAPMod **mods, int ignore_error ) +{ int err; LDAPMod **mp = NULL; @@ -3268,7 +3274,9 @@ entry_apply_mods( Slapi_Entry *e, LDAPMod **mods ) for ( mp = mods; mp && *mp; mp++ ) { err = entry_apply_mod( e, *mp ); - if ( err != LDAP_SUCCESS ) { + if(err == ignore_error){ + (*mp)->mod_op = LDAP_MOD_IGNORE; + } else if ( err != LDAP_SUCCESS ) { break; } } diff --git a/ldap/servers/slapd/slapi-private.h b/ldap/servers/slapd/slapi-private.h index f7b4d04..ddeac62 100644 --- a/ldap/servers/slapd/slapi-private.h +++ b/ldap/servers/slapd/slapi-private.h @@ -331,7 +331,7 @@ int entry_next_deleted_attribute( const Slapi_Entry *e, Slapi_Attr **a); /* entry.c */ int entry_apply_mods( Slapi_Entry *e, LDAPMod **mods ); int is_type_protected(const char *type); - +int entry_apply_mods_ignore_error( Slapi_Entry *e, LDAPMod **mods, int ignore_error ); int slapi_entries_diff(Slapi_Entry **old_entries, Slapi_Entry **new_entries, int testall, const char *logging_prestr, const int force_update, void *plg_id); /* entrywsi.c */

1 0

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

389-commits November 2012