I have a two year image of one of my IPA servers that I am trying to bring live. Unfortunately all of the certs except the CA are expired. I have attempted to follow the instructions for updating the certs, but it has failed to update them. After careful and extensive digging, I have found that the issue is two replication agreements from other IPA servers that have since been rebuilt. Because of the expired certs I can't login to the web UI, so I can't terminate the agreements that way, and the IPA commands fail. Is there a way to terminate these agreements manually by removing the references to the two servers?
Randy Morgan
You could turn the clock back, remove the agreements, renew the certs to a future date, shutdown, reset the clock and renew again to get up and running. Make sure you’re doing it while the system is offline to prevent NTP. Also: make sure you don’t run in to this again by making regular recovery points (backups, snapshots, periodic master updates). I’m assuming this is a recovery action from total loss of everything? If not: don’t bother with that image, install a fresh master instead.
John
On 26 Sep 2019, at 23:59, Randy Morgan via FreeIPA-users freeipa-users@lists.fedorahosted.org wrote:
I have a two year image of one of my IPA servers that I am trying to bring live. Unfortunately all of the certs except the CA are expired. I have attempted to follow the instructions for updating the certs, but it has failed to update them. After careful and extensive digging, I have found that the issue is two replication agreements from other IPA servers that have since been rebuilt. Because of the expired certs I can't login to the web UI, so I can't terminate the agreements that way, and the IPA commands fail. Is there a way to terminate these agreements manually by removing the references to the two servers?
Randy Morgan
-- Randy Morgan CSR Department of Chemistry/BioChemistry Brigham Young University _______________________________________________ FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org To unsubscribe send an email to freeipa-users-leave@lists.fedorahosted.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedorahosted.org/archives/list/freeipa-users@lists.fedorahoste...
Tried everything you just suggested, and it doesn't work. Yes this is a recovery option, our environment has become damaged somehow, uncertain of all that happened. This image gives us a path forward, but I need these replication agreements to go away and the image to become a stand alone master. Once that happens I believe I can get the certs to update, but right now everything seems to be attempting to talk to IPA2 which is still running but the server was rebuilt after this image was made, so we can't talk with the server.
Randy
On 9/26/2019 4:05 PM, John Keates via FreeIPA-users wrote:
You could turn the clock back, remove the agreements, renew the certs to a future date, shutdown, reset the clock and renew again to get up and running. Make sure you’re doing it while the system is offline to prevent NTP. Also: make sure you don’t run in to this again by making regular recovery points (backups, snapshots, periodic master updates). I’m assuming this is a recovery action from total loss of everything? If not: don’t bother with that image, install a fresh master instead.
John
On 26 Sep 2019, at 23:59, Randy Morgan via FreeIPA-users freeipa-users@lists.fedorahosted.org wrote:
I have a two year image of one of my IPA servers that I am trying to bring live. Unfortunately all of the certs except the CA are expired. I have attempted to follow the instructions for updating the certs, but it has failed to update them. After careful and extensive digging, I have found that the issue is two replication agreements from other IPA servers that have since been rebuilt. Because of the expired certs I can't login to the web UI, so I can't terminate the agreements that way, and the IPA commands fail. Is there a way to terminate these agreements manually by removing the references to the two servers?
Randy Morgan
-- Randy Morgan CSR Department of Chemistry/BioChemistry Brigham Young University _______________________________________________ FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org To unsubscribe send an email to freeipa-users-leave@lists.fedorahosted.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedorahosted.org/archives/list/freeipa-users@lists.fedorahoste...
FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org To unsubscribe send an email to freeipa-users-leave@lists.fedorahosted.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedorahosted.org/archives/list/freeipa-users@lists.fedorahoste...
What do you mean by ‘rebuilt’? Also: is that image a CA master and how does it fail when you run it with the clock turned back and network unplugged/firewalled?
John
On 27 Sep 2019, at 00:10, Randy Morgan via FreeIPA-users freeipa-users@lists.fedorahosted.org wrote:
Tried everything you just suggested, and it doesn't work. Yes this is a recovery option, our environment has become damaged somehow, uncertain of all that happened. This image gives us a path forward, but I need these replication agreements to go away and the image to become a stand alone master. Once that happens I believe I can get the certs to update, but right now everything seems to be attempting to talk to IPA2 which is still running but the server was rebuilt after this image was made, so we can't talk with the server.
Randy
On 9/26/2019 4:05 PM, John Keates via FreeIPA-users wrote:
You could turn the clock back, remove the agreements, renew the certs to a future date, shutdown, reset the clock and renew again to get up and running. Make sure you’re doing it while the system is offline to prevent NTP. Also: make sure you don’t run in to this again by making regular recovery points (backups, snapshots, periodic master updates). I’m assuming this is a recovery action from total loss of everything? If not: don’t bother with that image, install a fresh master instead.
John
On 26 Sep 2019, at 23:59, Randy Morgan via FreeIPA-users freeipa-users@lists.fedorahosted.org wrote:
I have a two year image of one of my IPA servers that I am trying to bring live. Unfortunately all of the certs except the CA are expired. I have attempted to follow the instructions for updating the certs, but it has failed to update them. After careful and extensive digging, I have found that the issue is two replication agreements from other IPA servers that have since been rebuilt. Because of the expired certs I can't login to the web UI, so I can't terminate the agreements that way, and the IPA commands fail. Is there a way to terminate these agreements manually by removing the references to the two servers?
Randy Morgan
-- Randy Morgan CSR Department of Chemistry/BioChemistry Brigham Young University _______________________________________________ FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org To unsubscribe send an email to freeipa-users-leave@lists.fedorahosted.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedorahosted.org/archives/list/freeipa-users@lists.fedorahoste...
FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org To unsubscribe send an email to freeipa-users-leave@lists.fedorahosted.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedorahosted.org/archives/list/freeipa-users@lists.fedorahoste...
-- Randy Morgan CSR Department of Chemistry/BioChemistry Brigham Young University _______________________________________________ FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org To unsubscribe send an email to freeipa-users-leave@lists.fedorahosted.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedorahosted.org/archives/list/freeipa-users@lists.fedorahoste...
IPA2 was removed from the replica set and reinstalled to correct some OS issues (twice). After that it was brought back in as a replica again. IPA2 has become corrupted and we are unable to update the subsystem certs. It is the last standing replica for the original cluster. The image of IPA1 is a virtual server, so turning off the network can be done but that means working through the console, yuck. I can try it and see what happens. Yes this image is a CA master, or at least it was supposed to be when the image was made.
Randy
On 9/26/2019 4:14 PM, John Keates via FreeIPA-users wrote:
What do you mean by ‘rebuilt’? Also: is that image a CA master and how does it fail when you run it with the clock turned back and network unplugged/firewalled?
John
On 27 Sep 2019, at 00:10, Randy Morgan via FreeIPA-users freeipa-users@lists.fedorahosted.org wrote:
Tried everything you just suggested, and it doesn't work. Yes this is a recovery option, our environment has become damaged somehow, uncertain of all that happened. This image gives us a path forward, but I need these replication agreements to go away and the image to become a stand alone master. Once that happens I believe I can get the certs to update, but right now everything seems to be attempting to talk to IPA2 which is still running but the server was rebuilt after this image was made, so we can't talk with the server.
Randy
On 9/26/2019 4:05 PM, John Keates via FreeIPA-users wrote:
You could turn the clock back, remove the agreements, renew the certs to a future date, shutdown, reset the clock and renew again to get up and running. Make sure you’re doing it while the system is offline to prevent NTP. Also: make sure you don’t run in to this again by making regular recovery points (backups, snapshots, periodic master updates). I’m assuming this is a recovery action from total loss of everything? If not: don’t bother with that image, install a fresh master instead.
John
On 26 Sep 2019, at 23:59, Randy Morgan via FreeIPA-users freeipa-users@lists.fedorahosted.org wrote:
I have a two year image of one of my IPA servers that I am trying to bring live. Unfortunately all of the certs except the CA are expired. I have attempted to follow the instructions for updating the certs, but it has failed to update them. After careful and extensive digging, I have found that the issue is two replication agreements from other IPA servers that have since been rebuilt. Because of the expired certs I can't login to the web UI, so I can't terminate the agreements that way, and the IPA commands fail. Is there a way to terminate these agreements manually by removing the references to the two servers?
Randy Morgan
-- Randy Morgan CSR Department of Chemistry/BioChemistry Brigham Young University _______________________________________________ FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org To unsubscribe send an email to freeipa-users-leave@lists.fedorahosted.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedorahosted.org/archives/list/freeipa-users@lists.fedorahoste...
FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org To unsubscribe send an email to freeipa-users-leave@lists.fedorahosted.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedorahosted.org/archives/list/freeipa-users@lists.fedorahoste...
-- Randy Morgan CSR Department of Chemistry/BioChemistry Brigham Young University _______________________________________________ FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org To unsubscribe send an email to freeipa-users-leave@lists.fedorahosted.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedorahosted.org/archives/list/freeipa-users@lists.fedorahoste...
FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org To unsubscribe send an email to freeipa-users-leave@lists.fedorahosted.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedorahosted.org/archives/list/freeipa-users@lists.fedorahoste...
Randy Morgan via FreeIPA-users wrote:
IPA2 was removed from the replica set and reinstalled to correct some OS issues (twice). After that it was brought back in as a replica again. IPA2 has become corrupted and we are unable to update the subsystem certs. It is the last standing replica for the original cluster. The image of IPA1 is a virtual server, so turning off the network can be done but that means working through the console, yuck. I can try it and see what happens. Yes this image is a CA master, or at least it was supposed to be when the image was made.
The version you're running could make a difference in how things work.
Only the renewal master, and there should only be one, will actually renew the certificates. So for your "offline" server I'd make sure it is by going back in time so ipa can start and do ipa config-show and see if this master is the CA renewal master. If it isn't then set it as it and try the renewal again.
The current getcert list output may provide some insights as well.
Generally speaking the agreements probably won't work if the master has been down for a long time, as it seems. The changelog has limited size so depending on the amount of time and the number of changes it is probably exhaused so can't replay history fully over replication so it is throwing up its hands.
rob
On 9/26/2019 4:14 PM, John Keates via FreeIPA-users wrote:
What do you mean by ‘rebuilt’? Also: is that image a CA master and how does it fail when you run it with the clock turned back and network unplugged/firewalled?
John
On 27 Sep 2019, at 00:10, Randy Morgan via FreeIPA-users freeipa-users@lists.fedorahosted.org wrote:
Tried everything you just suggested, and it doesn't work. Yes this is a recovery option, our environment has become damaged somehow, uncertain of all that happened. This image gives us a path forward, but I need these replication agreements to go away and the image to become a stand alone master. Once that happens I believe I can get the certs to update, but right now everything seems to be attempting to talk to IPA2 which is still running but the server was rebuilt after this image was made, so we can't talk with the server.
Randy
On 9/26/2019 4:05 PM, John Keates via FreeIPA-users wrote:
You could turn the clock back, remove the agreements, renew the certs to a future date, shutdown, reset the clock and renew again to get up and running. Make sure you’re doing it while the system is offline to prevent NTP. Also: make sure you don’t run in to this again by making regular recovery points (backups, snapshots, periodic master updates). I’m assuming this is a recovery action from total loss of everything? If not: don’t bother with that image, install a fresh master instead.
John
On 26 Sep 2019, at 23:59, Randy Morgan via FreeIPA-users freeipa-users@lists.fedorahosted.org wrote:
I have a two year image of one of my IPA servers that I am trying to bring live. Unfortunately all of the certs except the CA are expired. I have attempted to follow the instructions for updating the certs, but it has failed to update them. After careful and extensive digging, I have found that the issue is two replication agreements from other IPA servers that have since been rebuilt. Because of the expired certs I can't login to the web UI, so I can't terminate the agreements that way, and the IPA commands fail. Is there a way to terminate these agreements manually by removing the references to the two servers?
Randy Morgan
-- Randy Morgan CSR Department of Chemistry/BioChemistry Brigham Young University _______________________________________________ FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org To unsubscribe send an email to freeipa-users-leave@lists.fedorahosted.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedorahosted.org/archives/list/freeipa-users@lists.fedorahoste...
FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org To unsubscribe send an email to freeipa-users-leave@lists.fedorahosted.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedorahosted.org/archives/list/freeipa-users@lists.fedorahoste...
-- Randy Morgan CSR Department of Chemistry/BioChemistry Brigham Young University _______________________________________________ FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org To unsubscribe send an email to freeipa-users-leave@lists.fedorahosted.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedorahosted.org/archives/list/freeipa-users@lists.fedorahoste...
FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org To unsubscribe send an email to freeipa-users-leave@lists.fedorahosted.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedorahosted.org/archives/list/freeipa-users@lists.fedorahoste...
freeipa-users@lists.fedorahosted.org