Hello All,
I upgraded our ipa server and after the upgrade ipa won't start again. further investigation shows that components of ipa starts but pki-tomcatd@pki-tomcat.service appears to be where the issue lies. checking the logs suggested that issue lies in the certificate database. on checking the directory /etc/pki/pki-tomcat/alias with certutils
[namead@ipasvr01 alias]$ sudo certutil -K -d . -f pwdfile.txt certutil: Checking token "NSS Certificate DB" in slot "NSS User Private Key and Certificate Services" < 0> rsa 9bb20dbec9d8dd63e1db53b0662eaf37a1518bf9 ocspSigningCert cert-pki-ca < 1> rsa 49d9f7a5f5ab3ed93d4037676b1bf9e236b89d0f subsystemCert cert-pki-ca < 2> rsa df374a636d9a424aaefefc6367dcb868f82f536d Server-Cert cert-pki-ca *< 3> rsa 7cebd0bbadddd5e581c328a99982e0ef5172d61f (orphan)* < 4> rsa 52839be82200bb2a9ff2034629c53cd90a0575a8 auditSigningCert cert-pki-ca < 5> rsa c4a6d42c22a874a69231a2d7446bccfe9ce0cbaa caSigningCert cert-pki-ca
Any help in the deleting the key would be appreciated.
Thanks
_Uz
Uzor Ide via FreeIPA-users wrote:
Hello All,
I upgraded our ipa server and after the upgrade ipa won't start again. further investigation shows that components of ipa starts but pki-tomcatd@pki-tomcat.service appears to be where the issue lies. checking the logs suggested that issue lies in the certificate database. on checking the directory /etc/pki/pki-tomcat/alias with certutils
[namead@ipasvr01 alias]$ sudo certutil -K -d . -f pwdfile.txt certutil: Checking token "NSS Certificate DB" in slot "NSS User Private Key and Certificate Services" < 0> rsa 9bb20dbec9d8dd63e1db53b0662eaf37a1518bf9 ocspSigningCert cert-pki-ca < 1> rsa 49d9f7a5f5ab3ed93d4037676b1bf9e236b89d0f subsystemCert cert-pki-ca < 2> rsa df374a636d9a424aaefefc6367dcb868f82f536d Server-Cert cert-pki-ca *< 3> rsa 7cebd0bbadddd5e581c328a99982e0ef5172d61f (orphan)* < 4> rsa 52839be82200bb2a9ff2034629c53cd90a0575a8 auditSigningCert cert-pki-ca < 5> rsa c4a6d42c22a874a69231a2d7446bccfe9ce0cbaa caSigningCert cert-pki-ca
Any help in the deleting the key would be appreciated.
I doubt the orphan is preventing the CA from starting, it is almost certainly something else.
A temporary server certificate is created during the CA creation. My guess is that this orphan is the key to that long-gone one-time certificate.
You don't say what version you can have. In my 4.4 master I also have an orphan key and my CA works fine.
In my 4.7.x master I do not have such a key so either the CA is more thorough in cleanup or it no longer creates this temporary cert.
What was in the log that made you think it was related to an orphaned key?
rob
log contained the following Jan 13 17:44:02 ipasvr01.domain.com pki-server[4808]: ERROR: */var/lib/pki/pki-tomcat/alias contains an incomplete NSS database* in SQL format That's what made me go to the NSS database and so the orphan key.
ipa server version is 4.4.4 and upgraded to 4.7.2
On Mon, Jan 14, 2019 at 9:59 AM Rob Crittenden rcritten@redhat.com wrote:
Uzor Ide via FreeIPA-users wrote:
Hello All,
I upgraded our ipa server and after the upgrade ipa won't start again. further investigation shows that components of ipa starts but pki-tomcatd@pki-tomcat.service appears to be where the issue lies. checking the logs suggested that issue lies in the certificate database. on checking the directory /etc/pki/pki-tomcat/alias with certutils
[namead@ipasvr01 alias]$ sudo certutil -K -d . -f pwdfile.txt certutil: Checking token "NSS Certificate DB" in slot "NSS User Private Key and Certificate Services" < 0> rsa 9bb20dbec9d8dd63e1db53b0662eaf37a1518bf9 ocspSigningCert cert-pki-ca < 1> rsa 49d9f7a5f5ab3ed93d4037676b1bf9e236b89d0f subsystemCert cert-pki-ca < 2> rsa df374a636d9a424aaefefc6367dcb868f82f536d Server-Cert cert-pki-ca *< 3> rsa 7cebd0bbadddd5e581c328a99982e0ef5172d61f (orphan)* < 4> rsa 52839be82200bb2a9ff2034629c53cd90a0575a8 auditSigningCert cert-pki-ca < 5> rsa c4a6d42c22a874a69231a2d7446bccfe9ce0cbaa caSigningCert cert-pki-ca
Any help in the deleting the key would be appreciated.
I doubt the orphan is preventing the CA from starting, it is almost certainly something else.
A temporary server certificate is created during the CA creation. My guess is that this orphan is the key to that long-gone one-time certificate.
You don't say what version you can have. In my 4.4 master I also have an orphan key and my CA works fine.
In my 4.7.x master I do not have such a key so either the CA is more thorough in cleanup or it no longer creates this temporary cert.
What was in the log that made you think it was related to an orphaned key?
rob
On 1/14/19 5:30 PM, Uzor Ide via FreeIPA-users wrote:
Hello All,
I upgraded our ipa server and after the upgrade ipa won't start again. further investigation shows that components of ipa starts but pki-tomcatd@pki-tomcat.service appears to be where the issue lies. checking the logs suggested that issue lies in the certificate database. on checking the directory /etc/pki/pki-tomcat/alias with certutils
[namead@ipasvr01 alias]$ sudo certutil -K -d . -f pwdfile.txt certutil: Checking token "NSS Certificate DB" in slot "NSS User Private Key and Certificate Services" < 0> rsa 9bb20dbec9d8dd63e1db53b0662eaf37a1518bf9 ocspSigningCert cert-pki-ca < 1> rsa 49d9f7a5f5ab3ed93d4037676b1bf9e236b89d0f subsystemCert cert-pki-ca < 2> rsa df374a636d9a424aaefefc6367dcb868f82f536d Server-Cert cert-pki-ca *< 3> rsa 7cebd0bbadddd5e581c328a99982e0ef5172d61f (orphan)* < 4> rsa 52839be82200bb2a9ff2034629c53cd90a0575a8 auditSigningCert cert-pki-ca < 5> rsa c4a6d42c22a874a69231a2d7446bccfe9ce0cbaa caSigningCert cert-pki-ca
Any help in the deleting the key would be appreciated.
The certutil command can delete a key from a NSS database (certutil -F -k <id> -d /etc/pki/pki-tomcat/alias). But before you delete this private key, can you explain how you deduced that it was the root cause? I wouldn't advise to delete a private key if you're not 100% sure you need to.
Pki failing to start after an upgrade often happens when the certificate "subsystemCert cert-pki-ca" stored in /etc/pki/pki-tomcat/alias does not match the content of the usercertificate or description stored in uid=pkidbuser,ou=people,o=ipaca.
flo
Thanks
_Uz
FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org To unsubscribe send an email to freeipa-users-leave@lists.fedorahosted.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedorahosted.org/archives/list/freeipa-users@lists.fedorahoste...
Am certainly not sure that the orphan key is the root cause. It just looked out of place and the log had the following error Jan 13 17:44:02 ipasvr01.domain.com pki-server[4808]: ERROR: */var/lib/pki/pki-tomcat/alias contains an incomplete NSS database* in SQL format However, I compared the certificates stored in "subsystemCert cert-pki-ca" and " uid=pkidbuser,ou=people,o=ipaca userCertificate".but haven't been able to detect any difference.
On Mon, Jan 14, 2019 at 10:02 AM Florence Blanc-Renaud flo@redhat.com wrote:
On 1/14/19 5:30 PM, Uzor Ide via FreeIPA-users wrote:
Hello All,
I upgraded our ipa server and after the upgrade ipa won't start again. further investigation shows that components of ipa starts but pki-tomcatd@pki-tomcat.service appears to be where the issue lies. checking the logs suggested that issue lies in the certificate database. on checking the directory /etc/pki/pki-tomcat/alias with certutils
[namead@ipasvr01 alias]$ sudo certutil -K -d . -f pwdfile.txt certutil: Checking token "NSS Certificate DB" in slot "NSS User Private Key and Certificate Services" < 0> rsa 9bb20dbec9d8dd63e1db53b0662eaf37a1518bf9 ocspSigningCert cert-pki-ca < 1> rsa 49d9f7a5f5ab3ed93d4037676b1bf9e236b89d0f subsystemCert cert-pki-ca < 2> rsa df374a636d9a424aaefefc6367dcb868f82f536d Server-Cert cert-pki-ca *< 3> rsa 7cebd0bbadddd5e581c328a99982e0ef5172d61f (orphan)* < 4> rsa 52839be82200bb2a9ff2034629c53cd90a0575a8 auditSigningCert cert-pki-ca < 5> rsa c4a6d42c22a874a69231a2d7446bccfe9ce0cbaa caSigningCert cert-pki-ca
Any help in the deleting the key would be appreciated.
The certutil command can delete a key from a NSS database (certutil -F -k <id> -d /etc/pki/pki-tomcat/alias). But before you delete this private key, can you explain how you deduced that it was the root cause? I wouldn't advise to delete a private key if you're not 100% sure you need to.
Pki failing to start after an upgrade often happens when the certificate "subsystemCert cert-pki-ca" stored in /etc/pki/pki-tomcat/alias does not match the content of the usercertificate or description stored in uid=pkidbuser,ou=people,o=ipaca.
flo
Thanks
_Uz
FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org To unsubscribe send an email to
freeipa-users-leave@lists.fedorahosted.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives:
https://lists.fedorahosted.org/archives/list/freeipa-users@lists.fedorahoste...
Uzor Ide via FreeIPA-users wrote:
Am certainly not sure that the orphan key is the root cause. It just looked out of place and the log had the following error Jan 13 17:44:02 ipasvr01.domain.com http://ipasvr01.domain.com pki-server[4808]: ERROR: */var/lib/pki/pki-tomcat/alias contains an incomplete NSS database* in SQL format However, I compared the certificates stored in "subsystemCert cert-pki-ca" and " uid=pkidbuser,ou=people,o=ipaca userCertificate".but haven't been able to detect any difference.
NSS supports two database formats, dbm and sqlite. 4.7 switched to the sqlite database format. This switch is generally transparent.
Look in the database directory and see what files you have. You should have the dbm files (cert8.db, key3.db and secmod.db) and the sqlite files (cert9.db, key4.db, pkcs11.txt).
Let us know what you find.
rob
On Mon, Jan 14, 2019 at 10:02 AM Florence Blanc-Renaud <flo@redhat.com mailto:flo@redhat.com> wrote:
On 1/14/19 5:30 PM, Uzor Ide via FreeIPA-users wrote: > Hello All, > > I upgraded our ipa server and after the upgrade ipa won't start again. > further investigation shows that components of ipa starts > but pki-tomcatd@pki-tomcat.service appears to be where the issue lies. > checking the logs suggested that issue lies in the certificate database. > on checking the directory /etc/pki/pki-tomcat/alias with certutils > > [namead@ipasvr01 alias]$ sudo certutil -K -d . -f pwdfile.txt > certutil: Checking token "NSS Certificate DB" in slot "NSS User Private > Key and Certificate Services" > < 0> rsa 9bb20dbec9d8dd63e1db53b0662eaf37a1518bf9 ocspSigningCert > cert-pki-ca > < 1> rsa 49d9f7a5f5ab3ed93d4037676b1bf9e236b89d0f subsystemCert > cert-pki-ca > < 2> rsa df374a636d9a424aaefefc6367dcb868f82f536d Server-Cert > cert-pki-ca > *< 3> rsa 7cebd0bbadddd5e581c328a99982e0ef5172d61f (orphan)* > < 4> rsa 52839be82200bb2a9ff2034629c53cd90a0575a8 > auditSigningCert cert-pki-ca > < 5> rsa c4a6d42c22a874a69231a2d7446bccfe9ce0cbaa caSigningCert > cert-pki-ca > > Any help in the deleting the key would be appreciated. The certutil command can delete a key from a NSS database (certutil -F -k <id> -d /etc/pki/pki-tomcat/alias). But before you delete this private key, can you explain how you deduced that it was the root cause? I wouldn't advise to delete a private key if you're not 100% sure you need to. Pki failing to start after an upgrade often happens when the certificate "subsystemCert cert-pki-ca" stored in /etc/pki/pki-tomcat/alias does not match the content of the usercertificate or description stored in uid=pkidbuser,ou=people,o=ipaca. flo > > Thanks > > _Uz > > _______________________________________________ > FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org <mailto:freeipa-users@lists.fedorahosted.org> > To unsubscribe send an email to freeipa-users-leave@lists.fedorahosted.org <mailto:freeipa-users-leave@lists.fedorahosted.org> > Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: https://lists.fedorahosted.org/archives/list/freeipa-users@lists.fedorahosted.org >
FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org To unsubscribe send an email to freeipa-users-leave@lists.fedorahosted.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedorahosted.org/archives/list/freeipa-users@lists.fedorahoste...
All the files you named are present plus the password file (pwdfile.txt). - pkcs11.txt - pwdfile.txt - key3.db - key4.db - cert8.db - cert9.db - secmod.db
On Tue, Jan 15, 2019 at 3:12 PM Rob Crittenden rcritten@redhat.com wrote:
Uzor Ide via FreeIPA-users wrote:
Am certainly not sure that the orphan key is the root cause. It just looked out of place and the log had the following error Jan 13 17:44:02 ipasvr01.domain.com http://ipasvr01.domain.com pki-server[4808]: ERROR: */var/lib/pki/pki-tomcat/alias contains an incomplete NSS database* in SQL format However, I compared the certificates stored in "subsystemCert cert-pki-ca" and " uid=pkidbuser,ou=people,o=ipaca userCertificate".but haven't been able to detect any difference.
NSS supports two database formats, dbm and sqlite. 4.7 switched to the sqlite database format. This switch is generally transparent.
Look in the database directory and see what files you have. You should have the dbm files (cert8.db, key3.db and secmod.db) and the sqlite files (cert9.db, key4.db, pkcs11.txt).
Let us know what you find.
rob
On Mon, Jan 14, 2019 at 10:02 AM Florence Blanc-Renaud <flo@redhat.com mailto:flo@redhat.com> wrote:
On 1/14/19 5:30 PM, Uzor Ide via FreeIPA-users wrote: > Hello All, > > I upgraded our ipa server and after the upgrade ipa won't start again. > further investigation shows that components of ipa starts > but pki-tomcatd@pki-tomcat.service appears to be where the issue lies. > checking the logs suggested that issue lies in the certificate database. > on checking the directory /etc/pki/pki-tomcat/alias with certutils > > [namead@ipasvr01 alias]$ sudo certutil -K -d . -f pwdfile.txt > certutil: Checking token "NSS Certificate DB" in slot "NSS User Private > Key and Certificate Services" > < 0> rsa 9bb20dbec9d8dd63e1db53b0662eaf37a1518bf9 ocspSigningCert > cert-pki-ca > < 1> rsa 49d9f7a5f5ab3ed93d4037676b1bf9e236b89d0f subsystemCert > cert-pki-ca > < 2> rsa df374a636d9a424aaefefc6367dcb868f82f536dServer-Cert
> cert-pki-ca > *< 3> rsa 7cebd0bbadddd5e581c328a99982e0ef5172d61f (orphan)* > < 4> rsa 52839be82200bb2a9ff2034629c53cd90a0575a8 > auditSigningCert cert-pki-ca > < 5> rsa c4a6d42c22a874a69231a2d7446bccfe9ce0cbaa caSigningCert > cert-pki-ca > > Any help in the deleting the key would be appreciated. The certutil command can delete a key from a NSS database (certutil-F
-k <id> -d /etc/pki/pki-tomcat/alias). But before you delete this private key, can you explain how you deduced that it was the root cause? I wouldn't advise to delete a private key if you're not 100% sure you need to. Pki failing to start after an upgrade often happens when the certificate "subsystemCert cert-pki-ca" stored in /etc/pki/pki-tomcat/alias does not match the content of the usercertificate or description stored in uid=pkidbuser,ou=people,o=ipaca. flo > > Thanks > > _Uz > > _______________________________________________ > FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org <mailto:freeipa-users@lists.fedorahosted.org> > To unsubscribe send an email to freeipa-users-leave@lists.fedorahosted.org <mailto:freeipa-users-leave@lists.fedorahosted.org> > Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives:https://lists.fedorahosted.org/archives/list/freeipa-users@lists.fedorahoste...
>
FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org To unsubscribe send an email to
freeipa-users-leave@lists.fedorahosted.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives:
https://lists.fedorahosted.org/archives/list/freeipa-users@lists.fedorahoste...
Uzor Ide wrote:
All the files you named are present plus the password file (pwdfile.txt).
- pkcs11.txt
- pwdfile.txt
- key3.db
- key4.db
- cert8.db
- cert9.db
- secmod.db
I'm not sure if you said which distribution you're using so let's be precise about the contents.
You'll want to compare the output of:
# certutil -L -d sql:/var/lib/pki/pki-tomcat/alias/
with
# certutil -L -d dbm:/var/lib/pki/pki-tomcat/alias/
and
# certutil -K -d sql:/var/lib/pki/pki-tomcat/alias/ -f /var/lib/pki/pki-tomcat/alias/pwdfile.txt
with
# certutil -K -d dbm:/var/lib/pki/pki-tomcat/alias/ -f /var/lib/pki/pki-tomcat/alias/pwdfile.txt
Presumably there is some difference that dogtag is detecting.
rob
On Tue, Jan 15, 2019 at 3:12 PM Rob Crittenden <rcritten@redhat.com mailto:rcritten@redhat.com> wrote:
Uzor Ide via FreeIPA-users wrote: > Am certainly not sure that the orphan key is the root cause. It just > looked out of place and the log had the following error > Jan 13 17:44:02 ipasvr01.domain.com <http://ipasvr01.domain.com> <http://ipasvr01.domain.com> > pki-server[4808]: ERROR: */var/lib/pki/pki-tomcat/alias contains an > incomplete NSS database* in SQL format > However, I compared the certificates stored in "subsystemCert > cert-pki-ca" and " uid=pkidbuser,ou=people,o=ipaca userCertificate".but > haven't been able to detect any difference. NSS supports two database formats, dbm and sqlite. 4.7 switched to the sqlite database format. This switch is generally transparent. Look in the database directory and see what files you have. You should have the dbm files (cert8.db, key3.db and secmod.db) and the sqlite files (cert9.db, key4.db, pkcs11.txt). Let us know what you find. rob > > On Mon, Jan 14, 2019 at 10:02 AM Florence Blanc-Renaud <flo@redhat.com <mailto:flo@redhat.com> > <mailto:flo@redhat.com <mailto:flo@redhat.com>>> wrote: > > On 1/14/19 5:30 PM, Uzor Ide via FreeIPA-users wrote: > > Hello All, > > > > I upgraded our ipa server and after the upgrade ipa won't start > again. > > further investigation shows that components of ipa starts > > but pki-tomcatd@pki-tomcat.service appears to be where the issue > lies. > > checking the logs suggested that issue lies in the certificate > database. > > on checking the directory /etc/pki/pki-tomcat/alias with certutils > > > > [namead@ipasvr01 alias]$ sudo certutil -K -d . -f pwdfile.txt > > certutil: Checking token "NSS Certificate DB" in slot "NSS User > Private > > Key and Certificate Services" > > < 0> rsa 9bb20dbec9d8dd63e1db53b0662eaf37a1518bf9 > ocspSigningCert > > cert-pki-ca > > < 1> rsa 49d9f7a5f5ab3ed93d4037676b1bf9e236b89d0f > subsystemCert > > cert-pki-ca > > < 2> rsa df374a636d9a424aaefefc6367dcb868f82f536d Server-Cert > > cert-pki-ca > > *< 3> rsa 7cebd0bbadddd5e581c328a99982e0ef5172d61f (orphan)* > > < 4> rsa 52839be82200bb2a9ff2034629c53cd90a0575a8 > > auditSigningCert cert-pki-ca > > < 5> rsa c4a6d42c22a874a69231a2d7446bccfe9ce0cbaa > caSigningCert > > cert-pki-ca > > > > Any help in the deleting the key would be appreciated. > The certutil command can delete a key from a NSS database (certutil -F > -k <id> -d /etc/pki/pki-tomcat/alias). But before you delete this > private key, can you explain how you deduced that it was the root > cause? > I wouldn't advise to delete a private key if you're not 100% sure you > need to. > > Pki failing to start after an upgrade often happens when the > certificate > "subsystemCert cert-pki-ca" stored in /etc/pki/pki-tomcat/alias does > not > match the content of the usercertificate or description stored in > uid=pkidbuser,ou=people,o=ipaca. > > flo > > > > Thanks > > > > _Uz > > > > _______________________________________________ > > FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org <mailto:freeipa-users@lists.fedorahosted.org> > <mailto:freeipa-users@lists.fedorahosted.org <mailto:freeipa-users@lists.fedorahosted.org>> > > To unsubscribe send an email to > freeipa-users-leave@lists.fedorahosted.org <mailto:freeipa-users-leave@lists.fedorahosted.org> > <mailto:freeipa-users-leave@lists.fedorahosted.org <mailto:freeipa-users-leave@lists.fedorahosted.org>> > > Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html > > List Guidelines: > https://fedoraproject.org/wiki/Mailing_list_guidelines > > List Archives: > https://lists.fedorahosted.org/archives/list/freeipa-users@lists.fedorahosted.org > > > > > > _______________________________________________ > FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org <mailto:freeipa-users@lists.fedorahosted.org> > To unsubscribe send an email to freeipa-users-leave@lists.fedorahosted.org <mailto:freeipa-users-leave@lists.fedorahosted.org> > Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: https://lists.fedorahosted.org/archives/list/freeipa-users@lists.fedorahosted.org >
freeipa-users@lists.fedorahosted.org
-
Florence Blanc-Renaud -
Rob Crittenden -
Uzor Ide