I swear I have been reading and trying everything I can find on here and elsewhere today and I'm still having problems fixing my certs.
As appears to be a common problem, certmonger didn't auto-renew any of them.
IPA v4.6.9 running on Ubuntu 18.04; only the one server
IPA RA is fine ldap and krbtgt are "CA UNREACHABLE": Server at https://ipa01.simplyws.com/ipa/xml failed request, will retry: 903 (RPC failed at server. an internal error has occurred). Everything else is NEED_CSR_GEN_PIN including HTTP
Possibly ipa-cert-fix or pki-server cert-fix would take care of it, but they aren't in this version and I'm reluctant to upgrade the distro without proper preparation.
Everything starts without any problems. With the date set, everything is functioning like normal as far as I can tell.
I have rolled back the date successfully making sure to respect the 'notbefore' on ra-agent.pem
I've tried both manually: getcert resubmit -i xxx and restarting certmonger to no avail...
cn=ipa,cn=cas,cn=ca,$BASEDN and ou=authorities,ou=ca,o=ipaca appear to be fine.
Everything in /var/log/pki/pki-tomcat/ca/debug is FINE
There are some errors about missing .jar files in /var/log/pki/pki-tomcat/pki/debug
/var/log/ipa and /var/log/dirsrv don't seem to have anything of note.
Any thoughts would be greatly appreciated!
Sean McLennan via FreeIPA-users wrote:
I swear I have been reading and trying everything I can find on here and elsewhere today and I'm still having problems fixing my certs.
As appears to be a common problem, certmonger didn't auto-renew any of them.
IPA v4.6.9 running on Ubuntu 18.04; only the one server
Did you build this yourself? What is the history of this installation? Were there other servers at some point?
IPA RA is fine
ldap and krbtgt are "CA UNREACHABLE": Server at https://ipa01.simplyws.com/ipa/xml failed request, will retry: 903 (RPC failed at server. an internal error has occurred).
Check the Apache error log.
Everything else is NEED_CSR_GEN_PIN including HTTP
This suggests the tracking is really messed up. Can you provide the output of getcert list?
Possibly ipa-cert-fix or pki-server cert-fix would take care of it, but they aren't in this version and I'm reluctant to upgrade the distro without proper preparation.
It wouldn't fix the 389 or Apache certs.
Everything starts without any problems. With the date set, everything is functioning like normal as far as I can tell.
I have rolled back the date successfully making sure to respect the 'notbefore' on ra-agent.pem
Does this suggest that the RA agent cert was renewed at some point?
I've tried both manually: getcert resubmit -i xxx and restarting certmonger to no avail...
cn=ipa,cn=cas,cn=ca,$BASEDN and ou=authorities,ou=ca,o=ipaca appear to be fine.
Everything in /var/log/pki/pki-tomcat/ca/debug is FINE
There are some errors about missing .jar files in /var/log/pki/pki-tomcat/pki/debug
/var/log/ipa and /var/log/dirsrv don't seem to have anything of note.
Any thoughts would be greatly appreciated! _______________________________________________ FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org To unsubscribe send an email to freeipa-users-leave@lists.fedorahosted.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedorahosted.org/archives/list/freeipa-users@lists.fedorahoste...
Thank you for the reply!
Did you build this yourself? What is the history of this installation? Were there other servers at some point?
No, it's just from Ubuntu's repositories. It's about two years old and there's nothing of particular note; it was a straight-forward install, no unusual functions. Never had another server connected to it—always planned one but it's waiting on priority and budget.
Check the Apache error log
Thank you, that was helpful—kind of forget it's even part of the install. Appears there is a PyAsn1 Error? Maybe a Python2.7 vs. 3.6 thing?
[Fri Oct 09 00:00:25.453485 2020] [wsgi:error] [pid 7034] [remote 10.1.5.4:59838] ipa: INFO: [xmlserver] host/ipa01.my.domain@MY.REALM: cert_request(u'MIIDvTCCAqUCAQAwNDEVMBMGA1UEChMMU0lNUExZV1MuQ09NMRswGQYDVQQDExJpcGEwMS5zaW1wbHl3cy5jb20wggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQCxh0AiaCsveX7JROzezR4lk6Nk2KmsmQ+s3yrOzjVzVqNWCk5w3Y4vVjefTvdZpHTwKDO8hD++wvEyavYe1eYDESX4Py7UfNtOQsuMHnbUAHDLyTZyVzDt5r2abUHmhgYlgHfrrrNYFnfgk951PQrLcPd+07neJQjulPtH0BSjCH05Y8Ss3UkTPvSBfBTfFWsUVKl6tOuMbbNDoFo9bAzwH3qzSTb0c5B1SYC2vx6A7fBEqrKIFI10eOKZjzzWI65V9zjhAz3ShKDh+RVKl7R5ulRnU1wgetX1+LC8BGu9bR4A1uza12ZK8PrPmKQjL7tYF385ht4lLG0IjI5DMSf/AgMBAAGgggFCMCUGCSqGSIb3DQEJFDEYHhYAUwBlAHIAdgBlAHIALQBDAGUAcgB0MIIBFwYJKoZIhvcNAQkOMYIBCDCCAQQwgZ0GA1UdEQEBAASBkjCBj4ISaXBhMDEuc2ltcGx5d3MuY29toDQGCisGAQQBgjcUAgOgJgwkbGRhcC9pcGEwMS5zaW1wbHl3cy5jb21AU0lNUExZV1MuQ09NoEMGBisGAQUCAqA5MDegDhsMU0lNUExZV1MuQ09NoSUwI6ADAgEBoRwwGhsEbGRhcBsSaXBhMDEuc2ltcGx5d3MuY29tMAwGA1UdEwEB/wQCMAAwIAYDVR0OAQEABBYEFB0fN/R8MTxO1C1eb/Rgic+H6mFgMDIGCSsGAQQBgjcUAgEBAAQiHiAAYwBhAEkAUABBAHMAZQByAHYAaQBjAGUAQwBlAHIAdDANBgkqhkiG9w0BAQsFAAOCAQEAYhcKonl8W3qFdTnWfvG1ZdjeI3LWPF8DdjhZ3RuOtvukWIvhxNawoRrTBk7mnbrjaFQK48FvsqW19fo1h5746G3q3TqTimgPpplRkK7G2E3+JVtt8g5zLAXdiXnwwejcgH4Q6Y+Xr3XKJoGMjQCL34TLO+mNE6zaAj03KzFbd5oOTiqRXvSSP/dhQj5yY6yIu94+ri6jKnKUpWjqo0EUkhPHtMqm9hAPwJQ99Ns5lLNGOX4TPmiP9bBSZW+pginZ4w8wyylc00+8QzPnbe9T+JJOjU/bfUytmsHbDEcpJ4xFIzZcuqBBfLWRDDfBFG4Ajk24uvNfuZhlW8NK07LsEA==', profile_id=u'caIPAserviceCert', principal=u'ldap/ipa01.my.domain@MY.REALM', add=True, version=u'2.51'): InternalError [Fri Oct 09 00:00:35.402170 2020] [wsgi:error] [pid 7033] [remote 10.1.5.4:59876] ipa: ERROR: non-public: PyAsn1Error: <TagSet object at 0x7f378035fd10 tags 0:32:16> not in asn1Spec: <OctetString schema object at 0x7f377b8f99d0 tagSet <TagSet object at 0x7f379ae94290 tags 0:0:4> encoding iso-8859-1> [Fri Oct 09 00:00:35.402274 2020] [wsgi:error] [pid 7033] [remote 10.1.5.4:59876] Traceback (most recent call last): [Fri Oct 09 00:00:35.402288 2020] [wsgi:error] [pid 7033] [remote 10.1.5.4:59876] File "/usr/lib/python2.7/dist-packages/ipaserver/rpcserver.py", line 367, in wsgi_execute [Fri Oct 09 00:00:35.402299 2020] [wsgi:error] [pid 7033] [remote 10.1.5.4:59876] result = command(*args, **options) [Fri Oct 09 00:00:35.402309 2020] [wsgi:error] [pid 7033] [remote 10.1.5.4:59876] File "/usr/lib/python2.7/dist-packages/ipalib/frontend.py", line 450, in __call__ [Fri Oct 09 00:00:35.402320 2020] [wsgi:error] [pid 7033] [remote 10.1.5.4:59876] return self.__do_call(*args, **options) [Fri Oct 09 00:00:35.402330 2020] [wsgi:error] [pid 7033] [remote 10.1.5.4:59876] File "/usr/lib/python2.7/dist-packages/ipalib/frontend.py", line 478, in __do_call [Fri Oct 09 00:00:35.402341 2020] [wsgi:error] [pid 7033] [remote 10.1.5.4:59876] ret = self.run(*args, **options) [Fri Oct 09 00:00:35.402351 2020] [wsgi:error] [pid 7033] [remote 10.1.5.4:59876] File "/usr/lib/python2.7/dist-packages/ipalib/frontend.py", line 800, in run [Fri Oct 09 00:00:35.402361 2020] [wsgi:error] [pid 7033] [remote 10.1.5.4:59876] return self.execute(*args, **options) [Fri Oct 09 00:00:35.402371 2020] [wsgi:error] [pid 7033] [remote 10.1.5.4:59876] File "/usr/lib/python2.7/dist-packages/ipaserver/plugins/cert.py", line 884, in execute [Fri Oct 09 00:00:35.402382 2020] [wsgi:error] [pid 7033] [remote 10.1.5.4:59876] self.obj._parse(result, all) [Fri Oct 09 00:00:35.402392 2020] [wsgi:error] [pid 7033] [remote 10.1.5.4:59876] File "/usr/lib/python2.7/dist-packages/ipaserver/plugins/cert.py", line 493, in _parse [Fri Oct 09 00:00:35.402402 2020] [wsgi:error] [pid 7033] [remote 10.1.5.4:59876] cert.san_general_names) [Fri Oct 09 00:00:35.402412 2020] [wsgi:error] [pid 7033] [remote 10.1.5.4:59876] File "/usr/lib/python2.7/dist-packages/ipalib/x509.py", line 318, in san_general_names [Fri Oct 09 00:00:35.402451 2020] [wsgi:error] [pid 7033] [remote 10.1.5.4:59876] gns = self.__pyasn1_get_san_general_names() [Fri Oct 09 00:00:35.402462 2020] [wsgi:error] [pid 7033] [remote 10.1.5.4:59876] File "/usr/lib/python2.7/dist-packages/ipalib/x509.py", line 350, in __pyasn1_get_san_general_names [Fri Oct 09 00:00:35.402473 2020] [wsgi:error] [pid 7033] [remote 10.1.5.4:59876] ext['extnValue'], asn1Spec=univ.OctetString())[0] [Fri Oct 09 00:00:35.402483 2020] [wsgi:error] [pid 7033] [remote 10.1.5.4:59876] File "/usr/lib/python2.7/dist-packages/pyasn1/codec/ber/decoder.py", line 1318, in __call__ [Fri Oct 09 00:00:35.402494 2020] [wsgi:error] [pid 7033] [remote 10.1.5.4:59876] '%s not in asn1Spec: %r' % (tagSet, asn1Spec) [Fri Oct 09 00:00:35.402505 2020] [wsgi:error] [pid 7033] [remote 10.1.5.4:59876] PyAsn1Error: <TagSet object at 0x7f378035fd10 tags 0:32:16> not in asn1Spec: <OctetString schema object at 0x7f377b8f99d0 tagSet <TagSet object at 0x7f379ae94290 tags 0:0:4> encoding iso-8859-1>
This suggests the tracking is really messed up. Can you provide the output of getcert list?
Below.
Possibly ipa-cert-fix or pki-server cert-fix would take care of it, but they aren't in this version and I'm reluctant to upgrade the distro without proper preparation.
It wouldn't fix the 389 or Apache certs.
Thanks—glad I didn't go through that then!
Everything starts without any problems. With the date set, everything is functioning like normal as far as I can tell.
I have rolled back the date successfully making sure to respect the 'notbefore' on ra-agent.pem
Does this suggest that the RA agent cert was renewed at some point?
Yeah, I suppose it must have been successfully renewed by certmonger... I didn't think too hard about it since it wasn't expired:
Owner: CN=IPA RA, O=MY.REALM Issuer: CN=Certificate Authority, O=MY.REALM Serial number: 11 Valid from: Sat Sep 12 02:33:38 MDT 2020 until: Fri Sep 02 02:33:38 MDT 2022 Certificate fingerprints: SHA1: A8:32:C6:B4:C1:BF:C8:54:6B:35:F6:C7:DF:68:FB:47:73:C7:B4:2C SHA256: 5F:E8:77:BA:72:E4:64:56:E7:23:54:32:56:0D:66:7A:03:04:0F:04:7C:CE:E6:25:44:4A:15:B1:06:81:05:4A Signature algorithm name: SHA256withRSA Subject Public Key Algorithm: 2048-bit RSA key Version: 3
-----------
Number of certificates and requests being tracked: 9. Request ID '20181021083324': status: MONITORING stuck: no key pair storage: type=FILE,location='/var/lib/ipa/ra-agent.key' certificate: type=FILE,location='/var/lib/ipa/ra-agent.pem' CA: dogtag-ipa-ca-renew-agent issuer: CN=Certificate Authority,O=MY.REALM subject: CN=IPA RA,O=MY.REALM expires: 2022-09-02 02:33:38 MDT key usage: digitalSignature,keyEncipherment,dataEncipherment eku: id-kp-serverAuth,id-kp-clientAuth pre-save command: /usr/lib/ipa/certmonger/renew_ra_cert_pre post-save command: /usr/lib/ipa/certmonger/renew_ra_cert track: yes auto-renew: yes Request ID '20181021083404': status: MONITORING stuck: no key pair storage: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='auditSigningCert cert-pki-ca',token='NSS Certificate DB',pin set certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='auditSigningCert cert-pki-ca',token='NSS Certificate DB' CA: dogtag-ipa-ca-renew-agent issuer: CN=Certificate Authority,O=MY.REALM subject: CN=localhost expires: 2022-09-05 12:15:19 MDT key usage: digitalSignature,keyEncipherment,dataEncipherment eku: id-kp-serverAuth,id-kp-clientAuth pre-save command: /usr/lib/ipa/certmonger/stop_pkicad post-save command: /usr/lib/ipa/certmonger/renew_ca_cert "auditSigningCert cert-pki-ca" track: yes auto-renew: yes Request ID '20181021083405': status: NEED_CSR_GEN_TOKEN stuck: yes key pair storage: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='ocspSigningCert cert-pki-ca',token='NSS Certificate DB',pin set certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='ocspSigningCert cert-pki-ca',token='NSS Certificate DB' CA: dogtag-ipa-ca-renew-agent issuer: CN=Certificate Authority,O=MY.REALM subject: CN=localhost expires: 2020-10-13 12:14:21 MDT key usage: digitalSignature,keyEncipherment,dataEncipherment eku: id-kp-serverAuth,id-kp-clientAuth pre-save command: /usr/lib/ipa/certmonger/stop_pkicad post-save command: /usr/lib/ipa/certmonger/renew_ca_cert "ocspSigningCert cert-pki-ca" track: yes auto-renew: yes Request ID '20181021083406': status: NEED_CSR_GEN_TOKEN stuck: yes key pair storage: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='subsystemCert cert-pki-ca',token='NSS Certificate DB',pin set certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='subsystemCert cert-pki-ca',token='NSS Certificate DB' CA: dogtag-ipa-ca-renew-agent issuer: CN=Certificate Authority,O=MY.REALM subject: CN=localhost expires: 2020-10-13 12:15:01 MDT key usage: digitalSignature,keyEncipherment,dataEncipherment eku: id-kp-serverAuth,id-kp-clientAuth pre-save command: /usr/lib/ipa/certmonger/stop_pkicad post-save command: /usr/lib/ipa/certmonger/renew_ca_cert "subsystemCert cert-pki-ca" track: yes auto-renew: yes Request ID '20181021083407': status: NEED_CSR_GEN_TOKEN stuck: yes key pair storage: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='caSigningCert cert-pki-ca',token='NSS Certificate DB',pin set certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='caSigningCert cert-pki-ca',token='NSS Certificate DB' CA: dogtag-ipa-ca-renew-agent issuer: CN=Certificate Authority,O=MY.REALM subject: CN=localhost expires: 2020-10-10 02:34:28 MDT key usage: digitalSignature,keyEncipherment,dataEncipherment eku: id-kp-serverAuth,id-kp-clientAuth pre-save command: /usr/lib/ipa/certmonger/stop_pkicad post-save command: /usr/lib/ipa/certmonger/renew_ca_cert "caSigningCert cert-pki-ca" track: yes auto-renew: yes Request ID '20181021083408': status: NEED_CSR_GEN_TOKEN stuck: yes key pair storage: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='Server-Cert cert-pki-ca',token='NSS Certificate DB',pin set certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='Server-Cert cert-pki-ca',token='NSS Certificate DB' CA: dogtag-ipa-ca-renew-agent issuer: CN=Certificate Authority,O=MY.REALM subject: CN=localhost expires: 2020-10-13 12:14:29 MDT key usage: digitalSignature,keyEncipherment,dataEncipherment eku: id-kp-serverAuth,id-kp-clientAuth pre-save command: /usr/lib/ipa/certmonger/stop_pkicad post-save command: /usr/lib/ipa/certmonger/renew_ca_cert "Server-Cert cert-pki-ca" track: yes auto-renew: yes Request ID '20181021083613': status: CA_UNREACHABLE ca-error: Server at https://ipa01.my.domain/ipa/xml failed request, will retry: 903 (RPC failed at server. an internal error has occurred). stuck: no key pair storage: type=NSSDB,location='/etc/dirsrv/slapd-MY-REALM',nickname='Server-Cert',token='NSS Certificate DB',pinfile='/etc/dirsrv/slapd-MY-REALM/pwdfile.txt' certificate: type=NSSDB,location='/etc/dirsrv/slapd-MY-REALM',nickname='Server-Cert',token='NSS Certificate DB' CA: IPA issuer: CN=Certificate Authority,O=MY.REALM subject: CN=ipa01.my.domain,O=MY.REALM expires: 2020-10-21 02:36:13 MDT dns: ipa01.my.domain principal name: ldap/ipa01.my.domain@MY.REALM key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment eku: id-kp-serverAuth,id-kp-clientAuth pre-save command: post-save command: /usr/lib/ipa/certmonger/restart_dirsrv MY-REALM track: yes auto-renew: yes Request ID '20181021083714': status: NEED_CSR_GEN_PIN stuck: yes key pair storage: type=FILE,location='/var/lib/ipa/private/httpd.key',pinfile='/var/lib/ipa/passwds/ipa01.my.domain-443-RSA' certificate: type=FILE,location='/var/lib/ipa/certs/httpd.crt' CA: IPA issuer: CN=Certificate Authority,O=MY.REALM subject: CN=ipa01.my.domain,O=MY.REALM expires: 2020-10-21 02:37:17 MDT dns: ipa01.my.domain principal name: HTTP/ipa01.my.domain@MY.REALM key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment eku: id-kp-serverAuth,id-kp-clientAuth pre-save command: post-save command: /usr/lib/ipa/certmonger/restart_httpd track: yes auto-renew: yes Request ID '20181021083724': status: CA_UNREACHABLE ca-error: Server at https://ipa01.my.domain/ipa/xml failed request, will retry: 903 (RPC failed at server. an internal error has occurred). stuck: no key pair storage: type=FILE,location='/var/lib/krb5kdc/kdc.key' certificate: type=FILE,location='/var/lib/krb5kdc/kdc.crt' CA: IPA issuer: CN=Certificate Authority,O=MY.REALM subject: CN=ipa01.my.domain,O=MY.REALM expires: 2020-10-21 02:37:25 MDT principal name: krbtgt/MY.REALM@MY.REALM key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment eku: id-kp-serverAuth,id-pkinit-KPKdc pre-save command: post-save command: /usr/lib/ipa/certmonger/renew_kdc_cert track: yes auto-renew: yes
Sean McLennan via FreeIPA-users wrote:
Thank you for the reply!
Did you build this yourself? What is the history of this installation? Were there other servers at some point?
No, it's just from Ubuntu's repositories. It's about two years old and there's nothing of particular note; it was a straight-forward install, no unusual functions. Never had another server connected to it—always planned one but it's waiting on priority and budget.
Check the Apache error log
Thank you, that was helpful—kind of forget it's even part of the install. Appears there is a PyAsn1 Error? Maybe a Python2.7 vs. 3.6 thing?
[Fri Oct 09 00:00:25.453485 2020] [wsgi:error] [pid 7034] [remote 10.1.5.4:59838] ipa: INFO: [xmlserver] host/ipa01.my.domain@MY.REALM: cert_request(u'MIIDvTCCAqUCAQAwNDEVMBMGA1UEChMMU0lNUExZV1MuQ09NMRswGQYDVQQDExJpcGEwMS5zaW1wbHl3cy5jb20wggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQCxh0AiaCsveX7JROzezR4lk6Nk2KmsmQ+s3yrOzjVzVqNWCk5w3Y4vVjefTvdZpHTwKDO8hD++wvEyavYe1eYDESX4Py7UfNtOQsuMHnbUAHDLyTZyVzDt5r2abUHmhgYlgHfrrrNYFnfgk951PQrLcPd+07neJQjulPtH0BSjCH05Y8Ss3UkTPvSBfBTfFWsUVKl6tOuMbbNDoFo9bAzwH3qzSTb0c5B1SYC2vx6A7fBEqrKIFI10eOKZjzzWI65V9zjhAz3ShKDh+RVKl7R5ulRnU1wgetX1+LC8BGu9bR4A1uza12ZK8PrPmKQjL7tYF385ht4lLG0IjI5DMSf/AgMBAAGgggFCMCUGCSqGSIb3DQEJFDEYHhYAUwBlAHIAdgBlAHIALQBDAGUAcgB0MIIBFwYJKoZIhvcNAQkOMYIBCDCCAQQwgZ0GA1UdEQEBAASBkjCBj4ISaXBhMDEuc2ltcGx5d3MuY29toDQGCisGAQQBgjcUAgOgJgwkbGRhcC9pcGEwMS5zaW1wbHl3cy5jb21AU0lNUExZV1MuQ09NoEMGBisGAQUCAqA5MDegDhsMU0lNUExZV1MuQ09NoSUwI6ADAgEBoRwwGhsEbGRhcBsSaXBhMDEuc2ltcGx5d3MuY29tMAwGA1UdEwEB/wQCMAAwIAYDVR0OAQEABBYEFB0fN/R8MTxO1C1eb/Rgic+H6mFgMDIGCSsGAQQBgjcUAgEBAAQiHiAAYwBhAEkAUABBAHMAZQByAHYAaQBjAGUAQwBlAHIAdDANBgkqhkiG9w0BAQsFAAOCAQEAYhcKonl8W3qFdTnWfvG1ZdjeI3LWPF8DdjhZ3RuOtvukWIvhxNawoRrTBk7mnbrjaFQK48FvsqW19fo1h5746G3q3TqTimgPpplRkK7G2E3+JVtt8g5zLAXdiXnwwejcgH4Q6Y+Xr3XKJoGMjQCL34TLO+mNE6zaAj03KzFbd5oOTiqRXvSSP/dhQj5yY6yIu94+ri6jKnKUpWjqo0EUkhPHtMqm9hAPwJQ99Ns5lLNGOX4TPmiP9bBSZW+pginZ4w8wyylc00+8QzPnbe9T+JJOjU/bfUytmsHbDEcpJ4xFIzZcuqBBfLWRDDfBFG4Ajk24uvNfuZhlW8NK07LsEA==', profile_id=u'caIPAserviceCert', principal=u'ldap/ipa01.my.domain@MY.REALM', add=True, version=u'2.51'): InternalError [Fri Oct 09 00:00:35.402170 2020] [wsgi:error] [pid 7033] [remote 10.1.5.4:59876] ipa: ERROR: non-public: PyAsn1Error: <TagSet object at 0x7f378035fd10 tags 0:32:16> not in asn1Spec: <OctetString schema object at 0x7f377b8f99d0 tagSet <TagSet object at 0x7f379ae94290 tags 0:0:4> encoding iso-8859-1> [Fri Oct 09 00:00:35.402274 2020] [wsgi:error] [pid 7033] [remote 10.1.5.4:59876] Traceback (most recent call last): [Fri Oct 09 00:00:35.402288 2020] [wsgi:error] [pid 7033] [remote 10.1.5.4:59876] File "/usr/lib/python2.7/dist-packages/ipaserver/rpcserver.py", line 367, in wsgi_execute [Fri Oct 09 00:00:35.402299 2020] [wsgi:error] [pid 7033] [remote 10.1.5.4:59876] result = command(*args, **options) [Fri Oct 09 00:00:35.402309 2020] [wsgi:error] [pid 7033] [remote 10.1.5.4:59876] File "/usr/lib/python2.7/dist-packages/ipalib/frontend.py", line 450, in __call__ [Fri Oct 09 00:00:35.402320 2020] [wsgi:error] [pid 7033] [remote 10.1.5.4:59876] return self.__do_call(*args, **options) [Fri Oct 09 00:00:35.402330 2020] [wsgi:error] [pid 7033] [remote 10.1.5.4:59876] File "/usr/lib/python2.7/dist-packages/ipalib/frontend.py", line 478, in __do_call [Fri Oct 09 00:00:35.402341 2020] [wsgi:error] [pid 7033] [remote 10.1.5.4:59876] ret = self.run(*args, **options) [Fri Oct 09 00:00:35.402351 2020] [wsgi:error] [pid 7033] [remote 10.1.5.4:59876] File "/usr/lib/python2.7/dist-packages/ipalib/frontend.py", line 800, in run [Fri Oct 09 00:00:35.402361 2020] [wsgi:error] [pid 7033] [remote 10.1.5.4:59876] return self.execute(*args, **options) [Fri Oct 09 00:00:35.402371 2020] [wsgi:error] [pid 7033] [remote 10.1.5.4:59876] File "/usr/lib/python2.7/dist-packages/ipaserver/plugins/cert.py", line 884, in execute [Fri Oct 09 00:00:35.402382 2020] [wsgi:error] [pid 7033] [remote 10.1.5.4:59876] self.obj._parse(result, all) [Fri Oct 09 00:00:35.402392 2020] [wsgi:error] [pid 7033] [remote 10.1.5.4:59876] File "/usr/lib/python2.7/dist-packages/ipaserver/plugins/cert.py", line 493, in _parse [Fri Oct 09 00:00:35.402402 2020] [wsgi:error] [pid 7033] [remote 10.1.5.4:59876] cert.san_general_names) [Fri Oct 09 00:00:35.402412 2020] [wsgi:error] [pid 7033] [remote 10.1.5.4:59876] File "/usr/lib/python2.7/dist-packages/ipalib/x509.py", line 318, in san_general_names [Fri Oct 09 00:00:35.402451 2020] [wsgi:error] [pid 7033] [remote 10.1.5.4:59876] gns = self.__pyasn1_get_san_general_names() [Fri Oct 09 00:00:35.402462 2020] [wsgi:error] [pid 7033] [remote 10.1.5.4:59876] File "/usr/lib/python2.7/dist-packages/ipalib/x509.py", line 350, in __pyasn1_get_san_general_names [Fri Oct 09 00:00:35.402473 2020] [wsgi:error] [pid 7033] [remote 10.1.5.4:59876] ext['extnValue'], asn1Spec=univ.OctetString())[0] [Fri Oct 09 00:00:35.402483 2020] [wsgi:error] [pid 7033] [remote 10.1.5.4:59876] File "/usr/lib/python2.7/dist-packages/pyasn1/codec/ber/decoder.py", line 1318, in __call__ [Fri Oct 09 00:00:35.402494 2020] [wsgi:error] [pid 7033] [remote 10.1.5.4:59876] '%s not in asn1Spec: %r' % (tagSet, asn1Spec) [Fri Oct 09 00:00:35.402505 2020] [wsgi:error] [pid 7033] [remote 10.1.5.4:59876] PyAsn1Error: <TagSet object at 0x7f378035fd10 tags 0:32:16> not in asn1Spec: <OctetString schema object at 0x7f377b8f99d0 tagSet <TagSet object at 0x7f379ae94290 tags 0:0:4> encoding iso-8859-1>
What version of python-pyasn1 and pyasn1-modules is installed? You might try upgrading/downgrading them to see if that helps.
rob
This suggests the tracking is really messed up. Can you provide the output of getcert list?
Below.
Possibly ipa-cert-fix or pki-server cert-fix would take care of it, but they aren't in this version and I'm reluctant to upgrade the distro without proper preparation.
It wouldn't fix the 389 or Apache certs.
Thanks—glad I didn't go through that then!
Everything starts without any problems. With the date set, everything is functioning like normal as far as I can tell.
I have rolled back the date successfully making sure to respect the 'notbefore' on ra-agent.pem
Does this suggest that the RA agent cert was renewed at some point?
Yeah, I suppose it must have been successfully renewed by certmonger... I didn't think too hard about it since it wasn't expired:
Owner: CN=IPA RA, O=MY.REALM Issuer: CN=Certificate Authority, O=MY.REALM Serial number: 11 Valid from: Sat Sep 12 02:33:38 MDT 2020 until: Fri Sep 02 02:33:38 MDT 2022 Certificate fingerprints: SHA1: A8:32:C6:B4:C1:BF:C8:54:6B:35:F6:C7:DF:68:FB:47:73:C7:B4:2C SHA256: 5F:E8:77:BA:72:E4:64:56:E7:23:54:32:56:0D:66:7A:03:04:0F:04:7C:CE:E6:25:44:4A:15:B1:06:81:05:4A Signature algorithm name: SHA256withRSA Subject Public Key Algorithm: 2048-bit RSA key Version: 3
Number of certificates and requests being tracked: 9. Request ID '20181021083324': status: MONITORING stuck: no key pair storage: type=FILE,location='/var/lib/ipa/ra-agent.key' certificate: type=FILE,location='/var/lib/ipa/ra-agent.pem' CA: dogtag-ipa-ca-renew-agent issuer: CN=Certificate Authority,O=MY.REALM subject: CN=IPA RA,O=MY.REALM expires: 2022-09-02 02:33:38 MDT key usage: digitalSignature,keyEncipherment,dataEncipherment eku: id-kp-serverAuth,id-kp-clientAuth pre-save command: /usr/lib/ipa/certmonger/renew_ra_cert_pre post-save command: /usr/lib/ipa/certmonger/renew_ra_cert track: yes auto-renew: yes Request ID '20181021083404': status: MONITORING stuck: no key pair storage: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='auditSigningCert cert-pki-ca',token='NSS Certificate DB',pin set certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='auditSigningCert cert-pki-ca',token='NSS Certificate DB' CA: dogtag-ipa-ca-renew-agent issuer: CN=Certificate Authority,O=MY.REALM subject: CN=localhost expires: 2022-09-05 12:15:19 MDT key usage: digitalSignature,keyEncipherment,dataEncipherment eku: id-kp-serverAuth,id-kp-clientAuth pre-save command: /usr/lib/ipa/certmonger/stop_pkicad post-save command: /usr/lib/ipa/certmonger/renew_ca_cert "auditSigningCert cert-pki-ca" track: yes auto-renew: yes Request ID '20181021083405': status: NEED_CSR_GEN_TOKEN stuck: yes key pair storage: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='ocspSigningCert cert-pki-ca',token='NSS Certificate DB',pin set certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='ocspSigningCert cert-pki-ca',token='NSS Certificate DB' CA: dogtag-ipa-ca-renew-agent issuer: CN=Certificate Authority,O=MY.REALM subject: CN=localhost expires: 2020-10-13 12:14:21 MDT key usage: digitalSignature,keyEncipherment,dataEncipherment eku: id-kp-serverAuth,id-kp-clientAuth pre-save command: /usr/lib/ipa/certmonger/stop_pkicad post-save command: /usr/lib/ipa/certmonger/renew_ca_cert "ocspSigningCert cert-pki-ca" track: yes auto-renew: yes Request ID '20181021083406': status: NEED_CSR_GEN_TOKEN stuck: yes key pair storage: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='subsystemCert cert-pki-ca',token='NSS Certificate DB',pin set certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='subsystemCert cert-pki-ca',token='NSS Certificate DB' CA: dogtag-ipa-ca-renew-agent issuer: CN=Certificate Authority,O=MY.REALM subject: CN=localhost expires: 2020-10-13 12:15:01 MDT key usage: digitalSignature,keyEncipherment,dataEncipherment eku: id-kp-serverAuth,id-kp-clientAuth pre-save command: /usr/lib/ipa/certmonger/stop_pkicad post-save command: /usr/lib/ipa/certmonger/renew_ca_cert "subsystemCert cert-pki-ca" track: yes auto-renew: yes Request ID '20181021083407': status: NEED_CSR_GEN_TOKEN stuck: yes key pair storage: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='caSigningCert cert-pki-ca',token='NSS Certificate DB',pin set certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='caSigningCert cert-pki-ca',token='NSS Certificate DB' CA: dogtag-ipa-ca-renew-agent issuer: CN=Certificate Authority,O=MY.REALM subject: CN=localhost expires: 2020-10-10 02:34:28 MDT key usage: digitalSignature,keyEncipherment,dataEncipherment eku: id-kp-serverAuth,id-kp-clientAuth pre-save command: /usr/lib/ipa/certmonger/stop_pkicad post-save command: /usr/lib/ipa/certmonger/renew_ca_cert "caSigningCert cert-pki-ca" track: yes auto-renew: yes Request ID '20181021083408': status: NEED_CSR_GEN_TOKEN stuck: yes key pair storage: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='Server-Cert cert-pki-ca',token='NSS Certificate DB',pin set certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='Server-Cert cert-pki-ca',token='NSS Certificate DB' CA: dogtag-ipa-ca-renew-agent issuer: CN=Certificate Authority,O=MY.REALM subject: CN=localhost expires: 2020-10-13 12:14:29 MDT key usage: digitalSignature,keyEncipherment,dataEncipherment eku: id-kp-serverAuth,id-kp-clientAuth pre-save command: /usr/lib/ipa/certmonger/stop_pkicad post-save command: /usr/lib/ipa/certmonger/renew_ca_cert "Server-Cert cert-pki-ca" track: yes auto-renew: yes Request ID '20181021083613': status: CA_UNREACHABLE ca-error: Server at https://ipa01.my.domain/ipa/xml failed request, will retry: 903 (RPC failed at server. an internal error has occurred). stuck: no key pair storage: type=NSSDB,location='/etc/dirsrv/slapd-MY-REALM',nickname='Server-Cert',token='NSS Certificate DB',pinfile='/etc/dirsrv/slapd-MY-REALM/pwdfile.txt' certificate: type=NSSDB,location='/etc/dirsrv/slapd-MY-REALM',nickname='Server-Cert',token='NSS Certificate DB' CA: IPA issuer: CN=Certificate Authority,O=MY.REALM subject: CN=ipa01.my.domain,O=MY.REALM expires: 2020-10-21 02:36:13 MDT dns: ipa01.my.domain principal name: ldap/ipa01.my.domain@MY.REALM key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment eku: id-kp-serverAuth,id-kp-clientAuth pre-save command: post-save command: /usr/lib/ipa/certmonger/restart_dirsrv MY-REALM track: yes auto-renew: yes Request ID '20181021083714': status: NEED_CSR_GEN_PIN stuck: yes key pair storage: type=FILE,location='/var/lib/ipa/private/httpd.key',pinfile='/var/lib/ipa/passwds/ipa01.my.domain-443-RSA' certificate: type=FILE,location='/var/lib/ipa/certs/httpd.crt' CA: IPA issuer: CN=Certificate Authority,O=MY.REALM subject: CN=ipa01.my.domain,O=MY.REALM expires: 2020-10-21 02:37:17 MDT dns: ipa01.my.domain principal name: HTTP/ipa01.my.domain@MY.REALM key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment eku: id-kp-serverAuth,id-kp-clientAuth pre-save command: post-save command: /usr/lib/ipa/certmonger/restart_httpd track: yes auto-renew: yes Request ID '20181021083724': status: CA_UNREACHABLE ca-error: Server at https://ipa01.my.domain/ipa/xml failed request, will retry: 903 (RPC failed at server. an internal error has occurred). stuck: no key pair storage: type=FILE,location='/var/lib/krb5kdc/kdc.key' certificate: type=FILE,location='/var/lib/krb5kdc/kdc.crt' CA: IPA issuer: CN=Certificate Authority,O=MY.REALM subject: CN=ipa01.my.domain,O=MY.REALM expires: 2020-10-21 02:37:25 MDT principal name: krbtgt/MY.REALM@MY.REALM key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment eku: id-kp-serverAuth,id-pkinit-KPKdc pre-save command: post-save command: /usr/lib/ipa/certmonger/renew_kdc_cert track: yes auto-renew: yes _______________________________________________ FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org To unsubscribe send an email to freeipa-users-leave@lists.fedorahosted.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedorahosted.org/archives/list/freeipa-users@lists.fedorahoste...
What version of python-pyasn1 and pyasn1-modules is installed? You might try upgrading/downgrading them to see if that helps.
There are two versions: python-pyasn1(-modules) python3-pyasn1(-modules)
I tried to uninstall the first with apt—doing so was also going to remove all of freeipa(!) so I did not go through with that!
Digging a bit more, it appears that ipa is using the Apache2 wsgi mod to run python code and mod_wsgi is specifically Python 2.7. Also, in the apache error trace all the ipa-specific python files are in /usr/lib/python2.7/dist-packages/ and those don't exist anywhere else. So I'm thinking it's unlikely to be a version problem.
I'm not sure how to get any more information... is there a way to try and manually make the same requests on the command line and maybe get something more useful? Or is there away to manually just replace the expired cert with a new on?
Thanks,
Sean
Sean McLennan via FreeIPA-users wrote:
What version of python-pyasn1 and pyasn1-modules is installed? You might try upgrading/downgrading them to see if that helps.
There are two versions: python-pyasn1(-modules) python3-pyasn1(-modules)
I tried to uninstall the first with apt—doing so was also going to remove all of freeipa(!) so I did not go through with that!
Digging a bit more, it appears that ipa is using the Apache2 wsgi mod to run python code and mod_wsgi is specifically Python 2.7. Also, in the apache error trace all the ipa-specific python files are in /usr/lib/python2.7/dist-packages/ and those don't exist anywhere else. So I'm thinking it's unlikely to be a version problem.
I'm not sure how to get any more information... is there a way to try and manually make the same requests on the command line and maybe get something more useful? Or is there away to manually just replace the expired cert with a new on?
I'm not great with Debian-based systems but apt show python-pyasn1 should provide the version of pyasn1 that is installed.
IPA 4.6.x is python2-based.
The problem isn't the request it's an ASN.1 parsing error. I'm guessing that the CA is issuing the new cert ok but because of the parsing issue it is blow up inside IPA so it can't be further processed.
So solving the python-pyasn1 issue could just fix everything. You might try downgrading it.
RHEL-7, which has IPA 4.6.6 uses python2-pyasn1-0.1.9-7.el7.
rob
I'm not great with Debian-based systems but apt show python-pyasn1 should provide the version of pyasn1 that is installed.
IPA 4.6.x is python2-based.
The problem isn't the request it's an ASN.1 parsing error. I'm guessing that the CA is issuing the new cert ok but because of the parsing issue it is blow up inside IPA so it can't be further processed.
So solving the python-pyasn1 issue could just fix everything. You might try downgrading it.
RHEL-7, which has IPA 4.6.6 uses python2-pyasn1-0.1.9-7.el7.
Took a while to get pyasn downgraded, but I still get the same error. :(
Sean
I'm not great with Debian-based systems but apt show python-pyasn1 should provide the version of pyasn1 that is installed.
IPA 4.6.x is python2-based.
The problem isn't the request it's an ASN.1 parsing error. I'm guessing that the CA is issuing the new cert ok but because of the parsing issue it is blow up inside IPA so it can't be further processed.
So solving the python-pyasn1 issue could just fix everything. You might try downgrading it.
RHEL-7, which has IPA 4.6.6 uses python2-pyasn1-0.1.9-7.el7.
I thought I would just bite the bullet and try upgrading the distribution and then presumably IPA, but it looks like Ubuntu has pulled freeipa-server from 20.04 entirely because of a bug in bind. :( And there doesn't appear to be backport or anything.
It occurred to me to look in the webui and after working around another bug on the Authenication>Certificates page, it is clear that new certs are being issued everytime certmonger tries—I now have >50 of the same two certs (two are created each time certmonger is restarted). If I try to view any of those, I get the identical PyASN1 error both on screen and in the apache log
Inferring from the logs and getcert list, I believe they are the certs in: /var/lib/krb5kdc and /etc/dirsrv/slapd-MYREALM-COM/Server-Cert
Are each of those being stored in the back end some where they might be exported? Or are they lost because they are not being written to disk?
Is there a way I can just generate new certificates or somehow manually bypass certmonger?
Thanks,
Sean
Sean McLennan via FreeIPA-users wrote:
I'm not great with Debian-based systems but apt show python-pyasn1 should provide the version of pyasn1 that is installed.
IPA 4.6.x is python2-based.
The problem isn't the request it's an ASN.1 parsing error. I'm guessing that the CA is issuing the new cert ok but because of the parsing issue it is blow up inside IPA so it can't be further processed.
So solving the python-pyasn1 issue could just fix everything. You might try downgrading it.
RHEL-7, which has IPA 4.6.6 uses python2-pyasn1-0.1.9-7.el7.
I thought I would just bite the bullet and try upgrading the distribution and then presumably IPA, but it looks like Ubuntu has pulled freeipa-server from 20.04 entirely because of a bug in bind. :( And there doesn't appear to be backport or anything.
It occurred to me to look in the webui and after working around another bug on the Authenication>Certificates page, it is clear that new certs are being issued everytime certmonger tries—I now have >50 of the same two certs (two are created each time certmonger is restarted). If I try to view any of those, I get the identical PyASN1 error both on screen and in the apache log
Inferring from the logs and getcert list, I believe they are the certs in: /var/lib/krb5kdc and /etc/dirsrv/slapd-MYREALM-COM/Server-Cert
Are each of those being stored in the back end some where they might be exported? Or are they lost because they are not being written to disk?
It wouldn't help since there are a bunch of other certificates that also need to be renewed and won't w/o a working CA.
Is there a way I can just generate new certificates or somehow manually bypass certmonger?
certmonger isn't the problem.
ipa-certfix can renew offline and would fix it but as discussed that isn't available.
What version of pyasn1 and pyasn1-modules do you have now? The version in RHEL that works with the 4.6 release hasn't been updated since 2016. It's upstream version 0.1.9 and modules version 0.0.8.
rob
What version of pyasn1 and pyasn1-modules do you have now? The version in RHEL that works with the 4.6 release hasn't been updated since 2016. It's upstream version 0.1.9 and modules version 0.0.8.
pyasn1: 0.4.2-3 pyasn1-modules: 0.2.1-0.2
But after you mentioned the version that RHEL used before I did try down-grading both to 0.1.9 and 0.0.8 and got exactly the same errors.
:(
Sean
Sean McLennan via FreeIPA-users wrote:
What version of pyasn1 and pyasn1-modules do you have now? The version in RHEL that works with the 4.6 release hasn't been updated since 2016. It's upstream version 0.1.9 and modules version 0.0.8.
pyasn1: 0.4.2-3 pyasn1-modules: 0.2.1-0.2
But after you mentioned the version that RHEL used before I did try down-grading both to 0.1.9 and 0.0.8 and got exactly the same errors.
Did you happen to restart Apache after downgrading the packages?
rob
:(
Sean _______________________________________________ FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org To unsubscribe send an email to freeipa-users-leave@lists.fedorahosted.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedorahosted.org/archives/list/freeipa-users@lists.fedorahoste...
I've have expected the traceback to change at least. Can you provide the latest?
I've already reverted it and I didn't specifically copy them. Because the older versions were not available in my repos, I had to install them with pip and to be sure that ipa was using the right one, I moved the distro's version and linked the old one.
I probably still have the tracebacks somewhere in my log files, but I'm not sure how I would distinguish which is which...
At this point, I'm not seeing a work around to fix this. What are my options here? Rebuilding from scratch on a better distro? Is there a way to migrate content?
Thanks,
Sean
You might try looking at the apt history log to see what else has been updated and try to determine which version you were running. Ubuntu has unattended upgrades enabled by default now so I wouldn't be surprised if that's what broke your install.
Sean McLennan via FreeIPA-users wrote:
I've have expected the traceback to change at least. Can you provide the latest?
I've already reverted it and I didn't specifically copy them. Because the older versions were not available in my repos, I had to install them with pip and to be sure that ipa was using the right one, I moved the distro's version and linked the old one.
I probably still have the tracebacks somewhere in my log files, but I'm not sure how I would distinguish which is which...
At this point, I'm not seeing a work around to fix this. What are my options here? Rebuilding from scratch on a better distro? Is there a way to migrate content?
The traceback you provided earlier is very confusing. The line numbers don't line up with the upstream release-4-6-9 tag so I can't quite tell where it's failing.
It looks like it's failing to get the SAN from the CSR to ensure you are only requesting valid values. Which means the cert wouldn't have been renewed yet, but you mentioned that there are a whole ton of already issued certs.
And some of other failures are completely unrelated to this since they renew directly against the CA.
A kludge that might work is to setup a CentOS 7 build as a new replica with a CA while time is forced in the past. If the new server comes up ok follow the downstream RHEL docs to make it the renewal master and see if you can get the certs to renew.
rob
Holy crap I managed it! Nearly.
don't line up with the upstream release-4-6-9 tag so I can't quite tell where it's failing.
I am fairly certain that somehow I didn't revert to the older pynas1 like I thought somehow... I'm not sure what I did differently, but this time, when apache restarted everything was very broken.
HOWEVER, in desperation I started looking at the code in comparison to the latest release and in x509.py ln 349, __pyasn1_get_san_general_names there was this:
der = ext['extnValue'] if pyasn1.__version__.startswith('0.3'): # pyasn1 <= 0.3.7 needs explicit unwrap of ANY container # see https://pagure.io/freeipa/issue/7685 der = decoder.decode(der, asn1Spec=univ.OctetString())[0]
copying the entire 'if' did not work, but knowing that my pyasn1 is
0.3.7, I tried just pulling it entirely and it worked.
My relief is palpable.
NOW, I still have an issue with the rest of the 'stuck' certs that NEED_CSR_GEN_PIN—I've tried restarting certmonger and getcert resubmit individually, but still no luck...
Any advice now?
Sean
Sean McLennan via FreeIPA-users wrote:
Holy crap I managed it! Nearly.
don't line up with the upstream release-4-6-9 tag so I can't quite tell where it's failing.
I am fairly certain that somehow I didn't revert to the older pynas1 like I thought somehow... I'm not sure what I did differently, but this time, when apache restarted everything was very broken.
HOWEVER, in desperation I started looking at the code in comparison to the latest release and in x509.py ln 349, __pyasn1_get_san_general_names there was this:
der = ext['extnValue'] if pyasn1.__version__.startswith('0.3'): # pyasn1 <= 0.3.7 needs explicit unwrap of ANY container # see https://pagure.io/freeipa/issue/7685 der = decoder.decode(der, asn1Spec=univ.OctetString())[0]
copying the entire 'if' did not work, but knowing that my pyasn1 is
0.3.7, I tried just pulling it entirely and it worked.
My relief is palpable.
NOW, I still have an issue with the rest of the 'stuck' certs that NEED_CSR_GEN_PIN—I've tried restarting certmonger and getcert resubmit individually, but still no luck...
Any advice now?
I'd suggest stopping certmonger and looking for the actual request file in /var/lib/certmonger/request (grep for id=<request id>).
Make sure that the value in key_pin matches the value in /etc/pki/pki-tomcat/alias/pwdfile.txt
If it doesn't fix it, then restart certmonger.
rob
NOW, I still have an issue with the rest of the 'stuck' certs that NEED_CSR_GEN_PIN—I've tried restarting certmonger and getcert resubmit individually, but still no luck...
Any advice now?
I'd suggest stopping certmonger and looking for the actual request file in /var/lib/certmonger/request (grep for id=<request id>).
Make sure that the value in key_pin matches the value in /etc/pki/pki-tomcat/alias/pwdfile.txt
If it doesn't fix it, then restart certmonger.
It wasn't and I did fix it (I think). The pin is in a file: /var/lib/ipa/passwords/ipa01.mydomain.com-443-RSA
Assuming this is for the associated key, I copied that value into /etc/pki/pki-tomcat/alias/pwdfile.txt — that did not work. I did the opposite (copied pwdfile.txt to the pin file) just for good measure. Also did not work.
Restarted ipa after each change before restarting certmonger.
Also I failed to notice that that there's only one that is NEED_CSR_GEN_PIN and the rest are NEED_CSR_GEN_TOKEN
Thanks again,
Sean
Another quick observation about the NEED_CSR_GEN_TOKEN|PIN...
Although I have rolled the date back, in the GUI (Authentication > Certificates) those certs are still showing as Status EXPIRED.
For example, I run keytool on /var/lib/ipa/certs/httpd.crt and valid until Oct 21 (system date is Oct 8). It's serial number a/10. In the GUI, I view 10; compare the fingerprints to make sure I'm looking at the same one, same 'valid until', but in the main list it still says EXPIRED.
Is it possible somewhere it got flagged as expired and that's blocking it's renewal?
Sean
On 27.10.2020 23.30, Sean McLennan via FreeIPA-users wrote:
I swear I have been reading and trying everything I can find on here and elsewhere today and I'm still having problems fixing my certs.
As appears to be a common problem, certmonger didn't auto-renew any of them.
IPA v4.6.9 running on Ubuntu 18.04; only the one server
Where did you get this? 4.6.9 was never in ubuntu, 18.04 shipped with a 4.7.0 pre-release version..
Where did you get this? 4.6.9 was never in ubuntu, 18.04 shipped with a 4.7.0 pre-release version..
Oh. I have only ever installed from the repositories. The 4.6.9 came from:
ipa --version VERSION: 4.6.90.pre1+git20180411, API_VERSION: 2.229
but as soon as you said that I checked apt show and all the freeipa packages are:
4.7.0~pre1+git20180411-2ubuntu2
I'm sorry, it didn't occur to me that they would be different. :( That likely explains why the trace-back line-numbers were not working for Rob.
Sean
On 2020-11-03 2:30 p.m., Rob Crittenden via FreeIPA-users wrote:
I'd suggest stopping certmonger and looking for the actual request file in /var/lib/certmonger/request (grep for id=<request id>).
Make sure that the value in key_pin matches the value in /etc/pki/pki-tomcat/alias/pwdfile.txt
I observed something that feels relevant. I was following the instructions here: https://floblanc.wordpress.com/2016/12/19/troubleshooting-certmonger-issues-...
The log files don't show exactly what he says—I'm not sure if that's a version issue or something else. Not really finding errors (see below)
I don't seem to have 'ipaCert' anywhere?
certutil -L -d /etc/pki/pki-tomcat/alias/ -n 'subsystemCert cert-pki-ca'
fails with
certutil: Could not find cert: subsystemCert cert-pki-ca : PR_FILE_NOT_FOUND_ERROR: File not found
as do all the others that are stuck in that location.
certutil -L -d /etc/pki/pki-tomcat/alias/
produces:
Certificate Nickname Trust Attributes SSL,S/MIME,JAR/XPI
auditSigningCert cert-pki-ca u,u,u
Even if they were expired, shouldn't the others show up in the list? And of course, that date is rolled back so they shouldn't be expired...
More details and a summary of the state of things:
restarting certmonger and/or resubmitting certs does not cause certmonger to throw any errors; pki-tomcatd has too messages when it starts that don't stop it from starting: usr/share/pki/scripts/config: line 41: break: only meaningful in a `for', `while', or `until' loop cat: /usr/share/tomcat/conf/catalina.policy: No such file or directory
Nothing shows up with respect to the renewals shows up in /var/log/ipa/renew.log even when I modified the ca with 'dogtag-ipa-ca-renew-agent-submit -vv' There are a couple of things: ipalib.plugable DEBUG ipaserver.plugins.virtual is not a valid plugin module ipalib.plugable DEBUG ipaserver.plugins.sudo is not a valid plugin module
If there are other errors elsewhere, I'm not sure where to look.
The passwords in /etc/pki/pki-tomcat/alias/pwdfile.txt (PIN1) and /var/lib/ipa/passwds/ipa01.mydomain.com-443-RSA (PIN2) are /not/ the same. A third (different) password is in /etc/dirsrv/slapd-MYREALM-COM/pwdfile.txt (PIN3). Notes about key_pin and key_pin_file are from the individual request files in /var/lib/certmonger/requests/
These certs are now fine: type=FILE,location='/var/lib/ipa/*ra-agent.pem*' (no PIN in request) type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='*auditSigningCert cert-pki-ca*',token='NSS Certificate DB' (request uses PIN1) type=NSSDB,location='/etc/dirsrv/*slapd-MYREALM-COM',nickname='Server-Cert'*,token='NSS Certificate DB' (PIN3) type=FILE,location='/var/lib/krb5kdc/*kdc.crt*'
These are broken:
Request ID '20181021083405': status: NEED_CSR_GEN_TOKEN stuck: yes key pair storage: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='ocspSigningCert cert-pki-ca',token='NSS Certificate DB',pin set certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='ocspSigningCert cert-pki-ca',token='NSS Certificate DB' key_pin:PIN1
Request ID '20181021083406': status: NEED_CSR_GEN_TOKEN stuck: yes key pair storage: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='subsystemCert cert-pki-ca',token='NSS Certificate DB',pin set certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='subsystemCert cert-pki-ca',token='NSS Certificate DB' key_pin:PIN1
Request ID '20181021083407': status: NEED_CSR_GEN_TOKEN stuck: yes key pair storage: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='caSigningCert cert-pki-ca',token='NSS Certificate DB',pin set certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='caSigningCert cert-pki-ca',token='NSS Certificate DB' key_pin:PIN1
Request ID '20181021083408': status: NEED_CSR_GEN_TOKEN stuck: yes key pair storage: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='Server-Cert cert-pki-ca',token='NSS Certificate DB',pin set certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='Server-Cert cert-pki-ca',token='NSS Certificate DB' key_pin:PIN1
Request ID '20181021083714': status: NEED_CSR_GEN_PIN stuck: yes key pair storage: type=FILE,location='/var/lib/ipa/private/httpd.key',pinfile='/var/lib/ipa/passwds/ipa01.mydomain.com-443-RSA' certificate: type=FILE,location='/var/lib/ipa/certs/httpd.crt' key_pin:PIN2
It appears that almost all of my remaining "stuck" certs no longer exist in the NSS DB... (see below) is it possible for me to issue new ones?
Thanks,
Sean
On 2020-11-04 1:37 p.m., Sean McLennan via FreeIPA-users wrote:
On 2020-11-03 2:30 p.m., Rob Crittenden via FreeIPA-users wrote:
I'd suggest stopping certmonger and looking for the actual request file in /var/lib/certmonger/request (grep for id=<request id>).
Make sure that the value in key_pin matches the value in /etc/pki/pki-tomcat/alias/pwdfile.txt
I observed something that feels relevant. I was following the instructions here: https://floblanc.wordpress.com/2016/12/19/troubleshooting-certmonger-issues-...
The log files don't show exactly what he says—I'm not sure if that's a version issue or something else. Not really finding errors (see below)
I don't seem to have 'ipaCert' anywhere?
certutil -L -d /etc/pki/pki-tomcat/alias/ -n 'subsystemCert cert-pki-ca'
fails with
certutil: Could not find cert: subsystemCert cert-pki-ca : PR_FILE_NOT_FOUND_ERROR: File not found
as do all the others that are stuck in that location.
certutil -L -d /etc/pki/pki-tomcat/alias/
produces:
Certificate Nickname Trust Attributes SSL,S/MIME,JAR/XPI
auditSigningCert cert-pki-ca u,u,u
Even if they were expired, shouldn't the others show up in the list? And of course, that date is rolled back so they shouldn't be expired...
More details and a summary of the state of things:
restarting certmonger and/or resubmitting certs does not cause certmonger to throw any errors; pki-tomcatd has too messages when it starts that don't stop it from starting: usr/share/pki/scripts/config: line 41: break: only meaningful in a `for', `while', or `until' loop cat: /usr/share/tomcat/conf/catalina.policy: No such file or directory
Nothing shows up with respect to the renewals shows up in /var/log/ipa/renew.log even when I modified the ca with 'dogtag-ipa-ca-renew-agent-submit -vv' There are a couple of things: ipalib.plugable DEBUG ipaserver.plugins.virtual is not a valid plugin module ipalib.plugable DEBUG ipaserver.plugins.sudo is not a valid plugin module
If there are other errors elsewhere, I'm not sure where to look.
The passwords in /etc/pki/pki-tomcat/alias/pwdfile.txt (PIN1) and /var/lib/ipa/passwds/ipa01.mydomain.com-443-RSA (PIN2) are /not/ the same. A third (different) password is in /etc/dirsrv/slapd-MYREALM-COM/pwdfile.txt (PIN3). Notes about key_pin and key_pin_file are from the individual request files in /var/lib/certmonger/requests/
These certs are now fine: type=FILE,location='/var/lib/ipa/*ra-agent.pem*' (no PIN in request) type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='*auditSigningCert cert-pki-ca*',token='NSS Certificate DB' (request uses PIN1) type=NSSDB,location='/etc/dirsrv/*slapd-MYREALM-COM',nickname='Server-Cert'*,token='NSS Certificate DB' (PIN3) type=FILE,location='/var/lib/krb5kdc/*kdc.crt*'
These are broken:
Request ID '20181021083405': status: NEED_CSR_GEN_TOKEN stuck: yes key pair storage: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='ocspSigningCert cert-pki-ca',token='NSS Certificate DB',pin set certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='ocspSigningCert cert-pki-ca',token='NSS Certificate DB' key_pin:PIN1
Request ID '20181021083406': status: NEED_CSR_GEN_TOKEN stuck: yes key pair storage: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='subsystemCert cert-pki-ca',token='NSS Certificate DB',pin set certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='subsystemCert cert-pki-ca',token='NSS Certificate DB' key_pin:PIN1
Request ID '20181021083407': status: NEED_CSR_GEN_TOKEN stuck: yes key pair storage: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='caSigningCert cert-pki-ca',token='NSS Certificate DB',pin set certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='caSigningCert cert-pki-ca',token='NSS Certificate DB' key_pin:PIN1
Request ID '20181021083408': status: NEED_CSR_GEN_TOKEN stuck: yes key pair storage: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='Server-Cert cert-pki-ca',token='NSS Certificate DB',pin set certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='Server-Cert cert-pki-ca',token='NSS Certificate DB' key_pin:PIN1
Request ID '20181021083714': status: NEED_CSR_GEN_PIN stuck: yes key pair storage: type=FILE,location='/var/lib/ipa/private/httpd.key',pinfile='/var/lib/ipa/passwds/ipa01.mydomain.com-443-RSA' certificate: type=FILE,location='/var/lib/ipa/certs/httpd.crt' key_pin:PIN2
FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org To unsubscribe send an email to freeipa-users-leave@lists.fedorahosted.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedorahosted.org/archives/list/freeipa-users@lists.fedorahoste...
Sean McLennan via FreeIPA-users wrote:
It appears that almost all of my remaining "stuck" certs no longer exist in the NSS DB... (see below) is it possible for me to issue new ones?
The confusion centers around the fact that you reported running IPA 4.6.9 which is very different from the version you are actually running, 4.6.90.
There is no ipaCert any more so ignore that bit.
The missing certs are the real problem. You can look in /root/cacerts.p12 to see if the private keys exist there. The password is the Directory Manager password.
# pk12util -l /root/cacert.p12 |grep Friend
The names will appear twice, one for the private key and one for the public cert.
rob
On 2020-11-04 1:37 p.m., Sean McLennan via FreeIPA-users wrote:
On 2020-11-03 2:30 p.m., Rob Crittenden via FreeIPA-users wrote:
I'd suggest stopping certmonger and looking for the actual request file in /var/lib/certmonger/request (grep for id=<request id>).
Make sure that the value in key_pin matches the value in /etc/pki/pki-tomcat/alias/pwdfile.txt
I observed something that feels relevant. I was following the instructions here: https://floblanc.wordpress.com/2016/12/19/troubleshooting-certmonger-issues-...
The log files don't show exactly what he says—I'm not sure if that's a version issue or something else. Not really finding errors (see below)
I don't seem to have 'ipaCert' anywhere?
certutil -L -d /etc/pki/pki-tomcat/alias/ -n 'subsystemCert cert-pki-ca'
fails with
certutil: Could not find cert: subsystemCert cert-pki-ca : PR_FILE_NOT_FOUND_ERROR: File not found
as do all the others that are stuck in that location.
certutil -L -d /etc/pki/pki-tomcat/alias/
produces:
Certificate Nickname Trust Attributes SSL,S/MIME,JAR/XPI
auditSigningCert cert-pki-ca u,u,u
Even if they were expired, shouldn't the others show up in the list? And of course, that date is rolled back so they shouldn't be expired...
More details and a summary of the state of things:
restarting certmonger and/or resubmitting certs does not cause certmonger to throw any errors; pki-tomcatd has too messages when it starts that don't stop it from starting: usr/share/pki/scripts/config: line 41: break: only meaningful in a `for', `while', or `until' loop cat: /usr/share/tomcat/conf/catalina.policy: No such file or directory
Nothing shows up with respect to the renewals shows up in /var/log/ipa/renew.log even when I modified the ca with 'dogtag-ipa-ca-renew-agent-submit -vv' There are a couple of things: ipalib.plugable DEBUG ipaserver.plugins.virtual is not a valid plugin module ipalib.plugable DEBUG ipaserver.plugins.sudo is not a valid plugin module
If there are other errors elsewhere, I'm not sure where to look.
The passwords in /etc/pki/pki-tomcat/alias/pwdfile.txt (PIN1) and /var/lib/ipa/passwds/ipa01.mydomain.com-443-RSA (PIN2) are /not/ the same. A third (different) password is in /etc/dirsrv/slapd-MYREALM-COM/pwdfile.txt (PIN3). Notes about key_pin and key_pin_file are from the individual request files in /var/lib/certmonger/requests/
These certs are now fine: type=FILE,location='/var/lib/ipa/*ra-agent.pem*' (no PIN in request) type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='*auditSigningCert cert-pki-ca*',token='NSS Certificate DB' (request uses PIN1) type=NSSDB,location='/etc/dirsrv/*slapd-MYREALM-COM',nickname='Server-Cert'*,token='NSS Certificate DB' (PIN3) type=FILE,location='/var/lib/krb5kdc/*kdc.crt*'
These are broken:
Request ID '20181021083405': status: NEED_CSR_GEN_TOKEN stuck: yes key pair storage: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='ocspSigningCert cert-pki-ca',token='NSS Certificate DB',pin set certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='ocspSigningCert cert-pki-ca',token='NSS Certificate DB' key_pin:PIN1
Request ID '20181021083406': status: NEED_CSR_GEN_TOKEN stuck: yes key pair storage: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='subsystemCert cert-pki-ca',token='NSS Certificate DB',pin set certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='subsystemCert cert-pki-ca',token='NSS Certificate DB' key_pin:PIN1
Request ID '20181021083407': status: NEED_CSR_GEN_TOKEN stuck: yes key pair storage: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='caSigningCert cert-pki-ca',token='NSS Certificate DB',pin set certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='caSigningCert cert-pki-ca',token='NSS Certificate DB' key_pin:PIN1
Request ID '20181021083408': status: NEED_CSR_GEN_TOKEN stuck: yes key pair storage: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='Server-Cert cert-pki-ca',token='NSS Certificate DB',pin set certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='Server-Cert cert-pki-ca',token='NSS Certificate DB' key_pin:PIN1
Request ID '20181021083714': status: NEED_CSR_GEN_PIN stuck: yes key pair storage: type=FILE,location='/var/lib/ipa/private/httpd.key',pinfile='/var/lib/ipa/passwds/ipa01.mydomain.com-443-RSA' certificate: type=FILE,location='/var/lib/ipa/certs/httpd.crt' key_pin:PIN2
FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org To unsubscribe send an email to freeipa-users-leave@lists.fedorahosted.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedorahosted.org/archives/list/freeipa-users@lists.fedorahoste...
FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org To unsubscribe send an email to freeipa-users-leave@lists.fedorahosted.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedorahosted.org/archives/list/freeipa-users@lists.fedorahoste...
Thanks for replying
The missing certs are the real problem. You can look in /root/cacerts.p12 to see if the private keys exist there. The password is the Directory Manager password.
# pk12util -l /root/cacert.p12 |grep Friend
The names will appear twice, one for the private key and one for the public cert.
This is what I get: pk12util: PKCS12 decode not verified: SEC_ERROR_PKCS12_INVALID_MAC: Unable to import. Invalid MAC. Incorrect password or corrupt file. Friendly Name: caSigningCert cert-pki-ca Friendly Name: ocspSigningCert cert-pki-ca Friendly Name: subsystemCert cert-pki-ca Friendly Name: auditSigningCert cert-pki-ca Friendly Name: caSigningCert cert-pki-ca Friendly Name: ocspSigningCert cert-pki-ca Friendly Name: subsystemCert cert-pki-ca Friendly Name: auditSigningCert cert-pki-ca Friendly Name: Server-Cert cert-pki-ca
Sean
Sean McLennan via FreeIPA-users wrote:
Thanks for replying
The missing certs are the real problem. You can look in /root/cacerts.p12 to see if the private keys exist there. The password is the Directory Manager password.
# pk12util -l /root/cacert.p12 |grep Friend
The names will appear twice, one for the private key and one for the public cert.
This is what I get: pk12util: PKCS12 decode not verified: SEC_ERROR_PKCS12_INVALID_MAC: Unable to import. Invalid MAC. Incorrect password or corrupt file. Friendly Name: caSigningCert cert-pki-ca Friendly Name: ocspSigningCert cert-pki-ca Friendly Name: subsystemCert cert-pki-ca Friendly Name: auditSigningCert cert-pki-ca Friendly Name: caSigningCert cert-pki-ca Friendly Name: ocspSigningCert cert-pki-ca Friendly Name: subsystemCert cert-pki-ca Friendly Name: auditSigningCert cert-pki-ca Friendly Name: Server-Cert cert-pki-ca
Ok you probably have all you need but the error message means the password is wrong. Without the password you're still stuck.
rob
/root/cacerts.p12 to see if the private keys exist there. The password is the Directory Manager password.
# pk12util -l /root/cacert.p12 |grep Friend
The names will appear twice, one for the private key and one for the public cert.
This is what I get: pk12util: PKCS12 decode not verified: SEC_ERROR_PKCS12_INVALID_MAC: Unable to import. Invalid MAC. Incorrect password or corrupt file. Friendly Name: caSigningCert cert-pki-ca Friendly Name: ocspSigningCert cert-pki-ca Friendly Name: subsystemCert cert-pki-ca Friendly Name: auditSigningCert cert-pki-ca Friendly Name: caSigningCert cert-pki-ca Friendly Name: ocspSigningCert cert-pki-ca Friendly Name: subsystemCert cert-pki-ca Friendly Name: auditSigningCert cert-pki-ca Friendly Name: Server-Cert cert-pki-ca
Ok you probably have all you need but the error message means the password is wrong. Without the password you're still stuck.
So if it's supposed to be the Directory Manager password, I'm sure I have that one right because I can use it for basic 'ldapsearch'es.
/root/cacert.p12 was last modified in Dec 2018 (~2 months after install?) and I have never changed the Directory Manager password since I installed freeipa.
With no real expectation they would work, I tried a couple other passwords I have related to ipa as well as the passwords in /etc/pki/pki-tomcat/alias/pwdfile.txt and /var/lib/ipa/passwds/ipa01.mydomain.com-443-RSA. They didn't.
Anywhere else I can look?
Sean
Sean McLennan via FreeIPA-users wrote:
/root/cacerts.p12 to see if the private keys exist there. The password is the Directory Manager password.
# pk12util -l /root/cacert.p12 |grep Friend
The names will appear twice, one for the private key and one for the public cert.
This is what I get: pk12util: PKCS12 decode not verified: SEC_ERROR_PKCS12_INVALID_MAC: Unable to import. Invalid MAC. Incorrect password or corrupt file. Friendly Name: caSigningCert cert-pki-ca Friendly Name: ocspSigningCert cert-pki-ca Friendly Name: subsystemCert cert-pki-ca Friendly Name: auditSigningCert cert-pki-ca Friendly Name: caSigningCert cert-pki-ca Friendly Name: ocspSigningCert cert-pki-ca Friendly Name: subsystemCert cert-pki-ca Friendly Name: auditSigningCert cert-pki-ca Friendly Name: Server-Cert cert-pki-ca
Ok you probably have all you need but the error message means the password is wrong. Without the password you're still stuck.
So if it's supposed to be the Directory Manager password, I'm sure I have that one right because I can use it for basic 'ldapsearch'es.
/root/cacert.p12 was last modified in Dec 2018 (~2 months after install?) and I have never changed the Directory Manager password since I installed freeipa.
With no real expectation they would work, I tried a couple other passwords I have related to ipa as well as the passwords in /etc/pki/pki-tomcat/alias/pwdfile.txt and /var/lib/ipa/passwds/ipa01.mydomain.com-443-RSA. They didn't.
Anywhere else I can look?
It isn't stored anywhere.
I can't explain why cacert.p12 would be updated other than generating a domain level 0 replica installation file which wouldn't make sense with this version of IPA.
I'd see if you have any backups or other copies of these missing certificates somewhere.
rob
It isn't stored anywhere.
I can't explain why cacert.p12 would be updated other than generating a domain level 0 replica installation file which wouldn't make sense with this version of IPA.
I'd see if you have any backups or other copies of these missing certificates somewhere.
Actually, I've been rooting around and I found that there is also /root/cacert.p12.bkp that dates from the original install. The two files are identical and similarly cacert.p12.bkp produces the same error.
I did actually attempt to bring up a replica in May 2019 but it failed. I still had it sitting around as a virtual machine so I spun it up and it was very empty.
After looking at the instructions for changing the Directory Manager password here: https://www.freeipa.org/page/Howto/Change_Directory_Manager_Password I noticed /root/dm_password which *does* match my Directory Manager password.
There error message also says the file could be corrupt, but I'm assuming if that was the case, I wouldn't get any output from pk12util at all...
So the only other thing I can think of (and am about to explore) is that like with pyasn1, it's updates are causing the problem... does that bring up anything for you?
The only other thing that I noticed is that /etc/pki/pki-tomcat/alias/pwdfile.txt (which does match /root/keydb_pin like I think it's supposed to) is the only password/pin file I've seen that has an EOL... It seems like a stretch that that could have somehow impacted cacert.p12, but I still don't really understand how or when any of these are being used.
Cheers,
Sean
After looking at the instructions for changing the Directory Manager password here: https://www.freeipa.org/page/Howto/Change_Directory_Manager_Password I noticed /root/dm_password which *does* match my Directory Manager password.
Oh gawd I just realized this means that I followed those instructions and (attempted to) change the password, didn't I. :(
I have no memory of this. :(
Sean
pk12util: PKCS12 decode not verified: SEC_ERROR_PKCS12_INVALID_MAC: Unable to import. Invalid MAC. Incorrect password or corrupt file. Friendly Name: caSigningCert cert-pki-ca Friendly Name: ocspSigningCert cert-pki-ca Friendly Name: subsystemCert cert-pki-ca Friendly Name: auditSigningCert cert-pki-ca Friendly Name: caSigningCert cert-pki-ca Friendly Name: ocspSigningCert cert-pki-ca Friendly Name: subsystemCert cert-pki-ca Friendly Name: auditSigningCert cert-pki-ca Friendly Name: Server-Cert cert-pki-ca
Ok you probably have all you need but the error message means the password is wrong. Without the password you're still stuck.
So if it's supposed to be the Directory Manager password, I'm sure I have that one right because I can use it for basic 'ldapsearch'es.
OK, so after some some serious forensic work, I think I know what happened. Unfortunately, all of this was going on at a profoundly stressful time and I have no recollection of any of this. I had a snapshot of the server just after setup; in that I confirmed that the password was not what I thought it was immediately after it was set up. I also confirmed through a backup of my password manager that it was the password that I intended to use. So, very likely I managed to repeat a typo twice during set up.
In December I was setting up a trust with a previous Active Directory instance and discovered then that the DM password did not work. I changed it in LDAP by directly editing 'dse.ldif'. I made an attempt to fix cacert.p12, but it failed and I just replaced the file with the original (hence the dm_password, the key_pin and cacert.p12.bkp in root). Setting up the trust worked once the password was changed, and so I moved on... haven't had any problems in between so obviously just forgot about the whole thing (like I said, stressful time; cognition-impacting drugs...). I determined all this cause I have the full command history for the server.
This, I'm sure, is also why I was never able to get a replica running. Sigh.
So—thanks for listening to my sob-story.... Given it's most likely a typo of what I think it is, I may try and brute-force it, but assuming that's not going to work, am I rebuilding from scratch?
Thanks,
Sean
Sean McLennan via FreeIPA-users wrote:
pk12util: PKCS12 decode not verified: SEC_ERROR_PKCS12_INVALID_MAC: Unable to import. Invalid MAC. Incorrect password or corrupt file. Friendly Name: caSigningCert cert-pki-ca Friendly Name: ocspSigningCert cert-pki-ca Friendly Name: subsystemCert cert-pki-ca Friendly Name: auditSigningCert cert-pki-ca Friendly Name: caSigningCert cert-pki-ca Friendly Name: ocspSigningCert cert-pki-ca Friendly Name: subsystemCert cert-pki-ca Friendly Name: auditSigningCert cert-pki-ca Friendly Name: Server-Cert cert-pki-ca
Ok you probably have all you need but the error message means the password is wrong. Without the password you're still stuck.
So if it's supposed to be the Directory Manager password, I'm sure I have that one right because I can use it for basic 'ldapsearch'es.
OK, so after some some serious forensic work, I think I know what happened. Unfortunately, all of this was going on at a profoundly stressful time and I have no recollection of any of this. I had a snapshot of the server just after setup; in that I confirmed that the password was not what I thought it was immediately after it was set up. I also confirmed through a backup of my password manager that it was the password that I intended to use. So, very likely I managed to repeat a typo twice during set up.
In December I was setting up a trust with a previous Active Directory instance and discovered then that the DM password did not work. I changed it in LDAP by directly editing 'dse.ldif'. I made an attempt to fix cacert.p12, but it failed and I just replaced the file with the original (hence the dm_password, the key_pin and cacert.p12.bkp in root). Setting up the trust worked once the password was changed, and so I moved on... haven't had any problems in between so obviously just forgot about the whole thing (like I said, stressful time; cognition-impacting drugs...). I determined all this cause I have the full command history for the server.
This, I'm sure, is also why I was never able to get a replica running. Sigh.
So—thanks for listening to my sob-story.... Given it's most likely a typo of what I think it is, I may try and brute-force it, but assuming that's not going to work, am I rebuilding from scratch?
Sadly yes.
rob
freeipa-users@lists.fedorahosted.org
-
John Petrini -
Rob Crittenden -
Sean McLennan -
Timo Aaltonen