Hi All,
I have a werid issue with FreeIPA. I can't do anything with certificates. I also can't upgrade FreeIPA. If I run ipa-server-update I receive this: Unexpected error - see /var/log/ipaupgrade.log for details: NetworkError: cannot connect to ' https://freeipa.corp.mydomain.com:8443/ca/rest/account/login': [X509: KEY_VALUES_MISMATCH] key values mismatch (_ssl.c:2593) The ipa-server-upgrade command failed. See /var/log/ipaupgrade.log for more information
If I try to vew certificates via web interface I see this error message: IPA Error 907: NetworkError cannot connect to ' https://freeipa.corp.mydomain.com:443/ca/agent/ca/displayBySerial': [X509: KEY_VALUES_MISMATCH] key values mismatch (_ssl.c:2593)
Below is the list of my certificates: # getcert list | egrep '^Request|status:|subject:' Request ID '20171205153653': status: MONITORING subject: CN=freeipa.corp.mydomain.com,O=CORP.MYDOMAIN.COM Request ID '20180912151607': status: CA_UNREACHABLE subject: CN=CA Audit,O=CORP.MYDOMAIN.COM Request ID '20180912151608': status: CA_UNREACHABLE subject: CN=OCSP Subsystem,O=CORP.MYDOMAIN.COM Request ID '20180912151609': status: CA_UNREACHABLE subject: CN=CA Subsystem,O=CORP.MYDOMAIN.COM Request ID '20180912151610': status: MONITORING subject: CN=Certificate Authority,O=CORP.MYDOMAIN.COM Request ID '20180912151611': status: MONITORING subject: CN=user,O=CORP.MYDOMAIN.COM Request ID '20180912151612': status: CA_UNREACHABLE subject: CN=freeipa.corp.mydomain.com,O=CORP.MYDOMAIN.COM Request ID '20180912151613': status: MONITORING subject: CN=freeipa.corp.mydomain.com,O=CORP.MYDOMAIN.COM Request ID '20180912151615': status: MONITORING subject: CN=freeipa.corp.mydomain.com,O=CORP.MYDOMAIN.COM Request ID '20190212162113': status: MONITORING subject: CN=mail.corp.mydomain.com,O=CORP.MYDOMAIN.COM Request ID '20191017155747': status: MONITORING subject: CN=analytics-stage.corp.mydomain.com,O=CORP.MYDOMAIN.COM Request ID '20191026094947': status: MONITORING subject: CN=nas.corp.mydomain.com,O=CORP.MYDOMAIN.COM Request ID '20191026102844': status: MONITORING subject: CN=pe.corp.mydomain.com,O=CORP.MYDOMAIN.COM Request ID '20191027134809': status: CA_UNREACHABLE subject: Request ID '20191027135053': status: CA_REJECTED subject: Request ID '20191027135738': status: CA_UNREACHABLE subject:
I tried to set time on the server back and I tried to restart certmonger - but result is always the same - SSL error KEY_VALUES_MISMATCH. FreeIPA, version: 4.6.4 How to solve this issue?
Dmitri Moudraninets via FreeIPA-users wrote:
Hi All,
I have a werid issue with FreeIPA. I can't do anything with certificates. I also can't upgrade FreeIPA. If I run ipa-server-update I receive this: Unexpected error - see /var/log/ipaupgrade.log for details: NetworkError: cannot connect to 'https://freeipa.corp.mydomain.com:8443/ca/rest/account/login': [X509: KEY_VALUES_MISMATCH] key values mismatch (_ssl.c:2593) The ipa-server-upgrade command failed. See /var/log/ipaupgrade.log for more information
It suggests that private and public keys don't match for the RA agent cert.
Verify that the output of the following matches: # openssl x509 -noout -modulus -in /var/lib/ipa/ra-agent.pem # openssl rsa -noout -modulus -in /var/lib/ipa/ra-agent.key
If I try to vew certificates via web interface I see this error message: IPA Error 907: NetworkError cannot connect to 'https://freeipa.corp.mydomain.com:443/ca/agent/ca/displayBySerial': [X509: KEY_VALUES_MISMATCH] key values mismatch (_ssl.c:2593)
same
Below is the list of my certificates:
[snip]
We need more/all the context.
I tried to set time on the server back and I tried to restart certmonger
- but result is always the same - SSL error KEY_VALUES_MISMATCH.
FreeIPA, version: 4.6.4 How to solve this issue?
This doesn't seem to be an expiration issue, though I can't confirm based on the context provided.
Is it failing only one this one master or all?
rob
Hi Rob,
Both master and replica are failing. The output of the following commands is different on both FreeIPA servers. # openssl x509 -noout -modulus -in /var/lib/ipa/ra-agent.pem # openssl rsa -noout -modulus -in /var/lib/ipa/ra-agent.key
Is this a known issue?
ср, 20 нояб. 2019 г. в 22:24, Rob Crittenden rcritten@redhat.com:
Dmitri Moudraninets via FreeIPA-users wrote:
Hi All,
I have a werid issue with FreeIPA. I can't do anything with certificates. I also can't upgrade FreeIPA. If I run ipa-server-update I receive this: Unexpected error - see /var/log/ipaupgrade.log for details: NetworkError: cannot connect to 'https://freeipa.corp.mydomain.com:8443/ca/rest/account/login': [X509: KEY_VALUES_MISMATCH] key values mismatch (_ssl.c:2593) The ipa-server-upgrade command failed. See /var/log/ipaupgrade.log for more information
It suggests that private and public keys don't match for the RA agent cert.
Verify that the output of the following matches: # openssl x509 -noout -modulus -in /var/lib/ipa/ra-agent.pem # openssl rsa -noout -modulus -in /var/lib/ipa/ra-agent.key
If I try to vew certificates via web interface I see this error message: IPA Error 907: NetworkError cannot connect to 'https://freeipa.corp.mydomain.com:443/ca/agent/ca/displayBySerial': [X509: KEY_VALUES_MISMATCH] key values mismatch (_ssl.c:2593)
same
Below is the list of my certificates:
[snip]
We need more/all the context.
I tried to set time on the server back and I tried to restart certmonger
- but result is always the same - SSL error KEY_VALUES_MISMATCH.
FreeIPA, version: 4.6.4 How to solve this issue?
This doesn't seem to be an expiration issue, though I can't confirm based on the context provided.
Is it failing only one this one master or all?
rob
Dmitri Moudraninets wrote:
Hi Rob,
Both master and replica are failing. The output of the following commands is different on both FreeIPA servers. # openssl x509 -noout -modulus -in /var/lib/ipa/ra-agent.pem # openssl rsa -noout -modulus -in /var/lib/ipa/ra-agent.key
Is this a known issue?
No.
Do the cert and key match between the two masters? e.g. do they fail in exactly the same way?
What is the history of this? Did this happen in conjunction with troubleshooting another problem?
Can you provide the output of:
# getcert list -f /var/lib/ipa/ra-agent.pem # openssl x509 -text -in /var/lib/ipa/ra-agent.pem
rob
ср, 20 нояб. 2019 г. в 22:24, Rob Crittenden <rcritten@redhat.com mailto:rcritten@redhat.com>:
Dmitri Moudraninets via FreeIPA-users wrote: > Hi All, > > > I have a werid issue with FreeIPA. I can't do anything with > certificates. I also can't upgrade FreeIPA. If I run ipa-server-update I > receive this: > Unexpected error - see /var/log/ipaupgrade.log for details: > NetworkError: cannot connect to > 'https://freeipa.corp.mydomain.com:8443/ca/rest/account/login': [X509: > KEY_VALUES_MISMATCH] key values mismatch (_ssl.c:2593) > The ipa-server-upgrade command failed. See /var/log/ipaupgrade.log for > more information It suggests that private and public keys don't match for the RA agent cert. Verify that the output of the following matches: # openssl x509 -noout -modulus -in /var/lib/ipa/ra-agent.pem # openssl rsa -noout -modulus -in /var/lib/ipa/ra-agent.key > > If I try to vew certificates via web interface I see this error message: > IPA Error 907: NetworkError > cannot connect to > 'https://freeipa.corp.mydomain.com:443/ca/agent/ca/displayBySerial': > [X509: KEY_VALUES_MISMATCH] key values mismatch (_ssl.c:2593) same > > Below is the list of my certificates: [snip] We need more/all the context. > I tried to set time on the server back and I tried to restart certmonger > - but result is always the same - SSL error KEY_VALUES_MISMATCH. > FreeIPA, version: 4.6.4 > How to solve this issue? This doesn't seem to be an expiration issue, though I can't confirm based on the context provided. Is it failing only one this one master or all? rob
-- WBR Dmitry
Hi Rob,
Yes both masters are failing the same way. Output of openssl x509 -noout -modulus -in /var/lib/ipa/ra-agent.pem is the same on both masters. Output of openssl rsa -noout -modulus -in /var/lib/ipa/ra-agent.key is also the same on both masters. But the output of the first command is not the same as the output of the second command.
I can't remember that I troubleshoot any other problems but we tried to generate some personal certificates for some users. Also we tried to generate certificates with key files for some of our internal services. We did that for the first time and it worked at the end. Also I changed the admin password not so long ago.
Below you can find the output of the requested commands:
[root@second_master ~]# getcert list -f /var/lib/ipa/ra-agent.pem Number of certificates and requests being tracked: 9. Request ID '20180912151730': status: MONITORING stuck: no key pair storage: type=FILE,location='/var/lib/ipa/ra-agent.key' certificate: type=FILE,location='/var/lib/ipa/ra-agent.pem' CA: dogtag-ipa-ca-renew-agent issuer: CN=Certificate Authority,O=CORP.MYDOMAIN.DE subject: CN=dmud,O=CORP.MYDOMAIN.DE *<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< I see a username here. Does it have to be like that?* expires: 2021-10-29 09:39:47 UTC email: dmud@corp.mydomain.de key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment eku: id-kp-serverAuth,id-kp-clientAuth pre-save command: /usr/libexec/ipa/certmonger/renew_ra_cert_pre post-save command: /usr/libexec/ipa/certmonger/renew_ra_cert track: yes auto-renew: yes [root@second_master ~]# openssl x509 -text -in /var/lib/ipa/ra-agent.pem Certificate: Data: Version: 3 (0x2) Serial Number: 28 (0x1c) Signature Algorithm: sha256WithRSAEncryption Issuer: O=CORP.MYDOMAIN.DE, CN=Certificate Authority Validity Not Before: Oct 29 10:39:47 2019 GMT Not After : Oct 29 09:39:47 2021 GMT Subject: O=CORP.MYDOMAIN.DE, CN=dmud Subject Public Key Info: Public Key Algorithm: rsaEncryption Public-Key: (2048 bit) Modulus: 00:ba:09:81:99:9b:17:99:07:5a:10:28:c8:7a:03: ... 66:5f Exponent: 65537 (0x10001) X509v3 extensions: X509v3 Authority Key Identifier:
keyid:D2:91:B5:38:D3:4A:AE:3D:39:4D:8E:9E:FF:6F:15:08:BB:72:70:BF
X509v3 Subject Key Identifier: DE:5F:8B:60:34:0B:C8:88:96:FF:FC:F4:1C:0E:AC:09:BD:8D:51:0A X509v3 Subject Alternative Name: email:dmud@corp.mydomain.de Authority Information Access: OCSP - URI:http://ipa-ca.corp.mydomain.de/ca/ocsp
X509v3 Key Usage: critical Digital Signature, Non Repudiation, Key Encipherment, Data Encipherment X509v3 Extended Key Usage: TLS Web Server Authentication, TLS Web Client Authentication X509v3 CRL Distribution Points:
Full Name: URI:http://ipa-ca.corp.mydomain.de/ipa/crl/MasterCRL.bin CRL Issuer: DirName: O = ipaca, CN = Certificate Authority
Signature Algorithm: sha256WithRSAEncryption 06:d2:32:01:29:d2:67:d4:fe:0a:0d:d2:f6:5b:22:a9:18:92: ... a8:d1:54:a2 -----BEGIN CERTIFICATE----- MIIERzCCAy+gAwIBAgIBHDANBgkqhkiG9w0BAQsFADA+MRwwGgYDVQQKDBNDT1JQ ... U3qp7LokWOwmHnfDayEQ+11mkJb/rugYaG8p5Gkrfiqo6my+B5mIqNFUog== -----END CERTIFICATE-----
[root@first_master ~]# getcert list -f /var/lib/ipa/ra-agent.pem Number of certificates and requests being tracked: 16. Request ID '20180912151611': status: MONITORING stuck: no key pair storage: type=FILE,location='/var/lib/ipa/ra-agent.key' certificate: type=FILE,location='/var/lib/ipa/ra-agent.pem' CA: dogtag-ipa-ca-renew-agent issuer: CN=Certificate Authority,O=CORP.MYDOMAIN.DE subject: CN=dmud,O=CORP.MYDOMAIN.DE expires: 2021-10-29 09:39:47 UTC email: dmud@corp.mydomain.de key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment eku: id-kp-serverAuth,id-kp-clientAuth pre-save command: /usr/libexec/ipa/certmonger/renew_ra_cert_pre post-save command: /usr/libexec/ipa/certmonger/renew_ra_cert track: yes auto-renew: yes [root@first_master ~]# openssl x509 -text -in /var/lib/ipa/ra-agent.pem Certificate: Data: Version: 3 (0x2) Serial Number: 28 (0x1c) Signature Algorithm: sha256WithRSAEncryption Issuer: O=CORP.MYDOMAIN.DE, CN=Certificate Authority Validity Not Before: Oct 29 10:39:47 2019 GMT Not After : Oct 29 09:39:47 2021 GMT Subject: O=CORP.MYDOMAIN.DE, CN=dmud Subject Public Key Info: Public Key Algorithm: rsaEncryption Public-Key: (2048 bit) Modulus: 00:ba:09:81:99:9b:17:99:07:5a:10:28:c8:7a:03: ... 66:5f Exponent: 65537 (0x10001) X509v3 extensions: X509v3 Authority Key Identifier:
keyid:D2:91:B5:38:D3:4A:AE:3D:39:4D:8E:9E:FF:6F:15:08:BB:72:70:BF
X509v3 Subject Key Identifier: DE:5F:8B:60:34:0B:C8:88:96:FF:FC:F4:1C:0E:AC:09:BD:8D:51:0A X509v3 Subject Alternative Name: email:dmud@corp.mydomain.de Authority Information Access: OCSP - URI:http://ipa-ca.corp.mydomain.de/ca/ocsp
X509v3 Key Usage: critical Digital Signature, Non Repudiation, Key Encipherment, Data Encipherment X509v3 Extended Key Usage: TLS Web Server Authentication, TLS Web Client Authentication X509v3 CRL Distribution Points:
Full Name: URI:http://ipa-ca.corp.mydomain.de/ipa/crl/MasterCRL.bin CRL Issuer: DirName: O = ipaca, CN = Certificate Authority
Signature Algorithm: sha256WithRSAEncryption 06:d2:32:01:29:d2:67:d4:fe:0a:0d:d2:f6:5b:22:a9:18:92: ... a8:d1:54:a2 -----BEGIN CERTIFICATE----- MIIERzCCAy+gAwIBAgIBHDANBgkqhkiG9w0BAQsFADA+MRwwGgYDVQQKDBNDT1JQ ... U3qp7LokWOwmHnfDayEQ+11mkJb/rugYaG8p5Gkrfiqo6my+B5mIqNFUog== -----END CERTIFICATE-----
чт, 21 нояб. 2019 г. в 15:24, Rob Crittenden rcritten@redhat.com:
Dmitri Moudraninets wrote:
Hi Rob,
Both master and replica are failing. The output of the following commands is different on both FreeIPA servers. # openssl x509 -noout -modulus -in /var/lib/ipa/ra-agent.pem # openssl rsa -noout -modulus -in /var/lib/ipa/ra-agent.key
Is this a known issue?
No.
Do the cert and key match between the two masters? e.g. do they fail in exactly the same way?
What is the history of this? Did this happen in conjunction with troubleshooting another problem?
Can you provide the output of:
# getcert list -f /var/lib/ipa/ra-agent.pem # openssl x509 -text -in /var/lib/ipa/ra-agent.pem
rob
ср, 20 нояб. 2019 г. в 22:24, Rob Crittenden <rcritten@redhat.com mailto:rcritten@redhat.com>:
Dmitri Moudraninets via FreeIPA-users wrote: > Hi All, > > > I have a werid issue with FreeIPA. I can't do anything with > certificates. I also can't upgrade FreeIPA. If I run ipa-server-update I > receive this: > Unexpected error - see /var/log/ipaupgrade.log for details: > NetworkError: cannot connect to > 'https://freeipa.corp.mydomain.com:8443/ca/rest/account/login':
[X509:
> KEY_VALUES_MISMATCH] key values mismatch (_ssl.c:2593) > The ipa-server-upgrade command failed. See /var/log/ipaupgrade.log
for
> more information It suggests that private and public keys don't match for the RA agent cert. Verify that the output of the following matches: # openssl x509 -noout -modulus -in /var/lib/ipa/ra-agent.pem # openssl rsa -noout -modulus -in /var/lib/ipa/ra-agent.key > > If I try to vew certificates via web interface I see this error message: > IPA Error 907: NetworkError > cannot connect to > 'https://freeipa.corp.mydomain.com:443/ca/agent/ca/displayBySerial
':
> [X509: KEY_VALUES_MISMATCH] key values mismatch (_ssl.c:2593) same > > Below is the list of my certificates: [snip] We need more/all the context. > I tried to set time on the server back and I tried to restart certmonger > - but result is always the same - SSL error KEY_VALUES_MISMATCH. > FreeIPA, version: 4.6.4 > How to solve this issue? This doesn't seem to be an expiration issue, though I can't confirm based on the context provided. Is it failing only one this one master or all? rob
-- WBR Dmitry
Dmitri Moudraninets wrote:
Hi Rob,
Yes both masters are failing the same way. Output of openssl x509 -noout -modulus -in /var/lib/ipa/ra-agent.pem is the same on both masters. Output of openssl rsa -noout -modulus -in /var/lib/ipa/ra-agent.key is also the same on both masters. But the output of the first command is not the same as the output of the second command.
I can't remember that I troubleshoot any other problems but we tried to generate some personal certificates for some users. Also we tried to generate certificates with key files for some of our internal services. We did that for the first time and it worked at the end. Also I changed the admin password not so long ago.
Below you can find the output of the requested commands:
[root@second_master ~]# getcert list -f /var/lib/ipa/ra-agent.pem Number of certificates and requests being tracked: 9. Request ID '20180912151730': status: MONITORING stuck: no key pair storage: type=FILE,location='/var/lib/ipa/ra-agent.key' certificate: type=FILE,location='/var/lib/ipa/ra-agent.pem' CA: dogtag-ipa-ca-renew-agent issuer: CN=Certificate Authority,O=CORP.MYDOMAIN.DE http://CORP.MYDOMAIN.DE subject: CN=dmud,O=CORP.MYDOMAIN.DE http://CORP.MYDOMAIN.DE *<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< I see a username here. Does it have to be like that?* expires: 2021-10-29 09:39:47 UTC email: dmud@corp.mydomain.de mailto:dmud@corp.mydomain.de key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment eku: id-kp-serverAuth,id-kp-clientAuth pre-save command: /usr/libexec/ipa/certmonger/renew_ra_cert_pre post-save command: /usr/libexec/ipa/certmonger/renew_ra_cert track: yes auto-renew: yes
Right, someone overwrote the RA agent certificate.
Look to see if the user entry in the CA has the right cert:
$ ldapsearch -LLL -o ldif-wrap=no -x -D 'cn=directory manager' -W -b uid=ipara,ou=People,o=ipaca usercertificate
Put the base64 value of the usercertificate attribute into a file and add a prefix/suffix around it:
-----BEGIN CERTIFICATE----- MII....blah= -----END CERTIFICATE-----
$ openssl x509 -text -in /path/to/file
If the Subject is O = CORP.MYDOMAIN.DE, CN = IPA RA then that's a good start. Also look at the expires date to be sure it is still valid.
Assuming that is ok then re-run the openssl modulus commands to ensure they are the same.
Assuming that too is ok then you have the proper, valid RA agent cert. In that case I'd move the current file out of the way, who knows what it is, then run:
# openssl x509 -in /path/to/file -out /var/lib/ipa/ra-agent.pem (just to properly format the agent cert) # chown root:ipaapi /var/lib/ipa/ra-agent.pem # chmod 0440 /var/lib/ipa/ra-agent.pem # restorecon /var/lib/ipa/ra-agent.pem
Then try something like: ipa cert-show 1
This will exercise the RA agent cert and as long as you don't get an error back things are working again.
The cert is common among all masters so you can copy the file to your other master(s), ensuring proper ownership, permissions and SELinux context.
rob
Hi Rob,
ldapsearch -LLL -o ldif-wrap=no -x -D 'cn=directory manager' -W -b uid=ipara,ou=People,o=ipaca usercertificate
shows me the following:
Issuer: O=CORP.MYDOMAIN.DE, CN=Certificate Authority Validity Not Before: Dec 5 15:32:12 2017 GMT Not After : *Nov 25 15:32:12 2019* GMT
It's going to expire on Monday. Can it be a problem? I tried this command: openssl x509 -text -in /var/lib/ipa/ra-agent.pem
and it shows the following: Certificate: Data: Version: 3 (0x2) Serial Number: 28 (0x1c) Signature Algorithm: sha256WithRSAEncryption Issuer: O=CORP.MYDOMAIN.DE, CN=Certificate Authority Validity Not Before: Oct 29 10:39:47 2019 GMT Not After : Oct 29 09:39:47 2021 GMT Subject: O=CORP.MYDOMAIN.DE, CN=dmud Subject Public Key Info: Public Key Algorithm: rsaEncryption Public-Key: (2048 bit) Modulus: 00:ba:09:81:99:9b:17:99:07:5a:10:28:c8:7a:03: ... 18:db:02:ce:b4:66:ce:5a:e9:12:af:d3:da:bf:f7: 66:5f Exponent: 65537 (0x10001) X509v3 extensions: X509v3 Authority Key Identifier: keyid:D2:...70:BF
X509v3 Subject Key Identifier: DE:...:51:0A X509v3 Subject Alternative Name: email:dmud@corp.mydomain.de Authority Information Access: OCSP - URI:http://ipa-ca.corp.mydomain.de/ca/ocsp
I did nothing to /var/lib/ipa/ra-agent.pem yet.
чт, 21 нояб. 2019 г. в 16:54, Rob Crittenden rcritten@redhat.com:
Dmitri Moudraninets wrote:
Hi Rob,
Yes both masters are failing the same way. Output of openssl x509 -noout -modulus -in /var/lib/ipa/ra-agent.pem is the same on both masters. Output of openssl rsa -noout -modulus -in /var/lib/ipa/ra-agent.key is also the same on both masters. But the output of the first command is not the same as the output of the second command.
I can't remember that I troubleshoot any other problems but we tried to generate some personal certificates for some users. Also we tried to generate certificates with key files for some of our internal services. We did that for the first time and it worked at the end. Also I changed the admin password not so long ago.
Below you can find the output of the requested commands:
[root@second_master ~]# getcert list -f /var/lib/ipa/ra-agent.pem Number of certificates and requests being tracked: 9. Request ID '20180912151730': status: MONITORING stuck: no key pair storage: type=FILE,location='/var/lib/ipa/ra-agent.key' certificate: type=FILE,location='/var/lib/ipa/ra-agent.pem' CA: dogtag-ipa-ca-renew-agent issuer: CN=Certificate Authority,O=CORP.MYDOMAIN.DE http://CORP.MYDOMAIN.DE subject: CN=dmud,O=CORP.MYDOMAIN.DE http://CORP.MYDOMAIN.DE *<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< I see a username here. Does it have to be like that?* expires: 2021-10-29 09:39:47 UTC email: dmud@corp.mydomain.de mailto:dmud@corp.mydomain.de key usage:
digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
eku: id-kp-serverAuth,id-kp-clientAuth pre-save command: /usr/libexec/ipa/certmonger/renew_ra_cert_pre post-save command: /usr/libexec/ipa/certmonger/renew_ra_cert track: yes auto-renew: yes
Right, someone overwrote the RA agent certificate.
Look to see if the user entry in the CA has the right cert:
$ ldapsearch -LLL -o ldif-wrap=no -x -D 'cn=directory manager' -W -b uid=ipara,ou=People,o=ipaca usercertificate
Put the base64 value of the usercertificate attribute into a file and add a prefix/suffix around it:
-----BEGIN CERTIFICATE----- MII....blah= -----END CERTIFICATE-----
$ openssl x509 -text -in /path/to/file
If the Subject is O = CORP.MYDOMAIN.DE, CN = IPA RA then that's a good start. Also look at the expires date to be sure it is still valid.
Assuming that is ok then re-run the openssl modulus commands to ensure they are the same.
Assuming that too is ok then you have the proper, valid RA agent cert. In that case I'd move the current file out of the way, who knows what it is, then run:
# openssl x509 -in /path/to/file -out /var/lib/ipa/ra-agent.pem (just to properly format the agent cert) # chown root:ipaapi /var/lib/ipa/ra-agent.pem # chmod 0440 /var/lib/ipa/ra-agent.pem # restorecon /var/lib/ipa/ra-agent.pem
Then try something like: ipa cert-show 1
This will exercise the RA agent cert and as long as you don't get an error back things are working again.
The cert is common among all masters so you can copy the file to your other master(s), ensuring proper ownership, permissions and SELinux context.
rob
Dmitri Moudraninets wrote:
Hi Rob,
ldapsearch -LLL -o ldif-wrap=no -x -D 'cn=directory manager' -W -b uid=ipara,ou=People,o=ipaca usercertificate
shows me the following:
Issuer: O=CORP.MYDOMAIN.DE http://CORP.MYDOMAIN.DE, CN=Certificate Authority Validity Not Before: Dec 5 15:32:12 2017 GMT Not After : *Nov 25 15:32:12 2019* GMT
It's going to expire on Monday. Can it be a problem?
You didn't provide the cert subject so I can't be sure this is the right cert. If it contains CN = IPA RA then it is.
And yes, it expires in two days. What you'd need to do is restore it per my previous instruction into /var/lib/ipa/ra-agent.pem on the renewal master (ipa config-show to see which one it is).
Then run:
# getcert resubmit -f /var/lib/ipa/ra-agent.pem
That should renew the cert.
On the other masters I'd run the same command and that may fix things there as well.
rob
I tried this command: openssl x509 -text -in /var/lib/ipa/ra-agent.pem
and it shows the following: Certificate: Data: Version: 3 (0x2) Serial Number: 28 (0x1c) Signature Algorithm: sha256WithRSAEncryption Issuer: O=CORP.MYDOMAIN.DE http://CORP.MYDOMAIN.DE, CN=Certificate Authority Validity Not Before: Oct 29 10:39:47 2019 GMT Not After : Oct 29 09:39:47 2021 GMT Subject: O=CORP.MYDOMAIN.DE, CN=dmud Subject Public Key Info: Public Key Algorithm: rsaEncryption Public-Key: (2048 bit) Modulus: 00:ba:09:81:99:9b:17:99:07:5a:10:28:c8:7a:03: ... 18:db:02:ce:b4:66:ce:5a:e9:12:af:d3:da:bf:f7: 66:5f Exponent: 65537 (0x10001) X509v3 extensions: X509v3 Authority Key Identifier: keyid:D2:...70:BF
X509v3 Subject Key Identifier: DE:...:51:0A X509v3 Subject Alternative Name: email:dmud@corp.mydomain.de mailto:email%3Admud@corp.mydomain.de Authority Information Access: OCSP - URI:http://ipa-ca.corp.mydomain.de/ca/ocsp
I did nothing to /var/lib/ipa/ra-agent.pem yet.
чт, 21 нояб. 2019 г. в 16:54, Rob Crittenden <rcritten@redhat.com mailto:rcritten@redhat.com>:
Dmitri Moudraninets wrote: > Hi Rob, > > Yes both masters are failing the same way. Output of openssl x509 -noout > -modulus -in /var/lib/ipa/ra-agent.pem is the same on both masters. > Output of openssl rsa -noout -modulus -in /var/lib/ipa/ra-agent.key is > also the same on both masters. But the output of the first command is > not the same as the output of the second command. > > I can't remember that I troubleshoot any other problems but we tried to > generate some personal certificates for some users. Also we tried to > generate certificates with key files for some of our internal services. > We did that for the first time and it worked at the end. Also I changed > the admin password not so long ago. > > > Below you can find the output of the requested commands: > > > [root@second_master ~]# getcert list -f /var/lib/ipa/ra-agent.pem > Number of certificates and requests being tracked: 9. > Request ID '20180912151730': > status: MONITORING > stuck: no > key pair storage: type=FILE,location='/var/lib/ipa/ra-agent.key' > certificate: type=FILE,location='/var/lib/ipa/ra-agent.pem' > CA: dogtag-ipa-ca-renew-agent > issuer: CN=Certificate Authority,O=CORP.MYDOMAIN.DE <http://CORP.MYDOMAIN.DE> > <http://CORP.MYDOMAIN.DE> > subject: CN=dmud,O=CORP.MYDOMAIN.DE <http://CORP.MYDOMAIN.DE> <http://CORP.MYDOMAIN.DE> > *<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< I see a username here. Does it have > to be like that?* > expires: 2021-10-29 09:39:47 UTC > email: dmud@corp.mydomain.de <mailto:dmud@corp.mydomain.de> <mailto:dmud@corp.mydomain.de <mailto:dmud@corp.mydomain.de>> > key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment > eku: id-kp-serverAuth,id-kp-clientAuth > pre-save command: /usr/libexec/ipa/certmonger/renew_ra_cert_pre > post-save command: /usr/libexec/ipa/certmonger/renew_ra_cert > track: yes > auto-renew: yes Right, someone overwrote the RA agent certificate. Look to see if the user entry in the CA has the right cert: $ ldapsearch -LLL -o ldif-wrap=no -x -D 'cn=directory manager' -W -b uid=ipara,ou=People,o=ipaca usercertificate Put the base64 value of the usercertificate attribute into a file and add a prefix/suffix around it: -----BEGIN CERTIFICATE----- MII....blah= -----END CERTIFICATE----- $ openssl x509 -text -in /path/to/file If the Subject is O = CORP.MYDOMAIN.DE <http://CORP.MYDOMAIN.DE>, CN = IPA RA then that's a good start. Also look at the expires date to be sure it is still valid. Assuming that is ok then re-run the openssl modulus commands to ensure they are the same. Assuming that too is ok then you have the proper, valid RA agent cert. In that case I'd move the current file out of the way, who knows what it is, then run: # openssl x509 -in /path/to/file -out /var/lib/ipa/ra-agent.pem (just to properly format the agent cert) # chown root:ipaapi /var/lib/ipa/ra-agent.pem # chmod 0440 /var/lib/ipa/ra-agent.pem # restorecon /var/lib/ipa/ra-agent.pem Then try something like: ipa cert-show 1 This will exercise the RA agent cert and as long as you don't get an error back things are working again. The cert is common among all masters so you can copy the file to your other master(s), ensuring proper ownership, permissions and SELinux context. rob
-- WBR Dmitry
Hi Rob,
I did the following: I removed original ra-agent.pem and ra-agent key and openssl x509 -in /root/debug.cert -out /var/lib/ipa/ra-agent.pem chown root:ipaapi /var/lib/ipa/ra-agent.pem chmod 0440 /var/lib/ipa/ra-agent.pem restorecon /var/lib/ipa/ra-agent.pem
Successfully restarted FreeIPA: Directory Service: RUNNING krb5kdc Service: RUNNING kadmin Service: RUNNING named Service: RUNNING httpd Service: RUNNING ipa-custodia Service: RUNNING ntpd Service: RUNNING pki-tomcatd Service: RUNNING smb Service: RUNNING winbind Service: RUNNING ipa-otpd Service: RUNNING ipa-dnskeysyncd Service: RUNNING ipa: INFO: The ipactl command was successful
Now GUI shows different error: cannot connect to ' https://freeipa.corp.mydomain.de:443/ca/agent/ca/displayBySerial': [Errno 2] No such file or directory
[root@freeipa ~]# getcert list -f /var/lib/ipa/ra-agent.pem Number of certificates and requests being tracked: 16. Request ID '20180912151611': status: NEED_CSR stuck: no key pair storage: type=FILE,location='/var/lib/ipa/ra-agent.key' certificate: type=FILE,location='/var/lib/ipa/ra-agent.pem' CA: dogtag-ipa-ca-renew-agent issuer: CN=Certificate Authority,O=CORP.MYDOMAIN.DE subject: CN=IPA RA,O=CORP.MYDOMAIN.DE expires: 2019-11-25 15:32:12 UTC key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment eku: id-kp-serverAuth,id-kp-clientAuth pre-save command: /usr/libexec/ipa/certmonger/renew_ra_cert_pre post-save command: /usr/libexec/ipa/certmonger/renew_ra_cert track: yes auto-renew: yes
How to proceed further?
сб, 23 нояб. 2019 г. в 20:26, Rob Crittenden rcritten@redhat.com:
Dmitri Moudraninets wrote:
Hi Rob,
ldapsearch -LLL -o ldif-wrap=no -x -D 'cn=directory manager' -W -b uid=ipara,ou=People,o=ipaca usercertificate
shows me the following:
Issuer: O=CORP.MYDOMAIN.DE <http://CORP.MYDOMAIN.DE>,
CN=Certificate Authority Validity Not Before: Dec 5 15:32:12 2017 GMT Not After : *Nov 25 15:32:12 2019* GMT
It's going to expire on Monday. Can it be a problem?
You didn't provide the cert subject so I can't be sure this is the right cert. If it contains CN = IPA RA then it is.
And yes, it expires in two days. What you'd need to do is restore it per my previous instruction into /var/lib/ipa/ra-agent.pem on the renewal master (ipa config-show to see which one it is).
Then run:
# getcert resubmit -f /var/lib/ipa/ra-agent.pem
That should renew the cert.
On the other masters I'd run the same command and that may fix things there as well.
rob
I tried this command: openssl x509 -text -in /var/lib/ipa/ra-agent.pem
and it shows the following: Certificate: Data: Version: 3 (0x2) Serial Number: 28 (0x1c) Signature Algorithm: sha256WithRSAEncryption Issuer: O=CORP.MYDOMAIN.DE http://CORP.MYDOMAIN.DE, CN=Certificate Authority Validity Not Before: Oct 29 10:39:47 2019 GMT Not After : Oct 29 09:39:47 2021 GMT Subject: O=CORP.MYDOMAIN.DE, CN=dmud Subject Public Key Info: Public Key Algorithm: rsaEncryption Public-Key: (2048 bit) Modulus: 00:ba:09:81:99:9b:17:99:07:5a:10:28:c8:7a:03: ... 18:db:02:ce:b4:66:ce:5a:e9:12:af:d3:da:bf:f7: 66:5f Exponent: 65537 (0x10001) X509v3 extensions: X509v3 Authority Key Identifier: keyid:D2:...70:BF
X509v3 Subject Key Identifier: DE:...:51:0A X509v3 Subject Alternative Name: email:dmud@corp.mydomain.de
mailto:email%3Admud@corp.mydomain.de Authority Information Access: OCSP - URI:http://ipa-ca.corp.mydomain.de/ca/ocsp
I did nothing to /var/lib/ipa/ra-agent.pem yet.
чт, 21 нояб. 2019 г. в 16:54, Rob Crittenden <rcritten@redhat.com mailto:rcritten@redhat.com>:
Dmitri Moudraninets wrote: > Hi Rob, > > Yes both masters are failing the same way. Output of openssl x509 -noout > -modulus -in /var/lib/ipa/ra-agent.pem is the same on both masters. > Output of openssl rsa -noout -modulus -in
/var/lib/ipa/ra-agent.key is
> also the same on both masters. But the output of the first command
is
> not the same as the output of the second command. > > I can't remember that I troubleshoot any other problems but we tried to > generate some personal certificates for some users. Also we tried
to
> generate certificates with key files for some of our internal services. > We did that for the first time and it worked at the end. Also I changed > the admin password not so long ago. > > > Below you can find the output of the requested commands: > > > [root@second_master ~]# getcert list -f /var/lib/ipa/ra-agent.pem > Number of certificates and requests being tracked: 9. > Request ID '20180912151730': > status: MONITORING > stuck: no > key pair storage: type=FILE,location='/var/lib/ipa/ra-agent.key' > certificate: type=FILE,location='/var/lib/ipa/ra-agent.pem' > CA: dogtag-ipa-ca-renew-agent > issuer: CN=Certificate Authority,O=CORP.MYDOMAIN.DE <http://CORP.MYDOMAIN.DE> > <http://CORP.MYDOMAIN.DE> > subject: CN=dmud,O=CORP.MYDOMAIN.DE <http://CORP.MYDOMAIN.DE> <http://CORP.MYDOMAIN.DE> > *<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< I see a username here. Does it
have
> to be like that?* > expires: 2021-10-29 09:39:47 UTC > email: dmud@corp.mydomain.de <mailto:dmud@corp.mydomain.de> <mailto:dmud@corp.mydomain.de <mailto:dmud@corp.mydomain.de>> > key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment > eku: id-kp-serverAuth,id-kp-clientAuth > pre-save command: /usr/libexec/ipa/certmonger/renew_ra_cert_pre > post-save command: /usr/libexec/ipa/certmonger/renew_ra_cert > track: yes > auto-renew: yes Right, someone overwrote the RA agent certificate. Look to see if the user entry in the CA has the right cert: $ ldapsearch -LLL -o ldif-wrap=no -x -D 'cn=directory manager' -W -b uid=ipara,ou=People,o=ipaca usercertificate Put the base64 value of the usercertificate attribute into a file and add a prefix/suffix around it: -----BEGIN CERTIFICATE----- MII....blah= -----END CERTIFICATE----- $ openssl x509 -text -in /path/to/file If the Subject is O = CORP.MYDOMAIN.DE <http://CORP.MYDOMAIN.DE>, CN = IPA RA then that's a good start. Also look at the expires date to be sure it is still valid. Assuming that is ok then re-run the openssl modulus commands to
ensure
they are the same. Assuming that too is ok then you have the proper, valid RA agent
cert.
In that case I'd move the current file out of the way, who knows
what it
is, then run: # openssl x509 -in /path/to/file -out /var/lib/ipa/ra-agent.pem
(just to
properly format the agent cert) # chown root:ipaapi /var/lib/ipa/ra-agent.pem # chmod 0440 /var/lib/ipa/ra-agent.pem # restorecon /var/lib/ipa/ra-agent.pem Then try something like: ipa cert-show 1 This will exercise the RA agent cert and as long as you don't get an error back things are working again. The cert is common among all masters so you can copy the file to your other master(s), ensuring proper ownership, permissions and SELinux context. rob
-- WBR Dmitry
Dmitri Moudraninets wrote:
Hi Rob,
I did the following: I removed original ra-agent.pem and ra-agent key and openssl x509 -in /root/debug.cert -out /var/lib/ipa/ra-agent.pem chown root:ipaapi /var/lib/ipa/ra-agent.pem chmod 0440 /var/lib/ipa/ra-agent.pem restorecon /var/lib/ipa/ra-agent.pem
You removed the key!? I sure hope you have a backup of it.
Put it back and I think that will resolve things.
Successfully restarted FreeIPA: Directory Service: RUNNING krb5kdc Service: RUNNING kadmin Service: RUNNING named Service: RUNNING httpd Service: RUNNING ipa-custodia Service: RUNNING ntpd Service: RUNNING pki-tomcatd Service: RUNNING smb Service: RUNNING winbind Service: RUNNING ipa-otpd Service: RUNNING ipa-dnskeysyncd Service: RUNNING ipa: INFO: The ipactl command was successful
The agent cert is not required for the CA to operate.
Now GUI shows different error: cannot connect to 'https://freeipa.corp.mydomain.de:443/ca/agent/ca/displayBySerial': [Errno 2] No such file or directory
[root@freeipa ~]# getcert list -f /var/lib/ipa/ra-agent.pem Number of certificates and requests being tracked: 16. Request ID '20180912151611': status: NEED_CSR stuck: no key pair storage: type=FILE,location='/var/lib/ipa/ra-agent.key' certificate: type=FILE,location='/var/lib/ipa/ra-agent.pem' CA: dogtag-ipa-ca-renew-agent issuer: CN=Certificate Authority,O=CORP.MYDOMAIN.DE http://CORP.MYDOMAIN.DE subject: CN=IPA RA,O=CORP.MYDOMAIN.DE http://CORP.MYDOMAIN.DE expires: 2019-11-25 15:32:12 UTC key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment eku: id-kp-serverAuth,id-kp-clientAuth pre-save command: /usr/libexec/ipa/certmonger/renew_ra_cert_pre post-save command: /usr/libexec/ipa/certmonger/renew_ra_cert track: yes auto-renew: yes
This shows that the certificate has the right subject now which is good but you removed its private key so it won't work.
rob
сб, 23 нояб. 2019 г. в 20:26, Rob Crittenden <rcritten@redhat.com mailto:rcritten@redhat.com>:
Dmitri Moudraninets wrote: > Hi Rob, > > ldapsearch -LLL -o ldif-wrap=no -x -D 'cn=directory manager' -W > -b uid=ipara,ou=People,o=ipaca usercertificate > > shows me the following: > > Issuer: O=CORP.MYDOMAIN.DE <http://CORP.MYDOMAIN.DE> <http://CORP.MYDOMAIN.DE>, > CN=Certificate Authority > Validity > Not Before: Dec 5 15:32:12 2017 GMT > Not After : *Nov 25 15:32:12 2019* GMT > > It's going to expire on Monday. Can it be a problem? You didn't provide the cert subject so I can't be sure this is the right cert. If it contains CN = IPA RA then it is. And yes, it expires in two days. What you'd need to do is restore it per my previous instruction into /var/lib/ipa/ra-agent.pem on the renewal master (ipa config-show to see which one it is). Then run: # getcert resubmit -f /var/lib/ipa/ra-agent.pem That should renew the cert. On the other masters I'd run the same command and that may fix things there as well. rob > I tried this command: > openssl x509 -text -in /var/lib/ipa/ra-agent.pem > > and it shows the following: > Certificate: > Data: > Version: 3 (0x2) > Serial Number: 28 (0x1c) > Signature Algorithm: sha256WithRSAEncryption > Issuer: O=CORP.MYDOMAIN.DE <http://CORP.MYDOMAIN.DE> <http://CORP.MYDOMAIN.DE>, > CN=Certificate Authority > Validity > Not Before: Oct 29 10:39:47 2019 GMT > Not After : Oct 29 09:39:47 2021 GMT > Subject: O=CORP.MYDOMAIN.DE <http://CORP.MYDOMAIN.DE>, CN=dmud > Subject Public Key Info: > Public Key Algorithm: rsaEncryption > Public-Key: (2048 bit) > Modulus: > 00:ba:09:81:99:9b:17:99:07:5a:10:28:c8:7a:03: > ... > 18:db:02:ce:b4:66:ce:5a:e9:12:af:d3:da:bf:f7: > 66:5f > Exponent: 65537 (0x10001) > X509v3 extensions: > X509v3 Authority Key Identifier: > keyid:D2:...70:BF > > X509v3 Subject Key Identifier: > DE:...:51:0A > X509v3 Subject Alternative Name: > email:dmud@corp.mydomain.de <mailto:email%3Admud@corp.mydomain.de> > <mailto:email%3Admud@corp.mydomain.de <mailto:email%253Admud@corp.mydomain.de>> > Authority Information Access: > OCSP - URI:http://ipa-ca.corp.mydomain.de/ca/ocsp > > > I did nothing to /var/lib/ipa/ra-agent.pem yet. > > > чт, 21 нояб. 2019 г. в 16:54, Rob Crittenden <rcritten@redhat.com <mailto:rcritten@redhat.com> > <mailto:rcritten@redhat.com <mailto:rcritten@redhat.com>>>: > > Dmitri Moudraninets wrote: > > Hi Rob, > > > > Yes both masters are failing the same way. Output of openssl x509 > -noout > > -modulus -in /var/lib/ipa/ra-agent.pem is the same on both masters. > > Output of openssl rsa -noout -modulus -in /var/lib/ipa/ra-agent.key is > > also the same on both masters. But the output of the first command is > > not the same as the output of the second command. > > > > I can't remember that I troubleshoot any other problems but we > tried to > > generate some personal certificates for some users. Also we tried to > > generate certificates with key files for some of our internal > services. > > We did that for the first time and it worked at the end. Also I > changed > > the admin password not so long ago. > > > > > > Below you can find the output of the requested commands: > > > > > > [root@second_master ~]# getcert list -f /var/lib/ipa/ra-agent.pem > > Number of certificates and requests being tracked: 9. > > Request ID '20180912151730': > > status: MONITORING > > stuck: no > > key pair storage: type=FILE,location='/var/lib/ipa/ra-agent.key' > > certificate: type=FILE,location='/var/lib/ipa/ra-agent.pem' > > CA: dogtag-ipa-ca-renew-agent > > issuer: CN=Certificate Authority,O=CORP.MYDOMAIN.DE <http://CORP.MYDOMAIN.DE> > <http://CORP.MYDOMAIN.DE> > > <http://CORP.MYDOMAIN.DE> > > subject: CN=dmud,O=CORP.MYDOMAIN.DE <http://CORP.MYDOMAIN.DE> <http://CORP.MYDOMAIN.DE> > <http://CORP.MYDOMAIN.DE> > > *<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< I see a username here. Does it have > > to be like that?* > > expires: 2021-10-29 09:39:47 UTC > > email: dmud@corp.mydomain.de <mailto:dmud@corp.mydomain.de> <mailto:dmud@corp.mydomain.de <mailto:dmud@corp.mydomain.de>> > <mailto:dmud@corp.mydomain.de <mailto:dmud@corp.mydomain.de> <mailto:dmud@corp.mydomain.de <mailto:dmud@corp.mydomain.de>>> > > key usage: > digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment > > eku: id-kp-serverAuth,id-kp-clientAuth > > pre-save command: /usr/libexec/ipa/certmonger/renew_ra_cert_pre > > post-save command: /usr/libexec/ipa/certmonger/renew_ra_cert > > track: yes > > auto-renew: yes > > Right, someone overwrote the RA agent certificate. > > Look to see if the user entry in the CA has the right cert: > > $ ldapsearch -LLL -o ldif-wrap=no -x -D 'cn=directory manager' -W -b > uid=ipara,ou=People,o=ipaca usercertificate > > Put the base64 value of the usercertificate attribute into a file and > add a prefix/suffix around it: > > -----BEGIN CERTIFICATE----- > MII....blah= > -----END CERTIFICATE----- > > $ openssl x509 -text -in /path/to/file > > If the Subject is O = CORP.MYDOMAIN.DE <http://CORP.MYDOMAIN.DE> <http://CORP.MYDOMAIN.DE>, CN > = IPA RA then that's a good > start. Also look at the expires date to be sure it is still valid. > > Assuming that is ok then re-run the openssl modulus commands to ensure > they are the same. > > Assuming that too is ok then you have the proper, valid RA agent cert. > In that case I'd move the current file out of the way, who knows what it > is, then run: > > # openssl x509 -in /path/to/file -out /var/lib/ipa/ra-agent.pem (just to > properly format the agent cert) > # chown root:ipaapi /var/lib/ipa/ra-agent.pem > # chmod 0440 /var/lib/ipa/ra-agent.pem > # restorecon /var/lib/ipa/ra-agent.pem > > Then try something like: ipa cert-show 1 > > This will exercise the RA agent cert and as long as you don't get an > error back things are working again. > > The cert is common among all masters so you can copy the file to your > other master(s), ensuring proper ownership, permissions and SELinux > context. > > rob > > > > -- > WBR > Dmitry
-- With best regards/Mit freundlichen Grüßen
Moudraninets Dmitry, RHCSA http://www.linkedin.com/in/moudraninets http://www.xing.com/profile/Dmitry_Mudraninets
freeipa-users@lists.fedorahosted.org