So I went back and read, reread, studied what you wrote and I think I’m following you. I’m really unfamiliar with certs and the tools around it so forgive the ignorance.
So what I ended up doing is spinning up a CentOS7 VM and installing everything on it, adding it to the FreeIPA realm etc. and followed your instructions/email.
I ran the
modutil -dbdir sql:./mozilla/firefox/9zd63dro.default/ -list
It returns the list of the PKCS #11 Modules like I listed in my previous email. However, it only showed a single item “NSS Internal PKCS #11 Module”.
To look at what keys it had I ran
certutil -d sql:./mozilla/firefox/9zd63dro.default/ -h “NSS Internal PKCS #11” -L
This seemed like it returned all of the system wide certs. Including my self signed internal lan cert from freeipa. Should it have? That’s where I’m getting confused with your comment in your email when you mentioned the p11-kit-proxy and where it’s coming from, how it was added (if needed) as you said it was providing all of the system wide certs?
At this point this is where things took a detour and I think it’s part of my confusion, which I think is unrelated, but I was using Firefox, all of the certs are there in the system based on the commands you showed. However, every time i would visit my http server Firefox would throw a
SEC_ERROR_REVOKED_CERTIFICATE
I pulled my hair out for 2 hours, deleting the .mozilla folder, recreating it, looking at certs, trying to manually copy certs into the cert db etc.
Until I got fed up and tried Chrome...i downloaded chrome installed it ran it, checked the certs db looked at the certs and verified my internal cert was listed just like firefox. I visited the http server in chrome and it worked perfectly. No changes, which I believe is what you would expect.
I then went and tried the same thing on Ubuntu. I know you mentioned that I have to add the certs manually as Ubuntu doesn’t have the same functionality. So I just manually added my ipa.crt to firefox and then got a
SEC_ERROR_REVOKED_CERTIFICATE
installed chrome on ubuntu machine and manually imported the ipa.crt into chrome, went to the http and chrome worked fine.
So now I have no idea where I’m getting this
SEC_ERROR_REVOKED_CERTIFICATE
So now on a freeipa realm joined host. It seems that
CentOS7 - Firefox gets a - SEC_ERROR_REVOKED_CERTIFICATE Chrome - Works out of the box
Ubuntu 18.04 - Firefox gets after manually adding cert- SEC_ERROR_REVOKED_CERTIFICATE Chrome - works after manually adding the ipa.ca cert through GUI.
Is there some obvious reason why firefox would throw that error message but Chrome wouldn’t?
This stuff is making my head spin.
On Thu, Oct 10, 2019 at 2:45 PM Kevin Vasko kvasko@gmail.com wrote:
So you are saying that if the p11-kit-trust module is available it should be automatically adding the system wide trust store into the internal Firefox cert store?
This is the out of my commands. I have the cert store thats create in my home directory.
But there is no p11-kit-proxy do I have to add that myself? If so how do I do that?
modutil -dbdir sql:/home/<username>/.mozilla/firefox/9zd63dro.default-release/ -list
Listing of PKCS #11 Modules
- NSS Internal PKCS #11 Module uri:
pkcs11:library-manufacturer=Mozilla%20Foundation;library-description=NSS%20Internal%20Crypto%20Services;library-version=3.35 slots: 2 slots attached status: loaded
slot: NSS Internal Cryptographic Services token: NSS Generic Crypto Services uri: pkcs11:token=NSS%20Generic%20Crypto%20Services;manufacturer=Mozilla%20Foundation;serial=0000000000000000;model=NSS%203 slot: NSS User Private Key and Certificate Services token: NSS Certificate DB uri: pkcs11:token=NSS%20Certificate%20DB;manufacturer=Mozilla%20Foundation;serial=0000000000000000;model=NSS%203
I have the p11-kit-trust module.
$ p11-kit list-modules p11-kit-trust: p11-kit-trust.so library-description: PKCS#11 Kit Trust Module library-manufacturer: PKCS#11 Kit library-version: 0.23 token: System Trust manufacturer: PKCS#11 Kit model: p11-kit-trust serial-number: 1 hardware-version: 0.23 flags: write-protected token-initialized
On Thu, Oct 10, 2019 at 11:09 AM Alexander Bokovoy abokovoy@redhat.com wrote: On to, 10 loka 2019, Kevin Vasko wrote:
Alexander, Unless I'm misunderstanding the information I don't think it will matter though because Firefox and Chrome use their own certificates stores. I found that information after I posted this question. Speaking specifically for firefox (and Chrome looks to be similar)...I'm concluding that why I'm not seeing it work is because of this... "Since Firefox does not use the operating system's certificate store by default, these CA certificates must be added in to Firefox using one of the following methods. " taken from here https://wiki.mozilla.org/CA/AddRootToFirefox
On RHEL/Fedora we do have some magic: https://fedoraproject.org/wiki/Changes/NSSLoadP11KitModules On my Fedora 30 system I have this for my Firefox profile: $ modutil -dbdir sql:/home/abokovoy/.mozilla/firefox/$profile/ -list Listing of PKCS #11 Modules
- NSS Internal PKCS #11 Module uri: pkcs11:library-manufacturer=Mozilla%20Foundation;library-description=NSS%20Internal%20Crypto%20Services;library-version=3.46 slots: 2 slots attached status: loaded slot: NSS Internal Cryptographic Services token: NSS Generic Crypto Services uri: pkcs11:token=NSS%20Generic%20Crypto%20Services;manufacturer=Mozilla%20Foundation;serial=0000000000000000;model=NSS%203 slot: NSS User Private Key and Certificate Services token: NSS Certificate DB uri: pkcs11:token=NSS%20Certificate%20DB;manufacturer=Mozilla%20Foundation;serial=0000000000000000;model=NSS%203
- mPollux library name: /usr/lib64/libcryptoki.so uri: pkcs11:library-manufacturer=Fujitsu%20Finland%20Oy;library-description=mPollux%20DigiSign%20Client;library-version=0.1 slots: There are no slots attached to this module status: loaded
- p11-kit-proxy library name: p11-kit-proxy.so uri: pkcs11:library-manufacturer=PKCS%2311%20Kit;library-description=PKCS%2311%20Kit%20Proxy%20Module;library-version=1.1 slots: There are no slots attached to this module status: loaded
As you can see, there are three tokens attached. Number 1 is the NSS internal 'token', that's how NSS database looks like typically. Number 2 is a crypto token inserted by the Fujitsu Finland Oy which is used for my governmental ID operations through Firefox. Number three is the proxy for system-wide crypto tokens in Fedora. If I query that token separately, I can see a lot of certificates inside Firefox NSS database. If I omit -h option, certificates from all tokens get listed. $ certutil -d sql:/home/abokovoy/.mozilla/firefox/$profile/ -h p11-kit-proxy -L |wc -l 249 Exactly same story is with Chrome/Chromium, only that they use different store than Firefox: $ modutil -dbdir sql:/home/abokovoy/.pki/nssdb -list Listing of PKCS #11 Modules
- NSS Internal PKCS #11 Module uri: pkcs11:library-manufacturer=Mozilla%20Foundation;library-description=NSS%20Internal%20Crypto%20Services;library-version=3.46 slots: 2 slots attached status: loaded slot: NSS Internal Cryptographic Services token: NSS Generic Crypto Services uri: pkcs11:token=NSS%20Generic%20Crypto%20Services;manufacturer=Mozilla%20Foundation;serial=0000000000000000;model=NSS%203 slot: NSS User Private Key and Certificate Services token: NSS Certificate DB uri: pkcs11:token=NSS%20Certificate%20DB;manufacturer=Mozilla%20Foundation;serial=0000000000000000;model=NSS%203
- DigiSign PKCS#11 Module library name: /usr/lib64/libcryptoki.so uri: pkcs11:library-manufacturer=Fujitsu%20Finland%20Oy;library-description=mPollux%20DigiSign%20Client;library-version=0.1 slots: There are no slots attached to this module status: loaded
- p11-kit-proxy library name: p11-kit-proxy.so uri: pkcs11:library-manufacturer=PKCS%2311%20Kit;library-description=PKCS%2311%20Kit%20Proxy%20Module;library-version=1.1 slots: There are no slots attached to this module status: loaded
In past, people did manual work to pick up all the certs like https://blog.xelnor.net/firefox-systemcerts/ but it is not really needed anymore if you have p11-kit-proxy on your system. By default p11-kit-proxy has two modules: $ p11-kit list-modules p11-kit-trust: p11-kit-trust.so library-description: PKCS#11 Kit Trust Module library-manufacturer: PKCS#11 Kit library-version: 0.23 token: System Trust manufacturer: PKCS#11 Kit model: p11-kit-trust serial-number: 1 hardware-version: 0.23 flags: write-protected token-initialized token: Default Trust manufacturer: PKCS#11 Kit model: p11-kit-trust serial-number: 1 hardware-version: 0.23 flags: write-protected token-initialized opensc: opensc-pkcs11.so library-description: OpenSC smartcard framework library-manufacturer: OpenSC Project library-version: 0.19 It is the first one that brings all the system-wide certificates into NSS and other databases. For OpenSSL applications it can be brought in via PKCS#11 engine support.
So I at this point I don't think anything is wrong with ipa-install-client and it is performing correctly at this point adding it to the cert store. Given that the exception that you mentioned, that there is a difference in ipa-install-client adding it to the the NSS database on RHEL/Fedora/CentOS and not on the Ubuntu/Debian variants. However, I still don't think that will matter since Firefox/Chrome aren't reading either the NSS database or the crt bundle from what I understand. I'm going to keep digging to see if I find a solution for getting FF/Chrome to look at my certs and will post back on what I find. -Kevin On Thu, Oct 10, 2019 at 9:17 AM Alexander Bokovoy abokovoy@redhat.com wrote:
On to, 10 loka 2019, Kevin Vasko via FreeIPA-users wrote:
I actually manually checked the system wide crt files on each distribution I'm using, Ubuntu, CentOS and RHEL6/7. In all cases my /etc/ipa/ca.crt did appear to be in the each of their respective *.crt files. That indicates to me that there isn't any problem with the ipa-install-client on any of the distributions like I originally thought. Rob it does look like Ubuntu is adding it to the /etc/ssl/certs/ca-certificates.crt with the ipa-install-client as I didn't do it manually on any of my systems, so it does appear they are doing it somehow. These are the locations I checked. "/etc/ssl/certs/ca-certificates.crt", // Debian/Ubuntu/Gentoo etc. "/etc/pki/tls/certs/ca-bundle.crt", // Fedora/RHEL 6 "/etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem", // CentOS/RHEL 7 What appears to be the problem is (unless I'm mistaken) Firefox nor Chrome are using the system wide cert locations apparently and only using their own cert store. At least according to this article: https://thomas-leister.de/en/how-to-import-ca-root-certificate/
On RHEL/Fedora/CentOS we import system wide cert store automatically to NSS databases through p11-kit. On Ubuntu/Debian/Gentoo you need to do that manually.
It kind of is backed up by this article on the Mozilla page. https://support.mozilla.org/en-US/kb/setting-certificate-authorities-firefox So based off of this information I'm going to have to manually add the root certificates to each Chrome and Firefox cert store on the client machines, which is a bummer. Sorry for the noise. On Thu, Oct 10, 2019 at 8:40 AM Rob Crittenden rcritten@redhat.com wrote:
Kevin Vasko via FreeIPA-users wrote: > Kees Bakker, > If it is, I'm certainly not seeing it done on Ubuntu 16.04 or Ubuntu > 18.04 and based on Rob's comment it might not be done if I'm > understanding him correctly. Assuming I'm reading the code right it is not being executed on Debian/Ubuntu. At least not in the source. It's possible it is patched into the package in the distribution. rob > -Kevin > On Thu, Oct 10, 2019 at 8:19 AM Kees Bakker via FreeIPA-users > freeipa-users@lists.fedorahosted.org wrote: >> On 10-10-19 14:35, Rob Crittenden via FreeIPA-users wrote >>> Kevin Vasko via FreeIPA-users wrote: >>>> How would I validate that certs are getting added properly on a CentOS machine system wide store? >>>> I’m going to test it today to find out if this is a problem unique to Ubuntu/CentOS. >>> On Fedora the chain is put into >>> /etc/pki/ca-trust/source/anchors/ipa-ca.crt and update-ca-trust is executed. >>> There is no Debian/Ubuntu equivalent in the upstream source (it's >>> possible it is done in packaging). You could try something like: >>> cp /etc/ipa/ca.crt /usr/local/share/ca-certificates/ipa-ca.crt >>> update-ca-certificates >> This is already done by ipa-client-install >> _______________________________________________ >> FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org >> To unsubscribe send an email to freeipa-users-leave@lists.fedorahosted.org >> Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ >> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines >> List Archives: https://lists.fedorahosted.org/archives/list/freeipa-users@lists.fedorahoste... > _______________________________________________ > FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org > To unsubscribe send an email to freeipa-users-leave@lists.fedorahosted.org > Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: https://lists.fedorahosted.org/archives/list/freeipa-users@lists.fedorahoste...
FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org To unsubscribe send an email to freeipa-users-leave@lists.fedorahosted.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedorahosted.org/archives/list/freeipa-users@lists.fedorahoste...
-- / Alexander Bokovoy Sr. Principal Software Engineer Security / Identity Management Engineering Red Hat Limited, Finland
-- / Alexander Bokovoy Sr. Principal Software Engineer Security / Identity Management Engineering Red Hat Limited, Finland
On to, 10 loka 2019, Kevin Vasko wrote:
So I went back and read, reread, studied what you wrote and I think I’m following you. I’m really unfamiliar with certs and the tools around it so forgive the ignorance.
So what I ended up doing is spinning up a CentOS7 VM and installing everything on it, adding it to the FreeIPA realm etc. and followed your instructions/email.
I ran the
modutil -dbdir sql:./mozilla/firefox/9zd63dro.default/ -list
It returns the list of the PKCS #11 Modules like I listed in my previous email. However, it only showed a single item “NSS Internal PKCS #11 Module”.
To look at what keys it had I ran
certutil -d sql:./mozilla/firefox/9zd63dro.default/ -h “NSS Internal PKCS #11” -L
This seemed like it returned all of the system wide certs. Including my self signed internal lan cert from freeipa. Should it have? That’s where I’m getting confused with your comment in your email when you mentioned the p11-kit-proxy and where it’s coming from, how it was added (if needed) as you said it was providing all of the system wide certs?
At this point this is where things took a detour and I think it’s part of my confusion, which I think is unrelated, but I was using Firefox, all of the certs are there in the system based on the commands you showed. However, every time i would visit my http server Firefox would throw a
SEC_ERROR_REVOKED_CERTIFICATE
I pulled my hair out for 2 hours, deleting the .mozilla folder, recreating it, looking at certs, trying to manually copy certs into the cert db etc.
Until I got fed up and tried Chrome...i downloaded chrome installed it ran it, checked the certs db looked at the certs and verified my internal cert was listed just like firefox. I visited the http server in chrome and it worked perfectly. No changes, which I believe is what you would expect.
I then went and tried the same thing on Ubuntu. I know you mentioned that I have to add the certs manually as Ubuntu doesn’t have the same functionality. So I just manually added my ipa.crt to firefox and then got a
SEC_ERROR_REVOKED_CERTIFICATE
installed chrome on ubuntu machine and manually imported the ipa.crt into chrome, went to the http and chrome worked fine.
So now I have no idea where I’m getting this
SEC_ERROR_REVOKED_CERTIFICATE
So now on a freeipa realm joined host. It seems that
CentOS7 - Firefox gets a - SEC_ERROR_REVOKED_CERTIFICATE Chrome - Works out of the box
Ubuntu 18.04 - Firefox gets after manually adding cert- SEC_ERROR_REVOKED_CERTIFICATE Chrome - works after manually adding the ipa.ca cert through GUI.
Is there some obvious reason why firefox would throw that error message but Chrome wouldn’t?
This stuff is making my head spin.
For that host certificate Firefox thinks it is revoked by its issuer. Did you fiddle with the certificates? Perhaps, it would be easier to find out what certificate is that and check its status in IPA or whoever issued it?
I'm 100% positive I did nothing with this cert.
To validate, I spun up a brand new machine completely from scratch.
1. ran yum update 2. installed Gnome 3. installed ipa with my normal "sudo ipa-client-install --domain=exaple.com --realm=EXAMPLE.COM --enable-dns-updates --mkhomedir" 4. started Gnome with "startx" 5. Went to URL with Firefox, firefox errored with the "SEC_ERROR_REVOKED_CERTIFICATE" 6. installed chrome 7. went to same URL with Chrome, chrome works.
LSB Version: :core-4.1-amd64:core-4.1-noarch Distributor ID: CentOS Description: CentOS Linux release 7.7.1908 (Core) Release: 7.7.1908 Codename: Core Linux testmachine.example.com 3.10.0-1062.1.2.el7.x86_64 #1 SMP Mon Sep 30 14:19:46 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux firefox.x86_64 60.9.0-1.el7.centos @updates ipa-client.x86_64 4.6.5-11.el7.centos @base ipa-client-common.noarch 4.6.5-11.el7.centos @base ipa-common.noarch 4.6.5-11.el7.centos @base chrome-gnome-shell.x86_64 10.1-4.el7 @base google-chrome-stable.x86_64 77.0.3865.120-1 @google-chrome
How can I validate the certificate?
On Fri, Oct 11, 2019 at 12:11 AM Alexander Bokovoy abokovoy@redhat.com wrote:
On to, 10 loka 2019, Kevin Vasko wrote:
So I went back and read, reread, studied what you wrote and I think I’m following you. I’m really unfamiliar with certs and the tools around it so forgive the ignorance.
So what I ended up doing is spinning up a CentOS7 VM and installing everything on it, adding it to the FreeIPA realm etc. and followed your instructions/email.
I ran the
modutil -dbdir sql:./mozilla/firefox/9zd63dro.default/ -list
It returns the list of the PKCS #11 Modules like I listed in my previous email. However, it only showed a single item “NSS Internal PKCS #11 Module”.
To look at what keys it had I ran
certutil -d sql:./mozilla/firefox/9zd63dro.default/ -h “NSS Internal PKCS #11” -L
This seemed like it returned all of the system wide certs. Including my self signed internal lan cert from freeipa. Should it have? That’s where I’m getting confused with your comment in your email when you mentioned the p11-kit-proxy and where it’s coming from, how it was added (if needed) as you said it was providing all of the system wide certs?
At this point this is where things took a detour and I think it’s part of my confusion, which I think is unrelated, but I was using Firefox, all of the certs are there in the system based on the commands you showed. However, every time i would visit my http server Firefox would throw a
SEC_ERROR_REVOKED_CERTIFICATE
I pulled my hair out for 2 hours, deleting the .mozilla folder, recreating it, looking at certs, trying to manually copy certs into the cert db etc.
Until I got fed up and tried Chrome...i downloaded chrome installed it ran it, checked the certs db looked at the certs and verified my internal cert was listed just like firefox. I visited the http server in chrome and it worked perfectly. No changes, which I believe is what you would expect.
I then went and tried the same thing on Ubuntu. I know you mentioned that I have to add the certs manually as Ubuntu doesn’t have the same functionality. So I just manually added my ipa.crt to firefox and then got a
SEC_ERROR_REVOKED_CERTIFICATE
installed chrome on ubuntu machine and manually imported the ipa.crt into chrome, went to the http and chrome worked fine.
So now I have no idea where I’m getting this
SEC_ERROR_REVOKED_CERTIFICATE
So now on a freeipa realm joined host. It seems that
CentOS7 - Firefox gets a - SEC_ERROR_REVOKED_CERTIFICATE Chrome - Works out of the box
Ubuntu 18.04 - Firefox gets after manually adding cert- SEC_ERROR_REVOKED_CERTIFICATE Chrome - works after manually adding the ipa.ca cert through GUI.
Is there some obvious reason why firefox would throw that error message but Chrome wouldn’t?
This stuff is making my head spin.
For that host certificate Firefox thinks it is revoked by its issuer. Did you fiddle with the certificates? Perhaps, it would be easier to find out what certificate is that and check its status in IPA or whoever issued it?
-- / Alexander Bokovoy Sr. Principal Software Engineer Security / Identity Management Engineering Red Hat Limited, Finland
So following these instructions I found out that the certs are NOT revoked.
https://serverfault.com/questions/590504/how-do-i-check-if-my-ssl-certificat...
The one thing I did find is that in Firefox if I uncheck "Query OCSP responder servers to confirm the current validity of certificates". Everything works.
Why thats a problem with firefox I'm not sure...I'm still looking into it though...
On Fri, Oct 11, 2019 at 10:43 AM Kevin Vasko kvasko@gmail.com wrote:
I'm 100% positive I did nothing with this cert.
To validate, I spun up a brand new machine completely from scratch.
- ran yum update
- installed Gnome
- installed ipa with my normal "sudo ipa-client-install
--domain=exaple.com --realm=EXAMPLE.COM --enable-dns-updates --mkhomedir" 4. started Gnome with "startx" 5. Went to URL with Firefox, firefox errored with the "SEC_ERROR_REVOKED_CERTIFICATE" 6. installed chrome 7. went to same URL with Chrome, chrome works.
LSB Version: :core-4.1-amd64:core-4.1-noarch Distributor ID: CentOS Description: CentOS Linux release 7.7.1908 (Core) Release: 7.7.1908 Codename: Core Linux testmachine.example.com 3.10.0-1062.1.2.el7.x86_64 #1 SMP Mon Sep 30 14:19:46 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux firefox.x86_64 60.9.0-1.el7.centos @updates ipa-client.x86_64 4.6.5-11.el7.centos @base ipa-client-common.noarch 4.6.5-11.el7.centos @base ipa-common.noarch 4.6.5-11.el7.centos @base chrome-gnome-shell.x86_64 10.1-4.el7 @base google-chrome-stable.x86_64 77.0.3865.120-1 @google-chrome
How can I validate the certificate?
On Fri, Oct 11, 2019 at 12:11 AM Alexander Bokovoy abokovoy@redhat.com wrote:
On to, 10 loka 2019, Kevin Vasko wrote:
So I went back and read, reread, studied what you wrote and I think I’m following you. I’m really unfamiliar with certs and the tools around it so forgive the ignorance.
So what I ended up doing is spinning up a CentOS7 VM and installing everything on it, adding it to the FreeIPA realm etc. and followed your instructions/email.
I ran the
modutil -dbdir sql:./mozilla/firefox/9zd63dro.default/ -list
It returns the list of the PKCS #11 Modules like I listed in my previous email. However, it only showed a single item “NSS Internal PKCS #11 Module”.
To look at what keys it had I ran
certutil -d sql:./mozilla/firefox/9zd63dro.default/ -h “NSS Internal PKCS #11” -L
This seemed like it returned all of the system wide certs. Including my self signed internal lan cert from freeipa. Should it have? That’s where I’m getting confused with your comment in your email when you mentioned the p11-kit-proxy and where it’s coming from, how it was added (if needed) as you said it was providing all of the system wide certs?
At this point this is where things took a detour and I think it’s part of my confusion, which I think is unrelated, but I was using Firefox, all of the certs are there in the system based on the commands you showed. However, every time i would visit my http server Firefox would throw a
SEC_ERROR_REVOKED_CERTIFICATE
I pulled my hair out for 2 hours, deleting the .mozilla folder, recreating it, looking at certs, trying to manually copy certs into the cert db etc.
Until I got fed up and tried Chrome...i downloaded chrome installed it ran it, checked the certs db looked at the certs and verified my internal cert was listed just like firefox. I visited the http server in chrome and it worked perfectly. No changes, which I believe is what you would expect.
I then went and tried the same thing on Ubuntu. I know you mentioned that I have to add the certs manually as Ubuntu doesn’t have the same functionality. So I just manually added my ipa.crt to firefox and then got a
SEC_ERROR_REVOKED_CERTIFICATE
installed chrome on ubuntu machine and manually imported the ipa.crt into chrome, went to the http and chrome worked fine.
So now I have no idea where I’m getting this
SEC_ERROR_REVOKED_CERTIFICATE
So now on a freeipa realm joined host. It seems that
CentOS7 - Firefox gets a - SEC_ERROR_REVOKED_CERTIFICATE Chrome - Works out of the box
Ubuntu 18.04 - Firefox gets after manually adding cert- SEC_ERROR_REVOKED_CERTIFICATE Chrome - works after manually adding the ipa.ca cert through GUI.
Is there some obvious reason why firefox would throw that error message but Chrome wouldn’t?
This stuff is making my head spin.
For that host certificate Firefox thinks it is revoked by its issuer. Did you fiddle with the certificates? Perhaps, it would be easier to find out what certificate is that and check its status in IPA or whoever issued it?
-- / Alexander Bokovoy Sr. Principal Software Engineer Security / Identity Management Engineering Red Hat Limited, Finland
On pe, 11 loka 2019, Kevin Vasko wrote:
So following these instructions I found out that the certs are NOT revoked.
https://serverfault.com/questions/590504/how-do-i-check-if-my-ssl-certificat...
The one thing I did find is that in Firefox if I uncheck "Query OCSP responder servers to confirm the current validity of certificates". Everything works.
Why thats a problem with firefox I'm not sure...I'm still looking into it though...
As I said, look at the CA that issued that certificate. If it is marked as revoked, it would be present in OCSP and also in CA logs *why* it is revoked.
If this certificate is issued by IPA CA, go to Web UI, Authentication tab, Certificates, and search there for the certificate serial number and subject. See the status of it.
On Fri, Oct 11, 2019 at 10:43 AM Kevin Vasko kvasko@gmail.com wrote:
I'm 100% positive I did nothing with this cert.
To validate, I spun up a brand new machine completely from scratch.
- ran yum update
- installed Gnome
- installed ipa with my normal "sudo ipa-client-install
--domain=exaple.com --realm=EXAMPLE.COM --enable-dns-updates --mkhomedir" 4. started Gnome with "startx" 5. Went to URL with Firefox, firefox errored with the "SEC_ERROR_REVOKED_CERTIFICATE" 6. installed chrome 7. went to same URL with Chrome, chrome works.
LSB Version: :core-4.1-amd64:core-4.1-noarch Distributor ID: CentOS Description: CentOS Linux release 7.7.1908 (Core) Release: 7.7.1908 Codename: Core Linux testmachine.example.com 3.10.0-1062.1.2.el7.x86_64 #1 SMP Mon Sep 30 14:19:46 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux firefox.x86_64 60.9.0-1.el7.centos @updates ipa-client.x86_64 4.6.5-11.el7.centos @base ipa-client-common.noarch 4.6.5-11.el7.centos @base ipa-common.noarch 4.6.5-11.el7.centos @base chrome-gnome-shell.x86_64 10.1-4.el7 @base google-chrome-stable.x86_64 77.0.3865.120-1 @google-chrome
How can I validate the certificate?
On Fri, Oct 11, 2019 at 12:11 AM Alexander Bokovoy abokovoy@redhat.com wrote:
On to, 10 loka 2019, Kevin Vasko wrote:
So I went back and read, reread, studied what you wrote and I think I’m following you. I’m really unfamiliar with certs and the tools around it so forgive the ignorance.
So what I ended up doing is spinning up a CentOS7 VM and installing everything on it, adding it to the FreeIPA realm etc. and followed your instructions/email.
I ran the
modutil -dbdir sql:./mozilla/firefox/9zd63dro.default/ -list
It returns the list of the PKCS #11 Modules like I listed in my previous email. However, it only showed a single item “NSS Internal PKCS #11 Module”.
To look at what keys it had I ran
certutil -d sql:./mozilla/firefox/9zd63dro.default/ -h “NSS Internal PKCS #11” -L
This seemed like it returned all of the system wide certs. Including my self signed internal lan cert from freeipa. Should it have? That’s where I’m getting confused with your comment in your email when you mentioned the p11-kit-proxy and where it’s coming from, how it was added (if needed) as you said it was providing all of the system wide certs?
At this point this is where things took a detour and I think it’s part of my confusion, which I think is unrelated, but I was using Firefox, all of the certs are there in the system based on the commands you showed. However, every time i would visit my http server Firefox would throw a
SEC_ERROR_REVOKED_CERTIFICATE
I pulled my hair out for 2 hours, deleting the .mozilla folder, recreating it, looking at certs, trying to manually copy certs into the cert db etc.
Until I got fed up and tried Chrome...i downloaded chrome installed it ran it, checked the certs db looked at the certs and verified my internal cert was listed just like firefox. I visited the http server in chrome and it worked perfectly. No changes, which I believe is what you would expect.
I then went and tried the same thing on Ubuntu. I know you mentioned that I have to add the certs manually as Ubuntu doesn’t have the same functionality. So I just manually added my ipa.crt to firefox and then got a
SEC_ERROR_REVOKED_CERTIFICATE
installed chrome on ubuntu machine and manually imported the ipa.crt into chrome, went to the http and chrome worked fine.
So now I have no idea where I’m getting this
SEC_ERROR_REVOKED_CERTIFICATE
So now on a freeipa realm joined host. It seems that
CentOS7 - Firefox gets a - SEC_ERROR_REVOKED_CERTIFICATE Chrome - Works out of the box
Ubuntu 18.04 - Firefox gets after manually adding cert- SEC_ERROR_REVOKED_CERTIFICATE Chrome - works after manually adding the ipa.ca cert through GUI.
Is there some obvious reason why firefox would throw that error message but Chrome wouldn’t?
This stuff is making my head spin.
For that host certificate Firefox thinks it is revoked by its issuer. Did you fiddle with the certificates? Perhaps, it would be easier to find out what certificate is that and check its status in IPA or whoever issued it?
-- / Alexander Bokovoy Sr. Principal Software Engineer Security / Identity Management Engineering Red Hat Limited, Finland
Welp, I'm an idiot and you are completely 100% correct.
It was indeed revoked, but the http servers certificate was revoked and not the client..which is where I was focusing 100% of my debugging. Which clears up a LOT of things. I originally was loading the ca.crt on an Ubuntu machine a few days prior to this and it was working completely fine. After a few days I was getting the "SEC_ERROR_REVOKED_CERTIFICATE" when I went back to try it again.
However, what doesn't make sense to me is all of the commands I was running to check the certs were telling me that the certs were 100% okay and not revoked...
I ran this command which is supposedly supposed to tell me if my cert is okay with OCSP
openssl ocsp -issuer /etc/ipa/ca.crt -cert /etc/ipa/ca.crt -text -url http://ipa-ca.exmple.com/ca/ocsp -header "HOST" "ipa.exmple.com"
I was getting a
-----END CERTIFICATE----- Response verify OK /etc/ipa/ca.crt: good
And there was nothing in the result saying that it was expired on my client machines.
https://serverfault.com/questions/590504/how-do-i-check-if-my-ssl-certificat...
However, when I checked the UI like you suggested, the actual HTTP servers certificate was no longer listed. Oddly enough I had another stale browser window up and the certificate was there, BUT I guess in all of my toying with the the certs in removing them and recreating them, I guess I deleted all of them, and when I checked the stale page it showed up. I assume that other than the "valid from:" and the "valid to:" (which is like 2 years in the future) that a certificate wouldn't be revoked automatically? I did check on the http server that certmonger was running and says that "auto-renew: yes".
Can you maybe describe the appropriate way to debug this in the future? I was obviously doing it incorrectly. Which CA logs are you meaning? Are you meaning on the freeIPA servers? Are you meaning the http service itself? Where are you meaning "present in OCSP"? The key to this was my seeing the certificates for the http/service not showing up in the FreeIPA server UI. Once I recreated the http/service certificate the Firefox error went away.
Regardless much appreciated...now to figure out how to handle Ubuntu since like you said Fedora/CentOS work out of the box with the system wide certs...
On Sat, Oct 12, 2019 at 12:49 AM Alexander Bokovoy abokovoy@redhat.com wrote:
On pe, 11 loka 2019, Kevin Vasko wrote:
So following these instructions I found out that the certs are NOT revoked.
https://serverfault.com/questions/590504/how-do-i-check-if-my-ssl-certificat...
The one thing I did find is that in Firefox if I uncheck "Query OCSP responder servers to confirm the current validity of certificates". Everything works.
Why thats a problem with firefox I'm not sure...I'm still looking into it though...
As I said, look at the CA that issued that certificate. If it is marked as revoked, it would be present in OCSP and also in CA logs *why* it is revoked.
If this certificate is issued by IPA CA, go to Web UI, Authentication tab, Certificates, and search there for the certificate serial number and subject. See the status of it.
On Fri, Oct 11, 2019 at 10:43 AM Kevin Vasko kvasko@gmail.com wrote:
I'm 100% positive I did nothing with this cert.
To validate, I spun up a brand new machine completely from scratch.
- ran yum update
- installed Gnome
- installed ipa with my normal "sudo ipa-client-install
--domain=exaple.com --realm=EXAMPLE.COM --enable-dns-updates --mkhomedir" 4. started Gnome with "startx" 5. Went to URL with Firefox, firefox errored with the "SEC_ERROR_REVOKED_CERTIFICATE" 6. installed chrome 7. went to same URL with Chrome, chrome works.
LSB Version: :core-4.1-amd64:core-4.1-noarch Distributor ID: CentOS Description: CentOS Linux release 7.7.1908 (Core) Release: 7.7.1908 Codename: Core Linux testmachine.example.com 3.10.0-1062.1.2.el7.x86_64 #1 SMP Mon Sep 30 14:19:46 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux firefox.x86_64 60.9.0-1.el7.centos @updates ipa-client.x86_64 4.6.5-11.el7.centos @base ipa-client-common.noarch 4.6.5-11.el7.centos @base ipa-common.noarch 4.6.5-11.el7.centos @base chrome-gnome-shell.x86_64 10.1-4.el7 @base google-chrome-stable.x86_64 77.0.3865.120-1 @google-chrome
How can I validate the certificate?
On Fri, Oct 11, 2019 at 12:11 AM Alexander Bokovoy abokovoy@redhat.com wrote:
On to, 10 loka 2019, Kevin Vasko wrote:
So I went back and read, reread, studied what you wrote and I think I’m following you. I’m really unfamiliar with certs and the tools around it so forgive the ignorance.
So what I ended up doing is spinning up a CentOS7 VM and installing everything on it, adding it to the FreeIPA realm etc. and followed your instructions/email.
I ran the
modutil -dbdir sql:./mozilla/firefox/9zd63dro.default/ -list
It returns the list of the PKCS #11 Modules like I listed in my previous email. However, it only showed a single item “NSS Internal PKCS #11 Module”.
To look at what keys it had I ran
certutil -d sql:./mozilla/firefox/9zd63dro.default/ -h “NSS Internal PKCS #11” -L
This seemed like it returned all of the system wide certs. Including my self signed internal lan cert from freeipa. Should it have? That’s where I’m getting confused with your comment in your email when you mentioned the p11-kit-proxy and where it’s coming from, how it was added (if needed) as you said it was providing all of the system wide certs?
At this point this is where things took a detour and I think it’s part of my confusion, which I think is unrelated, but I was using Firefox, all of the certs are there in the system based on the commands you showed. However, every time i would visit my http server Firefox would throw a
SEC_ERROR_REVOKED_CERTIFICATE
I pulled my hair out for 2 hours, deleting the .mozilla folder, recreating it, looking at certs, trying to manually copy certs into the cert db etc.
Until I got fed up and tried Chrome...i downloaded chrome installed it ran it, checked the certs db looked at the certs and verified my internal cert was listed just like firefox. I visited the http server in chrome and it worked perfectly. No changes, which I believe is what you would expect.
I then went and tried the same thing on Ubuntu. I know you mentioned that I have to add the certs manually as Ubuntu doesn’t have the same functionality. So I just manually added my ipa.crt to firefox and then got a
SEC_ERROR_REVOKED_CERTIFICATE
installed chrome on ubuntu machine and manually imported the ipa.crt into chrome, went to the http and chrome worked fine.
So now I have no idea where I’m getting this
SEC_ERROR_REVOKED_CERTIFICATE
So now on a freeipa realm joined host. It seems that
CentOS7 - Firefox gets a - SEC_ERROR_REVOKED_CERTIFICATE Chrome - Works out of the box
Ubuntu 18.04 - Firefox gets after manually adding cert- SEC_ERROR_REVOKED_CERTIFICATE Chrome - works after manually adding the ipa.ca cert through GUI.
Is there some obvious reason why firefox would throw that error message but Chrome wouldn’t?
This stuff is making my head spin.
For that host certificate Firefox thinks it is revoked by its issuer. Did you fiddle with the certificates? Perhaps, it would be easier to find out what certificate is that and check its status in IPA or whoever issued it?
-- / Alexander Bokovoy Sr. Principal Software Engineer Security / Identity Management Engineering Red Hat Limited, Finland
-- / Alexander Bokovoy Sr. Principal Software Engineer Security / Identity Management Engineering Red Hat Limited, Finland
On ma, 14 loka 2019, Kevin Vasko wrote:
Welp, I'm an idiot and you are completely 100% correct.
It was indeed revoked, but the http servers certificate was revoked and not the client..which is where I was focusing 100% of my debugging. Which clears up a LOT of things. I originally was loading the ca.crt on an Ubuntu machine a few days prior to this and it was working completely fine. After a few days I was getting the "SEC_ERROR_REVOKED_CERTIFICATE" when I went back to try it again.
However, what doesn't make sense to me is all of the commands I was running to check the certs were telling me that the certs were 100% okay and not revoked...
I ran this command which is supposedly supposed to tell me if my cert is okay with OCSP
openssl ocsp -issuer /etc/ipa/ca.crt -cert /etc/ipa/ca.crt -text -url http://ipa-ca.exmple.com/ca/ocsp -header "HOST" "ipa.exmple.com"
I was getting a
-----END CERTIFICATE----- Response verify OK /etc/ipa/ca.crt: good
And there was nothing in the result saying that it was expired on my client machines.
CA certificate is not revoked, service certificate is. So you are verifying status of a wrong certificate in the command above.
Can you maybe describe the appropriate way to debug this in the future? I was obviously doing it incorrectly. Which CA logs are you meaning? Are you meaning on the freeIPA servers? Are you meaning the http service itself? Where are you meaning "present in OCSP"? The key to this was my seeing the certificates for the http/service not showing up in the FreeIPA server UI. Once I recreated the http/service certificate the Firefox error went away.
Since I don't know what your setup is (are you using integrated CA or you are trying to use some external CA?), I was trying to give a generic answer that would be valid in both cases.
There is no need to revoke IPA services certificates in the course of normal action. So I guess you did that by your explicit act.
FreeIPA CA (Dogtag) is automatically maintaining its OCSP responder. This means when you revoke a certificate, it is added to OCSP at next synchronization point in time. After that 'openssl ocsp' command would be able to see it is revoked. However, you need to test the right certificate -- instead of passing '-cert /etc/ipa/ca.crt', you need to pass the cert you want to test for revokation.
On Mon, Oct 14, 2019 at 05:50:47PM +0300, Alexander Bokovoy via FreeIPA-users wrote:
On ma, 14 loka 2019, Kevin Vasko wrote:
Welp, I'm an idiot and you are completely 100% correct.
It was indeed revoked, but the http servers certificate was revoked and not the client..which is where I was focusing 100% of my debugging. Which clears up a LOT of things. I originally was loading the ca.crt on an Ubuntu machine a few days prior to this and it was working completely fine. After a few days I was getting the "SEC_ERROR_REVOKED_CERTIFICATE" when I went back to try it again.
However, what doesn't make sense to me is all of the commands I was running to check the certs were telling me that the certs were 100% okay and not revoked...
I ran this command which is supposedly supposed to tell me if my cert is okay with OCSP
openssl ocsp -issuer /etc/ipa/ca.crt -cert /etc/ipa/ca.crt -text -url http://ipa-ca.exmple.com/ca/ocsp -header "HOST" "ipa.exmple.com"
I was getting a
-----END CERTIFICATE----- Response verify OK /etc/ipa/ca.crt: good
And there was nothing in the result saying that it was expired on my client machines.
CA certificate is not revoked, service certificate is. So you are verifying status of a wrong certificate in the command above.
Can you maybe describe the appropriate way to debug this in the future? I was obviously doing it incorrectly. Which CA logs are you meaning? Are you meaning on the freeIPA servers? Are you meaning the http service itself? Where are you meaning "present in OCSP"? The key to this was my seeing the certificates for the http/service not showing up in the FreeIPA server UI. Once I recreated the http/service certificate the Firefox error went away.
Since I don't know what your setup is (are you using integrated CA or you are trying to use some external CA?), I was trying to give a generic answer that would be valid in both cases.
There is no need to revoke IPA services certificates in the course of normal action. So I guess you did that by your explicit act.
FreeIPA CA (Dogtag) is automatically maintaining its OCSP responder. This means when you revoke a certificate, it is added to OCSP at next synchronization point in time.
For clarification: under default configuration OCSP responses will immediately show that cert is revoked. CRL updates happen on a schedule (every 15 minutes by default).
There is a mode where OCSP reads from CRL cache, but this is not the default configuration.
After that 'openssl ocsp' command would be able to see it is revoked. However, you need to test the right certificate -- instead of passing '-cert /etc/ipa/ca.crt', you need to pass the cert you want to test for revokation.
Well that’s the thing, I didn’t realize the service certificate was revoked as I thought the entire point of validating the client cert was to validate the entire “chain” with OCSP.
Im using IPAs internal cert system.
Yeah, I kept reissueing tickets when I was trying to get the post command script to work. I guess in the process I deleted one to many certs and didn’t realize it.
So if I would have ran the command on the services cert I should have seen it’s revoked?
Is there a command to do exactly what FF is doing for OCSP to validate the cert? Or should I just manually check each cert, client and service?
Thanks again for all of the help. I think at this point I’m wrapping my head around it at least.
Now to go mess with Ubuntu to get Firefox to read those system certs since now I know CentOS 100% works...
-Kevin
On Oct 14, 2019, at 9:50 AM, Alexander Bokovoy abokovoy@redhat.com wrote:
On ma, 14 loka 2019, Kevin Vasko wrote:
Welp, I'm an idiot and you are completely 100% correct.
It was indeed revoked, but the http servers certificate was revoked and not the client..which is where I was focusing 100% of my debugging. Which clears up a LOT of things. I originally was loading the ca.crt on an Ubuntu machine a few days prior to this and it was working completely fine. After a few days I was getting the "SEC_ERROR_REVOKED_CERTIFICATE" when I went back to try it again.
However, what doesn't make sense to me is all of the commands I was running to check the certs were telling me that the certs were 100% okay and not revoked...
I ran this command which is supposedly supposed to tell me if my cert is okay with OCSP
openssl ocsp -issuer /etc/ipa/ca.crt -cert /etc/ipa/ca.crt -text -url http://ipa-ca.exmple.com/ca/ocsp -header "HOST" "ipa.exmple.com"
I was getting a
-----END CERTIFICATE----- Response verify OK /etc/ipa/ca.crt: good
And there was nothing in the result saying that it was expired on my client machines.
CA certificate is not revoked, service certificate is. So you are verifying status of a wrong certificate in the command above.
Can you maybe describe the appropriate way to debug this in the future? I was obviously doing it incorrectly. Which CA logs are you meaning? Are you meaning on the freeIPA servers? Are you meaning the http service itself? Where are you meaning "present in OCSP"? The key to this was my seeing the certificates for the http/service not showing up in the FreeIPA server UI. Once I recreated the http/service certificate the Firefox error went away.
Since I don't know what your setup is (are you using integrated CA or you are trying to use some external CA?), I was trying to give a generic answer that would be valid in both cases.
There is no need to revoke IPA services certificates in the course of normal action. So I guess you did that by your explicit act.
FreeIPA CA (Dogtag) is automatically maintaining its OCSP responder. This means when you revoke a certificate, it is added to OCSP at next synchronization point in time. After that 'openssl ocsp' command would be able to see it is revoked. However, you need to test the right certificate -- instead of passing '-cert /etc/ipa/ca.crt', you need to pass the cert you want to test for revokation.
-- / Alexander Bokovoy Sr. Principal Software Engineer Security / Identity Management Engineering Red Hat Limited, Finland
On ti, 15 loka 2019, Kevin Vasko via FreeIPA-users wrote:
Well that’s the thing, I didn’t realize the service certificate was revoked as I thought the entire point of validating the client cert was to validate the entire “chain” with OCSP.
Im using IPAs internal cert system.
Yeah, I kept reissueing tickets when I was trying to get the post command script to work. I guess in the process I deleted one to many certs and didn’t realize it.
So if I would have ran the command on the services cert I should have seen it’s revoked?
Is there a command to do exactly what FF is doing for OCSP to validate the cert? Or should I just manually check each cert, client and service?
Regarding 'exactly what FF is doing for OCSP to validate', this is hard. Might be it is using the same functionality that is exposed by nss tools in the /usr/lib64/nss/unsupported-tools/vfyserv (in nss-tools package in Fedora/CentOS/RHEL)? Because NSS library handles it for Firefox.
freeipa-users@lists.fedorahosted.org
-
Alexander Bokovoy -
Fraser Tweedale -
Kevin Vasko