Hi, on CentOS 7 I installed Freeipa using "yum install ipa-server". Everything including client is on the same machine itself. All went well, I can now login to the web as "admin" and create user account etc. And "kinit admin", "kinit list" etc all worked as expected right after installation.
But a couple days later, even though I can still login as "admin" web user, on the server ssh session I get the following (I replaced REALM name with "REALM" here):
# kinit kinit: Client 'root@REALM' not found in Kerberos database while getting initial credentials
# kinit admin kinit: Pre-authentication failed: Invalid argument while getting initial credentials
# kinit list kinit: Client 'list@REALM' not found in Kerberos database while getting initial credentials
# env KRB5_TRACE=/dev/stdout kinit admin 2>&1 [11612] 1578511115.54729: Getting initial credentials for admin@REALM [11612] 1578511115.54731: Sending unauthenticated request [11612] 1578511115.54732: Sending request (167 bytes) to REALM [11612] 1578511115.54733: Initiating TCP connection to stream 127.0.0.1:88 [11612] 1578511115.54734: Sending TCP request to stream 127.0.0.1:88 [11612] 1578511115.54735: Received answer (240 bytes) from stream 127.0.0.1:88 [11612] 1578511115.54736: Terminating TCP connection to stream 127.0.0.1:88 [11612] 1578511115.54737: Response was from master KDC [11612] 1578511115.54738: Received error from KDC: -1765328359/Additional pre-authentication required [11612] 1578511115.54741: Preauthenticating using KDC method data [11612] 1578511115.54742: Processing preauth types: PA-PK-AS-REQ (16), PA-PK-AS-REP_OLD (15), PA-PK-AS-REQ_OLD (14), PA-FX-FAST (136), PA-PKINIT-KX (147), PA-FX-COOKIE (133) [11612] 1578511115.54743: Received cookie: MIT [11612] 1578511115.54744: PKINIT client has no configured identity; giving up [11612] 1578511115.54745: Preauth module pkinit (147) (info) returned: 0/Success [11612] 1578511115.54746: PKINIT client has no configured identity; giving up [11612] 1578511115.54747: Preauth module pkinit (16) (real) returned: 22/Invalid argument [11612] 1578511115.54748: PKINIT client has no configured identity; giving up [11612] 1578511115.54749: Preauth module pkinit (14) (real) returned: 22/Invalid argument kinit: Pre-authentication failed: Invalid argument while getting initial credentials
# klist -ek Keytab name: FILE:/etc/krb5.keytab KVNO Principal ---- -------------------------------------------------------------------------- 2 host/ipa.host.name@REALM (aes256-cts-hmac-sha1-96) 2 host/ipa.host.name@REALM (aes128-cts-hmac-sha1-96) 2 host/ipa.host.name@REALM (des3-cbc-sha1) 2 host/ipa.host.name@REALM (arcfour-hmac) 2 host/ipa.host.name@REALM (camellia128-cts-cmac) 2 host/ipa.host.name@REALM (camellia256-cts-cmac)
So looks like I lost "admin" in kerboros?
The only thing I think I did, is I have changed the server's time and hwclock time, by 9 minutes.
Thanks!
John Louis via FreeIPA-users freeipa-users@lists.fedorahosted.org writes:
Hi, on CentOS 7 I installed Freeipa using "yum install ipa-server". Everything including client is on the same machine itself. All went well, I can now login to the web as "admin" and create user account etc. And "kinit admin", "kinit list" etc all worked as expected right after installation.
But a couple days later, even though I can still login as "admin" web user, on the server ssh session I get the following (I replaced REALM name with "REALM" here):
# kinit kinit: Client 'root@REALM' not found in Kerberos database while getting initial credentials
# kinit admin kinit: Pre-authentication failed: Invalid argument while getting initial credentials
# kinit list kinit: Client 'list@REALM' not found in Kerberos database while getting initial credentials
Any logs from the KDC?
Thanks, --Robbie
Thanks so much.
/var/log/krb5kdc.log only contain the following few kind of lines, not necessarily in chronological order, and they repeated many times, so I just copied one line for each kind, but keep in mind each of them repeated many times:
krb5kdc: Invalid message type - while dispatching (udp) Jan 07 02:01:33 krb5kdc[2121](info): preauth (encrypted_timestamp) verify failure: Preauthentication failed Jan 07 02:01:33 krb5kdc[2121](info): AS_REQ (8 etypes {18 17 20 19 16 23 25 26}) 127.0.0.1: PREAUTH_FAILED: host/ipa.host.name@REALM for krbtgt/REALM@REALM, Preauthentication failed Jan 02 20:47:04 krb5kdc[2120](info): AS_REQ (8 etypes {18 17 20 19 16 23 25 26}) 127.0.0.1: CLIENT_NOT_FOUND: list@REALM for krbtgt/REALM@REALM, Client not found in Kerberos database Jan 02 20:47:33 krb5kdc[2120](info): AS_REQ (8 etypes {18 17 20 19 16 23 25 26}) 127.0.0.1: CLIENT_NOT_FOUND: root@REALM for krbtgt/REALM@REALM, Client not found in Kerberos database Jan 06 04:16:55 krb5kdc[2121](Error): TCP client 1.3.5.17.56660 wants 1195725856 bytes, cap is 1048572 Jan 07 01:31:42 krb5kdc[2120](info): AS_REQ (8 etypes {18 17 20 19 16 23 25 26}) 127.0.0.1: NEEDED_PREAUTH: admin@REALM for krbtgt/REALM@REALM, Additional pre-authentication required Jan 08 04:23:49 krb5kdc[2121](info): AS_REQ (8 etypes {18 17 16 23 25 26 20 19}) 127.0.0.1: NEEDED_PREAUTH: host/ipa.host.name@REALM for krbtgt/REALM@REALM, Additional pre-authentication required Jan 08 09:08:49 krb5kdc[2121](info): AS_REQ (8 etypes {18 17 16 23 25 26 20 19}) 127.0.0.1: ISSUE: authtime 1578492529, etypes {rep=18 tkt=18 ses=18}, host/ipa.host.name@REALM for krbtgt/REALM@REALM Jan 08 09:08:49 krb5kdc[2120](info): TGS_REQ (8 etypes {18 17 20 19 16 23 25 26}) 127.0.0.1: ISSUE: authtime 1578492529, etypes {rep=18 tkt=18 ses=18}, host/ipa.host.name@REALM for ldap/ipa.host.name@REALM
John Louis via FreeIPA-users freeipa-users@lists.fedorahosted.org writes:
Thanks so much.
/var/log/krb5kdc.log only contain the following few kind of lines, not necessarily in chronological order, and they repeated many times, so I just copied one line for each kind, but keep in mind each of them repeated many times:
It would have been more helpful if you had posted the logfile on a pastebin somewhere.
krb5kdc: Invalid message type - while dispatching (udp)
You said this was all on one machine, right? Is it perhaps network exposed?
Jan 07 02:01:33 krb5kdc[2121](info): preauth (encrypted_timestamp) verify failure: Preauthentication failed Jan 07 02:01:33 krb5kdc[2121](info): AS_REQ (8 etypes {18 17 20 19 16 23 25 26}) 127.0.0.1: PREAUTH_FAILED: host/ipa.host.name@REALM for krbtgt/REALM@REALM, Preauthentication failed Jan 02 20:47:04 krb5kdc[2120](info): AS_REQ (8 etypes {18 17 20 19 16 23 25 26}) 127.0.0.1: CLIENT_NOT_FOUND: list@REALM for krbtgt/REALM@REALM, Client not found in Kerberos database Jan 02 20:47:33 krb5kdc[2120](info): AS_REQ (8 etypes {18 17 20 19 16 23 25 26}) 127.0.0.1: CLIENT_NOT_FOUND: root@REALM for krbtgt/REALM@REALM, Client not found in Kerberos database Jan 06 04:16:55 krb5kdc[2121](Error): TCP client 1.3.5.17.56660 wants 1195725856 bytes, cap is 1048572
That's an enormous request...
Jan 07 01:31:42 krb5kdc[2120](info): AS_REQ (8 etypes {18 17 20 19 16 23 25 26}) 127.0.0.1: NEEDED_PREAUTH: admin@REALM for krbtgt/REALM@REALM, Additional pre-authentication required Jan 08 04:23:49 krb5kdc[2121](info): AS_REQ (8 etypes {18 17 16 23 25 26 20 19}) 127.0.0.1: NEEDED_PREAUTH: host/ipa.host.name@REALM for krbtgt/REALM@REALM, Additional pre-authentication required Jan 08 09:08:49 krb5kdc[2121](info): AS_REQ (8 etypes {18 17 16 23 25 26 20 19}) 127.0.0.1: ISSUE: authtime 1578492529, etypes {rep=18 tkt=18 ses=18}, host/ipa.host.name@REALM for krbtgt/REALM@REALM Jan 08 09:08:49 krb5kdc[2120](info): TGS_REQ (8 etypes {18 17 20 19 16 23 25 26}) 127.0.0.1: ISSUE: authtime 1578492529, etypes {rep=18 tkt=18 ses=18}, host/ipa.host.name@REALM for ldap/ipa.host.name@REALM
I'd check the kvno on all principals against what's in their keytabs. If that's not illuminating, we may need to look for data problems in LDAP (which hopefully someone else can explain).
Thanks, --Robbie
Thanks. Yea all this is installed on one server. I just blocked udp access on the firewall per your suggestion.
I have pasted the entire log at
Lines like "TCP client 1.3.5.17.56660 wants 1195725856 bytes, cap is 1048572" are removed above.
Also there is a size limit on that pastebin website, so I just pasted recent lines to fit into that limit.
I'd check the kvno on all principals against what's in their keytabs.
How do I do that? Sorry I read some kerberos documentation but don't quite understand. I did:
# kvno ldap/ipa.example.com@REALM kvno: Credentials cache keyring 'persistent:0:0' not found while getting client principal name
What else I can do to resolve this problem?
I can still login as "admin" to the web portable, and even to create users there. But this kerberos problem is a timed bomb, and I can't do some server work which requires it.
Thanks for your help!
John Louis via FreeIPA-users freeipa-users@lists.fedorahosted.org writes:
Thanks. Yea all this is installed on one server. I just blocked udp access on the firewall per your suggestion.
I have pasted the entire log at
Lines like "TCP client 1.3.5.17.56660 wants 1195725856 bytes, cap is 1048572" are removed above.
Also there is a size limit on that pastebin website, so I just pasted recent lines to fit into that limit.
I'd check the kvno on all principals against what's in their keytabs.
How do I do that?
List the keytab in question; e.g., `klist -ekt /path/to/keytab`, then kinit, and run the kvno command as you did.
Thanks, --Robbie
Thanks. Here they are:
# klist -ekt /etc/krb5.keytab Keytab name: FILE:/etc/krb5.keytab KVNO Timestamp Principal ---- ------------------- ------------------------------------------------------ 2 12/28/2019 21:23:47 host/ipa.example.com@REALM (aes256-cts-hmac-sha1-96) 2 12/28/2019 21:23:47 host/ipa.example.com@REALM (aes128-cts-hmac-sha1-96) 2 12/28/2019 21:23:47 host/ipa.example.com@REALM (des3-cbc-sha1) 2 12/28/2019 21:23:47 host/ipa.example.com@REALM (arcfour-hmac) 2 12/28/2019 21:23:47 host/ipa.example.com@REALM (camellia128-cts-cmac) 2 12/28/2019 21:23:47 host/ipa.example.com@REALM (camellia256-cts-cmac) [root@se log]# kinit kinit: Client 'root@REALM' not found in Kerberos database while getting initial credentials # kvno ldap/ipa.example.com@REALM kvno: Credentials cache keyring 'persistent:0:0' not found while getting client principal name
John Louis via FreeIPA-users wrote:
Thanks. Here they are:
# klist -ekt /etc/krb5.keytab Keytab name: FILE:/etc/krb5.keytab KVNO Timestamp Principal
2 12/28/2019 21:23:47 host/ipa.example.com@REALM (aes256-cts-hmac-sha1-96) 2 12/28/2019 21:23:47 host/ipa.example.com@REALM (aes128-cts-hmac-sha1-96) 2 12/28/2019 21:23:47 host/ipa.example.com@REALM (des3-cbc-sha1) 2 12/28/2019 21:23:47 host/ipa.example.com@REALM (arcfour-hmac) 2 12/28/2019 21:23:47 host/ipa.example.com@REALM (camellia128-cts-cmac) 2 12/28/2019 21:23:47 host/ipa.example.com@REALM (camellia256-cts-cmac) [root@se log]# kinit kinit: Client 'root@REALM' not found in Kerberos database while getting initial credentials # kvno ldap/ipa.example.com@REALM kvno: Credentials cache keyring 'persistent:0:0' not found while getting client principal name
Note that a raw kinit will use the user you are currently logged in as (root). To kinit to the admin user specify it on the cli:
# kinit admin # kvno host/ipa.example.com
You used ldap which is a different service than host.
rob
Thanks for the explanation. Here they are;
# kinit admin kinit: Pre-authentication failed: Invalid argument while getting initial credentials # kvno host/ipa.example.com kvno: Credentials cache keyring 'persistent:0:0' not found while getting client principal name
John Louis via FreeIPA-users freeipa-users@lists.fedorahosted.org writes:
Thanks for the explanation. Here they are;
# kinit admin kinit: Pre-authentication failed: Invalid argument while getting initial credentials
Show with KRB5_TRACE output please. (KRB5_TRACE=/dev/stderr kinit admin)
Thanks, --Robbie
Thanks. These are very similar to what was provided in the beginning. Here is exactly what you asked:
# KRB5_TRACE=/dev/stderr kinit admin [1567] 1579125111.129826: Getting initial credentials for admin@REALM [1567] 1579125111.129828: Sending unauthenticated request [1567] 1579125111.129829: Sending request (167 bytes) to REALM [1567] 1579125111.129830: Initiating TCP connection to stream 127.0.0.1:88 [1567] 1579125111.129831: Terminating TCP connection to stream 127.0.0.1:88 [1567] 1579125111.129832: Sending initial UDP request to dgram 127.0.0.1:88 kinit: Cannot contact any KDC for realm 'REALM' while getting initial credentials
John Louis via FreeIPA-users freeipa-users@lists.fedorahosted.org writes:
Thanks. These are very similar to what was provided in the beginning. Here is exactly what you asked:
# KRB5_TRACE=/dev/stderr kinit admin [1567] 1579125111.129826: Getting initial credentials for admin@REALM [1567] 1579125111.129828: Sending unauthenticated request [1567] 1579125111.129829: Sending request (167 bytes) to REALM [1567] 1579125111.129830: Initiating TCP connection to stream 127.0.0.1:88 [1567] 1579125111.129831: Terminating TCP connection to stream 127.0.0.1:88 [1567] 1579125111.129832: Sending initial UDP request to dgram 127.0.0.1:88 kinit: Cannot contact any KDC for realm 'REALM' while getting initial credentials
It looks like your KDC isn't running. Can you check why it's not?
Thanks, --Robbie
On to, 16 tammi 2020, Robbie Harwood via FreeIPA-users wrote:
John Louis via FreeIPA-users freeipa-users@lists.fedorahosted.org writes:
Thanks. These are very similar to what was provided in the beginning. Here is exactly what you asked:
# KRB5_TRACE=/dev/stderr kinit admin [1567] 1579125111.129826: Getting initial credentials for admin@REALM [1567] 1579125111.129828: Sending unauthenticated request [1567] 1579125111.129829: Sending request (167 bytes) to REALM [1567] 1579125111.129830: Initiating TCP connection to stream 127.0.0.1:88 [1567] 1579125111.129831: Terminating TCP connection to stream 127.0.0.1:88 [1567] 1579125111.129832: Sending initial UDP request to dgram 127.0.0.1:88 kinit: Cannot contact any KDC for realm 'REALM' while getting initial credentials
It looks like your KDC isn't running. Can you check why it's not?
Also, why is it 127.0.0.1? FreeIPA deployment does not really set it up this way, so what did you do after installation?
sorry it's actually not literally "127.0.0.1", but the actual IP. In all my replies above, I just replaced that actual IP with "127.0.0.1". I hope this won't confuse you.
"It looks like your KDC isn't running. Can you check why it's not?"
How do I do that? Here is what I can think of:
# systemctl status krb5kdc.service ● krb5kdc.service - Kerberos 5 KDC Loaded: loaded (/usr/lib/systemd/system/krb5kdc.service; disabled; vendor preset: disabled) Active: active (running) since Thu 2020-01-16 14:05:28 EST; 23s ago Process: 1611 ExecStart=/usr/sbin/krb5kdc -P /var/run/krb5kdc.pid $KRB5KDC_ARGS (code=exited, status=0/SUCCESS) Main PID: 1612 (krb5kdc) CGroup: /system.slice/krb5kdc.service ├─1612 /usr/sbin/krb5kdc -P /var/run/krb5kdc.pid -w 2 ├─1613 /usr/sbin/krb5kdc -P /var/run/krb5kdc.pid -w 2 └─1614 /usr/sbin/krb5kdc -P /var/run/krb5kdc.pid -w 2
Jan 16 14:05:27 se.lixing.us systemd[1]: Starting Kerberos 5 KDC... Jan 16 14:05:28 se.lixing.us systemd[1]: Started Kerberos 5 KDC.
# systemctl status kadmin.service ● kadmin.service - Kerberos 5 Password-changing and Administration Loaded: loaded (/usr/lib/systemd/system/kadmin.service; disabled; vendor preset: disabled) Active: active (running) since Thu 2020-01-16 14:05:28 EST; 4min 11s ago Process: 1617 ExecStart=/usr/sbin/_kadmind -P /var/run/kadmind.pid $KADMIND_ARGS (code=exited, status=0/SUCCESS) Main PID: 1618 (kadmind) CGroup: /system.slice/kadmin.service └─1618 /usr/sbin/kadmind -P /var/run/kadmind.pid
Jan 16 14:05:28 se.lixing.us systemd[1]: Starting Kerberos 5 Password-changing and Administration... Jan 16 14:05:28 se.lixing.us systemd[1]: Started Kerberos 5 Password-changing and Administration.
# lsof -i -P |grep :88 krb5kdc 1613 root 6u IPv4 18720 0t0 UDP *:88 krb5kdc 1613 root 7u IPv6 18721 0t0 UDP *:88 krb5kdc 1613 root 8u IPv4 18724 0t0 TCP *:88 (LISTEN) krb5kdc 1613 root 9u IPv6 18725 0t0 TCP *:88 (LISTEN) krb5kdc 1614 root 6u IPv4 18720 0t0 UDP *:88 krb5kdc 1614 root 7u IPv6 18721 0t0 UDP *:88 krb5kdc 1614 root 8u IPv4 18724 0t0 TCP *:88 (LISTEN) krb5kdc 1614 root 9u IPv6 18725 0t0 TCP *:88 (LISTEN)
So looks like it is running?
sorry looks like the output for this command is different now. I think I had rebooted it a few times these days. All other output remain the same though, such as "kinit admin" and "kvno host/REALM".
# KRB5_TRACE=/dev/stderr kinit admin 2>&1 [2294] 1579218609.798442: Getting initial credentials for admin@REALM [2294] 1579218609.798444: Sending unauthenticated request [2294] 1579218609.798445: Sending request (167 bytes) to REALM [2294] 1579218609.798446: Initiating TCP connection to stream 127.0.0.1:88 [2294] 1579218609.798447: Sending TCP request to stream 127.0.0.1:88 [2294] 1579218609.798448: Received answer (240 bytes) from stream 127.0.0.1:88 [2294] 1579218609.798449: Terminating TCP connection to stream 127.0.0.1:88 [2294] 1579218609.798450: Response was from master KDC [2294] 1579218609.798451: Received error from KDC: -1765328359/Additional pre-authentication required [2294] 1579218609.798454: Preauthenticating using KDC method data [2294] 1579218609.798455: Processing preauth types: PA-PK-AS-REQ (16), PA-PK-AS-REP_OLD (15), PA-PK-AS-REQ_OLD (14), PA-FX-FAST (136), PA-PKINIT-KX (147), PA-FX-COOKIE (133) [2294] 1579218609.798456: Received cookie: MIT [2294] 1579218609.798457: PKINIT client has no configured identity; giving up [2294] 1579218609.798458: Preauth module pkinit (147) (info) returned: 0/Success [2294] 1579218609.798459: PKINIT client has no configured identity; giving up [2294] 1579218609.798460: Preauth module pkinit (16) (real) returned: 22/Invalid argument [2294] 1579218609.798461: PKINIT client has no configured identity; giving up [2294] 1579218609.798462: Preauth module pkinit (14) (real) returned: 22/Invalid argument kinit: Pre-authentication failed: Invalid argument while getting initial credentials
On to, 16 tammi 2020, John Louis via FreeIPA-users wrote:
sorry looks like the output for this command is different now. I think I had rebooted it a few times these days. All other output remain the same though, such as "kinit admin" and "kvno host/REALM".
# KRB5_TRACE=/dev/stderr kinit admin 2>&1 [2294] 1579218609.798442: Getting initial credentials for admin@REALM [2294] 1579218609.798444: Sending unauthenticated request [2294] 1579218609.798445: Sending request (167 bytes) to REALM [2294] 1579218609.798446: Initiating TCP connection to stream 127.0.0.1:88 [2294] 1579218609.798447: Sending TCP request to stream 127.0.0.1:88 [2294] 1579218609.798448: Received answer (240 bytes) from stream 127.0.0.1:88 [2294] 1579218609.798449: Terminating TCP connection to stream 127.0.0.1:88 [2294] 1579218609.798450: Response was from master KDC [2294] 1579218609.798451: Received error from KDC: -1765328359/Additional pre-authentication required [2294] 1579218609.798454: Preauthenticating using KDC method data [2294] 1579218609.798455: Processing preauth types: PA-PK-AS-REQ (16), PA-PK-AS-REP_OLD (15), PA-PK-AS-REQ_OLD (14), PA-FX-FAST (136), PA-PKINIT-KX (147), PA-FX-COOKIE (133) [2294] 1579218609.798456: Received cookie: MIT [2294] 1579218609.798457: PKINIT client has no configured identity; giving up [2294] 1579218609.798458: Preauth module pkinit (147) (info) returned: 0/Success [2294] 1579218609.798459: PKINIT client has no configured identity; giving up [2294] 1579218609.798460: Preauth module pkinit (16) (real) returned: 22/Invalid argument [2294] 1579218609.798461: PKINIT client has no configured identity; giving up [2294] 1579218609.798462: Preauth module pkinit (14) (real) returned: 22/Invalid argument kinit: Pre-authentication failed: Invalid argument while getting initial credentials
So, according to the preauth types required for this user do not include password-based authentication. KDC tells that only PKINIT and FAST channel are accepted (and anything within the FAST channel). Can you tell a bit more about how this user is configured to be authenticated?
If you enabled OTP method on this user, then you should be using something like this:
kinit -n -c a-file kinit -T a-file admin
It is expected that OTP-enforced user authentication cannot be done at the moment with a single 'kinit' because Kerberos requires an encrypted channel first to pass the OTP details to a RADIUS server. And you need to use some other credentials to create that channel. In example above I'm using anonymous PKINIT (kinit -n) to create it.
When you login to Web UI, a server side part of the application does this for you automatically. When you login over SSH, SSSD does it for you using host credentials (host/... principal) to build the FAST channel. With 'kinit' you are on your own.
See, for example, https://access.redhat.com/solutions/3233521
Thank you so much. That perfectly explained what happened, and all my problems are resolved now.
What happened is, I went to the web interface and require admin to login with two factors. That works well on that web interface. But then I ssh'ed to the server withOUT kerberos or SSSD. Therefore got the above "kinit" problem.
After using the commands you pointed out:
kinit -n -c a-file kinit -T a-file admin
I now regained "kinit" capability even with two factors: just enter "password + OTPassword" in the second command above.
Thanks so much for all your help above!
freeipa-users@lists.fedorahosted.org
-
Alexander Bokovoy -
John Louis -
Rob Crittenden -
Robbie Harwood