Hello the list,
We just had a bit of fuss involved user logins. We're using sssd 1.16.1 on a client and FreeIPA 4.5.4 (ok, it's really RHIdM)
We had a lot of users having issues logging and/or resetting their passwords on a host with 2FA enabled, and it turns out when they're using an advanced SSH client (e.g. MobaXterm) that also starts a SFTP session they can't login and we see error like:
Sep 11 00:09:05 lander sshd[27408]: pam_sss(sshd:auth): received for user testuser: 4 (System error)
Sep 11 00:09:06 lander sshd[27380]: error: PAM: Authentication failure for testuser from remote.local
If the SFTP file browser is disabled, or it's protocol is set to use SCP then logins progress normally.
In FreeIPA we've enabled 2FA on a per-host basis and the HBAC rule only allows sshd services, so if these were the cause of the '4 (System error)' failures then it'd be much better if the error reports were more meaningful.
Does anyone have any advice on setting up SFTP so that it works (and ideally, doesn't need repeated entry of credentials).
Regards,
Aaron
On Tue, 11 Sep 2018, Aaron Hicks via FreeIPA-users wrote:
Hello the list,
We just had a bit of fuss involved user logins. We're using sssd 1.16.1 on a client and FreeIPA 4.5.4 (ok, it's really RHIdM)
We had a lot of users having issues logging and/or resetting their passwords on a host with 2FA enabled, and it turns out when they're using an advanced SSH client (e.g. MobaXterm) that also starts a SFTP session they can't login and we see error like:
Sep 11 00:09:05 lander sshd[27408]: pam_sss(sshd:auth): received for user testuser: 4 (System error)
Sep 11 00:09:06 lander sshd[27380]: error: PAM: Authentication failure for testuser from remote.local
If the SFTP file browser is disabled, or it's protocol is set to use SCP then logins progress normally.
In FreeIPA we've enabled 2FA on a per-host basis and the HBAC rule only allows sshd services, so if these were the cause of the '4 (System error)' failures then it'd be much better if the error reports were more meaningful.
Does anyone have any advice on setting up SFTP so that it works (and ideally, doesn't need repeated entry of credentials).
Can you check into sssd domain logs (after setting debug_level=9 for a domain) what exactly happened there for such a session?
Alexander Bokovoy via FreeIPA-users wrote:
On Tue, 11 Sep 2018, Aaron Hicks via FreeIPA-users wrote:
Hello the list,
We just had a bit of fuss involved user logins. We're using sssd 1.16.1 on a client and FreeIPA 4.5.4 (ok, it's really RHIdM)
We had a lot of users having issues logging and/or resetting their passwords on a host with 2FA enabled, and it turns out when they're using an advanced SSH client (e.g. MobaXterm) that also starts a SFTP session they can't login and we see error like:
Sep 11 00:09:05 lander sshd[27408]: pam_sss(sshd:auth): received for user testuser: 4 (System error)
Sep 11 00:09:06 lander sshd[27380]: error: PAM: Authentication failure for testuser from remote.local
If the SFTP file browser is disabled, or it's protocol is set to use SCP then logins progress normally.
In FreeIPA we've enabled 2FA on a per-host basis and the HBAC rule only allows sshd services, so if these were the cause of the '4 (System error)' failures then it'd be much better if the error reports were more meaningful.
Does anyone have any advice on setting up SFTP so that it works (and ideally, doesn't need repeated entry of credentials).
Can you check into sssd domain logs (after setting debug_level=9 for a domain) what exactly happened there for such a session?
Sure seems like an hbac issue to me. You can allow the sftp service as well to see if that alleviates the issue.
To change the message you'd want to file a bug against sssd.
rob
Ah, sftp is a subsystem within sshd, so it does not and can not have it's own HBAC rule, it uses any rule that authorises sshd.
-----Original Message----- From: Rob Crittenden rcritten@redhat.com Sent: Wednesday, 12 September 2018 12:07 AM To: FreeIPA users list freeipa-users@lists.fedorahosted.org Cc: Aaron Hicks aaron.hicks@nesi.org.nz; Alexander Bokovoy abokovoy@redhat.com Subject: Re: [Freeipa-users] Re: sftp file broswer causes 4 (System Error)
Alexander Bokovoy via FreeIPA-users wrote:
On Tue, 11 Sep 2018, Aaron Hicks via FreeIPA-users wrote:
Hello the list,
We just had a bit of fuss involved user logins. We're using sssd 1.16.1 on a client and FreeIPA 4.5.4 (ok, it's really RHIdM)
We had a lot of users having issues logging and/or resetting their passwords on a host with 2FA enabled, and it turns out when they're using an advanced SSH client (e.g. MobaXterm) that also starts a SFTP session they can't login and we see error like:
Sep 11 00:09:05 lander sshd[27408]: pam_sss(sshd:auth): received for user testuser: 4 (System error)
Sep 11 00:09:06 lander sshd[27380]: error: PAM: Authentication failure for testuser from remote.local
If the SFTP file browser is disabled, or it's protocol is set to use SCP then logins progress normally.
In FreeIPA we've enabled 2FA on a per-host basis and the HBAC rule only allows sshd services, so if these were the cause of the '4 (System error)' failures then it'd be much better if the error reports were more meaningful.
Does anyone have any advice on setting up SFTP so that it works (and ideally, doesn't need repeated entry of credentials).
Can you check into sssd domain logs (after setting debug_level=9 for a domain) what exactly happened there for such a session?
Sure seems like an hbac issue to me. You can allow the sftp service as well to see if that alleviates the issue.
To change the message you'd want to file a bug against sssd.
rob
On Tue, 2018-09-11 at 14:10 +1200, Aaron Hicks via FreeIPA-users wrote:
Hello the list,
We just had a bit of fuss involved user logins. We're using sssd 1.16.1 on a client and FreeIPA 4.5.4 (ok, it's really RHIdM)
We had a lot of users having issues logging and/or resetting their passwords on a host with 2FA enabled, and it turns out when they're using an advanced SSH client (e.g. MobaXterm) that also starts a SFTP session they can't login and we see error like:
Sep 11 00:09:05 lander sshd[27408]: pam_sss(sshd:auth): received for user testuser: 4 (System error)
Sep 11 00:09:06 lander sshd[27380]: error: PAM: Authentication failure for testuser from remote.local
If the SFTP file browser is disabled, or it's protocol is set to use SCP then logins progress normally.
In FreeIPA we've enabled 2FA on a per-host basis and the HBAC rule only allows sshd services, so if these were the cause of the '4 (System error)' failures then it'd be much better if the error reports were more meaningful.
Does anyone have any advice on setting up SFTP so that it works (and ideally, doesn't need repeated entry of credentials).
You should find out if your client supports using a master connection for SSH, instead of trying to open multiple different connection for SSH and SFTP. In the end it is a client issue if it can't properly prompt for credentials when it uses multiple different authenticated connections (I assume this client is caching passwords and trying to resubmit old 2FA codes in the process ? [Caching of password seem already bad in itself if that's the case, how long does it hold onto your creds? will it leak them?])
HTH, Simo.
Hi Simo,
Yes, we recognise this as a client side issue. This was as much a FYI post for people in the future searching for similar issues to latch onto. I've also made similar comments back to the developers of the MobaXterm client we observed this with. We now ask our users to switch the file browser protocol to SCP which I think uses the master connection method you've recommended.
Regards,
Aaron
-----Original Message----- From: Simo Sorce simo@redhat.com Sent: Thursday, 13 September 2018 4:20 AM To: FreeIPA users list freeipa-users@lists.fedorahosted.org Cc: Aaron Hicks aaron.hicks@nesi.org.nz Subject: Re: [Freeipa-users] sftp file broswer causes 4 (System Error)
On Tue, 2018-09-11 at 14:10 +1200, Aaron Hicks via FreeIPA-users wrote:
Hello the list,
We just had a bit of fuss involved user logins. We're using sssd 1.16.1 on a client and FreeIPA 4.5.4 (ok, it's really RHIdM)
We had a lot of users having issues logging and/or resetting their passwords on a host with 2FA enabled, and it turns out when they're using an advanced SSH client (e.g. MobaXterm) that also starts a SFTP session they can't login and we see error like:
Sep 11 00:09:05 lander sshd[27408]: pam_sss(sshd:auth): received for user testuser: 4 (System error)
Sep 11 00:09:06 lander sshd[27380]: error: PAM: Authentication failure for testuser from remote.local
If the SFTP file browser is disabled, or it's protocol is set to use SCP then logins progress normally.
In FreeIPA we've enabled 2FA on a per-host basis and the HBAC rule only allows sshd services, so if these were the cause of the '4 (System error)' failures then it'd be much better if the error reports were more meaningful.
Does anyone have any advice on setting up SFTP so that it works (and ideally, doesn't need repeated entry of credentials).
You should find out if your client supports using a master connection for SSH, instead of trying to open multiple different connection for SSH and SFTP. In the end it is a client issue if it can't properly prompt for credentials when it uses multiple different authenticated connections (I assume this client is caching passwords and trying to resubmit old 2FA codes in the process ? [Caching of password seem already bad in itself if that's the case, how long does it hold onto your creds? will it leak them?])
HTH, Simo.
-- Simo Sorce Sr. Principal Software Engineer Red Hat, Inc
Hi all. I wonder who and how this is been resolved? I have centos 7 where an sftp server is running. Authentication is with freeIPA 4.5.4. all the users connect to the sftp server normally but when there are multiple connections randomly I got this error
Nov 7 08:30:09 sftp sshd[23487]: pam_sss(sshd:account): Access denied for user nifi_sftp: 4 (System error)
Not sure why. The same user doesn't have any issue connecting manually but when different connections from 3 nodes (running a open source sftp client called NIFI from apache.org) I got that error. I have to say that I tried to reproduce with a script running multiple connections at the same time and I get the same errors. If I use controlmaster mechanism on ssh client I dont' get the error at all.
Any idea? cheers
On Mon, Sep 17, 2018 at 3:43 AM Aaron Hicks via FreeIPA-users < freeipa-users@lists.fedorahosted.org> wrote:
Hi Simo,
Yes, we recognise this as a client side issue. This was as much a FYI post for people in the future searching for similar issues to latch onto. I've also made similar comments back to the developers of the MobaXterm client we observed this with. We now ask our users to switch the file browser protocol to SCP which I think uses the master connection method you've recommended.
Regards,
Aaron
-----Original Message----- From: Simo Sorce simo@redhat.com Sent: Thursday, 13 September 2018 4:20 AM To: FreeIPA users list freeipa-users@lists.fedorahosted.org Cc: Aaron Hicks aaron.hicks@nesi.org.nz Subject: Re: [Freeipa-users] sftp file broswer causes 4 (System Error)
On Tue, 2018-09-11 at 14:10 +1200, Aaron Hicks via FreeIPA-users wrote:
Hello the list,
We just had a bit of fuss involved user logins. We're using sssd 1.16.1 on a client and FreeIPA 4.5.4 (ok, it's really RHIdM)
We had a lot of users having issues logging and/or resetting their passwords on a host with 2FA enabled, and it turns out when they're using an advanced SSH client (e.g. MobaXterm) that also starts a SFTP session they can't login and we see error like:
Sep 11 00:09:05 lander sshd[27408]: pam_sss(sshd:auth): received for user testuser: 4 (System error)
Sep 11 00:09:06 lander sshd[27380]: error: PAM: Authentication failure for testuser from remote.local
If the SFTP file browser is disabled, or it's protocol is set to use SCP then logins progress normally.
In FreeIPA we've enabled 2FA on a per-host basis and the HBAC rule only allows sshd services, so if these were the cause of the '4 (System
error)'
failures then it'd be much better if the error reports were more
meaningful.
Does anyone have any advice on setting up SFTP so that it works (and ideally, doesn't need repeated entry of credentials).
You should find out if your client supports using a master connection for SSH, instead of trying to open multiple different connection for SSH and SFTP. In the end it is a client issue if it can't properly prompt for credentials when it uses multiple different authenticated connections (I assume this client is caching passwords and trying to resubmit old 2FA codes in the process ? [Caching of password seem already bad in itself if that's the case, how long does it hold onto your creds? will it leak them?])
HTH, Simo.
-- Simo Sorce Sr. Principal Software Engineer Red Hat, Inc
FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org To unsubscribe send an email to freeipa-users-leave@lists.fedorahosted.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedorahosted.org/archives/list/freeipa-users@lists.fedorahoste...
On ke, 07 marras 2018, Alfredo De Luca via FreeIPA-users wrote:
Hi all. I wonder who and how this is been resolved? I have centos 7 where an sftp server is running. Authentication is with freeIPA 4.5.4. all the users connect to the sftp server normally but when there are multiple connections randomly I got this error
Nov 7 08:30:09 sftp sshd[23487]: pam_sss(sshd:account): Access denied for user nifi_sftp: 4 (System error)
Not sure why. The same user doesn't have any issue connecting manually but when different connections from 3 nodes (running a open source sftp client called NIFI from apache.org) I got that error. I have to say that I tried to reproduce with a script running multiple connections at the same time and I get the same errors. If I use controlmaster mechanism on ssh client I dont' get the error at all.
Any idea?
Use sssd debugging to demonstrate why pam_sss is denying access. https://docs.pagure.org/SSSD.sssd/users/troubleshooting.html
You'd need logs from the sssd_<domain>.log and sssd_pam.log related to the time when there is an attempt to connect with NIFI. Use debug_level=9 in domain and pam sections to show all logs and provide them somewhere we can look up.
Hi alexander. Thanks for your info. Here are 2 logs. One is the pam.log and the other one is the domain.log at the time when we got the error below.
Nov 8 17:09:06 sftp-test sshd[25100]: pam_sss(sshd:account): Access denied for user nifi_sftp: 4 (System error)
The user to search is nifi_sftp.
Thanks heaps and let me know if you need more info Cheers
On Wed, Nov 7, 2018 at 3:49 PM Alexander Bokovoy abokovoy@redhat.com wrote:
On ke, 07 marras 2018, Alfredo De Luca via FreeIPA-users wrote:
Hi all. I wonder who and how this is been resolved? I have centos 7 where an sftp server is running. Authentication is with freeIPA 4.5.4. all the users connect to the sftp server normally but when there are multiple connections randomly I got this error
Nov 7 08:30:09 sftp sshd[23487]: pam_sss(sshd:account): Access denied for user nifi_sftp: 4 (System error)
Not sure why. The same user doesn't have any issue connecting manually but when different connections from 3 nodes (running a open source sftp client called NIFI from apache.org) I got that error. I have to say that I tried to reproduce with a script running multiple connections at the same time and I get the same errors. If I use controlmaster mechanism on ssh client I dont' get the error at all.
Any idea?
Use sssd debugging to demonstrate why pam_sss is denying access. https://docs.pagure.org/SSSD.sssd/users/troubleshooting.html
You'd need logs from the sssd_<domain>.log and sssd_pam.log related to the time when there is an attempt to connect with NIFI. Use debug_level=9 in domain and pam sections to show all logs and provide them somewhere we can look up.
-- / Alexander Bokovoy Sr. Principal Software Engineer Security / Identity Management Engineering Red Hat Limited, Finland
On to, 08 marras 2018, Alfredo De Luca via FreeIPA-users wrote:
Hi alexander. Thanks for your info. Here are 2 logs. One is the pam.log and the other one is the domain.log at the time when we got the error below.
Nov 8 17:09:06 sftp-test sshd[25100]: pam_sss(sshd:account): Access denied for user nifi_sftp: 4 (System error)
The user to search is nifi_sftp.
Thanks heaps and let me know if you need more info
Do you have SELinux enabled? Disabled?
From the looks of sssd_<domain>.log you have trouble with setting SELinux for the user:
Thu Nov 8 17:09:06 2018) [sssd[be[novalocal]]] [selinux_child_done] (0x0020): selinux_child_parse_response failed: [22][Invalid argument]
This means that most likely you have SELinux disabled completely yet SSSD attempts to set up SELinux context and considers its failure a hard fail.
Setting
selinux_provider = none
in [domain/novalocal] section should help if you are not using SELinux.
Cheers
On Wed, Nov 7, 2018 at 3:49 PM Alexander Bokovoy abokovoy@redhat.com wrote:
On ke, 07 marras 2018, Alfredo De Luca via FreeIPA-users wrote:
Hi all. I wonder who and how this is been resolved? I have centos 7 where an sftp server is running. Authentication is with freeIPA 4.5.4. all the users connect to the sftp server normally but when there are multiple connections randomly I got this error
Nov 7 08:30:09 sftp sshd[23487]: pam_sss(sshd:account): Access denied for user nifi_sftp: 4 (System error)
Not sure why. The same user doesn't have any issue connecting manually but when different connections from 3 nodes (running a open source sftp client called NIFI from apache.org) I got that error. I have to say that I tried to reproduce with a script running multiple connections at the same time and I get the same errors. If I use controlmaster mechanism on ssh client I dont' get the error at all.
Any idea?
Use sssd debugging to demonstrate why pam_sss is denying access. https://docs.pagure.org/SSSD.sssd/users/troubleshooting.html
You'd need logs from the sssd_<domain>.log and sssd_pam.log related to the time when there is an attempt to connect with NIFI. Use debug_level=9 in domain and pam sections to show all logs and provide them somewhere we can look up.
-- / Alexander Bokovoy Sr. Principal Software Engineer Security / Identity Management Engineering Red Hat Limited, Finland
-- *Alfredo*
thanks Alexander. We don't have selinux enabled so good point from you. I will implement the solution you suggested soon and let you know. Thanks heaps
Alfredo
On Thu, Nov 8, 2018 at 9:05 PM Alexander Bokovoy abokovoy@redhat.com wrote:
On to, 08 marras 2018, Alfredo De Luca via FreeIPA-users wrote:
Hi alexander. Thanks for your info. Here are 2 logs. One is the pam.log and the other one is the domain.log at the time when we got the error below.
Nov 8 17:09:06 sftp-test sshd[25100]: pam_sss(sshd:account): Access
denied
for user nifi_sftp: 4 (System error)
The user to search is nifi_sftp.
Thanks heaps and let me know if you need more info
Do you have SELinux enabled? Disabled?
From the looks of sssd_<domain>.log you have trouble with setting SELinux for the user:
Thu Nov 8 17:09:06 2018) [sssd[be[novalocal]]] [selinux_child_done] (0x0020): selinux_child_parse_response failed: [22][Invalid argument]
This means that most likely you have SELinux disabled completely yet SSSD attempts to set up SELinux context and considers its failure a hard fail.
Setting
selinux_provider = none
in [domain/novalocal] section should help if you are not using SELinux.
Cheers
On Wed, Nov 7, 2018 at 3:49 PM Alexander Bokovoy abokovoy@redhat.com wrote:
On ke, 07 marras 2018, Alfredo De Luca via FreeIPA-users wrote:
Hi all. I wonder who and how this is been resolved? I have centos 7 where an sftp server is running. Authentication is with freeIPA 4.5.4. all the users connect to the sftp server normally but when there are multiple connections randomly I got this error
Nov 7 08:30:09 sftp sshd[23487]: pam_sss(sshd:account): Access denied
for
user nifi_sftp: 4 (System error)
Not sure why. The same user doesn't have any issue connecting manually
but
when different connections from 3 nodes (running a open source sftp
client
called NIFI from apache.org) I got that error. I have to say that I tried to reproduce with a script running multiple connections at the same time and I get the same errors. If I use controlmaster mechanism on ssh client I dont' get the error at all.
Any idea?
Use sssd debugging to demonstrate why pam_sss is denying access. https://docs.pagure.org/SSSD.sssd/users/troubleshooting.html
You'd need logs from the sssd_<domain>.log and sssd_pam.log related to the time when there is an attempt to connect with NIFI. Use debug_level=9 in domain and pam sections to show all logs and provide them somewhere we can look up.
-- / Alexander Bokovoy Sr. Principal Software Engineer Security / Identity Management Engineering Red Hat Limited, Finland
-- *Alfredo*
-- / Alexander Bokovoy Sr. Principal Software Engineer Security / Identity Management Engineering Red Hat Limited, Finland
Hi Alexander. Spot on... we fixed the issue with your suggestion. Thanks heaps Appreciated.
regards
On Fri, Nov 9, 2018 at 12:43 PM Alfredo De Luca alfredo.deluca@gmail.com wrote:
thanks Alexander. We don't have selinux enabled so good point from you. I will implement the solution you suggested soon and let you know. Thanks heaps
Alfredo
On Thu, Nov 8, 2018 at 9:05 PM Alexander Bokovoy abokovoy@redhat.com wrote:
On to, 08 marras 2018, Alfredo De Luca via FreeIPA-users wrote:
Hi alexander. Thanks for your info. Here are 2 logs. One is the pam.log and the other one is the domain.log
at
the time when we got the error below.
Nov 8 17:09:06 sftp-test sshd[25100]: pam_sss(sshd:account): Access
denied
for user nifi_sftp: 4 (System error)
The user to search is nifi_sftp.
Thanks heaps and let me know if you need more info
Do you have SELinux enabled? Disabled?
From the looks of sssd_<domain>.log you have trouble with setting SELinux for the user:
Thu Nov 8 17:09:06 2018) [sssd[be[novalocal]]] [selinux_child_done] (0x0020): selinux_child_parse_response failed: [22][Invalid argument]
This means that most likely you have SELinux disabled completely yet SSSD attempts to set up SELinux context and considers its failure a hard fail.
Setting
selinux_provider = none
in [domain/novalocal] section should help if you are not using SELinux.
Cheers
On Wed, Nov 7, 2018 at 3:49 PM Alexander Bokovoy abokovoy@redhat.com wrote:
On ke, 07 marras 2018, Alfredo De Luca via FreeIPA-users wrote:
Hi all. I wonder who and how this is been resolved? I have centos 7 where an sftp server is running. Authentication is
with
freeIPA 4.5.4. all the users connect to the sftp server normally but when there are multiple connections randomly I got this error
Nov 7 08:30:09 sftp sshd[23487]: pam_sss(sshd:account): Access
denied for
user nifi_sftp: 4 (System error)
Not sure why. The same user doesn't have any issue connecting
manually but
when different connections from 3 nodes (running a open source sftp
client
called NIFI from apache.org) I got that error. I have to say that I tried to reproduce with a script running multiple connections at the same time and I get the same errors. If I use controlmaster mechanism on ssh client I dont' get the error at all.
Any idea?
Use sssd debugging to demonstrate why pam_sss is denying access. https://docs.pagure.org/SSSD.sssd/users/troubleshooting.html
You'd need logs from the sssd_<domain>.log and sssd_pam.log related to the time when there is an attempt to connect with NIFI. Use debug_level=9 in domain and pam sections to show all logs and provide them somewhere we can look up.
-- / Alexander Bokovoy Sr. Principal Software Engineer Security / Identity Management Engineering Red Hat Limited, Finland
-- *Alfredo*
-- / Alexander Bokovoy Sr. Principal Software Engineer Security / Identity Management Engineering Red Hat Limited, Finland
-- *Alfredo*
freeipa-users@lists.fedorahosted.org
-
Aaron Hicks -
Alexander Bokovoy -
Alfredo De Luca -
Rob Crittenden -
Simo Sorce