After upgrading to sys-auth/pambase-20201010 sys-libs/pam-1.4.0_p20200829 I cannot login via ssh + pw. ssh + keys still works, which is currently the only way I have access to this server. Log: Oct 13 08:29:24 xxx sshd[7408]: pam_krb5(sshd:auth): user xxx authenticated as xxx@XXX.DE Oct 13 08:29:24 xxx sshd[7408]: pam_faillock(sshd:auth): Unknown option: conf Oct 13 08:29:24 xxx sshd[7408]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=x.x.x.x user=xxx Oct 13 08:29:25 xxx sshd[7391]: error: PAM: Permission denied for xxx from x.x.x.x 'Unknown option: conf' is reported as an own minor bug #748228, but is not relevant here. The auth-failure occures on this line: auth sufficient pam_unix.so nullok try_first_pass Most probably in /etc/pam.d/system-auth but /etc/pam.d/system-login has the same line after including system-auth. As I understand 'sufficient' should not require the line to succeed, if the authentication was already established (in krb5 in this case). So why does this fail? Reproducible: Always ================================================================= Package Settings ================================================================= sys-libs/pam-1.4.0_p20200829::gentoo was built with the following: USE="berkdb filecaps nis pie (split-usr) -audit -debug (-selinux)" ABI_X86="(64) -32 (-x32)" CFLAGS="-O2 -pipe" CXXFLAGS="-O2 -pipe" FEATURES="assume-digests binpkg-docompress binpkg-dostrip binpkg-logs binpkg-multi-instance buildpkg config-protect-if-modified distlocks ebuild-locks fixlafiles ipc-sandbox merge-sync multilib-strict network-sandbox news parallel-fetch pid-sandbox preserve-libs protect-owned qa-unresolved-soname-deps sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync xattr" sys-auth/pambase-20201010::gentoo was built with the following: USE="nullok pam_krb5 passwdqc sha512 -caps -debug -elogind -gnome-keyring -minimal -mktemp -pam_ssh -pwhistory -pwquality -securetty (-selinux) -systemd" ABI_X86="(64)" CFLAGS="-O2 -pipe" CXXFLAGS="-O2 -pipe" FEATURES="assume-digests binpkg-docompress binpkg-dostrip binpkg-logs binpkg-multi-instance buildpkg config-protect-if-modified distlocks ebuild-locks fixlafiles ipc-sandbox merge-sync multilib-strict network-sandbox news parallel-fetch pid-sandbox preserve-libs protect-owned qa-unresolved-soname-deps sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync xattr"
workaround: comment out #auth [default=die] pam_faillock.so authfail Other than that this seems to be related to https://bugs.gentoo.org/747868 and other pambase-20201010 tickets.
Thanks, that works for now.
(In reply to Manuel Mommertz from comment #2) > Thanks, that works for now. A cleaner approach would be to use <sys-auth/pambase minimal> if you can do without the add'l pam modules until pambase-20201010 is fixed.
*** This bug has been marked as a duplicate of bug 748294 ***
Sorry to say, but while bug #748294 is fixed, now, this is NOT a duplicate of it. The line mentioned in comment #1 is still in /etc/pam.d/system-auth and needs to be commented to allow a successful login. :( Oct 14 06:44:36 xxx sshd[26747]: pam_krb5(sshd:auth): user xxx authenticated as xxx@XXX.DE Oct 14 06:44:36 xxx sshd[26747]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=x.x.x.x user=xxx Oct 14 06:44:38 xxx sshd[26740]: error: PAM: Permission denied for xxx from x.x.x.x
I tend to agree with that. pam_krb5(su:auth): pam_sm_authenticate: exit (success) pam_unix(su:auth): authentication failure; [...] pam_authenticate: Permission denied FAILED su for X by Y Same goes for tty login: pam_krb5(login:auth): pam_sm_authenticate: exit (success) pam_unix(login:auth): authentication failure; [...] FAILED LOGIN (1) on '/dev/tty2' FOR 'Y', Permission denied When deploying with USE=-minimal authentication succeeds.
pambase-20202013: # diff system-auth system-auth.useminimal 5,7d4 < auth required pam_faillock.so preauth < auth sufficient pam_unix.so nullok try_first_pass < auth [default=die] pam_faillock.so authfail 11d7 < account required pam_faillock.so # diff system-login system-login.useminimal 7d6 < account required pam_faillock.so 11d9 < session optional pam_lastlog.so silent 13,14d10 < session optional pam_motd.so motd=/etc/motd < session optional pam_mail.so Unless the logic in pambase might require a closer look the culprit might be pam_faillock.so, which is part of pam...
After digging through pam man-pages I conclude that the logic of authentication handling in gentoo has to change, if we want to support faillock. with sys-auth/pambase-20200304 the auth-part of /etc/pam.d/system-auth looked like this (with krb5): auth required pam_env.so auth [success=1 default=ignore] pam_krb5.so ignore_root try_first_pass auth required pam_unix.so try_first_pass likeauth nullok auth optional pam_permit.so if krb5 verified credentials successful, success=1 jumps over the following line (pam_unix) directly to pam_permit. pam_permit always succeeds, so the execution reaches end-of-file with a success. if krb5 reports failure, because of default=ignore, the failure is not stored and execution continues with pam_unix.so if pam_unix.so succeeds, the result is the same as when pam_krb5 succeeds. if pam_unix.so fails, 'required' stores the failure but continues with the next line. The success of pam_permit is not stored, as optional means, it is only stored if there was no failure before. So in all cases, the execution continues to the end of the file. With sys-auth/pambase-20201013 the following lines are appended to the file: auth required pam_faillock.so preauth auth sufficient pam_unix.so nullok try_first_pass auth [default=die] pam_faillock.so authfail Which now checks AGAIN for the credentials, but only with pam_unix. I would suggest to replace all this with: auth required pam_faillock.so preauth auth required pam_env.so auth sufficient pam_krb5.so ignore_root try_first_pass auth sufficient pam_unix.so try_first_pass likeauth nullok auth [default=die] pam_faillock.so authfail Which stops execution as soon as correct credentials are found. reaching the last line only when the login failed. This drops the useless pam_permit.so. If you insist on keeping it, this would be a solution: auth required pam_faillock.so preauth auth required pam_env.so auth sufficient pam_krb5.so ignore_root try_first_pass auth [success=done new_authtok_reqd=done default=bad] pam_unix.so try_first_pass likeauth nullok auth sufficient pam_permit.so auth [default=die] pam_faillock.so authfail
And with respect to bug #747793, other files need to use 'substacks' instead of 'include'.
To add to it: Apparently this also clears the record of failed attempts without having to call faillock.so again with authsucc. I haven't found anything on this in the man pages, thoguh.
(In reply to SacredRide from comment #10) > To add to it: > > Apparently this also clears the record of failed attempts without having to > call faillock.so again with authsucc. I haven't found anything on this in > the man pages, thoguh. It actually is mentioned in the man-page of pam_faillock (but not very prominent): Due to complications in the way the PAM stack can be configured it is also possible to call pam_faillock as an account module. So the this line surfs the same purpose as authsucc: account required pam_faillock.so
(In reply to Manuel Mommertz from comment #5) > Sorry to say, but while bug #748294 is fixed, now, this is NOT a duplicate > of it. The line mentioned in comment #1 is still in /etc/pam.d/system-auth > and needs to be commented to allow a successful login. :( > > Oct 14 06:44:36 xxx sshd[26747]: pam_krb5(sshd:auth): user xxx authenticated > as xxx@XXX.DE > Oct 14 06:44:36 xxx sshd[26747]: pam_unix(sshd:auth): authentication > failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=x.x.x.x user=xxx > Oct 14 06:44:38 xxx sshd[26740]: error: PAM: Permission denied for xxx from > x.x.x.x Although would be nice if you provided more detailed explanation to avoid a confusion:)
Looks like faillock must be made no-op module for now, it is the shortest way.
(In reply to Mikle Kolyada from comment #12) > Although would be nice if you provided more detailed explanation to avoid a > confusion:) Hard to provide, without knowing all this pam-stuff which I only slowly begin to understand, now. But I mentioned, that it was not the same as bug #748228, as the other bug. If you have any suggestions how I can improve the next time, let me know. :) (In reply to Mikle Kolyada from comment #13) > Looks like faillock must be made no-op module for now, it is the shortest way. It would be the shortest way, to remove the faillock support, yes. And then take some time to implement it correctly. Supporting all the use-flag combinations right is hard stuff, obviously. Thanks for doing this. :)
(In reply to Manuel Mommertz from comment #14) > (In reply to Mikle Kolyada from comment #12) > > Although would be nice if you provided more detailed explanation to avoid a > > confusion:) > > Hard to provide, without knowing all this pam-stuff which I only slowly > begin to understand, now. But I mentioned, that it was not the same as bug > #748228, as the other bug. If you have any suggestions how I can improve the > next time, let me know. :) > > (In reply to Mikle Kolyada from comment #13) > > Looks like faillock must be made no-op module for now, it is the shortest way. > It would be the shortest way, to remove the faillock support, yes. And then > take some time to implement it correctly. Supporting all the use-flag > combinations right is hard stuff, obviously. Thanks for doing this. :) both answers are no: 1.) no it is not hard if you read journalctl -f and /var/log/messages (for the future references to ease the debug for you) 2.) no, faillock is fine here
(In reply to Mikle Kolyada from comment #15) > 1.) no it is not hard if you read journalctl -f and /var/log/messages (for > the future references to ease the debug for you) I don't use systemd so journalctl is not available. The output of /var/log/messages was provided in my initial post. I mentioned that the 'Unknown option' was not relevant for this bug and pointed to the pam-config-line which caused the auth-failure. What more do you need from a bug-reporter? To be fair, I just see that I forgot to provide the relevant part of /etc/pam.d/system-auth. It is now available in comment #8, but it should have been given right away. > 2.) no, faillock is fine here Then what do you mean by making it 'no-op'? Setting the control-field to [default=ignore]? This would not help in any way. faillock authfail has to be run conditionally only when login does not succeed. And as I don't see a way to run a module conditionally depending on the previous success, you have to make the condition on the module that might succeed. So either jumping out with [success=done] or jumping over with [success=N]. Where the latter is a bit more complex as you have to get the 'N' right and keep it right when possibly changing config in the future. Therefor I would prefer [success=done].
(In reply to Mikle Kolyada from comment #13) > Looks like faillock must be made no-op module for now, it is the shortest > way. I only now realize, that this would indeed work... faillock would always increase its counter, even if the login is successful. But as it is set to [default=ignore] it cannot stop the login. And then with successful login the counter is reset. So no side effects. Apologize for not getting it sooner. :)
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/proj/pambase.git/commit/?id=eb138196aa2d3cb860d5eb5ab1d05985df34ad2c commit eb138196aa2d3cb860d5eb5ab1d05985df34ad2c Author: Sam James <sam@gentoo.org> AuthorDate: 2020-10-20 02:32:28 +0000 Commit: Sam James <sam@gentoo.org> CommitDate: 2020-10-20 02:38:20 +0000 templates/system-auth.tpl: use faillock in minimal case Bug: https://bugs.gentoo.org/748405 Signed-off-by: Sam James <sam@gentoo.org> templates/system-auth.tpl | 7 ++----- 1 file changed, 2 insertions(+), 5 deletions(-)
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=00a599149912784b80e9fa5925c122f3a193aede commit 00a599149912784b80e9fa5925c122f3a193aede Author: Sam James <sam@gentoo.org> AuthorDate: 2020-10-20 02:40:38 +0000 Commit: Sam James <sam@gentoo.org> CommitDate: 2020-10-20 02:40:43 +0000 sys-auth/pambase: add 20201020 Includes faillock in minimal case. Bug: https://bugs.gentoo.org/748405 Package-Manager: Portage-3.0.8, Repoman-3.0.1 Signed-off-by: Sam James <sam@gentoo.org> sys-auth/pambase/Manifest | 1 + sys-auth/pambase/pambase-20201020.ebuild | 99 ++++++++++++++++++++++++++++++++ 2 files changed, 100 insertions(+)
Still not working with pambase-20201020. (reduced)system-auth now looks like this: auth required pam_env.so auth [success=1 default=ignore] pam_krb5.so debug ignore_root try_first_pass auth optional pam_permit.so auth requisite pam_faillock.so preauth auth [success=1 default=ignore] pam_unix.so nullok try_first_pass auth [default=die] pam_faillock.so authfail I don't even have an idea what this should accomplish. If pam_krb5.so succeeds it stores success and jumps over pam_permit.so. If pam_krb5.so fails the fail is ignored and pam_permit.so succeeds and stores success. So in both cases it reaches pam_unix.so with a success stored. No difference whether the credentials for kerberos where verified or not. Then pam_unix.so is run and that decides if the login will succeed (jump over next line) or not (run next line with [default=die], so login fails).
Created attachment 668624 [details, diff] fix login with kerberos I created a patch which fixes the login problems for me. Additionally I added some comments. The comment regarding pam_permit.so was found in the template of pambase-20200304. I think it is very important for understanding and should not get lost.
(In reply to Manuel Mommertz from comment #20) pambase is broken for quite a while, at least since the publication the news item about migration to pam-1.4 (I don't remember is that was broken before). After each update (which always produces a configuration which would lock users out of the box) I have to manually review the update and then delete the suggested files. I imagine how many administrators have broken their setups by blindly accepting the changes. > So in both cases it reaches pam_unix.so with a success stored. No difference > whether the credentials for kerberos where verified or not. There is also a third case, in the account section: pam_krb5 would result in 'ignore' (because the previous auth was successful but not password-based). Then the control is passed to pam_unix which, if there is no corresponding passwd/shadow entry (because the user is served by and stored in Kerberos), will fail with PAM_NEW_AUTHTOK_REQD triggering password change logic each time (and not allowing the user in because after password change a new login is required, at least for sshd). Additionally to breakage of pambase, the migration brought breakage to pam documentation (pam man pages, except those for pam modules are no more installed) which complicates debug for people like me who do not store pam API details in memory.
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/proj/pambase.git/commit/?id=99919c4b2b59af27e7ad1daa6fbe8c614a8463c0 commit 99919c4b2b59af27e7ad1daa6fbe8c614a8463c0 Author: Sam James <sam@gentoo.org> AuthorDate: 2020-10-26 08:32:29 +0000 Commit: Sam James <sam@gentoo.org> CommitDate: 2020-10-26 22:48:06 +0000 templates/system-auth.tpl: skip pam_unix with krb5 Before this change, success on pam_krb5 would result in jumping one line (over pam_permit) back into pam_unix. Incidentally, we did the later stanza correctly. This was a regression from old pambase. Bug: https://bugs.gentoo.org/748405 Signed-off-by: Sam James <sam@gentoo.org> templates/system-auth.tpl | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-)
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=72038665ef97d89a5e96a7752e4d147b75f0a06c commit 72038665ef97d89a5e96a7752e4d147b75f0a06c Author: Sam James <sam@gentoo.org> AuthorDate: 2020-10-27 00:16:18 +0000 Commit: Sam James <sam@gentoo.org> CommitDate: 2020-10-27 00:17:07 +0000 sys-auth/pambase: bump to 20201026 Sam James (3): templates/system-login.tpl: always need faillock templates/system-auth.tpl: skip pam_unix with krb5 templates/system-auth.tpl: fix libcap module name Bug: https://bugs.gentoo.org/750524 Bug: https://bugs.gentoo.org/748405 Package-Manager: Portage-3.0.8, Repoman-3.0.2 Signed-off-by: Sam James <sam@gentoo.org> sys-auth/pambase/Manifest | 1 + sys-auth/pambase/pambase-20201026.ebuild | 99 ++++++++++++++++++++++++++++++++ 2 files changed, 100 insertions(+)
Thanks for your patience, all. We are getting there. Please let us know if this doesn't fix your problem.
(In reply to Larry the Git Cow from comment #23) > Incidentally, we did the later stanza correctly. This was a regression > from old pambase. There is still a subtle problem, as explained in comment #22. For the ok/fail return values, the code looks ok. But for the ignore return value from pam_krb5 module in the account section, the path is to the next module in the stack which is pam_unix. The logic inside pam_unix is a kind of (knowingly) broken in case when there is no user entry in passwd/shadow: the module will return PAM_NEW_AUTHTOK_REQD which will cause the service to request to change the password (and not allow to log in). Tomorrow I will have a chance to verify this theory on a real box. I already hit this some day in the past but would like to be sure that I am theorizing correctly. In my configuration PAM+Kerberos + sshd with all three: PAM, GSS-API and PKI enabled I am using no_pass_expiry with pam_unix in account section. For the more general case I guess this needs to be rethinked, I am not sure of all possible implications of this hack.
(In reply to Alexander Bezrukov from comment #26) > For the ok/fail return values, the code looks ok. But for the ignore return > value from pam_krb5 module in the account section, the path is to the next > module in the stack which is pam_unix. The logic inside pam_unix is a kind > of (knowingly) broken in case when there is no user entry in passwd/shadow: > the module will return PAM_NEW_AUTHTOK_REQD which will cause the service to > request to change the password (and not allow to log in). Yeah, we were talking about this. I'm not sure (right now) what the best case is when pam_krb5 fails. Relying on some entry in shadow isn't good. We _could_ duplicate the logic in the template (to not just rely on pam primitives with jumping) but I don't love that idea. > [snip]
(In reply to Sam James from comment #27) > Yeah, we were talking about this. I'm not sure (right now) what the best > case is when pam_krb5 fails. Relying on some entry in shadow isn't good. Failing is another case, also not very straightforward: what if a client laptop is used standalone in a foreign (to its Kerberos domain) network: there would be no response from the KDC and pam_krb5 will time out. It is ok but each local user authentication would take a considerable amount of time. Please also take in consideration that for the root user pam_krb5 WILL (and should) fail in many setups: either because of ignore_root or because of minimum_uid. Maybe pam_unix should go before pam_krb5? My custom solution is that pam_krb5 occurs before pam_unix on stationery boxes and vice versa on mobile client computers which often travel to foreign networks. What I am writing about (also in comment #22) is when pam_krb5's pam_acct_mgmt() returns 'ignore' because the authentication was successful but not password-based. This is how this would look in the logs (I redacted sensitive data): krb5kdc[4506]: TGS_REQ (1 etypes {aes256-cts-hmac-sha1-96(18)}) x.x.x.x: ISSUE: authtime 1603275380, etypes {rep=aes256-cts-hmac-sha1-96(18), tkt=aes256-cts-hmac-sha1-96(18), ses=aes256-cts-hmac-sha1-96(18)}, <principal> for <TGT service> sshd[28269]: Authorized to <user>, krb5 principal <principal> (krb5_kuserok) sshd[28269]: pam_krb5(sshd:account): pam_sm_acct_mgmt: entry sshd[28269]: pam_krb5(sshd:account): skipping non-Kerberos login sshd[28269]: pam_krb5(sshd:account): pam_sm_acct_mgmt: exit (ignore) And if pam_unix (without the no_pass_expiry option) follows, the following would appear in the logs: sshd[28269]: pam_unix(sshd:account): expired password for user <user> (password aged) sshd[28269]: Accepted gssapi-with-mic for <user> from x.x.x.x port 51844 ssh2: <principal> sshd[28269]: pam_unix(sshd:session): session opened for user <user>(uid=<uid>) by (uid=0) passwd[28274]: pam_unix(passwd:chauthtok): user "<user>" does not exist in /etc/passwd sshd[28273]: Received disconnect from x.x.x.x port 51844:11: disconnected by user sshd[28273]: Disconnected from user <user> x.x.x.x port 51844 sshd[28269]: pam_unix(sshd:session): session closed for user <user> This is an excerpt from a very old auth.log. I cannot be 100% sure about actual pam.d contents at that time. Tomorrow I will try to reproduce this theory on a real non-production box with pam.d from the most recent pambase.
Thanks Sam for working on this. :) Positive news: * Password-based login with kerberos works * Password-based login via passwd works * Wrong password fails and (says it) locks user after 3 failed attempts But... Line 2 of my system-auth looks like this now: auth [success=4 default=ignore] pam_krb5.so debug ignore_root try_first_pass Counting lines after, success should be 3, not 4, to jump to pam_permit.so. If kerberos-login succeeds it jumps over faillock, so you can still login after 3 failed attempts. And because it jumps over the faillock in account-section, too, the counter is not reseted on a successfull login. I think faillock should work for kerberos the same as for normal unix login. Apart from that, some notes: * As the pam config is quite complex I highly recommend to add comments, like in the patch I provided. * Why does faillock occur in system-login? The file includes system-auth which handles faillock. I don't see a reason to have it occure in system-login again. Regarding the comments from Alexander: Missing documentation is tracked in bug 747796. For return value 'ignore' from krb5, I have no experience with it and therefore cannot comment on this.
(In reply to Alexander Bezrukov from comment #28) > Tomorrow I will try to reproduce this > theory on a real non-production box with pam.d from the most recent pambase. I corrected the off-by-one error Manuel Mommertz wrote about in comment 29 and confirmed my theory which I wrote about in comment 22, comment 26 and comment 28. If authentication was passwordless (e.g. based on public key) and the user is not in passwd/shadow files, then the password changing logic was triggered by pam_unix returning PAM_NEW_AUTHTOK_REQD and the user is never allowed in.
Created attachment 669725 [details, diff] respect faillock with kerberos From my side the login works with pambase-20201028.1. However I apply the attached patch to have faillock protect kerberos-logins, too.
(In reply to Manuel Mommertz from comment #31) > Created attachment 669725 [details, diff] [details, diff] > respect faillock with kerberos > > From my side the login works with pambase-20201028.1. However I apply the > attached patch to have faillock protect kerberos-logins, too. I did not wrap faillock into kerberos on purpose. because kadmin has its own restrictions for an account. Otherwise everything has been fixed I see.
