After system update today of pam/pambase my auto-unlock of my keyring with lightdm and setup to allow automatic usage of stored SSH keys stopped working. I hadn't had +gnome-keyring on pambase before with my prior-to-update-working setup, but I added it and still no dice. I'm having trouble tracking down the exact changes that caused it, and the old packages are no longer in-tree to attempt selective downgrade to confirm issue. I'm happy to provide more information if you need it, and I can attempt a selective downgrade to confirm issue if the prior versions are restored in-tree. ❯ emerge --info pambase Portage 3.0.4 (python 3.7.8-final-0, default/linux/amd64/17.1/systemd, gcc-9.3.0, glibc-2.31-r6, 5.8.14-gentoo x86_64) ================================================================= System Settings ================================================================= System uname: Linux-5.8.14-gentoo-x86_64-Intel-R-_Core-TM-_i7-8750H_CPU_@_2.20GHz-with-gentoo-2.7 KiB Mem: 16295236 total, 11178648 free KiB Swap: 20971516 total, 20971516 free Timestamp of repository gentoo: Sat, 10 Oct 2020 16:00:01 +0000 Head commit of repository gentoo: 40c1d8ffaa7fb77a67d9f02cdd21c01bfaf76de4 Timestamp of repository steam-overlay: Mon, 21 Sep 2020 16:07:17 +0000 Head commit of repository steam-overlay: ae0b747be5b634a27f049c19897cbd656c38d5d1 sh bash 5.0_p18 ld GNU ld (Gentoo 2.34 p6) 2.34.0 app-shells/bash: 5.0_p18::gentoo dev-java/java-config: 2.3.1::gentoo dev-lang/perl: 5.30.3::gentoo dev-lang/python: 2.7.18-r4::gentoo, 3.7.8-r2::gentoo, 3.8.5::gentoo dev-util/cmake: 3.17.4-r1::gentoo sys-apps/baselayout: 2.7::gentoo sys-apps/sandbox: 2.18::gentoo sys-devel/autoconf: 2.13-r1::gentoo, 2.69-r5::gentoo sys-devel/automake: 1.16.1-r1::gentoo sys-devel/binutils: 2.34-r2::gentoo sys-devel/gcc: 9.3.0-r1::gentoo sys-devel/gcc-config: 2.3.1::gentoo sys-devel/libtool: 2.4.6-r6::gentoo sys-devel/make: 4.2.1-r4::gentoo sys-kernel/linux-headers: 5.8::gentoo (virtual/os-headers) sys-libs/glibc: 2.31-r6::gentoo Repositories: gentoo location: /usr/portage sync-type: rsync sync-uri: rsync://rsync.gentoo.org/gentoo-portage priority: -1000 sync-rsync-verify-jobs: 1 sync-rsync-verify-metamanifest: yes sync-rsync-verify-max-age: 24 sync-rsync-extra-opts: steam-overlay location: /var/db/repos/steam-overlay sync-type: git sync-uri: https://github.com/gentoo-mirror/steam-overlay.git masters: gentoo Installed sets: @leela-deps, @ranger-plugins ACCEPT_KEYWORDS="amd64" ACCEPT_LICENSE="* -@EULA" CBUILD="x86_64-pc-linux-gnu" CFLAGS="-march=native -O2 -pipe -Wa,-mbranches-within-32B-boundaries" CHOST="x86_64-pc-linux-gnu" CONFIG_PROTECT="/etc /usr/share/config /usr/share/gnupg/qualified.txt" CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/dconf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo" CXXFLAGS="-march=native -O2 -pipe -Wa,-mbranches-within-32B-boundaries" DISTDIR="/usr/portage/distfiles" ENV_UNSET="CARGO_HOME DBUS_SESSION_BUS_ADDRESS DISPLAY GOBIN GOPATH PERL5LIB PERL5OPT PERLPREFIX PERL_CORE PERL_MB_OPT PERL_MM_OPT XAUTHORITY XDG_CACHE_HOME XDG_CONFIG_HOME XDG_DATA_HOME XDG_RUNTIME_DIR" FCFLAGS="-march=native -O2 -pipe -Wa,-mbranches-within-32B-boundaries" FEATURES="assume-digests binpkg-docompress binpkg-dostrip binpkg-logs config-protect-if-modified distlocks ebuild-locks fixlafiles ipc-sandbox merge-sync multilib-strict network-sandbox news parallel-fetch pid-sandbox preserve-libs protect-owned qa-unresolved-soname-deps sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync xattr" FFLAGS="-march=native -O2 -pipe -Wa,-mbranches-within-32B-boundaries" GENTOO_MIRRORS="http://distfiles.gentoo.org" LANG="en_US.UTF-8" LDFLAGS="-Wl,-O1 -Wl,--as-needed" MAKEOPTS="-j10" PKGDIR="/usr/portage/packages" PORTAGE_CONFIGROOT="/" PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages --exclude=/.git" PORTAGE_TMPDIR="/var/tmp" USE="X a52 aac acl acpi alsa amd64 apparmor audit berkdb blas bluetooth bzip2 cairo cblas-external cjk cli color-management crypt cryptsetup cuda cups dbus dri dts dvb emboss encode epub eselect-ldso exif fam fdk fftw flac fortran gdbm gif glamor gnome-keyring gpm gtk gui heif iconv icu idn ipv6 jpeg lapack lcms libglvnd libnotify libtirpc mad mng mp3 mp4 mpeg mpris multilib ncurses networkmanager nls nptl nvenc nvidia ogg opencl openexr opengl openmp opus pam pango pcre pdf pkcs11 png policykit ppds pulseaudio qt5 readline sdl seccomp security-key smartcard smp spell split-usr ssl startup-notification svg systemd tcpd threads tiff truetype udev udisks unicode upower usb v4l vaapi vdpau vim-syntax vorbis vulkan webp wxwidgets x264 x265 xattr xcb xinerama xml xv xvid xvmc zlib zsh-completion zstd" ABI_X86="64" ADA_TARGET="gnat_2018" ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87x ca0106 cmipci emu10k1x ens1370 ens1371 es1938 es1968 fm801 hda-intel intel8x0 intel8x0m maestro3 trident usb-audio via82xx via82xx-modem ymfpci" APACHE2_MODULES="authn_core authz_core socache_shmcb unixd actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="karbon sheets words" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" CPU_FLAGS_X86="aes avx avx2 f16c fma3 mmx mmxext pclmul popcnt rdrand sse sse2 sse3 sse4_1 sse4_2 ssse3" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock greis isync itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf skytraq superstar2 timing tsip tripmate tnt ublox ubx" INPUT_DEVICES="libinput wacom" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php7-2 php7-3 php7-4" POSTGRES_TARGETS="postgres10 postgres11" PYTHON_SINGLE_TARGET="python3_7" PYTHON_TARGETS="python2_7 python3_7" RUBY_TARGETS="ruby25 ruby26" USERLAND="GNU" VIDEO_CARDS="intel i965 nvidia" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account" Unset: CC, CPPFLAGS, CTARGET, CXX, EMERGE_DEFAULT_OPTS, INSTALL_MASK, LC_ALL, LINGUAS, PORTAGE_BINHOST, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS ================================================================= Package Settings ================================================================= sys-auth/pambase-20200917::gentoo was built with the following: USE="gnome-keyring nullok passwdqc sha512 systemd -caps -debug (-elogind) -minimal -mktemp -pam_krb5 -pam_ssh -pwhistory -pwquality -securetty (-selinux)" ABI_X86="(64)"
please provide journalctl -f or any equivalent file containing logged information about failed attempts (pam logs what it does)
Created attachment 664549 [details] system log when keyring unlock was working
Created attachment 664552 [details] system log when it's broke
So for context, I have this in my .zshrc to allow the ssh autologin: if [ -n "$DESKTOP_SESSION" ];then eval $(gnome-keyring-daemon --start) export SSH_AUTH_SOCK fi (see e.g. arch wiki) I ran two commands: sudo journalctl -b -2 -a --no-pager | grep -m 200 -i 'lightdm\|pam\|keyring\|gcr' > working.txt sudo journalctl -b -a --no-pager | grep -m 200 -i 'lightdm\|pam\|keyring\|gcr' > broke.txt to hopefully filter out as much as possible to the changed stuff. I'm looking through them now as well. Please let me know if it's missing something you need.
There's new gcr log file stuff that wasn't there before, but I hadn't set (or needed to set) +gnome-keyring USE flag on pambase before so that could where it's coming from. I was hoping if indeed the change is coming from pam/pambase (as it was one of the few updated packages) you would have an idea what changed. I'm still trying to explore to see if there's other additional info I can find so feel free to let me know or contact me in freenode #gentoo (I'm aphysically)
Not sure it is strictly pam relevant: 1.) sys-libs/pam does not ship gnome-keyring module at all. 2.) what pambase is using is shipped by the gnome-base/gnome-keyring package 3.) there were no changes made to the stack other than making gnome-keyring an optional module for pambase (it is literally just means adding 'if' conditions, so nothing really drastic) What version of gnome keyring are you using?
I have gnome-base/gnome-keyring-3.36.0 you can see the difference fairly early in boot from the older working Oct 07 23:37:17 ares lightdm[2790]: gkr-pam: gnome-keyring-daemon started properly and unlocked keyring to the newer Oct 10 11:17:39 ares lightdm[2735]: gkr-pam: gnome-keyring-daemon started properly I'm trying right now to look through the more complete logs to try to see if there's something else relevant I can find. If you think it's worth it I can try the older package to A/B test if it's really pambase if you re-add it to tree. Although, given the change was gnome-keyring related it does seem like less of a coincidence.
What I see from the logs where pam-gcr stopped working for you is: /usr/bin/ssh-add command failed: Child process exited with code 1 Which might be relevant. Just to be sure, could you try to log-in without lightdm and then try using startx? I do not except the case where lightdm's pam stack might be involved (but unlikely it is the case), and then see logs again? Also, as far as I understand you are using gnome-keyring as a mean of ssh pass unlock right? Do not you happen to have something other than keyring for that? Say, pam_ssh?
(FWIW, I’m about to head to bed here but I agree with Mikle’s analysis right now. It’s be a hell of a coincidence but I don’t think we changed anything related.)
Created attachment 664561 [details] working log up until unlock success
Created attachment 664564 [details] fail log up until unlock failure
I hadn't thought the ssh-add command failure was especially important since it occurred after the first obvious failure deviation (i.e. it had already failed to say "and unlocked keyring". I grabbed the 500 lines of system log from everything leading up until where it first states it either does/doesn't unlock keyring in the working and non-working system log. I don't think I have it configured to unlock if I login without lightdm: my main reason for using gnome-keyring was to get the keyring to automatically unlock when I logged in with my display manager. You can find on the gentoo-wiki for lightdm where you add the appropriate pam stuff to lgihtdm in summary, you: in /etc/pam.d/lightdm: Add auth optional pam_gnome_keyring.so at the end of the auth section and session optional pam_gnome_keyring.so auto_start at the end of the session section I think there are ways to get auto unlock outside of that as well, but I haven't tried them. Do you want me to look into trying to get a different auto unlock to work? My last lightdm compile was 08/05/2020.
To be perfectly clear (I realize I could have been clearer before): gnome-keyring does display a password prompt for an ssh key when I, say, try to ssh into a box where I have a key stored in my keyring. The issue is that the gnome-keyring should have been automatically unlocked by lightdm via pam, and that's what stopped working for me.
but if you modify /etc/pam.d/lightdm it is not pam at all (itself). I mean yes, it is releated to the stack, but it is not pam/pambase causing it.
But I am wondering, can you please try to change all the `include` lines to the `substack` ones here: https://gitweb.gentoo.org/repo/gentoo.git/tree/x11-misc/lightdm/files/lightdm-autologin (well, it is the /etc/pam.d/lightdm file actually), and try again?
I think some (most?) displaymanagers ship this config/behavior by default (I think lightdm is supposed to be one of them at least according to arch-wiki, but for whatever reason it required adding the modification in Gentoo. Not sure if the wiki is wrong or one of arch/gentoo change default behavior) Do you think it's relatively certain the recent pam/pambase changes didn't trigger the problem (like, say, that some change could have triggered a bug in lightdm's handling of some new behavior)? I'm going to try this include suggestion, but first I will boot into the new pambase you released to isolate what change at a time. I'll get back to you shortly.
Changing all of the 'include' in /etc/pam.d/lightdm to 'substack' resulted in the autounlock working as expected. The file now looks like: auth substack system-local-login auth optional pam_gnome_keyring.so account substack system-local-login password substack system-local-login session substack system-local-login session optional pam_gnome_keyring.so auto_start Can you provide any insight on what this did and why it could have been suddenly needed? Is this a change I want to keep around? Is this something lightdm needs to change and perhaps other DMs in gentoo as well?
(In reply to Gregory Beauregard from comment #16) > I think some (most?) displaymanagers ship this config/behavior by default (I > think lightdm is supposed to be one of them at least according to arch-wiki, > but for whatever reason it required adding the modification in Gentoo. Not > sure if the wiki is wrong or one of arch/gentoo change default behavior) Very often distros have to ship theIr own pam stack files, what upstream ships often does not fit in practice, so we ship our own config (so does Arch). > [snip]
The bug has been closed via the following commit(s): https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=78187c218bcde260ef8ff50ddb3e3c34db5cb55f commit 78187c218bcde260ef8ff50ddb3e3c34db5cb55f Author: Mikle Kolyada <zlogene@gentoo.org> AuthorDate: 2020-10-11 06:58:21 +0000 Commit: Mikle Kolyada <zlogene@gentoo.org> CommitDate: 2020-10-11 06:58:21 +0000 x11-misc/lightdm: include ⟶ substack pambase files Acked-by: Lars Wendler <polynomial-c@gentoo.org> Closes: https://bugs.gentoo.org/747625 Package-Manager: Portage-3.0.4, Repoman-3.0.1 Signed-off-by: Mikle Kolyada <zlogene@gentoo.org> x11-misc/lightdm/files/lightdm-autologin | 4 ++-- .../lightdm/{lightdm-1.30.0-r1.ebuild => lightdm-1.30.0-r2.ebuild} | 0 2 files changed, 2 insertions(+), 2 deletions(-)
