Booting sys-apps/systemd-219-r2 (and earlier versions) sometimes hangs during boot. Unable to login. After enabling kernel option 'systemd.log_level=debug systemd.log_target=kmsg log_buf_len=10M' and connecting to debug shell journalctl -b shows: systemd[1]: Failed to register name: Connection timed out systemd-logind[1872]: Failed to enable subscription: Connection timed out systemd-logind[1872]: Failed to fully start up daemon: Connection timed out systemd[1]: Failed to set up API bus: Connection timed out The journal shows systemd-logind is scheduled for restart but the restart rarely happens. Interestingly typing 'kill 1' ing the debug shell enables the boot to continue successfully. Reproducible: Sometimes Steps to Reproduce: 1. Boot with systemd emerged. 2. Repeat 1 until failure to complete boot occurrs. 3. Actual Results: Boot hangs after systemd.logind fails Expected Results: Boot succeeds emerge --info Portage 2.2.14 (python 2.7.9-final-0, default/linux/amd64/13.0/desktop, gcc-4.8.3, glibc-2.19-r1, 3.19.2-gentoo-64-sd x86_64) ================================================================= System uname: Linux-3.19.2-gentoo-64-sd-x86_64-AMD_FX-tm-9590_Eight-Core_Processor-with-gentoo-2.2 KiB Mem: 32859072 total, 31166872 free KiB Swap: 46137328 total, 46137328 free Timestamp of tree: Sun, 22 Mar 2015 02:45:01 +0000 ld GNU ld (Gentoo 2.24 p1.4) 2.24 distcc 3.1 x86_64-pc-linux-gnu [disabled] ccache version 3.1.9 [disabled] app-shells/bash: 4.3_p33-r1 dev-java/java-config: 2.2.0 dev-lang/perl: 5.20.1-r4 dev-lang/python: 2.7.9-r1, 3.3.5-r1, 3.4.1 dev-util/ccache: 3.1.9-r4 dev-util/cmake: 2.8.12.2-r1 dev-util/pkgconfig: 0.28-r1 sys-apps/baselayout: 2.2 sys-apps/openrc: 0.13.11 sys-apps/sandbox: 2.6-r1 sys-devel/autoconf: 2.13, 2.69 sys-devel/automake: 1.11.6-r1, 1.13.4 sys-devel/binutils: 2.24-r3 sys-devel/gcc: 4.7.3-r1, 4.8.3 sys-devel/gcc-config: 1.7.3 sys-devel/libtool: 2.4.4 sys-devel/make: 4.1-r1 sys-kernel/linux-headers: 3.18 (virtual/os-headers) sys-libs/glibc: 2.19-r1 Repositories: gentoo local ikelos kodi-overlay jkolo calculate Installed sets: @kernels ACCEPT_KEYWORDS="amd64" ACCEPT_LICENSE="*" CBUILD="x86_64-pc-linux-gnu" CFLAGS="-march=amdfam10 -mtune=amdfam10 -O2 -pipe" CHOST="x86_64-pc-linux-gnu" CONFIG_PROTECT="/etc /usr/lib64/libreoffice/program/sofficerc /usr/share/config /usr/share/gnupg/qualified.txt /usr/share/themes/oxygen-gtk/gtk-2.0 /var/lib/hsqldb" CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/dconf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/php/apache2-php5.5/ext-active/ /etc/php/cgi-php5.5/ext-active/ /etc/php/cli-php5.5/ext-active/ /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo /etc/texmf/language.dat.d /etc/texmf/language.def.d /etc/texmf/updmap.d /etc/texmf/web2c" CXXFLAGS="-march=amdfam10 -mtune=amdfam10 -O2 -pipe" DISTDIR="/mnt/portage.autofs/distfiles" EMERGE_DEFAULT_OPTS="--with-bdeps=y -j9 --load-average=8 --quiet-build=y" FCFLAGS="-O2 -pipe" FEATURES="assume-digests binpkg-logs buildpkg config-protect-if-modified distlocks ebuild-locks fixlafiles merge-sync news parallel-fetch parallel-install preserve-libs protect-owned sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync" FFLAGS="-O2 -pipe" GENTOO_MIRRORS="http://mirror.qubenet.net/mirror/gentoo/ http://mirrors.linuxant.fr/distfiles.gentoo.org/ ftp://mirror.qubenet.net/mirror/gentoo/ http://www.mirrorservice.org/sites/www.ibiblio.org/gentoo/ http://ftp.snt.utwente.nl/pub/os/linux/gentoo ftp://ftp.free.fr/mirrors/ftp.gentoo.org/ http://mirror.leaseweb.com/gentoo/ http://gentoo.modulix.net/gentoo/ http://linux.rz.ruhr-uni-bochum.de/download/gentoo-mirror/ ftp://mirror.bytemark.co.uk/gentoo/" LANG="en_GB.UTF-8" LDFLAGS="-Wl,-O1 -Wl,--as-needed" MAKEOPTS="-j17 --load-average=16" PKGDIR="/mnt/portage.autofs/packages" PORTAGE_CONFIGROOT="/" PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages --exclude=lost+found/" PORTAGE_TMPDIR="/var/tmp" PORTDIR="/usr/portage" PORTDIR_OVERLAY="/usr/local/portage /var/lib/layman/ikelos /var/lib/layman/kodi-overlay /var/lib/layman/jkolo /var/lib/layman/calculate" SYNC="rsync://datastoreone/gentoo-portage" USE="X a52 aac accessibility acl acpi aes alsa amarok amd64 amr authdaemond avx bash-completion berkdb bluetooth bluray branding bzip2 cairo caps cdda cdr cli cracklib crypt cups cxx dbus dri dts dv dvd dvdr dvdread emboss encode exif fam firefox flac fma3 fma4 fortran gdbm gif glamor gpm gtk iconv ieee1394 ipv6 jpeg kde lcms ldap libnotify lm_sensors lock mad mmx mmxext mng modules mp3 mp4 mpeg multilib mysql ncurses network nls nptl ofx ogg opengl openmp oss pam pango pcre pdf png policykit popcnt ppds qt3 qt3support qt4 readline samba sasl sdl semantic-desktop session spell sse sse2 sse3 sse4_1 sse4_2 sse4a ssl ssse3 startup-notification svg systemd tcpd thunar tiff truetype udev udisks uk_bleb uk_rt unicode upower usb v4l v4l2 vdpau vorbis wxwidgets x264 xcb xine xinerama xml xop xv xvid xvmc zlib" ABI_X86="64" ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87x ca0106 cmipci emu10k1x ens1370 ens1371 es1938 es1968 fm801 hda-intel intel8x0 intel8x0m maestro3 trident usb-audio via82xx via82xx-modem ymfpci" APACHE2_MODULES="actions alias auth_basic auth_digest authn_anon authn_dbd authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache dav dav_fs dav_lock dbd deflate dir disk_cache env expires ext_filter file_cache filter headers ident imagemap include info log_config logio mem_cache mime mime_magic negotiation proxy proxy_ajp proxy_balancer proxy_connect proxy_http rewrite setenvif so speling status unique_id userdir usertrack vhost_alias cgi cgid" CALLIGRA_FEATURES="kexi words flow plan sheets stage tables krita karbon braindump author" CAMERAS="ptp2" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" CPU_FLAGS_X86="aes avx fma3 fma4 mmx mmxext popcnt sse sse2 sse3 sse4_1 sse4_2 sse4a ssse3 xop" DRACUT_MODULES="lvm syslog" DVB_CARDS="usb-dib0700" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf superstar2 timing tsip tripmate tnt ublox ubx" GRUB_PLATFORMS="efi-32 efi-64 pc multiboot mount" INPUT_DEVICES="evdev" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" LINGUAS="en en_GB" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php5-5" PYTHON_SINGLE_TARGET="python2_7" PYTHON_TARGETS="python2_7 python3_3" QEMU_SOFTMMU_TARGETS="i386 x86_64" QEMU_USER_TARGETS="i386 x86_64" RUBY_TARGETS="ruby19 ruby20" SANE_BACKENDS="hp5590 hp net abaton agfafocus apple artec artec_eplus48u as6e avision bh canon canon630u canon_dr canon_pp cardscan coolscan coolscan2 coolscan3 dc210 dc240 dc25 dell1600n_net dmc epjitsu epson epson2 fujitsu genesys gt68xx hp3500 hp3900 hp4200 hp5400 hpljm1005 hpsj5s hs2p ibm kodak kvs1025 kvs20xx leo lexmark ma1509 magicolor matsushita microtek microtek2 mustek mustek_pp mustek_usb nec niash p5 pie pixma plustek plustek_pp pnm qcam ricoh rts8891 s9036 sceptre sharp sm3600 sm3840 snapscan sp15c st400 stv680 tamarack teco1 teco2 teco3 test u12 umax umax1220u umax_pp xerox_mfp" USERLAND="GNU" VIDEO_CARDS="nvidia" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account" Unset: CPPFLAGS, CTARGET, INSTALL_MASK, LC_ALL, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS, USE_PYTHON
While attempting to collect log I found 1000's of lines containing 'systemd-udevd[878]: maximum number (24) of children reached' so I added 'udev.children-max=64' to the kernel options. This removed the noise but had no effect on thesystemd-logind issue. I have added logs from 'dmesg', 'journal -b -l' and 'journal -b -l' after using 'kill 1' in the debug shell.
Created attachment 399464 [details] dmesg.txt
Created attachment 399466 [details] journalctl -b -l
Created attachment 399468 [details] journalctl -b -l after kill 1 in debug shell and boot completed
This output is not from the same boot as the attached log but 'systemctl list-jobs' typically shows JOB UNIT TYPE STATE 279 xinetd.service start waiting 292 rpc-statd.service start waiting 256 gentoo-local-10-uam-poll-dvd-cd.service start waiting 305 remote-fs-pre.target start waiting 319 kdm.service start waiting 311 distccd.service start waiting 268 mysqld.service start waiting 280 systemd-user-sessions.service start waiting 261 rpcbind.target start waiting 312 nmbd.service start waiting 138 multi-user.target start waiting 283 gentoo-local-91-check-efibootmgr-setings.service start waiting 287 nfs-server.service start waiting 294 rpc-statd-notify.service start waiting 617 systemd-logind.service restart waiting 282 gentoo-local-90-grub2-set-default-0.service start waiting 296 systemd-networkd-wait-online.service start waiting 258 network.target start waiting 313 smbd.service start waiting 289 nfs-mountd.service start waiting 299 remote-fs.target start waiting 304 nfs-client.target start waiting 260 rpcbind.service start waiting 270 vmware.target start waiting 295 network-online.target start waiting 314 apache2.service start waiting 269 postfix.service start waiting 137 graphical.target start waiting 28 jobs listed.
Created attachment 399472 [details] journalctl -b -l from a successfully boot
For comparission I have added 'journalctl -b -l' from a successfully boot
I found https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/1423811 It looks close to my issue, so I reversed the update referred to ( http://cgit.freedesktop.org/systemd/systemd/patch/?id=13790add4bf648fed816361794d8277a75253410 ), re-emerged systemd with the revesed patch applied and created a new initramfs. With the patch applied I rebooted 40 times without hitting the systemd-logind issue. Without the patch the fail would occurr about every 5th boot.
Created attachment 399566 [details, diff] Patch to revert offending systemd commit.
BTW this issue occurs on most of my boxes, with varing frequency, including the mythtv server. :-( I have also install the revert patch on the mythtv server and will update this bug should the issue reoccur.
i had the same problem on 3 different systems, 1 on virtualbox, 2 with real hardware. I solved it by downgrading to systemd-218 for now. The problem seems to occur on several distributions, see http://comments.gmane.org/gmane.comp.sysutils.systemd.devel/28525 and https://bugs.archlinux.org/task/44016 .
As far as i can see, this does not happen with systemd-220-r2.
Thanks for feedback