Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!

Bug 950264

Summary: sys-kernel/gentoo-sources-6.12.16 hangs on boot with loglevel<7
Product: Gentoo Linux Reporter: Christopher Head <bugs>
Component: Current packagesAssignee: Gentoo Kernel Bug Wranglers and Kernel Maintainers <kernel>
Status: UNCONFIRMED ---    
Severity: normal CC: bugs
Priority: Normal    
Version: unspecified   
Hardware: AMD64   
OS: Linux   
Whiteboard:
Package list:
Runtime testing required: ---
Attachments: Kernel config file with default log level 7
GRUB configuration file
Bootconfig file
Dmesg output from a successful boot

Description Christopher Head 2025-02-25 05:42:57 UTC
Created attachment 919879 [details]
Kernel config file with default log level 7

On one specific VPS, out of all my machines, upgrading to gentoo-sources-6.12.16 caused a hang on boot. On trying to debug, I eventually discovered that it boots OK with loglevel=7 on the kernel command line, and hangs with loglevel=6 (or less). I’m using sys-kernel/genkernel-4.3.16-r2 to build an initramfs, but not the kernel itself. I’ll attach the Kconfig (with log level 7; changing CONFIG_CONSOLE_LOGLEVEL_foo=7 to a lower number, or specifying it on the command line with loglevel=, both cause the hang).

$ emerge --info sys-kernel/gentoo-sources
Portage 3.0.66.1 (python 3.12.9-final-0, default/linux/amd64/23.0/no-multilib/systemd, gcc-14, glibc-2.40-r8, 6.12.16-gentoo x86_64)
=================================================================
                         System Settings
=================================================================
System uname: Linux-6.12.16-gentoo-x86_64-Intel-R-_Xeon-R-_Silver_4210R_CPU_@_2.40GHz-with-glibc2.40
KiB Mem:     2042648 total,   1670360 free
KiB Swap:    1047548 total,   1047548 free
Timestamp of repository gentoo: Mon, 24 Feb 2025 11:45:00 +0000
Head commit of repository gentoo: 895bd61078d5f5e3c79ca10b7498463f93e1f850
Head commit of repository hawk777: 92c44772e64a30cc433503350db2aa383db487e9

sh bash 5.2_p37
ld GNU ld (Gentoo 2.43 p3) 2.43.1
app-misc/pax-utils:        1.3.8::gentoo
app-shells/bash:           5.2_p37::gentoo
dev-build/autoconf:        2.72-r1::gentoo
dev-build/automake:        1.16.5-r2::gentoo
dev-build/cmake:           3.31.5::gentoo
dev-build/libtool:         2.5.4::gentoo
dev-build/make:            4.4.1-r100::gentoo
dev-build/meson:           1.6.1::gentoo
dev-lang/perl:             5.40.0-r1::gentoo
dev-lang/python:           3.12.9::gentoo, 3.13.2::gentoo
dev-lang/rust-bin:         1.84.1-r2::gentoo
sys-apps/baselayout:       2.17::gentoo
sys-apps/sandbox:          2.39::gentoo
sys-apps/systemd:          256.10::gentoo
sys-devel/binutils:        2.43-r2::gentoo
sys-devel/binutils-config: 5.5.2::gentoo
sys-devel/gcc:             14.2.1_p20241221::gentoo
sys-devel/gcc-config:      2.12.1::gentoo
sys-kernel/linux-headers:  6.6-r1::gentoo (virtual/os-headers)
sys-libs/glibc:            2.40-r8::gentoo
Repositories:

gentoo
    location: /var/cache/repos/gentoo
    sync-type: rsync
    sync-uri: rsync://rsync.ca.gentoo.org/gentoo-portage
    priority: 1
    volatile: True
    sync-rsync-verify-jobs: 1
    sync-rsync-extra-opts: 
    sync-rsync-verify-max-age: 3
    sync-rsync-verify-metamanifest: yes

hawk777
    location: /var/cache/repos/hawk777
    sync-type: git
    sync-uri: /var/lib/git/portage
    masters: gentoo
    priority: 2
    volatile: True

Installed sets: @admin, @dns, @letsencrypt, @mail, @pim, @web, @wedding
ACCEPT_KEYWORDS="amd64"
ACCEPT_LICENSE="@FREE"
CBUILD="x86_64-pc-linux-gnu"
CFLAGS="-O2 -pipe -march=native -fomit-frame-pointer"
CHOST="x86_64-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/share/gnupg/qualified.txt /var/bind"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/sandbox.d"
CXXFLAGS="-O2 -pipe -march=native -fomit-frame-pointer"
DISTDIR="/var/cache/distfiles"
EMERGE_DEFAULT_OPTS="--with-bdeps y --complete-graph y -1 --quiet-build --unordered-display"
ENV_UNSET="CARGO_HOME DBUS_SESSION_BUS_ADDRESS DISPLAY GDK_PIXBUF_MODULE_FILE GOBIN GOPATH PERL5LIB PERL5OPT PERLPREFIX PERL_CORE PERL_MB_OPT PERL_MM_OPT XAUTHORITY XDG_CACHE_HOME XDG_CONFIG_HOME XDG_DATA_HOME XDG_RUNTIME_DIR XDG_STATE_HOME"
FCFLAGS="-O2 -pipe -march=native -fomit-frame-pointer"
FEATURES="assume-digests binpkg-docompress binpkg-dostrip binpkg-logs buildpkg-live config-protect-if-modified distlocks ebuild-locks fixlafiles ipc-sandbox merge-sync merge-wait multilib-strict network-sandbox news parallel-fetch pid-sandbox pkgdir-index-trusted preserve-libs protect-owned qa-unresolved-soname-deps sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync xattr"
FFLAGS="-O2 -pipe -march=native -fomit-frame-pointer"
GENTOO_MIRRORS="https://gentoo.mirrors.ovh.net/gentoo-distfiles/ https://mirror.reenigne.net/gentoo/ https://mirror.csclub.uwaterloo.ca/gentoo-distfiles/"
INSTALL_MASK="grub-mkconfig"
LANG="en_CA.utf8"
LDFLAGS="-Wl,-O1 -Wl,--as-needed -Wl,-z,pack-relative-relocs"
LEX="flex"
PKGDIR="/var/cache/binpkgs"
PORTAGE_CONFIGROOT="/"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages --exclude=/.git"
PORTAGE_TMPDIR="/var/cache"
RUSTFLAGS="-Ctarget-cpu=native"
SHELL="/bin/bash"
USE="acl amd64 brotli bzip2 caps iconv icu idn ipv6 libtirpc lto ncurses nls pam pcre readline seccomp ssl systemd test-rust udev unicode xattr zlib" ABI_X86="64" ADA_TARGET="gcc_14" APACHE2_MODULES="authn_core authz_core socache_shmcb unixd actions alias auth_basic authn_anon authn_dbm authn_file authz_dbm authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir env expires ext_filter file_cache filter headers include info log_config logio mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="karbon sheets words" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" CPU_FLAGS_X86="mmx mmxext sse sse2 aes avx avx2 avx512_vnni avx512bw avx512cd avx512dq avx512f avx512vl f16c fma3 pclmul popcnt rdrand sse3 sse4_1 sse4_2 ssse3" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock greis isync itrax navcom oceanserver oncore rtcm104v2 rtcm104v3 sirf skytraq superstar2 tsip tripmate tnt ublox" GUILE_SINGLE_TARGET="3-0" GUILE_TARGETS="3-0" INPUT_DEVICES="libinput" KERNEL="linux" L10N="en en-CA" LCD_DEVICES="bayrad cfontz glk hd44780 lb216 lcdm001 mtxorb text" LUA_SINGLE_TARGET="lua5-1" LUA_TARGETS="luajit" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php8-2" POSTGRES_TARGETS="postgres16" PYTHON_SINGLE_TARGET="python3_12" PYTHON_TARGETS="python3_12" RUBY_TARGETS="ruby32" VIDEO_CARDS="amdgpu fbdev intel nouveau radeon radeonsi vesa dummy" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipp2p iface geoip fuzzy condition tarpit sysrq proto logmark ipmark dhcpmac delude chaos account"
Unset:  ADDR2LINE, AR, ARFLAGS, AS, ASFLAGS, CC, CCLD, CONFIG_SHELL, CPP, CPPFLAGS, CTARGET, CXX, CXXFILT, ELFEDIT, EXTRA_ECONF, F77FLAGS, FC, GCOV, GPROF, LC_ALL, LD, LFLAGS, LIBTOOL, LINGUAS, MAKE, MAKEFLAGS, MAKEOPTS, NM, OBJCOPY, OBJDUMP, PORTAGE_BINHOST, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS, PYTHONPATH, RANLIB, READELF, SIZE, STRINGS, STRIP, YACC, YFLAGS

=================================================================
                        Package Settings
=================================================================

sys-kernel/gentoo-sources-6.6.74::gentoo was built with the following:
USE="symlink -build -experimental"


sys-kernel/gentoo-sources-6.12.16::gentoo was built with the following:
USE="symlink -build -experimental"
Comment 1 Mike Pagano gentoo-dev 2025-02-25 16:14:33 UTC
Anything on the screen ?
Comment 2 Christopher Head 2025-02-26 07:05:51 UTC
With loglevel 6, there are a few console messages, but nothing obviously relevant:

Booting a command line

[    0.000000] Linux version 6.12.16-gentoo (root@<hostname elided>) (gcc (Gentoo 14.2.1_p20241221 p7) 14.2.1 20241221, GNU ld (Gentoo 2.43 p3) 2.43.1) #4 Mon Feb 24 21:46:34 PST 2025
[    0.011275] Kernel command line: BOOT_IMAGE=/root/boot/vmlinuz bootconfig loglevel=6 -- quiet
[    0.011305] Unknown kernel command line parameters "BOOT_IMAGE=/root/boot/vmlinuz", will be passed to user space.
[    0.011319] random: crng init done
Comment 3 Mike Pagano gentoo-dev 2025-02-26 17:48:17 UTC
Please attach your full grub.cfg
Comment 4 Christopher Head 2025-02-27 05:49:27 UTC
Created attachment 919948 [details]
GRUB configuration file

Sure, though there’s not much interesting about it.
Comment 5 Christopher Head 2025-02-27 05:50:22 UTC
Created attachment 919949 [details]
Bootconfig file

And before you ask, here’s the bootconfig.
Comment 6 Mike Pagano gentoo-dev 2025-02-27 12:25:28 UTC
Can I see a full dmesg of a working kernel?


I've never seen a grub.cfg file like that. You make it with grub-mkconfig ?

Another maybe dumb question, your boot partition if off /root ?
Comment 7 Mike Pagano gentoo-dev 2025-02-27 12:27:11 UTC
Also, 

can you test with vanilla-sources and see if the same issue happens ?
Comment 8 Christopher Head 2025-02-28 07:19:48 UTC
Created attachment 920031 [details]
Dmesg output from a successful boot

Here’s the dmesg output.
Comment 9 Christopher Head 2025-02-28 07:52:29 UTC
(In reply to Mike Pagano from comment #6)
> I've never seen a grub.cfg file like that. You make it with grub-mkconfig ?

No, hand-written. Given how simple it is to write a grub.cfg, I never quite found it worth the time to learn how the automation stuff works.

> Another maybe dumb question, your boot partition if off /root ?

No separate boot partition. There is a BIOS Boot Partition (not mounted, not a filesystem, just for GRUB to use, needed because it’s a GPT disk), a btrfs partition (which is root, and /boot is inside it), and a swap partition.

> can you test with vanilla-sources and see if the same issue happens ?

I tried vanilla-sources-6.12.16 (since that’s the same version number as the current stable gentoo-sources). Behaviour is exactly the same: hangs with loglevel=6, boots fine with loglevel=7.
Comment 10 Christopher Head 2025-02-28 17:14:38 UTC
By the way, in case you’re wondering how I concluded this is the kernel hanging and not something else:

1. On this machine, if I boot with “debug loglevel=7”, I get a lot of kernel output, then the initramfs banner (“>>> Genkernel 4.3.16” and so on), and then the rescue shell.
2. On any other machine, if I boot with “debug loglevel=6”, I get short-ish kernel output but still the initramfs banner and rescue shell.
3. On this machine, if I boot with “debug loglevel=6”, I get only the very short kernel output I pasted above, and no initramfs banner.

So by (1) I conclude that the initramfs on this machine is working fine if we can just get to it, by (2) I conclude that even with loglevel=6 the banner should still be printed if the get to the initramfs at all, and therefore by (3) we must not be getting to the initramfs.

And it can’t be anything *before* the kernel hanging because those few kernel messages *are* printed. So if it’s not hanging pre-kernel, and it’s not hanging after reaching the initramfs, the kernel is the only thing left.