Bug 216798 – amd64 with hardware/software raid root fails to boot

• Product: Gentoo Linux
• Status: RESOLVED TEST-REQUEST

[Description Marc Duponcheel 2008-04-08 00:20:30 UTC]

I have an amd64 server (called opteron) an amd64 desktop (called
desktop) and an x86 laptop (called laptop) with equal bleeding edge,
often unmasked, portage packages. They are updated almost daily.

 When I rebooted them this weekend, the x86 booted fine but both amd64
stopped and could not mount rootfs. Kernel panics.

 However, if I boot softlevel=single I do get a prompt and I can
continue the boot with, say, 'telinit 5'.

 The desktop has a soft md raid1 root.

 The opteron has a hard raid6 root.

 It is kind of hard to capture console (it contains garbage [FF...FF])
dmesg mentiones (on opteron):
 ...
Command line: root=/dev/sda3 softlevel=single[FF...
..FF]<6>BIOS-provided physical RAM map:
BIOS-e820: 0000000000000000 - 000000000009f400 (usable)
 ...

 The desktop dmesg looks good:
 ...
md: ... autorun DONE.
kjournald starting.  Commit interval 5 seconds
EXT3-fs: mounted filesystem with ordered data mode.
VFS: Mounted root (ext3 filesystem) readonly.
Freeing unused kernel memory: 212k freed
 ...


 But note that the above outputa are from successfull booting
(softlevel=single).  The unsuccessfull fails to VFS mount root, panics
and hangs.


 Here are some relevant packages

sys-apps/baselayout-2.0.0
sys-apps/openrc-0.2.1-r1
sys-apps/sysvinit-2.86-r10
sys-devel/gcc-4.3.0

 The kernel is 2.6.24-gentoo-r4


 Note that baselayout+openrc have been working well for weeks, so
personally I suspect that maybe there is a 64bit specific bug revealed
by compiling openrc or sysvinit with the latest (unmasked) gcc.

 I am happy to provide any input but I must say I typically only can
reboot during weekends (desktop is my workstation) and next weekend I
go on vacation for 2 weeks.


 I experiment with Linux for 15 years now so I am not complaining if
anything goes wrong... I only want to be helpful mentioning real
issues and contributing to solve them.

 Thanks!

[Comment 1 Marc Duponcheel 2008-04-08 00:37:35 UTC]

opteron ~ # emerge --info
Portage 2.1.5_rc2 (default-linux/amd64/2007.0, gcc-4.3.0, glibc-2.7-r2, 2.6.24-gentoo-r4 x86_64)
=================================================================
System uname: 2.6.24-gentoo-r4 x86_64 Dual-Core AMD Opteron(tm) Processor 2218
Timestamp of tree: Mon, 07 Apr 2008 20:16:01 +0000
distcc 2.18.3 x86_64-pc-linux-gnu (protocols 1 and 2) (default port 3632) [disabled]
ccache version 2.4 [disabled]
app-shells/bash:     3.2_p33
dev-java/java-config: 1.3.7, 2.1.5
dev-lang/python:     2.5.1-r5
dev-python/pycrypto: 2.0.1-r6
dev-util/ccache:     2.4-r7
sys-apps/baselayout: 2.0.0
sys-apps/sandbox:    1.2.18.1-r2
sys-devel/autoconf:  2.13, 2.61-r1
sys-devel/automake:  1.4_p6, 1.5, 1.6.3, 1.7.9-r1, 1.8.5-r3, 1.9.6-r2, 1.10.1
sys-devel/binutils:  2.18-r1
sys-devel/gcc-config: 1.4.0-r4
sys-devel/libtool:   1.5.26
virtual/os-headers:  2.6.24
ACCEPT_KEYWORDS="amd64"
CBUILD="x86_64-pc-linux-gnu"
CFLAGS="-march=opteron -O2 -pipe"
CHOST="x86_64-pc-linux-gnu"
CONFIG_PROTECT="/etc /var/bind"
CONFIG_PROTECT_MASK="/etc/env.d /etc/env.d/java/ /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/php/apache2-php5/ext-active/ /etc/php/cgi-php5/ext-active/ /etc/php/cli-php5/ext-active/ /etc/revdep-rebuild /etc/terminfo /etc/texmf/web2c /etc/udev/rules.d"
CXXFLAGS="-march=opteron -O2 -pipe"
DISTDIR="/usr/portage/distfiles"
FEATURES="collision-protect distlocks metadata-transfer parallel-fetch sandbox sfperms strict unmerge-orphans userfetch"
GENTOO_MIRRORS="ftp://ftp.belnet.be/mirror/rsync.gentoo.org/gentoo/ ftp://ftp.snt.utwente.nl/pub/os/linux/gentoo http://ftp.belnet.be/mirror/rsync.gentoo.org/gentoo/ http://trumpetti.atm.tut.fi/gentoo/ ftp://ftp.tu-clausthal.de/pub/linux/gentoo/ ftp://sunsite.informatik.rwth-aachen.de/pub/Linux/gentoo"
LANG="en_US.UTF-8"
LC_ALL="en_US.UTF-8"
LDFLAGS=""
LINGUAS="en fr nl"
MAKEOPTS="-j5"
PKGDIR="/usr/portage/packages"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --stats --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages"
PORTAGE_TMPDIR="/var/tmp"
PORTDIR="/usr/portage"
PORTDIR_OVERLAY="/usr/portage/local/layman/enlightenment /usr/local/portage /usr/portage/local/layman/perl"
SYNC="rsync://rsync.europe.gentoo.org/gentoo-portage"
USE="3dnow 3dnowext a52 aac acl acpi alsa amd64 ao apache2 apm audiofile avahi avi bash-completion berkdb bluetooth bzip2 cairo caps cdda cddb cdio cdparanoia cdr cjk clamav cli cracklib crypt ctype cups curl daap dba dbus dlloader doc dv dvb dvd dvdr dvdread encode ethereal examples exif expat fam fame fbcon ffmpeg flac font-server fontconfig foomaticdb fortran ftp gcj gd gdbm gif gimpprint gmp gnutls gphoto2 gpm gtk2 guile hal icecast iconv icu id3tag ident idn ieee1394 imagemagick imap imlib ipod ipv6 isdnlog jack jpeg jpeg2k ladspa lame ldap libsamplerate libwww lirc live lm_sensors logrotate lua mad mbox midi mikmod mime mmx mmxext mng modplug mp3 mpeg mudflap musepack musicbrainz mysql ncurses network nfs nls nptl nptlonly oav ogg openal openexr opengl openmp oss pam pango pcre pdf perl php pic png portaudio posix ppds pppd pulseaudio python quicktime quotas radio readline reflection ruby samba sasl scanner server session shout skey slang slp sndfile snmp sockets socks5 sox speex spell spl sse sse2 sse3 ssl stream svg syslog tcl tcpd tetex theora threads tiff truetype truetype-fonts type1-fonts udev unicode usb userlocales v4l v4l2 vorbis x264 xattr xcb xine xml xorg xpm xv xvid zeroconf zlib" ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87x ca0106 cmipci emu10k1x ens1370 ens1371 es1938 es1968 fm801 hda-intel intel8x0 intel8x0m maestro3 trident usb-audio via82xx via82xx-modem ymfpci" ALSA_PCM_PLUGINS="adpcm alaw asym copy dmix dshare dsnoop empty extplug file hooks iec958 ioplug ladspa lfloat linear meter mulaw multi null plug rate route share shm softvol" APACHE2_MODULES="actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" ELIBC="glibc" INPUT_DEVICES="keyboard mouse" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LINGUAS="en fr nl" LIRC_DEVICES="serial sir" USERLAND="GNU" VIDEO_CARDS="ati radeon vesa"
Unset:  CPPFLAGS, CTARGET, EMERGE_DEFAULT_OPTS, INSTALL_MASK, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS

Portage 2.1.5_rc2 (default-linux/x86/2007.0, gcc-4.3.0, glibc-2.7-r2, 2.6.24-gentoo-r4 i686)
=================================================================
System uname: 2.6.24-gentoo-r4 i686 Intel(R) Pentium(R) M processor 2.13GHz
Timestamp of tree: Mon, 07 Apr 2008 20:16:01 +0000
distcc 2.18.3 i686-pc-linux-gnu (protocols 1 and 2) (default port 3632) [disabled]
ccache version 2.4 [disabled]
app-shells/bash:     3.2_p33
dev-java/java-config: 1.3.7, 2.1.5
dev-lang/python:     2.5.1-r5
dev-python/pycrypto: 2.0.1-r6
dev-util/ccache:     2.4-r7
sys-apps/baselayout: 2.0.0
sys-apps/sandbox:    1.2.18.1-r2
sys-devel/autoconf:  2.13, 2.61-r1
sys-devel/automake:  1.4_p6, 1.5, 1.6.3, 1.7.9-r1, 1.8.5-r3, 1.9.6-r2, 1.10.1
sys-devel/binutils:  2.18-r1
sys-devel/gcc-config: 1.4.0-r4
sys-devel/libtool:   1.5.26
virtual/os-headers:  2.6.24
ACCEPT_KEYWORDS="x86"
CBUILD="i686-pc-linux-gnu"
CFLAGS="-march=i686 -O2 -pipe"
CHOST="i686-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/kde/3.5/env /usr/kde/3.5/share/config /usr/kde/3.5/shutdown /usr/kde/4.0/env /usr/kde/4.0/share/config /usr/kde/4.0/shutdown /usr/share/config /var/lib/hsqldb"
CONFIG_PROTECT_MASK="/etc/env.d /etc/env.d/java/ /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/php/apache2-php5/ext-active/ /etc/php/cgi-php5/ext-active/ /etc/php/cli-php5/ext-active/ /etc/revdep-rebuild /etc/terminfo /etc/texmf/web2c /etc/udev/rules.d"
CXXFLAGS="-march=i686 -O2 -pipe"
DISTDIR="/usr/portage/distfiles"
FEATURES="collision-protect distlocks metadata-transfer parallel-fetch sandbox sfperms strict unmerge-orphans userfetch"
GENTOO_MIRRORS="ftp://ftp.belnet.be/mirror/rsync.gentoo.org/gentoo/ ftp://ftp.snt.utwente.nl/pub/os/linux/gentoo http://ftp.belnet.be/mirror/rsync.gentoo.org/gentoo/ http://trumpetti.atm.tut.fi/gentoo/ ftp://ftp.tu-clausthal.de/pub/linux/gentoo/ ftp://sunsite.informatik.rwth-aachen.de/pub/Linux/gentoo"
LANG="en_US.UTF-8"
LC_ALL="en_US.UTF-8"
LDFLAGS=""
LINGUAS="en fr nl"
MAKEOPTS="-j2"
PKGDIR="/usr/portage/packages"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --stats --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages"
PORTAGE_TMPDIR="/var/tmp"
PORTDIR="/usr/portage"
PORTDIR_OVERLAY="/usr/portage/local/layman/enlightenment /usr/local/portage /usr/portage/local/layman/perl"
SYNC="rsync://rsync.europe.gentoo.org/gentoo-portage"
USE="X a52 aac acl acpi alsa ao apm arts audiofile avahi avi bash-completion berkdb bitmap-fonts bluetooth bzip2 cairo caps cdda cddb cdio cdparanoia cdr cjk clamav cli cracklib crypt ctype cups curl daap dba dbus dga dlloader doc dri dv dvb dvd dvdr dvdread encode esd ethereal examples exif expat fam fame fbcon ffmpeg firefox flac font-server fontconfig foomaticdb fortran ftp gcj gd gdbm gif gimpprint glut gmp gnome gnutls gphoto2 gpm gstreamer gtk gtk2 gtkhtml guile hal htmlhandbook iconv icu id3tag idn ieee1394 imagemagick imap imlib ipod ipv6 irda isdnlog jack java jpeg jpeg2k kde ladspa lame ldap lesstif libnotify libsamplerate libwww lirc live lm_sensors logrotate lua mad mbox midi mikmod mime mmx mng modplug mono motif mozilla mp3 mpeg mudflap musepack musicbrainz mysql ncurses network nfs nls nptl nptlonly nsplugin oav ogg openal openexr opengl openmp oss pam pango pcre pdf perl php pic png portaudio posix ppds pppd pulseaudio python qt qt3 qt4 quicktime quotas radio rdesktop readline real reflection ruby samba sasl scanner sdl server session shout skey slang slp sndfile snmp sockets socks5 sox speex spell spl sse sse2 ssl stream svg svga syslog tcl tcltk tcpd tetex theora threads tiff truetype truetype-fonts type1-fonts udev unicode usb userlocales v4l v4l2 vorbis win32codecs x264 x86 xcb xine xml xorg xpm xulrunner xv xvid zeroconf zlib" ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87x ca0106 cmipci emu10k1 emu10k1x ens1370 ens1371 es1938 es1968 fm801 hda-intel intel8x0 intel8x0m maestro3 trident usb-audio via82xx via82xx-modem ymfpci" ALSA_PCM_PLUGINS="adpcm alaw asym copy dmix dshare dsnoop empty extplug file hooks iec958 ioplug ladspa lfloat linear meter mulaw multi null plug rate route share shm softvol" APACHE2_MODULES="actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" ELIBC="glibc" INPUT_DEVICES="keyboard mouse" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LINGUAS="en fr nl" LIRC_DEVICES="serial sir" USERLAND="GNU" VIDEO_CARDS="fglrx ati radeon vesa"
Unset:  CPPFLAGS, CTARGET, EMERGE_DEFAULT_OPTS, INSTALL_MASK, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS

[Comment 2 SpanKY 2008-04-08 01:02:03 UTC]

if the kernel is aborting with unable to mount the rootfs, you havent even made it to userspace which means openrc/baselayout cannot be involved.

a hardware raid solution also implies that the kernel has no idea it is running on a raid.  it only sees a single disk.  if your kernel lacks the device driver for the card in question, then your kernel is compiled incorrectly.

[Comment 3 Timo Gurr (RETIRED) 2008-04-08 02:13:59 UTC]

If you happen to use grub perhaps you ran into bug #216307.

[Comment 4 Marc Duponcheel 2008-04-08 03:31:50 UTC]

Hi SpanKY
The rootfs does mount if I choose grub.conf lines ending in
 softlevel=single
To get a default runlevel I then do
 # telinit 5

Hi Timo, the bug #216307 may match my 'opteron' hw raid system
 but root= is not last in my grub line:
kernel /vmlinuz-2.6.24-gentoo-r4 root=/dev/sda3 rootflags=usrquota,grpquota

However, for my sw raid system 'desktop' the line is
kernel /vmlinuz-2.6.24-gentoo-r4 root=/dev/md1
 so there it is last

Note that when I choose the line ending in
 opteron (hw raid)
root=/dev/sda3 softlevel=single
 desktop (sw raid)
root=/dev/md1 softlevel=single
 then it is not last and mount succeeds.

I will try the new -r5 grub patchset

[Comment 5 Marc Duponcheel 2008-04-08 03:49:40 UTC]

Hi all

The latest grub fixes my hw raid server. Great!

I won't try my sw raid server before Fri 11 Apr since it takes
me too much time to setup my desktops for work.

Thanks!

[Comment 6 Jan Kundrát (RETIRED) 2008-04-09 17:48:14 UTC]

(In reply to comment #5)
> The latest grub fixes my hw raid server. Great!

Glad to hear that.

> I won't try my sw raid server before Fri 11 Apr since it takes
> me too much time to setup my desktops for work.

Closing in the meanwhile, please re-open if problem persists.

[Comment 7 Marc Duponcheel 2008-04-10 21:36:17 UTC]

Thanks all for the help. bug# 216798 is a duplicate of bug# 216307