After having build a new kernel based on 2.6.27-r7 upon restart the bootprocess freezes at the point where it says: Waiting for uevents to be processed With udev-124 this message comes from the udev-start script with udev-135 this comes from /etc/init.d/udev script. The actual command in both cases is as far as I can tell : udevadm --settle Having used the running .config from my working 2.6.23 kernel to do a make oldconfig on a number of kernels from 2.6.24 and up it seems the major change appears from 2.6.23 to 2.6.24 that causes the freeze. This have been established by doing diffs on the .config files produced. Reproducible: Always Steps to Reproduce: 1.compile the kernel 2.install it 3.see it freeze when trying to process uevents Actual Results: The system freezes without any messages in the logs - the only message is the : Waiting for uevents to be processed sent to the screen Expected Results: The system boots :-) Portage 2.1.6.4 (default/linux/amd64/2008.0/desktop, gcc-4.1.2, glibc-2.8_p20080602-r1, 2.6.23-gentoo-r9 x86_64) ================================================================= System uname: Linux-2.6.23-gentoo-r9-x86_64-Dual_Core_AMD_Opteron-tm-_Processor_285-with-glibc2.2.5 Timestamp of tree: Sat, 21 Feb 2009 03:00:01 +0000 ccache version 2.4 [enabled] app-shells/bash: 3.2_p39 dev-java/java-config: 1.3.7-r1, 2.1.6-r1 dev-lang/python: 2.4.4-r13, 2.5.2-r7 dev-python/pycrypto: 2.0.1-r6 dev-util/ccache: 2.4-r7 dev-util/cmake: 2.4.6-r1 sys-apps/baselayout: 1.12.11.1 sys-apps/sandbox: 1.2.18.1-r2 sys-devel/autoconf: 2.13, 2.63 sys-devel/automake: 1.4_p6, 1.5, 1.6.3, 1.7.9-r1, 1.8.5-r3, 1.9.6-r2, 1.10.2 sys-devel/binutils: 2.18-r3 sys-devel/gcc-config: 1.4.0-r4 sys-devel/libtool: 1.5.26 virtual/os-headers: 2.6.27-r2 ACCEPT_KEYWORDS="amd64" CBUILD="x86_64-pc-linux-gnu" CFLAGS="-O2 -pipe -mtune=k8 -march=k8" CHOST="x86_64-pc-linux-gnu" CONFIG_PROTECT="/etc /usr/kde/3.5/env /usr/kde/3.5/share/config /usr/kde/3.5/shutdown /usr/share/config /var/bind" CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/env.d/java/ /etc/fonts/fonts.conf /etc/gconf /etc/php/apache2-php5/ext-active/ /etc/php/cgi-php5/ext-active/ /etc/php/cli-php5/ext-active/ /etc/revdep-rebuild /etc/splash /etc/terminfo /etc/udev/rules.d" CXXFLAGS="-O2 -pipe -mtune=k8 -march=k8" DISTDIR="/usr/portage/distfiles" FEATURES="ccache distlocks fixpackages parallel-fetch protect-owned sandbox sfperms strict unmerge-orphans userfetch userpriv" GENTOO_MIRRORS="ftp://ftp.linux.ee/pub/gentoo/distfiles/ http://ftp.linux.ee/pub/gentoo/distfiles/ ftp://pandemonium.tiscali.de/pub/gentoo/ http://gentoo.blueyonder.co.uk http://trumpetti.atm.tut.fi/gentoo/" LANG="da_DK.utf8" LDFLAGS="-Wl,-O1" LINGUAS="en da" MAKEOPTS="-j5" PKGDIR="/usr/portage/packages" PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --stats --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages" PORTAGE_TMPDIR="/var/tmp" PORTDIR="/usr/portage" PORTDIR_OVERLAY="/usr/local/portage-kristian-duus" SYNC="rsync://rsync.gentoo.org/gentoo-portage" USE="X acl acpi aiglx alsa amd64 apache2 avahi bacula-console bash-completion berkdb blender-game bluetooth branding browserplugin bzip2 cairo calendar cdr cgi clamd cli cracklib crypt ctype cups dbus divx dri dvd dvdr dvdread eds emboss encode esd evo extensions extraengine fam fax fbcondecor ffmpeg firefox foomaticdb fortran gdbm gif glitz gmedia gnome gocr gpm gstreamer gtk gtk2 hal hardenedphp hddtemp iconv ipv6 isdnlog java jpeg kde kqemu ldap libnotify lm_sensors mad masquerade midi mikmod mmx mp3 mpeg mudflap multilib mysql mysqli ncurses new-login nls nptl nptlonly nsplugin ocrad ogg openal openexr opengl openmp pam pcre pdf perl php png ppds pppd python qt3 qt3support qt4 quicktime readline realmedia reflection scanner sdl sensord session sip slang spell spl sse sse2 ssl startup-notification subject-rewrite suexec suhosin svg sysfs tcpd tesseract theora tiff tk truetype unicode usb vhosts vim-syntax vorbis wmp xml xorg xulrunner xv yaz zlib" ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87x ca0106 cmipci emu10k1x ens1370 ens1371 es1938 es1968 fm801 hda-intel intel8x0 intel8x0m maestro3 trident usb-audio via82xx via82xx-modem ymfpci" ALSA_PCM_PLUGINS="adpcm alaw asym copy dmix dshare dsnoop empty extplug file hooks iec958 ioplug ladspa lfloat linear meter mmap_emul mulaw multi null plug rate route share shm softvol" APACHE2_MODULES="actions alias auth_basic auth_digest authn_anon authn_dbd authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache dav dav_fs dav_lock dbd deflate dir disk_cache env expires ext_filter file_cache filter headers ident imagemap include info log_config logio mem_cache mime mime_magic negotiation proxy proxy_ajp proxy_balancer proxy_connect proxy_http rewrite setenvif so speling status unique_id userdir usertrack vhost_alias" ELIBC="glibc" INPUT_DEVICES="keyboard mouse evdev" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LINGUAS="en da" USERLAND="GNU" VIDEO_CARDS="mach64 mga vesa vga dummy" Unset: CPPFLAGS, CTARGET, EMERGE_DEFAULT_OPTS, FFLAGS, INSTALL_MASK, LC_ALL, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS
Created attachment 183057 [details] .config file for 2.6.27
Created attachment 183058 [details] Working .config for 2.6.23-r9
Can you try this? 1) Make a backup of /etc/udev/rules.d/70-persistent*.rules 2) delete 70-persistent*.rules 3) reboot If they don't regenerate you can copy them back. Then possibly, emerge udev again
Before deleting the files I would like to understand why this should solve the problem and what will happen if I still cannot boot ? And last but not least what will happen to my fallback scenario - ie. the 2.6.23 kernel.
You're backing up the files, so you can restore even from a rescue cd if necessary. I think wikipedia could do a better job of teaching you udev than I can. http://en.wikipedia.org/wiki/Udev Look specifically at udev rules
I have finally had a chance to reboot the system again after deleting the 70-persistant* files. Booting with 2.6.27 it freezes as before. Booting with 2.6.23-r9 regenerates the files as they were before the deletion.
I'm wondering if this is a hardware issue. Do you have any extraneous hardware in your system? Any tv cards?
(In reply to comment #7) > I'm wondering if this is a hardware issue. Do you have any extraneous hardware > in your system? Any tv cards? I don't have any TV cards in the machine and I was about to say no to extraneous hardware. But I actually have an older SCSI adapter that I've stopped using but which is still in the system. 04:04.0 SCSI storage controller: LSI Logic / Symbios Logic 53c875 (rev 26) I still have difficulty understanding why it will boot with the older kernel and not with the newer. I would expect erratic behaviour with the old kernel if I had a hardware issue.
> 04:04.0 SCSI storage controller: LSI Logic / Symbios Logic 53c875 (rev 26) > > I still have difficulty understanding why it will boot with the older kernel > and not with the newer. I would expect erratic behaviour with the old kernel if > I had a hardware issue. > I have this same card connected to 2 tape libraries (a Compaq StorageWorks TL892 and a Sun StorEdge L9) and with 2.6.27-r8 I can't even boot. Upon booting I get this error message "sym1: SCSI parity error detected: SCR1=1 DBC=1100001e SBCL=ae" printed non-stop on my screen. With 2.6.25-r3, my system is rock solid. Maybe there is a link between Kristian's problem and mine Charles
(In reply to comment #8) > (In reply to comment #7) > I don't have any TV cards in the machine and I was about to say no to > extraneous hardware. But I actually have an older SCSI adapter that I've > stopped using but which is still in the system. Would you be willing to pull the unused card? Any chance of testing with gentoo-sources-2.6.29-r1.
(In reply to comment #10) > (In reply to comment #8) > > (In reply to comment #7) > > I don't have any TV cards in the machine and I was about to say no to > > extraneous hardware. But I actually have an older SCSI adapter that I've > > stopped using but which is still in the system. > > Would you be willing to pull the unused card? > Any chance of testing with gentoo-sources-2.6.29-r1. > Sorry for the late reply - I'm willing to pull the card - I just need to find time to do it. As for a newer kernel - unless you tell me that it contains changes in the udev or the the module for the scsi adapter that might directly resolve the problem, i'd rather wait and do just the adapter for now.
Please reopen when you've had a chance to remove the card. If that doesn't help, here's something else to Compile your kernel with CONFIG_MAGIC_SYSRQ=y Modify your bootloader to pass the "debug" parameter to the kernel Reproduce the hang Press alt+sysrq+m Take photo of screen Upload photo here :) thanks!
(In reply to comment #12) > Please reopen when you've had a chance to remove the card. > If that doesn't help, here's something else to > > Compile your kernel with CONFIG_MAGIC_SYSRQ=y > Modify your bootloader to pass the "debug" parameter to the kernel > Reproduce the hang > Press alt+sysrq+m > Take photo of screen > Upload photo here :) I have removed the card and it unfortunately didn't solve the problem. I will try again with the latest kernel adding the parameters you specified - and last but not least grab a camera :-)
I have added the debug option to the kernel and tried to do the alt+sysrq(prnt scr)+m but the machine is locked hard waiting for the uevents to be processed. I have tried pushing the numlock button to see if the keyboard is responding, but it seems like it will not even do that. Two additional notes: This last attempt was done with gentoo sources2.6.28-gentoo-r5. When I posted this I did do a large number of diffs to establish when the change that broke this was introduced. As far as I could tell this is going from 2.6.23.x to 2.6.24
Can you boot with the debug parameter anyway? It may cause a message to be printed to the screen when it crashes. ALso try disabling the framebuffer if you are using one. Would you be able to set up a serial console? Also please confirm that you aren't using any out-of-kernel modules (packages which you emerge in portage every time you upgrade your kernel, for example proprietary graphics drivers)
(In reply to comment #9) > > 04:04.0 SCSI storage controller: LSI Logic / Symbios Logic 53c875 (rev 26) > > > > I still have difficulty understanding why it will boot with the older kernel > > and not with the newer. I would expect erratic behaviour with the old kernel if > > I had a hardware issue. > > > > I have this same card connected to 2 tape libraries (a Compaq StorageWorks > TL892 and a Sun StorEdge L9) and with 2.6.27-r8 I can't even boot. Upon booting > I get this error message "sym1: SCSI parity error detected: SCR1=1 DBC=1100001e > SBCL=ae" printed non-stop on my screen. With 2.6.25-r3, my system is rock > solid. Maybe there is a link between Kristian's problem and mine > > Charles > I just wan to add that I tried with 2.6.28-r5 today and I have the same problem. Charles
(In reply to comment #15) > Can you boot with the debug parameter anyway? It may cause a message to be > printed to the screen when it crashes. ALso try disabling the framebuffer if > you are using one. > > Would you be able to set up a serial console? > > Also please confirm that you aren't using any out-of-kernel modules (packages > which you emerge in portage every time you upgrade your kernel, for example > proprietary graphics drivers) > The last couple of times I had the debug option enabled but it didn't affect the output at the point where it freezes. As for a serial console - I have the equipment to connect to the serial console - but is there anything that I need in the kernel to actually get a connection on the serial console ? As for proprietary modules, I did have a matrox module in there - but as a result of debugging this kernel problem and upgrading to Xorg 1.5 I have removed the matrox module. This have not changed anything. As a result of upgrading to latest X I noticed that I had not upgraded to the latest gcc. Could the difference between gcc 4.1.2 and 4.3.2 explain why the newer kernels wont boot ?
This has been awhile. Have you made any progress here?
Created attachment 202378 [details] .config file for 2.6.30-gentoo-r4 latest config file that still will not boot
I just tried the latest gentoo kernel (2.6.30-r4) with the same result as before. During boot I noticed that it said : please use probe_mask=0x3f I have then tried adding ide_generic.probe_mask=0x3f on the kernel commandline - but without any change in the result. I should probably have mentioned that the mainboard is a Tyan Thunder K8S Pro (S2882). My next attempt will be to remove the KVM module - as it seems my Opterons do not support it.
Anything to report here?
(In reply to comment #21) > Anything to report here? > No change unfortunately - it still freezes at the point where it says : Waiting for uevents to be processed Can I ask someone to do a diff of the 23 and 30 kernel and tell me if anything seems off ?
I have just tried by changing CONFIG_UEVENT_HELPER_PATH="/sbin/hotplug" to CONFIG_UEVENT_HELPER_PATH="/sbin/udevadm" as I found this to be a difference between a laptop I have and the server. But unfortunately still no change.
Created attachment 208681 [details] Photo of the screen when the machine freezes I have turned udev debug information on and attached a picture of the screen in the hope that someone can spot what might be wrong.
# udevadm monitor --env --kernel --udev & # udevadm trigger ... (lots of stuff here) # udevadm settle at this point it waits for a long time, then: UDEV [1239520736.717056] add /devices/pci0000:00/0000:00:03.3/usb1/1-8/1-8:1.0/host4/target4:0:0/4:0:0:0/block/sdb (block) UDEV_LOG=3 ACTION=add DEVPATH=/devices/pci0000:00/0000:00:03.3/usb1/1-8/1-8:1.0/host4/target4:0:0/4:0:0:0/block/sdb SUBSYSTEM=block DEVNAME=/dev/sdb DEVTYPE=disk SEQNUM=886 ID_VENDOR=Generic- ID_VENDOR_ENC=Generic- ID_VENDOR_ID=0bda ID_MODEL=xD_SDMMC_MS_Pro ID_MODEL_ENC=xD\x2fSDMMC\x2fMS\x2fPro\x20 ID_MODEL_ID=0116 ID_REVISION=1.00 ID_SERIAL=Generic-_xD_SDMMC_MS_Pro_20021111153705700-0:0 ID_SERIAL_SHORT=20021111153705700 ID_TYPE=disk ID_INSTANCE=0:0 ID_BUS=usb ID_USB_INTERFACES=:080650: ID_USB_INTERFACE_NUM=00 ID_USB_DRIVER=usb-storage ID_PATH=pci-0000:00:03.3-usb-0:8:1.0-scsi-0:0:0:0 MAJOR=8 MINOR=16 DEVLINKS=/dev/block/8:16 /dev/disk/by-id/usb-Generic-_xD_SDMMC_MS_Pro_20021111153705700-0:0 /dev/disk/by-path/pci-0000:00:03.3-usb-0:8:1.0-scsi-0:0:0:0 Without my MicroSD adapter it runs just well. I am running kernel 2.6.31 with udev 146-r1.
By the way, it also works fine if a card is present in the adapter. Looks like a driver bug...
Greetings, could you people try booting a recent vanilla kernel? Looking at bug #299287 it seems quite likely that a patch for fbcondecor that is included in gentoo-sources is the cause of the issue.
Thank you George for a good suggestion - I thought I had tried with a vanilla kernel. But looking at grub I can see I haven't. I have compiled a new vanilla kernel - I will let you know when I have had a chance to try it out. Regards Kristian
Feel free to reopen with test results
Upgrading to 2.6.31-gentoo-r6 seemed to solve my problems.
