Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 132530 - /sbin/rc: system hangs for 15-30 minutes while doing rm -rf $(ls -d1 ...
Summary: /sbin/rc: system hangs for 15-30 minutes while doing rm -rf $(ls -d1 ...
Status: RESOLVED WORKSFORME
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: [OLD] Core system (show other bugs)
Hardware: All Linux
: High normal (vote)
Assignee: Gentoo's Team for Core System packages
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2006-05-06 23:51 UTC by Eugene Pavlovsky
Modified: 2006-11-14 01:10 UTC (History)
0 users

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Eugene Pavlovsky 2006-05-06 23:51:56 UTC
Sometimes when I shutdown, my system kinda hangs after the "Setting hardware clock using the system clock" message. It does something with the HDD (LED flashing wildly) for about 15-30 minutes, then finally continues to the next step...
If I forcibly turn off the computer, then on next boot, after the "Setting system clock using the hardware clock", the same stuff happens. It's very depressing...
PC is IBM ThinkPad R51, the root filesystem is ReiserFS 3 (noatime,notail options). HDD is Fujitsu MHT2060AT. I'm using the PIIx IDE driver.
Comment 1 SpanKY gentoo-dev 2006-05-07 01:55:10 UTC
either a kernel or a hardware bug
Comment 2 Daniel Drake (RETIRED) gentoo-dev 2006-05-07 03:38:27 UTC
Are there any BIOS updates available?
Comment 3 Eugene Pavlovsky 2006-05-09 09:14:43 UTC
I already have the latest BIOS (3.20), my kernel is 2.6.16 with swsusp2 and radeonfb (D2 suspend) patches.
Comment 4 Daniel Drake (RETIRED) gentoo-dev 2006-05-10 08:23:21 UTC
And this has always happened? it wasn't introduced after a kernel upgrade or something like that?
Comment 5 Eugene Pavlovsky 2006-05-10 09:47:22 UTC
I don't remember the first time it happened, that was probably several months ago.  This usually occured at "shutdown -h now" before I went to sleep, so I just give it some time and in 15-30 minutes (maybe faster sometimes) it shuts down. Lately I interrupted the process (i.e. emergency unmount SysRq-U then hold power button for 4 seconds). I've booted it up immediately, and this freeze it happened on boot. I gave it several minutes of wait then went to do some chores, when I came back it was already booted up.

I update kernels regularly (as soon as new stable releases come out), I didn't change anything IDE or filesystems related in kernel config.
Comment 6 Daniel Drake (RETIRED) gentoo-dev 2006-05-15 08:47:42 UTC
Can you reproduce this by /etc/init.d/clock stop during usual system operation? If so, could you add some debugging into the init script so that you know exactly which command (and its parameters) cause the delay?
Comment 7 Eugene Pavlovsky 2006-05-15 13:44:54 UTC
Tried /etc/init.d/clock stop. Worked ok. I haven't had this problem since the time I posted this bug.
Comment 8 Daniel Drake (RETIRED) gentoo-dev 2006-05-21 08:37:26 UTC
How about if you repeatedly start and stop the clock service?
Comment 9 Eugene Pavlovsky 2006-05-21 12:01:31 UTC
Tried stop/start several hundred times (in a loop). Didn't reproduce the bug.
Yesterday this strange disk activity happened again, on shutdown. There was a message from /etc/init.d/clock, then "shutting down wpa_supplicant on eth1", then the system has frozen. So I guess maybe the /etc/init.d/clock is not the culprit. Then again, it may be. I tried to use magic sysrq key, the SysRq-? worked, displaying the list of magic sysrq commands, but show memory / processes etc. magic sysrq commands didn't work at all... As this bug is hard to reproduce, I think some logging should be added somewhere so when this bug happens again, we'll get some information logged about it. I don't know where or what to log, though.
Comment 10 Daniel Drake (RETIRED) gentoo-dev 2006-05-21 13:14:51 UTC
Disabling RC_PARALLEL_STARTUP in /etc/conf.d/rc might help make the problem more obvious
As for sysrq not giving output, you probably just need to increase log level. Press alt+sysrq+9 first.
Comment 11 Eugene Pavlovsky 2006-05-27 10:12:37 UTC
One question about sysrq - even with sysrq+9, it only writes the info (mem/tasks) to syslog, not to screen, how to make it print it to screen?

About this bug - I finally found the problem! It's not in /etc/init.d/clock, but in /sbin/rc itself. When I ran into the problem again, I did sysrq-f (oom_kill) and it killed an "rm -rf $(ls -d1 ..." process called from /sbin/rc. I found this code (it's there in two places) in /sbin/rc. Here's the process that hangs for a long time:

        # Clear $svcdir from stale entries, but leave the caches around, as it
        # should help speed things up a bit
        rm -rf $(ls -d1 "${svcdir}/"* 2>/dev/null | \
                 grep -ve '\(depcache\|deptree\|envcache\)')

I've changed the bug title to reflect this. I hope you can tell me how this can be fixed. Where are these "${svcdir}/"* ?
Comment 12 Daniel Drake (RETIRED) gentoo-dev 2006-05-29 05:34:44 UTC
Can't comment on the sysrq thing - it works here. Maybe try it on a real live VT rather than when the system is rebooting to see if that makes a difference?

Please post "emerge --info" output
Comment 13 Eugene Pavlovsky 2006-05-29 07:26:46 UTC
Tried SysRq+9 then SysRq+P on a real VT while system is running. It showed "SysRq: Show Regs" but nothing else - it went to syslog. Maybe syslog somehow captures this stuff exclusively when it's running?

My emerge --info:
Portage 2.1_rc3-r1 (default-linux/x86/2005.1, gcc-3.4.6, glibc-2.4-r3, 2.6.16 i686)
=================================================================
System uname: 2.6.16 i686 Intel(R) Pentium(R) M processor 1.70GHz
Gentoo Base System version 1.12.0
dev-lang/python:     2.4.3-r1
dev-python/pycrypto: 2.0.1-r5
dev-util/ccache:     [Not Present]
dev-util/confcache:  [Not Present]
sys-apps/sandbox:    1.2.18.1
sys-devel/autoconf:  2.13, 2.59-r7
sys-devel/automake:  1.4_p6, 1.5, 1.6.3, 1.7.9-r1, 1.8.5-r3, 1.9.6-r2
sys-devel/binutils:  2.16.1-r2
sys-devel/libtool:   1.5.22
virtual/os-headers:  [Not Present]
ACCEPT_KEYWORDS="x86 ~x86"
AUTOCLEAN="yes"
CBUILD="i686-pc-linux-gnu"
CFLAGS="-march=pentium-m -O3 -pipe -mmmx -msse -msse2 -mfpmath=sse"
CHOST="i686-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/lib/mozilla/defaults/pref /usr/share/X11/xkb /var/qmail/alias /var/qmail/control"
CONFIG_PROTECT_MASK="/etc/gconf /etc/revdep-rebuild /etc/terminfo /etc/env.d"
CXXFLAGS="-march=pentium-m -O3 -pipe -mmmx -msse -msse2 -mfpmath=sse"
DISTDIR="/usr/portage/distfiles"
FEATURES="autoconfig distlocks metadata-transfer sandbox sfperms strict"
GENTOO_MIRRORS="ftp://mirror.aiya.ru/pub/gentoo/ ftp://ftp.linux.ee/pub/gentoo/distfiles/ ftp://ftp.rhnet.is/pub/gentoo/"
LANG="en_US.UTF-8"
LC_ALL=""
LINGUAS="en ru"
MAKEOPTS="-j2"
PKGDIR="/usr/portage/packages"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --delete-after --stats --timeout=180 --exclude='/distfiles' --exclude='/local' --exclude='/packages'"
PORTAGE_TMPDIR="/var/tmp"
PORTDIR="/usr/portage"
PORTDIR_OVERLAY="/usr/local/portage"
SYNC="rsync://rsync.europe.gentoo.org/gentoo-portage"
USE="x86 X aac acpi alsa anthy apache2 async avi berkdb bitmap-fonts bzip2 cairo canna caps cardbus cjk cli crypt cups curl dba dlloader dri dts dvd dvdr emboss encode fbcon font-server fontconfig foomaticdb fortran ftp gd gdbm gif gimpprint gmp gnome gpm gtk gtk2 guile iconv ipv6 isdnlog jack javascript jpeg ladspa lcd libg++ libwww logrotate lua lzo mad maildir matroska mikmod mmap mmx motif mozdevelop moznocompose moznoirc mozsvg mp3 mpeg mpm-worker musepack mysql ncurses nls no-old-linux nptl nptlonly nsplugin offensive ogg oggvorbis opengl oscar pam pcre pdf php png ppds pppd python qt quicktime readline reflection rtc sdk sdl session spell spl sse sse2 ssl svgz tcpd threads tiff truetype truetype-fonts type1-fonts unicode userlocales utf8 vorbis wifi win32codecs xml xml2 xorg xv xvmc zlib elibc_glibc input_devices_keyboard input_devices_mouse input_devices_synaptics input_devices_evdev kernel_linux linguas_en linguas_ru userland_GNU video_cards_radeon"
Unset:  CTARGET, EMERGE_DEFAULT_OPTS, INSTALL_MASK, LDFLAGS, PORTAGE_RSYNC_EXTRA_OPTS
Comment 14 Daniel Drake (RETIRED) gentoo-dev 2006-05-29 07:52:29 UTC
When you did alt+sysrq+9 did it print "Changing loglevel to 9" on the console?

I seem to remember having a problem with that actually, when I had both USB and PS/2 keyboards attached, the loglevel change only worked on one of them..weird.

You can alternatively do this: echo 9 > /proc/sysrq-trigger

Anyway, reassigning to base-system. Something in /sbin/rc is causing this hang
Comment 15 Eugene Pavlovsky 2006-05-29 09:16:08 UTC
No, it didn't print anything. I only have one keyboard (it's a notebook).
Doing "echo 9 > /proc/sysrq-trigger" worked, SysRq-M dumped lots of info to console.

Yes, /sbin/rc should be debugged and fixed. I'll try doing smth myself, too.
Comment 16 SpanKY gentoo-dev 2006-06-07 10:14:51 UTC
change the line to 'rm -rvf' and see what gets printed out
Comment 17 Eugene Pavlovsky 2006-11-13 02:23:37 UTC
I've a new notebook (sold the one I had this bug on) and haven't had this bug occur to me on the fresh gentoo install i've made on it. Can probably resolve the bug
Comment 18 Roy Marples (RETIRED) gentoo-dev 2006-11-13 04:39:24 UTC
Fair enough.
Unless you're a vserver then the code in question in /sbin/rc won't get called anyway using baselayout-1.13
Comment 19 Eugene Pavlovsky 2006-11-14 01:10:22 UTC
Thanks