This morning I was doing some work on my alpha box and noticed slowness for some
applications (screen, uptime, /etc/init.d/klive) and no response from others
(/etc/init.d/apache2, /etc/init.d/jabber, /etc/init.d/mysql). I looked at the
load averages and they kept climbing very rapidly.
08:07:40 up 17 days, 22:34, 3 users, load average: 55.45, 46.99, 42.09
The system was so bogged down that I couldn't run many commands, so I just
rebooted. I checked the logs and found nothing useful in the system log or the
mysql logs. However in the kernel log I did find this:
Nov 1 05:04:53 [kernel] [5733083.383995] scheduling while atomic:
I'm not sure how or why it happened or if I can expect it to happen again. I
just upgraded to mysql-4.1.14 about 3 days ago and I'm not sure if that had
something to do with it. Can someone tell me if the cause of this is a kernel
bug, a mysql bug, or both?
Reproducible: Didn't try
Steps to Reproduce:
System became unresponsive, forcing me to reboot.
It should have kept running smoothly without bringing my load average up to 55.45.
Portage 18.104.22.168-r3 (default-linux/alpha/2005.0, gcc-3.3.2,
glibc-22.214.171.12441102-r1, 126.96.36.199 alpha)
System uname: 188.8.131.52 alpha EV56
Gentoo Base System version 1.6.13
dev-lang/python: 2.3.5-r2, 2.4.2
sys-devel/autoconf: 2.13, 2.59-r6
sys-devel/automake: 1.4_p6, 1.5, 1.6.3, 1.7.9-r1, 1.8.5-r3, 1.9.6-r1
CFLAGS="-mieee -O3 -mcpu=ev4"
CONFIG_PROTECT="/etc /usr/kde/2/share/config /usr/kde/3/share/config
/usr/lib/X11/xkb /usr/lib/mozilla/defaults/pref /usr/share/config
CONFIG_PROTECT_MASK="/etc/gconf /etc/terminfo /etc/env.d"
CXXFLAGS="-mieee -O3 -mcpu=ev4"
FEATURES="autoconfig distlocks sandbox sfperms strict"
USE="alpha X apache2 arts berkdb bitmap-fonts crypt cups curl eds encode esd fam
font-server foomatic foomaticdb fortran gd gdbm gif gnome gpm gstreamer gtk gtk2
imlib jabber jpeg kde libg++ libwww mad mikmod motif mozilla mp3 mpeg mysql
ncurses nls nptl nptlonly ogg oggvorbis opengl oss pam pdflib perl png postgres
python qt quicktime readline sdl spell ssl tcpd tiff truetype truetype-fonts
type1-fonts udev vorbis xml2 xmms xv zlib linguas_en userland_GNU kernel_linux
Unset: ASFLAGS, CTARGET, LANG, LC_ALL
Created attachment 71870 [details]
Kernel Config 184.108.40.206
Created attachment 71871 [details]
MySQL Config File
I think there are known problems with 2.6.13 on alpha.
You should try 2.6.14 if possible, but I'm not sure if those issues are resolved
I just re-read the 2.6.14 changelog and there are some atomic fixes for alpha
in 2.6.14, but unfortunately 2.6.14 doesn't compile on alpha. However
2.6.14-git4 has a 2.6.14 compile fix. I'll try the fix; unfortunately
it will be hard to tell if it worked because I ran 220.127.116.11 for 17 days without
a problem, and I don't know how to reproduce the error. Thanks for the advice!
I applied the patch and got 2.6.14 to compile. However the "scheduling while
atomic" thing is happening again. See output below:
pan ~ # uptime
07:49:27 up 2 days, 18:54, 1 user, load average: 42.17, 42.05, 42.01
pan ~ # tail /var/log/kernel/current
Nov 4 05:25:13 [kernel] [4426177.025788] scheduling while atomic:
pan ~ # cat /proc/version
Linux version 2.6.14 (root@pan) (gcc version 3.3.2 20040119 (Gentoo Linux
3.3.2-r7, propolice-3.3-7)) #1 Tue Nov 1 12:05:18 EST 2005
I read the ChangeLog for 2.6.14-git7 and there are a lot of atomicity fixes.
I'll try that kernel and report back if this happens again.
I haven't gotten the 'scheduling while atomic' error in over a week. Upgrading
to the latest kernel solved the problem.