Hey everyone, About two hours ago i made an emerge --update world on my system, an x86 server running Gentoo (2.6.10) There were ~50 packages emerged, and glibc was in those. After that Apache 2.0.53 starts very greately but writes into log that the process with pid blah-blah-blah got a segfault. Writes about 2-3 times a second. At the beginning i thought that this may be an apache error 'cos it was merged along with glibc, after a while i unmerged 2.0.53 and emerged 2.0.52 - suprisingly error was the same, though before that emerge stuff worked very neatly. I'm terribly sorry for taking your time guys if it's something that i did wrong in here - but it seemed to me that everything was done normally. I was using the same configs with 2.0.52 and 2.0.53 (i symlinked the config locations 'cos 2.0.53 uses different config locations than the prior versions) I'm trying to emerge back onto prior version of glibc, i'll report the progress in here. Reproducible: Always Steps to Reproduce: 1. emerge --update world 2. emerge apache-2.0.53 3. /etc/init.d/apache2 start 4. tail -n 30 /var/log/apache2/error_log Actual Results: no answer from HTTPd, segfault stuff in logs. Expected Results: Httpd should operate normally. No segfault messages should appear.
What version of glibc ? Also, could you please post the ouput of "emerge info". And last but not least, does the old Windows trick of reboot-and-see work for you ? :P
This problem effects me as well. Rebooting did not fix the problem. Here is the information you requested. xolotl:~$ epm -qa|grep apache apache-2.0.52-r1 xolotl:~$ epm -qa|grep glibc glibc-2.3.4.20041102-r1 xolotl:~$ emerge info Portage 2.0.51.19 (default-linux/x86/2004.3, gcc-3.3.5, glibc-2.3.4.20041102-r1, 2.6.10-gentoo-r6 i686) ================================================================= System uname: 2.6.10-gentoo-r6 i686 Intel(R) Pentium(R) 4 CPU 1.60GHz Gentoo Base System version 1.4.16 Python: dev-lang/python-2.3.4-r1 [2.3.4 (#1, Mar 16 2005, 11:17:19)] distcc 2.16 i686-pc-linux-gnu (protocols 1 and 2) (default port 3632) [enabled] ccache version 2.3 [enabled] dev-lang/python: 2.3.4-r1 sys-devel/autoconf: 2.59-r6, 2.13 sys-devel/automake: 1.7.9-r1, 1.8.5-r3, 1.5, 1.4_p6, 1.6.3, 1.9.4 sys-devel/binutils: 2.15.92.0.2-r1 sys-devel/libtool: 1.5.10-r4 virtual/os-headers: 2.6.8.1-r2 ACCEPT_KEYWORDS="x86" AUTOCLEAN="yes" CFLAGS="-O2 -march=pentium4 -fomit-frame-pointer -pipe -fprefetch-loop-arrays -mfpmath=sse" CHOST="i686-pc-linux-gnu" CONFIG_PROTECT="/etc /usr/kde/2/share/config /usr/kde/3.4/env /usr/kde/3.4/share/config /usr/kde/3.4/shutdown /usr/kde/3/share/config /usr/lib/X11/xkb /usr/lib/mozilla/defaults/pref /usr/share/config /usr/share/texmf/dvipdfm/config/ /usr/share/texmf/dvips/config/ /usr/share/texmf/tex/generic/config/ /usr/share/texmf/tex/platex/config/ /usr/share/texmf/xdvi/ /var/qmail/control" CONFIG_PROTECT_MASK="/etc/gconf /etc/terminfo /etc/env.d" CXXFLAGS="-O2 -march=pentium4 -fomit-frame-pointer -pipe -fprefetch-loop-arrays -mfpmath=sse" DISTDIR="/usr/portage/distfiles" FEATURES="autoaddcvs autoconfig ccache distcc distlocks sandbox sfperms" GENTOO_MIRRORS="http://gentoo.seren.com/gentoo/ ftp://ftp.ussg.iu.edu/pub/linux/gentoo http://gentoo.mirrors.pair.com/ ftp://gentoo.mirrors.pair.com/ ftp://gentoo.mirrors.pair.com/" MAKEOPTS="-j6" PKGDIR="/usr/portage/packages" PORTAGE_TMPDIR="/var/tmp" PORTDIR="/usr/portage" SYNC="rsync://rsync.gentoo.org/gentoo-portage" USE="x86 X Xaw3d a52 aac aalib acl acpi aim alsa apache2 apm arts audiofile avi bash-completion bcmath berkdb bindist bitmap-fonts bonobo bzlib cdb cdparanoia cdr crypt cups curl dba dbm dga dio divx4linux dv dvd dvdr dvdread emacs emboss encode esd ethereal evo fam ffmpeg fftw flac flatfile font-server foomaticdb fortran freetype ftp gd gdbm gif gmp gnome gnutls gphoto2 gpm gstreamer gtk gtk2 gtkhtml hal icq imagemagick imap imlib inifile ipv6 jabber java jpeg kde libg++ libwww mad maildir mbox mcal mhash mikmod mime mmap mmx mng mono motif mozilla mp3 mpeg mpi msn ncurses nls nptl offensive ofx ogg oggvorbis openal opengl oscar oss pam pcre pdflib perl php pic pie png pnp posix python qt quicktime readline samba sdl session sharedext sharedmem sockets socks5 sox speex spell spl sse ssl svg svga sysfs sysvipc tcpd tetex theora threads tiff truetype truetype-fonts type1-fonts unicode usb usepackagedmakefiles vcd vhosts wxwindows xine xinerama xml xml2 xmms xosd xpm xprint xsl xv xvid yahoo zlib" Unset: ASFLAGS, CBUILD, CTARGET, LANG, LC_ALL, LDFLAGS, PORTDIR_OVERLAY
when *exactly* does it segfault? maybe only on page requests? please try *remerging* apr and apr-util. also, please try running apache *without* any 3rdparty modules, like mod_php / subversion's mod_dav_svn etc. Does it still segfault?
Apache segfaults on page load. I remerged apr and apr-util, and then remerged Apache for good measure, and it still segfaults. No third party modules are being loaded, only the defaults.
Hey sorry for not answering the topic. Well now, everything that was written above doesn't work. In general, in 2.6.10 kernel we get a serial port error on the kernel log, coming up with the same frequency as the apache errors. Rolling back to the previous versions on glibc, gcc etc. didn't help. I have rebuilt a kernel it didn't help, in addition i installed the newer version of kernel 2.6.11 - no other effect than the serial port error disappearance, (maybe because the we have switched off serial port support due to not using it) THE ONLY POSSIBLE SOLUTION TO ME WAS REBUILDING A PACKAGE FROM SOURCE BY HAND, and hey ? it helped. From all this stuff i belive i can make a conclusion that, a problem is explictly in the building process using the new glibc libraries. I couldn't try to rebuild apache2.0.52/53 from emerge because configure brought me a whole bunch of errors, the best of which was that my compiler can't make executables. And that concerned a lot of packages starting from baselayout to automake:) but that neat stuff is for the second topic. I was recovering my system from that glibc stuff using a livecd downloaded yesterday. In addition, here is emerge --info: Portage 2.0.51-r2 (default-linux/x86/2004.3, gcc-3.3.5, glibc-2.3.4.20040808-r1, 2.6.11-gentoo i686) ================================================================= System uname: 2.6.11-gentoo i686 Intel(R) Pentium(R) 4 CPU 2.80GHz Gentoo Base System version 1.4.16 Autoconf: sys-devel/autoconf-2.59-r6,sys-devel/autoconf-2.13 Automake: sys-devel/automake-1.7.9-r1,sys-devel/automake-1.8.5-r3,sys-devel/automake-1.5,sys-devel/automake-1.4_p6,sys-devel/automake-1.6.3,sys-devel/automake-1.9.5 Binutils: sys-devel/binutils-2.15.92.0.2-r1 Headers: sys-kernel/linux-headers-2.4.22-r1 Libtools: sys-devel/libtool-1.5.10-r4 ACCEPT_KEYWORDS="x86" AUTOCLEAN="yes" CFLAGS="-O2 -mcpu=i686 -fomit-frame-pointer --force" CHOST="i386-pc-linux-gnu" COMPILER="" CONFIG_PROTECT="/etc /usr/X11R6/lib/X11/xkb /usr/kde/2/share/config /usr/kde/3/share/config /usr/share/config /var/bind /var/qmail/control" CONFIG_PROTECT_MASK="/etc/gconf /etc/terminfo /etc/env.d" CXXFLAGS="-O2 -mcpu=i686 -fomit-frame-pointer --force" DISTDIR="/usr/portage/distfiles" FEATURES="autoaddcvs autoconfig ccache distlocks sandbox sfperms" GENTOO_MIRRORS="http://gentoo.osuosl.org http://distro.ibiblio.org/pub/Linux/distributions/gentoo" MAKEOPTS="-j2" PKGDIR="/usr/portage/packages" PORTAGE_TMPDIR="/var/tmp" PORTDIR="/usr/portage" PORTDIR_OVERLAY="" SYNC="rsync://rsync.gentoo.org/gentoo-portage" USE="x86 X apm arts avi berkdb bitmap-fonts crypt cups curl emboss encode font-server foomaticdb fortran freetds gdbm gif gnome gpm gtk gtk2 imagemagick imlib ipv6 java jpeg junit kde ldap libg++ libwww mad mikmod motif mp3 mpeg mysql ncurses nls odbc oggvorbis opengl oss pam pdflib perl png python qt quicktime readline ruby samba sdl slang spell sqlite ssl svga tcltk tcpd tiff truetype truetype-fonts type1-fonts xml2 xmms xv zlib" And another trick, my apache starts to segfault straight from the beginning, it doesn't wait till the request comes.
Now mine crashes sometimes at startup, sometimes at page-load. Each time I rebuild it does either one or the other, but the behavior never changes until I rebuild. I.e., one build crashes only at page-load, another build crashes every time at start-up. (FYI, I've been rebuilding apr and apr-util each time I rebuilt, and I used gcc 3.3.) Here is a backtrace from the at-startup crash. I can't get one for the at-page-load crash because gdb won't give me useful stack data for threads, and at this moment I'm not motivated enough to find a way around this limitation since I've learned that 2.0.53 works fine. (gdb) bt #0 0xb7c2d028 in strcmp () from /lib/tls/libc.so.6 #1 0x0806f0a9 in ap_process_resource_config (s=0x80c7288, fname=0x80f30e8 "/usr/lib/apache2/conf/modules.d/*.conf", conftree=0xbffff344, p=0x80ad0a8, ptemp=0x80d9158) at config.c:1598 #2 0x08082e00 in include_config (cmd=0xbffff5c0, dummy=0xbffff428, name=0x80f3048 "conf/modules.d/*.conf") at core.c:2298 #3 0x0806d234 in invoke_cmd (cmd=0x809e108, parms=0xbffff5c0, mconfig=0xbffff428, args=0x80de185 "") at config.c:691 #4 0x0806e862 in execute_now (cmd_line=0x80f3000 "Include", args=0x80de170 "conf/modules.d/*.conf", parms=0xbffff5c0, p=0x80ad0a8, ptemp=0x80d9158, sub_tree=0xbffff428, parent=0x0) at config.c:1334 #5 0x0806dd6d in ap_build_config_sub (p=0x80ad0a8, temp_pool=0x80d9158, l=0x80de168 "Include conf/modules.d/*.conf", parms=0xbffff5c0, current=0xbffff484, curr_parent=0xbffff480, conftree=0x80a4034) at config.c:913 #6 0x0806e33a in ap_build_config (parms=0xbffff5c0, p=0x80ad0a8, temp_pool=0x80d9158, conftree=0x80a4034) at config.c:1126 #7 0x0806ed4b in process_resource_config_nofnmatch (s=0x80c7288, fname=0x80c7b20 "/etc/apache2/conf/apache2.conf", conftree=0x80a4034, p=0x80ad0a8, ptemp=0x80d9158, depth=0) at config.c:1511 #8 0x0806eead in ap_process_resource_config (s=0x80c7288, fname=0x80c7b20 "/etc/apache2/conf/apache2.conf", conftree=0x80a4034, p=0x80ad0a8, ptemp=0x80d9158) at config.c:1547 #9 0x0806fadc in ap_read_config (process=0x80ab120, ptemp=0x80d9158, filename=0x8099d60 "/etc/apache2/conf/apache2.conf", conftree=0x80a4034) at config.c:1890 #10 0x080724dd in main (argc=6, argv=0xbffff884) at main.c:531 (gdb) up #1 0x0806f0a9 in ap_process_resource_config (s=0x80c7288, fname=0x80f30e8 "/usr/lib/apache2/conf/modules.d/*.conf", conftree=0xbffff344, p=0x80ad0a8, ptemp=0x80d9158) at config.c:1598 1598 if (strcmp(dirent.name, ".") (gdb) print dirent.name $5 = 0x20 <Address 0x20 out of bounds> (gdb) print dirent $4 = {pool = 0x80ad0a8, valid = 33562624, protection = 6, filetype = APR_NOFILE, user = 135213149, group = 248697717, inode = 974006, device = 163208757248, nlink = 17, size = 41, csize = 16, atime = 580735722366906600, mtime = 578402377095181080, ctime = 579992764830184212, fname = 0x80f305d "", name = 0x20 <Address 0x20 out of bounds>, filehand = 0x80ad0a8}
I'm having some trouble with apache as well, which might be related: $ /usr/sbin/apache2 Segmentation fault $ emerge info Portage 2.0.51.19 (default-linux/ppc/2005.0, gcc-3.4.3, glibc-2.3.4.20041102-r1, 2.6.11 ppc) ================================================================= System uname: 2.6.11 ppc 7447A, altivec supported Gentoo Base System version 1.4.16 Python: dev-lang/python-2.3.4-r1 [2.3.4 (#1, Feb 12 2005, 13:40:52)] dev-lang/python: 2.3.4-r1 sys-devel/autoconf: 2.59-r6, 2.13 sys-devel/automake: 1.7.9-r1, 1.8.5-r3, 1.5, 1.4_p6, 1.6.3, 1.9.4 sys-devel/binutils: 2.15.90.0.3-r4 sys-devel/libtool: 1.5.10-r4 virtual/os-headers: 2.4.22 ACCEPT_KEYWORDS="ppc" AUTOCLEAN="yes" CFLAGS="-O2 -pipe -mcpu=7400 -maltivec -mabi=altivec" CHOST="powerpc-unknown-linux-gnu" CONFIG_PROTECT="/etc /usr/kde/2/share/config /usr/kde/3/share/config /usr/lib/X11/xkb /usr/lib/mozilla/defaults/pref /usr/share/config /usr/share/texmf/dvipdfm/config/ /usr/share/texmf/dvips/config/ /usr/share/texmf/tex/generic/config/ /usr/share/texmf/tex/platex/config/ /usr/share/texmf/xdvi/ /var/qmail/control" CONFIG_PROTECT_MASK="/etc/gconf /etc/terminfo /etc/env.d" CXXFLAGS="-O2 -pipe -mcpu=7400 -maltivec -mabi=altivec" DISTDIR="/usr/portage/distfiles" FEATURES="autoaddcvs autoconfig ccache distlocks noclean sandbox sfperms" GENTOO_MIRRORS="http://ftp.rhnet.is/pub/gentoo" LC_ALL="en_US.utf8" MAKEOPTS="-j2" PKGDIR="/usr/portage/packages" PORTAGE_TMPDIR="/var/tmp" PORTDIR="/usr/portage" PORTDIR_OVERLAY="/usr/portage/local" SYNC="rsync://ftp.rhnet.is/gentoo-portage" USE="X aalib aim alsa altivec apache2 bash-completion berkdb bitmap-fonts bzlib cdparanoia cdr cjk crypt cups curl directfb divx4linux doc dvd emboss encode esd exif fam flac fortran gd gdbm gif gnome gphoto2 gpm gstreamer gtk gtk2 guile icq imagemagick imlib innodb jabber jpeg latex libwww mad matroska motif mozilla mp3 mpeg msn mysql ncurses network nls nodrm offensive oggvorbis opengl oss pam perl png postgres ppc python qt readline rtc ruby samba sdl speex spell ssl svg tcltk tcpd tetex theora tiff truetype truetype-fonts type1-fonts unicode v4l v4l2 xml xml2 xmms xprint xv xvid yahoo zlib linguas_en_US.utf8" Unset: ASFLAGS, CBUILD, CTARGET, LANG, LDFLAGS
screw it, using 1.3 again.
I'm getting this same problem, apache 2.0.53 segfaulting on page load. Some pages load fine (text-only) but pages with images seems to segfault. Seems like nothing has happened with this bug for two months, does anyone have a fix? I didn't have this problem yesterday when I used apache for hours developing a website, it started today when I upgraded to 2.0.54 (I've downgraded to 2.0.53 again, but I still have the same problem). I have the same glibc version as I had when it worked, so that can't be the problem. Here is my info: Portage 2.0.51.21-r1 (default-linux/x86/2005.0, gcc-3.4.3-20050110, glibc-2.3.5-r0, 2.6.11-tpc1-tpc01 i686) ================================================================= System uname: 2.6.11-tpc1-tpc01 i686 Mobile Intel(R) Pentium(R) 4 - M CPU 2.00GHz Gentoo Base System version 1.6.11 dev-lang/python: 2.3.5 sys-apps/sandbox: 1.2.5-r1 sys-devel/autoconf: 2.13, 2.59-r6 sys-devel/automake: 1.4_p6, 1.5, 1.6.3, 1.7.9-r1, 1.8.5-r3, 1.9.5 sys-devel/binutils: 2.15.92.0.2-r8 sys-devel/libtool: 1.5.16 virtual/os-headers: 2.6.11 ACCEPT_KEYWORDS="x86 ~x86" AUTOCLEAN="yes" CBUILD="i686-pc-linux-gnu" CFLAGS="-march=pentium4 -O2 -pipe" CHOST="i686-pc-linux-gnu" CONFIG_PROTECT="/etc /usr/kde/2/share/config /usr/kde/3.4/env /usr/kde/3.4/share/config /usr/kde/3.4/shutdown /usr/kde/3/share/config /usr/lib/X11/xkb /usr/share/config /var/qmail/control" CONFIG_PROTECT_MASK="/etc/gconf /etc/terminfo /etc/env.d" CXXFLAGS="-march=pentium4 -O2 -pipe" DISTDIR="/usr/portage/distfiles" FEATURES="autoconfig distlocks fixpackages sandbox sfperms strict" GENTOO_MIRRORS="http://ftp.du.se/pub/os/gentoo/ http://mirror.pudas.net/gentoo/ http://distfiles.gentoo.org" LANG="sv_SE.ISO-8859-1" LC_ALL="sv_SE.ISO-8859-1" LINGUAS="sv" MAKEOPTS="-j4" PKGDIR="/usr/portage/packages" PORTAGE_TMPDIR="/var/tmp" PORTDIR="/usr/portage" PORTDIR_OVERLAY="/usr/local/portage/tpwch" SYNC="rsync://rsync3.se.gentoo.org/gentoo-portage" USE="x86 X aalib acpi alsa apache2 avi bash-completion berkdb bitmap-fonts bzlib crypt cups curl curlwrappers dba dga divx4linux dvd encode fbcon ffmpeg flac ftp gd gif gpm gtk gtk2 imagemagick java javascript jpeg kde kdeenablefinal libcaca live lua mad mikmod mime mmx mmxext motif mozilla mp3 mpeg mysql ncurses nls nptl nptlonly offensive ogg oggvorbis opengl pcre pda pdflib perl php pic png posix python qt quicktime readline real rtc samba sdl session sockets spell sse sse2 ssl theora threads tidy truetype truetype-fonts type1-fonts usb userlocales vorbis win32codecs wxwindows xml xml2 xmms xosd xpm xv xvid zeroconf zlib linguas_sv userland_GNU kernel_linux elibc_glibc" Unset: ASFLAGS, CTARGET, LDFLAGS
I have "resolved" same probleme : Downgrade "dev-libs/apr-0.9.6-r3" to "dev-libs/apr-0.9.6-r2" Downgrade "net-www/apache-2.0.54" to "net-www/apache-2.0.53" Downgrade apr before apache. Downgrade only apache don't work.
I think this may be related to a message that pops up briefly when emerging apache about having to rebuild any add-on modules. I had to re-emerge mysqltool and mod_php before it would work. I haven't figured out whether I can get it to work with the latest versions of everything, but simply reverting versions wasn't enough.
I ran into the same annoying problem. As suggested in Comment #10 I downgraded apr and then tried downgrading apache2 itself. The latter step did not work as emerge complained about some missing directory. Anyway, downgrading apr, and re-emerging apache-2.0.54 afterwards plus rebuilding the apache modules was enough to get back to a working version.
...forgot to mention that I also rebuild the apr-utils after downgrading apr.
Dang! After restarting apache server it happily began segfaulting again. Guess I have to start all over again. This is really annoying! : (
Apparently the Apache crashes on my machine were caused be a crazed mod_php. After cleaning out any and all packages related to apache, removing old symlinks as suggested in the emerge process and reemerging everything I got a running webserver again. Here the versions that seem to work: net-www/apache: 2.0.54-r5 dev-libs/apr: 0.9.6-r3 dev-libs/apr-util: 0.9.6-r2 dev-php/mod_php: 4.3.11-r2
I believe these may have been symptoms of the removal of Large File Support which caused many segfaults for many users. Check http://www.gentoo.org/proj/en/apache/troubleshooting.xml and see if the instructions there work. If you are still having issues with segfaults after going through that guide, feel free to reopen this bug, and/or join us in #gentoo-apache on irc.freenode.net.
