I have had Apache running on an ARM for well over a year and after a recent upgrade (world) my system around November 20th, Apache no longer works. I have had the server running since August or September -- I did not do any upgrades since then as it became mission critical. Now, at start-up, I immediately get a Segmentation fault. Reproducible: Always Steps to Reproduce: /etc/init.d/apache2 start Actual Results: plug linux # /etc/init.d/apache2 start /etc/init.d/apache2: line 43: 16248 Segmentation fault ${APACHE2} ${APACHE2_OPTS} -t > /dev/null 2>&1 * apache2 has detected an error in your setup: /etc/init.d/apache2: line 43: 16250 Segmentation fault ${APACHE2} ${APACHE2_OPTS} -t plug linux # Expected Results: "started ok" plug apache2 # gcc-config -l [1] armv5tel-softfloat-linux-gnueabi-4.4.5 * [2] armv5tel-softfloat-linux-gnueabi-4.5.1 plug apache2 # plug ~ # gdb /usr/sbin/apache2 GNU gdb (Gentoo 7.0.1 p1) 7.0.1 Copyright (C) 2009 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "armv5tel-softfloat-linux-gnueabi". For bug reporting instructions, please see: <http://bugs.gentoo.org/>... Reading symbols from /usr/sbin/apache2...(no debugging symbols found)...done. (gdb) run Starting program: /usr/sbin/apache2 [Thread debugging using libthread_db enabled] Program received signal SIGSEGV, Segmentation fault. 0x4028fc8c in apr_palloc () from /usr/lib/libapr-1.so.0 (gdb) q A debugging session is active. Inferior 1 [process 8515] will be killed. Quit anyway? (y or n) y plug ~ # /usr/lib/libapr-1.la /usr/lib/libapreq2.so.3.7.1 /usr/lib/libapr-1.so /usr/lib/libaprutil-1.a /usr/lib/libapr-1.so.0 /usr/lib/libaprutil-1.la /usr/lib/libapr-1.so.0.4.2 /usr/lib/libaprutil-1.so /usr/lib/libapreq2.a /usr/lib/libaprutil-1.so.0 /usr/lib/libapreq2.la /usr/lib/libaprutil-1.so.0.3.10 /usr/lib/libapreq2.so plug ~ # ls -la /usr/lib/libapr* -rw-r--r-- 1 root root 227644 Aug 22 11:23 /usr/lib/libapr-1.a -rw-r--r-- 1 root root 965 Aug 22 11:23 /usr/lib/libapr-1.la lrwxrwxrwx 1 root root 17 Aug 22 11:23 /usr/lib/libapr-1.so -> libapr-1.so.0.4.2 lrwxrwxrwx 1 root root 17 Aug 22 11:23 /usr/lib/libapr-1.so.0 -> libapr-1.so.0.4.2 -rwxr-xr-x 1 root root 150496 Aug 22 11:23 /usr/lib/libapr-1.so.0.4.2 -rw-r--r-- 1 root root 59306 Aug 20 23:13 /usr/lib/libapreq2.a -rw-r--r-- 1 root root 993 Aug 20 23:11 /usr/lib/libapreq2.la lrwxrwxrwx 1 root root 18 Aug 20 23:13 /usr/lib/libapreq2.so -> libapreq2.so.3.7.1 lrwxrwxrwx 1 root root 18 Aug 20 23:13 /usr/lib/libapreq2.so.3 -> libapreq2.so.3.7.1 -rwxr-xr-x 1 root root 50860 Aug 20 23:13 /usr/lib/libapreq2.so.3.7.1 -rw-r--r-- 1 root root 166866 Nov 21 18:08 /usr/lib/libaprutil-1.a -rw-r--r-- 1 root root 1020 Nov 21 18:08 /usr/lib/libaprutil-1.la lrwxrwxrwx 1 root root 22 Nov 21 18:08 /usr/lib/libaprutil-1.so -> libaprutil-1.so.0.3.10 lrwxrwxrwx 1 root root 22 Nov 21 18:08 /usr/lib/libaprutil-1.so.0 -> libaprutil-1.so.0.3.10 -rwxr-xr-x 1 root root 113044 Nov 21 18:08 /usr/lib/libaprutil-1.so.0.3.10 plug ~ # eix libapr [I] www-apache/libapreq2 Available versions: (2) (*)2.08-r2 (~*)2.12 {perl} Installed versions: 2.12(2)(11:13:19 PM 08/20/2010)(perl) Homepage: http://httpd.apache.org/apreq/ Description: A library for manipulating client request data via the Apache API plug ~ # I tried to re-emerge www-apache/libapreq2 and it failed complaining: " Bad apache2 binary (/usr/sbin/apache2)" I seem to be in a catch-22: the debugger suggests the problem is with libapreq, but attempting to re-emerger libapreq complains that the Apache binary is "bad". What may I provide to help determine what the problem is?
Note: Just in case this is related to the recent Perl upgrade (although I cannot see how), I had not performed the "perl-cleaner --all" for the upgrade of Perl and am now doing so: 72 packages need to be recompiled.
I ran "perl-cleaner --all" which re-emerged 71 of 72 packages, the last package was www-apache/libapreq2-2.12 failed. I have a bug pending for that: see Bug # 347908. I then tried starting Apache and got the same result: plug apache2 # /usr/sbin/apache2ctl --verbose start /etc/init.d/apache2: line 43: 16614 Segmentation fault ${APACHE2} ${APACHE2_OPTS} -t > /dev/null 2>&1 * apache2 has detected an error in your setup: /etc/init.d/apache2: line 43: 16616 Segmentation fault ${APACHE2} ${APACHE2_OPTS} -t plug apache2 #
I removed dev-libs/apr (emerge -C dev-libs/apr) and then tried running apache: plug ~ # /usr/sbin/apache2 start /usr/sbin/apache2: error while loading shared libraries: libapr-1.so.0: cannot open shared object file: No such file or directory plug ~ # I re-emerged dev-libs/apr-1.3.9 (using >dev-libs/apr-1.3.9 in /etc/portage/package.mask) and it emerged with the following: >>> Completed installing apr-1.3.9 into /var/tmp/portage/dev-libs/apr-1.3.9/image/ strip: armv5tel-softfloat-linux-gnueabi-strip --strip-unneeded -R .comment usr/lib/libapr-1.so.0.3.9 usr/lib/libapr-1.a * QA Notice: The following files contain runtime text relocations * Text relocations force the dynamic linker to perform extra * work at startup, waste system resources, and may pose a security * risk. On some architectures, the code may not even function * properly, if at all. * For more information, see http://hardened.gentoo.org/pic-fix-guide.xml * Please include the following list of files in your report: * TEXTREL usr/lib/libapr-1.so.0.3.9 removing executable bit: usr/lib/libapr-1.la >>> Installing (1 of 1) dev-libs/apr-1.3.9 >>> Auto-cleaning packages... >>> No outdated packages were found on your system. * GNU info directory index is up-to-date. * IMPORTANT: 7 config files in '/etc' need updating. * See the CONFIGURATION FILES section of the emerge * man page to learn how to update config files. plug ~ # I then tried running apache: plug ~ # /usr/sbin/apache2 start Segmentation fault plug ~ # GNU debug shows the problem is with apr: plug ~ # gdb /usr/sbin/apache2 GNU gdb (Gentoo 7.0.1 p1) 7.0.1 Copyright (C) 2009 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "armv5tel-softfloat-linux-gnueabi". For bug reporting instructions, please see: <http://bugs.gentoo.org/>... Reading symbols from /usr/sbin/apache2...(no debugging symbols found)...done. (gdb) run Starting program: /usr/sbin/apache2 [Thread debugging using libthread_db enabled] Program received signal SIGSEGV, Segmentation fault. 0x402907f8 in apr_palloc () from /usr/lib/libapr-1.so.0 (gdb) I tried reading http://hardened.gentoo.org/pic-fix-guide.xml and the prerequisite http://www.gentoo.org/proj/en/hardened/pic-guide.xml and was unable to "fix" what may be the problem, the instructions "Just review the build output and see if the command to compile it was invoked with -fPIC. If not, go fix the build system as you do not need to dig into the source. " presuppose that I know how to "fix the build system". I'll attach the emerge output; perhaps someone reviewing this bug knows what to look for.
Created attachment 257022 [details] full "emerge dev-libs/apr" with (package.mask ">dev-libs/apr-1.3.9") I'll try another emerge and de-activate the package mask allowing the high watermark of dev-libs/apr to install.
Created attachment 257025 [details] emerge of apr-1.4.2 from: plug ~ # FEATURES="keepwork" emerge dev-libs/apr >dev-libs-apr_emerge.txt * WARNING: The FEATURES variable contains one or more values that * should be disabled under normal circumstances: keepwork plug ~ # pwd /root plug ~ # I'll zip up the work area and it that is wanted, I can provide it, or a part thereof.
high watermark of dev-libs/apr (1.4.2) still produces segmentation fault: Program received signal SIGSEGV, Segmentation fault. 0x402910e4 in apr_palloc () from /usr/lib/libapr-1.so.0
I re-emerge apr and emerged the new update to apr-util and my Apache instance suddenly started working again. [I] dev-libs/apr Available versions: (1) 1.3.9!t 1.4.2!t {doc older-kernels-compatibility +urandom} Installed versions: 1.4.2(1)!t(07:01:50 AM 01/16/2011)(urandom -doc -older-kernels-compatibility) Homepage: http://apr.apache.org/ Description: Apache Portable Runtime Library [I] dev-libs/apr-util Available versions: (1) 1.3.9!t ~1.3.9-r1!t 1.3.10!t {berkdb doc freetds gdbm ldap mysql odbc postgres sqlite sqlite3} Installed versions: 1.3.10(1)!t(07:09:20 AM 01/16/2011)(berkdb gdbm ldap mysql postgres -doc -freetds -odbc -sqlite -sqlite3) Homepage: http://apr.apache.org/ Description: Apache Portable Runtime Utility Library
I'm back. After upgrading my system, I have the same problem: plug conf.d # gdb /usr/sbin/apache2 GNU gdb (Gentoo 7.2 p1) 7.2 Copyright (C) 2010 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "armv5tel-softfloat-linux-gnueabi". For bug reporting instructions, please see: <http://bugs.gentoo.org/>... Reading symbols from /usr/sbin/apache2...(no debugging symbols found)...done. (gdb) run Starting program: /usr/sbin/apache2 [Thread debugging using libthread_db enabled] Program received signal SIGSEGV, Segmentation fault. 0x4029c334 in apr_palloc () from /usr/lib/libapr-1.so.0 (gdb) plug conf.d # eix dev-libs/apr [I] dev-libs/apr Available versions: (1) 1.4.4!t {doc elibc_FreeBSD older-kernels-compatibility +urandom +uuid} Installed versions: 1.4.4(1)!t(11:21:06 PM 07/02/2011)(urandom uuid -doc -elibc_FreeBSD -older-kernels-compatibility) Homepage: http://apr.apache.org/ Description: Apache Portable Runtime Library [I] dev-libs/apr-util Available versions: (1) 1.3.11!t {berkdb doc freetds gdbm ldap mysql odbc postgres sqlite sqlite3} Installed versions: 1.3.11(1)!t(11:35:38 PM 07/02/2011)(berkdb gdbm ldap mysql postgres -doc -freetds -odbc -sqlite -sqlite3) Homepage: http://apr.apache.org/ Description: Apache Portable Runtime Utility Library Found 2 matches. plug conf.d # I tried emerge both packages above from dev-libs, but I still get the SEGMENTATION FAULT>
