Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 19837 - Apache starts once, but won't start 2nd time until "emerge mod_php" again
Summary: Apache starts once, but won't start 2nd time until "emerge mod_php" again
Status: VERIFIED FIXED
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: [OLD] Server (show other bugs)
Hardware: x86 Linux
: High critical (vote)
Assignee: PHP Bugs
URL: http://forums.gentoo.org/viewtopic.ph...
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2003-04-23 10:27 UTC by Nathan Mahon
Modified: 2003-09-25 18:27 UTC (History)
1 user (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
output from strace -> SEGFAULT -> just using -D PHP4 this time, too. (strace.out,111.09 KB, text/plain)
2003-04-25 00:48 UTC, Nathan Mahon
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Nathan Mahon 2003-04-23 10:27:24 UTC
Apache works fine, after a fresh ebuild. I can use phpmyadmin and squirrelmail,
under SSL. after the first restart of apache, apache does neither start properly
or complain to console or logfile. Apache does start if i remove the -D PHP4
from the start script, which lead me to believe mod_php was to blame... though
it worked after the initial build.  I can "emerge mod_php" and suddenly it works
fine ... until the next apache restart.

Reproducible: Always
Steps to Reproduce:
1. /etc/init.d/apache stop
2. /etc/init.d/apache start


Actual Results:  
while monitoring with: ( while sleep 1; do ps acx | grep apache; done )&
I see apache start it's initial process, which lasts for ~7 seconds. it then
changes into a second process which lasts for 3 seconds.  after that, there are
no apache processes running at all.

Expected Results:  
I expected apache to start and serve pages like it does initially.

Portage 2.0.47-r10 (default-x86-1.4, gcc-3.2.2, glibc-2.3.1-r4)
=================================================================
System uname: 2.4.20 i586 AMD-K6(tm) 3D processor
GENTOO_MIRRORS="http://gentoo.oregonstate.edu/ http://distro.ibiblio.org/pub/Lin
ux/distributions/gentoo"
CONFIG_PROTECT="/etc /var/qmail/control /usr/share/config /usr/kde/2/share/confi
g /usr/kde/3/share/config /var/bind /usr/X11R6/lib/X11/xkb"
CONFIG_PROTECT_MASK="/etc/gconf /etc/env.d"
PORTDIR="/usr/portage"
DISTDIR="/usr/portage/distfiles"
PKGDIR="/usr/portage/packages"
PORTAGE_TMPDIR="/var/tmp"
PORTDIR_OVERLAY=""
USE="x86 oss apm crypt cups gif imlib libg++ mikmod mmx ncurses pdflib quicktime
 spell xmms xv zlib alsa gdbm berkdb slang readline svga X gpm tcpd pam oggvorbi
s gtk motif opengl mozilla -java -gnome -kde -arts -nls -postgres jpeg png truet
ype xml xml2 avi aalib mpeg encode fbcon 3dnow sdl python perl qt mysql imap lib
www maildir sasl ssl"
COMPILER="gcc3"
CHOST="i586-pc-linux-gnu"
CFLAGS="-march=k6-2 -O3 -pipe -fomit-frame-pointer"
CXXFLAGS="-march=k6-2 -O3 -pipe -fomit-frame-pointer"
ACCEPT_KEYWORDS="x86"
MAKEOPTS="-j2"
AUTOCLEAN="yes"
SYNC="rsync://rsync.gentoo.org/gentoo-portage"
FEATURES="sandbox ccache"
========================================================
uname -a 
Linux kavlax.vaevictus.net 2.4.20 #1 SMP Tue Apr 8 11:11:15 CDT 2003 i586
AMD-K6(tm) 3D processor AuthenticAMD GNU/Linux
========================================================
root@n8linux vae #  emerge -p apache mod_php mod_perl mod_ssl

These are the packages that I would merge, in order:

Calculating dependencies ...done!
[ebuild   R  ] net-www/apache-1.3.27-r3  
[ebuild   R  ] dev-php/mod_php-4.3.1  
[ebuild   R  ] dev-perl/mod_perl-1.27-r1  
[ebuild   R  ] net-www/mod_ssl-2.8.14  

========================================================
Comment 1 Nathan Mahon 2003-04-23 10:31:46 UTC
[Wed Apr 23 00:13:28 2003] [notice] Apache/1.3.27 (Unix)  (Gentoo/Linux) mod_ssl/2.8.14 OpenSSL/0.9.6i mod_perl/1.27 PHP/4.3.1 configured -- resuming normal operations

btw, you might notice the root@n8linux bit on the emerge -p line above, which is indeed from a separate system, but is exactly the same output from kavlax.  So don't be alarmed by that.  Sorry about any confusion there.
Comment 2 Robin Johnson archtester Gentoo Infrastructure gentoo-dev Security 2003-04-23 11:50:41 UTC
Can you reproduce this with either of mod_perl or mod_ssl disabled ?

The fact that Apache seems to start initially, but dies very quickly indicates that it is dying for some reason. 

If you can, I recommend getting the output of strace -ff and digging thru the output files to find the one of the actual httpd binary to see where it dies.

I saw something very similar on a Redhat9 box recently, I didn't work on it further as it vanished when mod_ssl was disabled.
Comment 3 Nathan Mahon 2003-04-25 00:48:51 UTC
Created attachment 11100 [details]
output from strace -> SEGFAULT -> just using -D PHP4 this time, too.
Comment 4 Nathan Mahon 2003-04-25 12:57:03 UTC
I should clarify ... I'm probably completely off about the emerge mod_php helping.
This morning, being grumpy, I tried to fix it with re-emerging mod_php again.  This didn't help initially.  I then did this:

while ! ( sleep 25 && ps cax |grep apache ) ; do /etc/init.d/apache zap start; done

...  and after 3 or 4 attempts to start apache, it was successful.  So ... the emerge is probably not related... but I don't see how it can segfault 3 times in a row and then start to full production value with php ssl perl and the whole lot.
Comment 5 Nathan Mahon 2003-04-27 14:09:03 UTC
I had to reboot yesterday, and now I can't get apache to start at all, it seems.
well... at least now it's being more consistant... though that means I can't get it to work at all.
Comment 6 Robin Johnson archtester Gentoo Infrastructure gentoo-dev Security 2003-04-27 22:59:27 UTC
please do the strace with 'strace -ff' or find out how apache calls 'httpd' and strace that with -ff directly.

Looking at the current strace output, it could be possible that you have some bad RAM, but i'm not certain on that.
Comment 7 Nathan Mahon 2003-04-28 03:00:48 UTC
I thought I did do the strace with -ff.
also, I ran memtest86 for 16 hours very recently, and found *no* problems with anything.
Comment 8 Joshua Kinard gentoo-dev 2003-04-28 11:16:26 UTC
I don't know if this is of any help, but There's been some similar oddities like this when attempting to use apache + mod_ssl on sparc64.  Using mod_ssl, and apache just silently dies.  I've straced it, and it seems apache loads mod_ssl, mod_ssl attempts to bind to port 443, and gets an error back from the system saying "Address in Use".  This is only seen on sparc64, works fine on other archs.  I'm not sure if it might be related to this bug in any way, shape or form, but thought to mention it just in case.
Comment 9 Robin Johnson archtester Gentoo Infrastructure gentoo-dev Security 2003-04-28 13:22:41 UTC
*scrathes head*
Nathan: if it is ok with you, could I please get temporary root access to your machine to dig out some more information and try a few things to solve this?
I can work in a supervised shared screen session if you wish.

If it is ok, please create a user account on the machine, and email me all the details (username/password/rootpassword/hostname) in a GPG encrypted email. 
My key is available here http://search.keyserver.net:11371/pks/lookup?template=netensearch%2Cnetennomatch%2Cnetenerror&search=0x34884E85&op=vindex&fingerprint=on&submit=Get+List
Key ID: 34884E85
FingerPrint: 11AC BA4F 4778 E3F6 E4ED F38E B27B 944E 3488 4E85
Comment 10 Nathan Mahon 2003-04-28 17:59:44 UTC
sent.  Lemme know if you have issues.
Comment 11 Robin Johnson archtester Gentoo Infrastructure gentoo-dev Security 2003-04-29 04:45:40 UTC
How it will show up always: if you do '/etc/init.d/apache start' it always starts fine.
Conditions to make it show the error: '-D PHP4' in  conf.d/apache (other flags don't matter on or off, i have confirmed it exists with them off) AND you need to be starting from a non-interactive shell. A simple shell script works for this: '/etc/init.d/apache start' inside a file with chmod +x and run will show it happening.

It's gremlins I say!
Comment 12 Robin Johnson archtester Gentoo Infrastructure gentoo-dev Security 2003-04-29 05:29:46 UTC
files changed:
ok, now it is happening all the time with just '-D PHP4'
The really strange thing is that there is NO error output. the mmap2 operation completes successfully, but right after that there is a segfault. However even with ulimit -c set properly, there is NO core file dumpped.

/etc/php4/php.ini - grep for robbat2, there are 3 lines there commented out
/var/log/apache/php.err - php was set to send errors here
/etc/conf.d/apache - made copy of flags and played with them

As a longshort, could you please try jark's solution from the forums:
emerge -C apache php mod_php mod_ssl 
emerge sync 
emerge -u sablotron 
USE="-java -qt" emerge apache php mod_php mod_ssl
but back up your apache+php config files first!
As another idea, try emerging php+mod_php with USE="-crypt -java -qt".

Comment 13 Nathan Mahon 2003-04-29 13:07:23 UTC
Will do.
Comment 14 Nathan Mahon 2003-04-29 14:52:15 UTC
It's working!
You Da Man!
is there any explaination why this happened?
Comment 15 Robin Johnson archtester Gentoo Infrastructure gentoo-dev Security 2003-04-29 15:03:18 UTC
which part of my suggestions provided the solution?
i want to track this down futher now.
Comment 16 Nathan Mahon 2003-04-29 16:52:25 UTC
Sure... I can give you access again if you'd like it, (i rotated passwords already)... but ... anyway... 
I simply:
--------------------------------------------------------
emerge -C apache php mod_php mod_ssl 
emerge sync 
emerge -u sablotron 
USE="-java -qt" emerge apache php mod_php mod_ssl
--------------------------------------------------------
and I'd note that when the problem was working before: I'd get 7 seconds with one process, 3 seconds with a 2nd, and then one or two actual spawned handling daemons. Now, I *quickly* get the 5 spawned daemons I was expecting.

Comment 17 Robin Johnson archtester Gentoo Infrastructure gentoo-dev Security 2003-04-29 18:01:59 UTC
I don't need access again.
Could you try both of these
USE="-java" emerge apache php mod_php mod_ssl
USE="-qt" emerge apache php mod_php mod_ssl
and see if you you reduce the bug to happening under one of them only?

Comment 18 Nathan Mahon 2003-04-30 11:21:19 UTC
it'll take a while... but i'll work on it.
I'm rebuilding without any USE specified at all, just to verify that I still get the problem... then I'll do the other two.  :D  Sure wish I had more horsepower in that case. 
Comment 19 Nathan Mahon 2003-04-30 12:34:21 UTC
Phase one: Verify problem with no "USE=" modification. finished. (apache doesn't work.).  :D
Phase two: Verify problem with "USE=-java" -- in progress.
Comment 20 Nathan Mahon 2003-04-30 16:35:51 UTC
Phase two: Verify problem with "USE=-java" -- finished: Apache still broken: probably not java.
Phase three: Verify problem with "USE=-qt" -- finished: BINGO. "USE=-qt" is the culprit.  Apache is now serving pages.  :D

You need anything else tested?
Comment 21 Robin Johnson archtester Gentoo Infrastructure gentoo-dev Security 2003-04-30 19:16:21 UTC
could you please try 4.3.1-r2] (not -r3) of mod_php with your original USE flags? -r1 and -r2 had some QT fixes and other bits.
Comment 22 Nathan Mahon 2003-05-01 12:09:55 UTC
i did: # emerge /usr/portage/dev-php/mod_php/mod_php-4.3.1-r2.ebuild
and It appears to be working for me, after an apache restart of course.
Comment 23 Nathan Mahon 2003-05-01 12:16:17 UTC
oops... that was on the wrong machine... lemme try it again.
Comment 24 Nathan Mahon 2003-05-01 14:30:43 UTC
nope ... that didn't help.  emerging mod_php (4.3.1-r2) still doesn't allow apache to start properly.
Comment 25 Donny Davies (RETIRED) gentoo-dev 2003-05-01 15:25:21 UTC
looks like you're having fun with this.
Comment 26 Robin Johnson archtester Gentoo Infrastructure gentoo-dev Security 2003-05-01 15:34:18 UTC
thanks for testing. I'll be careful with QT in my new php eclass.
Could you tell me what version of QT you have installed?
then i'll close this bug.
Comment 27 Nathan Mahon 2003-05-02 09:30:03 UTC
3.1.2, it seems.

n8
Comment 28 Robin Johnson archtester Gentoo Infrastructure gentoo-dev Security 2003-07-10 14:34:08 UTC
Move to php-bugs
Comment 29 Robin Johnson archtester Gentoo Infrastructure gentoo-dev Security 2003-07-10 14:41:53 UTC
Closing this now, inconclusive.
Nobody else has seen anything like it, so I suspect it is more of a gremlin in QT.
Comment 30 Robin Johnson archtester Gentoo Infrastructure gentoo-dev Security 2003-09-25 18:27:32 UTC
closing again
Comment 31 Robin Johnson archtester Gentoo Infrastructure gentoo-dev Security 2003-09-25 18:27:43 UTC
closing properly.