Bug 81161

Summary:	net-www/apache starts too many server processes
Product:	Gentoo Linux	Reporter:	Andre Hinrichs <andre.hinrichs>
Component:	[OLD] Core system	Assignee:	Apache Team - Bugzilla Reports <apache-bugs>
Status:	RESOLVED CANTFIX
Severity:	trivial	CC:	m.debruijne
Priority:	High
Version:	unspecified
Hardware:	All
OS:	Linux
Whiteboard:
Package list:		Runtime testing required:	---
Attachments:	my complete configuration my /etc/conf.d/apache2

Description Andre Hinrichs 2005-02-07 13:25:26 UTC

Today I discovered, that apache is starting 58 server processes.
One owned by root, 57 owned by user apache.
The configuration file says:

MaxSpareServers     10

Since I very often check the process list, I'm pretty sure that this was
not the case yesterday. The difference is, that I started a complete world
rebuild five days ago which was finished today. Don't know if this changed
anything. Even after reboot the "problem" does not vanish. I even don't know
if this is a real problem. Just doesn't fit into my understanding of a clean
process list...
And because of this I state this as a trivial bug... for now...
Additionally I've to say that I updated my USE variable before starting the
rebuild to consider the very many changes there...


Reproducible: Always
Steps to Reproduce:

Actual Results:  
apache starts 58 processes


Expected Results:  
apache should start only 5 processes (MinSpareServers)


Portage 2.0.51-r15 (default-linux/x86/2004.0, gcc-3.3.5,
glibc-2.3.4.20040808-r1, 2.6.9 i686)
=================================================================
System uname: 2.6.9 i686 Pentium III (Coppermine)
Gentoo Base System version 1.4.16
Python:              dev-lang/python-2.3.4 [2.3.4 (#1, Feb  4 2005, 23:20:17)]
distcc 2.16 i686-pc-linux-gnu (protocols 1 and 2) (default port 3632) [disabled]
dev-lang/python:     2.3.4
sys-devel/autoconf:  2.59-r6, 2.13
sys-devel/automake:  1.7.9-r1, 1.8.5-r3, 1.5, 1.4_p6, 1.6.3, 1.9.4
sys-devel/binutils:  2.15.92.0.2-r1
sys-devel/libtool:   1.5.10-r4
virtual/os-headers:  2.6.8.1-r2
ACCEPT_KEYWORDS="x86"
AUTOCLEAN="yes"
CFLAGS="-O3 -march=i686 -funroll-loops -pipe"
CHOST="i686-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/kde/2/share/config /usr/kde/3.3/env
/usr/kde/3.3/share/config /usr/kde/3.3/shutdown /usr/kde/3/share/config
/usr/lib/X11/xkb /usr/share/config /usr/share/texmf/dvipdfm/config/
/usr/share/texmf/dvips/config/ /usr/share/texmf/tex/generic/config/
/usr/share/texmf/tex/platex/config/ /usr/share/texmf/xdvi/ /var/bind
/var/qmail/control"
CONFIG_PROTECT_MASK="/etc/gconf /etc/terminfo /etc/env.d"
CXXFLAGS="-O3 -march=i686 -funroll-loops -pipe"
DISTDIR="/usr/portage/distfiles"
FEATURES="autoaddcvs autoconfig ccache distlocks keeptemp sandbox sfperms strict"
GENTOO_MIRRORS="http://192.168.200.1/
http://ftp-stud.fht-esslingen.de/pub/Mirrors/gentoo/
ftp://ftp.linux.ee/pub/gentoo/distfiles/
http://ftp.linux.ee/pub/gentoo/distfiles/ http://src.gentoo.pl"
MAKEOPTS="-j2"
PKGDIR="/usr/portage/packages"
PORTAGE_TMPDIR="/var/tmp"
PORTDIR="/usr/portage"
PORTDIR_OVERLAY="/usr/local/portage"
SYNC="rsync://192.168.200.1/gentoo-portage"
USE="x86 X Xaw3d a52 aac aalib acl alsa apache2 apm arts audiofile avi
bitmap-fonts bonobo bzlib cdparanoia cdr cpdflib crypt cups curl dbm divx4linux
dv dvd dvdread emacs encode esd expat f77 fam ffmpeg flac flash font-server
foomaticdb fortran ftp gd gdbm ggi gif gmp gnome gnutls gphoto2 gpm gtk gtk2
gtkhtml guile icq imagemagick imap imlib ipv6 java jpeg kde kdeenablefinal lcms
ldap leim libg++ libwww mad maildir matroska mhash mikmod milter mime ming mmx
mng motif mozilla mp3 mpeg mysql mysqli nas ncurses nls nocardbus odbc ogg
oggvorbis openal opengl oss pam pcmcia pcre pda pdflib perl php plotutils png
posix ppds profile python qt quicktime readline recode sasl scanner sdl shared
sharedmem simplexml slang slp sndfile soap sockets sox speex spell spl ssl svg
svga sziptcltk tcpd tetex theora threads tidy tiff tokenizer truetype
truetype-fonts type1-fonts unicode usb vcd wmf wxwindows xine xml xml2 xmlrpc
xmms xpm xsl xv xvid zlib"
Unset:  ASFLAGS, CBUILD, CTARGET, LANG, LC_ALL, LDFLAGS

Comment 1 Christian Parpart (RETIRED) gentoo-dev

2005-02-07 14:44:29 UTC

interesting. very interesting. what pages/applications does your webserver serve? Well, I ask, because in my case, it just spawns those 10 childs on rcstart, but may extend up to 250, since I've a web-based realtime chat running on it. It may be of interest, what you are doing with your apache, and whether this is a default installation you're using, or not. If not, what did you change/apply/etc?

chipig? any ideas else?

Comment 2 Paul Querna 2005-02-07 21:49:24 UTC

Please post your entire configuration. This sounds like a config error to me.

Comment 3 Andre Hinrichs 2005-02-08 04:38:07 UTC

Created attachment 50721 [details]
my complete configuration

The server acts only as a download server for portage distfiles in a local
network
So, no special configurations were done...

Comment 4 Andre Hinrichs 2005-02-08 04:40:49 UTC

Created attachment 50722 [details]
my /etc/conf.d/apache2

Comment 5 Peter Gantner (a.k.a. nephros) 2005-02-09 12:43:11 UTC

hmm, I think there is some confusion with regard to processes <=> threads here.

which program are you using to count those processes?

my apache.conf has the following (defaults) set:
# MinSpareServers: minimum number of server processes which are kept spare
# MaxSpareServers: maximum number of server processes which are kept spare
MinSpareServers      5
MaxSpareServers     10
# MinSpareThreads: minimum number of worker threads which are kept spare
# MaxSpareThreads: maximum number of worker threads which are kept spare
MinSpareThreads     25
MaxSpareThreads     75

ps output corresonds to this:
Processes:
[scourge]~ # ps ax|grep apache|grep -v grep
17794 ?        Ss     0:00 /usr/sbin/apache2 -k start -D PHP5 -D DAV -D DAV_FS
17795 ?        S      0:00 /usr/sbin/apache2 -k start -D PHP5 -D DAV -D DAV_FS
17796 ?        Sl     0:00 /usr/sbin/apache2 -k start -D PHP5 -D DAV -D DAV_FS
17797 ?        Sl     0:00 /usr/sbin/apache2 -k start -D PHP5 -D DAV -D DAV_FS

threads:
[scourge]~ # ps axH|grep apache|grep -v grep|wc -l
56
[scourge]~ # ps axH|grep apache|grep -v grep|uniq 
17794 ?        Ss     0:00 /usr/sbin/apache2 -k start -D PHP5 -D DAV -D DAV_FS
17795 ?        S      0:00 /usr/sbin/apache2 -k start -D PHP5 -D DAV -D DAV_FS
17796 ?        Sl     0:00 /usr/sbin/apache2 -k start -D PHP5 -D DAV -D DAV_FS
17797 ?        Sl     0:00 /usr/sbin/apache2 -k start -D PHP5 -D DAV -D DAV_FS

This is apache-2.0.52-r2 on a NPTL 2.6.11-rc1 system, glibc-2.3.4.20041102 procps-3.2.5

Comment 6 Andre Hinrichs 2005-02-15 11:41:24 UTC

sorry for being late. was ill...

# ps ax|grep apache|grep -v grep
 7937 ?        Ss     0:00 /usr/sbin/apache2 -k start
 8074 ?        S      0:00 /usr/sbin/apache2 -k start
 8075 ?        S      0:00 /usr/sbin/apache2 -k start
 8076 ?        S      0:00 /usr/sbin/apache2 -k start
 8077 ?        S      0:00 /usr/sbin/apache2 -k start
 8078 ?        S      0:00 /usr/sbin/apache2 -k start
 8081 ?        S      0:00 /usr/sbin/apache2 -k start
 8082 ?        S      0:00 /usr/sbin/apache2 -k start
 8083 ?        S      0:00 /usr/sbin/apache2 -k start
 8084 ?        S      0:00 /usr/sbin/apache2 -k start
 8085 ?        S      0:00 /usr/sbin/apache2 -k start
 8086 ?        S      0:00 /usr/sbin/apache2 -k start
 8087 ?        S      0:00 /usr/sbin/apache2 -k start
 8088 ?        S      0:00 /usr/sbin/apache2 -k start
 8089 ?        S      0:00 /usr/sbin/apache2 -k start
 8090 ?        S      0:00 /usr/sbin/apache2 -k start
 8091 ?        S      0:00 /usr/sbin/apache2 -k start
 8092 ?        S      0:00 /usr/sbin/apache2 -k start
 8093 ?        S      0:00 /usr/sbin/apache2 -k start
 8094 ?        S      0:00 /usr/sbin/apache2 -k start
 8095 ?        S      0:00 /usr/sbin/apache2 -k start
 8096 ?        S      0:00 /usr/sbin/apache2 -k start
 8097 ?        S      0:00 /usr/sbin/apache2 -k start
 8098 ?        S      0:00 /usr/sbin/apache2 -k start
 8099 ?        S      0:00 /usr/sbin/apache2 -k start
 8100 ?        S      0:00 /usr/sbin/apache2 -k start
 8101 ?        S      0:00 /usr/sbin/apache2 -k start
 8102 ?        S      0:00 /usr/sbin/apache2 -k start
 8103 ?        S      0:00 /usr/sbin/apache2 -k start
 8104 ?        S      0:00 /usr/sbin/apache2 -k start
 8105 ?        S      0:00 /usr/sbin/apache2 -k start
 8106 ?        S      0:00 /usr/sbin/apache2 -k start
 8107 ?        S      0:00 /usr/sbin/apache2 -k start
 8108 ?        S      0:00 /usr/sbin/apache2 -k start
 8109 ?        S      0:00 /usr/sbin/apache2 -k start
 8110 ?        S      0:00 /usr/sbin/apache2 -k start
 8111 ?        S      0:00 /usr/sbin/apache2 -k start
 8112 ?        S      0:00 /usr/sbin/apache2 -k start
 8113 ?        S      0:00 /usr/sbin/apache2 -k start
 8114 ?        S      0:00 /usr/sbin/apache2 -k start
 8115 ?        S      0:00 /usr/sbin/apache2 -k start
 8116 ?        S      0:00 /usr/sbin/apache2 -k start
 8117 ?        S      0:00 /usr/sbin/apache2 -k start
 8118 ?        S      0:00 /usr/sbin/apache2 -k start
 8119 ?        S      0:00 /usr/sbin/apache2 -k start
 8120 ?        S      0:00 /usr/sbin/apache2 -k start
 8121 ?        S      0:00 /usr/sbin/apache2 -k start
 8122 ?        S      0:00 /usr/sbin/apache2 -k start
 8123 ?        S      0:00 /usr/sbin/apache2 -k start
 8124 ?        S      0:00 /usr/sbin/apache2 -k start
 8125 ?        S      0:00 /usr/sbin/apache2 -k start
 8126 ?        S      0:00 /usr/sbin/apache2 -k start
 8127 ?        S      0:00 /usr/sbin/apache2 -k start
 8128 ?        S      0:00 /usr/sbin/apache2 -k start
 8129 ?        S      0:00 /usr/sbin/apache2 -k start
 8130 ?        S      0:00 /usr/sbin/apache2 -k start
 8131 ?        S      0:00 /usr/sbin/apache2 -k start
 8132 ?        S      0:00 /usr/sbin/apache2 -k start


ps axH  produces exactly the same output...

Comment 7 Charles Leahy 2005-08-09 02:53:31 UTC

I am now also experiencing this behaviour and I am convinced it is not a
configuration problem. I recently installed apache with mpm-worker and started
it with the default gentoo config to test things worked. ps showed that 58
processes were being launched in a bizzare tree as shown in full bellow:

21076 ?        Ss     0:00 /usr/sbin/apache2 -DINFO -DSUPHP -d /usr/lib/apache2
21081 ?        S      0:00  \_ /usr/sbin/apache2 -DINFO -DSUPHP -d /usr/lib/apac
21082 ?        S      0:00  |   \_ /usr/sbin/apache2 -DINFO -DSUPHP -d /usr/lib/
21085 ?        S      0:00  |       \_ /usr/sbin/apache2 -DINFO -DSUPHP -d /usr/
21088 ?        S      0:00  |       \_ /usr/sbin/apache2 -DINFO -DSUPHP -d /usr/
21091 ?        S      0:00  |       \_ /usr/sbin/apache2 -DINFO -DSUPHP -d /usr/
21092 ?        S      0:00  |       \_ /usr/sbin/apache2 -DINFO -DSUPHP -d /usr/
21095 ?        S      0:00  |       \_ /usr/sbin/apache2 -DINFO -DSUPHP -d /usr/
21097 ?        S      0:00  |       \_ /usr/sbin/apache2 -DINFO -DSUPHP -d /usr/
21098 ?        S      0:00  |       \_ /usr/sbin/apache2 -DINFO -DSUPHP -d /usr/
21101 ?        S      0:00  |       \_ /usr/sbin/apache2 -DINFO -DSUPHP -d /usr/
21103 ?        S      0:00  |       \_ /usr/sbin/apache2 -DINFO -DSUPHP -d /usr/
21104 ?        S      0:00  |       \_ /usr/sbin/apache2 -DINFO -DSUPHP -d /usr/
21107 ?        S      0:00  |       \_ /usr/sbin/apache2 -DINFO -DSUPHP -d /usr/
21109 ?        S      0:00  |       \_ /usr/sbin/apache2 -DINFO -DSUPHP -d /usr/
21110 ?        S      0:00  |       \_ /usr/sbin/apache2 -DINFO -DSUPHP -d /usr/
21113 ?        S      0:00  |       \_ /usr/sbin/apache2 -DINFO -DSUPHP -d /usr/
21115 ?        S      0:00  |       \_ /usr/sbin/apache2 -DINFO -DSUPHP -d /usr/
21116 ?        S      0:00  |       \_ /usr/sbin/apache2 -DINFO -DSUPHP -d /usr/
21119 ?        S      0:00  |       \_ /usr/sbin/apache2 -DINFO -DSUPHP -d /usr/
21121 ?        S      0:00  |       \_ /usr/sbin/apache2 -DINFO -DSUPHP -d /usr/
21122 ?        S      0:00  |       \_ /usr/sbin/apache2 -DINFO -DSUPHP -d /usr/
21125 ?        S      0:00  |       \_ /usr/sbin/apache2 -DINFO -DSUPHP -d /usr/
21127 ?        S      0:00  |       \_ /usr/sbin/apache2 -DINFO -DSUPHP -d /usr/
21128 ?        S      0:00  |       \_ /usr/sbin/apache2 -DINFO -DSUPHP -d /usr/
21131 ?        S      0:00  |       \_ /usr/sbin/apache2 -DINFO -DSUPHP -d /usr/
21133 ?        S      0:00  |       \_ /usr/sbin/apache2 -DINFO -DSUPHP -d /usr/
21134 ?        S      0:00  |       \_ /usr/sbin/apache2 -DINFO -DSUPHP -d /usr/
21137 ?        S      0:00  |       \_ /usr/sbin/apache2 -DINFO -DSUPHP -d /usr/
21084 ?        S      0:00  \_ /usr/sbin/apache2 -DINFO -DSUPHP -d /usr/lib/apac
21086 ?        S      0:00      \_ /usr/sbin/apache2 -DINFO -DSUPHP -d /usr/lib/
21089 ?        S      0:00          \_ /usr/sbin/apache2 -DINFO -DSUPHP -d /usr/
21090 ?        S      0:00          \_ /usr/sbin/apache2 -DINFO -DSUPHP -d /usr/
21093 ?        S      0:00          \_ /usr/sbin/apache2 -DINFO -DSUPHP -d /usr/
21094 ?        S      0:00          \_ /usr/sbin/apache2 -DINFO -DSUPHP -d /usr/
21096 ?        S      0:00          \_ /usr/sbin/apache2 -DINFO -DSUPHP -d /usr/
21099 ?        S      0:00          \_ /usr/sbin/apache2 -DINFO -DSUPHP -d /usr/
21100 ?        S      0:00          \_ /usr/sbin/apache2 -DINFO -DSUPHP -d /usr/
21102 ?        S      0:00          \_ /usr/sbin/apache2 -DINFO -DSUPHP -d /usr/
21105 ?        S      0:00          \_ /usr/sbin/apache2 -DINFO -DSUPHP -d /usr/
21106 ?        S      0:00          \_ /usr/sbin/apache2 -DINFO -DSUPHP -d /usr/
21108 ?        S      0:00          \_ /usr/sbin/apache2 -DINFO -DSUPHP -d /usr/
21111 ?        S      0:00          \_ /usr/sbin/apache2 -DINFO -DSUPHP -d /usr/
21112 ?        S      0:00          \_ /usr/sbin/apache2 -DINFO -DSUPHP -d /usr/
21114 ?        S      0:00          \_ /usr/sbin/apache2 -DINFO -DSUPHP -d /usr/
21117 ?        S      0:00          \_ /usr/sbin/apache2 -DINFO -DSUPHP -d /usr/
21118 ?        S      0:00          \_ /usr/sbin/apache2 -DINFO -DSUPHP -d /usr/
21120 ?        S      0:00          \_ /usr/sbin/apache2 -DINFO -DSUPHP -d /usr/
21123 ?        S      0:00          \_ /usr/sbin/apache2 -DINFO -DSUPHP -d /usr/
21124 ?        S      0:00          \_ /usr/sbin/apache2 -DINFO -DSUPHP -d /usr/
21126 ?        S      0:00          \_ /usr/sbin/apache2 -DINFO -DSUPHP -d /usr/
21129 ?        S      0:00          \_ /usr/sbin/apache2 -DINFO -DSUPHP -d /usr/
21130 ?        S      0:00          \_ /usr/sbin/apache2 -DINFO -DSUPHP -d /usr/
21132 ?        S      0:00          \_ /usr/sbin/apache2 -DINFO -DSUPHP -d /usr/
21135 ?        S      0:00          \_ /usr/sbin/apache2 -DINFO -DSUPHP -d /usr/
21136 ?        S      0:00          \_ /usr/sbin/apache2 -DINFO -DSUPHP -d /usr/
21138 ?        S      0:00          \_ /usr/sbin/apache2 -DINFO -DSUPHP -d /usr/

This list was generated with ps axf, all of the above are definitely processes
and not threads.

The following shows modules compiled into apache:
charm ~ # apache2 -l
Compiled in modules:
  core.c
  worker.c
  http_core.c
  mod_so.c

I am currently using the new layout (net-www/apache-2.0.54-r13) however
experience the same behaviour with the old layout (-r8). Apache was compiled
with the following use flags:

+apache2 -debug -doc -ldap -mpm-leader -mpm-peruser -mpm-prefork -mpm-threadpool
+mpm-worker -no-suexec (-selinux) +ssl -static-modules +threads

In both the new and old layouts this was observed with the default config,
however I have since modified the configs to be suitable for my use and
commented out any lines in httpd.conf directed at any mpm other than worker but
the same 58 processes are still observed.

mod_info shows worker.c as being configured as follows:
Current Configuration:
    StartServers 2
    ThreadsPerChild 25
    MinSpareThreads 25
    MaxSpareThreads 75
    MaxClients 150
    Listen 80
    User apache
    Group apache

I am running gentoo on a Sun Netra T1 AC200. My emerge --info follows:

charm ~ # emerge --info
Portage 2.0.51.22-r2 (default-linux/sparc/sparc64/2005.0, gcc-3.3.5-20050130,
glibc-2.3.3.20040420-r2, 2.4.31-sparc sparc64)
=================================================================
System uname: 2.4.31-sparc sparc64 sun4u
Gentoo Base System version 1.6.13
dev-lang/python:     2.3.5
sys-apps/sandbox:    1.2.11
sys-devel/autoconf:  2.13, 2.59-r6
sys-devel/automake:  1.4_p6, 1.5, 1.6.3, 1.7.9-r1, 1.8.5-r3, 1.9.5
sys-devel/binutils:  2.15.92.0.2-r10
sys-devel/libtool:   1.5.18-r1
virtual/os-headers:  2.4.23
ACCEPT_KEYWORDS="sparc"
AUTOCLEAN="yes"
CBUILD="sparc-unknown-linux-gnu"
CFLAGS="-O2 -mcpu=ultrasparc -pipe"
CHOST="sparc-unknown-linux-gnu"
CONFIG_PROTECT="/etc /usr/kde/2/share/config /usr/kde/3/share/config
/usr/share/config /var/bind /var/qmail/control"
CONFIG_PROTECT_MASK="/etc/gconf /etc/terminfo /etc/env.d"
CXXFLAGS="-O2 -mcpu=ultrasparc -pipe"
DISTDIR="/usr/portage/distfiles"
FEATURES="autoconfig ccache distlocks sandbox sfperms strict"
GENTOO_MIRRORS="http://gentoo.blueyonder.co.uk"
PKGDIR="/usr/portage/packages"
PORTAGE_TMPDIR="/var/tmp"
PORTDIR="/usr/portage"
PORTDIR_OVERLAY="/usr/local/portage"
SYNC="rsync://rsync.gentoo.org/gentoo-portage"
USE="sparc adns apache2 authdaemond bash-completion berkdb bind-mysql bzip2
crypt curl extensions gcc64 gd imagemagick imap innodb ithreads jpeg libwww
maildir memlimit mime mpeg mpm-worker mysql ncurses nls pam perl php png python
readline sasl session shaper sockets softquota spell ssl tcpd threads tiff
truetype unicode vhosts xml xml2 zlib userland_GNU kernel_linux elibc_glibc"
Unset:  ASFLAGS, CTARGET, LANG, LC_ALL, LDFLAGS, LINGUAS, MAKEOPTS

This is purely speculation to me but it would seem that mpm-worker creates two
children (as it should) but then with it comes to make listener and worker
threads it instead creates processes. It would appear to create to correct
number of children given my configuration, but as processes and not threads.

Thanks,

Charles Leahy

Comment 8 Charles Leahy 2005-08-09 03:00:11 UTC

Sorry, one correction, 57 and *not* 58 processes are being started.

Comment 9 Charles Leahy 2005-08-09 07:21:07 UTC

I have now tested this with a clean install on another sparc box using the
default gentoo apache confs, and the following use flags:

[ebuild   R   ] net-www/apache-2.0.54-r13  +apache2 -debug +doc -ldap
-mpm-leader -mpm-peruser -mpm-prefork -mpm-threadpool +mpm-worker -no-suexec
(-selinux) +ssl -static-modules +threads 0 kB

The same problem occurs, this time launching 59 processes. Again, apache2 -l
shows worker.c being built in.

On both boxes I am using:

[ebuild   R   ] dev-libs/apr-0.9.6-r3  -ipv6 0 kB 
[ebuild   R   ] dev-libs/apr-util-0.9.6-r2  +berkdb -gdbm -ldap 0 kB 

Thanks

Comment 10 Michael Stewart (vericgar) (RETIRED) gentoo-dev

2005-08-09 18:45:12 UTC

From what I can tell by reading the source code, it looks like this may be an
issue with pthread_create. Can you run ps -eLf so we can check the threads for
each process? I suspect threading is not working at all on your machine. Is
there another threaded application you can check to see if it's behaving the
same way?

Comment 11 johan 2005-09-03 00:48:03 UTC

I have the same problem. pthreads are definately broken. I tried switching from
SMP to non-SMP and recompiling glibc, but no joy. This is very annoying.

Comment 12 Michael Stewart (vericgar) (RETIRED) gentoo-dev

2005-09-17 20:10:08 UTC

It seems confirmed this is an issue with pthreads, and not an apache issue.
I'm not sure how to fix that, other then starting completely over and installing
again. There's not anything the apache team can about this though.