Today I discovered, that apache is starting 58 server processes. One owned by root, 57 owned by user apache. The configuration file says: MaxSpareServers 10 Since I very often check the process list, I'm pretty sure that this was not the case yesterday. The difference is, that I started a complete world rebuild five days ago which was finished today. Don't know if this changed anything. Even after reboot the "problem" does not vanish. I even don't know if this is a real problem. Just doesn't fit into my understanding of a clean process list... And because of this I state this as a trivial bug... for now... Additionally I've to say that I updated my USE variable before starting the rebuild to consider the very many changes there... Reproducible: Always Steps to Reproduce: Actual Results: apache starts 58 processes Expected Results: apache should start only 5 processes (MinSpareServers) Portage 2.0.51-r15 (default-linux/x86/2004.0, gcc-3.3.5, glibc-2.3.4.20040808-r1, 2.6.9 i686) ================================================================= System uname: 2.6.9 i686 Pentium III (Coppermine) Gentoo Base System version 1.4.16 Python: dev-lang/python-2.3.4 [2.3.4 (#1, Feb 4 2005, 23:20:17)] distcc 2.16 i686-pc-linux-gnu (protocols 1 and 2) (default port 3632) [disabled] dev-lang/python: 2.3.4 sys-devel/autoconf: 2.59-r6, 2.13 sys-devel/automake: 1.7.9-r1, 1.8.5-r3, 1.5, 1.4_p6, 1.6.3, 1.9.4 sys-devel/binutils: 2.15.92.0.2-r1 sys-devel/libtool: 1.5.10-r4 virtual/os-headers: 2.6.8.1-r2 ACCEPT_KEYWORDS="x86" AUTOCLEAN="yes" CFLAGS="-O3 -march=i686 -funroll-loops -pipe" CHOST="i686-pc-linux-gnu" CONFIG_PROTECT="/etc /usr/kde/2/share/config /usr/kde/3.3/env /usr/kde/3.3/share/config /usr/kde/3.3/shutdown /usr/kde/3/share/config /usr/lib/X11/xkb /usr/share/config /usr/share/texmf/dvipdfm/config/ /usr/share/texmf/dvips/config/ /usr/share/texmf/tex/generic/config/ /usr/share/texmf/tex/platex/config/ /usr/share/texmf/xdvi/ /var/bind /var/qmail/control" CONFIG_PROTECT_MASK="/etc/gconf /etc/terminfo /etc/env.d" CXXFLAGS="-O3 -march=i686 -funroll-loops -pipe" DISTDIR="/usr/portage/distfiles" FEATURES="autoaddcvs autoconfig ccache distlocks keeptemp sandbox sfperms strict" GENTOO_MIRRORS="http://192.168.200.1/ http://ftp-stud.fht-esslingen.de/pub/Mirrors/gentoo/ ftp://ftp.linux.ee/pub/gentoo/distfiles/ http://ftp.linux.ee/pub/gentoo/distfiles/ http://src.gentoo.pl" MAKEOPTS="-j2" PKGDIR="/usr/portage/packages" PORTAGE_TMPDIR="/var/tmp" PORTDIR="/usr/portage" PORTDIR_OVERLAY="/usr/local/portage" SYNC="rsync://192.168.200.1/gentoo-portage" USE="x86 X Xaw3d a52 aac aalib acl alsa apache2 apm arts audiofile avi bitmap-fonts bonobo bzlib cdparanoia cdr cpdflib crypt cups curl dbm divx4linux dv dvd dvdread emacs encode esd expat f77 fam ffmpeg flac flash font-server foomaticdb fortran ftp gd gdbm ggi gif gmp gnome gnutls gphoto2 gpm gtk gtk2 gtkhtml guile icq imagemagick imap imlib ipv6 java jpeg kde kdeenablefinal lcms ldap leim libg++ libwww mad maildir matroska mhash mikmod milter mime ming mmx mng motif mozilla mp3 mpeg mysql mysqli nas ncurses nls nocardbus odbc ogg oggvorbis openal opengl oss pam pcmcia pcre pda pdflib perl php plotutils png posix ppds profile python qt quicktime readline recode sasl scanner sdl shared sharedmem simplexml slang slp sndfile soap sockets sox speex spell spl ssl svg svga sziptcltk tcpd tetex theora threads tidy tiff tokenizer truetype truetype-fonts type1-fonts unicode usb vcd wmf wxwindows xine xml xml2 xmlrpc xmms xpm xsl xv xvid zlib" Unset: ASFLAGS, CBUILD, CTARGET, LANG, LC_ALL, LDFLAGS
interesting. very interesting. what pages/applications does your webserver serve? Well, I ask, because in my case, it just spawns those 10 childs on rcstart, but may extend up to 250, since I've a web-based realtime chat running on it. It may be of interest, what you are doing with your apache, and whether this is a default installation you're using, or not. If not, what did you change/apply/etc? chipig? any ideas else?
Please post your entire configuration. This sounds like a config error to me.
Created attachment 50721 [details] my complete configuration The server acts only as a download server for portage distfiles in a local network So, no special configurations were done...
Created attachment 50722 [details] my /etc/conf.d/apache2
hmm, I think there is some confusion with regard to processes <=> threads here. which program are you using to count those processes? my apache.conf has the following (defaults) set: # MinSpareServers: minimum number of server processes which are kept spare # MaxSpareServers: maximum number of server processes which are kept spare MinSpareServers 5 MaxSpareServers 10 # MinSpareThreads: minimum number of worker threads which are kept spare # MaxSpareThreads: maximum number of worker threads which are kept spare MinSpareThreads 25 MaxSpareThreads 75 ps output corresonds to this: Processes: [scourge]~ # ps ax|grep apache|grep -v grep 17794 ? Ss 0:00 /usr/sbin/apache2 -k start -D PHP5 -D DAV -D DAV_FS 17795 ? S 0:00 /usr/sbin/apache2 -k start -D PHP5 -D DAV -D DAV_FS 17796 ? Sl 0:00 /usr/sbin/apache2 -k start -D PHP5 -D DAV -D DAV_FS 17797 ? Sl 0:00 /usr/sbin/apache2 -k start -D PHP5 -D DAV -D DAV_FS threads: [scourge]~ # ps axH|grep apache|grep -v grep|wc -l 56 [scourge]~ # ps axH|grep apache|grep -v grep|uniq 17794 ? Ss 0:00 /usr/sbin/apache2 -k start -D PHP5 -D DAV -D DAV_FS 17795 ? S 0:00 /usr/sbin/apache2 -k start -D PHP5 -D DAV -D DAV_FS 17796 ? Sl 0:00 /usr/sbin/apache2 -k start -D PHP5 -D DAV -D DAV_FS 17797 ? Sl 0:00 /usr/sbin/apache2 -k start -D PHP5 -D DAV -D DAV_FS This is apache-2.0.52-r2 on a NPTL 2.6.11-rc1 system, glibc-2.3.4.20041102 procps-3.2.5
sorry for being late. was ill... # ps ax|grep apache|grep -v grep 7937 ? Ss 0:00 /usr/sbin/apache2 -k start 8074 ? S 0:00 /usr/sbin/apache2 -k start 8075 ? S 0:00 /usr/sbin/apache2 -k start 8076 ? S 0:00 /usr/sbin/apache2 -k start 8077 ? S 0:00 /usr/sbin/apache2 -k start 8078 ? S 0:00 /usr/sbin/apache2 -k start 8081 ? S 0:00 /usr/sbin/apache2 -k start 8082 ? S 0:00 /usr/sbin/apache2 -k start 8083 ? S 0:00 /usr/sbin/apache2 -k start 8084 ? S 0:00 /usr/sbin/apache2 -k start 8085 ? S 0:00 /usr/sbin/apache2 -k start 8086 ? S 0:00 /usr/sbin/apache2 -k start 8087 ? S 0:00 /usr/sbin/apache2 -k start 8088 ? S 0:00 /usr/sbin/apache2 -k start 8089 ? S 0:00 /usr/sbin/apache2 -k start 8090 ? S 0:00 /usr/sbin/apache2 -k start 8091 ? S 0:00 /usr/sbin/apache2 -k start 8092 ? S 0:00 /usr/sbin/apache2 -k start 8093 ? S 0:00 /usr/sbin/apache2 -k start 8094 ? S 0:00 /usr/sbin/apache2 -k start 8095 ? S 0:00 /usr/sbin/apache2 -k start 8096 ? S 0:00 /usr/sbin/apache2 -k start 8097 ? S 0:00 /usr/sbin/apache2 -k start 8098 ? S 0:00 /usr/sbin/apache2 -k start 8099 ? S 0:00 /usr/sbin/apache2 -k start 8100 ? S 0:00 /usr/sbin/apache2 -k start 8101 ? S 0:00 /usr/sbin/apache2 -k start 8102 ? S 0:00 /usr/sbin/apache2 -k start 8103 ? S 0:00 /usr/sbin/apache2 -k start 8104 ? S 0:00 /usr/sbin/apache2 -k start 8105 ? S 0:00 /usr/sbin/apache2 -k start 8106 ? S 0:00 /usr/sbin/apache2 -k start 8107 ? S 0:00 /usr/sbin/apache2 -k start 8108 ? S 0:00 /usr/sbin/apache2 -k start 8109 ? S 0:00 /usr/sbin/apache2 -k start 8110 ? S 0:00 /usr/sbin/apache2 -k start 8111 ? S 0:00 /usr/sbin/apache2 -k start 8112 ? S 0:00 /usr/sbin/apache2 -k start 8113 ? S 0:00 /usr/sbin/apache2 -k start 8114 ? S 0:00 /usr/sbin/apache2 -k start 8115 ? S 0:00 /usr/sbin/apache2 -k start 8116 ? S 0:00 /usr/sbin/apache2 -k start 8117 ? S 0:00 /usr/sbin/apache2 -k start 8118 ? S 0:00 /usr/sbin/apache2 -k start 8119 ? S 0:00 /usr/sbin/apache2 -k start 8120 ? S 0:00 /usr/sbin/apache2 -k start 8121 ? S 0:00 /usr/sbin/apache2 -k start 8122 ? S 0:00 /usr/sbin/apache2 -k start 8123 ? S 0:00 /usr/sbin/apache2 -k start 8124 ? S 0:00 /usr/sbin/apache2 -k start 8125 ? S 0:00 /usr/sbin/apache2 -k start 8126 ? S 0:00 /usr/sbin/apache2 -k start 8127 ? S 0:00 /usr/sbin/apache2 -k start 8128 ? S 0:00 /usr/sbin/apache2 -k start 8129 ? S 0:00 /usr/sbin/apache2 -k start 8130 ? S 0:00 /usr/sbin/apache2 -k start 8131 ? S 0:00 /usr/sbin/apache2 -k start 8132 ? S 0:00 /usr/sbin/apache2 -k start ps axH produces exactly the same output...
I am now also experiencing this behaviour and I am convinced it is not a configuration problem. I recently installed apache with mpm-worker and started it with the default gentoo config to test things worked. ps showed that 58 processes were being launched in a bizzare tree as shown in full bellow: 21076 ? Ss 0:00 /usr/sbin/apache2 -DINFO -DSUPHP -d /usr/lib/apache2 21081 ? S 0:00 \_ /usr/sbin/apache2 -DINFO -DSUPHP -d /usr/lib/apac 21082 ? S 0:00 | \_ /usr/sbin/apache2 -DINFO -DSUPHP -d /usr/lib/ 21085 ? S 0:00 | \_ /usr/sbin/apache2 -DINFO -DSUPHP -d /usr/ 21088 ? S 0:00 | \_ /usr/sbin/apache2 -DINFO -DSUPHP -d /usr/ 21091 ? S 0:00 | \_ /usr/sbin/apache2 -DINFO -DSUPHP -d /usr/ 21092 ? S 0:00 | \_ /usr/sbin/apache2 -DINFO -DSUPHP -d /usr/ 21095 ? S 0:00 | \_ /usr/sbin/apache2 -DINFO -DSUPHP -d /usr/ 21097 ? S 0:00 | \_ /usr/sbin/apache2 -DINFO -DSUPHP -d /usr/ 21098 ? S 0:00 | \_ /usr/sbin/apache2 -DINFO -DSUPHP -d /usr/ 21101 ? S 0:00 | \_ /usr/sbin/apache2 -DINFO -DSUPHP -d /usr/ 21103 ? S 0:00 | \_ /usr/sbin/apache2 -DINFO -DSUPHP -d /usr/ 21104 ? S 0:00 | \_ /usr/sbin/apache2 -DINFO -DSUPHP -d /usr/ 21107 ? S 0:00 | \_ /usr/sbin/apache2 -DINFO -DSUPHP -d /usr/ 21109 ? S 0:00 | \_ /usr/sbin/apache2 -DINFO -DSUPHP -d /usr/ 21110 ? S 0:00 | \_ /usr/sbin/apache2 -DINFO -DSUPHP -d /usr/ 21113 ? S 0:00 | \_ /usr/sbin/apache2 -DINFO -DSUPHP -d /usr/ 21115 ? S 0:00 | \_ /usr/sbin/apache2 -DINFO -DSUPHP -d /usr/ 21116 ? S 0:00 | \_ /usr/sbin/apache2 -DINFO -DSUPHP -d /usr/ 21119 ? S 0:00 | \_ /usr/sbin/apache2 -DINFO -DSUPHP -d /usr/ 21121 ? S 0:00 | \_ /usr/sbin/apache2 -DINFO -DSUPHP -d /usr/ 21122 ? S 0:00 | \_ /usr/sbin/apache2 -DINFO -DSUPHP -d /usr/ 21125 ? S 0:00 | \_ /usr/sbin/apache2 -DINFO -DSUPHP -d /usr/ 21127 ? S 0:00 | \_ /usr/sbin/apache2 -DINFO -DSUPHP -d /usr/ 21128 ? S 0:00 | \_ /usr/sbin/apache2 -DINFO -DSUPHP -d /usr/ 21131 ? S 0:00 | \_ /usr/sbin/apache2 -DINFO -DSUPHP -d /usr/ 21133 ? S 0:00 | \_ /usr/sbin/apache2 -DINFO -DSUPHP -d /usr/ 21134 ? S 0:00 | \_ /usr/sbin/apache2 -DINFO -DSUPHP -d /usr/ 21137 ? S 0:00 | \_ /usr/sbin/apache2 -DINFO -DSUPHP -d /usr/ 21084 ? S 0:00 \_ /usr/sbin/apache2 -DINFO -DSUPHP -d /usr/lib/apac 21086 ? S 0:00 \_ /usr/sbin/apache2 -DINFO -DSUPHP -d /usr/lib/ 21089 ? S 0:00 \_ /usr/sbin/apache2 -DINFO -DSUPHP -d /usr/ 21090 ? S 0:00 \_ /usr/sbin/apache2 -DINFO -DSUPHP -d /usr/ 21093 ? S 0:00 \_ /usr/sbin/apache2 -DINFO -DSUPHP -d /usr/ 21094 ? S 0:00 \_ /usr/sbin/apache2 -DINFO -DSUPHP -d /usr/ 21096 ? S 0:00 \_ /usr/sbin/apache2 -DINFO -DSUPHP -d /usr/ 21099 ? S 0:00 \_ /usr/sbin/apache2 -DINFO -DSUPHP -d /usr/ 21100 ? S 0:00 \_ /usr/sbin/apache2 -DINFO -DSUPHP -d /usr/ 21102 ? S 0:00 \_ /usr/sbin/apache2 -DINFO -DSUPHP -d /usr/ 21105 ? S 0:00 \_ /usr/sbin/apache2 -DINFO -DSUPHP -d /usr/ 21106 ? S 0:00 \_ /usr/sbin/apache2 -DINFO -DSUPHP -d /usr/ 21108 ? S 0:00 \_ /usr/sbin/apache2 -DINFO -DSUPHP -d /usr/ 21111 ? S 0:00 \_ /usr/sbin/apache2 -DINFO -DSUPHP -d /usr/ 21112 ? S 0:00 \_ /usr/sbin/apache2 -DINFO -DSUPHP -d /usr/ 21114 ? S 0:00 \_ /usr/sbin/apache2 -DINFO -DSUPHP -d /usr/ 21117 ? S 0:00 \_ /usr/sbin/apache2 -DINFO -DSUPHP -d /usr/ 21118 ? S 0:00 \_ /usr/sbin/apache2 -DINFO -DSUPHP -d /usr/ 21120 ? S 0:00 \_ /usr/sbin/apache2 -DINFO -DSUPHP -d /usr/ 21123 ? S 0:00 \_ /usr/sbin/apache2 -DINFO -DSUPHP -d /usr/ 21124 ? S 0:00 \_ /usr/sbin/apache2 -DINFO -DSUPHP -d /usr/ 21126 ? S 0:00 \_ /usr/sbin/apache2 -DINFO -DSUPHP -d /usr/ 21129 ? S 0:00 \_ /usr/sbin/apache2 -DINFO -DSUPHP -d /usr/ 21130 ? S 0:00 \_ /usr/sbin/apache2 -DINFO -DSUPHP -d /usr/ 21132 ? S 0:00 \_ /usr/sbin/apache2 -DINFO -DSUPHP -d /usr/ 21135 ? S 0:00 \_ /usr/sbin/apache2 -DINFO -DSUPHP -d /usr/ 21136 ? S 0:00 \_ /usr/sbin/apache2 -DINFO -DSUPHP -d /usr/ 21138 ? S 0:00 \_ /usr/sbin/apache2 -DINFO -DSUPHP -d /usr/ This list was generated with ps axf, all of the above are definitely processes and not threads. The following shows modules compiled into apache: charm ~ # apache2 -l Compiled in modules: core.c worker.c http_core.c mod_so.c I am currently using the new layout (net-www/apache-2.0.54-r13) however experience the same behaviour with the old layout (-r8). Apache was compiled with the following use flags: +apache2 -debug -doc -ldap -mpm-leader -mpm-peruser -mpm-prefork -mpm-threadpool +mpm-worker -no-suexec (-selinux) +ssl -static-modules +threads In both the new and old layouts this was observed with the default config, however I have since modified the configs to be suitable for my use and commented out any lines in httpd.conf directed at any mpm other than worker but the same 58 processes are still observed. mod_info shows worker.c as being configured as follows: Current Configuration: StartServers 2 ThreadsPerChild 25 MinSpareThreads 25 MaxSpareThreads 75 MaxClients 150 Listen 80 User apache Group apache I am running gentoo on a Sun Netra T1 AC200. My emerge --info follows: charm ~ # emerge --info Portage 2.0.51.22-r2 (default-linux/sparc/sparc64/2005.0, gcc-3.3.5-20050130, glibc-2.3.3.20040420-r2, 2.4.31-sparc sparc64) ================================================================= System uname: 2.4.31-sparc sparc64 sun4u Gentoo Base System version 1.6.13 dev-lang/python: 2.3.5 sys-apps/sandbox: 1.2.11 sys-devel/autoconf: 2.13, 2.59-r6 sys-devel/automake: 1.4_p6, 1.5, 1.6.3, 1.7.9-r1, 1.8.5-r3, 1.9.5 sys-devel/binutils: 2.15.92.0.2-r10 sys-devel/libtool: 1.5.18-r1 virtual/os-headers: 2.4.23 ACCEPT_KEYWORDS="sparc" AUTOCLEAN="yes" CBUILD="sparc-unknown-linux-gnu" CFLAGS="-O2 -mcpu=ultrasparc -pipe" CHOST="sparc-unknown-linux-gnu" CONFIG_PROTECT="/etc /usr/kde/2/share/config /usr/kde/3/share/config /usr/share/config /var/bind /var/qmail/control" CONFIG_PROTECT_MASK="/etc/gconf /etc/terminfo /etc/env.d" CXXFLAGS="-O2 -mcpu=ultrasparc -pipe" DISTDIR="/usr/portage/distfiles" FEATURES="autoconfig ccache distlocks sandbox sfperms strict" GENTOO_MIRRORS="http://gentoo.blueyonder.co.uk" PKGDIR="/usr/portage/packages" PORTAGE_TMPDIR="/var/tmp" PORTDIR="/usr/portage" PORTDIR_OVERLAY="/usr/local/portage" SYNC="rsync://rsync.gentoo.org/gentoo-portage" USE="sparc adns apache2 authdaemond bash-completion berkdb bind-mysql bzip2 crypt curl extensions gcc64 gd imagemagick imap innodb ithreads jpeg libwww maildir memlimit mime mpeg mpm-worker mysql ncurses nls pam perl php png python readline sasl session shaper sockets softquota spell ssl tcpd threads tiff truetype unicode vhosts xml xml2 zlib userland_GNU kernel_linux elibc_glibc" Unset: ASFLAGS, CTARGET, LANG, LC_ALL, LDFLAGS, LINGUAS, MAKEOPTS This is purely speculation to me but it would seem that mpm-worker creates two children (as it should) but then with it comes to make listener and worker threads it instead creates processes. It would appear to create to correct number of children given my configuration, but as processes and not threads. Thanks, Charles Leahy
Sorry, one correction, 57 and *not* 58 processes are being started.
I have now tested this with a clean install on another sparc box using the default gentoo apache confs, and the following use flags: [ebuild R ] net-www/apache-2.0.54-r13 +apache2 -debug +doc -ldap -mpm-leader -mpm-peruser -mpm-prefork -mpm-threadpool +mpm-worker -no-suexec (-selinux) +ssl -static-modules +threads 0 kB The same problem occurs, this time launching 59 processes. Again, apache2 -l shows worker.c being built in. On both boxes I am using: [ebuild R ] dev-libs/apr-0.9.6-r3 -ipv6 0 kB [ebuild R ] dev-libs/apr-util-0.9.6-r2 +berkdb -gdbm -ldap 0 kB Thanks
From what I can tell by reading the source code, it looks like this may be an issue with pthread_create. Can you run ps -eLf so we can check the threads for each process? I suspect threading is not working at all on your machine. Is there another threaded application you can check to see if it's behaving the same way?
I have the same problem. pthreads are definately broken. I tried switching from SMP to non-SMP and recompiling glibc, but no joy. This is very annoying.
It seems confirmed this is an issue with pthreads, and not an apache issue. I'm not sure how to fix that, other then starting completely over and installing again. There's not anything the apache team can about this though.
