Bugzilla Bug 149467

The current spawn-fcgi init script isn't very flexible. It only allows you to
run one FastCGI application. It is quite likely that people would like to run
more than one. I personally run PHP, Rails and Trac all from the same machine.
I have written a new init script and config file that allows you to run several
FastCGI applications by symlinking. It works in a similar way to the net.*
scripts except that the configuration files are separate. For example, for
Trac, you would symlink from /etc/init.d/spawn-fcgi to /etc/init.d/fcgi.trac
and you would copy the /etc/conf.d/spawn-fcgi configuration file to
/etc/conf.d/fcgi.trac. Simple. (-: The user is prevented from starting
/etc/init.d/spawn-fcgi directly.

Please also see bug #149465.

Created an attachment (id=98347) [details]
spawn-fcgi init script

Created an attachment (id=98348) [details]
spawn-fcgi conf file

Created an attachment (id=98349) [details]
spawn-fcgi init script

Oops. Last minute fixes!

# Thilo Bangert <bangert@gentoo.org> (30 Sep 2006)
# packages have been absorbed by lighttpd - pending removal (30 Oct), bug
149599
net-www/spawn-fcgi
dev-libs/localizer

Will be removed from portage -> WONTFIX.

I'm fairly certain the spawn-fcgi executable is not being removed from the
lighttpd package. It isn't strictly needed in order to run FastCGI applications
but it is if you want to be able to start/stop your applications independently
of the web server. These scripts are still very relevant to the lighttpd
package. I'm giving you something useful here, please don't throw it back in my
face.

As I expected, spawn-fcgi is still being installed as part of lighttpd. Please
reconsider these scripts.

has been removed

WILL SOMEONE PLEASE READ WHAT I AM WRITING!! spawn-fcgi is included AS PART OF
LIGHTTPD. Check it for yourself. Run "ls /usr/bin/spawn-fcgi". Check the
lighttpd ebuild. Just stop ignoring me.

reassigning to hopefully the right people

hhm, i have not looked at the script - but it sounds indeed usefull. 

i don't know what the spawn-fcgi init.d script starts now, but i don't like the
idea of adding something which breaks existing setup's...

do you think it would be possible to modify your startup script in a way that
it keeps current setups running and only add's this extra functionality if it
is wanted? (again, i haven't looked at the script yet - sorry)

thanks

Phew. (-:

Hmmm. It might be possible but it would be very awkward and confusing because
the new script works in a very different way. It's not supposed to be run
directly. By default, the old script starts PHP. A nicer solution would be to
include a small function in the ebuild or elsewhere that would convert an
existing config file to the new format and rename any spawn-fcgi symlinks in
/etc/runlevel.

Hmm, migrating to this new script will be tricky.  I'm planning on moving away
from our own set of configs and moving to what upstream uses (rather what they
will soon be using), so I'd prefer to kill these two birds with just the one
stone.

Regardless, I'm working on this - thanks muchly for the bug and the improved
scripts!

Has there been any progress made in this?
I'm running different php-versions as fcgi and would therefore prefer to use
the "portage-way" of maintaining the spawn-fcgi scripts.

I still stand by my scripts. I haven't seen any progress with what beu was
talking about. I think he's left Gentoo actually.

Don't know if this is the right place, but a remark: on my setup, it seems that
the init script for spawn-fcgi has a problem: the php-cgi process spawned
remains tied to pid file. So it puts all the statistics in pid file and at
"stop" you can imagine the mess.

I'm using spawn-fcgi with nginx, but this seems nor related to this.

maybe thr problem lies here?
        env - ${E} ${EX} 2>${SPAWNFCGI_PID}

You are referring to the current scripts, not my new ones. I understand what
you're saying though. Perhaps another reason to give my scripts a try?

(In reply to comment #16)
> You are referring to the current scripts, not my new ones. I understand what
> you're saying though. Perhaps another reason to give my scripts a try?
> 

Yes, indeed. With your script all works just fine; with the old one my box
always get stuck on shutdown, forcing a reset leading do a unwanted fsck.
I use it to fire up both php-cgi and a script that takes care of perl cgi
handling (something found on internet, wandering if there is something packaged
:) )

I strongly suggest to adopt your script, my only advice is to change (as I did
on my box) the variables according to the script now bundled with gentoo
ebuild, so it's possible to replace the actual one seamlessly, without having
to make changes on configuration script.

Actually, I'm thinking to make a custom ebuild able to install only spawn-fcgi
script and possibly the init script: I'm using nginx and not lighttp and now
I've to keep lighty installed just for spawn-fcgi. Maybe this will be useful to
anyone using nginx...

(In reply to comment #15)
> ... it seems that
> the init script for spawn-fcgi has a problem: the php-cgi process spawned
> remains tied to pid file. So it puts all the statistics in pid file and at
> "stop" you can imagine the mess. ...

This problem is driving me nuts too.
Could, please, some dev look into it or even better finish and include the
init-scripts from this bug-report?

thanks a lot.

I am, in fact, disappointed to find that www-servers/spawn-fcgi was removed
(bug 149599).  www-servers/spawn-fcgi might have been added to support Lighttpd
initially, but there are many other web servers that don't have built-in tools
to spawn FastCGI daemons.  These web servers, like NginX, would benefit greatly
from a stand-alone www-servers/spawn-fcgi package.  It should not be depended
upon by any package, but simply be available as a generic FastCGI spawner for
use with any web server.

Googling about using PHP and NginX yields a lot of custom-hacked init scripts
for launching php-fastcgi (none specific to Gentoo that I could find) and
instructions for "borrowing" Lighttpd's spawn-fcgi.

If www-servers/spawn-fcgi is re-added as a stand-alone tool, that would be a
good chance to look into James's improved init script as well.

I totally agree. Currently I have lighttpd installed just for not having to
manually install/configure spawn-fcgi.

Do you think you could add the fix in your script, so it DOESN'T ignore the
FCGI_WEB_SERVER_ADDRS from the config files?

See my initial bug report and quick fix in
http://bugs.gentoo.org/show_bug.cgi?id=215247

Also consider adding chroot support:
http://bugs.gentoo.org/show_bug.cgi?id=223233

Created an attachment (id=158397) [details]
spawn-fcgi conf file

Created an attachment (id=158401) [details]
spawn-fcgi init script

Here is an update.

It includes a fix that prevented "stop" from working on OpenRC systems. I'm not
sure whether the RC_SVCNAME variable is set on pre-OpenRC systems. Hopefully it
is!

I have changed FCGI_WEB_SERVER_ADDRS to FCGI_ADDRESS and it actually works now.
I thought this variable was just picked up automatically by spawn-fcgi but now
I realise it's not. It's important to note that only one IP can actually be
specified. I checked the source code to verify this.

I have also added an FCGI_CHROOT option.

Another interesting point to mention is that when starting spawn-fcgi with the
-u (user) option, that user's additional groups do not take effect. I think
spawn-fcgi would have to be modified for this to work. An easy workaround is to
launch it through sudo instead. I could do this by default if sudo is available
but unusual sudo configurations would break it.

Exchange RC_SVCNAME with SVCNAME and it will work.

Besides of that it seems to work well.

Even better, this also seems to have fixed some weird "unexpected end-of-file"
error in the interaction of lighttpd <-> trac.

Created an attachment (id=158413) [details]
spawn-fcgi init script

Done.

Created an attachment (id=158415) [details]
New init.d script with the abovementioned fixes

Fixed the init.d script:
- RC_SVCNAME -> SVCNAME
- Made the shutdown process work by killing in a loop, instead of killing
concated pids (123 + 456 -> 123456 -> no such process)
- Also let a failed shutdown (namely the kill command) spit out an error
message, since otherwise it might be hard to debug (no syslog messages, i.e.)

Actually I killed them in a loop before but couldn't think why I'd done it like
that. Now I realise why. Line breaks act as separators but spawn-fcgi doesn't
add a line break like all the other PID files on my system. Sorry!

Created an attachment (id=158417) [details]
spawn-fcgi init script

You missed an instance of RC_SVCNAME.

Created an attachment (id=171677) [details]
patch to the old spawn-fcgi

I think the maintainer makes a good point about using the upstream methods, but
a general purpose FCGI spawner does sound very useful.

However, as long as we're using the old Gentoo spawn-fcgi, please apply the
above diff. It actually captures the php-cgi PID for stopping and restarting
the service.

-p

Maybe the name of new script be changed to /etc/init.d/gentoo-spawn-fcgi etc
...
And then this could finally get in! I ignore if any Gentoo dev. is actually
running multiple fastcgi websites with the current scripts without heavy
hacking (which could be agilely factorized): if so then maybe we could know
more about it? Else we need this feature in; without BC break of course please
:)
Thanks for sharing the script. Cheers.

I'd be pushing for this more if it weren't for the fact that most of my servers
run Ubuntu. XD Not really my choice though.

Please fix the BC break to make it acceptable into Gentoo.
Thanks again for the scripts! Cheers.

I'd really love to see:
- The spawn-fcgi functionality being removed from net-www/lighttpd
- The spawn-fcgi ebuild re-added to the portage tree
- The spawn-fcgi making use of this improved script
- Having 'fastcgi? ( net-www/spawn-fcgi )' in net-www/lighttpd,
net-www/nginx,...

I'm running nginx here and this just feels really strange having lighttpd
installed only to provide the spawn-fcgi bin which seems to be split from
lighttpd now anyways: http://redmine.lighttpd.net/projects/spawn-fcgi

I'm now using nginx as well so I have the same issue.

Created an attachment (id=183637) [details]
Patch to add check whether both of FCGI_PORT and FCGI_SOCKET are defined

This patch adds a check whether both of FCGI_PORT and FCGI_SOCKET are defined.
Until now the script just failed silently and only "--debug" showed the actual
issue.
Now the script checks for this before.

Created an attachment (id=183638) [details]
Patch to add check whether both of FCGI_PORT and FCGI_SOCKET are defined

My previous patch checked for FCGI_ADDRESS ... don't ask me why ;-)

This one does now correctly by checking for FCGI_PORT and FCGI_SOCKET

(From update of attachment 171677 [details])
this patch is a dupe of #252268 - please only open separate bugs for seperate
bugs ;-)

www-servers/spawn-fcgi is now in portage using this script - please test.
thanks for your contribution and perseverance.

feedback welcome!

kind regards
Thilo

Is this supposed to work with php-cgi? If so how?

I copied /etc/conf.d/spawn-fcgi /etc/conf.d/spawn-fcgi.test and set:
FCGI_PROGRAM=/usr/bin/php-cgi
In spawn-fcgi.test

I symlinked /etc/init.d/spawn-fcgi to /etc/init.d/spawn-fgi.test

And yet when I try to start it I get:

 # /etc/init.d/spawn-fcgi.test start
 * Starting FastCGI application test ...
spawn-fcgi: child spawned successfully: PID: 10595        [ !! ]

I end up with no php-cgi processes running.

If I mod the init script slightly to echo the start-stop-daemon line and then
manually paste that into my shell, everything starts up fine. Which is odd,
because it's clearly not working from within the init script.

Any suggestions?

(In reply to comment #42)
> Is this supposed to work with php-cgi? If so how?
> 
> I copied /etc/conf.d/spawn-fcgi /etc/conf.d/spawn-fcgi.test and set:
> FCGI_PROGRAM=/usr/bin/php-cgi
> In spawn-fcgi.test
> 
> I symlinked /etc/init.d/spawn-fcgi to /etc/init.d/spawn-fgi.test
> 
> And yet when I try to start it I get:
> 
>  # /etc/init.d/spawn-fcgi.test start
>  * Starting FastCGI application test ...
> spawn-fcgi: child spawned successfully: PID: 10595        [ !! ]
> 
> I end up with no php-cgi processes running.
> 
> If I mod the init script slightly to echo the start-stop-daemon line and then
> manually paste that into my shell, everything starts up fine. Which is odd,
> because it's clearly not working from within the init script.
> 
> Any suggestions?
> 

I can confirm this. The script spawns the process, gets a PID and immediately
kills this pid (= fails).
If it helps, I can post the relevant --debug output of the script.

(In reply to comment #43)
> I can confirm this. The script spawns the process, gets a PID and immediately
> kills this pid (= fails).
> If it helps, I can post the relevant --debug output of the script.

I had this troubles too - that's because the script does nearly no error
checking at all and some arguments are incompatible.

I did a rewrite of the script which still needs some polishing, but I'll post
this rewrite here as soon as possible.

Regards, Elias P.

Hang on, the version I posted works fine with PHP though. Something must have
been changed. Having a look...

thanks for the testing and feedback everybody.

the script was indeed changed - it now uses start-stop-daemon among other
things.

thomasg: debug output is always welcome. just stating that it doesnt work is
not very helpful.

elias p.: the patch you uploaded has been applied. if there is more you deem
necessary it is more than welcome.

php-cgi works overhere. make sure that /usr/bin/spawn-fcgi and
/etc/init.d/spawn-fcgi are in fact the one from the www-servers/spawn-fcgi
package:
$ equery belongs /etc/init.d/spawn-fcgi

and that you have updated your conf files (using etc-update or dispatch-conf).
more testing and input welcome.

Created an attachment (id=187181) [details]
debug log when starting spawn-fcgi using /usr/bin/php-cgi as FCGI_PROGRAM

# equery b /etc/init.d/spawn-fcgi
[ Searching for file(s) /etc/init.d/spawn-fcgi in *... ]
www-servers/spawn-fcgi-1.6.1 (/etc/init.d/spawn-fcgi)
# equery b /etc/conf.d/spawn-fcgi
[ Searching for file(s) /etc/conf.d/spawn-fcgi in *... ]
www-servers/spawn-fcgi-1.6.1 (/etc/conf.d/spawn-fcgi)

my /etc/conf.d/spawn-fcgi.php differs from the /etc/conf.d/spawn-fcgi like
this:
--- /etc/conf.d/spawn-fcgi      2009-04-03 11:42:06.000000000 +0100
+++ /etc/conf.d/spawn-fcgi.php  2009-04-02 14:26:08.000000000 +0100
@@ -22,7 +22,7 @@
 # extension but not always. For PHP, you should usually point this to
 # /usr/bin/php-cgi.
 #
-FCGI_PROGRAM=/path/to/application.fcgi
+FCGI_PROGRAM=/usr/bin/php-cgi

 # The number of child processes to spawn. The default is 1. For PHP
 # applications, set this to 1 and use PHP_FCGI_CHILDREN instead.
@@ -37,8 +37,8 @@
 # The user and group to run your application as. If you do not specify these,
 # the application will be run as root:root.
 #
-FCGI_USER=
-FCGI_GROUP=
+FCGI_USER=nginx
+FCGI_GROUP=nginx

 # If your application requires additional environment variables, you need to
 # allow them here. Only the variables specified here, plus a few others
@@ -50,5 +50,5 @@
 # of child processes to spawn. The second is the number of requests to be
 # served by a single PHP process before it is restarted.
 #
-PHP_FCGI_CHILDREN=5
+PHP_FCGI_CHILDREN=2
 PHP_FCGI_MAX_REQUESTS=500


Entire debug log attached as a text file, this was the output from:
/etc/init.d/spawn-fcgi.php start --debug 1>>output.txt 2>>output.txt

i have comitted a new version of the init script - i dont think it will fix the
issue described here, but who knows. you will have to remerge spawn-fcgi.

also try running strace before php, like so:
FCGI_PROGRAM="/usr/bin/strace -o /tmp/spawn-fcgi -ff /usr/bin/php-cgi"

can somebody on amd64 confirm that it work for them with php5?

a posixified version of the init script has been deployed. remerge spawn-fcgi
to test it.

Created an attachment (id=187283) [details]
Main process strace information

Using 

FCGI_PROGRAM="/usr/bin/strace -o /tmp/spawn-fcgi -ff /usr/bin/php-cgi"

Things seem to work normally. I get 5 php-cgi instances running and the various
files under /tmp created by strace. For what it's worth, at this point nothing
dies. If I think proceed to browse to my php app everything works as expected,
even if the /etc/init.d/spawn-fcgi start does return [ !! ]

If I revert back to just
FCGI_PROGRAM="/usr/bin/php-cgi"
I get the same behaviour as before.

Created an attachment (id=187285) [details]
strace output from one of the children

Strace output from one child, I diffed it against all other children and they
returned no differences.

hhm, strange. i am at a complete loss....

one last thing you could try: call strace inbetween start-stop-daemon and
spawn-fcgi... your terminal won't return using the followin patch, but it may
give us some insight.

how many people are experiencing this problem?


--- spawn-fcgi  2009-04-06 20:08:44.000000000 +0200
+++ spawn-fcgi.orig     2009-04-06 20:06:40.000000000 +0200
@@ -75,8 +75,8 @@
                [ -n "${FCGI_SOCKET}" ] && SOCKET_OPTION="-s
${FCGI_SOCKET}-${X}"
                [ -n "${FCGI_PORT}" ] && INET_OPTION="-a ${FCGI_ADDRESS} -p
$((${FCGI_PORT} + ${X} - 1))"

-               start-stop-daemon --start --pidfile ${P} ${E} --exec strace --
\
-               -o /tmp/spawn-fcgi -ff ${SPAWNFCGI} ${SOCKET_OPTION}
${INET_OPTION} -P ${P} ${OPTIONS} -- ${FCGI_PROGRAM}
+               start-stop-daemon --start --pidfile ${P} ${E} --exec
${SPAWNFCGI} -- \
+                       ${SOCKET_OPTION} ${INET_OPTION} -P ${P} ${OPTIONS} --
${FCGI_PROGRAM}
                RETVAL=$?

                # Stop on error. Don't want to spawn a mess!

Sorry I didn't try this until now, been very busy. I guessed right first time
what the problem was. The PID file directory doesn't get created automatically
and you removed the line to create it. Add that back and all will be well.

Tested and working on amd64.

the pid file directory is created by the ebuild. it shouldnt be necessary to
create it in the startup script...

anyway - i have added it back. please remerge spawn-fcgi and retest.
gary: does this fix your problem?

Hmm you're right, I hadn't checked the ebuild. I just assumed my directory was
there from before. Maybe that's not it. It works fine for me other than that
though.

To confuse matters, if I apply the patch in comment 52, although the terminal
doesn't return, it seems to work fine. I get 5 php-cgi children started.

Are the files generated by strace any use if it works with the above patch?

It still doesn't work if I revert back to the original init script.

What I also find odd is that if I add an echo in front of the original
start-stop-daemon line just to see what it's actually executing, and then copy
and paste that into a shell, it works as I would expect it to. Yet when the
exact same line runs in the init script RETVAL=1 for some reason

ok - i am closing this bug (the new init script exists).
the issue with the init script not working is now tracked at bug 265454 -
anybody having that problem is kindly asked to CC himself there (also please
attach emerge --info)...

thanks for your input and help!
kind regards
Thilo