Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 300619 - [EBUILD] nginx 0.8.32 with a bunch of additional modules
Summary: [EBUILD] nginx 0.8.32 with a bunch of additional modules
Status: RESOLVED WONTFIX
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: [OLD] Server (show other bugs)
Hardware: All Linux
: High enhancement
Assignee: Benedikt Böhm (RETIRED)
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2010-01-11 19:52 UTC by steveb
Modified: 2011-04-19 19:02 UTC (History)
2 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
www-servers/nginx/files/nginx.conf-r4 (nginx.conf-r4,1.38 KB, text/plain)
2010-01-11 19:54 UTC, steveb
Details
www-servers/nginx/files/nginx.init-r2 (nginx.init-r2,1.62 KB, text/plain)
2010-01-11 19:54 UTC, steveb
Details
www-servers/nginx/files/nginx.logrotate (nginx.logrotate,332 bytes, text/plain)
2010-01-11 19:54 UTC, steveb
Details
www-servers/nginx/files/udplog_0_8_32.patch (udplog_0_8_32.patch,997 bytes, patch)
2010-01-11 19:55 UTC, steveb
Details | Diff
www-servers/nginx/nginx-0.8.32-r1.ebuild (nginx-0.8.32-r1.ebuild,18.80 KB, text/plain)
2010-01-11 19:55 UTC, steveb
Details
nginx-0.8.32.patch (nginx-0.8.32.patch,21.16 KB, patch)
2010-01-11 22:40 UTC, steveb
Details | Diff
clean-nginx-0.8.32.patch (clean-nginx-0.8.32.patch,9.07 KB, patch)
2010-01-14 13:14 UTC, steveb
Details | Diff
www-servers/nginx/nginx-0.8.34-r1.ebuild (nginx-0.8.34-r1.ebuild,19.09 KB, text/plain)
2010-03-04 17:53 UTC, steveb
Details
eclass/nginx.eclass (nginx.eclass,22.90 KB, text/plain)
2010-03-25 10:09 UTC, steveb
Details
www-servers/nginx/nginx-0.8.34-r2.ebuild (nginx-0.8.34-r2.ebuild,322 bytes, text/plain)
2010-03-25 10:13 UTC, steveb
Details

Note You need to log in before you can comment on or make changes to this bug.
Description steveb 2010-01-11 19:52:21 UTC
Here my nginx 0.8.32 Ebuild with a bunch of additional modules for nginx.

Reproducible: Always
Comment 1 steveb 2010-01-11 19:54:08 UTC
Created attachment 216107 [details]
www-servers/nginx/files/nginx.conf-r4
Comment 2 steveb 2010-01-11 19:54:29 UTC
Created attachment 216108 [details]
www-servers/nginx/files/nginx.init-r2
Comment 3 steveb 2010-01-11 19:54:42 UTC
Created attachment 216109 [details]
www-servers/nginx/files/nginx.logrotate
Comment 4 steveb 2010-01-11 19:55:02 UTC
Created attachment 216111 [details, diff]
www-servers/nginx/files/udplog_0_8_32.patch
Comment 5 steveb 2010-01-11 19:55:23 UTC
Created attachment 216112 [details]
www-servers/nginx/nginx-0.8.32-r1.ebuild
Comment 6 Dirkjan Ochtman (RETIRED) gentoo-dev 2010-01-11 20:06:41 UTC
Please send a diff for all files you changed, it's not clear exactly what you want to change.
Comment 7 steveb 2010-01-11 20:25:49 UTC
(In reply to comment #6)
> Please send a diff for all files you changed, it's not clear exactly what you
> want to change.
> 
Have you looked at the Ebuild included in this bug report? You really want me to send a diff?

The Ebuild I included has 25 additional modules for nginx. And it allows one to compile nginx without any HTTP stuff (aka if you want to use nginx as a Mail/SMTP/POP proxy). The nginx Ebuild currently in Portage does not provide that kind of flexibility.

Beside that this Ebuild here should produce leaner nginx binary since it does only link against Perl, zlib, etc if one is using HTTPD functionality (since those modules are only used in the HTTPD code).
Comment 8 Dirkjan Ochtman (RETIRED) gentoo-dev 2010-01-11 22:17:27 UTC
I've looked at it briefly, but diffs are much more useful.
Comment 9 steveb 2010-01-11 22:40:06 UTC
Created attachment 216135 [details, diff]
nginx-0.8.32.patch

(In reply to comment #8)
> I've looked at it briefly, but diffs are much more useful.
> 
It's not hard to download the Ebuild from b.g.o and make the diff. Anyway... since you insist in having a diff: here you go.

I did not added udplog_0_8_32.patch to the diff since that file is a new file and not a change.
Comment 10 Dirkjan Ochtman (RETIRED) gentoo-dev 2010-01-14 09:40:14 UTC
Thanks for attaching a patch. However, we will not be supporting nginx extensions that are not part of the nginx tree for the foreseeable future. I'd still be interested in supporting the http use flag, if you can do a patch for that.
Comment 11 steveb 2010-01-14 11:31:12 UTC
(In reply to comment #10)
> Thanks for attaching a patch. However, we will not be supporting nginx
> extensions that are not part of the nginx tree for the foreseeable future.
>
Bummer! Might I ask: why?


> I'd
> still be interested in supporting the http use flag, if you can do a patch for
> that.
> 
That's how I started. Long, long ago. I think nginx was at that time not in Portage. Later I looked at the stock Gentoo Ebuild and continued with my own Ebuild. I even asked Igor about some compile options that I had the impression where not optimal in the stock Gentoo Ebuild: http://nginx.org/pipermail/nginx/2009-October/016106.html

Anyway... I am going to remove all the additional patches and make a Ebuild without the 3th party modules and post it here.
Comment 12 steveb 2010-01-14 13:14:41 UTC
Created attachment 216494 [details, diff]
clean-nginx-0.8.32.patch

(In reply to comment #10)
> I'd
> still be interested in supporting the http use flag, if you can do a patch for
> that.
> 
Okay. Here you go. A patch that patches the current 0.8.31 Ebuild to enable the http USE flag.

The new Ebuild is using EAPI 2 so a bunch of things are reordered to follow the Gentoo style guide.

The original Ebuild just installs the logrotate script regardless if the user has logrotate installed or not. That is fixed in the new Ebuild.

I added as well the fix for the AIO module on ARM platforms. See http://nginx.org/pipermail/nginx/2009-September/015726.html for more infos.

I added as well the "xslt" USE flag so that users having enabled it can use XSLT in nginx.

And I cleaned the Ebuild to not include a double / since "$ROOT" is already "/" or is containing a path wiht "/" at the end. So there is no need to add an additional "/" after using "$ROOT".


btw: It's a pity Gentoo will not add additional modules to nginx. I know that I went to far with adding 25 3th party modules but some of them are really useful. Stuff like the PAM module that allows you to use PAM for authentication or the upstream_fair that allows you to do fair load balancing against upstream servers are real lifesavers. Is there any reason why Gentoo wont add 3th party modules to nginx?
Comment 13 Oleg Gawriloff 2010-02-09 23:27:37 UTC
> btw: It's a pity Gentoo will not add additional modules to nginx. I know that I
> went to far with adding 25 3th party modules but some of them are really
> useful. Stuff like the PAM module that allows you to use PAM for authentication
> or the upstream_fair that allows you to do fair load balancing against upstream
> servers are real lifesavers. Is there any reason why Gentoo wont add 3th party
> modules to nginx?

I've agree. Why we can't have new port, for example nginx-modules or smth like this?
Comment 14 Dirkjan Ochtman (RETIRED) gentoo-dev 2010-02-10 08:06:37 UTC
Because I don't want to spend the time to maintain it.
Comment 15 steveb 2010-02-11 12:27:19 UTC
(In reply to comment #13)
> I've agree. Why we can't have new port, for example nginx-modules or smth like
> this?
> 
Because nginx build system does not offer that possibility.
Comment 16 steveb 2010-03-04 17:53:34 UTC
Created attachment 222061 [details]
www-servers/nginx/nginx-0.8.34-r1.ebuild
Comment 17 Benedikt Böhm (RETIRED) gentoo-dev 2010-03-04 18:25:10 UTC
please take a look at my overlay, it will soon be merged into the main tree. it contains NGINX_ADD_MODULES and USE_EXPANDed modules

http://git.xnull.de/gitweb/?p=overlay.git;a=tree;f=www-servers/nginx
Comment 18 Benedikt Böhm (RETIRED) gentoo-dev 2010-03-07 10:45:11 UTC
NGINX_ADD_MODULES support is in 0.8.34-r1 now
Comment 19 steveb 2010-03-07 12:15:31 UTC
(In reply to comment #18)
> NGINX_ADD_MODULES support is in 0.8.34-r1 now
> 
Can you elaborate how to add custom modules to nginx with the new Ebuild? Is there any documentation available for adding custom modules to nginx Ebuild?
Comment 20 Benedikt Böhm (RETIRED) gentoo-dev 2010-03-07 12:46:10 UTC
just add NGINX_ADD_MODULES to make.conf, with paths to the module sources:

NGINX_ADD_MODULES="/usr/src/nginx/foo_module/"
Comment 21 steveb 2010-03-07 12:58:57 UTC
(In reply to comment #20)
> just add NGINX_ADD_MODULES to make.conf, with paths to the module sources:
> 
> NGINX_ADD_MODULES="/usr/src/nginx/foo_module/"
> 
Ouch. This is IMHO totally unhandy. For many reasons:
- Module documentation does not get installed that way
- Some modules (aka: nginx_udplog_module, mod-zip-module, nginx_upstream_jvm_route, ngx_supervisord, etc) require patching of nginx and the above method does not provide a mechanism for doing that
- Some modules depend on HTTP and the mechanism above does not provide an option to set dependencies

It is however better then nothing but still I find it very unpractical. Somehow nginx is in Gentoo land a redheaded stepchild.
Comment 22 Benedikt Böhm (RETIRED) gentoo-dev 2010-03-07 13:26:23 UTC
The problem is that nginx does not support run-time loading of modules and it just is too much overhead to maintain all the third-party modules in the nginx ebuild itself. if you need a lot of custom modules you are probably better of creating an nginx ebuild in your overlay. if needed this could be simplified with some eclass magic.
Comment 23 steveb 2010-03-07 14:09:49 UTC
(In reply to comment #22)
> The problem is that nginx does not support run-time loading of modules
>
Right.


> and it
> just is too much overhead to maintain all the third-party modules in the nginx
> ebuild itself.
>
This obviously does not apply to the passenger module currently added in 0.8.34-r1. I guess you self are needing that module and that is the reason it is included in r1. Right? Why is the new nginx Ebuild not constant regarding that issue? Is there any reason that the passenger module has made it into the Ebuild and the others not? IMHO all third party modules should use the new mechanism included in r1.


> if you need a lot of custom modules you are probably better of
> creating an nginx ebuild in your overlay.
>
This is what I currently do.


> if needed this could be simplified
> with some eclass magic.
> 
This would +/- move the complexity from the Ebuild into a Eclass. I mean the maintenance complexity. Still one would need to update the Eclass when a new version of a module gets out.
Comment 24 Benedikt Böhm (RETIRED) gentoo-dev 2010-03-07 15:15:19 UTC
(In reply to comment #23)
> This obviously does not apply to the passenger module currently added in
> 0.8.34-r1.

passenger is quite hard to get right in nginx, and needs patches. if passenger wouldn't be such a popular solutions i woouldn't have added it. and yes, one of my customers uses it, so i can at least test it.

most other modules are simple C files, and can be added via NGINX_ADD_MODULES. i don't think documentation needs to be installed in this case, since you already know what you're doing.

if you have a module that requires some more logic please open another bug, and i can consider adding it to the ebuild.
Comment 25 steveb 2010-03-25 10:09:38 UTC
Created attachment 225187 [details]
eclass/nginx.eclass

I know, I know. This bug report is closed. Anyway... I replaced my other Ebuild to use a nginx Eclass. The mechanism introduced by Benedikt Böhm <hollow@gentoo.org> in 0.8.34-r1 for adding modules to nginx is now included in this Eclass.
Comment 26 steveb 2010-03-25 10:13:32 UTC
Created attachment 225189 [details]
www-servers/nginx/nginx-0.8.34-r2.ebuild

The Ebuild using the new Eclass. The version numbers of the modules can be overwritten inside the Ebuild (if needed). If the versions are not overwritten in the Ebuild then the version numbers from the Eclass are used. They are:
-------------------------
USE_RUBY="ruby18"
RUBY_OPTIONAL="yes"
PASSENGER_PV="2.2.11"
MODULE_ACCEPT_LANGUAGE_PV="02262ce"
MODULE_ACCESSKEY_PV="2.0.3"
MODULE_AUTH_REQUEST_PV="0.2"
MODULE_BYTES_FILTER_PV="57365655ee44"
MODULE_CACHE_PURGE_PV="1.0"
MODULE_CHUNKIN_PV="cb610a5"
MODULE_DRIZZLE_PV="aa3269a"
MODULE_ECHO_PV="d388079"
MODULE_EVAL_PV="e85e11e"
MODULE_EVENTS_PV="49ab119a468c"
MODULE_GUNZIP_PV="d1b72979e4f1"
MODULE_H264_PV="2.2.7"
MODULE_HEADERS_MORE_PV="db9913e"
MODULE_IP_TOS_FILTER_PV="a23404790f33"
MODULE_MODZIP_PV="1.1.5"
MODULE_MOGILEFS_PV="1.0.3"
MODULE_AUTH_PAM_PV="1.1"
MODULE_PUSH_PV="0.692"
MODULE_RDS_JSON_PV="9cf3bef"
MODULE_REDIS_PV="0.3.1"
MODULE_SLOWFS_CACHE_PV="1.3"
MODULE_SUPERVISORD_PV="1.3"
MODULE_UDPLOG_PV="1.0.0"
MODULE_UPLOAD_PROGRESS_PV="c740674"
MODULE_UPLOAD_PV="2.0.12"
MODULE_UPSTREAM_FAIR_PV="2131c73"
MODULE_UPSTREAM_JVM_ROUTE_PV="0.1"
MODULE_UPSTREAM_KEEPALIVE_PV="2ce9d8a1ca93"
MODULE_XSS_PV="58732b0"
-------------------------

Beside the added modules everything should work like with the current nginx Ebuild in portage. So you can still use NGINX_MODULES_HTTP, NGINX_MODULES_MAIL and NGINX_ADD_MODULES.
Comment 27 steveb 2011-04-19 17:36:57 UTC
(In reply to comment #24)
> (In reply to comment #23)
> > This obviously does not apply to the passenger module currently added in
> > 0.8.34-r1.
> 
> passenger is quite hard to get right in nginx, and needs patches. if passenger
> wouldn't be such a popular solutions i woouldn't have added it. and yes, one of
> my customers uses it, so i can at least test it.
> 
> most other modules are simple C files, and can be added via NGINX_ADD_MODULES.
> i don't think documentation needs to be installed in this case, since you
> already know what you're doing.
> 
> if you have a module that requires some more logic please open another bug, and
> i can consider adding it to the ebuild.
> 
Have you now soften your policy regarding adding more modules to nginx? I see the latest nginx Ebuilds to include such simple modules as ey-balancer, headers more, cache purge, push, etc... all of them could be ultra easy managed with NGINX_ADD_MODULES. But somehow they manage to get into stock nginx Ebuild. I understood your argument regarding the inclusion of Passanger. But those other modules that are allowed to get included into the nginx Ebuild do not fall into the same 'some more logic' argument as Passanger is.

So something has changed. Are we now allowed to request the inclusion of more modules? Or is there another logic/way how to get modules into stock nginx Ebuild without the usage of NGINX_ADD_MODULES? Why do such simple modules like ey-balancer manage to get into stock nginx Ebuild while other modules are denied and you point to the usage of NGINX_ADD_MODULES in those cases? Can you explain the policy/reason?
Comment 28 Benedikt Böhm (RETIRED) gentoo-dev 2011-04-19 17:51:45 UTC
(In reply to comment #27)
> (In reply to comment #24)
> > (In reply to comment #23)
> > > This obviously does not apply to the passenger module currently added in
> > > 0.8.34-r1.
> > 
> > passenger is quite hard to get right in nginx, and needs patches. if passenger
> > wouldn't be such a popular solutions i woouldn't have added it. and yes, one of
> > my customers uses it, so i can at least test it.
> > 
> > most other modules are simple C files, and can be added via NGINX_ADD_MODULES.
> > i don't think documentation needs to be installed in this case, since you
> > already know what you're doing.
> > 
> > if you have a module that requires some more logic please open another bug, and
> > i can consider adding it to the ebuild.
> > 
> Have you now soften your policy regarding adding more modules to nginx?

from the ebuild:

 - keep the following 3 requirements in mind before adding external modules:
  * alive upstream
  * sane packaging
  * builds cleanly

> I see
> the latest nginx Ebuilds to include such simple modules as ey-balancer, headers
> more, cache purge, push, etc... all of them could be ultra easy managed with
> NGINX_ADD_MODULES. But somehow they manage to get into stock nginx Ebuild. I
> understood your argument regarding the inclusion of Passanger. But those other
> modules that are allowed to get included into the nginx Ebuild do not fall into
> the same 'some more logic' argument as Passanger is.

since passenger clearly violates the last two of the above rules it has been removed from latest nginx ebuilds. regarding ey-balancer: i'm not sure it should be considered as alive upstream and i've also encountered a few problems with, so i'm not sure it'll stay.

regarding other modules: if they fulfill the rules above you can open bugs for them