Created attachment 424504 [details] external ngx-devel-kit livebuild So I don't have to touch the official nginx.ebuild on every update, I wrote separate (live) ebuilds for the modules I need (s. attachment). Those ebuilds install the modules' sources to /usr/src and install a file into /etc/env.d/ with the form: SPACE_SEPARATED="NGINX_ADD_MODULES" NGINX_ADD_MODULES="/usr/src/module" Because I can't explicitly avtivate ngx-devel-kit, I made an ebuild for that aswell. The only disadvantage i see right now is, that nginx won't be rebuilt if a module is updated. An ewarn mentions this. If you like, you could conditionally depend on those modules by USE-flag to overcome the updating issue and to enable users, to select modules implicitly by changing USE-flags. Then, NGINX_ADD_MODULES should probably changed to something like NGINX_UNMAINTAINED_MODULES ...
Created attachment 424506 [details] ngx-http-form-input livebuild
Created attachment 424508 [details] ngx-http-set-misc livebuild
Created attachment 424568 [details] external ngx-devel-kit livebuild - Fixed bug, so ngx-devel-kit is always at the beginning of the NGINX_ADD_MODULES envvar. - Fixed \n not interpreted by echo. - Removed stale $NGINX_ADD_MODULES
Created attachment 424570 [details] ngx-http-form-input livebuild - Removed stale $NGINX_ADD_MODULES
Created attachment 424572 [details] ngx-http-set-misc livebuild - Removed stale $NGINX_ADD_MODULES
Thanks for your contribution! I'll have a think about your approach and get back to you with comments.
Looks like upstream is also working on a solution: https://www.nginx.com/blog/dynamic-modules-nginx-1-9-11/ I hope we will get something like PECL.
So, I've thought about this for a while. The TL;DR is that we should stay off this in the 1.9.x series and see what happens with nginx 2.0 (or 1.10). I've also spoken to the other maintainers to see if we share conclusion. Here's my rationale: 1. We've had a bit of a bad rep with the 1.6, 1.7 series where we were a bit too trigger happy with 3rd party module changes. This is avoided in the 1.8/1.9 series by only bumping (if not specific bugfixes are filed against a module) 1.9 modules. The intent is to have 1.9 brew into the next stable and then let that cool down. 2. There's not a lot of 3rd party modules that supports this just yet 3. The internal api/abi of nginx is moving pretty quick in development series. I reckon this will change with the introduction of 3rd party modules, but again; I feel we would make a premature decision by jumping on this too quickly. With above said I think the next mainline series might be a good place to explore this. Closing as WONTFIX for now.
So is patching the main ebuild still the preferred method to include 3rd-party modules for 1.10.x or can we have separate ebuilds for them now, since nginx supports dynamic modules?
Situation hasn't really changed: Most of our provided 3rd-party modules can't be build as dynamic module yet. And we don't have a solution yet ready for the ebuild to support something like that (will need a complete rewrite and probably an nginx eclass for the modules). We have that on our todo list (I am re-opening this bug report to track status). But it will take some time...
I need to get an ebuild for another 3rd party nginx module and wondered if I still need the "dirty hack" from my initial comment. > Most of our provided 3rd-party modules can't be build as dynamic module yet. There still are lots of those modules and I don't expect that to change in the near future. This is still the main problem I see now. It could be resolved with epatch for every module but that would add more maintenance overhead so I'd like to avoid it. Also, I suppose some people would still like to be able to compile a module directly inside the nginx source tree, so it wouldn't be a good standard solution. The livebuilds I attached in 2016 still work flawlessly for me (but I still need to manually re-emerge nginx after every update, which is annoying). Are there new developments to get 3rd party modules cleanly integrated or any plans?
I was thinking about writing a eclass to build dynamic modules. As I see it depends on the whole nginx source code on build time. What do you think is a good way to get the nginx source in the eclass? Something like the gestreamer eclass where there use the same version on all plugins. This could mean to use a nginx version variable in the modules ebuild. Or is there a better way?
(In reply to jonas.licht from comment #12) > Or is there a better way? Anther way is to copy the nginx src to the system.
I've just wrote an eclass for building dynamic nginx modules. Together with one demo package I opened a PR: https://github.com/gentoo/gentoo/pull/16053
It would be nice to have this ability for distributing third party nginx modules from overlays. Now it is painful: overlay should reflect any update of nginx in the main tree and it is the manual work. Can I help somehow? Or at least kindly ask to give attention to the problem with third party modules. (I would use it to ship nginx upstream module for tarantool as part of tarantool overlay.)
(In reply to Alexander Turenko from comment #15) > It would be nice to have this ability for distributing third party nginx > modules from overlays. Now it is painful: overlay should reflect any update > of nginx in the main tree and it is the manual work. > > Can I help somehow? Or at least kindly ask to give attention to the problem > with third party modules. > > (I would use it to ship nginx upstream module for tarantool as part of > tarantool overlay.) Hi Alexander, my PR is now open for nearly one and a half year. With this eclass we can now provide modules in an overlay. Only on a version change of the nginx server, maintance in the overlay is needed. Maybe we can combine our force and start an overlay for 3rd party nginx modules. What to you think? Best regards, Jonas
The main problem that we have now with shipping 3rd-party modules for nginx is about synchronizing nginx ebuild with one from the main Gentoo tree. If we'll fork it, we'll again meet the same problem. Here I would expect a decision from Gentoo Developers.
Just came here to note that my proposed solution works unmodified until today for 6 years since when I opened the issue. Also I would guess that 2.x doesn't come anytime soon and the number of 1.x plugins is growing. Externally linked plugins don't seem to have made any progress. Maybe someone is interested to add this to her overlay, we could look over it and maybe improve/update it so there's no need to manually rebuild nginx when an ngx-* plugin ebuild is rebuilt.
(In reply to Alexander Turenko from comment #17) > The main problem that we have now with shipping 3rd-party modules for nginx > is about synchronizing nginx ebuild with one from the main Gentoo tree. If > we'll fork it, we'll again meet the same problem. > > Here I would expect a decision from Gentoo Developers. But with my proposal you do not need to change anything on the main nginx ebuild.
> But with my proposal you do not need to change anything on the main nginx ebuild. Mine does neither. It's all in the plugins' ebuild. The only downside is, that nginx needs to be rebuilt after the 3rd party module has been emerged (since only module sources are installed, registered via ENV vars and then picked up by the next nginx compile to have it built in - which has obvious advantages, which is why nginx dev's chose that method. Even if externally linked modules would work flawlessly, you'd still want compiled-in modules working without listing them all in the main ebuild.) The problem of rebuilding manually could be solved without major changes, except for livebuilds (didn't attempt it, since it works very well for me as is). My impression is, that ebuild maintainers seek for a perfect solution that doesn't exist and probably never will. Ironically, that results in the worst possible solution that we got now: A central, hardcoded plugin list controlled by USE flags which concentrates all the maintenance overhead in a single ebuild. Anyone please correct me if I'm wrong or missing something.
I have posted a RFC for a new www-nginx category and Jonas's nginx-module.eclass with a few changes for building the dynamic modules on gentoo-dev. If this gets accepted, we might have support for separately packaged dynamic modules. Of course, we'd then need maintainers to move the modules currently packaged in www-servers/nginx to their own packages.
