Gentoo is great in supporting SLOTS/multiple versions of the same software. However, the systemd directive RuntimeDirectory defeats this feature for postgresql socket access. Currently, systemd postgresl*.service units include: RuntimeDirectory=postgresql RuntimeDirectoryMode=1775 This doesn't support simultaneous running postgresql* services (of the same or different SLOT/versions on different PGPORT sockets) because the systemd directive RuntimeDirectory doesn't support service sharing since it will remove/recreate the specified directory at any (postgresql*) service stop/restart resulting in removal of (still needed) socket & lock files of any other running postgresql* service. Instead /usr/lib/tmpfiles.d/postgresql-${SLOT}.conf should be used. This can easily be fixed by 1. removing RuntimeDirectory* directives from dev-db/postgresql/files/postgresql.service* 2. adding dev-db/postgresql/files/postgresql.tmpfiles containing: d /run/postgresql 1775 postgres postgres - 3. including systemd_newtmpfilesd in all postgresql SLOT/ebuilds: systemd_newtmpfilesd "${FILESDIR}"/${PN}.tmpfiles ${PN}-${SLOT}.conf Additionally (for consistency), the following strings in ebuilds (they all affect /run/postgresql) should be replaced: - 0775 should be 1755 (fperms of /run/postgresql) - root:postgres should be postgres:postgres (dev-db/postgresql/files/postgresql.*) Note: like any other postgresql package SLOT/versions), I guess postgresql-9999.ebuild should be migrated to EAPI="6" too?
(In reply to Evert from comment #0) > Gentoo is great in supporting SLOTS/multiple versions of the same software. You're welcome! Thank you for the compliment. > However, the systemd directive RuntimeDirectory defeats this feature for > postgresql socket access. > > Currently, systemd postgresl*.service units include: > RuntimeDirectory=postgresql > RuntimeDirectoryMode=1775 > > This doesn't support simultaneous running postgresql* services (of the same > or different SLOT/versions on different PGPORT sockets) because the systemd > directive RuntimeDirectory doesn't support service sharing since it will > remove/recreate the specified directory at any (postgresql*) service > stop/restart resulting in removal of (still needed) socket & lock files of > any other running postgresql* service. > Instead /usr/lib/tmpfiles.d/postgresql-${SLOT}.conf should be used. Okay, yeah, that's not good. Who's bright idea was it to make systemd work that way? > Additionally (for consistency), the following strings in ebuilds (they all > affect /run/postgresql) should be replaced: > - 0775 should be 1755 (fperms of /run/postgresql) Prefix is a special case. If there aren't any adverse effects, this can be done. > - root:postgres should be postgres:postgres It should definitely be root:postgres. This prevents any other program or user in the postgres group from deleting a socket they don't own. > Note: like any other postgresql package SLOT/versions), I guess > postgresql-9999.ebuild should be migrated to EAPI="6" too? There were some additional challenges to bumping the live ebuild to EAPI 6. It'll be revisited eventually, but there really isn't any need to do so at this time. > This can easily be fixed by > 1. removing RuntimeDirectory* directives from > dev-db/postgresql/files/postgresql.service* > 2. adding dev-db/postgresql/files/postgresql.tmpfiles containing: > d /run/postgresql 1775 postgres postgres - > 3. including systemd_newtmpfilesd in all postgresql SLOT/ebuilds: > systemd_newtmpfilesd "${FILESDIR}"/${PN}.tmpfiles ${PN}-${SLOT}.conf If you provide a set of files suitable for use in systemd and get someone else who uses systemd to verify your changes as good, I'll replace the set we have now. I neither have the time nor interest to learn how to use a systemd. Plus, I use Emacs, so I'm already running two OSes simultaneously and don't want a third.
OpenRC now has tmpfiles.d support, to be able to share the temporary (PID file, socket file,...) directory configuration with systemd. So rather than having, say, a start_pre() function in the init script create /run/foo, you could add a tmpfiles.d entry that creates /run/foo and it would work for both OpenRC and systemd. So, we could at least test *that* part of it while still running OpenRC.
The postgresql init scripts do not use "start_pre". The socket dir is checked/created in the "checkconfig" function. Besides that, postgresql.init-9.3-r1 which is used for pg-9.3+ supports the use of multiple PG_SOCKET_DIRECTORIES for some reason. On top of that, the socket directory is user/admin configurable in postgresql.confd-9.3. So, if you want to use tmpfiles.d for OpenRC too, you need to get rid of the configurable PG_SOCKET_DIRECTORY / PG_SOCKET_DIRECTORIES too.
(In reply to Michael Orlitzky from comment #2) > OpenRC now has tmpfiles.d support, to be able to share the temporary (PID > file, socket file,...) directory configuration with systemd. So rather than > having, say, a start_pre() function in the init script create /run/foo, you > could add a tmpfiles.d entry that creates /run/foo and it would work for > both OpenRC and systemd. > > So, we could at least test *that* part of it while still running OpenRC. No, I won't hobble the initscripts for the reasons given in response to Evert. (In reply to Evert from comment #3) > The postgresql init scripts do not use "start_pre". The socket dir is > checked/created in the "checkconfig" function. I'll probably change checkconfig to start_pre at some point. Functionally, there's no benefit, so I won't make this change until there's some other worthwhile change to make. > Besides that, postgresql.init-9.3-r1 which is used for pg-9.3+ supports the > use of multiple PG_SOCKET_DIRECTORIES for some reason. On top of that, the > socket directory is user/admin configurable in postgresql.confd-9.3. That "some reason" is because PostgreSQL 9.3+ supports multiple, simultaneous socket directory locations. This allows an administrator to expose the Unix socket in chroots and various filesystem namespaces. > So, if you want to use tmpfiles.d for OpenRC too, you need to get rid of the > configurable PG_SOCKET_DIRECTORY / PG_SOCKET_DIRECTORIES too. So, yeah, not happening. The OpenRC init scripts are going to continue exploiting the full capabilities of PostgreSQL and its utilities. Unfortunately, systemd is incapable of doing the same, from what little I understand about it.
Created attachment 497444 [details] unix socket support for simultaneous running postgresql-* services when using systemd As per your request here the new postgresql ebuild files. - OpenRC files left unchanged. - emerge postgresql:9.{2..6} # tested & OK, also creates /run/postgresql without the need for reboot. - systemctl start/restart postgresql-9.{5,6} # tested & OK - socket/lock files of simultaneous running services: tested & OK # cat dev-db/postgresql/files/postgresql.tmpfiles d /run/postgresql 1775 root postgres - # diff dev-db/postgresql/files/postgresql.service dev-db/postgresql/files/postgresql.service-9.2 50,53d49 < # Make sure the required runtimedir is present < RuntimeDirectory=postgresql < RuntimeDirectoryMode=1775 < # diff dev-db/postgresql/files/postgresql.service-9.6 dev-db/postgresql/files/postgresql.service-9.6-r1 51,54d50 < # Make sure the required runtimedir is present < RuntimeDirectory=postgresql < RuntimeDirectoryMode=1775 < # diff dev-db/postgresql/postgresql-9.2.23.ebuild dev-db/postgresql/postgresql-9.2.23-r1.ebuild 224c224 < "${FILESDIR}/${PN}.service" | \ --- > "${FILESDIR}/${PN}.service-9.2" | \ 225a226 > systemd_newtmpfilesd "${FILESDIR}"/${PN}.tmpfiles ${PN}-${SLOT}.conf 233c234 < fperms 0775 /run/postgresql --- > fperms 1775 /run/postgresql 272a274 > systemd_tmpfiles_create ${PN}-${SLOT}.conf diff for 9.3, 9.4 and 9.5 are the same (different line numbers only). # diff dev-db/postgresql/postgresql-9.6.5.ebuild dev-db/postgresql/postgresql-9.6.5-r1.ebuild 268c268 < "${FILESDIR}/${PN}.service-9.6" | \ --- > "${FILESDIR}/${PN}.service-9.6-r1" | \ 270a271 > systemd_newtmpfilesd "${FILESDIR}"/${PN}.tmpfiles ${PN}-${SLOT}.conf 278c279 < fperms 0775 /run/postgresql --- > fperms 1775 /run/postgresql 317a319 > systemd_tmpfiles_create ${PN}-${SLOT}.conf diff for 10_rc1 and 9999 are the same (different line numbers only).
So basically the changes I proposed: - removed RuntimeDirectory* directives from the (new) systemd units - added files/postgresql.tmpfiles - honoured root:postgres for /run/postgresql - added systemd_newtmpfilesd to the new ebuilds - assumed fperms 1755 is a non-issue for "use prefix" - also needed to add systemd_tmpfiles_create in pkg_postinst which makes sure /run/postgresql exists at the end of emerge and eliminates the need for reboot (in case /run/postgresql didn't exist yet) - tested emerge of all these new ebuilds - OK - tested systemctl start/stop/restart & creation/removal/keeping of socket/lock files of simultaneous running (9.5 and 9.6) services - OK Conclusion: This should all work fine. Question: You mentioned you need someone else to verify my changes. Do you have someone in mind?
I hate to add one more thing to think about to the discussion, but: OpenRC will honor systemd's tmpfiles entries by default these days. So, for example, if I set PG_SOCKET_DIRECTORIES="/run/postgresql-openrc" in /etc/conf.d/postgresql-9.5, and create $ cat /usr/lib64/tmpfiles.d/postgres.conf d /run/postgresql 1775 root postgres - and then start the postgresql-9.5 service (with OpenRC), I wind up with *both* /run/postgresql (from the tmpfiles entry) /run/postgresql-openrc (from the OpenRC service script) A priori that's not a problem -- just a tiny bit more motivation to standardize on the directory. Otherwise we might accidentally update the permissions/ownership on the tmpfiles entry in the future and accidentally affect OpenRC or vice-versa. I don't see a way to pass a user-configurable list of directories (i.e. PG_SOCKET_DIRECTORIES) to the tmpfiles.d processor. We could document how a user might do that himself, but I don't know enough about why you might want multiple socket directories to discuss this intelligently.
(In reply to Michael Orlitzky from comment #7) > I hate to add one more thing to think about to the discussion, but: OpenRC > will honor systemd's tmpfiles entries by default these days. So, for > example, if I set > > PG_SOCKET_DIRECTORIES="/run/postgresql-openrc" > > in /etc/conf.d/postgresql-9.5, and create > > $ cat /usr/lib64/tmpfiles.d/postgres.conf > d /run/postgresql 1775 root postgres - > > and then start the postgresql-9.5 service (with OpenRC), I wind up with > *both* > > /run/postgresql (from the tmpfiles entry) > /run/postgresql-openrc (from the OpenRC service script) > > A priori that's not a problem -- just a tiny bit more motivation to > standardize on the directory. Otherwise we might accidentally update the > permissions/ownership on the tmpfiles entry in the future and accidentally > affect OpenRC or vice-versa. Hmm, then maybe we want to conditionally install the tmpfile. systemd is already a USE flag in the ebuild, so it should be as simple as `use systemd && ...`. We can document the tmpfile, but it'll still be surprising to most when this random directory keeps getting created despite what's set in /etc/conf.d/postgresql-${SLOT}. If anyone even notices it. This could also be considered a security risk.
OK clear, the tmpfiles file is currently not desired on OpenRC systems so I moved systemd_newtmpfilesd into the "if use systemd" block. Notes: - for 9.2..9.5 I needed to "backport" the "if use systemd" block (including IUSE) which is fine and even more consistent over the current ebuilds. - systemd_newtmpfilesd (systemd.eclass) is similar to newtmpfiles (tmpfiles.eclass) so other than it's in a different eclass I'm not sure why this function exists twice. - systemd_tmpfiles_create doesn't need "use systemd && ..." since it checks for the systemd-tmpfiles executable which only exists on systemd systems anyway. - systemd_tmpfiles_create (systemd.eclass) is similar to tmpfiles_process (tmpfiles.eclass) but the latter supports OpenRC too. - systemd_tmpfiles_create is NOT used in any other ebuild, so it might be better to use "use systemd && tmpfiles_process ..." instead. Since tmpfiles.eclass supports both systemd & OpenRC, I guess this eclass is created later and may deprecate the corresponding functions in systemd.eclass. If that's the case, it might be better to inherit tmpfiles in the ebuild too and replace - systemd_newtmpfilesd for newtmpfiles - systemd_tmpfiles_create for "use systemd && tmpfiles_process" Please Let me know which option you prefer so I can attach the new preferred ebuild files.
(In reply to Evert from comment #9) > Please Let me know which option you prefer so I can attach the new preferred > ebuild files. I don't know enough to have a preference. Maybe the folks at #gentoo-systemd would be able to give some better advice on which eclass to use. My gut tells me the systemd eclass [1] is the one we want to continue using as the tmpfiles eclass [2] only has a few functions. [1]: https://devmanual.gentoo.org/eclass-reference/systemd.eclass/index.html [2]: https://devmanual.gentoo.org/eclass-reference/tmpfiles.eclass/index.html P.S.: You asked previously if I had someone in mind to verify your changes, and I don't. I'll take the verification of anyone, though. It needn't be a Gentoo dev.
Created attachment 497984 [details] unix socket support for simultaneous running postgresql-* services when using systemd Here are the new additional ebuild files. As per your advice I use the systemd eclass functions. If the tmpfiles eclass might be preferred in the future, it will simply be a matter of changing 2 function names. I saw postgresql-10.0-r1.ebuild has been replaced with postgresql-10.0.ebuild so I made that change too. emerge dev-db/postgresql:9.{2..6} # tested & OK I even tested with USE=-server and/or USE=-systemd, all behave as expected. Since the changes are not shocking and I'm carefull & confident, I suggest we don't need another user to test. Well, you know how to use vimdiff and/or meld too ;-)
Of course I meant: I saw postgresql-10_rc1-r1.ebuild has been replaced ...
commit 9d489165499bc17cf58f7872b13327b160355af3 (HEAD -> master, origin/master, origin/HEAD) Author: Aaron W. Swenson <titanofold@gentoo.org> Date: Sat Oct 7 16:19:36 2017 -0400 dev-db/postgresql: Improved systemd support Now multiple slots can run simultaneously. Thank you Evert. Bump live ebuild EAPI to 6 and switched to git-r3 eclass. Gentoo-Bug: https://bugs.gentoo.org/632496 Package-Manager: Portage-2.3.8, Repoman-2.3.3
