When installing acct-{group,user} from binpkgs the RDEPEND means that the packages can be installed in parallel so preinst checks can fail. For EAPI 8 it should be possible to use IDEPEND, but sam mentions that there are some additional issues with --root that could require the use of BDEPEND. Reproducible: Always
> When installing acct-{group,user} from binpkgs the RDEPEND means that the packages can be installed in parallel so preinst checks can fail. Could you expand on this? I don't really understand what you mean. Maybe provide an example or two.
So it seems like this is a difficult race condition to trigger. I have an example below of what _should_ trigger the issue, but have not gotten it to cooperate. I think the way I actually triggered it involved building many more packages at the same time (e.g. with emerge --color=y --with-bdeps=y --onlydeps --onlydeps-with-rdeps=n -j4 -q --keep-going -uDN ${targets} where targets will pull in many many packages including dozens of acct-{user,group} packages). emerge --buildpkgonly -j8 acct-{user,group}/{messagebus,nginx,apache,redis,rabbitmq} emerge --onlydeps acct-user/messagebus # this shows the error message that can occur emerge --getbinpkgonly -1q -j12 acct-{user,group}/{messagebus,nginx,apache,redis,rabbitmq} # there is a race condition here # clean up and try again until the race happens (very difficult it seems?) emerge -C acct-{user,group}/{messagebus,nginx,apache,rabbitmq} for u in {messagebus,nginx,apache,redis,rabbitmq}; do userdel ${u}; done
Back up a minute: you haven't actually explained the problem or provided an example of an error message that would appear.
Anyway, neither BDEPEND or IDEPEND are appropriate here. Those specify dependencies relevant BROOT, but we are creating the users/groups in ROOT.
I ran into bug 903083 while attempting to produce a failure of some sort using the instructions you provided so far.
Created attachment 858999 [details] build.log for an acct-user failure My previous comment had a very confusing typo (switching nodeps for onlydeps). The build.log attached here was run with the correct command: emerge --nodeps acct-user/messagebus This simulates the issue, which is that it is possible for portage to attempt to install the acct-user and acct-group packages at the same time, and if preinst for acct-user runs before acct-group finishes, then the error seen in the build.log will appear. I have only see the race condition happen once and I was not able to grab the environment that it happened in. I know that setting FEATURES=-ebuild-locks will absolutely cause this to happen, however when I encountered the issue I was running with FEATURES=ebuild-locks explicitly because this is a known issue for acct- ebuilds. It might be possible to demonstrate the issue by putting a sleep command in an acct-group ebuild so that it will stall, allowing the acct-user ebuild to reach presintall and fail to find the corresponding group.
Running "emerge --nodeps acct-user/messagebus" tells Portage to ignore RDEPEND, so acct-group/messagebus wont get installed at all. That doesn't provide a useful simulation. As far as I am aware, RDEPEND should be satisfied before installation occurs and pkg_preinst and pkg_postinst are invoked. If Portage does something different, that's probably a bug in Portage.
I'm unable to get Portage to install acct-group/messagebus and acct-user/messagebus at the same time using a simple test script (see below). It always installs them one after the other. > % sudo ./test.sh > + rm -rf /testroot > + mkdir -p /testroot/etc > + touch /testroot/etc/group /testroot/etc/gshadow /testroot/etc/passwd /testroot/etc/shadow > + chmod 0600 /testroot/etc/gshadow /testroot/etc/shadow > + export 'FEATURES=-ebuild-locks parallel-install' > + FEATURES='-ebuild-locks parallel-install' > + emerge --root /testroot -v1k -j2 acct-group/messagebus acct-user/messagebus > > These are the packages that would be merged, in order: > > Calculating dependencies... done! > Dependency resolution took 1.76 s. > > [binary N ] acct-group/messagebus-0-r1-1::gentoo to /testroot/ 0 KiB > [binary N ] acct-user/messagebus-0-r1-1::gentoo to /testroot/ 0 KiB > > Total: 2 packages (2 new, 2 binaries), Size of downloads: 0 KiB > > >>> Running pre-merge checks for acct-group/messagebus-0-r1 > >>> Running pre-merge checks for acct-user/messagebus-0-r1 > >>> Emerging binary (1 of 2) acct-group/messagebus-0-r1::gentoo for /testroot/ > >>> Installing (1 of 2) acct-group/messagebus-0-r1::gentoo to /testroot/ > >>> Completed (1 of 2) acct-group/messagebus-0-r1::gentoo to /testroot/ > >>> Emerging binary (2 of 2) acct-user/messagebus-0-r1::gentoo for /testroot/ > >>> Installing (2 of 2) acct-user/messagebus-0-r1::gentoo to /testroot/ > >>> Completed (2 of 2) acct-user/messagebus-0-r1::gentoo to /testroot/ > >>> Jobs: 2 of 2 complete Load avg: 0.46, 0.35, 0.45 > > * Messages for package acct-group/messagebus-0-r1 merged to /testroot/: > * Log file: /var/log/portage/acct-group:messagebus-0-r1:20230326-050504.log > > * Adding group messagebus > > * Messages for package acct-user/messagebus-0-r1 merged to /testroot/: > * Log file: /var/log/portage/acct-user:messagebus-0-r1:20230326-050509.log > > * Adding user messagebus > > * GNU info directory index is up-to-date. If I throw some other packages in there, the order gets mixed up a bit, but the groups always finish before their respective users. > % sudo ./test.sh > + rm -rf /testroot > + mkdir -p /testroot/etc > + touch /testroot/etc/group /testroot/etc/gshadow /testroot/etc/passwd /testroot/etc/shadow > + chmod 0600 /testroot/etc/gshadow /testroot/etc/shadow > + export 'FEATURES=-ebuild-locks parallel-install' > + FEATURES='-ebuild-locks parallel-install' > + emerge --root /testroot -v1k -j4 acct-group/apache acct-group/messagebus acct-user/apache acct-user/messagebus > > These are the packages that would be merged, in order: > > Calculating dependencies... done! > Dependency resolution took 1.78 s. > > [binary N ] acct-group/apache-0-r2-1::gentoo to /testroot/ 0 KiB > [binary N ] acct-group/messagebus-0-r1-1::gentoo to /testroot/ 0 KiB > [binary N ] acct-user/apache-0-r2-3::gentoo to /testroot/ 0 KiB > [binary N ] acct-user/messagebus-0-r1-1::gentoo to /testroot/ 0 KiB > > Total: 4 packages (4 new, 4 binaries), Size of downloads: 0 KiB > > >>> Running pre-merge checks for acct-group/apache-0-r2 > >>> Running pre-merge checks for acct-group/messagebus-0-r1 > >>> Running pre-merge checks for acct-user/apache-0-r2 > >>> Running pre-merge checks for acct-user/messagebus-0-r1 > >>> Emerging binary (1 of 4) acct-group/apache-0-r2::gentoo for /testroot/ > >>> Emerging binary (2 of 4) acct-group/messagebus-0-r1::gentoo for /testroot/ > >>> Installing (1 of 4) acct-group/apache-0-r2::gentoo to /testroot/ > >>> Completed (1 of 4) acct-group/apache-0-r2::gentoo to /testroot/ > >>> Emerging binary (3 of 4) acct-user/apache-0-r2::gentoo for /testroot/ > >>> Installing (2 of 4) acct-group/messagebus-0-r1::gentoo to /testroot/ > >>> Completed (2 of 4) acct-group/messagebus-0-r1::gentoo to /testroot/ > >>> Emerging binary (4 of 4) acct-user/messagebus-0-r1::gentoo for /testroot/ > >>> Installing (3 of 4) acct-user/apache-0-r2::gentoo to /testroot/ > >>> Completed (3 of 4) acct-user/apache-0-r2::gentoo to /testroot/ > >>> Installing (4 of 4) acct-user/messagebus-0-r1::gentoo to /testroot/ > >>> Completed (4 of 4) acct-user/messagebus-0-r1::gentoo to /testroot/ > >>> Jobs: 4 of 4 complete Load avg: 0.84, 0.43, 0.42 > > * Messages for package acct-group/apache-0-r2 merged to /testroot/: > * Log file: /var/log/portage/acct-group:apache-0-r2:20230326-051302.log > > * Adding group apache > > * Messages for package acct-group/messagebus-0-r1 merged to /testroot/: > * Log file: /var/log/portage/acct-group:messagebus-0-r1:20230326-051302.log > > * Adding group messagebus > > * Messages for package acct-user/apache-0-r2 merged to /testroot/: > * Log file: /var/log/portage/acct-user:apache-0-r2:20230326-051309.log > > * Adding user apache > > * Messages for package acct-user/messagebus-0-r1 merged to /testroot/: > * Log file: /var/log/portage/acct-user:messagebus-0-r1:20230326-051309.log > > * Adding user messagebus > > * GNU info directory index is up-to-date.
Darn. What if FEATURES=parallel-install is set?
(In reply to Tom Gillespie from comment #9) > Darn. What if FEATURES=parallel-install is set? I had FEATURES="-ebuild-locks parallel-install" set in my test script.
Since neither of us can reproduce a failure, I'm closing this as NEEDINFO.
Ok. I have a repro. It works running directly on the official docker images, it is not specific to the musl image as I tested against gentoo/stage3:hardened and gentoo/stage3:latest as well. Maybe the issues is cause by running inside docker? docker pull gentoo/portage:latest docker pull gentoo/stage3:amd64-musl-hardened docker create -v /var/db/repos/gentoo --name local-portage-snap gentoo/portage:latest /bin/true docker run \ --volumes-from local-portage-snap \ gentoo/stage3:amd64-musl-hardened \ sh -c 'emerge --buildpkgonly acct-group/mail acct-group/messagebus acct-user/messagebus; emerge --color=y --with-bdeps=y --nodeps -j4 -q --keep-going -u =acct-group/mail-0-r1 =acct-group/messagebus-0-r1 =acct-user/messagebus-0-r1 || { emerge --info; cat /var/tmp/portage/acct-user/messagebus-0-r1/temp/environment;}' Interestingly when running on gentoo/stage3:latest image I get a bunch of messages that don't show up on the hardened images: Unable to unshare: EPERM (for FEATURES="ipc-sandbox network-sandbox pid-sandbox") Not sure if it is related.
Created attachment 859456 [details] emerge --info in the failing docker environment
(In reply to Tom Gillespie from comment #12) You just "reproduced" the issue with emerge --nodeps. As I said before, this tells Portage to ignore all dependencies (including RDEPEND), which makes the test invalid.
Even if this is not valid, I think the behavior might be undesired. I would expect for dependencies to be calculated within the set of packages being directly emerged. If this is not done then the user has to manually determine the merge order and cannot rely on portage. This means that it is currently impossible to safely emerge multiple packages with --nodeps if there are any dependencies within the set to be emerged. This might be a feature request instead for --nodeps to enforce dependency ordering within the set of atoms listed for emerge? e.g. emerge --nodeps acct-user/messagebus acct-group/messagebus should enforce ordering because there is a RDEPEND between the two, whereas emerge --nodeps acct-user/messagebus should still produce the error because acct-group/messagebus is not and would not be installed.
(In reply to Tom Gillespie from comment #15) emerge --nodeps currently disables all dependency calculations. That is probably intentional since calculating dependencies can be an expensive operation. What you are asking for seems like a new feature. If you would like to pursue it, please file a new bug with Product = "Portage Development".
