Today, during my "emerge --jobs=4 --keep-going ..." two packages failed due to errors in the postinst phase. However, emerge has completely ignored and hidden those errors: * The packages were counted as "complete", not as "failed". * Both packages were registered as "installed successfully", emerge continued without recalculating the dependencies. * emerge did not display any error messages or error logs, neither when the packages failed nor at the end. If I hadn't checked the error logs manually afterwards, I wouldn't have noticed at all that 2 packages failed (one was just a doc install failure, but the other one was a more serious problem with the binaries installed).
We should log the failed postinst phase via elog.
There's a patch in the following branch: https://github.com/zmedico/portage/tree/bug_547778
The patch is posted for review here: https://archives.gentoo.org/gentoo-portage-dev/message/cd5447f975d6c592f96f94f0a6babc7d
(In reply to Klaus Kusche from comment #0) > * emerge did not display any error messages or error logs, > neither when the packages failed nor at the end. If it called eerror or die, then you should have seen a corresponding eerror message. It's the responsibility of the ebuild to call eerror or die when appropriate, so you should file a separate bug for the ebuild if it doesn't do that.
One was a failing readme.gentoo_print_elog (no readme.gentoo) in postinst, which definitely *did* call die, the other was bug 547752 = 547782, where the whole postinst step ends with fail status if I remember correctly. In both cases, the error was correctly recorded as an error in the elog, but when emerging silently (e.g. "--jobs=4 --keep-going" or "--quiet-build --quiet-fail"), it was neither counted nor displayed nor noticeable in any other way during the emerge. If I remember correctly, the error was shown when emerging "normally" (not silently), but I'd have to try again to make sure.
(In reply to Zac Medico from comment #3) > The patch is posted for review here: > > https://archives.gentoo.org/gentoo-portage-dev/message/ > cd5447f975d6c592f96f94f0a6babc7d This is in the master branch now. (In reply to Klaus Kusche from comment #5) > In both cases, the error was correctly recorded as an error in the elog, > but when emerging silently (e.g. "--jobs=4 --keep-going" or > "--quiet-build --quiet-fail"), it was neither counted nor displayed > nor noticeable in any other way during the emerge. Check your PORTAGE_ELOG_SYSTEM setting. The default setting enables the echo module, which will display the eerror messages just before emerge exits. > If I remember correctly, the error was shown when emerging "normally" > (not silently), but I'd have to try again to make sure. You'll see it when it occurs, but you won't with --jobs or --quiet-build because all of that ebuild output is directed to the build log. Either way, you'll see it before emerge exits if you have PORTAGE_ELOG_SYSTEM="echo" enabled. You can use this command to check: portageq envvar PORTAGE_ELOG_SYSTEM
> (In reply to Klaus Kusche from comment #5) > > In both cases, the error was correctly recorded as an error in the elog, > > but when emerging silently (e.g. "--jobs=4 --keep-going" or > > "--quiet-build --quiet-fail"), it was neither counted nor displayed > > nor noticeable in any other way during the emerge. > > Check your PORTAGE_ELOG_SYSTEM setting. The default setting enables the echo > module, which will display the eerror messages just before emerge exits. I have PORTAGE_ELOG_CLASSES="info log warn error" PORTAGE_ELOG_SYSTEM="save" > > If I remember correctly, the error was shown when emerging "normally" > > (not silently), but I'd have to try again to make sure. > > You'll see it when it occurs, but you won't with --jobs or --quiet-build > because all of that ebuild output is directed to the build log. Either way, > you'll see it before emerge exits if you have PORTAGE_ELOG_SYSTEM="echo" > enabled. You can use this command to check: > > portageq envvar PORTAGE_ELOG_SYSTEM The problem is *not* that I did not see the error messages. The problem is that portage completely *ignored* the errors: * It registered the failed packages as "installed" in /var/db/pkg. * It counted the packages as "completed", not as "failed". The "failed" count was still displayed as zero at the end of the emerge. * It did *not* recalculate the dependencies, it just continued emerging (also packages which depend on the failed packages). I don't want emerge to display the full error messages, I just want to have a list of the names of the failed packages at the end (as it is done for packages failing in other phases).
(In reply to Klaus Kusche from comment #7) > The problem is that portage completely *ignored* the errors: > * It registered the failed packages as "installed" in /var/db/pkg. It registers them in /var/db/pkg *before* the postinst phase is executed, because at that point irreversible changes have been made to the installed system. Doing a rollback at that point could be accomplished in a couple of different ways: 1) Unmerging the files that were just installed, and mergw the previously installed version if there was one (a backup binary package would have to have been created by FEATURES=unmerge-backup) 2) Rollback to a previous checkpoint using filesystem snapshot capabilities (see bug 40127). > * It counted the packages as "completed", not as "failed". > The "failed" count was still displayed as zero at the end of the emerge. > * It did *not* recalculate the dependencies, it just continued emerging > (also packages which depend on the failed packages). Yes, in that sense, postinst failures currently are not treated as a "rel" failures. If postinst needs to do something that is so critical that we need to count it as a "real" failure, then it's probably doing something that it really should not be doing. > I don't want emerge to display the full error messages, > I just want to have a list of the names of the failed packages at the end > (as it is done for packages failing in other phases). For what package(s) is the postinst phase so critical?
(In reply to Zac Medico from comment #8) > (In reply to Klaus Kusche from comment #7) > > The problem is that portage completely *ignored* the errors: > > * It registered the failed packages as "installed" in /var/db/pkg. > > It registers them in /var/db/pkg *before* the postinst phase is executed, > because at that point irreversible changes have been made to the installed > system. Doing a rollback at that point could be accomplished in a couple of > different ways: > > 1) Unmerging the files that were just installed, and mergw the previously > installed version if there was one (a backup binary package would have to > have been created by FEATURES=unmerge-backup) > > 2) Rollback to a previous checkpoint using filesystem snapshot capabilities > (see bug 40127). Ok, makes sense to have them marked as "installed". > > * It counted the packages as "completed", not as "failed". > > The "failed" count was still displayed as zero at the end of the emerge. > > * It did *not* recalculate the dependencies, it just continued emerging > > (also packages which depend on the failed packages). > > Yes, in that sense, postinst failures currently are not treated as a "rel" > failures. If postinst needs to do something that is so critical that we need > to count it as a "real" failure, then it's probably doing something that it > really should not be doing. > > > I don't want emerge to display the full error messages, > > I just want to have a list of the names of the failed packages at the end > > (as it is done for packages failing in other phases). I *still* want to be informed in some way that something went wrong and that I have to check the logs and perhaps correct things. > For what package(s) is the postinst phase so critical? For the two errors I really had: One uncritical (just doc missing). One quite critical: fcaps failed, executables migth lack needed caps. Just checked the postinst's of some of the packages installed here: * Database and cache updates of all kinds (mime cache, icon cache, fonts, texmf update, XML catalog, hwdb, docbook...) * Certs rehash (!) * Essential chown, chgrp, fcaps (e.g. in polkit, eselect, openldap, cdrtools, sandbox, man, wireshark) * eselect updates (e.g. mesa, python, xorg, ...) * Creating users and groups (systemd) * Creating / moving dir's and files (openrc, portage, gcc-config), creating links (gawk, bash, clang) * Generating unique ids (dbus, dhcpcd) * Runlevel manipulation (kmod, udev, ...) * Restarting daemons So, postinst is not just writing nice hints to the log. Postinst is doing essential things which need my attention when they fail.
... and which should perhaps be fixed *before* depending packages are emerged.
(In reply to Klaus Kusche from comment #9) > I *still* want to be informed in some way that something went wrong > and that I have to check the logs and perhaps correct things. I suppose we could make it show a message in the foreground which is similar to a regular failure message, which includes the path of the build log. For fcaps, I've filed bug 548516 which will allow fcaps to be called before the package is installed, during the (sandboxed) src_install phase.
There's an additional patch in the following branch: https://github.com/zmedico/portage/tree/bug_547778 I've posted it for review here: https://archives.gentoo.org/gentoo-portage-dev/message/936447e45de05184d55668e66d585ae1 This makes postinst failures behave more like "real" failures in terms of output and the final emerge return code. However, in other respects, these packages will be treated as successful installations. This means that they will not cause emerge to immediately exit, and they will not trigger recalculation of dependencies when --keep-going is enabled.
This is in the master branch: https://gitweb.gentoo.org/proj/portage.git/commit/?id=601fc103087bcecf46d34d26aad5a939a1cb087c
Released in portage-2.2.19
