I hate hiding things behind the qa feature, but the scripts in https://github.com/gentoo/portage/tree/master/bin/postinst-qa-check.d are SUPER PAINFULLY slow on my system where the entire rootfs is in tmpfs, which I find unacceptable. At any given moment, it appears that this is all my system is running. Please hide these expensive checks behind FEATURES=qa or some new flag at your discretion. I run FEATURES=qa on all dev boxes, but on my build box I won't see the logs unless it's a hard failure so this kind of thing is pointless.
If it's doing something prohibitively expensive (as in bug 631454), then we really need to fix it, not just hide it behind a flag.
I guess it's probably calling eqatag for an obscene number of files, as in bug 631454. We need to do something about that, it's just not acceptable.
The fact that your root fs is on tmpfs explains the issue, since copying the files there would bump the ctime, and we're using -newercm (-cnewer) in postinst-qa-check.d.
I think what we need to do is test for a positive result *before* the package is merged, and then simply disable the check in that case.
A reasonable approach would be to have preinst-qa-check.d scripts that fix up the state, so that any problems found by the postinst-qa-check.d scripts can be blamed on the current package with a high level of confidence.
Patch sent for review: https://archives.gentoo.org/gentoo-portage-dev/message/ce99a55a610bff7fcfd92cb6fca19699 https://github.com/gentoo/portage/pull/224
forgive my naivity on this, but the current checks seem to check the entire filesystem instead of just the files you are installing, and the suggested patch doesn't appear to change that. Wouldn't it be possible to check just the files you are installing? As an example, I've been installing virtual/jpeg and stuck on this check for nearly 3 hours right now.....
thanks to dwfreed for noticing, please return BEFORE the find in the case of known impossibility, such as parallel-install. Although if this check ran on just the files being installed (before merge) that wouldn't be a limitation.
(In reply to Rick Farina (Zero_Chaos) from comment #7) > forgive my naivity on this, but the current checks seem to check the entire > filesystem instead of just the files you are installing, and the suggested > patch doesn't appear to change that. The cost of the 'find' calls is relatively small, so that should not be a problem. > Wouldn't it be possible to check just the files you are installing? As an > example, I've been installing virtual/jpeg and stuck on this check for > nearly 3 hours right now..... The really costly part is the eqatag call with an extremely large number of false-positive files, and the proposed patch will fix that. (In reply to Rick Farina (Zero_Chaos) from comment #8) > thanks to dwfreed for noticing, please return BEFORE the find in the case of > known impossibility, such as parallel-install. Although if this check ran > on just the files being installed (before merge) that wouldn't be a > limitation. It's useful to keep the find calls in this case, since it will detect when update-desktop-database, update-mime-database, or gtk-update-icon-cache needs to be called.
I will yield to your wisdom on this matter, as usual. Is this patch ready for testing or should I hold off for some reason?
(In reply to Rick Farina (Zero_Chaos) from comment #10) > I will yield to your wisdom on this matter, as usual. Is this patch ready > for testing or should I hold off for some reason? It's ready, please test.
to further clarify, I am running FEATURES=parallel-install and this is still an issue for me (pre-patch). I'm adding this note because I honestly don't fully understand what is going on here and so I'm trying to add detail which might be useful.
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/proj/portage.git/commit/?id=2f01a9fdc68740642cca25c0fcc7f1d1fc14c0ee commit 2f01a9fdc68740642cca25c0fcc7f1d1fc14c0ee Author: Zac Medico <zmedico@gentoo.org> AuthorDate: 2017-10-26 06:06:51 +0000 Commit: Zac Medico <zmedico@gentoo.org> CommitDate: 2017-10-27 18:52:08 +0000 postinst_qa_check: initialize preinst state (bug 635474) In order to prevent false-positives during postinst_qa_check, use a preinst_qa_check function to initialize a baseline state for the postinst checks. Bug: https://bugs.gentoo.org/635474 bin/misc-functions.sh | 18 +++++++++++------- bin/postinst-qa-check.d/50gnome2-utils | 3 +++ bin/postinst-qa-check.d/50xdg-utils | 6 ++++++ bin/preinst-qa-check.d/50gnome2-utils | 1 + bin/preinst-qa-check.d/50xdg-utils | 1 + pym/portage/package/ebuild/doebuild.py | 1 + 6 files changed, 23 insertions(+), 7 deletions(-)}
This all seems good in my testing.
(In reply to Rick Farina (Zero_Chaos) from comment #14) > This all seems good in my testing. Great, thanks!
