It is useful to have default definitions for an ebuild, that allows autotools / Makefile projects to "just work" in the common cases out of the box. It is useful to have src_prepare automatically handle PATCHES, eapply_user, and similar. On the other hand, running phases isn't a no-op in terms of time, even if it's a no-op in terms of ${WORKDIR} and ${D} I/O. This is part of why categories such as virtual, acct-user, acct-group, and app-alternatives are so slow -- even though virtual packages need to do nothing other than updating the vdb, and even though acct-* packages only need to invoke src_install and pkg_*, and even though app-alternatives can be done purely via pkg_*. EAPI 9 could define something like UNUSED_PHASES="src_unpack src_prepare src_configure src_compile src_test src_install", which eclasses used by categories which shall never do any of this work can define. Individual ebuilds could also define it if e.g. they know that there's a Makefile, but no configure stage, but the savings there may not be very compelling. Alternative: NEEDED_PHASES which defaults to all phases with a default, but can be set by eclasses / ebuilds to "" to cancel out the default ones or specify exactly which ones they use. The time spent to install virtual packages is *very* noticeable at the moment, because there's quite literally nothing to do and the fact that it isn't instantaneous is inherently suboptimal. This could help lower that gap.
I'm not sure this is worth adding as the slowness is largely due to design choices so in effect it would be adding required complexity for other implementations that don't exhibit such exaggerated symptoms. For example, if I recall correctly pkgcore (which also uses a native bash backend) can merge virtual/* pkgs around an order of magnitude faster than portage while pkgcraft currently is around four orders of magnitude faster for similar functionality. Therefore (while I realize it would be a lot of work) I'd suggest portage devs to look more at merging pkgcore's ebuild daemon (ebd) support before trying to work around its performance issues with PMS changes. Integrating ebd support would also significantly boost performance in a number of other places such as cache regen if done properly.
... and as a followup, things such as T are defined for all phases, but if no phases are run and no files are installed, it doesn't need to be created. This could then allow virtual packages to run purely in python by updating the vdb (or writing out a binpackage that consists purely of metadata). Using a faster mechanism to run phases does sound interesting though. Still I think it would be neat if phases don't have to be run at all when you don't use them.
Just want to point out ${PORTAGE_BIN_PATH}/{pre,post}inst-qa-check.d/50xdg-utils here as a potential easy target to fix. The amount of files it touches on every single ebuild merge scales linearly with how much desktop cruft someone has installed, so regular users receive death by a thousand papercuts over time while it doesn't show up on headless/CI servers at all. Here's a `perf stat --repeat=10 -- emerge --nodeps -1 acct-user/man`: 7.0484 +- 0.0155 seconds time elapsed ( +- 0.22% ) And after I do a `touch /usr/local/lib/{pre,post}inst-qa-check.d/50xdg-utils` to disable it: 6.8230 +- 0.0138 seconds time elapsed ( +- 0.20% ) That's 220ms, or 3-4% of an ebuild that does nothing, under optimal conditions (fast workstation, everything fits in memory). The difference gets much more pronounced when I'm updating my slower machines over nfs+chroot.
(In reply to Enne Eziarc from comment #3) Thanks. Please feel free to file any other bugs for such issues too. I know Kerin is interested and sort of poking at these while he does an audit of various parts of Portage, but I am myself too.
Let's call the xdg thing bug 162450 (or a need for triggers) unless there's a specific issue with that check being slower than it needs to be.
I"m certain I mentioned the xdg-utils thing to you on IRC, at least. :) In particular, when I looked at it the xdg check was a major contributing factor to virtual package speeds specifically. They probably shouldn't run at all for packages that don't have a meaningful image (or any at all), like virtuals! But also the check is kinda dumb and should be rm -rf'ed. Its fully-from-scratch replacement should operate on the image directory instead.
We could easily tag ebuilds with a PROPERTIES value to modify this behavior, like PROPERTIES=virtual proposed back in 2008 here: https://marc.info/?l=gentoo-dev&m=121969287407879&w=2
One thing about a package selecting which phases it needs to support is that packages which do use some phases, but aren't virtuals, could avoid running phases they don't need, while still running phases they do need. As I pointed out, there are several non-virtual categories which nonetheless don't need to run all phases. e.g. acct-user acct-group both define src_install (but not any other src phases) app-alternatives typically defines a package-specific src_install, and shouldn't need to run pkg_* phases or most src_* ones. So they cannot do pure vdb transactions but could avoid quite a bit of the current weight.
I would also like to be able to skip src_test even when running with global FEATURES=test. Many packages don't have a testsuite. Currently, what usually happens is one runs the tests anyway -- portage reports the tests succeeded, because it attempted to run emake check -n && emake test -n and discovered nothing to do. This isn't even a no-op function, you have to fork out to a phase helper and then run GNU Make just to have it print make: *** No rule to make target 'check'. Stop. to /dev/null You can also RESTRICT="test" but this then makes it look like things are broken... it would be nice to have a clean way to describe this which is introspectable, and it would be nice to not have so much stuff be logic which is specific to FEATURES="test" handling.
