I've been doing some upgrades on some systems that had been overdue for awhile, and I kept having random files removed improperly while unmerging packages. I'll detail a couple of cases here (the first I went into in great detail in the forum post I linked to above). One (debianutils/mktemp) - /bin/mktemp used to be a part of sys-apps/debianutils, but got moved out to its own package at sys-apps/mktemp. When you emerge either of these from a system using the old debianutils, Portage will install mktemp first and then debianutils, so when debianutils unmerges the old version, it ends up considering /bin/mktemp an "orphaned" binary and removes it. Then, in order to fix the system, I had to manually reinstall mktemp. Two (cracklib): The dictionary files at /usr/lib/cracklib_dict.* used to be owned by the cracklib package (ie: inside /var/db/pkg/sys-libs/cracklib-$VER/CONTENTS). Newer versions just generate them in the postinstall step, and they're not considered part of the official package. Because of this, they end up being considered "orphaned" and removed. After my cracklib upgrade, any call to "passwd" would result in the message "passwd: Critical error - immediate abort" until I regenerated the cracklib dictionaries. I'm sure that there's other random files strewn about which have gotten removed which I haven't noticed yet (because nothing's broken to let me know that they're missing). When I add "-unmerge-orphans" to my FEATURES list, these files remain in place and my system remains happy. Given all that, it seems to me that having unmerge-orphans be in the default list may not be a great idea. I wasn't even aware that that option existed until I started tracking down these options. The default make.conf.example doesn't mention it either, or its possible negative side-effects. Obviously that's up to you, but IMO it's more damaging than helpful. Reproducible: Always
I fail to see how making something not default fixes any of the issues you've mentioned here, it just hides them under the carpet for people that don't enable the feature. - your sys-apps/mktemp would never happen if another feature (collision-protect) was enabled by default, not to mention the PDEPEND misbehaviour introduced by Bug 180045 (which is basically something I still fail to grok - the only answer I got wrt the whacky stuff going on see Bug 180045 Comment #6 and following there, the only 'answer' I got there was that 'the ebuild is correct', probably because it deletes libs it itself needs, makes sense ;P) - sys-libs/cracklib is plain broken, installing vital stuff orphaned by default (mis-fix for Bug 114690), there's Bug 195017 about this ATM.
Even when unmerge-orphans is disabled, a file collision between two or more packages can still cause serious problems for at least the following 2 reasons: 1) Different packages may install incompatible versions of a file. Depending on the function served by the file, this may or may not be a serious problem. 2) When two packages "own" the same file, the file can be removed when whichever package was installed last is uninstalled, leaving the first package with a missing file. Given the above two scenarios, there is very little benefit from disabling unmerge-orphans. Leaving orphaned files around can also lead to other sorts of unique problems. For example, people have reported problems with orphaned files in /etc/udev/rules.d/ causing udev to behave unexpectedly. The commonly accepted solution is to disallow packages from installing files in the same locations. The collision-protect feature enables us to detect collisions and fix them. In cases where a file moves from one package to another between versions, we can use blockers to indicate which package versions shouldn't be installed simultaneously, and in some cases it's possible for emerge to use blockers to adjust merge order so that file collisions do not occur. I have written a blog post that demonstrates this: http://planet.gentoo.org/developers/zmedico/2007/08/19/using_blockers_to_adjust_merge_order Given collision-protect and blockers, we have the tools necessary to solve file collision issues. However, these tools won't help us unless they are used effectively. Perhaps we should enable collision-protect by default. I would prefer to do that than disable unmerge-orphans in an attempt to hide problems.
Is there a solution for cases like cracklib, where files which used to be considered part of the package no longer are? Collision detection wouldn't help there. Is the cracklib ebuild just Not Doing Something Correctly when it decides that /usr/lib/cracklib_dict.* don't belong to it?
I'm not sure why the cracklib ebuild would intentionally create orphans. Normally, that's not a desirable thing to do.
The problem with enabling collision-protect by default is that it is quite annoying when it stops the a merge because of collisions with orphaned files. For example, sometimes packages decide to start installing python *.py[co] bytecode files which results in a collision with the files that were previously generated at runtime. A useful alternative would be to implement the "soft" collision-protect feature described in bug 140594 and have it log collisions via the elog system. It should be alright to enable that unconditionally, and that way we could collect more collision data from users without have to annoy them too much with halted merges.
(In reply to comment #2) > 1) Different packages may install incompatible versions of a file. Depending on > the function served by the file, this may or may not be a serious problem. > > 2) When two packages "own" the same file, the file can be removed when > whichever package was installed last is uninstalled, leaving the first package > with a missing file. Why should this happen? This causes file collisions. Two ebuilds installing the same files is illegal. The only problem is that file-collision feature isn't enabled by default. Not that I propose disabling the unmerge-orphans feature, but...
(In reply to comment #5) I've been using this collision-protect for ages, it's perfectly usable. The python-updater collision is solved, so generally there's just Bug 136382 left which can be trivially fixed, plus the hardcoded CONFIG_PROTECT on kernel modules (COLLISION_IGNORE="/lib/modules" works just fine there). If something leaves orphaned *.py[co] files somewhere, such ebuild is broken and needs to be fixed to use python_mod_{optimize,cleanup} properly, I've filed quite a couple of bugs about this which are all fixed now, if there are more ebuilds broken like this, they need to be fixed as well. :) Also, not really sure whether overwriting a binary with a completely different one is less annoying than preventing such broken behaviour by default which results in a stopped merge. It's pretty much too late to spit out a log when this already happened. :)
Starting with portage-2.1.3.13 an eerror message will be generated whenever a file collision is detected. The message is logged via elog and the "echo" elog module, which is enabled by default, will echo the eerror message just before emerge exits. These error messages should increase awareness of file collisions so that bug reports are filed when necessary and the problems are corrected as soon as possible. This will increase the likelihood that file collision problems will be solved before ebuilds are marked stable, so the only people who will really need to enable collision-protect will be users of unstable ebuilds.
Zac, you're __way__ too careful here. A lot of people run their systems with "collision-protect" for years, now. Of course there will be stuff showing up, because a number of people will have a messy system, but the tree itself should be quite o.k. in this regard. Just an announcement, so people know what they're up to add stick it in; And while we are at it "userpriv" and "usersandbox" as well. Who disagrees has no problem disabling the features.
(In reply to comment #9) > Zac, you're __way__ too careful here. A lot of people run their systems with > "collision-protect" for years, now. Of course there will be stuff showing up, > because a number of people will have a messy system, but the tree itself should > be quite o.k. in this regard. I agree that the tree is probably mostly alright. However, since unmerge-orphans is relatively new, the majority of users are likely to have lots of orphans on their systems that can trigger false positives. I think that using elog to log all collisions is a good compromise, and a logical first step toward enabling collision-protect by default. > Just an announcement, so people know what they're up to add stick it in; And > while we are at it "userpriv" and "usersandbox" as well. Who disagrees has no > problem disabling the features. There are already some features that have been enabled in the base profile. We can enable more at any time, but it requires discussion on the -dev mailing list.
If we're going to change defaults, I'd suggest we do it in 2.2 as it's much easier for people to deal with such changes if they're tied to a "big" version jump rather than appearing out of nowhere. The main problem I have with enabling collision-protect by default is that a) in my experience more often than not the error is harmless (i.e. there isn't a real collision, just a file being tracked now where it wasn't previously) b) there is no friendly interface to restart the merge operation (without restarting the whole build), which can be extremely annyoing
*** Bug 224259 has been marked as a duplicate of this bug. ***
(In reply to comment #12) > *** Bug 224259 has been marked as a duplicate of this bug. *** You gotta be kidding me? You knew this problem for over a half year and yet let me run into this?
In portage-2.2_rc9 there is a new "protect-owned" feature which is enabled by default via /etc/make.globals. Here is the description from the make.conf.5 man page: This is identical to the collision-protect feature except that files may be overwritten if they are not explicitly listed in the contents of a currently installed package. This is particularly useful on systems that have lots of orphan files that have been left behind by older versions of portage that did not support the unmerge-orphans feature. Like collision-protect, the COLLISION_IGNORE variable can be used to selectively disable this feature. It is recommended to leave either protect-owned or collision-protect enabled at all times, since otherwise file collisions between packages may result in files being overwritten or uninstalled at inappropriate times. If collision-protect is enabled then it takes precedence over protect-owned.
