Hello folks From time to time, an ebuild will be split down to two or more ebuilds and vice versa. In most cases it's "just" an important software which somebody uses, however a few month ago, even a system core package (e2fsprogs-libs) was newly invented by joining two packages (ss and com_err) which itself were slits of (e2fsprogs if I remember correctly). Here, this approach of emerge -C may end up deadly for the portage system, as some other system software including wget just doesn't work afterwards. There are some failsafe mechanisms which may be used manually (like "quickpkg ss com_err" before demerging them) however, portage itself should be able to handle such things automatically. That's where this proposal comes in: When splitting a package into two or more packages, the all new packages should contain a FRIENDS variable naming the base package. This FRIENDS variable shall have the effect that collision protection considers all FRIENDS packages as the same, meaning, that this package may overwrite files from the other packages. Example: Splitting e2fsprogs-libs-1 down to e2fsprogs-libs-2 + ss-2 + com_err-2: * e2fsprogs-libs-2 -> DEPEND="ss-2 com_err-2" * ss-2 (new) -> FRIENDS="<e2fsprogs-libs-2" * com_err-2 (new) -> FRIENDS="<e2fsprogs-libs-2" The FRIENDS variable should be implemented in a way, that more than one package may be mentioned and of course, that versions may be named etc. - just like DEPEND. Of course, this feature must be considered while demerging the old e2fsprogs package if FEATURES=unmerge-orphans is activated (in my opinion, that feature should be deactivated by default anyways or better yet: just being dropped completely again, but YMMV - this shall not be a political discussion). Regards, Bodo
It's not a bad idea (in that it's simple and direct, always a good thing imo.) It shouldn't need a valuable cache metadata slot (i think there's only 4 or 5 spare outta 20) aiui, since it doesn't affect resolution-- assuming blockers etc are still used as currently. If it does, which would be what one wanted with a simple declaration, it's a tougher sell. If you were going to do this, I'd suggest SPLIT, as 'friend' has C++ connotations which you're clearly aiming for (packages have close inter-dependency and should be considered part of the same package) but I'm not sure it's right for this specific case. Declaring that two packages share src is a viable option, and a more generic use-case. To get round the metadata restriction, PROPERTIES are now in portage, but as yet they only apply to binary values. Let's see what portage/Gentoo bods think. > Of course, this feature must be considered while demerging the old e2fsprogs > package if FEATURES=unmerge-orphans is activated (in my opinion, that feature > should be deactivated by default anyways or better yet: just being dropped > completely again, but YMMV - this shall not be a political discussion). > Heh, nw; it clearly has usage or it wouldn't be around, and imo is better for default as packages often add cruft all over the shop. Could it be refined? I'd imagine so; patches are always welcome for review, if nothing else, which gets your code to a better level. (But do that on another bug;)
(In reply to comment #0) > From time to time, an ebuild will be split down to two or more ebuilds and vice > versa. In most cases it's "just" an important software which somebody uses, > however a few month ago, even a system core package (e2fsprogs-libs) was newly > invented by joining two packages (ss and com_err) which itself were slits of > (e2fsprogs if I remember correctly). Here, this approach of emerge -C may end > up deadly for the portage system, as some other system software including wget > just doesn't work afterwards. There are some failsafe mechanisms which may be > used manually (like "quickpkg ss com_err" before demerging them) however, > portage itself should be able to handle such things automatically. I guess you're not aware that portage-2.1.6 is already capable of solving this automatically. It could still use some improvement for some cases (see bug #248110, comment #12), but it's possible to use masks as a workaround (like qt-4.3* was masked as discussed in bug #248110). Backtracking support, which is planned for bug #1343, will make such masking unnecessary. The FRIENDS variable that you suggest would actually be redundant since the necessary information is already included in the existing dependencies. All that's really needed is to analyze the dependencies and backtrack when necessary (but #1343).
(In reply to comment #0) > Of course, this feature must be considered while demerging the old e2fsprogs > package if FEATURES=unmerge-orphans is activated (in my opinion, that feature > should be deactivated by default anyways or better yet: just being dropped > completely again, but YMMV - this shall not be a political discussion). Since portage-2.1.6, FEATURES=protect-owned is enabled by default and this makes FEATURES=unmerge-orphans much safer since merges will automatically abort when a file collision with another package is detected.
(In reply to comment #2) > I guess you're not aware that portage-2.1.6 is already capable of solving this > automatically. 1233111184: >>> emerge (1 of 3) app-admin/eselect-1.0.11-r1 to / 1233111234: >>> unmerge success: app-admin/eselect-1.0.10 1233111234: ::: completed emerge (1 of 3) app-admin/eselect-1.0.11-r1 to / 1233111234: >>> emerge (2 of 3) app-admin/eselect-news-20080320 to / 1233111249: --- AUTOCLEAN: Nothing unmerged. 1233111249: ::: completed emerge (2 of 3) app-admin/eselect-news-20080320 to / 1233111249: >>> emerge (3 of 3) sys-apps/portage-2.1.6.4 to / 1233111425: --- AUTOCLEAN: Nothing unmerged. 1233111425: ::: completed emerge (3 of 3) sys-apps/portage-2.1.6.4 to / 1233111430: *** exiting successfully. Then I started my e2fsprogs update ride consisting of: 1233112318: *** emerge --deep --ask --update --verbose system [canceled] 1233112527: *** emerge --oneshot --ask --verbose e2fsprogs [canceled] 1233112590: *** emerge --oneshot --ask --verbose =e2fsprogs-1.41.0 [canceled] 1233166428: *** emerge --oneshot e2fsprogs-libs [canceled] Here I did the quickpkg - or in other words: I gave up ... 1233167046: Started emerge on: Jan 28, 2009 19:24:06 1233167057: >>> unmerge success: sys-libs/ss-1.40.9 1233167060: >>> unmerge success: sys-libs/com_err-1.40.9 1233167060: *** exiting successfully. 1233167113: >>> emerge (1 of 1) sys-fs/e2fsprogs-1.41.3 to / 1233167376: ::: completed emerge (1 of 1) sys-fs/e2fsprogs-1.41.3 to / 1233167382: *** exiting successfully. So, either you have a regression from portage-2.1.6 tp portage-2.1.6.4 or your statement is just wrong (or the new feature is just not used by the offending packages). > The FRIENDS variable that you suggest would actually be redundant since the > necessary information is already included in the existing dependencies. The e2fsprogs-block is not a bug of missing information. It's a problem of explicitely DEPEND="!$old-version" to prevent the collision protect to kick in. Regards, Bodo
(In reply to comment #3) > Since portage-2.1.6, FEATURES=protect-owned is enabled by default and this > makes FEATURES=unmerge-orphans much safer since merges will automatically abort > when a file collision with another package is detected. Exactly that package splitting problem is the reason for me switching of this unmerge-orphans feature completely, because in most cases, if new-package-2 is blocked because of old-package-1, a simple # FEATURES="-collision-protect" emerge -1O =new-package-2 # emerge -1 =old-package-2 just works fine. However, this proposal would make this workarounds unneccessarry but must be considered in the unmerge-orphans feature, because effectively it's going to have the same result. Regards, Bodo
(In reply to comment #1) > If you were going to do this, I'd suggest SPLIT, as 'friend' has C++ > connotations which you're clearly aiming for [...] I first thought about SPLIT but then you had the problem, that it's not obvious, that it could be used for joins as well. So I searched a more general word - friends. Despite the fact I'm a C++ programmer, the C++ keyword didn't inspire my choose for that. However, you could use other terms as well, like ASSUME_SAME or something like that. > To get round the metadata restriction, PROPERTIES are now in portage, but as > yet they only apply to binary values. Can you provide any link where that feature is described? I found some rfc from a mailing list but I didn't really understand what that guy actually wants and his example doesn't really make sense to me. Regards, Bodo
(In reply to comment #4) > So, either you have a regression from portage-2.1.6 tp portage-2.1.6.4 or your > statement is just wrong (or the new feature is just not used by the offending > packages). Well, it's just that there are some things that can prevent it from working optimally. For example, if you have either sys-libs/ss or sys-libs/com_err listed in /var/lib/portage/world then it's taken as an indication that these packages should not be uninstalled. In portage-2.1.6.7, which doesn't have stable keywords yet, the output is improved in order to make it easier to recognize cases like this. For example, see bug #234907, comment #138. > > The FRIENDS variable that you suggest would actually be redundant since the > > necessary information is already included in the existing dependencies. > > The e2fsprogs-block is not a bug of missing information. It's a problem of > explicitely DEPEND="!$old-version" to prevent the collision protect to kick in. So, are you agreeing with me that your suggested FRIENDS variable would be redundant? As you can see in bug #234907, comment #37, emerge is capable of solving this conflict by temporarily installing conflicting packages simultaneously. You can see some other examples of this in my blog: http://blogs.gentoo.org/zmedico/2008/05/09/blocking_package_file_collisions
(In reply to comment #5) > just works fine. However, this proposal would make this workarounds > unneccessarry but must be considered in the unmerge-orphans feature, because > effectively it's going to have the same result. As I've explained, the workarounds are already unnecessary. If a particular blocker is not solving automatically, the latest version of portage (which does not have stable keywords yet) will show additional information about why the blocker will not solve (for example, see bug #234907, comment #138). I can assure you that your suggested FRIENDS variable would be redundant. If you still believe that it would not be redundant, you are going to have to do a very good job of explaining why this information cannot be gleaned from the existing dependencies. Otherwise, ebuild developers will certainly reject it as redundant and unnecessary.
Aside from blocker resolution enhancements via bug 1343, another thing we can do to make things easier is to prompt the user to remove packages from the world file as suggested in bug #257009.
(In reply to comment #7) > So, are you agreeing with me that your suggested FRIENDS variable would be > redundant? As you can see in bug #234907, comment #37, emerge is capable of > solving this conflict by temporarily installing conflicting packages > simultaneously. You can see some other examples of this in my blog: $ emerge --version Portage 2.1.6.4 (default/linux/x86/2008.0/desktop, gcc-4.1.2, glibc-2.6.1-r0, 2.6.25 i686) >>> Installing app-text/scrollkeeper-dtd-1.0 * checking 1 files for package collisions * This package will overwrite one or more files that may belong to other * packages (see list below). You can use a command such as `portageq * owners / <filename>` to identify the installed package that owns a * file. If portageq reports that only one package owns a file then do * NOT file a bug report. A bug report is only useful if it identifies at * least two or more packages that are known to install the same file(s). * If a collision occurs and you can not explain where the file came from * then you should simply ignore the collision since there is not enough * information to determine if a real problem exists. Please do NOT file * a bug report at http://bugs.gentoo.org unless you report exactly which * two packages install the same file(s). Once again, please do NOT file * a bug report unless you have completely understood the above message. * * package app-text/scrollkeeper-dtd-1.0 NOT merged * * Detected file collision(s): * * /usr/share/xml/scrollkeeper/dtds/scrollkeeper-omf.dtd * * Searching all installed packages for file collisions... * * Press Ctrl-C to Stop * * app-text/scrollkeeper-0.3.14-r2 * /usr/share/xml/scrollkeeper/dtds/scrollkeeper-omf.dtd * * Package 'app-text/scrollkeeper-dtd-1.0' NOT merged due to file * collisions. If necessary, refer to your elog messages for the whole * content of the above message. >>> Failed to install app-text/scrollkeeper-dtd-1.0, Log file: Is this your definition of "emerge is capable of solving this conflict by temporarily installing conflicting packages simultaneously"? The dependency list doesn't contain any meaning of WHY there is a blocking. It can be because the new package just overwrites files formerly belonging to other packages or it can be, because the new package actually fails compiling while the older version of another package is still being installed. There are many reasons, why two packages are blocking each other, I'm just wondering why I as a user have to describe the problems to you, the developers. In fact, it should be the other way. Regards, Bodo PS: As I obviously still need to resolve such problems by hand, how can I switch back from the broken depencendy solver to the functioning one from previous versions without downgrading portage completely?
(In reply to comment #10) > Is this your definition of "emerge is capable of solving this conflict by > temporarily installing conflicting packages simultaneously"? No. If there's a blocker and you do not use --nodeps then file collisions are handled automatically. So, the case you show above did not meet these requirements for some reason. > The dependency list doesn't contain any meaning of WHY there is a blocking. It > can be because the new package just overwrites files formerly belonging to > other packages or it can be, because the new package actually fails compiling > while the older version of another package is still being installed. Since EAPI 2 we hav 2 different kinds of blockers. For the file overwriting case you use the same old !atom syntax. For a compile time blocker, use the new !!atom syntax. > There are many reasons, why two packages are blocking each other, I'm just > wondering why I as a user have to describe the problems to you, the developers. > In fact, it should be the other way. I'm sorry, I should have explained the above earlier. I did now, since you asked. FWIW, the new blocker syntax is briefly documented in `man 5 ebuild`. > PS: As I obviously still need to resolve such problems by hand, how can I > switch back from the broken depencendy solver to the functioning one from > previous versions without downgrading portage completely? Trust me. You'll like the new one once you've gotten accustomed to it.
(In reply to comment #11) > No. If there's a blocker and you do not use --nodeps then file collisions are > handled automatically. So, the case you show above did not meet these > requirements for some reason. I did NOT use --nodeps: 1235222111: Started emerge on: Feb 21, 2009 14:15:11 1235222111: *** emerge --deep --oneshot --ask --update --verbose gnome-base/gdm [...] 1235226975: >>> emerge (13 of 120) app-text/scrollkeeper-dtd-1.0 to / 1235226981: === (13 of 120) Cleaning (app-text/scrollkeeper-dtd-1.0::/var/portage/app-text/scrollkeeper-dtd/scrollkeeper-dtd-1.0.ebuild) 1235226981: === (13 of 120) Compiling/Merging (app-text/scrollkeeper-dtd-1.0::/var/portage/app-text/scrollkeeper-dtd/scrollkeeper-dtd-1.0.ebuild) 1235227002: === (13 of 120) Merging (app-text/scrollkeeper-dtd-1.0::/var/portage/app-text/scrollkeeper-dtd/scrollkeeper-dtd-1.0.ebuild) 1235227052: *** Finished. Cleaning up... 1235227052: *** exiting unsuccessfully with status '1'. 1235227067: *** terminating. Here the file collision occured I then resolved it by issuing the command # FEATURES="-collision-protect -protect-owned" ebuild /usr/portage/app-text/scrollkeeper-dtd/scrollkeeper-dtd-1.0.ebuild qmerge followed by # emerge --resume --skip-first > Since EAPI 2 we hav 2 different kinds of blockers. For the file overwriting > case you use the same old !atom syntax. For a compile time blocker, use the > new!!atom syntax. Ok, I didn't know that. In this case I apologise for my accusations. However, in my opinion, the here proposed FRIEND feature would have been much more intuitive. > Trust me. You'll like the new one once you've gotten accustomed to it. Obviously, it doesn't work as expected, yet. However, as soon as it works, most probably I will in fact get accustomed to it. Regards, Bodo
(In reply to comment #12) > Obviously, it doesn't work as expected, yet. However, as soon as it works, most > probably I will in fact get accustomed to it. I used portage-2.1.6.7 to try same upgrade and it handled the blockers successfully: # emerge -v1q app-text/scrollkeeper-dtd app-text/scrollkeeper app-text/rarian [ebuild N ] app-text/scrollkeeper-dtd-1.0 [ebuild N ] app-text/rarian-0.8.1 USE="-debug" [ebuild U ] app-text/scrollkeeper-9999-r1 [0.3.14-r2] USE="(-nls%*)" [blocks b ] <app-text/scrollkeeper-9999 ("<app-text/scrollkeeper-9999" is blocking app-text/rarian-0.8.1) [blocks b ] <app-text/scrollkeeper-9999-r1 ("<app-text/scrollkeeper-9999-r1" is blocking app-text/scrollkeeper-dtd-1.0) >>> Verifying ebuild manifests >>> Starting parallel fetch >>> Emerging (1 of 3) app-text/scrollkeeper-dtd-1.0 >>> Installing app-text/scrollkeeper-dtd-1.0 >>> Emerging (2 of 3) app-text/rarian-0.8.1 >>> Installing app-text/rarian-0.8.1 >>> Emerging (3 of 3) app-text/scrollkeeper-9999-r1 >>> Installing app-text/scrollkeeper-9999-r1 >>> Jobs: 3 of 3 complete Load avg: 1.20, 0.85, 0.47 * GNU info directory index is up-to-date. I'm not sure why the file collision occurred in your case. Please file a separate bug if you are able to reproduce that problem.
so... if I'm understanding this right, we don't need this so the bug can be closed?
(In reply to Alex Xu (Hello71) from comment #14) > so... if I'm understanding this right, we don't need this so the bug can be > closed? No progress since six years. Closing.
