Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!

Bug 185377

Summary: Implement repository dependencies
Product: Portage Development Reporter: David Carlos Manuelda <StormByte>
Component: Core - Interface (emerge)Assignee: Portage team <dev-portage>
Status: RESOLVED FIXED    
Severity: enhancement CC: ag, clemenswaechter, coldwind, de.techno, esigra, gentoo, n-roeser, pesa
Priority: High Keywords: InVCS
Version: unspecified   
Hardware: All   
OS: Linux   
Whiteboard:
Package list:
Runtime testing required: ---
Bug Depends on:    
Bug Blocks: 155723, 240187, 340675, 358927    

Description David Carlos Manuelda 2007-07-15 05:42:23 UTC
I have couple of overlays using layman, one of them is xeffects. From this overlay I am interested only in couple of packages, but this overlay is huge. When I update kde, it takes by deffect the overlay's ones, despite same version is available in main tree (for example kde things).

¿Could it be possible to select wich ebuilds to use (something like ignore overlays for that ebuild...?

Reproducible: Always
Comment 1 Marius Mauch (RETIRED) gentoo-dev 2007-07-15 08:23:53 UTC
Zac, are you planning to integrate repository deps in the new resolver?
Comment 2 Zac Medico gentoo-dev 2007-07-15 10:38:41 UTC
Repository deps are easy enough to do.  For this bug though, maybe what's needed is the ability to prioritize repositories.  For example, make the resolver prefer ebuilds from the main tree over the ones in the xeffects overlay whenever possible.  If necessary, packages from the overlay would still be able to use repository deps to pull in their own version of a package.
Comment 3 Marius Mauch (RETIRED) gentoo-dev 2007-07-15 12:13:38 UTC
Well, repo deps would be a requirement for this issue one way or the other.
Comment 4 Zac Medico gentoo-dev 2007-07-15 12:48:02 UTC
(In reply to comment #0)
> When I update kde, it takes by deffect the overlay's ones, despite same version
> is available in main tree (for example kde things).

FWIW, you can make the main tree highest priority by appending it to PORTDIR_OVERLAY like this:

PORTDIR="/usr/portage"
PORTDIR_OVERLAY="/usr/local/portage /usr/portage"

That will cause the main tree to be preferred if it contains the highest visible version of a package.
Comment 5 Mike McQuaid 2007-07-16 22:37:37 UTC
*** Bug 185607 has been marked as a duplicate of this bug. ***
Comment 6 David Carlos Manuelda 2007-07-25 15:38:50 UTC
I suggest:

How about making by default, a highest priority for the main tree? For example, if a ebuild does not exists in the main tree, pick it from an overlay, and from the portage's tree if exists.
I use overlays for this proposit, to install a software WICH IS NOT in the portage's tree (like knetworkmanager) but not to prefer overlays for customization of kde for example..

Can this be made for default?
Comment 7 Davide Pesavento gentoo-dev 2007-08-23 16:12:05 UTC
Isn't this similar to bug #170288 ?
Comment 8 David Carlos Manuelda 2007-09-14 03:09:51 UTC
Is a bit similar, but not equal.

I was thinking about it, and I suggest one thing:

How about the possibility to add another "operator"?

For example, there are >=, ~, etc operator for ebuilds selection which portage understands, why about adding a "*" operator which will select IN-TREE ebuild rather than overlay ones whenever possible?

This way, I could have xeffects overlay (which have most kde things common with main tree), and I could emerge *kdm (which will install /usr/portage/kde-base/kdm/kdm-version.ebuild)

What do you think about this (portage-devs)?
Comment 9 Marius Mauch (RETIRED) gentoo-dev 2007-09-14 10:23:06 UTC
That's what I was referring to in comment #1.
Comment 10 Bernhard Frauendienst 2007-09-20 18:32:12 UTC
There is actually another usage for this feature: I'm using some kde/qt ebuilds from xeffects overlay, and partly depend on each other. The patched kdelibs for example requires the patched qt version.
This is of course not defined in the DEPEND section of the ebuild (since ebuilds cannot depend on certain USE flags to be existant or on a certain repository/overlay). Now if one of those ebuild gets bumped in the main tree, a world update will emerge the unpatched version (of qt for example) which breaks the patched apps from the overlay (kicker for example).

As a workaround I excluded the relevant packages to be rsynced, but this doesn't really help if only certain (old) slots are in the overlay (like qt-3*... qt-4* is only in the main tree).

So it would be great to have a /etc/portage/package.repository* (for example), which allows the user to select *certain package atoms* to be prioritized or forced from a certain repository/overlay. 

To prefer the main tree over certain overlays, one could use a line in package.repository:
> category-name/package main "some other overlay"

This would first use the main tree version, then the one from "some other overlay" (could be the reponame, or maybe rather a path name?), and then versions from any other overlay.

To complete my qt-example, I would use package.repository.force with a line like
> =x11-libs/qt-3* xeffects
to force qt-3* to be taken from the xeffects overlay, even if there's a newer version in the main tree.

For the sake of completeness, also package.repository.mask and package.repository.unmask could be considered. 
Comment 11 Alec Warner archtester Gentoo Infrastructure gentoo-dev Security 2007-09-22 23:11:51 UTC
(In reply to comment #10)
> There is actually another usage for this feature: I'm using some kde/qt ebuilds
> from xeffects overlay, and partly depend on each other. The patched kdelibs for
> example requires the patched qt version.
> This is of course not defined in the DEPEND section of the ebuild (since
> ebuilds cannot depend on certain USE flags to be existant or on a certain
> repository/overlay). Now if one of those ebuild gets bumped in the main tree, a
> world update will emerge the unpatched version (of qt for example) which breaks
> the patched apps from the overlay (kicker for example).
> 
> As a workaround I excluded the relevant packages to be rsynced, but this
> doesn't really help if only certain (old) slots are in the overlay (like
> qt-3*... qt-4* is only in the main tree).
> 
> So it would be great to have a /etc/portage/package.repository* (for example),
> which allows the user to select *certain package atoms* to be prioritized or
> forced from a certain repository/overlay. 
> 
> To prefer the main tree over certain overlays, one could use a line in
> package.repository:
> > category-name/package main "some other overlay"
> 
> This would first use the main tree version, then the one from "some other
> overlay" (could be the reponame, or maybe rather a path name?), and then
> versions from any other overlay.
> 
> To complete my qt-example, I would use package.repository.force with a line
> like
> > =x11-libs/qt-3* xeffects
> to force qt-3* to be taken from the xeffects overlay, even if there's a newer
> version in the main tree.
> 
> For the sake of completeness, also package.repository.mask and
> package.repository.unmask could be considered. 
> 

So again, these are all just hacks around not hacking repository dependencies.  Ideally the overlay ebuilds would DEPEND on other overlay ebuilds and we avoid this whole addition of silly files ;)

I liked Zacs idea of just reordering PORTAGE_OVERLAY, simple, already implemented, no effort for probably 80-90% coverage of cases ;)
Comment 12 Marius Mauch (RETIRED) gentoo-dev 2007-10-06 16:39:37 UTC
(In reply to comment #11)

> So again, these are all just hacks around not hacking repository dependencies. 
> Ideally the overlay ebuilds would DEPEND on other overlay ebuilds and we avoid
> this whole addition of silly files ;)

Not really ideal as you've changed a spec that's too loose into one that's probably too strict (who says that only the listed repo has a compatible version?). But probably better than the status quo.
Comment 13 Marius Mauch (RETIRED) gentoo-dev 2008-06-08 23:26:00 UTC
*** Bug 225469 has been marked as a duplicate of this bug. ***
Comment 14 Marius Mauch (RETIRED) gentoo-dev 2008-07-02 10:00:57 UTC
*** Bug 230485 has been marked as a duplicate of this bug. ***
Comment 15 Santiago M. Mola (RETIRED) gentoo-dev 2008-09-03 14:15:12 UTC
I usually use 5-7 overlays with Gentoo. When using Paludis, it's enough for me to be able to mask/unmask/unkeyword packages by repo (as in ::repo deps), and in just a couple of cases modifying the priority.

That's an optimal solution for my day to day use.
Comment 16 Christian Harms 2010-03-02 23:37:42 UTC
(In reply to comment #11)
> I liked Zacs idea of just reordering PORTAGE_OVERLAY, simple, already
> implemented, no effort for probably 80-90% coverage of cases ;)

I was about to do this, admiring the elegance of the solution to something that was bugging me as well, when I stumbled over
$ cat /usr/portage/local/layman/make.conf
PORTDIR_OVERLAY="
/usr/portage/local/layman/science
/usr/portage/local/layman/games
/usr/portage/local/layman/sunrise
/usr/portage/local/layman/gamerlay
/usr/portage/local/layman/kde-sunset
$PORTDIR_OVERLAY
"

as curious as I might be, I think for your %age estimation to be correct, one should remember to source layman's make.conf before setting
PORTDIR_OVERLAY="/usr/local/portage /usr/portage $PORTDIR_OVERLAY"
(or somesuch, those happen to be my wanted priorities)
Comment 17 Zac Medico gentoo-dev 2010-10-01 16:12:24 UTC
This is fixed by the 'reposyntax' patches, like this one:

http://git.overlays.gentoo.org/gitweb/?p=proj/portage.git;a=commit;h=e177e729bad3659cb2610f511928f90868bbebc3
Comment 18 Zac Medico gentoo-dev 2010-10-06 21:15:44 UTC
This is fixed in 2.2_rc90, but I'll leave this bug open until it's in an unmasked release.
Comment 19 Zac Medico gentoo-dev 2010-10-12 12:59:28 UTC
*** Bug 340675 has been marked as a duplicate of this bug. ***
Comment 20 Zac Medico gentoo-dev 2010-11-21 08:28:24 UTC
*** Bug 346209 has been marked as a duplicate of this bug. ***
Comment 21 Navid Zamani 2011-03-19 19:33:28 UTC
How come nobody here just recommended adding syntax to DEPEND atoms, allowing for repository selection?
Example: =kde-base/kdelibs:3.5@kde-sunset
Where “@overlayname” is saying “From the overlay “overlayname”.
Sipmle, intuitive, backwards-compatible, only one place to change (I assume)… And as someone here said, allowing for much more than simple overlay priority ordering. :)
Comment 22 Zac Medico gentoo-dev 2011-03-19 19:42:16 UTC
(In reply to comment #21)
> How come nobody here just recommended adding syntax to DEPEND atoms, allowing
> for repository selection?
> Example: =kde-base/kdelibs:3.5@kde-sunset
> Where “@overlayname” is saying “From the overlay “overlayname”.

All gentoo's package managers support atom::repo syntax. It's already decided an nobody wants to change the syntax since that would only be extra work for everyone involved.

> Sipmle, intuitive, backwards-compatible, only one place to change (I assume)…
> And as someone here said, allowing for much more than simple overlay priority
> ordering. :)

These kinds of syntax changes are never backwards-compatible.
Comment 23 Zac Medico gentoo-dev 2011-06-06 13:04:52 UTC
This is fixed in 2.1.10.
Comment 24 dE 2011-06-07 00:19:16 UTC
(In reply to comment #23)
> This is fixed in 2.1.10.

OK..... thanks!

Any info about this?
Comment 25 Zac Medico gentoo-dev 2011-06-07 00:32:46 UTC
It looks like we forgot to document this feature. However, it's really simple. Any place you'd use an atom, use atom::repo to specify the repo that the atom should match. For example, sys-apps/portage::gentoo would match portage from the "gentoo" repo.
Comment 26 dE 2011-06-07 14:46:20 UTC
This will work in package.*?
Comment 27 Zac Medico gentoo-dev 2011-06-07 15:02:23 UTC
Yes, it works in /etc/portage/package.* and also in atoms given as emerge command line arguments. In package.*, you can use atom::repo together with wildcard atoms (from bug 340675). For example you can use */*::gentoo to match all ebuilds from the gentoo repo.

In case it's not obvious, I should also note that the repo name of a given overlay is specified in the profiles/repo_name file. This file is documented in the portage(5) man page.
Comment 28 Zac Medico gentoo-dev 2011-06-08 07:04:08 UTC
We currently have some documentation for the wildcards here:

http://git.overlays.gentoo.org/gitweb/?p=proj/portage.git;a=commit;h=b1692269047db9f6c59f5203629ab2fc68423086

What I'd like to do is move the wildcard docs into an "Extended Atom Syntax" section, and include some atom::repo documentation there.
Comment 29 Zac Medico gentoo-dev 2011-06-09 22:09:37 UTC
(In reply to comment #28)
> What I'd like to do is move the wildcard docs into an "Extended Atom Syntax"
> section, and include some atom::repo documentation there.

This is in git now:

http://git.overlays.gentoo.org/gitweb/?p=proj/portage.git;a=commit;h=ffbaba42bebda213154dbba32e32c245c19e16b2
Comment 30 Zac Medico gentoo-dev 2011-06-10 09:44:10 UTC
The new docs are included in portage-2.1.10.1 and 2.2.0_alpha39.