Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!

Bug 16365

Summary: Emerge can't handle circular dependencies
Product: Portage Development Reporter: Hiel Van Campen <hielvc>
Component: UnclassifiedAssignee: Portage team <dev-portage>
Status: RESOLVED FIXED    
Severity: normal CC: cesarb, craig.lawson, denys.duchier, gentoo, jason, johnm, news, oldium.pro, re-vax, sascha-gentoo-bugzilla, sn.ml, virginsnow
Priority: High Keywords: InVCS
Version: unspecified   
Hardware: x86   
OS: Linux   
Whiteboard:
Package list:
Runtime testing required: ---
Bug Depends on: 147766    
Bug Blocks: 155723, 147007    
Attachments: Ignore blockers on packages about to be updated

Description Hiel Van Campen 2003-02-25 14:40:38 UTC
This problem does not show all the time. I have seen through various versions of
portage but not my current version, which is 1 day old. Ok the problem that I
have seen is taht when doing "emerge -u? system, world , gnome, or kde about 1
out three times I will get an error saying sefault file.. When I restart emerge
(up arrow enter) instead trying to rebuild the last package, indecting that it
did segfault it starts emergeing a different package. Ex: emerge segfault on
gl-updates, there was nothing in the first 3 dirs it made to make. quit with
segfault error I restared and it emerged ttmkdir next and about 3 pkges latter
did gl-update with no probs. This only seems to occur when there are akot of
files to emerge. It could also be related to bug# 1661 where doing parallel
emerge -uf world and emerge -u world can get in to trouble. This happened when
gnome 2.2 was relessed. I started emerge -f gnome and had down loaded 20 files,
Figureing that was a good enough lead I stared emerge -ub gonome in vc4 and
about ten packages in, emerge -ub started downloading instead of just building.
checking it out, both emerges were downloading the same package at the same time
& one stoping screwed the other. The emerge -f had 40 to 43 pkgs downloaded by
that time.

ANy way the emerge out step has occoured many times, and  99% of them I wasnt
during parrallel build as with the above example, single instance of emerge
running. 

Reproducible: Sometimes
Steps to Reproduce:
1 make a new system from gentoo-grp cd.and the load the grp pckges 
2.emerge -u world 
3.I havent tried this but rac & kanapus got my dander up so if I get a new hd
installed today I can try it latter this wk. This happened after 




Portage 2.0.46-r12 (default-x86-1.4, gcc-3.2.2, glibc-2.3.1-r2)
=================================================================
System uname: 2.4.20-gentoo-r1 i686 AMD Athlon(tm) XP 1800+
GENTOO_MIRRORS="ftp://mirror.iawnet.sandia.gov/pub/gentoo
http://gentoo.oregonstate.edu/
http://distro.ibiblio.org/pub/Linux/distributions/gentoo"
CONFIG_PROTECT="/etc /var/qmail/control /usr/kde/2/share/config
/usr/kde/3/share/config /usr/X11R6/lib/X11/xkb /usr/share/config"
CONFIG_PROTECT_MASK="/etc/gconf /etc/env.d"
PORTDIR="/home/portage"
DISTDIR="/home/portage/distfiles"
PKGDIR="/home/portage/packages"
PORTAGE_TMPDIR="/var/tmp"
PORTDIR_OVERLAY=""
USE="x86 oss 3dnow apm arts avi crypt cups encode gif jpeg kde libg++ libwww
mikmod mmx mpeg ncurses nls pdflib png qtmt quicktime spell truetype xml2 xmms
xv zlib gdbm berkdb slang readline svga tcltk java guile sdl gpm tcpd pam ssl
perl python esd imlib oggvorbis qt motif opengl mozilla cdr X gtk gnome alsa"
COMPILER="gcc3"
CHOST="i686-pc-linux-gnu"
CFLAGS="-march=athlon-xp -O2 -pipe"
CXXFLAGS="-march=athlon-xp -O2 -pipe"
ACCEPT_KEYWORDS="x86"
MAKEOPTS="-j2"
AUTOCLEAN="yes"
SYNC="rsync://rsync.gentoo.org/gentoo-portage"
FEATURES="sandbox ccache"
Comment 1 Hiel Van Campen 2003-12-28 08:10:07 UTC
This just a question which about the troubles with dependencies. The same prob that catalyst should solve. SO close this sucker!

Have a great New Years

hielvc
Comment 2 Hiel Van Campen 2004-09-17 11:20:39 UTC
Still does ;^) but not quit as often,those nasty dependence trees.
Comment 3 Jason Stubbs (RETIRED) gentoo-dev 2005-10-07 07:30:11 UTC
*** Bug 11272 has been marked as a duplicate of this bug. ***
Comment 4 Jason Stubbs (RETIRED) gentoo-dev 2005-10-07 07:30:59 UTC
*** Bug 16538 has been marked as a duplicate of this bug. ***
Comment 5 Jason Stubbs (RETIRED) gentoo-dev 2005-10-07 08:34:02 UTC
*** Bug 37453 has been marked as a duplicate of this bug. ***
Comment 6 Jason Stubbs (RETIRED) gentoo-dev 2005-10-07 08:58:24 UTC
*** Bug 41019 has been marked as a duplicate of this bug. ***
Comment 7 Jason Stubbs (RETIRED) gentoo-dev 2005-10-07 09:04:57 UTC
*** Bug 44632 has been marked as a duplicate of this bug. ***
Comment 8 Jason Stubbs (RETIRED) gentoo-dev 2005-10-09 20:22:18 UTC
*** Bug 108664 has been marked as a duplicate of this bug. ***
Comment 9 Denys Duchier 2005-10-10 02:09:36 UTC
(In reply to comment #8)  
> *** Bug 108664 has been marked as a duplicate of this bug. ***  
  
May I ask what the difficulty is in _noticing_ a circularity in the  
dependency graph and informing the user of it? 
 
Comment 10 Denys Duchier 2005-10-10 07:01:04 UTC
I noticed something that looks odd in the computation on the depgraph: 
 
% emerge -e cyrus-sasl -pv --debug 
 
gives me (among a lot of output): 
 
Parent:    ebuild / net-nds/openldap-2.2.28-r1 merge 
Depstring: >=sys-libs/ncurses-5.1 tcpd? ( >=sys-apps/tcp-wrappers-7.6 ) ssl? 
( >=dev-libs/openssl-0.9.6 ) readline? ( >=sys-libs/readline-4.1 ) sasl? 
( >=dev-libs/cyrus-sasl-2.1.7-r3 ) odbc? ( dev-db/unixODBC ) slp? 
( >=net-libs/openslp-1.0 ) perl? ( >=dev-lang/perl-5.6 ) samba? 
( >=dev-libs/openssl-0.9.6 ) kerberos? ( virtual/krb5 ) berkdb? 
( >=sys-libs/db-4.2.52_p1 ) !berkdb? ( gdbm? ( >=sys-libs/gdbm-1.8.0 ) !gdbm? 
( >=sys-libs/db-4.2.52_p1 ) ) >=sys-devel/libtool-1.5.18-r1 
>=sys-apps/sed-4 !bootstrap? ( sys-devel/patch ) !bootstrap? 
( sys-devel/patch ) >=sys-libs/ncurses-5.1 tcpd? 
( >=sys-apps/tcp-wrappers-7.6 ) ssl? ( >=dev-libs/openssl-0.9.6 ) readline? 
( >=sys-libs/readline-4.1 ) sasl? ( >=dev-libs/cyrus-sasl-2.1.7-r3 ) odbc? 
( dev-db/unixODBC ) slp? ( >=net-libs/openslp-1.0 ) perl? 
( >=dev-lang/perl-5.6 ) samba? ( >=dev-libs/openssl-0.9.6 ) kerberos? 
( virtual/krb5 ) berkdb? ( >=sys-libs/db-4.2.52_p1 ) !berkdb? ( gdbm? 
( >=sys-libs/gdbm-1.8.0 ) !gdbm? ( >=sys-libs/db-4.2.52_p1 ) ) 
Candidates: ['>=sys-libs/readline-4.1', '>=dev-libs/openssl-0.9.6', 
'dev-db/unixODBC', '>=net-libs/openslp-1.0', '>=sys-devel/libtool-1.5.18-r1', 
'>=sys-apps/tcp-wrappers-7.6'] 
 
notice how cyrus-sasl is missing from the "Candidates" 
 
while: 
 
% emerge -e openldap -pv --debug 
 
gives me: 
 
Parent:    ebuild / net-nds/openldap-2.2.28-r1 merge 
Depstring: >=sys-libs/ncurses-5.1 tcpd? ( >=sys-apps/tcp-wrappers-7.6 ) ssl? 
( >=dev-libs/openssl-0.9.6 ) readline? ( >=sys-libs/readline-4.1 ) sasl? 
( >=dev-libs/cyrus-sasl-2.1.7-r3 ) odbc? ( dev-db/unixODBC ) slp? 
( >=net-libs/openslp-1.0 ) perl? ( >=dev-lang/perl-5.6 ) samba? 
( >=dev-libs/openssl-0.9.6 ) kerberos? ( virtual/krb5 ) berkdb? 
( >=sys-libs/db-4.2.52_p1 ) !berkdb? ( gdbm? ( >=sys-libs/gdbm-1.8.0 ) !gdbm? 
( >=sys-libs/db-4.2.52_p1 ) ) >=sys-devel/libtool-1.5.18-r1 
>=sys-apps/sed-4 !bootstrap? ( sys-devel/patch ) !bootstrap? 
( sys-devel/patch ) >=sys-libs/ncurses-5.1 tcpd? 
( >=sys-apps/tcp-wrappers-7.6 ) ssl? ( >=dev-libs/openssl-0.9.6 ) readline? 
( >=sys-libs/readline-4.1 ) sasl? ( >=dev-libs/cyrus-sasl-2.1.7-r3 ) odbc? 
( dev-db/unixODBC ) slp? ( >=net-libs/openslp-1.0 ) perl? 
( >=dev-lang/perl-5.6 ) samba? ( >=dev-libs/openssl-0.9.6 ) kerberos? 
( virtual/krb5 ) berkdb? ( >=sys-libs/db-4.2.52_p1 ) !berkdb? ( gdbm? 
( >=sys-libs/gdbm-1.8.0 ) !gdbm? ( >=sys-libs/db-4.2.52_p1 ) ) 
Candidates: ['>=sys-apps/tcp-wrappers-7.6', '>=dev-lang/perl-5.6', 
'>=dev-libs/openssl-0.9.6', 'dev-db/unixODBC', '>=sys-libs/db-4.2.52_p1', 
'>=net-libs/openslp-1.0', '>=sys-devel/libtool-1.5.18-r1', 'sys-devel/patch', 
'>=sys-libs/ncurses-5.1', '>=sys-libs/readline-4.1', '>=sys-apps/sed-4', 
'>=dev-libs/cyrus-sasl-2.1.7-r3'] 
 
where it correctly appears. 
 
I have the sinking feeling that emerge does not actually build a correct 
and complete depgraph :-/ 
 
Can anyone enlighten me as to what the core problem is? 
Comment 11 Jason Stubbs (RETIRED) gentoo-dev 2005-10-10 07:34:49 UTC
The core problem is that emerge does not actually build a correct and complete 
depgraph. ;) 
 
Instead it assumes that any package that already exists in the graph has had 
its dependencies taken care of. Based on this (incorrect) assumption, it then 
pulls leaf packages from the graph one at a time in the order that they were 
added. 
Comment 12 Denys Duchier 2005-10-10 08:35:13 UTC
(In reply to comment #11) 
 
is this dep_zapdeps's doing? 
 
Maybe, instead of discarding dependencies, it should return a pair 
(deps_todo,deps_done) so that depgraph building can add appropriate 
edges for the deps_done without processing them again. 
 
... or have I completely misunderstood the problem? 
 
Comment 13 Jason Stubbs (RETIRED) gentoo-dev 2005-10-10 16:15:05 UTC
emerge, line 928. 
Comment 14 Denys Duchier 2005-10-11 03:51:36 UTC
(In reply to comment #13) 
> emerge, line 928.  
 
Interesting, but I don't think that explains the missing "Candidates". 
Does it? 
Comment 15 Jason Stubbs (RETIRED) gentoo-dev 2005-10-23 03:35:56 UTC
*** Bug 93795 has been marked as a duplicate of this bug. ***
Comment 16 Alec Warner (RETIRED) archtester gentoo-dev Security 2006-02-10 22:56:14 UTC
*** Bug 122432 has been marked as a duplicate of this bug. ***
Comment 17 kevin asher 2006-02-10 23:13:30 UTC
OK, so this is a known limitation.  Is there a known workaround?
Is there any kind of test to predict when an merge will fail due
to circular dependency? (emerge -p won't tell you because it
doesn't actually invoke the ebuild)

How do the devs get from stage2 to stage3, anyway, given this
limitation?  I have trouble imagining devs tweaking package.mask
every time they want to build a stage3 tarball. There's *got* to
be a workaround!
Comment 18 Oldrich Jedlicka 2006-07-10 02:26:14 UTC
I'm really starting to be desperate with this "feature" (and other features). I just now tried to update my system, I have the newest portage 2.1.1_pre2-r6 and I tried to update the system and stopped with java-config-1.2.11 update.

java-config depends (RDEPENDS) on java-config-wrapper and java-config-wrapper depends (DEPENDS) on java-config. Update could run perfectly, but...

# emerge -uav --oneshot =dev-java/java-config-1.3.0-r2
[blocks B     ] <dev-java/java-config-1.3 (is blocking dev-java/java-config-wrapper-0.9-r5)
[ebuild  N    ] dev-java/java-config-wrapper-0.9-r5  4 kB
[ebuild     U ] dev-java/java-config-1.3.0-r2 [1.2.11-r1] 13 kB

or when I start with --tree

# emerge -uav --tree --oneshot =dev-java/java-config-1.3.0-r2
[blocks B     ] <dev-java/java-config-1.3 (is blocking dev-java/java-config-wrapper-0.9-r5)
[ebuild     U ] dev-java/java-config-1.3.0-r2 [1.2.11-r1] 13 kB
[ebuild  N    ]  dev-java/java-config-wrapper-0.9-r5  4 kB

The order is different with tree and without tree. The update without the specified version selects java-config-2*, but it does not update java-config-1*. "Deep" update of world selects both versions, but in wrong order (as for emerge without --tree):

# emerge -uav --deep world
[blocks B     ] <dev-java/java-config-1.3 (is blocking dev-java/java-config-wrapper-0.9-r5)
...
[ebuild  N    ] dev-java/java-config-wrapper-0.9-r5  4 kB
[ebuild  NS   ] dev-java/java-config-2.0.26-r3  14 kB
[ebuild     U ] dev-java/java-config-1.3.0-r2 [1.2.11-r1] 13 kB

Heh? I really do not want to solve those problems by hand again and again every year...
Comment 19 Zac Medico gentoo-dev 2006-07-25 20:35:53 UTC
(In reply to comment #14)
> I don't think that explains the missing "Candidates". 

Those missing "Candidates" are now added to the digraph in >=portage-2.1.1_pre3-r5 (see bug 126748).  It still isn't able to automatically negotiate circular dependencies but at least they should be reported correctly so that a bug can be filed.
Comment 20 Zac Medico gentoo-dev 2006-08-20 21:24:25 UTC
(In reply to comment #19)
> Those missing "Candidates" are now added to the digraph

Well, more of them, anyway.  As shown by bug 144527, there are still more missing candidates.  I'm thinking about adding a boolean parameter to dep_check and dep_zapdeps so that dep_check will be able to return all of the selected atoms, even the ones that are already satisfied.  That will allow us to build a complete digraph and detect circular deps that currently go undetected. 
Comment 21 Jason Stubbs (RETIRED) gentoo-dev 2006-09-18 07:04:01 UTC
Created attachment 97335 [details, diff]
Ignore blockers on packages about to be updated

Processes blockers after the dep graph has been built and removes any blockers that only apply to packages that are about to be updated. This fixes the scenario listed in comment #18.

This patch applies on top of the patches from bug #147766
Comment 22 Zac Medico gentoo-dev 2006-09-20 15:21:48 UTC
(In reply to comment #21)
> Ignore blockers on packages about to be updated

This patch is in svn r4486.  It does leave the system in a slightly inconsistent state for a period of time.  I think it's well worth it though, considering that is solves a common case of bug 79606 in a very logical way.  If the system is left in an inconsistent state, the user will eventually receive a complaint from emerge if necessary.

Comment 23 Zac Medico gentoo-dev 2006-09-22 19:27:17 UTC
This bug is solved by the patches from bug 147766 which have been released in 2.1.2_pre1-r1.