Bugzilla Bug 16365

This problem does not show all the time. I have seen through various versions
of
portage but not my current version, which is 1 day old. Ok the problem that I
have seen is taht when doing "emerge -u? system, world , gnome, or kde about 1
out three times I will get an error saying sefault file.. When I restart emerge
(up arrow enter) instead trying to rebuild the last package, indecting that it
did segfault it starts emergeing a different package. Ex: emerge segfault on
gl-updates, there was nothing in the first 3 dirs it made to make. quit with
segfault error I restared and it emerged ttmkdir next and about 3 pkges latter
did gl-update with no probs. This only seems to occur when there are akot of
files to emerge. It could also be related to bug# 1661 where doing parallel
emerge -uf world and emerge -u world can get in to trouble. This happened when
gnome 2.2 was relessed. I started emerge -f gnome and had down loaded 20 files,
Figureing that was a good enough lead I stared emerge -ub gonome in vc4 and
about ten packages in, emerge -ub started downloading instead of just building.
checking it out, both emerges were downloading the same package at the same
time
& one stoping screwed the other. The emerge -f had 40 to 43 pkgs downloaded by
that time.

ANy way the emerge out step has occoured many times, and  99% of them I wasnt
during parrallel build as with the above example, single instance of emerge
running. 

Reproducible: Sometimes
Steps to Reproduce:
1 make a new system from gentoo-grp cd.and the load the grp pckges 
2.emerge -u world 
3.I havent tried this but rac & kanapus got my dander up so if I get a new hd
installed today I can try it latter this wk. This happened after 




Portage 2.0.46-r12 (default-x86-1.4, gcc-3.2.2, glibc-2.3.1-r2)
=================================================================
System uname: 2.4.20-gentoo-r1 i686 AMD Athlon(tm) XP 1800+
GENTOO_MIRRORS="ftp://mirror.iawnet.sandia.gov/pub/gentoo
http://gentoo.oregonstate.edu/
http://distro.ibiblio.org/pub/Linux/distributions/gentoo"
CONFIG_PROTECT="/etc /var/qmail/control /usr/kde/2/share/config
/usr/kde/3/share/config /usr/X11R6/lib/X11/xkb /usr/share/config"
CONFIG_PROTECT_MASK="/etc/gconf /etc/env.d"
PORTDIR="/home/portage"
DISTDIR="/home/portage/distfiles"
PKGDIR="/home/portage/packages"
PORTAGE_TMPDIR="/var/tmp"
PORTDIR_OVERLAY=""
USE="x86 oss 3dnow apm arts avi crypt cups encode gif jpeg kde libg++ libwww
mikmod mmx mpeg ncurses nls pdflib png qtmt quicktime spell truetype xml2 xmms
xv zlib gdbm berkdb slang readline svga tcltk java guile sdl gpm tcpd pam ssl
perl python esd imlib oggvorbis qt motif opengl mozilla cdr X gtk gnome alsa"
COMPILER="gcc3"
CHOST="i686-pc-linux-gnu"
CFLAGS="-march=athlon-xp -O2 -pipe"
CXXFLAGS="-march=athlon-xp -O2 -pipe"
ACCEPT_KEYWORDS="x86"
MAKEOPTS="-j2"
AUTOCLEAN="yes"
SYNC="rsync://rsync.gentoo.org/gentoo-portage"
FEATURES="sandbox ccache"

This just a question which about the troubles with dependencies. The same prob
that catalyst should solve. SO close this sucker!

Have a great New Years

hielvc

Still does ;^) but not quit as often,those nasty dependence trees.

*** Bug 11272 has been marked as a duplicate of this bug. ***

*** Bug 16538 has been marked as a duplicate of this bug. ***

*** Bug 37453 has been marked as a duplicate of this bug. ***

*** Bug 41019 has been marked as a duplicate of this bug. ***

*** Bug 44632 has been marked as a duplicate of this bug. ***

*** Bug 108664 has been marked as a duplicate of this bug. ***

(In reply to comment #8)  
> *** Bug 108664 has been marked as a duplicate of this bug. ***  
  
May I ask what the difficulty is in _noticing_ a circularity in the  
dependency graph and informing the user of it? 
 

I noticed something that looks odd in the computation on the depgraph: 
 
% emerge -e cyrus-sasl -pv --debug 
 
gives me (among a lot of output): 
 
Parent:    ebuild / net-nds/openldap-2.2.28-r1 merge 
Depstring: >=sys-libs/ncurses-5.1 tcpd? ( >=sys-apps/tcp-wrappers-7.6 ) ssl? 
( >=dev-libs/openssl-0.9.6 ) readline? ( >=sys-libs/readline-4.1 ) sasl? 
( >=dev-libs/cyrus-sasl-2.1.7-r3 ) odbc? ( dev-db/unixODBC ) slp? 
( >=net-libs/openslp-1.0 ) perl? ( >=dev-lang/perl-5.6 ) samba? 
( >=dev-libs/openssl-0.9.6 ) kerberos? ( virtual/krb5 ) berkdb? 
( >=sys-libs/db-4.2.52_p1 ) !berkdb? ( gdbm? ( >=sys-libs/gdbm-1.8.0 ) !gdbm? 
( >=sys-libs/db-4.2.52_p1 ) ) >=sys-devel/libtool-1.5.18-r1 
>=sys-apps/sed-4 !bootstrap? ( sys-devel/patch ) !bootstrap? 
( sys-devel/patch ) >=sys-libs/ncurses-5.1 tcpd? 
( >=sys-apps/tcp-wrappers-7.6 ) ssl? ( >=dev-libs/openssl-0.9.6 ) readline? 
( >=sys-libs/readline-4.1 ) sasl? ( >=dev-libs/cyrus-sasl-2.1.7-r3 ) odbc? 
( dev-db/unixODBC ) slp? ( >=net-libs/openslp-1.0 ) perl? 
( >=dev-lang/perl-5.6 ) samba? ( >=dev-libs/openssl-0.9.6 ) kerberos? 
( virtual/krb5 ) berkdb? ( >=sys-libs/db-4.2.52_p1 ) !berkdb? ( gdbm? 
( >=sys-libs/gdbm-1.8.0 ) !gdbm? ( >=sys-libs/db-4.2.52_p1 ) ) 
Candidates: ['>=sys-libs/readline-4.1', '>=dev-libs/openssl-0.9.6', 
'dev-db/unixODBC', '>=net-libs/openslp-1.0', '>=sys-devel/libtool-1.5.18-r1', 
'>=sys-apps/tcp-wrappers-7.6'] 
 
notice how cyrus-sasl is missing from the "Candidates" 
 
while: 
 
% emerge -e openldap -pv --debug 
 
gives me: 
 
Parent:    ebuild / net-nds/openldap-2.2.28-r1 merge 
Depstring: >=sys-libs/ncurses-5.1 tcpd? ( >=sys-apps/tcp-wrappers-7.6 ) ssl? 
( >=dev-libs/openssl-0.9.6 ) readline? ( >=sys-libs/readline-4.1 ) sasl? 
( >=dev-libs/cyrus-sasl-2.1.7-r3 ) odbc? ( dev-db/unixODBC ) slp? 
( >=net-libs/openslp-1.0 ) perl? ( >=dev-lang/perl-5.6 ) samba? 
( >=dev-libs/openssl-0.9.6 ) kerberos? ( virtual/krb5 ) berkdb? 
( >=sys-libs/db-4.2.52_p1 ) !berkdb? ( gdbm? ( >=sys-libs/gdbm-1.8.0 ) !gdbm? 
( >=sys-libs/db-4.2.52_p1 ) ) >=sys-devel/libtool-1.5.18-r1 
>=sys-apps/sed-4 !bootstrap? ( sys-devel/patch ) !bootstrap? 
( sys-devel/patch ) >=sys-libs/ncurses-5.1 tcpd? 
( >=sys-apps/tcp-wrappers-7.6 ) ssl? ( >=dev-libs/openssl-0.9.6 ) readline? 
( >=sys-libs/readline-4.1 ) sasl? ( >=dev-libs/cyrus-sasl-2.1.7-r3 ) odbc? 
( dev-db/unixODBC ) slp? ( >=net-libs/openslp-1.0 ) perl? 
( >=dev-lang/perl-5.6 ) samba? ( >=dev-libs/openssl-0.9.6 ) kerberos? 
( virtual/krb5 ) berkdb? ( >=sys-libs/db-4.2.52_p1 ) !berkdb? ( gdbm? 
( >=sys-libs/gdbm-1.8.0 ) !gdbm? ( >=sys-libs/db-4.2.52_p1 ) ) 
Candidates: ['>=sys-apps/tcp-wrappers-7.6', '>=dev-lang/perl-5.6', 
'>=dev-libs/openssl-0.9.6', 'dev-db/unixODBC', '>=sys-libs/db-4.2.52_p1', 
'>=net-libs/openslp-1.0', '>=sys-devel/libtool-1.5.18-r1', 'sys-devel/patch', 
'>=sys-libs/ncurses-5.1', '>=sys-libs/readline-4.1', '>=sys-apps/sed-4', 
'>=dev-libs/cyrus-sasl-2.1.7-r3'] 
 
where it correctly appears. 
 
I have the sinking feeling that emerge does not actually build a correct 
and complete depgraph :-/ 
 
Can anyone enlighten me as to what the core problem is? 

The core problem is that emerge does not actually build a correct and complete 
depgraph. ;) 
 
Instead it assumes that any package that already exists in the graph has had 
its dependencies taken care of. Based on this (incorrect) assumption, it then 
pulls leaf packages from the graph one at a time in the order that they were 
added. 

(In reply to comment #11) 
 
is this dep_zapdeps's doing? 
 
Maybe, instead of discarding dependencies, it should return a pair 
(deps_todo,deps_done) so that depgraph building can add appropriate 
edges for the deps_done without processing them again. 
 
... or have I completely misunderstood the problem? 
 

emerge, line 928. 

(In reply to comment #13) 
> emerge, line 928.  
 
Interesting, but I don't think that explains the missing "Candidates". 
Does it? 

*** Bug 93795 has been marked as a duplicate of this bug. ***

*** Bug 122432 has been marked as a duplicate of this bug. ***

OK, so this is a known limitation.  Is there a known workaround?
Is there any kind of test to predict when an merge will fail due
to circular dependency? (emerge -p won't tell you because it
doesn't actually invoke the ebuild)

How do the devs get from stage2 to stage3, anyway, given this
limitation?  I have trouble imagining devs tweaking package.mask
every time they want to build a stage3 tarball. There's *got* to
be a workaround!

I'm really starting to be desperate with this "feature" (and other features). I
just now tried to update my system, I have the newest portage 2.1.1_pre2-r6 and
I tried to update the system and stopped with java-config-1.2.11 update.

java-config depends (RDEPENDS) on java-config-wrapper and java-config-wrapper
depends (DEPENDS) on java-config. Update could run perfectly, but...

# emerge -uav --oneshot =dev-java/java-config-1.3.0-r2
[blocks B     ] <dev-java/java-config-1.3 (is blocking
dev-java/java-config-wrapper-0.9-r5)
[ebuild  N    ] dev-java/java-config-wrapper-0.9-r5  4 kB
[ebuild     U ] dev-java/java-config-1.3.0-r2 [1.2.11-r1] 13 kB

or when I start with --tree

# emerge -uav --tree --oneshot =dev-java/java-config-1.3.0-r2
[blocks B     ] <dev-java/java-config-1.3 (is blocking
dev-java/java-config-wrapper-0.9-r5)
[ebuild     U ] dev-java/java-config-1.3.0-r2 [1.2.11-r1] 13 kB
[ebuild  N    ]  dev-java/java-config-wrapper-0.9-r5  4 kB

The order is different with tree and without tree. The update without the
specified version selects java-config-2*, but it does not update
java-config-1*. "Deep" update of world selects both versions, but in wrong
order (as for emerge without --tree):

# emerge -uav --deep world
[blocks B     ] <dev-java/java-config-1.3 (is blocking
dev-java/java-config-wrapper-0.9-r5)
...
[ebuild  N    ] dev-java/java-config-wrapper-0.9-r5  4 kB
[ebuild  NS   ] dev-java/java-config-2.0.26-r3  14 kB
[ebuild     U ] dev-java/java-config-1.3.0-r2 [1.2.11-r1] 13 kB

Heh? I really do not want to solve those problems by hand again and again every
year...

(In reply to comment #14)
> I don't think that explains the missing "Candidates". 

Those missing "Candidates" are now added to the digraph in
>=portage-2.1.1_pre3-r5 (see bug 126748).  It still isn't able to automatically
negotiate circular dependencies but at least they should be reported correctly
so that a bug can be filed.

(In reply to comment #19)
> Those missing "Candidates" are now added to the digraph

Well, more of them, anyway.  As shown by bug 144527, there are still more
missing candidates.  I'm thinking about adding a boolean parameter to dep_check
and dep_zapdeps so that dep_check will be able to return all of the selected
atoms, even the ones that are already satisfied.  That will allow us to build a
complete digraph and detect circular deps that currently go undetected. 

Created an attachment (id=97335) [details]
Ignore blockers on packages about to be updated

Processes blockers after the dep graph has been built and removes any blockers
that only apply to packages that are about to be updated. This fixes the
scenario listed in comment #18.

This patch applies on top of the patches from bug #147766

(In reply to comment #21)
> Ignore blockers on packages about to be updated

This patch is in svn r4486.  It does leave the system in a slightly
inconsistent state for a period of time.  I think it's well worth it though,
considering that is solves a common case of bug 79606 in a very logical way. 
If the system is left in an inconsistent state, the user will eventually
receive a complaint from emerge if necessary.

This bug is solved by the patches from bug 147766 which have been released in
2.1.2_pre1-r1.