Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 374397 - portage goes into endless loop when trying to emerge some packages
Summary: portage goes into endless loop when trying to emerge some packages
Status: RESOLVED FIXED
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: [OLD] Core system (show other bugs)
Hardware: All Linux
: Normal blocker (vote)
Assignee: Portage team
URL:
Whiteboard:
Keywords: InVCS
Depends on:
Blocks: 373933
  Show dependency tree
 
Reported: 2011-07-07 22:22 UTC by Piergiorgio Beruto
Modified: 2015-08-03 01:07 UTC (History)
0 users

See Also:
Package list:
Runtime testing required: ---


Attachments
debug log (debug.log,513.64 KB, text/plain)
2011-07-09 21:04 UTC, Piergiorgio Beruto
Details
output circular dependency debug graph earlier (circ_debug.patch,2.99 KB, patch)
2011-07-09 22:59 UTC, Zac Medico
Details | Diff
new debug log (debug.log,514.54 KB, text/plain)
2011-07-09 23:18 UTC, Piergiorgio Beruto
Details
fix exponential loop triggered by REQUIRED_USE in php ebuild (circ_fix.patch,1.66 KB, patch)
2011-07-10 01:34 UTC, Zac Medico
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Piergiorgio Beruto 2011-07-07 22:22:44 UTC
After updating to latest portage, emerge goes into endless loop when trying to emerge some packages, like syslog-ng.
Also happens if I try to run "perl-cleaner --all".
Happened on gentoo x86 virtualbox.
Also happens when trying to install a brand new x86_64 (amd64) system following the guide; everything halts at trying to emerge syslog-ng (same issue, CPU is @100%, last line being printed is "Calculating dependencies... done!").
Trying to emerge --debug yields no more prints after that one.
Also happens with --pretend option.

Reproducible: Always

Steps to Reproduce:
1. upgrade to portage 2.1.10.4
2. emerge --pretend -e world
Actual Results:  
emerge halts after printing "Calculating dependencies... done!" then takes 100% of the cpu

Expected Results:  
emerge prints what packages would be emerged

Portage 2.1.10.4 (default/linux/amd64/10.0/no-multilib, gcc-4.4.5, glibc-2.12.2-r0, 2.6.38-gentoo-r6 x86_64)
=================================================================
System uname: Linux-2.6.38-gentoo-r6-x86_64-Intel-R-_Core-TM-_i7_CPU_950_@_3.07GHz-with-gentoo-2.0.2
Timestamp of tree: Thu, 07 Jul 2011 20:15:01 +0000
app-shells/bash:          4.1_p9
dev-lang/python:          2.7.1-r1, 3.1.3-r1
sys-apps/baselayout:      2.0.2
sys-apps/openrc:          0.8.2-r1
sys-apps/sandbox:         2.4
sys-devel/autoconf:       2.65-r1
sys-devel/automake:       1.11.1
sys-devel/binutils:       2.20.1-r1
sys-devel/gcc:            4.4.5
sys-devel/gcc-config:     1.4.1-r1
sys-devel/libtool:        2.2.10
sys-devel/make:           3.82
sys-kernel/linux-headers: 2.6.36.1 (virtual/os-headers)
sys-libs/glibc:           2.12.2
Repositories: gentoo
ACCEPT_KEYWORDS="amd64 ~x86"
ACCEPT_LICENSE="* -@EULA"
CBUILD="x86_64-pc-linux-gnu"
CFLAGS="-march=native -O2 -pipe"
CHOST="x86_64-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/share/gnupg/qualified.txt"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/gconf /etc/gentoo-release /etc/sandbox.d /etc/terminfo"
CXXFLAGS="-march=native -O2 -pipe"
DISTDIR="/usr/portage/distfiles"
FEATURES="assume-digests binpkg-logs distlocks ebuild-locks fixlafiles fixpackages news parallel-fetch protect-owned sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch"
FFLAGS=""
GENTOO_MIRRORS="http://distfiles.gentoo.org"
LDFLAGS="-Wl,-O1 -Wl,--as-needed"
LINGUAS="en_US"
MAKEOPTS="-j3"
PKGDIR="/usr/portage/packages"
PORTAGE_CONFIGROOT="/"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --stats --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages"
PORTAGE_TMPDIR="/var/tmp"
PORTDIR="/usr/portage"
PORTDIR_OVERLAY=""
SYNC="rsync://shelob/gentoo-portage"
USE="64bit X acl acpi alsa amd64 apng bash-completion berkdb branding bzip2 cairo cli consolekit cracklib crypt cscope cups cxx dbus declarative device-mapper dga dri fbcon firefox fontconfig fortran gd gdbm gif git gpm graphviz gtk iconv java javascript jbig jpeg kde libnotify lzma mercurial mmap mmx mng modules mono mudflap mysql ncurses networkmanager nls nptl nptlonly nsplugin opengl openmp pam pcre pdf perl php png policykit pppd python qt3support qt4 readline semantic-desktop session sockets sql sse sse2 sse3 ssl ssse3 startup-notification subversion svg sysfs tcpd threads tiff truetype udev unicode usb vim vim-pager vim-syntax webkit wxwidgets xinetd xml xmlrpc xorg xpm xrandr zip zlib" ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87x ca0106 cmipci emu10k1x ens1370 ens1371 es1938 es1968 fm801 hda-intel intel8x0 intel8x0m maestro3 trident usb-audio via82xx via82xx-modem ymfpci" ALSA_PCM_PLUGINS="adpcm alaw asym copy dmix dshare dsnoop empty extplug file hooks iec958 ioplug ladspa lfloat linear meter mmap_emul mulaw multi null plug rate route share shm softvol" APACHE2_MODULES="actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="braindump flow karbon kexi kpresenter krita tables words" CAMERAS="ptp2" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf superstar2 timing tsip tripmate tnt ubx" INPUT_DEVICES="evdev virtualbox" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LINGUAS="en_US" PHP_TARGETS="php5-3" RUBY_TARGETS="ruby18" USERLAND="GNU" VIDEO_CARDS="virtualbox" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account" 
Unset:  CPPFLAGS, CTARGET, EMERGE_DEFAULT_OPTS, INSTALL_MASK, LANG, LC_ALL, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS
Comment 1 Zac Medico gentoo-dev 2011-07-08 18:08:11 UTC
When it's hung like that, you can Pdb to get a backtrace like this:

1) kill -s SIGUSR1 <emerge pid>
2) the signal should make emerge should go to a Pdb prompt, then you can use the 'bt' command to make it display a backtrace
Comment 2 Piergiorgio Beruto 2011-07-08 23:23:09 UTC
here is the output of the backtrace from "emerge --pretend syslog-ng:"

These are the packages that would be merged, in order:

Calculating dependencies  ... done!
--Return--
> /usr/bin/emerge(28)debug_signal()->None
-> pdb.set_trace()
(Pdb)   /usr/bin/emerge(43)<module>()
-> retval = emerge_main()
  /usr/lib64/portage/pym/_emerge/main.py(1907)emerge_main()
-> myopts, myaction, myfiles, spinner)
  /usr/lib64/portage/pym/_emerge/actions.py(302)action_build()
-> mydepgraph.display_problems()
  /usr/lib64/portage/pym/_emerge/depgraph.py(6013)display_problems()
-> self._display_problems()
  /usr/lib64/portage/pym/_emerge/depgraph.py(6026)_display_problems()
-> self._dynamic_config._circular_deps_for_display)
  /usr/lib64/portage/pym/_emerge/depgraph.py(5485)_show_circular_deps()
-> circular_dependency_handler(self, mygraph)
  /usr/lib64/portage/pym/_emerge/resolver/circular_dependency.py(28)__init__()
-> self.solutions, self.suggestions = self._find_suggestions()
  /usr/lib64/portage/pym/_emerge/resolver/circular_dependency.py(179)_find_suggestions()
-> uselist=current_use, flat=True)
  /usr/lib64/portage/pym/portage/dep/__init__.py(400)use_reduce()
-> if is_active(stack[level][-1]):
  /usr/lib64/portage/pym/portage/dep/__init__.py(311)is_active()
-> if conditional.startswith("!"):
> /usr/bin/emerge(28)debug_signal()->None
-> pdb.set_trace()
(Pdb) 

It seems to me that it found some circular deps but it goes into an endless loop and fails to show the result...
Comment 3 Zac Medico gentoo-dev 2011-07-09 00:56:52 UTC
I've reviewed the code for this loop. It's hard to read, but it seems like it's guaranteed to terminate. It might take a long time if you have a large cycle involving lots of USE flags though. Maybe it will eventually complete if you let it run a bit longer.
Comment 4 Zac Medico gentoo-dev 2011-07-09 01:15:59 UTC
I've copied the _next_use_state code into a separate file and it seems to work as intended. We can get the same result with python's cartesian product function:

  itertools.product(("enabled", "disabled"), repeat=len(affecting_use))
Comment 5 Piergiorgio Beruto 2011-07-09 14:45:15 UTC
Hi,
I've been running "emerge --pretend syslog-ng" for more than 45 minutes (@core i7 950) with still no results...

I didn't understand your last comment, is there anything I shall do to help you debug this issue?
Comment 6 Zac Medico gentoo-dev 2011-07-09 16:26:07 UTC
Please attach a log created like this:

  emerge --pretend --debug syslog-ng &> debug.log

You can run `tail -f debug.log` from another shell, and kill it when it appears to get stuck in a loop.
Comment 7 Piergiorgio Beruto 2011-07-09 21:04:10 UTC
Created attachment 279539 [details]
debug log

Here you are,
I've waited for some minutes before actually stopping the command.
The wierd thing is that nothing is printed after "done".

I can upload the (partially installed) virtualbox image if you wish.
Comment 8 Zac Medico gentoo-dev 2011-07-09 22:59:12 UTC
Created attachment 279549 [details, diff]
output circular dependency debug graph earlier

Save as /tmp/circ_debug.patch and apply as follows:

  cd /usr/lib/portage
  patch -p1 < /tmp/circ_debug.patch

Please apply this patch and then create a new log as described in comment #6. This will give us more info about the nature of the cycle that we are dealing with.
Comment 9 Piergiorgio Beruto 2011-07-09 23:18:58 UTC
Created attachment 279551 [details]
new debug log

here you are,
hope this helps!
Comment 10 Zac Medico gentoo-dev 2011-07-10 01:34:09 UTC
Created attachment 279557 [details, diff]
fix exponential loop triggered by REQUIRED_USE in php ebuild

Save as /tmp/circ_fix.patch and apply as follows:

 patch /usr/lib/portage/pym/_emerge/resolver/circular_dependency.py /tmp/circ_fix.patch
Comment 11 Piergiorgio Beruto 2011-07-10 21:31:46 UTC
Ok, that seems to work in both my virtual machines.
Thank you for your assistance.
Comment 12 Zac Medico gentoo-dev 2011-07-13 00:28:55 UTC
Thanks for testing. This is fixed in 2.1.10.5 and 2.2.0_alpha44.