This happens for the 1st time in a chroot image at the tinderbox, that # emerge -1p sys-cluster/nova did not finished. I killed the process after 11 hours and tried the same command at the command line - similar picture, but I did not wait so long. At my desktop system the command emerge calculated the deps within seconds, so probably related to the USE flag settings ? # cat /etc/portage/make.conf # These settings were set by the catalyst build script that automatically # built this stage. # Please consult /usr/share/portage/config/make.conf.example for a more # detailed example. CFLAGS="-march=native -O2 -pipe" CXXFLAGS="${CFLAGS}" # WARNING: Changing your CHOST is not something that should be done lightly. # Please consult http://www.gentoo.org/doc/en/change-chost.xml before changing. CHOST="x86_64-pc-linux-gnu" # These are the USE flags that were used in addition to what is provided by the # profile used for building. #USE="bindist mmx sse sse2" PORTDIR="/usr/portage" DISTDIR="/var/tmp/distfiles" PKGDIR="${PORTDIR}/packages" USE="aes-ni apache2 btrfs cairo cgi corefonts cups dbus designer dnssec dot drmkms dvd ecc extraengine fpm gudev gui icu jadetex lapack libkms logrotate mbox minizip openssl pax_kernel pcre16 plasma policykit python qt3support qml spice avx avx2 mmx sse sse2 sse4 sse4_1 sse4_2 ssse3 uml usb usbredir xkb xslt xvfb xvmc zenmap -bindist -hpn -ldap -offensive -semantic-desktop fortran qt5 pkcs11 introspection threads tls scrpyt -ssh-askpass -pwquality -nss smartcard -ipv6 sockets gtk svg inifile pdo curl -sqlite -sqlite3 mysql -mysqli bzip2 -xz -xscreensaver -fontconfig widgets gd gif -jpeg png xmp uxa -v4l -v4l2 -video theora -xa opencv ogg ffmpeg opengl -sdl kvm -qemu libvirtd tcl tk imap alsa pulseaudio " CPU_FLAGS_X86="aes avx mmx mmxext popcnt sse sse2 sse3 sse4_1 sse4_2 ssse3" LINGUAS="en en_GB" ACCEPT_KEYWORDS="amd64 ~amd64" ACCEPT_LICENSE="*" EMERGE_DEFAULT_OPTS=" --accept-properties=-interactive --accept-restrict=-fetch --nospinner --tree --quiet-build" # do not use "fail-clean" here, it would delete the log files we want to send out # in planning: "test" # FEATURES="preserve-libs parallel-fetch ipc-sandbox network-sandbox" PORT_LOGDIR="/var/log/portage" PORTAGE_ELOG_CLASSES="qa error" PORTAGE_ELOG_SYSTEM="save" PORTAGE_ELOG_MAILURI="tinderbox@zwiebeltoralf.de www325.your-server.de:100025" # elog portage needs this ? PORTAGE_ELOG_MAILFROM="amd64-kde-unstable_20150716-153903 <tinderbox@zwiebeltoralf.de>" # From is lost with ssmtp, but nevertheless ... MAKEOPTS="-j1" GENTOO_MIRRORS="http://ftp.uni-erlangen.de/pub/mirrors/gentoo rsync://mirror.netcologne.de/gentoo/ ftp://sunsite.informatik.rwth-aachen.de/pub/Linux/gor.bytemark.co.uk/gentoo/ rsync://ftp.snt.utwente.nl/gentoo"
# emerge --info Portage 2.2.20 (python 2.7.10-final-0, default/linux/amd64/13.0/desktop/kde, gcc-4.9.3, glibc-2.21-r1, 4.0.8-hardened x86_64) ================================================================= System uname: Linux-4.0.8-hardened-x86_64-Intel-R-_Core-TM-_i7-3770_CPU_@_3.40GHz-with-gentoo-2.2 KiB Mem: 16164692 total, 2764448 free KiB Swap: 0 total, 0 free Timestamp of repository gentoo: Thu, 23 Jul 2015 09:45:01 +0000 sh bash 4.3_p39 ld GNU ld (Gentoo 2.25 p1.2) 2.25 app-shells/bash: 4.3_p39::gentoo dev-java/java-config: 2.2.0::gentoo dev-lang/perl: 5.22.0::gentoo dev-lang/python: 2.7.10::gentoo, 3.3.5-r1::gentoo dev-util/cmake: 3.2.3::gentoo dev-util/pkgconfig: 0.28-r3::gentoo sys-apps/baselayout: 2.2::gentoo sys-apps/openrc: 0.17::gentoo sys-apps/sandbox: 2.6-r1::gentoo sys-devel/autoconf: 2.13::gentoo, 2.69-r1::gentoo sys-devel/automake: 1.9.6-r4::gentoo, 1.10.3-r1::gentoo, 1.11.6-r1::gentoo, 1.12.6::gentoo, 1.13.4::gentoo, 1.14.1::gentoo, 1.15::gentoo sys-devel/binutils: 2.25-r1::gentoo sys-devel/gcc: 4.8.4::gentoo, 4.9.3::gentoo sys-devel/gcc-config: 1.8::gentoo sys-devel/libtool: 2.4.6-r1::gentoo sys-devel/make: 4.1-r1::gentoo sys-kernel/linux-headers: 4.1::gentoo (virtual/os-headers) sys-libs/glibc: 2.21-r1::gentoo Repositories: local location: /usr/local/portage masters: gentoo gentoo location: /usr/portage sync-type: rsync sync-uri: rsync://rsync.gentoo.org/gentoo-portage priority: 9999 ACCEPT_KEYWORDS="amd64 ~amd64" ACCEPT_LICENSE="*" CBUILD="x86_64-pc-linux-gnu" CFLAGS="-march=native -O2 -pipe" CHOST="x86_64-pc-linux-gnu" CONFIG_PROTECT="/etc /usr/lib64/tomoyo/conf /usr/share/config /usr/share/easy-rsa /usr/share/gnupg/qualified.txt /usr/share/maven-bin-3.2/conf /var/lib/hsqldb /var/spool/torque" CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/dconf /etc/env.d /etc/fonts/fonts.conf /etc/games/angband/edit/ /etc/gconf /etc/gentoo-release /etc/php/apache2-php5.5/ext-active/ /etc/php/apache2-php5.6/ext-active/ /etc/php/cgi-php5.5/ext-active/ /etc/php/cgi-php5.6/ext-active/ /etc/php/cli-php5.5/ext-active/ /etc/php/cli-php5.6/ext-active/ /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo /etc/texmf/language.dat.d /etc/texmf/language.def.d /etc/texmf/updmap.d /etc/texmf/web2c" CXXFLAGS="-march=native -O2 -pipe" DISTDIR="/var/tmp/distfiles" EMERGE_DEFAULT_OPTS=" --accept-properties=-interactive --accept-restrict=-fetch --nospinner --tree --quiet-build" FCFLAGS="-O2 -pipe" FEATURES="assume-digests binpkg-logs config-protect-if-modified distlocks ebuild-locks fixlafiles ipc-sandbox merge-sync network-sandbox news parallel-fetch preserve-libs protect-owned sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync" FFLAGS="-O2 -pipe" GENTOO_MIRRORS="http://ftp.uni-erlangen.de/pub/mirrors/gentoo rsync://mirror.netcologne.de/gentoo/ ftp://sunsite.informatik.rwth-aachen.de/pub/Linux/gor.bytemark.co.uk/gentoo/ rsync://ftp.snt.utwente.nl/gentoo" LANG="en_US.utf8" LDFLAGS="-Wl,-O1 -Wl,--as-needed" MAKEOPTS="-j1" PKGDIR="/usr/portage/packages" PORTAGE_CONFIGROOT="/" PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages" PORTAGE_TMPDIR="/var/tmp" USE="X a52 aac acl acpi aes-ni alsa amd64 apache2 avx avx2 berkdb bluetooth branding btrfs bzip2 cairo cdda cdr cgi cli consolekit corefonts cracklib crypt cups curl cxx dbus declarative designer dnssec dot dri drmkms dts dvd dvdr ecc emboss encode exif extraengine fam ffmpeg firefox flac fortran fpm gd gdbm gif glamor gpm gtk gudev gui iconv icu imap inifile introspection jadetex kde kipi kvm lapack lcms libkms libnotify libvirtd logrotate mad mbox minizip mmx mmxext mng modules mp3 mp4 mpeg multilib mysql ncurses nls nptl ogg opencv opengl openmp openssl pam pango pax_kernel pcre pcre16 pdf pdo phonon pkcs11 plasma png policykit ppds pulseaudio python qml qt3support qt4 qt5 readline scrpyt session smartcard sockets spell spice sse sse2 sse4 sse4_1 sse4_2 ssl ssse3 startup-notification svg tcl tcpd theora threads tiff tk tls truetype udev udisks uml unicode upower usb usbredir uxa vorbis widgets wxwidgets x264 xcb xcomposite xinerama xkb xml xmp xslt xv xvfb xvid xvmc zenmap zlib" ABI_X86="64" ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87x ca0106 cmipci emu10k1x ens1370 ens1371 es1938 es1968 fm801 hda-intel intel8x0 intel8x0m maestro3 trident usb-audio via82xx via82xx-modem ymfpci" APACHE2_MODULES="authn_core authz_core socache_shmcb unixd actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="kexi words flow plan sheets stage tables krita karbon braindump author" CAMERAS="ptp2" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" CPU_FLAGS_X86="aes avx mmx mmxext popcnt sse sse2 sse3 sse4_1 sse4_2 ssse3" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf superstar2 timing tsip tripmate tnt ublox ubx" INPUT_DEVICES="keyboard mouse evdev" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" LINGUAS="en en_GB" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php5-5" PYTHON_SINGLE_TARGET="python2_7" PYTHON_TARGETS="python2_7 python3_3" RUBY_TARGETS="ruby19 ruby20" USERLAND="GNU" VIDEO_CARDS="fbdev glint intel mach64 mga nouveau nv r128 radeon savage sis tdfx trident vesa via vmware dummy v4l" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account" Unset: CPPFLAGS, CTARGET, INSTALL_MASK, LC_ALL, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS, USE_PYTHON
happened at another chroot image (~amd64 desktop profile). Given the USE flag set below I had to remove "-qt4 qt5" from it - otherwise portage hangs infinitly at # emerge -1p qemu These are the packages that would be merged, in order: Calculating dependencies... done! ^C Exiting on signal 2 This works now: USE="aes-ni apache2 btrfs cairo cgi corefonts cups dbus designer dnssec dot drmkms ecc extraengine fpm gui icu jadetex lapack libkms logrotate mbox minizip pax_kernel pcre16 plasma policykit qml spice sqlite avx avx2 mmx sse sse2 sse4 sse4_1 sse4_2 ssse3 uml usb usbredir xkb xslt xvfb xvmc zenmap -bindist -hpn -ldap -offensive -semantic-desktop smartcard ipv6 -sockets -qt3support inifile -dvd bzip2 xz xscreensaver png -uxa video theora ffmpeg opengl -libvirtd imap -freetds mysql sqlite3 fortran -java -gudev -systemd " #-qt4 qt5 Adding either qt4 or qt5 and emerge doesn't do continue
Created attachment 407534 [details] amd64-desktop-unstable_20150723-224316 package list And because I do have other chroot images at that tinderbox where "qt4 qt5" is a valid pair in USE flags I attach here the package history of the image - I think, it rules.
I'm not sure this is an error in nova itself. At least the basic image I test on works with this... perhaps it's ipython or cmake (which are being pulled in)? USE="qt4 qt5 compute-only -memcached -rabbitmq" emerge -1p nova | grep qt [ebuild N ] dev-qt/qtchooser-0_p20150102 USE="(-qt5) {-test}" [ebuild N ] app-eselect/eselect-qtgraphicssystem-1.1.1 [ebuild N ] dev-qt/qtcore-4.8.6-r2 USE="exceptions glib iconv ssl (-aqua) -debug -icu -pch -qt3support" ABI_X86="32 (64) (-x32)" [ebuild N ] dev-qt/qttranslations-4.8.6-r1 ABI_X86="32 (64) (-x32)" [ebuild N ] dev-qt/qtscript-4.8.6-r2 USE="exceptions jit (-aqua) -debug -pch" ABI_X86="32 (64) (-x32)" [ebuild N ] dev-python/ipython-2.2.0-r1 USE="qt4 smp -doc -examples -matplotlib -mongodb -nbconvert -notebook -octave {-test} -wxwidgets" PYTHON_TARGETS="python2_7 python3_4 -python3_3" [ebuild N ] dev-qt/qtgui-4.8.6-r4 USE="accessibility exceptions glib xv (-aqua) -cups -debug -egl -gtkstyle -mng -nas -nis -pch -qt3support -tiff -trace -xinerama" ABI_X86="32 (64) (-x32)" [ebuild N ] dev-util/cmake-3.2.2 USE="ncurses qt4 -doc -emacs (-qt5) -system-jsoncpp {-test}"
also, this sounds like a portage bug (-qt4 qt5 resolves to this for me) USE="-qt4 qt5 compute-only -memcached -rabbitmq" emerge -1p nova | grep qt [ebuild N ] dev-util/cmake-3.2.2 USE="ncurses -doc -emacs -qt4 (-qt5) -system-jsoncpp {-test}" [ebuild N ] dev-python/ipython-2.2.0-r1 USE="smp -doc -examples -matplotlib -mongodb -nbconvert -notebook -octave -qt4 {-test} -wxwidgets" PYTHON_TARGETS="python2_7 python3_4 -python3_3"
It happens today at 3 new freshly installed chroot images too, eg : # emerge -1up app-emulation/ganeti-instance-debootstrap These are the packages that would be merged, in reverse order: Calculating dependencies ... done! shows no progress after x hours. However just tweaking qt4/qt5 USE flags doesn't helped at the first image I tested. Is there a debug output of portage I could force ? -> assigned to portage
(In reply to Toralf Förster from comment #6) > Is there a debug output of portage I could force ? You can use the --debug option. However, we can get more precise debugging information from a backtrace created with the debugger. In order to do this, use `kill -s SIGUSR1 <pid>` to send a SIGUSER1 to the emerge process. Then emerge should produce a pdb prompt. At the emerge prompt, type 'bt' and then press enter. Please post the backtrace that it produces.
Created attachment 408154 [details] debug output (In reply to Zac Medico from comment #7) > (In reply to Toralf Förster from comment #6) > > Is there a debug output of portage I could force ? > > You can use the --debug option. ok, an emerge -1 --debug -p app-emulation/ganeti-instance-debootstrap 2>&1 | tee /tmp/debug.log at the image volumes/amd64-13.0-stable_20150731-123855/ gives the attached output, ending with : ... done! circular dependency graph: (app-emulation/spice-0.12.5-r1:0/0::gentoo, ebuild scheduled for merge) depends on (app-emulation/qemu-2.3.0-r4:0/0::gentoo, ebuild scheduled for merge) (buildtime) (app-emulation/qemu-2.3.0-r4:0/0::gentoo, ebuild scheduled for merge) depends on (app-emulation/spice-0.12.5-r1:0/0::gentoo, ebuild scheduled for merge) (buildtime)
(In reply to Toralf Förster from comment #8) > circular dependency graph: > > (app-emulation/spice-0.12.5-r1:0/0::gentoo, ebuild scheduled for merge) > depends on > (app-emulation/qemu-2.3.0-r4:0/0::gentoo, ebuild scheduled for merge) > (buildtime) > (app-emulation/qemu-2.3.0-r4:0/0::gentoo, ebuild scheduled for merge) > depends on > (app-emulation/spice-0.12.5-r1:0/0::gentoo, ebuild scheduled for merge) > (buildtime) The circular dependency solver is going into an infinite loop on this, which I was able to reproduce and create a backtrace: (Pdb) bt /usr/lib/python-exec/python3.4/emerge(50)<module>() -> retval = emerge_main() /usr/lib64/python3.4/site-packages/_emerge/main.py(1155)emerge_main() -> return run_action(emerge_config) /usr/lib64/python3.4/site-packages/_emerge/actions.py(3191)run_action() -> emerge_config.args, spinner) /usr/lib64/python3.4/site-packages/_emerge/actions.py(328)action_build() -> mydepgraph.display_problems() /usr/lib64/python3.4/site-packages/_emerge/depgraph.py(8301)display_problems() -> self._dynamic_config._circular_deps_for_display) /usr/lib64/python3.4/site-packages/_emerge/depgraph.py(7732)_show_circular_deps() -> circular_dependency_handler(self, mygraph) /usr/lib64/python3.4/site-packages/_emerge/resolver/circular_dependency.py(39)__init__() -> self.solutions, self.suggestions = self._find_suggestions() /usr/lib64/python3.4/site-packages/_emerge/resolver/circular_dependency.py(181)_find_suggestions() -> uselist=current_use, flat=True) /usr/lib64/python3.4/site-packages/portage/dep/__init__.py(552)use_reduce() -> if is_active(stack[level][-1]): /usr/lib64/python3.4/site-packages/portage/dep/__init__.py(478)is_active() -> if useflag_re.match(flag) is None:
The circular dependency that triggers this is as follows: app-emulation/qemu-2.3.0-r4: DEPEND: spice? ( >=app-emulation/spice-0.12.0[static-libs(+)] ) app-emulation/spice-0.12.5-r1: DEPEND: client? ( smartcard? ( app-emulation/qemu[smartcard] ) )
This is the loop where it hangs in pym/_emerge/resolver/circular_dependency.py: for use_state in product(("disabled", "enabled"), repeat=len(affecting_use)): It uses itertools.product(), which can result in an extremely long loop.
The qemu ebuild repeats the spice dependency once for each of 28 different softmmu targets. This results in the extract_affecting_use function returning 29 USE flags (spice + 28 softmmu targets). When itertools.product is then called with repeat=29, it results in 2 ^ 29 = 536,870,912 combinations. Looping over these 536,870,912 combinations is what makes emerge appear to hang indefinitely. An obvious optimization would be to make the product exclude softmmu targets that the user has not enabled.
There's a patch in the following branch: https://github.com/zmedico/portage/tree/bug_555698 Thee resulting circular dependency solver output looks like this with QEMU_SOFTMMU_TARGETS="i386 x86_64" enabled: It might be possible to break this cycle by applying any of the following changes: - app-emulation/spice-0.12.5-r1 (Change USE: -smartcard) - app-emulation/qemu-2.3.0-r4 (Change USE: -qemu_softmmu_targets_i386 -qemu_softmmu_targets_x86_64) - app-emulation/qemu-2.3.0-r4 (Change USE: -spice) You can test it like this: echo '=sys-apps/portage-9999 **' >> /etc/portage/package.accept_keywords portage_LIVE_BRANCH=bug_555698 \ portage_LIVE_REPO=https://github.com/zmedico/portage.git \ emerge -1 =sys-apps/portage-9999 I've posted it for review here: https://archives.gentoo.org/gentoo-portage-dev/message/47b1553a8d73a28d7ca258b719939631
tested and solved the issue at that image
Thanks for testing. This is in the master branch now: https://gitweb.gentoo.org/proj/portage.git/commit/?id=024e2a5779e92fffbd0a0d30c8cff725c4552626
Released in portage-2.2.21
(In reply to Brian Dolbec from comment #16) > Released in portage-2.2.21 At least at my tinderbox I do run often into this issue for stable images - therefore me wonders if it would make sense to either back port this to a 2.2.20.2 release too -or- to stabilize one of the unstable versions ?
we're going to stabilize one of the newer versions. In fact it's today that lets us call for 2.2.24 stabilization.
