Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!

Bug 481628

Summary: sys-apps/portage: autounmask should solve SLOT conflicts involving USE deps solvable by USE adjustments
Product: Portage Development Reporter: Hendrik v. Raven <hendrik>
Component: Core - DependenciesAssignee: Portage team <dev-portage>
Status: CONFIRMED ---    
Severity: normal CC: boxcars, esigra, hendrik
Priority: Normal    
Version: 2.2   
Hardware: All   
OS: Linux   
Whiteboard:
Package list:
Runtime testing required: ---
Bug Depends on:    
Bug Blocks: 300071, 376695    
Attachments: output with portage-9999
output with portage 2.2.0
output with portage 2.1.12.2 (stable)

Description Hendrik v. Raven 2013-08-19 11:51:37 UTC
Created attachment 356412 [details]
output with portage-9999

portage 2.2.0 and portage nightly (commit: a500fbdae1098424e64356f8a24ac3c25bbff908) fail to calculate the required changes in package.use and instead prints a huge mass of conflicting dependencies (see logs). portage 2.1.12.2 prints a correct set of USE changes.

I removed all dependency use changes from package.use and let portage recalculate them, to remove unneeded. With the recent version of portage it was not possible to get these automaticaly. As far as I can see the biggest problems are the abi_x86_32 changes required by the
    app-emulation/emul-linux-x86* abi_x86_32
statement.

The logs are created by running
   emerge --update --newuse --deep --backtrack=30 @world
with the different portage versions

output of emerge -pgv portage:
[ebuild   R   ] sys-apps/portage-9999  USE="(ipc) (xattr) -build -doc -epydoc (-pypy2_0) -python2 -python3 (-selinux)" LINGUAS="-ru" PYTHON_TARGETS="python2_7 python3_3 (-pypy1_9) (-pypy2_0) -python2_6 (-python3_1) -python3_2 (-python3_4)" 

output of emerge --info (with portage-9999):
Portage 2.2.0_p8 (hardened/linux/amd64, gcc-4.8.1, glibc-2.17, 3.11.0-rc1 x86_64)
=================================================================
System uname: Linux-3.11.0-rc1-x86_64-Intel-R-_Core-TM-_i5-2520M_CPU_@_2.50GHz-with-gentoo-2.2
KiB Mem:     8061504 total,    602452 free
KiB Swap:   10485756 total,   9891320 free
Timestamp of tree: Mon, 19 Aug 2013 08:00:02 +0000
ld GNU ld (GNU Binutils) 2.23.2
distcc 3.1 x86_64-pc-linux-gnu [disabled]
app-shells/bash:          4.2_p45
dev-java/java-config:     2.2.0
dev-lang/python:          2.7.5-r2, 3.3.2-r2
dev-util/cmake:           2.8.11.1
dev-util/pkgconfig:       0.28
sys-apps/baselayout:      2.2
sys-apps/openrc:          0.12
sys-apps/sandbox:         2.6-r1
sys-devel/autoconf:       2.13, 2.69
sys-devel/automake:       1.11.6, 1.12.6, 1.13.4, 1.14
sys-devel/binutils:       2.23.2
sys-devel/gcc:            4.6.4, 4.7.3, 4.8.1
sys-devel/gcc-config:     1.8
sys-devel/libtool:        2.4.2
sys-devel/make:           3.82-r4
sys-kernel/linux-headers: 3.10 (virtual/os-headers)
sys-libs/glibc:           2.17
Repositories: gentoo local xmw gamerlay sunrise lorem_ipsum emacs steam-overlay gentoo-haskell
ACCEPT_KEYWORDS="amd64 ~amd64"
ACCEPT_LICENSE="* -@EULA"
CBUILD="x86_64-pc-linux-gnu"
CFLAGS="-O2 -pipe -march=native"
CHOST="x86_64-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/share/gnupg/qualified.txt /var/lib/hsqldb"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/php/apache2-php5.3/ext-active/ /etc/php/apache2-php5.4/ext-active/ /etc/php/cgi-php5.3/ext-active/ /etc/php/cgi-php5.4/ext-active/ /etc/php/cli-php5.3/ext-active/ /etc/php/cli-php5.4/ext-active/ /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo /etc/texmf/language.dat.d /etc/texmf/language.def.d /etc/texmf/updmap.d /etc/texmf/web2c"
CXXFLAGS="-O2 -pipe -march=native"
DISTDIR="/mnt/data/distfiles"
EMERGE_DEFAULT_OPTS="--keep-going --quiet-build=n --autounmask-write"
FCFLAGS="-O2 -pipe"
FEATURES="assume-digests binpkg-logs clean-logs compressdebug config-protect-if-modified distlocks ebuild-locks fixlafiles merge-sync news parallel-fetch parallel-install preserve-libs protect-owned sandbox sfperms sign splitdebug strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync xattr"
FFLAGS="-O2 -pipe"
GENTOO_MIRRORS="https://lore.xmw.de/gentoo/"
LANG="en_US.utf8"
LDFLAGS="-Wl,-O1 -Wl,--as-needed"
MAKEOPTS="-j1"
PKGDIR="/mnt/data/packages"
PORTAGE_CONFIGROOT="/"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages"
PORTAGE_TMPDIR="/tmp"
PORTDIR="/usr/portage"
PORTDIR_OVERLAY="/usr/local/portage /var/lib/layman/xmw /var/lib/layman/gamerlay /var/lib/layman/sunrise /var/lib/layman/lorem_ipsum /var/lib/layman/emacs /var/lib/layman/steam /var/lib/layman/haskell"
SYNC="rsync://lore.xmw.de/gentoo-portage"
USE="X aac acl acpi aes-ni alsa amd64 avx bash-completion berkdb bzip2 c++0x cairo cli cracklib crypt cxx device-mapper dri dts dvi emacs flac gdbm gif git gpm hardened iconv icu ipv6 jit jpeg justify latex matroska mmx mmxext modules mp3 mpeg mudflap multilib ncurses nls nptl ogg openal opengl openmp pam pcre pdf pgo png profile python readline sdl session smp speex spell sse sse2 sse3 sse4_1 sse4_2 ssl ssse3 svg tcpd theora threads tiff truetype udev unicode urandom usb vaapi vim vim-syntax vorbis x264 xft xinerama xorg zlib zsh-completion" ABI_X86="64" ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87x ca0106 cmipci emu10k1x ens1370 ens1371 es1938 es1968 fm801 hda-intel intel8x0 intel8x0m maestro3 trident usb-audio via82xx via82xx-modem ymfpci" APACHE2_MODULES="authn_core authz_core socache_shmcb unixd actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="kexi words flow plan sheets stage tables krita karbon braindump author" CAMERAS="ptp2" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf superstar2 timing tsip tripmate tnt ubx" INPUT_DEVICES="evdev mouse keyboard synaptics" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" LINGUAS="de en en_US" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php5-4" PYTHON_SINGLE_TARGET="python2_7" PYTHON_TARGETS="python2_7 python3_3" QEMU_SOFTMMU_TARGETS="arm i386 x86_64" QEMU_USER_TARGETS="arm i386 x86_64" RUBY_TARGETS="ruby20" USERLAND="GNU" VIDEO_CARDS="intel" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account"
USE_PYTHON="2.7 3.3"
Unset:  CPPFLAGS, CTARGET, INSTALL_MASK, LC_ALL, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS
Comment 1 Hendrik v. Raven 2013-08-19 11:52:09 UTC
Created attachment 356414 [details]
output with portage 2.2.0
Comment 2 Hendrik v. Raven 2013-08-19 11:52:34 UTC
Created attachment 356416 [details]
output with portage 2.1.12.2 (stable)
Comment 3 Zac Medico gentoo-dev 2013-08-19 20:44:02 UTC
Can you do a git-bisect to determine which change triggers this behavior?

You can run portage from a git checkout if you set PATH and PYTHONPATH as described here:

http://www.gentoo.org/proj/en/portage/doc/testing.xml#doc_chap3
Comment 4 Hendrik v. Raven 2013-08-20 09:49:30 UTC
I will run the bisect, and post the result
Comment 5 Hendrik v. Raven 2013-08-20 10:32:54 UTC
f74f301e869b3cac9060a1f35b89df673805c08a is the first bad commit
commit f74f301e869b3cac9060a1f35b89df673805c08a
Author: Zac Medico <zmedico@gentoo.org>
Date:   Wed Jun 19 14:07:17 2013 -0700

    depgraph: fix installed/newuse logic more

    The matched_pkgs_ignore_use logic, introduced in commit
    dc8d70479dfb54a1f2561ba3fe82d7a62d97964f, is questionable because we
    need to ensure that an installed package can be selected if there are
    no other available packages that match a given USE-dep.

:040000 040000 872f154f02fc02a64cc921d949c011656506af98 78064c87c7864c4faeebb3c84c42afd5462e518e M     pym
Comment 6 Zac Medico gentoo-dev 2013-08-20 11:11:57 UTC
Okay, thanks for that. I'll see about creating a test case to reproduce the issue, and a patch to fix it.
Comment 7 Zac Medico gentoo-dev 2013-08-22 00:52:54 UTC
I made this test case that produces a similar conflict to the one that you've reported:

http://git.overlays.gentoo.org/gitweb/?p=proj/portage.git;a=commit;h=fedb39bc9a2a9fd3ab37a08096cda6824c93b02a

Unfortunately, the conflict is not solved even when using a checkout of the code prior to commit f74f301e869b3cac9060a1f35b89df673805c08a.

It looks like we need to add code to make the SLOT-conflict handling interact with autounmask.
Comment 8 Hendrik v. Raven 2013-08-22 09:28:09 UTC
I tried to solve all the conflicts by downgrading to stable portage (2.1.12.2) and let it create the USE changes via autounmask. It took me several runs (like 5, didn't count exactly) of emerge -avuND world && dispatch-conf, because there were always new use changes.

However the list is still not complete, it seems to ignore some packages, which should be catched by --newuse, but are not. Example is texlive-core. I have texlive[xetex], but removed the required texlive-core[xetex] USE change. In my oppinion that should get triggered by --newuse, but it doesn't apear in the rebuild list. If i run emerge -av1 texlive-core it results in the same stack trace like in 2.2.0.

I'm not completly sure, why it's triggered with 2.2.0, but may it be, that that's a bug in the dependency caculation in 2.1.12.0, which covers the unmask bug in the old version?
Comment 9 Zac Medico gentoo-dev 2013-08-22 19:01:50 UTC
(In reply to Hendrik v. Raven from comment #8)
> However the list is still not complete, it seems to ignore some packages,
> which should be catched by --newuse, but are not. Example is texlive-core. I
> have texlive[xetex], but removed the required texlive-core[xetex] USE
> change. In my oppinion that should get triggered by --newuse, but it doesn't
> apear in the rebuild list. If i run emerge -av1 texlive-core it results in
> the same stack trace like in 2.2.0.

That sounds like essentially the same issue as bug #350230. If rebuilding the package with new USE settings will trigger a conflict, then it tends to use the installed package instead. We may be able to handle this better by coupling the conflict handler with autounmask.

> I'm not completly sure, why it's triggered with 2.2.0, but may it be, that
> that's a bug in the dependency caculation in 2.1.12.0, which covers the
> unmask bug in the old version?

The code changed in this commit causes the depgraph to choose the installed package more often, which can affect the success/failure of backtracking:

http://git.overlays.gentoo.org/gitweb/?p=proj/portage.git;a=commit;h=f74f301e869b3cac9060a1f35b89df673805c08a