Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bugzilla DB migration completed. Please report issues to Infra team via email via infra@gentoo.org or IRC

Bug 622462

Summary: sys-apps/portage-2.3.6 tries to wrongly unmask a hardmasked package instead of telling user to change USE conflicting flags with default --autounmask=y
Product: Portage Development Reporter: Pacho Ramos <pacho>
Component: Core - DependenciesAssignee: Portage team <dev-portage>
Status: RESOLVED FIXED    
Severity: normal CC: c.a.strahl, esigra, grknight, josef64
Priority: Normal Keywords: InVCS
Version: unspecified   
Hardware: All   
OS: Linux   
See Also: https://bugs.gentoo.org/show_bug.cgi?id=622480
https://bugs.gentoo.org/show_bug.cgi?id=640478
https://bugs.gentoo.org/show_bug.cgi?id=658648
https://bugs.gentoo.org/show_bug.cgi?id=667132
Whiteboard:
Package list:
Runtime testing required: ---
Bug Depends on:    
Bug Blocks: 651804    
Attachments: Bogus autounmask try of unmasking hardmasked qscintilla

Description Pacho Ramos gentoo-dev 2017-06-22 11:06:04 UTC
I am trying to update a one year old system and I am having some issues. In this case, portage is trying to unmask a hardmasked version of qscintilla. On the other hand, if I run emerge with --autounmask=n, it correctly tells me that the issue is a conflict between qt4 and qt5 USEs.

I keep for now the system stopped for the case you need something from their package database to improve this :/

Thanks a lot
Comment 1 Pacho Ramos gentoo-dev 2017-06-22 11:07:57 UTC
Created attachment 477594 [details]
Bogus autounmask try of unmasking hardmasked qscintilla
Comment 2 Pacho Ramos gentoo-dev 2017-06-22 11:08:46 UTC
This is the proper output running with "--autounmask=n" and showing the real culprit:
# emerge -auDN world --jobs=3 --autounmask=n

These are the packages that would be merged, in order:

Calculating dependencies... done!

!!! Multiple package instances within a single package slot have been pulled
!!! into the dependency graph, resulting in a slot conflict:

dev-lang/perl:0

  (dev-lang/perl-5.24.1-r2:0/5.24::gentoo, ebuild scheduled for merge) pulled in by
    (no parents that aren't satisfied by other packages in this slot)

  (dev-lang/perl-5.22.2:0/5.22::gentoo, installed) pulled in by
    dev-lang/perl:0/5.22=[-build(-)] required by (dev-perl/Authen-SASL-2.160.0-r1:0/0::gentoo, installed)
                 ^^^^^^^^                                                                                                                    
    (and 26 more with the same problem)

sys-libs/ncurses:0

  (sys-libs/ncurses-6.0-r1:0/6::gentoo, ebuild scheduled for merge) pulled in by
    (no parents that aren't satisfied by other packages in this slot)

  (sys-libs/ncurses-5.9-r5:0/5::gentoo, installed) pulled in by
    >=sys-libs/ncurses-5.7-r7:0/5= required by (media-sound/lame-3.99.5-r1:0/0::gentoo, installed)
                             ^^^^^                                                                                                    
    (and 1 more with the same problem)

NOTE: Use the '--verbose-conflicts' option to display parents omitted above

It may be possible to solve this problem by using package.mask to
prevent one of those packages from being selected. However, it is also
possible that conflicting dependencies exist such that they are
impossible to satisfy simultaneously.  If such a conflict exists in
the dependencies of two different packages, then those packages can
not be installed simultaneously.

For more information, see MASKED PACKAGES section in the emerge man
page or refer to the Gentoo Handbook.


emerge: there are no ebuilds built with USE flags to satisfy ">=x11-libs/qscintilla-2.9.3-r2:=[qt5(+)]".
!!! One of the following packages is required to complete your request:
- x11-libs/qscintilla-2.9.4::gentoo (Change USE: +qt5, this change violates use flag constraints defined by x11-libs/qscintilla-2.9.4: 'exactly-one-of ( qt4 qt5 )')
(dependency required by "sci-mathematics/octave-4.2.1::gentoo[gui]" [ebuild])
(dependency required by "sci-mathematics/qtoctave-0.10.1-r1::gentoo" [installed])
(dependency required by "@selected" [set])
(dependency required by "@world" [argument])
Comment 3 Pacho Ramos gentoo-dev 2017-06-22 11:09:08 UTC
# emerge --info
Portage 2.3.6 (python 2.7.10-final-0, default/linux/amd64/13.0/desktop/gnome/systemd, gcc-4.9.3, glibc-2.22-r4, 4.7.7-gentoo x86_64)
=================================================================
System uname: Linux-4.7.7-gentoo-x86_64-Intel-R-_Core-TM-2_Duo_CPU_E7500_@_2.93GHz-with-gentoo-2.2
KiB Mem:     3977756 total,    897624 free
KiB Swap:    6286332 total,   6259216 free
Timestamp of repository gentoo: Thu, 22 Jun 2017 08:45:01 +0000
sh bash 4.3_p48
ld GNU ld (Gentoo 2.25.1 p1.1) 2.25.1
ccache version 3.2.4 [enabled]
app-shells/bash:          4.3_p48::gentoo
dev-java/java-config:     2.2.0-r3::gentoo
dev-lang/perl:            5.22.2::gentoo
dev-lang/python:          2.7.10-r1::gentoo, 3.4.3-r1::gentoo
dev-util/ccache:          3.2.4::gentoo
dev-util/cmake:           3.5.2-r1::gentoo
dev-util/pkgconfig:       0.28-r2::gentoo
sys-apps/baselayout:      2.2::gentoo
sys-apps/openrc:          0.21.7::gentoo
sys-apps/sandbox:         2.10-r1::gentoo
sys-devel/autoconf:       2.13::gentoo, 2.69::gentoo
sys-devel/automake:       1.11.6-r1::gentoo, 1.14.1::gentoo, 1.15::gentoo
sys-devel/binutils:       2.25.1-r1::gentoo
sys-devel/gcc:            4.9.3::gentoo
sys-devel/gcc-config:     1.7.3::gentoo
sys-devel/libtool:        2.4.6::gentoo
sys-devel/make:           4.1-r1::gentoo
sys-kernel/linux-headers: 4.3::gentoo (virtual/os-headers)
sys-libs/glibc:           2.22-r4::gentoo
Repositories:

gentoo
    location: /usr/portage
    sync-type: rsync
    sync-uri: rsync://rsync.europe.gentoo.org/gentoo-portage
    priority: -1000

fisi35
    location: /usr/local/portage
    masters: gentoo
    priority: 0

ACCEPT_KEYWORDS="amd64"
ACCEPT_LICENSE="*"
CBUILD="x86_64-pc-linux-gnu"
CFLAGS="-O2 -pipe -march=native"
CHOST="x86_64-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/lib64/libreoffice/program/sofficerc /usr/share/gnupg/qualified.txt"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/dconf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo /etc/texmf/language.dat.d /etc/texmf/language.def.d /etc/texmf/updmap.d /etc/texmf/web2c"
CXXFLAGS="-O2 -pipe -march=native"
DISTDIR="/usr/distfiles"
EMERGE_DEFAULT_OPTS="--autounmask-write --keep-going --backtrack=10000"
FCFLAGS="-O2 -pipe"
FEATURES="assume-digests binpkg-logs ccache config-protect-if-modified distlocks ebuild-locks fixlafiles merge-sync news parallel-fetch preserve-libs protect-owned sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync xattr"
FFLAGS="-O2 -pipe"
GENTOO_MIRRORS="http://ftp.heanet.ie/pub/gentoo/"
LANG="es_ES.utf8"
LDFLAGS="-Wl,-O1 -Wl,--as-needed"
MAKEOPTS="-j3"
PKGDIR="/usr/portage"
PORTAGE_CONFIGROOT="/"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages --exclude=/.git"
PORTAGE_TMPDIR="/var/tmp"
USE="X a52 aac acl acpi alsa amd64 berkdb branding bzip2 cairo cdda cddb cdr cli colord cracklib crypt cups cxx dbus djvu dri dts dvd dvdr eds emboss encode evo exif fam ffmpeg firefox flac foomaticdb fortran gdbm gif gimp git glamor gnome gnome-keyring gnome-online-accounts google gpm gstreamer gtk iconv introspection java jpeg jpeg2k kpathsea latex lcms libnotify libsecret luatex lzma mad mms mng modules mp3 mp4 mpeg multilib nautilus ncurses networkmanager nls nptl ogg opengl openmp pam pango pch pcre pdf png policykit ppds pulseaudio qt3support qt4 raw readline sdl seccomp session spell ssl startup-notification subversion svg systemd t1lib tcpd theora tiff tracker truetype udev udisks unicode upower usb vdpau vorbis wxwidgets x264 xattr xcb xml xmp xv xvid zlib" ABI_X86="64" ALSA_CARDS="hda-intel" APACHE2_MODULES="authn_core authz_core socache_shmcb unixd actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="kexi words flow plan sheets stage tables krita karbon braindump author" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" CPU_FLAGS_X86="mmx mmxext sse sse2 sse3 sse4_1 ssse3" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock isync itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf skytraq superstar2 timing tsip tripmate tnt ublox ubx" INPUT_DEVICES="libinput keyboard mouse" KERNEL="linux" L10N="es es-ES en-GB en-US en" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" LINGUAS="es es_ES en_GB en_US en" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php5-6" PYTHON_SINGLE_TARGET="python3_4" PYTHON_TARGETS="python2_7 python3_4" RUBY_TARGETS="ruby21 ruby22" USERLAND="GNU" VIDEO_CARDS="nouveau vesa" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account"
Unset:  CC, CPPFLAGS, CTARGET, CXX, INSTALL_MASK, LC_ALL, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS, USE_PYTHON
Comment 4 Brian Evans Gentoo Infrastructure gentoo-dev 2017-06-22 14:15:51 UTC
In #gentoo, we've heard reports of users claiming that openssl-1.1.X should be unmasked instead of changing bindist USE.

Very annoying behavior that can have users doing something they shouldn't.
Comment 5 Pacho Ramos gentoo-dev 2017-06-22 14:35:03 UTC
Also, why once this is catched
> 
> emerge: there are no ebuilds built with USE flags to satisfy
> ">=x11-libs/qscintilla-2.9.3-r2:=[qt5(+)]".
> !!! One of the following packages is required to complete your request:
> - x11-libs/qscintilla-2.9.4::gentoo (Change USE: +qt5, this change violates
> use flag constraints defined by x11-libs/qscintilla-2.9.4: 'exactly-one-of (
> qt4 qt5 )')
> (dependency required by "sci-mathematics/octave-4.2.1::gentoo[gui]" [ebuild])
> (dependency required by "sci-mathematics/qtoctave-0.10.1-r1::gentoo"
> [installed])
> (dependency required by "@selected" [set])
> (dependency required by "@world" [argument])

emerge is still continuing with the backtracking? I would expect it to interrupt it (like when it does with --autounmask=n) because that constraint cannot be resolved without user interaction
Comment 6 Zac Medico gentoo-dev 2017-06-22 16:08:26 UTC
(In reply to Pacho Ramos from comment #5)
> emerge is still continuing with the backtracking? I would expect it to
> interrupt it (like when it does with --autounmask=n) because that constraint
> cannot be resolved without user interaction

The odd behavior is likely related to the "<dev-python/qscintilla-python-2.10 -qt4" package.use setting from this commit:

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=96f2d1c58eeca40e33565f72822761b048e23bbd

Because of that setting, in may be possible to satisfy the ">=x11-libs/qscintilla-2.9.3-r2:=[qt5(+)]" dependency without and USE changes, simply my unmasking the dev-python/qscintilla-python-2.10 ebuild.
Comment 7 Zac Medico gentoo-dev 2017-06-22 16:12:15 UTC
I meant the "<x11-libs/qscintilla-2.10 -qt4" setting from this commit:

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=8ed39aa2882fde741802656483f798257300ba43
Comment 8 Pacho Ramos gentoo-dev 2017-06-22 16:14:16 UTC
But I am not using plasma profile, I am using default/linux/amd64/13.0/desktop/gnome/systemd
Comment 9 Pacho Ramos gentoo-dev 2017-06-22 16:17:35 UTC
Also, even if it would be possible to solve it unmasking a hardmasked package, in general changing the USE flags should be preferred over trying to rely on a hardmasked package as, usual, packages there have some important issues. Then, probably the resolver should prefer to keep the package.mask entry if possible
Comment 10 Pacho Ramos gentoo-dev 2017-06-22 16:19:40 UTC
(In reply to Pacho Ramos from comment #9)
> Also, even if it would be possible to solve it unmasking a hardmasked
> package, in general changing the USE flags should be preferred over trying
> to rely on a hardmasked package as, usual, packages there have some
> important issues. Then, probably the resolver should prefer to keep the
> package.mask entry if possible

I have seen that using --autounmask-keep-masks=y results in that behavior exactly, the USE change is properly reported and it doesn't try to unmask the hardmasked package. This would be probably a saner default because allowing people to unmask so easily hardmasked packages can be a bit risky for them (depending on the affected package of course)
Comment 11 Pacho Ramos gentoo-dev 2017-06-23 09:13:40 UTC
In the other bug report (#622480) Zac has stated that:
(--autounmask-keep-masks allows keyword changes that are not necessarily desirable)

What changes are that ones? I was tempted to add that option as default option on my computers to not tend to unmask hardmasked packages... but now, I am not sure :|

Thanks!
Comment 12 Zac Medico gentoo-dev 2017-06-23 15:55:40 UTC
(In reply to Pacho Ramos from comment #11)
> In the other bug report (#622480) Zac has stated that:
> (--autounmask-keep-masks allows keyword changes that are not necessarily
> desirable)
> 
> What changes are that ones? I was tempted to add that option as default
> option on my computers to not tend to unmask hardmasked packages... but now,
> I am not sure :|

The --autounmask-keep-masks option only prevents package.mask changes. That means there can still be autounmask changes to package.accept_keywords. The proposed --autounmask-use-only option would not allow either package.mask or package.accept_keywords changes.
Comment 13 Pacho Ramos gentoo-dev 2017-06-24 08:55:40 UTC
Ah, ok. Then, I will probably rely on --autounmask-keep-masks because my main concern was to get hardmasked packages unmasked so easily :) 

Also, why is --autounmask-keep-masks not enabled by default? It looks safer to me to force people (by default) to add entries to "package.unmask" when they really want to do, after reading the mask entry and be ok with the reasons for the mask. Or should I open a separate bug report to suggest the change of the default?
Comment 14 Zac Medico gentoo-dev 2017-06-24 18:17:58 UTC
(In reply to Pacho Ramos from comment #13)
> Ah, ok. Then, I will probably rely on --autounmask-keep-masks because my
> main concern was to get hardmasked packages unmasked so easily :) 

In this case, the reason that it didn't stop at the USE change is that autounmask refuses to make USE changes that violate REQUIRED_USE. So, in order to avoid violating REQUIRED_USE, it tried to use package.unmask. Maybe we could adjust the logic here a bit, so that it tries to satisfy REQUIRED_USE before it resorts to packages.unmask.

> Also, why is --autounmask-keep-masks not enabled by default? It looks safer
> to me to force people (by default) to add entries to "package.unmask" when
> they really want to do, after reading the mask entry and be ok with the
> reasons for the mask. Or should I open a separate bug report to suggest the
> change of the default?

I think it's better to keep the default as-is, and adjust the logic to be smarter about satisfying REQUIRED_USE.
Comment 15 Pacho Ramos gentoo-dev 2017-07-01 07:49:42 UTC
(In reply to Zac Medico from comment #14)
> I think it's better to keep the default as-is, and adjust the logic to be
> smarter about satisfying REQUIRED_USE.

Ah, nice, if that would be feasible that would be great sure :)
Comment 16 Pacho Ramos gentoo-dev 2017-07-01 08:00:05 UTC
*** Bug 620416 has been marked as a duplicate of this bug. ***
Comment 17 Zac Medico gentoo-dev 2017-07-08 23:14:03 UTC
A standardized algorithm for automatic solving of REQUIRED_USE will be helpful, as proposed here:

https://wiki.gentoo.org/wiki/User:MGorny/GLEP:ReqUse
Comment 18 Clint 2017-10-12 14:46:03 UTC
Be great to see some progress on this,

at present (and for a long time now) >openssl-1.1 is broken and nothing has improved in the past year since this was masked.

If you keep it masked, you get circular deps and KDE requiring it - if you unmask you end up with rev deps unable to build, and if you switch to libressl then that comes with it's own pita problems.

Please devs....
Comment 19 Pacho Ramos gentoo-dev 2017-12-14 10:48:38 UTC
I am unsure if maybe --autounmask-keep-masks set as default could have avoided issues like bug 640478 (portage trying to unmask a masked for removal package instead of installing the new one). For now it has been workarounded switching the order of the deps... but I guess it could happen again easily in the future :/
Comment 20 Zac Medico gentoo-dev 2018-03-26 03:12:55 UTC
I have reproduced the issue involving the octave dependency on qscintilla[qt5]. The problem occurs in the depgraph _wrapped_select_pkg_highest_available_imp method, where the stable version of qscintilla is skipped because autounmask fails to generate a USE setting that satisfies REQUIRED_USE, causing the method to set use_match to False on line 6361:

https://gitweb.gentoo.org/proj/portage.git/tree/pym/_emerge/depgraph.py?h=portage-2.3.24#n6361

Since it's able to achieve a valid USE configuration with the masked version of qscintilla, it chooses that instead.

In the absence of GLEP73 REQUIRED_USE solving (bug 628004), the autounmask code should be fixed to recognize that this dependency can be satisfied with the stable version of qscintilla, and therefore it should choose that version even if REQUIRED_USE is unsatisfied for some reason.
Comment 22 Larry the Git Cow gentoo-dev 2018-03-27 16:58:51 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/proj/portage.git/commit/?id=d7f2fbd4a649fc81f00a22eb1633a193a988b5da

commit d7f2fbd4a649fc81f00a22eb1633a193a988b5da
Author:     Zac Medico <zmedico@gentoo.org>
AuthorDate: 2018-03-26 07:11:19 +0000
Commit:     Zac Medico <zmedico@gentoo.org>
CommitDate: 2018-03-27 16:56:03 +0000

    emerge --autounmask: prevent unmask due to unsatisfied REQUIRED_USE (bug 622462)
    
    Fix autounmask to generate USE changes that violate REQUIRED_USE, in
    order to prevent it from trying to create inappropriate keywords or
    package.unmask. This solves the problem reported in bug 622462, where
    it inappropriately unmasked a newer version of qscintilla. With this
    fix, it will generate USE changes that violate REQUIRED_USE, and then
    report the issue as follows:
    
    emerge: there are no ebuilds built with USE flags to satisfy ">=x11-libs/qscintilla-2.9.3-r2:=[qt5(+)]".
    !!! One of the following packages is required to complete your request:
    - x11-libs/qscintilla-2.9.4::test_repo (Change USE: +qt5, this change violates use flag constraints defined by x11-libs/qscintilla-2.9.4: 'exactly-one-of ( qt4 qt5 )')
    
    Note that it may be possible for autounmask to try harder to solve
    REQUIRED_USE in cases like this, but that it beyond the scope of this
    bug fix. The only goal of this patch is to correctly handle the case
    where satisfaction of REQUIRED_USE has failed.
    
    Bug: https://bugs.gentoo.org/622462

 pym/_emerge/depgraph.py                       | 20 +++++++++++++++++---
 pym/portage/tests/resolver/test_autounmask.py | 25 ++++++++++++++++++++++++-
 2 files changed, 41 insertions(+), 4 deletions(-)}
Comment 23 Zac Medico gentoo-dev 2018-06-21 17:28:27 UTC
Fixed in 2.3.40.