virtual/w3m-0 gets emerged whenever I update my box with an "emerge --update --newuse --changed-use --ask --newrepo -vt --deep --complete-graph=y --changed-deps=y --with-bdeps=y @world", only to be deinstalled afterwards when I run an "emerge --ask -v --depclean". This has happened to me for the last fortnight, at least. The only package in my box depending on virtual/w3m-0 is app-text/xmlto: === $ sudo equery d virtual/w3m * These packages depend on virtual/w3m: app-text/xmlto-0.0.26-r1 (text ? virtual/w3m) $ eix -I xmlto [I] app-text/xmlto Available versions: 0.0.26-r1 ~0.0.28-r1 {latex text} Installed versions: 0.0.26-r1(14:02:49 07/12/17)(latex text) Homepage: https://pagure.io/xmlto Description: script for converting XML and DocBook documents to a variety of output formats ===
(In reply to urcindalo from comment #0) > virtual/w3m-0 gets emerged whenever I update my box with an "emerge --update > --newuse --changed-use --ask --newrepo -vt --deep --complete-graph=y > --changed-deps=y --with-bdeps=y @world" Perhaps you ought to show us that output.
Also your output of emerge -vpt app-text/xmlto
(In reply to Jeroen Roovers from comment #1) > (In reply to urcindalo from comment #0) > > virtual/w3m-0 gets emerged whenever I update my box with an "emerge --update > > --newuse --changed-use --ask --newrepo -vt --deep --complete-graph=y > > --changed-deps=y --with-bdeps=y @world" > > Perhaps you ought to show us that output. Sure! Here you are, after sync'ing a few minutes ago: === These are the packages that would be merged, in reverse order: Calculating dependencies... done! [nomerge ] net-print/hplip-3.17.10-r1::gentoo USE="X hpcups hpijs kde (policykit) qt5 scanner snmp -doc -fax -libressl -libusb0 -minimal -parport -static-ppds" PYTHON_SINGLE_TARGET="python3_5 -python2_7 -python3_4 -python3_6" PYTHON_TARGETS="python2_7 python3_5 -python3_4 -python3_6" [nomerge ] dev-python/PyQt5-5.7.1::gentoo USE="dbus declarative gui network opengl printsupport svg webkit widgets -bluetooth -debug -designer -doc -examples (-gles2) -help -location -multimedia -positioning -sensors -serialport -sql -testlib -webchannel -webengine -websockets -x11extras -xmlpatterns" PYTHON_TARGETS="python2_7 python3_5 -python3_4 -python3_6" [ebuild UD ] dev-python/sip-4.19.3:0/12::gentoo [4.19.6:0/12::gentoo] USE="-debug -doc" PYTHON_TARGETS="python2_7 python3_5 -python3_4 -python3_6" 984 KiB [ebuild U ] net-misc/dropbox-45.3.88::gentoo [41.3.76::gentoo] USE="X librsync-bundled (-selinux)" PYTHON_SINGLE_TARGET="python3_5%* -python2_7% -python3_6%" PYTHON_TARGETS="python2_7 python3_5%* -python3_6%" 112.274 KiB [nomerge ] kde-plasma/plasma-meta-5.11.5:5::gentoo USE="crypt display-manager grub gtk handbook networkmanager pam sddm sdk wallpapers -bluetooth -plymouth -pulseaudio" [nomerge ] kde-plasma/kwin-5.11.5:5::gentoo USE="handbook -debug (-gles2) -multimedia {-test}" [nomerge ] media-libs/libepoxy-1.4.2::gentoo USE="X {-test}" ABI_X86="32 (64) (-x32)" [ebuild U ] x11-misc/util-macros-1.19.2::gentoo [1.19.1::gentoo] 83 KiB [nomerge ] kde-apps/kdeutils-meta-17.08.3-r1:5::gentoo USE="cups nls qt4 -floppy" [nomerge ] kde-apps/print-manager-17.08.3:5::gentoo USE="gtk -debug" [nomerge ] app-admin/system-config-printer-1.4.8::gentoo USE="(policykit) -doc -gnome-keyring" PYTHON_TARGETS="python2_7" [nomerge ] app-text/xmlto-0.0.26-r1::gentoo USE="latex text" [ebuild N ] virtual/w3m-0::gentoo 0 KiB [nomerge ] kde-apps/kdeutils-meta-17.08.3-r1:5::gentoo USE="cups nls qt4 -floppy" [nomerge ] kde-apps/sweeper-17.08.3:4/17.08::gentoo USE="handbook -debug" [nomerge ] kde-frameworks/kdelibs-4.14.37:4/4.14::gentoo USE="acl bzip2 handbook jpeg2k nls openexr plasma (policykit) qt3support spell ssl udev udisks upower (-altivec) -debug -doc (-fam) -kerberos -libressl -lzma {-test} -zeroconf" CPU_FLAGS_X86="mmx sse sse2 -3dnow" [ebuild U ] x11-apps/iceauth-1.0.8::gentoo [1.0.7::gentoo] 135 KiB Total: 5 packages (3 upgrades, 1 downgrade, 1 new), Size of downloads: 113.474 KiB Would you like to merge these packages? [Yes/No] ===
(In reply to Jeroen Roovers from comment #2) > Also your output of > > emerge -vpt app-text/xmlto My pleasure (virtual/w3m NOT installed, before answering YES to update my system with the command in the previous message): === $ sudo emerge -vpt app-text/xmlto Contraseña: These are the packages that would be merged, in reverse order: Calculating dependencies... done! [ebuild R ] app-text/xmlto-0.0.26-r1::gentoo USE="latex text" 0 KiB Total: 1 package (1 reinstall), Size of downloads: 0 KiB ===
(In reply to urcindalo from comment #3) > (In reply to Jeroen Roovers from comment #1) > > (In reply to urcindalo from comment #0) > > > virtual/w3m-0 gets emerged whenever I update my box with an "emerge --update > > > --newuse --changed-use --ask --newrepo -vt --deep --complete-graph=y > > > --changed-deps=y --with-bdeps=y @world" > > > > Perhaps you ought to show us that output. > > Sure! Here you are, after sync'ing a few minutes ago: After updating, this: === $ sudo emerge --ask -v --depclean * Always study the list of packages to be cleaned for any obvious * mistakes. Packages that are part of the world set will always * be kept. They can be manually added to this set with * `emerge --noreplace <atom>`. Packages that are listed in * package.provided (see portage(5)) will be removed by * depclean, even if they are part of the world set. * * As a safety measure, depclean will not remove any packages * unless *all* required dependencies have been resolved. As a * consequence of this, it often becomes necessary to run * `emerge --update --newuse --deep @world` prior to depclean. Calculating dependencies... done! . . . >>> Calculating removal order... >>> These are the packages that would be unmerged: virtual/w3m selected: 0 protected: none omitted: none All selected packages: =virtual/w3m-0 >>> 'Selected' packages are slated for removal. >>> 'Protected' and 'omitted' packages will not be removed. Would you like to unmerge these packages? [Yes/No] ===
It would be a bug in dependency resolver of Portage. Is this problem still occurring with sys-apps/portage-2.3.36? If yes, attach output of `emerge --info` and relevant installation/deinstallation commands with added --debug option.
This is an old bug which came back again few days ago. I didn't open a new bug since this report still open. # emerge -auvDN world These are the packages that would be merged, in order: Calculating dependencies... done! [ebuild N ] virtual/w3m-0::gentoo 0 KiB Total: 1 package (1 new), Size of downloads: 0 KiB Would you like to merge these packages? [Yes/No] # emerge -a --depclean * Always study the list of packages to be cleaned for any obvious * mistakes. Packages that are part of the world set will always * be kept. They can be manually added to this set with * `emerge --noreplace <atom>`. Packages that are listed in * package.provided (see portage(5)) will be removed by * depclean, even if they are part of the world set. * * As a safety measure, depclean will not remove any packages * unless *all* required dependencies have been resolved. As a * consequence of this, it often becomes necessary to run * `emerge --update --newuse --deep @world` prior to depclean. Calculating dependencies... done! >>> Calculating removal order... >>> These are the packages that would be unmerged: virtual/w3m selected: 0 protected: none omitted: none All selected packages: =virtual/w3m-0 >>> 'Selected' packages are slated for removal. >>> 'Protected' and 'omitted' packages will not be removed. Would you like to unmerge these packages? [Yes/No] # emerge --info Portage 2.3.87 (python 3.7.6-final-0, default/linux/amd64/17.1/desktop/plasma, gcc-9.2.0, glibc-2.30-r3, 5.4.18-gentoo x86_64) ================================================================= System uname: Linux-5.4.18-gentoo-x86_64-Intel-R-_Core-TM-_i7_CPU_950_@_3.07GHz-with-gentoo-2.6 KiB Mem: 16398268 total, 11588840 free KiB Swap: 4194300 total, 4194300 free Timestamp of repository gentoo: Fri, 07 Feb 2020 23:00:01 +0000 Head commit of repository gentoo: 3b5eba82dcb411018882763bd49fdb85a97291a0 sh bash 5.0_p11 ld GNU ld (Gentoo 2.34 p1) 2.34.0 distcc 3.3.3 x86_64-pc-linux-gnu [disabled] ccache version 3.7.7 [disabled] app-shells/bash: 5.0_p11::gentoo dev-java/java-config: 2.2.0-r4::gentoo dev-lang/perl: 5.30.1::gentoo dev-lang/python: 2.7.17-r1::gentoo, 3.6.10::gentoo, 3.7.6::gentoo, 3.8.1::gentoo, 3.9.0_alpha2::gentoo dev-util/ccache: 3.7.7::gentoo dev-util/cmake: 3.16.4::gentoo dev-util/pkgconfig: 0.29.2::gentoo sys-apps/baselayout: 2.6-r1::gentoo sys-apps/openrc: 0.42.1::gentoo sys-apps/sandbox: 2.18::gentoo sys-devel/autoconf: 2.13-r1::gentoo, 2.69-r5::gentoo sys-devel/automake: 1.13.4-r2::gentoo, 1.16.1-r2::gentoo sys-devel/binutils: 2.34::gentoo sys-devel/gcc: 9.2.0-r3::gentoo sys-devel/gcc-config: 2.2.1::gentoo sys-devel/libtool: 2.4.6-r6::gentoo sys-devel/make: 4.3::gentoo sys-kernel/linux-headers: 5.5::gentoo (virtual/os-headers) sys-libs/glibc: 2.30-r3::gentoo Repositories: gentoo location: /usr/portage sync-type: rsync sync-uri: rsync://rsync.gentoo.org/gentoo-portage priority: -1000 sync-rsync-extra-opts: sync-rsync-verify-metamanifest: yes sync-rsync-verify-max-age: 24 sync-rsync-verify-jobs: 2 local location: /usr/local/portage masters: gentoo priority: 0 ACCEPT_KEYWORDS="amd64 ~amd64" ACCEPT_LICENSE="* PUEL @EULA" CBUILD="x86_64-pc-linux-gnu" CFLAGS="-O2 -pipe -march=native -mtune=native" CHOST="x86_64-pc-linux-gnu" CONFIG_PROTECT="/etc /usr/share/config /usr/share/gnupg/qualified.txt" CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/dconf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo /etc/texmf/language.dat.d /etc/texmf/language.def.d /etc/texmf/updmap.d /etc/texmf/web2c" CPPFLAGS="-O2 -pipe -march=native -mtune=native" CXXFLAGS="-O2 -pipe -march=native -mtune=native" DISTDIR="/usr/portage/distfiles" EMERGE_DEFAULT_OPTS="--quiet-build=n --with-bdeps=y --verbose-conflicts --buildpkg --buildpkg-exclude='virtual/* sys-kernel/*-sources */*-bin x11-drivers/*'" ENV_UNSET="DBUS_SESSION_BUS_ADDRESS DISPLAY GOBIN PERL5LIB PERL5OPT PERLPREFIX PERL_CORE PERL_MB_OPT PERL_MM_OPT XAUTHORITY XDG_CACHE_HOME XDG_CONFIG_HOME XDG_DATA_HOME XDG_RUNTIME_DIR" FCFLAGS="-O2 -pipe" FEATURES="assume-digests binpkg-docompress binpkg-dostrip binpkg-logs buildpkg config-protect-if-modified distlocks ebuild-locks fixlafiles ipc-sandbox merge-sync multilib-strict network-sandbox news pid-sandbox preserve-libs protect-owned sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync xattr" FFLAGS="-O2 -pipe" GENTOO_MIRRORS="http://distfiles.gentoo.org" LANG="en_AU.UTF-8" LC_ALL="en_AU.UTF8" LDFLAGS="-Wl,-O1 -Wl,--as-needed" MAKEOPTS="-j9 -s" PKGDIR="/usr/portage/packages" PORTAGE_CONFIGROOT="/" PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages --exclude=/.git" PORTAGE_TMPDIR="/var/tmp" USE="X a52 aac aacs aalib acl acpi activities alsa amd64 asf berkdb bidi bluetooth bluray branding bzip2 cairo cdda cdio cdr cli consolekit crypt cups cxx dbus declarative directfb dri dts dv dvb dvd dvdr dvi emboss enca encode exif extensions extras faac faad fam fbcon ffmpeg flac fontconfig fortran gcj gd gdbm gif gpm graphviz gstreamer gtk gtk2 httpd iconv icu ilbc imagemagick imlib ipod java jpeg jpeg2k kde kdesu kipi kwallet lcms ldap legacy-systray libcaca libmpeg2 libnotify libtirpc live lua lzo mad matroska mjpeg mmx mng modplug mp3 mp4 mpeg multilib multislot mysql ncurses nls nptl ogg opengl openmp pam pango pcre pdf phonon plasma png policykit postproc ppds profile qml qt3support qt5 quicktime readline rtc rtmp samba scanner script sddm sdl seccomp semantic-desktop speex spell split-usr sql sqlite sse sse2 sse4_1 sse4_2 ssl ssse3 startup-notification subversion svg tcpd theora threads tiff toolame truetype twolame udev udisks unicode upower usb v4l vcd vlm vorbis vpx webkit widgets wxwidgets x264 x265 xanim xattr xcb xcomposite xine xml xv xvid xvmc zlib" ABI_X86="32 64" ADA_TARGET="gnat_2018" ALSA_CARDS="intel8x0" APACHE2_MODULES="authn_core authz_core socache_shmcb unixd actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="karbon sheets words" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" CPU_FLAGS_X86="mmx mmxext popcnt sse sse2 sse3 sse4_1 sse4_2 ssse3" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock greis isync itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf skytraq superstar2 timing tsip tripmate tnt ublox ubx" INPUT_DEVICES="evdev keyboard mouse" KERNEL="linux" L10N="en ar" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" LLVM_TARGETS="X86" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php7-2" POSTGRES_TARGETS="postgres10 postgres11" PYTHON_SINGLE_TARGET="python3_6" PYTHON_TARGETS="python2_7 python3_6 python3_7 python3_8" QEMU_SOFTMMU_TARGETS="x86_64 arm i386" QEMU_USER_TARGETS="arm armeb x86_64 i386" RUBY_TARGETS="ruby24 ruby25 ruby26 ruby27" USERLAND="GNU" VIDEO_CARDS="nvidia qxl" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account" Unset: CC, CTARGET, CXX, INSTALL_MASK, LINGUAS, PORTAGE_BINHOST, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS
I tried to re-merge xmlto and all dependencies. But it didn't fix the problem. # emerge -av1 www-client/lynx www-client/elinks www-client/w3m app-text/xmlto These are the packages that would be merged, in order: Calculating dependencies... done! [ebuild R ] www-client/elinks-0.13_pre_pre20180225::gentoo USE="X bzip2 gpm lua mouse nls samba ssl unicode xml zlib -bittorrent -brotli -debug -finger -ftp -gopher (-guile) -idn -ipv6 -javascript -libressl -nntp -perl -ruby -tre" 0 KiB [ebuild R ] www-client/w3m-0.5.3_p20190105::gentoo USE="X fbcon gpm imlib nls ssl unicode -gdk-pixbuf -libressl -lynxkeymap -nntp -xface" L10N="-de -ja" 0 KiB [ebuild R ] www-client/lynx-2.9.0_pre4::gentoo USE="bzip2 nls ssl unicode -cjk -gnutls -idn -ipv6 -libressl" 0 KiB [ebuild N ] virtual/w3m-0::gentoo 0 KiB [ebuild R ] app-text/xmlto-0.0.28-r1::gentoo USE="text -latex" 0 KiB Total: 5 packages (1 new, 4 reinstalls), Size of downloads: 0 KiB Would you like to merge these packages? [Yes/No]
virtual/w3m is installed because of app-text/xmlto dependency. # emerge -auvDNt world These are the packages that would be merged, in reverse order: Calculating dependencies... done! [nomerge ] net-misc/spice-gtk-0.37-r2::gentoo USE="gtk3 introspection mjpeg (policykit) usbredir -libressl -lz4 -pulseaudio -sasl -smartcard -static-libs -vala -webdav" [nomerge ] sys-devel/autoconf-2.69-r5:2.69::gentoo USE="-emacs" [nomerge ] dev-lang/perl-5.30.1:0/5.30::gentoo USE="berkdb gdbm -debug -doc -ithreads" [nomerge ] sys-libs/db-6.0.35-r2:6.0::gentoo USE="cxx java -doc -examples -tcl -test" ABI_X86="32 (64) (-x32)" [nomerge ] dev-java/java-config-2.2.0-r4:2::gentoo USE="-test" PYTHON_TARGETS="python2_7 python3_6 python3_7" [nomerge ] dev-python/setuptools-44.0.0::gentoo USE="-test" PYTHON_TARGETS="python2_7 python3_6 python3_7 python3_8 -pypy3" [nomerge ] dev-lang/python-3.8.1:3.8::gentoo USE="bluetooth gdbm ncurses readline sqlite ssl tk xml -build -examples -hardened -ipv6 -libressl -test -wininst" [nomerge ] net-wireless/bluez-5.52:0/3::gentoo USE="cups mesh obex readline udev -btpclient -debug -deprecated -doc -experimental -extra-tools -midi (-selinux) -systemd -test -test-programs -user-session" ABI_X86="32 (64) (-x32)" PYTHON_SINGLE_TARGET="python2_7" PYTHON_TARGETS="python2_7" [nomerge ] sys-apps/dbus-1.12.16::gentoo USE="X -debug -doc -elogind (-selinux) -static-libs -systemd -test -user-session" ABI_X86="32 (64) (-x32)" [nomerge ] app-text/xmlto-0.0.28-r1::gentoo USE="text -latex" [ebuild N ] virtual/w3m-0::gentoo 0 KiB Total: 1 package (1 new), Size of downloads: 0 KiB The dependency line in xmlto-0.0.28-r1.ebuild text? ( || ( virtual/w3m www-client/lynx www-client/elinks ) ) which means it depends on either virtual/w3m or www-client/lynx or www-client/elinks if the text USE flag is set. The use flag text is set and I have www-client/lynx and www-client/elinks installed. # emerge -pv www-client/lynx www-client/elinks These are the packages that would be merged, in order: Calculating dependencies... done! [ebuild R ] www-client/elinks-0.13_pre_pre20180225::gentoo USE="X bzip2 gpm lua mouse nls samba ssl unicode xml zlib -bittorrent -brotli -debug -finger -ftp -gopher (-guile) -idn -ipv6 -javascript -libressl -nntp -perl -ruby -tre" 0 KiB [ebuild R ] www-client/lynx-2.9.0_pre4::gentoo USE="bzip2 nls ssl unicode -cjk -gnutls -idn -ipv6 -libressl" 0 KiB Then, why emerge is trying to install virtual/w3m if the dependency condition is already met?
(In reply to email200202 from comment #9) > Then, why emerge is trying to install virtual/w3m if the dependency > condition is already met? Possibly related to bug 707108. What version of sys-apps/portage do you have?
# emerge -pv sys-apps/portage These are the packages that would be merged, in order: Calculating dependencies... done! [ebuild R ] sys-apps/portage-2.3.87::gentoo USE="(ipc) native-extensions rsync-verify xattr -build -doc -gentoo-dev (-selinux)" PYTHON_TARGETS="python2_7 python3_6 python3_7 python3_8 -pypy3" 0 KiB Total: 1 package (1 reinstall), Size of downloads: 0 KiB
(In reply to Zac Medico from comment #10) > (In reply to email200202 from comment #9) > > Then, why emerge is trying to install virtual/w3m if the dependency > > condition is already met? > > Possibly related to bug 707108. What version of sys-apps/portage do you have? sys-apps/portage-2.3.87 They seem to be related. bug 707108 was supposed to to be fixed in version 2.3.87. It could be side effect of the fix.
It looks like the fix for bug 706278 probably triggered it: https://gitweb.gentoo.org/proj/portage.git/commit/?id=f7d83d75c6b05a16ef07473917082dbd0cd9955c
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/proj/portage.git/commit/?id=1aaed33429f390957514eb81b050cae8c7201d67 commit 1aaed33429f390957514eb81b050cae8c7201d67 Author: Zac Medico <zmedico@gentoo.org> AuthorDate: 2020-02-08 07:05:02 +0000 Commit: Zac Medico <zmedico@gentoo.org> CommitDate: 2020-02-08 08:24:04 +0000 Add unit test for virtual/w3m depclean inconsistency (bug 649622) Bug: https://bugs.gentoo.org/649622 Signed-off-by: Zac Medico <zmedico@gentoo.org> lib/portage/tests/resolver/test_or_choices.py | 114 +++++++++++++++++++++++++- 1 file changed, 113 insertions(+), 1 deletion(-)
The removal of virtual/w3m is essentially indistinguishable from the removal of dev-util/cmake-bootstrap in the depclean test case for bug 703440. If we use that as a guide, then we need it to avoid pulling in virtual/w3m during the world update.
I have a working patch: https://github.com/gentoo/portage/pull/504
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/proj/portage.git/commit/?id=216c52d1ea447fbf06f6ac00e5d771780508601d commit 216c52d1ea447fbf06f6ac00e5d771780508601d Author: Zac Medico <zmedico@gentoo.org> AuthorDate: 2020-02-08 18:41:56 +0000 Commit: Zac Medico <zmedico@gentoo.org> CommitDate: 2020-02-08 18:48:45 +0000 dep_zapdeps: prefer choices with all in graph (bug 649622) Prefer choices where all packages have been pulled into the graph (except for choices that eliminate upgrades). This logic must be consistent between both update and removal (depclean) actions in order to avoid pulling in packages only to have them removed by depclean, as reported in bug 649622 for virtual/w3m. Bug: https://bugs.gentoo.org/649622 Signed-off-by: Zac Medico <zmedico@gentoo.org> lib/portage/dep/dep_check.py | 8 +++----- lib/portage/tests/resolver/test_or_choices.py | 9 ++++++--- 2 files changed, 9 insertions(+), 8 deletions(-)
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=87db6ee1103dc33c206111288a43ac619d27cefc commit 87db6ee1103dc33c206111288a43ac619d27cefc Author: Zac Medico <zmedico@gentoo.org> AuthorDate: 2020-02-09 01:18:34 +0000 Commit: Zac Medico <zmedico@gentoo.org> CommitDate: 2020-02-09 01:31:22 +0000 sys-apps/portage: Bump to version 2.3.88 #649622 prevent unecessary installation of virtual/w3m followed by removal by depclean #705736 preserve-libs: prevent unecessary preservation of system libraries which a package bundles #707820 generate API documentation with sphinx-apidoc #708448 support FEATURES=qa-unresolved-soname-deps so that the QA warning from bug 704320 can be disabled #708660 phase-helpers.sh: avoid passing an empty root value to portageq when ebuild IPC is disabled Bug: https://bugs.gentoo.org/706142 Bug: https://bugs.gentoo.org/649622 Bug: https://bugs.gentoo.org/705736 Bug: https://bugs.gentoo.org/707820 Bug: https://bugs.gentoo.org/708448 Bug: https://bugs.gentoo.org/708660 Package-Manager: Portage-2.3.88, Repoman-2.3.20 Signed-off-by: Zac Medico <zmedico@gentoo.org> profiles/arch/riscv/package.use.mask | 2 +- sys-apps/portage/Manifest | 1 + sys-apps/portage/portage-2.3.88.ebuild | 271 +++++++++++++++++++++++++++++++++ 3 files changed, 273 insertions(+), 1 deletion(-)
The problem is still there in Portage 2.3.88. World update is trying to install virtual/w3m and it is not required to satisfy the dependencies of app-text/xmlto since www-client/lynx and www-client/elinks are installed. --depclean is correct in uninstalling virtual/w3m but it shouldn't be installed in the first place.
(In reply to email200202 from comment #19) > The problem is still there in Portage 2.3.88. > > World update is trying to install virtual/w3m and it is not required to > satisfy the dependencies of app-text/xmlto since www-client/lynx and > www-client/elinks are installed. Do you have either www-client/lynx or www-client/elinks in /var/lib/portage/world? In the corresponding unit test, these packages are the world file. If at least one of them is in world then that should prevent virtual/w3m from being pulled in.
From https://forums.gentoo.org/viewtopic-t-1108096.html it looks like having www-client/w3m in your world file triggers the issue, so I'll create a unit test for that.
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/proj/portage.git/commit/?id=21d9b579dc3fda2105caf3328ac5f0afbd80b9a2 commit 21d9b579dc3fda2105caf3328ac5f0afbd80b9a2 Author: Zac Medico <zmedico@gentoo.org> AuthorDate: 2020-02-09 23:04:50 +0000 Commit: Zac Medico <zmedico@gentoo.org> CommitDate: 2020-02-09 23:15:01 +0000 test_virtual_w3m: test with www-client/w3m in @world Test for behavior reported in bug 649622 comment #10, where depclean removed virtual/w3m-0 even though www-client/w3m was in the world file. Since nothing is removed here, it means that we have not reproduced the behavior reported in this comment. Bug: https://bugs.gentoo.org/649622#c19 Signed-off-by: Zac Medico <zmedico@gentoo.org> lib/portage/tests/resolver/test_or_choices.py | 29 +++++++++++++++++++++++++++ 1 file changed, 29 insertions(+)
(In reply to email200202 from comment #19) > The problem is still there in Portage 2.3.88. > > World update is trying to install virtual/w3m and it is not required to > satisfy the dependencies of app-text/xmlto since www-client/lynx and > www-client/elinks are installed. It is intended for it to install virtual/w3m, since you have www-client/w3m installed and virtual/w3m is preferred because it is furthest to the left in the dependency string from app-text/xmlto: || ( virtual/w3m www-client/lynx www-client/elinks ) We consider virtual packages so have zero installation cost, so that having www-client/w3m installed is equivalent to having virtual/w3m as far as evaluation of dependencies is concerned. > --depclean is correct in uninstalling virtual/w3m but it shouldn't be > installed in the first place. This behavior is dependent on the content of your world file. This unit test case shows that virtual/w3m should not be removed if you have www-client/w3m in your world file: https://gitweb.gentoo.org/proj/portage.git/commit/?id=21d9b579dc3fda2105caf3328ac5f0afbd80b9a2 Do you have www-client/w3m in your world file or not?
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/proj/portage.git/commit/?id=c3e9a0fe26434edcc2c034c6b863789eb6afbc34 commit c3e9a0fe26434edcc2c034c6b863789eb6afbc34 Author: Zac Medico <zmedico@gentoo.org> AuthorDate: 2020-02-09 23:53:42 +0000 Commit: Zac Medico <zmedico@gentoo.org> CommitDate: 2020-02-09 23:56:16 +0000 test_virtual_w3m: test without www-client/w3m installed Add test case showing that virtual/w3m-0 is not pulled in when www-client/w3m is not installed and not in @world. Bug: https://bugs.gentoo.org/649622 Signed-off-by: Zac Medico <zmedico@gentoo.org> lib/portage/tests/resolver/test_or_choices.py | 39 +++++++++++++++++++++++---- 1 file changed, 34 insertions(+), 5 deletions(-)
(In reply to Zac Medico from comment #23) > (In reply to email200202 from comment #19) > > --depclean is correct in uninstalling virtual/w3m but it shouldn't be > > installed in the first place. > > This behavior is dependent on the content of your world file. This unit test > case shows that virtual/w3m should not be removed if you have www-client/w3m > in your world file: > > https://gitweb.gentoo.org/proj/portage.git/commit/ > ?id=21d9b579dc3fda2105caf3328ac5f0afbd80b9a2 > > Do you have www-client/w3m in your world file or not? I've got an idea for a test case now. I think your case is triggered only when www-client/w3m is not in @world and it is pulled in as a dependency of some other package (which is not xmlto).
If www-client/w3m is installed and depclean wants to remove virtual/w3m but not www-client/w3m, then we should have a look at this output: emerge -p --depclean --debug > debug.log 2>&1
I've obtained a debug log from https://forums.gentoo.org/viewtopic-t-1108096.html and it shows some complex behavior triggered by these dependencies: app-misc/neofetch-6.1.0: www-client/w3m[imlib] app-text/xmlto-0.0.28-r1: || ( virtual/w3m www-client/lynx www-client/elinks ) mail-client/neomutt-20191207: || ( www-client/lynx www-client/w3m www-client/elinks ) virtual/w3m-0: || ( www-client/w3m www-client/w3mmee ) x11-base/xorg-server-1.20.7: || ( www-client/links www-client/lynx www-client/w3m )
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/proj/portage.git/commit/?id=b98dbb357b711f64c8fc1e305c5bca38be203f1d commit b98dbb357b711f64c8fc1e305c5bca38be203f1d Author: Zac Medico <zmedico@gentoo.org> AuthorDate: 2020-02-10 01:10:14 +0000 Commit: Zac Medico <zmedico@gentoo.org> CommitDate: 2020-02-10 02:24:11 +0000 test_virtual_w3m: add realistic case, deps copied from real ebuilds Since virtual/w3m-0 is not removed in the depclean case, this test fails to reproduce bug 649622. Bug: https://bugs.gentoo.org/649622 Signed-off-by: Zac Medico <zmedico@gentoo.org> lib/portage/tests/resolver/test_or_choices.py | 142 ++++++++++++++++++++++++++ 1 file changed, 142 insertions(+)
Created attachment 613166 [details] depclean debug log from https://forums.gentoo.org/viewtopic-t-1108096.html In b98dbb357b711f64c8fc1e305c5bca38be203f1d there's a more realistic test case that copies w3m dependencies from real ebuilds found in the attached debug log, however the depclean test case does not remove virtual/w3m-0, so it does not reproduce the issue seen in the attached log from https://forums.gentoo.org/viewtopic-t-1108096.html.
(In reply to Zac Medico from comment #29) > Created attachment 613166 [details] > depclean debug log from https://forums.gentoo.org/viewtopic-t-1108096.html When comparing the real debug output to the unit test debug output, I can see that in the unit test the www-client/w3m package is pulled into the dependency graph earlier, which explains why it selects virtual/w3m instead of www-client/lynx when it evaluates the xmlto dependencies.
(In reply to Zac Medico from comment #30) > (In reply to Zac Medico from comment #29) > > Created attachment 613166 [details] > > depclean debug log from https://forums.gentoo.org/viewtopic-t-1108096.html > > When comparing the real debug output to the unit test debug output, I can > see that in the unit test the www-client/w3m package is pulled into the > dependency graph earlier, which explains why it selects virtual/w3m instead > of www-client/lynx when it evaluates the xmlto dependencies. The order of evaluation is pseudo-random, since it depends on the order of iteration for the unordered set that's returned from the PackageSet.getAtoms() method for the @world set.
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/proj/portage.git/commit/?id=ab472882b13ebe7a8a649c723928250dc8aeb740 commit ab472882b13ebe7a8a649c723928250dc8aeb740 Author: Zac Medico <zmedico@gentoo.org> AuthorDate: 2020-02-10 05:04:20 +0000 Commit: Zac Medico <zmedico@gentoo.org> CommitDate: 2020-02-10 05:05:23 +0000 depgraph: sort package set atoms for consistent results Traverse the unordered set returned from PackageSet.getAtoms() in sorted order, for more consistent results. Bug: https://bugs.gentoo.org/649622 Signed-off-by: Zac Medico <zmedico@gentoo.org> lib/_emerge/depgraph.py | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)
I've posted this message regarding inconsistent use of || preferences for www-client/elinks, links, lynx, w3m, and w3mmee: https://archives.gentoo.org/gentoo-dev/message/550d3859dea6d0fb0b39064628992634
(In reply to Zac Medico from comment #33) > I've posted this message regarding inconsistent use of || preferences for > www-client/elinks, links, lynx, w3m, and w3mmee: > > https://archives.gentoo.org/gentoo-dev/message/ > 550d3859dea6d0fb0b39064628992634 The inconsistent use of || preferences means that the default preference depends on the combination of packages that one has installed and the order that one has installed them, leading to unpredictable results. In portage, we can only mitigate the issue by trying to ensure that dependencies are processed in a very predictable order. That means sorting things, like in the commit referenced in comment #32.
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/proj/portage.git/commit/?id=350b8f486b96871386d721e48e768a77a1c4b1a6 commit 350b8f486b96871386d721e48e768a77a1c4b1a6 Author: Zac Medico <zmedico@gentoo.org> AuthorDate: 2020-02-11 20:39:37 +0000 Commit: Zac Medico <zmedico@gentoo.org> CommitDate: 2020-02-11 20:48:56 +0000 depgraph: sort nested package set names for consistent results Traverse the unordered set returned from PackageSet.getNonAtoms() in sorted order, for more consistent results. Use a reverse sort since we pop these sets from a stack. Bug: https://bugs.gentoo.org/649622 Signed-off-by: Zac Medico <zmedico@gentoo.org> lib/_emerge/depgraph.py | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-)
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/proj/portage.git/commit/?id=079f8c4a36ccc2ef5e25e7a57cd0707640f82592 commit 079f8c4a36ccc2ef5e25e7a57cd0707640f82592 Author: Zac Medico <zmedico@gentoo.org> AuthorDate: 2020-02-14 19:21:28 +0000 Commit: Zac Medico <zmedico@gentoo.org> CommitDate: 2020-02-14 23:10:37 +0000 depclean: ensure consistency with update actions (bug 649622) Make depclean traverse dependencies in the same order as update actions, in order to ensure consistency in decisions which are dependent on the order of dependency evaluation due to inconsistent use of || preferences in different packages. In unit tests, update test_virtual_w3m_realistic to assert that the order of graph traversal is deterministic and consistent between update and removal (depclean) actions. Bug: https://bugs.gentoo.org/649622 Signed-off-by: Zac Medico <zmedico@gentoo.org> lib/_emerge/actions.py | 33 +++++++--- lib/_emerge/depgraph.py | 21 +++++-- lib/portage/tests/resolver/ResolverPlayground.py | 77 +++++++++++++++--------- lib/portage/tests/resolver/test_or_choices.py | 12 +++- 4 files changed, 96 insertions(+), 47 deletions(-)
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/proj/portage.git/commit/?id=b3a945a9dad98a6d2064a9120206ad3cc9d46675 commit b3a945a9dad98a6d2064a9120206ad3cc9d46675 Author: Zac Medico <zmedico@gentoo.org> AuthorDate: 2020-02-15 00:53:00 +0000 Commit: Zac Medico <zmedico@gentoo.org> CommitDate: 2020-02-15 00:56:45 +0000 _expand_set_args: un-reverse nested set traversal Un-reverse traversal order for @profile @selected @system. Bug: https://bugs.gentoo.org/649622 Signed-off-by: Zac Medico <zmedico@gentoo.org> lib/_emerge/depgraph.py | 6 ++---- lib/portage/tests/resolver/test_or_choices.py | 4 ++-- 2 files changed, 4 insertions(+), 6 deletions(-)
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=76ea0ae6484979140a3f9204eeaa8dc775f6cf47 commit 76ea0ae6484979140a3f9204eeaa8dc775f6cf47 Author: Zac Medico <zmedico@gentoo.org> AuthorDate: 2020-02-15 01:13:16 +0000 Commit: Zac Medico <zmedico@gentoo.org> CommitDate: 2020-02-15 01:17:14 +0000 sys-apps/portage: Bump to version 2.3.89 #649622 depclean: ensure consistency with update actions, via consistent order of dependency traversal Bug: https://bugs.gentoo.org/706142 Bug: https://bugs.gentoo.org/649622 Package-Manager: Portage-2.3.89, Repoman-2.3.20 Signed-off-by: Zac Medico <zmedico@gentoo.org> sys-apps/portage/Manifest | 1 + sys-apps/portage/portage-2.3.89.ebuild | 271 +++++++++++++++++++++++++++++++++ 2 files changed, 272 insertions(+)
JFYI. I am experiensing the same behaviour. Portage 2.3.89. $ emerge -qcp virtual/w3m: 0 none none All selected packages: =virtual/w3m-0 $ equery d virtual/w3m * These packages depend on virtual/w3m: app-text/docbook-sgml-utils-0.6.14-r3 (virtual/w3m) app-text/xmlto-0.0.28-r1 (text ? virtual/w3m) $ equery d www-client/w3m * These packages depend on www-client/w3m: mail-client/mutt-1.13.4 (doc ? www-client/w3m) virtual/w3m-0 (www-client/w3m) x11-base/xorg-server-1.20.7 (doc ? www-client/w3m) I have www-client/w3m in my world file. I do not have virtual/w3m in my world file.
(In reply to Stefan Strogin from comment #39) > JFYI. I am experiensing the same behaviour. Portage 2.3.89. > > $ emerge -qcp > virtual/w3m: 0 none none > > All selected packages: =virtual/w3m-0 Can you confirm whether or not emerge -uD @world still wants to install this package? The aim of recent changes is to make the update and depclean actions consistent, but whether or not you'll ultimately have virtual/w3m installed is dependent on your installed packages as discussed in https://archives.gentoo.org/gentoo-dev/message/550d3859dea6d0fb0b39064628992634.
Seems not to be fixed for me. An "emerge -uD @world" still wants to pull virtual/w3m in. And "emerge --depclean" wants to remove it after that.
(In reply to Conrad Kostecki from comment #41) > Seems not to be fixed for me. An "emerge -uD @world" still wants to pull > virtual/w3m in. And "emerge --depclean" wants to remove it after that. I wonder if it's backtracking that triggers this inconsistency. Since depclean actions never backtrack, the order of dependency evaluation might be different for update actions which involve backtracking. A simple test would be to see if "emerge -uD --backtrack=0 @world" fails, which would indicate that backtracking could play a role. Backtracking will make it more difficult to achieve an order of dependency evaluation that is consistent with depclean, and the presence of inconsistencies discussed in https://archives.gentoo.org/gentoo-dev/message/550d3859dea6d0fb0b39064628992634 make results dependent on the order of dependency evaluation.
I have this same problem. I don't use w3m but it's a dependency of app-misc/neofetch.
(In reply to Zac Medico from comment #42) > (In reply to Conrad Kostecki from comment #41) [...] > I wonder if it's backtracking that triggers this inconsistency. Since > depclean actions never backtrack, the order of dependency evaluation might > be different for update actions which involve backtracking. A simple test > would be to see if "emerge -uD --backtrack=0 @world" fails, which would > indicate that backtracking could play a role. [...] Zac, I am experiencing the same problem with sys-apps/portage-2.3.89-r1 I also have app-text/xmlto[text] installed. I ran the following: # emerge -uDav --backtrack=0 @world It still wants to install virtual/w3m-0 and depclean wants to remove it. # equery d virtual/w3m * These packages depend on virtual/w3m app-text/docbook-sgml-utils-0.6.14-r3 (virtual/w3m) app-text/xmlto-0.0.28-r1 (text ? virtual/w3m)
(In reply to Terra from comment #44) > (In reply to Zac Medico from comment #42) > > (In reply to Conrad Kostecki from comment #41) > [...] > > I wonder if it's backtracking that triggers this inconsistency. Since > > depclean actions never backtrack, the order of dependency evaluation might > > be different for update actions which involve backtracking. A simple test > > would be to see if "emerge -uD --backtrack=0 @world" fails, which would > > indicate that backtracking could play a role. > [...] > > Zac, I am experiencing the same problem with sys-apps/portage-2.3.89-r1 > > I also have app-text/xmlto[text] installed. > > I ran the following: > # emerge -uDav --backtrack=0 @world > > It still wants to install virtual/w3m-0 and depclean wants to remove it. If we compare --debug logs for both calculations, then we should be able to see where the order of dependency evaluation diverges, and that should be correctable. > # equery d virtual/w3m > * These packages depend on virtual/w3m > app-text/docbook-sgml-utils-0.6.14-r3 (virtual/w3m) > app-text/xmlto-0.0.28-r1 (text ? virtual/w3m) These are the dependencies of those packages: app-text/docbook-sgml-utils-0.6.14-r3: || ( www-client/lynx www-client/links www-client/elinks virtual/w3m ) app-text/xmlto-0.0.28-r1: || ( virtual/w3m www-client/lynx www-client/elinks ) I guess you must have www-client/lynx or www-client/elinks installed. I wonder other package(s) pull those into your dependency graph.
(In reply to Zac Medico from comment #45) [..snip..] > I guess you must have www-client/lynx or www-client/elinks installed. I > wonder other package(s) pull those into your dependency graph. www-client/elinks is *not* installed www-client/lynx is installed $ equery d www-client/lynx * These packages depend on lynx: app-text/docbook-sgml-utils-0.6.14-r3 (www-client/lynx) app-text/xmlto-0.0.28-r1 (text ? www-client/lynx) sys-apps/smartmontools-7.0-r1 (update_drivedb ? www-client/lynx) x11-base/xorg-server-1.20.7 (doc ? www-client/lynx)
