Bug 205227 - app-portage/gentoolkit-0.2.4_pre7 - revdep-rebuild fails to handle removed packages
Bug#: 205227 Product:  Portage Development Version: 2.1 Platform: All
OS/Version: All Status: RESOLVED Severity: normal Priority: P2
Resolution: FIXED Assigned To: tools-portage@gentoo.org Reported By: davidepesa@gmail.com
Component: Tools
URL: 
Summary: app-portage/gentoolkit-0.2.4_pre7 - revdep-rebuild fails to handle removed packages
Keywords:  InSVN
Status Whiteboard: 
Opened: 2008-01-10 17:31 0000
Description:   Opened: 2008-01-10 17:31 0000 
# revdep-rebuild -p

WARNING
WARNING *** This is a rewritten version of revdep-rebuild ***
WARNING
WARNING
WARNING Please report any bugs to http://bugs.gentoo.org
WARNING
WARNING In the bug report please include the following information:
WARNING     emerge --info
WARNING     A copy of the output from the revdep-rebuild command
WARNING     A copy of the .revdep-rebuild* files as an attachment
WARNING
WARNING If the bug is severe, the previous version of revdep-rebuild is located
WARNING at: /usr/lib/gentoolkit/bin/revdep-rebuild
WARNING
WARNING
WARNING *** This is a rewritten version of revdep-rebuild ***
WARNING

 * Configuring search environment for revdep-rebuild

 * Checking reverse dependencies
 * Packages containing binaries and libraries broken by a package update
 * will be emerged.

 * Collecting system binaries and libraries
 * Generated new /root/.revdep-rebuild.1_files

 * Collecting complete LD_LIBRARY_PATH
 * Generated new /root/.revdep-rebuild.2_ldpath

 * Checking dynamic linking consistency
[ 79% ]  *   broken /usr/lib64/openoffice/program/i18npool.uno.so (requires
libicui18n.so.36
libicuuc.so.36)
 *   broken /usr/lib64/openoffice/program/libcui680lx.so (requires
libicuuc.so.36)
 *   broken /usr/lib64/openoffice/program/libfrm680lx.so (requires
libicui18n.so.36
libicuuc.so.36)
[ 80% ]  *   broken /usr/lib64/openoffice/program/liblng680lx.so (requires
libicuuc.so.36)
[ 81% ]  *   broken /usr/lib64/openoffice/program/libsvt680lx.so (requires
libicuuc.so.36)
 *   broken /usr/lib64/openoffice/program/libsvx680lx.so (requires
libicuuc.so.36)
 *   broken /usr/lib64/openoffice/program/libsw680lx.so (requires
libicuuc.so.36)
 *   broken /usr/lib64/openoffice/program/libvcl680lx.so (requires
libicule.so.36
libicuuc.so.36)
[ 100% ]
 * Generated new /root/.revdep-rebuild.3_rebuild

 * Assigning files to packages
 *   /usr/lib64/openoffice/program/i18npool.uno.so -> app-office/openoffice
 *   /usr/lib64/openoffice/program/libcui680lx.so -> app-office/openoffice
 *   /usr/lib64/openoffice/program/libfrm680lx.so -> app-office/openoffice
 *   /usr/lib64/openoffice/program/liblng680lx.so -> app-office/openoffice
 *   /usr/lib64/openoffice/program/libsvt680lx.so -> app-office/openoffice
 *   /usr/lib64/openoffice/program/libsvx680lx.so -> app-office/openoffice
 *   /usr/lib64/openoffice/program/libsw680lx.so -> app-office/openoffice
 *   /usr/lib64/openoffice/program/libvcl680lx.so -> app-office/openoffice
 * Generated new /root/.revdep-rebuild.4_packages_raw and
/root/.revdep-rebuild.4_package_owners

 * Cleaning list of packages to rebuild
 * Generated new /root/.revdep-rebuild.4_packages

 * Assigning packages to ebuilds
 * Generated new /root/.revdep-rebuild.4_ebuilds

 * Evaluating package order
 * Nothing to rebuild.
 * (The program should have already quit, so this is a minor bug.)


Then I tried:
# revdep-rebuild -e -i

WARNING
WARNING *** This is a rewritten version of revdep-rebuild ***
WARNING
WARNING
WARNING Please report any bugs to http://bugs.gentoo.org
WARNING
WARNING In the bug report please include the following information:
WARNING     emerge --info
WARNING     A copy of the output from the revdep-rebuild command
WARNING     A copy of the .revdep-rebuild* files as an attachment
WARNING
WARNING If the bug is severe, the previous version of revdep-rebuild is located
WARNING at: /usr/lib/gentoolkit/bin/revdep-rebuild
WARNING
WARNING
WARNING *** This is a rewritten version of revdep-rebuild ***
WARNING

 * Configuring search environment for revdep-rebuild

 * Checking reverse dependencies
 * Packages containing binaries and libraries broken by a package update
 * will be emerged.

 * Collecting system binaries and libraries
 * Generated new /root/.revdep-rebuild.1_files

 * Collecting complete LD_LIBRARY_PATH
 * Generated new /root/.revdep-rebuild.2_ldpath

 * Checking dynamic linking consistency
[ 79% ]  *   broken /usr/lib64/openoffice/program/i18npool.uno.so (requires
libicui18n.so.36
libicuuc.so.36)
 *   broken /usr/lib64/openoffice/program/libcui680lx.so (requires
libicuuc.so.36)
 *   broken /usr/lib64/openoffice/program/libfrm680lx.so (requires
libicui18n.so.36
libicuuc.so.36)
[ 80% ]  *   broken /usr/lib64/openoffice/program/liblng680lx.so (requires
libicuuc.so.36)
[ 81% ]  *   broken /usr/lib64/openoffice/program/libsvt680lx.so (requires
libicuuc.so.36)
 *   broken /usr/lib64/openoffice/program/libsvx680lx.so (requires
libicuuc.so.36)
 *   broken /usr/lib64/openoffice/program/libsw680lx.so (requires
libicuuc.so.36)
 *   broken /usr/lib64/openoffice/program/libvcl680lx.so (requires
libicule.so.36
libicuuc.so.36)
[ 100% ]
 * Generated new /root/.revdep-rebuild.3_rebuild

 * Assigning files to ebuilds
 * Generated new /root/.revdep-rebuild.4_ebuilds

 * Evaluating package order
 * Generated new /root/.revdep-rebuild.5_order

 * All prepared. Starting rebuild
emerge --oneshot  =!!! =All =ebuilds =that =could =satisfy
="=app-office/openoffice-2.3.0" =have =been =masked. =!!! =One =of =the
=following =masked =packages =is =required =to =complete =your =request: =-
=app-office/openoffice-2.3.0 =(masked =by: =EAPI =, =CHOST: =) =The =current
=version =of =portage =supports =EAPI ='1'. =You =must =upgrade =to =a =newer
=version =of =portage =before =EAPI =masked =packages =can =be =installed. =For
=more =information, =see =MASKED =PACKAGES =section =in =the =emerge =man =page
=or =refer =to =the =Gentoo =Handbook.
..........
Calculating dependencies /

!!! '=!!!' is not a valid package atom.
!!! Please check ebuild(5) for full details.
!!! (Did you specify a version but forget to prefix with '='?)
 * revdep-rebuild failed to emerge all packages.
 * you have the following choices:
 * - If emerge failed during the build, fix the problems and re-run
revdep-rebuild.
 * - Use /etc/portage/package.keywords to unmask a newer version of the
package.
 *   (and remove /root/.revdep-rebuild.5_order to be evaluated again)
 * - Modify the above emerge command and run it manually.
 * - Compile or unmerge unsatisfied packages manually,
 *   remove temporary files, and try again.
 *   (you can edit package/ebuild list first)
 * To remove temporary files, please run:
 * rm /root/.revdep-rebuild*.?_*
grep: /root/.revdep-rebuild.4_package_owners: No such file or directory

openoffice-2.3.0 has been removed from the Portage tree, so probably the cause
of this bug is that revdep-rebuild wrongly tries to re-emerge exactly the same
version of the package (even if -e was specified on the command line), then
portage prints out an error because that specific version doesn't exist but
revdep continues anyway, failing for a second time.


emerge --info:
Portage 2.1.4_rc14 (default-linux/amd64/2007.0/desktop, gcc-4.1.2,
glibc-2.7-r1, 2.6.24-rc6-git8 x86_64)
=================================================================
System uname: 2.6.24-rc6-git8 x86_64 Intel(R) Core(TM)2 CPU T7600 @ 2.33GHz
Timestamp of tree: Thu, 10 Jan 2008 13:16:01 +0000
distcc 2.18.3 x86_64-pc-linux-gnu (protocols 1 and 2) (default port 3632)
[disabled]
ccache version 2.4 [disabled]
app-shells/bash:     3.2_p33
dev-java/java-config: 1.3.7, 2.1.3
dev-lang/python:     2.5.1-r5
dev-util/ccache:     2.4-r7
sys-apps/baselayout: 2.0.0_rc6-r1
sys-apps/sandbox:    1.2.18.1-r2
sys-devel/autoconf:  2.13, 2.61-r1
sys-devel/automake:  1.7.9-r1, 1.8.5-r3, 1.9.6-r2, 1.10
sys-devel/binutils:  2.18-r1
sys-devel/gcc-config: 1.4.0-r4
sys-devel/libtool:   1.5.24
virtual/os-headers:  2.6.23-r3
ACCEPT_KEYWORDS="amd64 ~amd64"
CBUILD="x86_64-pc-linux-gnu"
CFLAGS="-O2 -march=nocona -pipe"
CHOST="x86_64-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/kde/3.5/env /usr/kde/3.5/share/config
/usr/kde/3.5/shutdown /usr/kde/3.9/env /usr/kde/3.9/share/config
/usr/kde/3.9/shutdown /usr/share/config"
CONFIG_PROTECT_MASK="/etc/env.d /etc/env.d/java/ /etc/fonts/fonts.conf
/etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/splash /etc/terminfo
/etc/texmf/web2c /etc/udev/rules.d"
CXXFLAGS="-O2 -march=nocona -pipe"
DISTDIR="/usr/portage/distfiles"
FEATURES="distlocks metadata-transfer parallel-fetch sandbox sfperms strict
unmerge-orphans userfetch userpriv usersandbox"
GENTOO_MIRRORS="http://mirror.ing.unibo.it/gentoo/                
ftp://ftp.gentoo.mesh-solutions.com/gentoo/"
LANG="it_IT.UTF-8"
LC_ALL="it_IT.UTF-8"
LINGUAS="it"
MAKEOPTS="-j3"
PKGDIR="/usr/portage/packages"
PORTAGE_RSYNC_EXTRA_OPTS="--prune-empty-dirs"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress
--force --whole-file --delete --delete-after --stats --timeout=180
--exclude=/distfiles --exclude=/local --exclude=/packages
--filter=H_**/files/digest-*"
PORTAGE_TMPDIR="/var/tmp"
PORTDIR="/usr/portage"
PORTDIR_OVERLAY="/usr/portage/local/layman/kde /usr/portage/local/layman/openrc
/usr/portage/local/pesa"
SYNC="rsync://rsync.europe.gentoo.org/gentoo-portage"
USE="X a52 aac acl acpi alsa amd64 audiofile avahi bash-completion berkdb
bitmap-fonts bluetooth bzip2 cairo caps cddb cdr cli cracklib crypt curl
curlwrappers dbus dri dts dv dvd dvdr dvdread emboss encode evo exif expat fam
ffmpeg fftw firefox flac ftp gd gdbm gif glut gmp gnutls gpm graphviz hal iconv
idn ieee1394 ipod ipv6 isdnlog jabber javascript jbig jpeg jpeg2k kde
kdeenablefinal kdexdeltas lame lcms ldap libsamplerate lirc lm_sensors lua lzo
mad matroska midi mmap mmx mng mozilla mp3 mpeg mplayer msn mudflap musepack
musicbrainz ncurses nls nptl nptlonly nsplugin offensive ogg opengl openmp pam
pcre pdf png pppd pulseaudio python qt3 qt3support qt4 quicktime readline
reflection ruby samba sasl sdl session slang sndfile snmp socks5 speex spell
spl sqlite sqlite3 sse sse2 ssl svg tcpd tetex theora threads tiff truetype
truetype-fonts type1-fonts unicode usb v4l vcd vorbis wifi wmf x264 xcb
xcomposite xine xinerama xml xorg xpm xv xvid zlib" ALSA_CARDS="hda-intel"
ALSA_PCM_PLUGINS="adpcm alaw asym copy dmix dshare dsnoop empty extplug file
hooks iec958 ioplug ladspa lfloat linear meter mulaw multi null plug rate route
share shm softvol" APACHE2_MODULES="actions alias auth_basic authn_alias
authn_anon authn_dbm authn_default authn_file authz_dbm authz_default
authz_groupfile authz_host authz_owner authz_user autoindex cache dav dav_fs
dav_lock deflate dir disk_cache env expires ext_filter file_cache filter
headers include info log_config logio mem_cache mime mime_magic negotiation
rewrite setenvif speling status unique_id userdir usertrack vhost_alias"
ELIBC="glibc" INPUT_DEVICES="keyboard mouse synaptics evdev" KERNEL="linux"
LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses
text" LINGUAS="it" LIRC_DEVICES="inputlirc sir" USERLAND="GNU"
VIDEO_CARDS="fglrx vesa"
Unset:  CPPFLAGS, CTARGET, EMERGE_DEFAULT_OPTS, INSTALL_MASK, LDFLAGS,
PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS

------- Comment #1 From Davide Pesavento 2008-01-10 17:32:52 0000 ------- 
Using app-portage/gentoolkit-0.2.4_pre7.

------- Comment #2 From Paul Varner 2008-01-10 18:54:02 0000 ------- 
What does the following command return?

portageq best_visible / app-office/openoffice:0

------- Comment #3 From Davide Pesavento 2008-01-10 19:11:37 0000 ------- 
Right. I locally masked >=app-office/openoffice-2.3.1.
That command returns nothing.

Anyway, revdep-rebuild doesn't honour the '-e' option on the command line, and
could try to validate emerge's output (bailing out with an appropriate error
message) rather than continuing and producing a wrong emerge command.

------- Comment #4 From Michael A. Smith 2008-01-22 22:44:46 0000 ------- 
Please upload the .revdep-rebuild* files from your home directory. (Run
revdep-rebuild with the -k flag to keep it from deleting temporary files.)

------- Comment #5 From Michael A. Smith 2008-01-23 00:01:36 0000 ------- 
in get_build_order()
<SNIP>
  REBUILD_GREP=$(emerge --nodeps $RAW_REBUILD_LIST | sed 's/\[[^]]*\]//g') &&
    emerge --deep $RAW_REBUILD_LIST | sed 's/\[[^]]*\]//g' |
    grep -F "$REBUILD_GREP" > $LIST.5_order || {
      eerror
      eerror 'Warning: Failed to resolve package order.'
      eerror 'Will merge in arbitrary order'
      eerror
</SNIP>

If $RAW_REBUILD_LIST contains the string '=app-office/openoffice-2.3.0' and
2.3.0 is not available, then emerge --nospinner --pretend --oneshot --quiet
--nodeps $RAW_REBUILD_LIST would create the error message in comment #0:

!!! All ebuilds that could satisfy "=app-office/openoffice-2.3.0" have been
masked.
!!! One of the following masked packages is required to complete your request:
- app-office/openoffice-2.3.0 (masked by: EAPI-1 , CHOST:)

For more information, see MASKED PACKAGES section in the emerge man page or
refer to the Gentoo Handbook.

But what does *not* make sense is two things that I haven't been able to figure
out yet.
1) Why is the error message 'Warning: Failed to resolve package order...' not
being triggered, and
2) Why is the output, which afaict goes to STDERR, getting captured in a STDIN
redirect?

------- Comment #6 From Michael A. Smith 2008-01-26 01:03:47 0000 ------- 
Created an attachment (id=141796) [details]
revdep-rebuild_r453_better_version_checking.patch

Patch against rev 453 that does several things slightly different:

- Doesn't hunt for versions unless -e is specified, by default just uses slots
- Fixes this bug and I think also in Bug #199507 where get_package_order()
didn't handle emerge failures because it was in a pipeline.
- Removes packages from the immediate build list if they cannot be merged
automatically because of blocking, masking, or treecleaning. Informs the user
of this.
- Fixed a minor bug caused by grepping for $LIST.4_package_owners even when
that file doesn't exist.

Hopefully didn't regress :)

------- Comment #7 From Jakub Moc (RETIRED) 2008-02-01 20:06:27 0000 ------- 
*** Bug 208485 has been marked as a duplicate of this bug. ***

------- Comment #8 From Davide Pesavento 2008-02-10 23:15:36 0000 ------- 
Patch works fine here.

[...]
 * Evaluating package order
 *
 * Portage could not find any version of the following packages it could build:
 *  app-office/openoffice:0
 *
 * (Perhaps they are masked, blocked, or removed from portage.)
 * Try to emerge them manually.
 *
 * Warning: Portage cannot rebuild any of the necessary packages.

------- Comment #9 From Paul Varner 2008-02-16 20:58:45 0000 ------- 
$ svn commit -m "Apply patch to better handle masked and removed packages. (Bug
205227)"
Sending        revdep-rebuild/revdep-rebuild
Transmitting file data .
Committed revision 462.

------- Comment #10 From Paul Varner 2008-02-21 01:52:14 0000 ------- 
Released in gentoolkit-0.2.4_rc2